New Feature: Using Private Git Repositories with AEM as a Cloud Service
In AEM as a Cloud Service or Adobe’s hosted AEM 6.5 product (AEM Managed Services or “AMS”), there has long been a requirement that deploying any code into any Adobe-hosted environment requires first getting that code into Adobe’s hosted git repository.
The reasoning is fairly simple - Adobe’s Cloud Manager CI/CD framework has to run a number of checks on the code, then execute builds on that code from their containerized build service, and then also to be able to tag releases in the repo once production deployments are complete.
Private vs Adobe Repos - Context
There have long been a number of complexities and limitations that AEMaaCS and AMS customers have had to deal with, in using this which has made it a highly-desirable and sought-after feature to be able to use a private git repo with cloud-hosted AEM, such as:
Maintaining two Repos
The Adobe git repo is only a deployment repository. It’s not meant to be the primary git that you work out of to develop your AEM site. It has no SLA for availability or backups. So, in running your project you do have to maintain your own internal repository as well as the Adobe repo for deployment, and then manage the sync between those repos.
Adobe repo lacks features like pull requests
There are a number of features that the Adobe git implementation lacks which makes certain early-stage pipelines a bit challenging. For example, the Adobe git does not have threading or pull requests.
This becomes a problem when the Adobe CI/CD pipeline is the only place where you can really run the full suite of build automation tests that would show up things like security violations, performance degradation, Sonarqube rule violations, etc.
So, if you have a new feature that you want to open a PR on that - say - you think makes the site faster, you have to open that PR on your own Github, your lead has to “approve” it even if we don’t know if it passes build checks, you then commit it through to the Adobe git at which point maybe it FAILS and then your PR was sort of worthless.
Security & User Management
There are no fine-grained controls on the Adobe git to control user management & access. So, you end up needing to implement your own security on your own git, then rely on build automation you create on your end to attempt to limit what branches, commits, changes, etc come through to the Adobe git. It can be done, but just adds an error-prone level of Rube Goldberg complexity.
Private Git Repositories for AEM as a Cloud Service & AMS: What did Adobe Release?
To be clear, this is a first general release of this feature (announced at Adobe Summit this year and in preview with various customers over the past year). As such, it doesn’t support EVERY feature you might want, and there are a number of limitations you should be aware of. But it’s a great step forward for those who live in this world, so I’ll try to outline a few things it can and can’t do.
What You Now Can Do:
-
Use Github.com in Cloud Manager Pipelines: You can now add a github.com git repository as a deployment repo in Adobe Cloud Manager.
Note: This specifically refers to the public https://github.com/ site and not self-hosted Github or other git providers. Adobe uses a Github App to be able to authenticate and maintain access for Git, and this is a required part of the process right now.
This is likely one of the major hurdles involved in implementing other git providers. -
Run Code Quality Pipelines on PRs: A very useful part of this “shift left” in the pipeline that this allows you to do, is to open PRs in github which can then kick off a Cloud Manager Code Quality Pipeline, which can then report back its results RIGHT INTO THE PR. This is huge to being able to validate if a PR is really approvable or whether it should go back for re-work.
-
PR pipelines are Auto-Created and Auto-Pruned: When you open up a PR which kicks off a code-quality pipeline in your Cloud Manager, a pipeline gets created specifically for that PR and will run & output specifically for that PR. When that PR is closed, Cloud Manager will auto-remove that pipeline so that you don’t end up getting clogged with a million PR pipelines.
This behaviour is configurable in your project.
-
Execute Full-Stack Pipelines: After doing a code-quality check and approving PRs, one can then set up Cloud Manager full-stack pipelines to be able to run, straight out of your Github repo, and without transiting the Cloud Manager Adobe Repo at all.
These deployments can work all the way through to production. -
Works on both AMS (AEM 6.5) and AEM as a Cloud Service: This supports both types of infrastructures, with the same caveats here.
What You Can’t Yet Do:
-
Use other Non-Github Git Repos: This feature only works on https://github.com. So, if you have “company.github.com” or “gitlab.com” or “bitbucket.com” or Azure repos or any other non-Github git provider, this won’t work for you yet.
Adobe has said that they’re actively working on prioritizing and enabling other git providers based on customer need - so if you have a NEED, waste no time in reaching out to any Adobe employee you know who will listen. -
No Config, Front-End or Web-Tier Pipelines: Currently this feature only works with the “Full-Stack pipeline” which deploys everything in your AEM project. If you use the other submodule pipelines such as the Web-Tier pipeline to deploy dispatcher-only code, or the Config pipeline to deploy your CDN configs, this feature isn’t for you yet.
-
Git Trigger Doesn’t Yet Work: If you’re trying to have a pipeline (code quality or deployment) start on a git trigger, so that it starts the deployment once code is committed, this does not yet work.
At this time, even if “Deployment Trigger: On Git Changes” is checked, you’ll still need to manually trigger the deployment. This is now also mentioned in the “Limitations” section of the documentation. (Hat tip to Sathish Balan for pointing this one out!)
Perfecting your AEM Pipeline
If this looks like something you might want to try yourself, the documentation is here on how to get started with self-managed Github in AEM.
However, if you’re not sure if this is something that you want to implement or not, or have other deployment or CI/CD worries you want to chat about, I’m happy to talk!
Also, for more on the subject (if you like podcasts), our CTO Dwayne Hale and I here discuss the merits and demerits of running cloud vs self-hosted AEM gear.
Tad Reeves
Principal Architect at Arbory Digital
AEM Architect & DevOps guy with 14 years experience on AEM/CQ and 25+ years in systems infrastructure. He’s been mountain biking longer than he’s been doing system administration, and though originally from Maine, makes his home in the mountains of Northwest Georgia.
Like what you heard? Have questions about what’s right for you? We’d love to talk! Contact Us
Podcast Episodes & Blog Posts
What is AEM? What is Adobe Experience Manager used for? We’ve attempted to do a basic explanation of what AEM is and does in 30 minutes or less - and somehow we managed to make it, despite the Fire Department randomly showing up about 19 minutes into the podcast recording!.
How much do you know about the tools at your disposal to optimize your site's performance in mainland China? And even if you don't have a Chinese-language site, do you need to be concerned with in-China performance? YOU DO!
In today’s war between cloud repatriation and blazingly-fast new Edge Delivery services, let’s revisit: is self-hosted AEM still a thing?