Decorative double Helix

Thinking Through AEM Infrastructure in China - Do You Need in-China Hosting?

I’ve gotten the opportunity to consult for a few different companies deploying infrastructure to China, as well as debugging China-specific performance issues for AEM environments, and it remains the most complicated country to solve for. There are variables, unknowns and gotchas galore, so I figured it might be helpful to put together a collection of some of the issues I’ve run into, in the case it might help you evaluate your infrastructure plans. Some of these issues may pertain only to sites that run on Adobe Experience Manager, others affect a broad range of platforms – I’ll try to make it a worthwhile read even if you don’t run AEM.

I was going to do this as one big supermegatron article, but just the first section I’m writing is already bloating out of control, so I’ll do this as a multi-part series, split up into a few topics:

How do you measure & decide whether or not you need a hosting solution for China?
Is AEM as a Cloud Service an option?
Does Adobe’s new Edge Delivery Services work in China?
Do you need a specific license to host in China?
What CDN considerations should one have for China?
Do you set up an entirely new infrastructure, or just put a Dispatcher/Publisher setup in China? Or just a Dispatcher?
What networking concerns should you be aware of with China infrastructure?
What about SSO/Site Login?
How will user-generated content (UGC) get handled, and how does that play with Chinese data residency law?
What about your custom backend calls like your site-search infrastructure, databases, etc?
What about Adobe Dynamic Media and Adobe Launch?
What if you use Brightcove or Youtube for your on-demand videos? What are video delivery options if you have a mainland China audience?

See, lots to talk about. I’ll start in this article just with the first bit – how to decide whether or not you even need China-specific infrastructure?

Edit: The second post in the series is up here: Tips for AEM Infrastructure in China: Designing an in-China or Near-China Site Architecture

Dealing with the Great Firewall: What’s Different about Accessing Sites in China

In case you’re not already well-versed with Internet access in China, one should know that China’s internet access is, for the most part, entirely behind the world’s most murky and complex web filtering apparatus. It’s a partially-automated, partially human-controlled access filter which can control ingress and egress using a number of mechanisms including DNS blocking, IP-based blocking, quality-of-service filtering (i.e. slowly degrading website usability instead of outright-blocking), man-in-the-middle attacks and packet spoofing. These mechanisms can have highly-variable effects on website traffic and website UX.

As an example: it’s popularly known that “you can’t see Twitter in China”. However, that’s not technically true. When last I visited China in 2018 – as a foreign visitor, in a hotel in Guangzhou (primarily used by foreign visitors) I had no problem accessing Flickr or Twitter, though I occasionally had trouble with Gmail and Google. However, Facebook, for me, worked the first time, was slower the second time, and by day 2 of my stay, I no longer found I could get the whole Facebook page to load. Note that this was my experience when using the hotel’s wifi. When I got myself a prepaid China Unicom SIM card for my burner cell phone, and then used it to tether my laptop, suddenly ALL US social media sites were blocked, Google, Facebook, Youtube, everything.

The view from my hotel in Guangzhou – where I had no problem accessing US sites usually assumed to be restricted in China

Furthermore, when traveling to a city like Fuzhou or Quanzhou (an out-of-the-way destination not frequented by foreigners) I found I couldn’t hit Facebook, Linkedin, Flickr or any of the usual social media biggies – sometimes even with a VPN.

Hong Kong and Macau, however, as they are “special administrative regions” of China, are not restrained by the same internet access filtering as the rest of China – and therefore resources hosted in datacenters located in Hong Kong will face delays, quality-of-service issues and potential blocking as they transit into the rest of China.

The main take-aways you should have from this are:

(a) Accessing any resource hosted outside of mainland China will potentially be subjected to either outright blocking, delay, or quality-of-service degradation, and

(b) Because of the variable nature of the implementation of the filtering apparatus, one can’t take a single website visit or a small cross-section of visits (i.e. the hear-say of a single office worker in Shanghai) as representative of what your potential users and customers are facing across China.

Measuring China Website Performance: Do You Need In-China Hosting?

That being said, the first and most important thing to quantify is to decide whether or not you actually need separate infrastructure for China or not. You need to decide whether or not there is a problem, and if there is a problem you need to decide what its root cause is. Remember your average website page view contains potentially hundreds of sub-requests, many of them to third-party resources. You need to discover:

(a) Are your users having a bad time?
(b) Is it the requests to YOUR gear that’s slow, or is it a third-party?
(c) Is that something you can fix by substituting locale-specific resources, or is there no substitute for getting some gear in China?

This evaluation takes a few parts, and can really turn into the scope of work for a large discovery project just to determine. And it could be REALLY IMPORTANT to determine. Have a look at this analysis from my own site, hosted in the USA:

Above is a “filterable geography” graph from New Relic Browser for the www.opsinventor.com site. Note that for the limited subset of visitors for this USA-hosted site, US visitors get ~7x faster load times than Hong Kong ~30x faster page load times than mainland China.

I hereby apologize to any of my mainland Chinese blog visitors.

Assuming first that you have a significant fraction of site visitors from China (data your Google Analytics / Adobe Analytics should be able to rapidly tell you), it’s important to then deeply understand what sort of user experience those visitors are currently getting. If you don’t have much traffic, no reason to put hundreds of thousands into new marketing infrastructure in the country. However, if you’re Tesla or Marriott and are doing billions in sales locally in China, it’s not really much of an option.

Geolocation report from Google Analytics for recent traffic for this site – looks like I do have enough China-specific traffic to bother with, were I to monetize this site at all…

Use Real-User Monitoring

The best and unequivocally-recommended way to do this is to employ a real-user monitoring (RUM) tool on your site such as New Relic Browser, Dynatrace or Datadog which uses javascript code injected into every page on your site to be able to conduct detailed timing & analysis of user interactions, and thereby give you a real view, by locale, of what people are seeing. This is the most real, as it also takes into account all factors like user hardware, the quality of real users’ Internet connections or company VPNs, etc.

Tools like this also allow you to dig into specific session traces, so that you can debug and iterate – looking at how long a specific visitor took to load various resources, comparing mobile vs desktop, etc to really understand whether a resource hang stopped the whole loading of a page for a user, or just silently timed out. This screenshot is from New Relic Browser, showing this in action:

Another tool which I’ve used with great success for geo-specific testing is good old WebPageTest, which allows you to test specific URLs on a variety of browsers, from a variety of specific locations including several in China itself as well as other APAC regions like Hong Kong, Japan and Singapore.

The nice thing about WebPageTest is that it will optionally also record an MP4 of the page loading in the browser, so that you can see what it would have looked like for a user in China loading a given page resource.

It’s essential to gather enough data first before any big infrastructure purchase, to make sure you know that you’ll be solving the right problem with the engineering resources you then devote to making a snappy and usable Chinese user experience.