The now–finalized HTTP/2 specification has rightfully garnered a lot of interest from the web performance community. The new protocol is aimed at addressing common network performance issues with the aging HTTP/1.x protocol, whilst preserving the existing semantics.

We began a small-scale rollout for static assets earlier this year. After building confidence in our new infrastructure, we began transitioning our static assets to HTTP/2. Surprisingly, some sections of our platform felt noticeably slower. This post will cover our investigation into the performance regressions we experienced by adopting HTTP/2.

Our story isn’t the panacea of web performance typically associated with HTTP/2. We hope sharing our sobering experience will help to balance the discussion.

Why HTTP/2?

For better or worse, the story of HTTP/2 has become tied to notions of free performance and how it will make everything we know about web performance wrong.

Unlike HTTP/1.x which creates a new connection per resource, HTTP/2 creates at most a single connection per hostname. That connection is a multiplexed stream utilizing a binary framing protocol. The binary framing is responsible for matching multiple concurrent requests to responses.

Slide from Ilya Grigorik’s presentation: HTTP/2 is here, let’s optimize!

No longer being limited to one transaction per connection, head-of-line blocking is largely eliminated. Creating fewer connections also means a reduced sensitivity to latency and TCP congestion controls. In combination, these properties can result in big performance wins because they reduce the volume and duration of round trips between server and client.

https://www.igvita.com/2012/07/19/latency-the-new-web-performance-bottleneck

Monitoring the performance of HTTP/2

We use Calibre for synthetic monitoring of end-user performance, collecting a diverse set of metrics. We push a small subset of this data to highly–visible Geckoboards throughout our offices.

Etsy–inspired performance dashboards in the Melbourne office

We used the following metrics as proxies for user-perceived page load performance and the success of HTTP/2. We chose these specific metrics because they’re affected by different aspects of page load life cycle.

• The DOMContentLoaded event is delayed by synchronous scripts.
• The time to first paint is delayed by render–blocking resources like CSS and fonts.
• The time to visually complete is delayed by non–render–blocking resources like images and potentially asynchronous scripts.
• Speed Index is affected by the rate of visual completion over time.

Testing and verifying the success of HTTP/2

We started by migrating our image thumbnail CDN to CloudFlare, which provides HTTP/2 out of the box. Initial benchmarks showed CloudFlare’s latency and response times to be comparable to those of our existing CDN.

As a design marketplace most of our pages are image–centric, commonly requiring upwards of 50 images.

Pages with many small resources are adversely affected by minor changes in connection latency with HTTP/1.x. For such latency-bound pages we expected visual completion be reached faster. How much faster would depend on the connection latency and number of images. We expected this trend to continue on high latency, low bandwidth 3G connections.

For bandwidth–bound pages we expected to see no appreciable change.

Enabling HTTP/2 for images alone doesn’t affect head–of–line blocking, so we didn’t expect any changes in time to first paint or DOMContentLoaded.

Reality

The results weren’t as clear cut. Below I’ll dig into some of the nuances, surprises and future considerations for our HTTP/2 rollout.

Testing

We enabled the HTTP/2 CDN behind a feature flag and over the next week we recorded approximately 100 Calibre snapshots with and without the new CDN. The Calibre Chrome agents are US-based with low–latency, high–bandwidth connections.

Case study: Designer portfolio

Designer portfolios are representative of a typical latency–bound 99designs page. Here we observed a 5% improvement in Speed Index and time to visual completion.

The time to first paint was comparable, but interestingly the initial render was more complete with HTTP/2.

More complete initial render with images served over HTTP/2

Case study: Discover design gallery

Our Discover design galleries are representative of the extremes of our platform. With 80 images, weighing in at around 10mb per page, these pages are bandwidth–bound so the effects from the reduction in latency should be marginal. As such, we expected no noticeable change in performance.

What we actually observed was a 5–10% regression on average in time to visual completion and Speed Index. Overall page load time had decreased however, suggesting that we were benefiting from reduced connection latency.

Delayed first paint, and visual completion times for HTTP/2

High latency testing

For gathering data on high latency connections we used WebPagetest with a 3G connection profile.

Initial paints continued to be more complete, but happened noticeably later. The overall page load continued to occur earlier.

Counter–intuitively, visual completeness was negatively delayed by an average of 15% for Designer profiles and 25% for Discover respectively.

TL;DR:

For a typical image rich, latency–bound page using a high–speed, low–latency connection, visual completion was achieved 5% faster on average.

For an extremely image–heavy, bandwidth–bound page using the same connection, visual completion was achieved 5–10% slower on average.

On a high–latency, low–speed connection we saw significant delays for page to reach visual completion.

In all tests we saw overall page load times improved, and more complete initial paints.

The postmortem

The data collected left us with one big question

Hypothesis 1: network saturation

HTTP/1.x traffic is bursty in nature due to opening many short-lived connections. This behavior is responsible for the network waterfall seen in dev tools.

HTTP/1.x staggered network waterfall

Initially we thought a single, long lived TCP connection loading megabytes of image data could be starving bandwidth from loading layout blocking resources like CSS, JS or fonts.

However, the network waterfalls didn’t reveal any changes to the loading behavior of layout blocking resources.

Hypothesis 2: altered loading priority

When using HTTP/1.x, browsers have a limit of approximately six simultaneous open connections to an origin. As resources are discovered, they’re added to a FIFO resource download queue. Limiting the number of open connections to an origin creates an implicit loading priority of resources.

Each queued resource represents a request–response round trip to an origin that must be completed before the resource can leave the queue. This behavior is what we know as the network waterfall.

HTTP/2’s framing protocol lets the browser stitch together multiple requests and responses, so we lose the document order priority queue.

Discover page network timeline for HTTP/1.x and HTTP/2

We considered that the HTTP/1.x best practice of putting