sitespeed.io 17.0 and Browsertime 12.0

Woohoo we shipped 17.0.0! There are many changes and you should read through the full changelog and focus on the new best practices, breaking changes, updated dashboards, block third parties when using WebPageReplay, new privacy advice and new delta TTFB metrics.

New best practices #

One of the new things in 17 is the support for one extra key in Graphite: the name of the test. Set a computer friendly name of your test by using --slug. Then use the slug in the graphite key by adding --graphite.addSlugToKey to your run. When you do that change, should also convert your graphite data and your dashboards. The plan is like this:

  • In April 2021 you can convert your data and use the slug. You need to add --graphite.addSlugToKey true else you will get a log warning that you miss the slug for your test. All default dashboards in sitespeed.io will use the slug, so to use them you should add that new key and convert your data.
  • In September 2021 --graphite.addSlugToKey true will be set to default, meaning if you haven’t upgraded your Graphite data yet, you need to set --graphite.addSlugToKey false to be able to run as before.
  • In November 2021 the CLI functionality will disappear and you need upgrade your Graphite metrics when you upgrade sitespeed.io.

You can read how to upgrade in the documentation.

When you have updated you will add one extra parameter to your test: --copyLatestFilesToBase. When you have done all these step you are ready for … using the new dashboards in Grafana with screenshot and video from last run. This is SUPER useful to be able to fast see what’s going on. Checkout what it looks like here and here. And read the documentation how to make sure you see them in Grafana.

We ship all new dashboards with some extra focus on Google Web Vitals (for sitespeed.io, WebPageTest, Google Page Speed Insights, Lighthouse and CRUX). We understand that these metrics is important for some users so lets focus on them. We also ship a couple new example dashboards for how to setup user journeys. Documentation for those will come soon.

What to think about when upgrading #

You need to do a plan on when you want to upgrade Graphite to be able to use those new dashboards. You can upgrade to 17.0.0 and continue as before but to be able to use the new things in the dashboards you need to upgrade your Graphite data.

Breaking changes #

  • We have changed some of the connectivity profiles, you can see the changes here. This means that if you use 3g connectivity using Throttle, your tests will have a faster TTFB than before. If you wanna hold on to the old settings you can do that by adding --browsertime.legacyConnectivityProfiles true to your tests.
  • If you have a budget using layoutShift that metric has now been renamed to cumulativeLayoutShift.
  • Read the full list of other changes in Browsertime.

Updated dashboards #

We have upgraded all dashboards! To use them you need to follow our new best practices. When that is done, you can upgrade the dashboards.

We have one dashboard for all metrics you collect from a page. Use that dashboard as an example of what you can do with your own dashboards. It looks like this and checkout that the dashboard now show a screenshot and a video from the latest run.

Page metrics

Scroll down on the dashboard and you will a lot of rows that you can expand. Expand them to see what metrics that is collected.

Page metrics example 2

We have also shipped a couple of example user journey dashboards. Here’s what it looks like for a user journey where you login to Wikipedia.

Login User Journey

To setup your own dashboards for your user journeys you should follow this guide.

We also updated the leaderboard dashboard to follow the setup, the CruX, WebPageTest and Google Page Speed Insights/Lighthouse. All dashboards include the ever popular Google Web Vitals.

Leaderboard

CruX

WebPageTest

Plus 1 dashboard

Block third parties when using WebPageReplay #

Many thanks to Inderpartap Singh Cheema that fixed so that you can use --chrome.blockDomainsExcept together with WebPageReplay in the Docker container, so you more easily can focus on the performance disregarding 3rd party marketing scripts.

New metrics: Delta TTFB #

We have a couple of new metrics that is the delta between TTFB and First Contentful Paint, Largest Contentful paint and First visual change #1528. You can use this if you have unstable TTFB and want to alert on front end metrics. The metrics are automatically sent to Graphite.

New privacy advice #

There’s a new version of the Coach with a new super important privacy advice: Make sure you disable Chrome’s new FLoC for your site. Read more about FLoC.

Privacy advice from the coach

/Peter