Documentation / Performance Budget
Performance Budget
Performance budget #
Have you heard of a performance budget? If not, please go read these excellent posts by Tim Kadlec. Don’t worry, we’ll be here when you get back. Setting a performance budget and Fast enough. You should also read Daniel Malls’ How to make a performance budget. Welcome back - let’s continue the setup of sitespeed.io performance budgets. :)
How it works #
When you run sitespeed.io configured with a budget, the script will exit with an exit status > 0 if the budget fails. Else the exit code is 0. It will log all budget items regardless if they pass or fail and generate a HTML report.
The log will look something like this:
[2019-01-20 19:58:18] ERROR: Failing budget timings.firstPaint for https://www.sitespeed.io/documentation/ with value 462 ms max limit 100 ms
[2019-01-20 19:58:18] ERROR: Failing budget size.total for https://www.sitespeed.io/documentation/ with value 23.6 KB max limit 1000 B
[2019-01-20 19:58:18] INFO: Budget: 3 working and 2 failing tests
And if you check the return code after your run and you have failing budgets the exit code will be larger than zero.
echo $?
1
The report looks like this. 
Timing metrics (like first visual change) uses the median metric of all runs. So if you wanna have more stable metrics, increase the number of iterations/runs that you test one URL.
The budget file #
In 8.0 we introduced a new way of configuring budget. You can configure default values and specific for a URL. In the budget file there are 5 couple of sections:
- timings - are Visual and technical metrics and are configured in milliseconds (ms)
- requests - the max number of requests per type or total
- transferSize - the max transfer size (over the wire) per type or total
- thirdParty - max number of requests or transferSize for third parties
- score - minimum score for Coach advice
Simple budget file #
The simplest version of a budget file that will check for SpeedIndex higher than 1000 ms looks like this:
{
"budget": {
"timings": {
"SpeedIndex":1000
}
}
}
Override per URL or alias #
All URLs that you test then needs to have a SpeedIndex faster than 1000. But if you have one URL that you know are slower? You can override budget per URL.
{
"budget": {
"https://www.sitespeed.io/documentation/": {
"timings": {
"SpeedIndex": 3000
}
},
"timings": {
"SpeedIndex":1000
}
}
}
If you use alias for URLs, you can use that instead:
{
"budget": {
"myAlias": {
"timings": {
"SpeedIndex": 3000
}
},
"timings": {
"SpeedIndex":1000
}
}
User Timing API metrics #
You can use User Timing API metrics in your budget. Both marks and measurements will be picked up under the name usertimings. Sitespeed.io will first look for a mark with that name, and if that do not exist it will look for a measurement.
{
"budget": {
"usertimings": {
"headerLogo":1000
}
}
}
Metrics from scripting #
You can use metrics from your scripts in your budget.
{
"budget": {
"scriptingmetrics": {
"myOwnMetric": 20
}
}
}
Full example #
Here is an example of a fully configured budget file.
{
"budget": {
"timings": {
"firstPaint": 1000,
"pageLoadTime": 2000,
"fullyLoaded": 2000,
"FirstVisualChange": 1000,
"LastVisualChange": 1200,
"SpeedIndex": 1200,
"PerceptualSpeedIndex":1200,
"VisualReadiness": 200,
"VisualComplete95": 1190
},
"requests": {
"total": 89,
"html": 1,
"javascript": 0,
"css": 1,
"image": 50,
"font": 0,
"httpErrors" : 0
},
"transferSize": {
"total": 400000,
"html": 20000,
"javascript": 0,
"css": 10000,
"image": 200000,
"font": 0
},
"thirdParty": {
"transferSize": 0,
"requests": 0
},
"score": {
"bestpractice": 100,
"privacy": 100,
"performance": 100
}
}
}
If you use WebPageTest you can configure:
{
"budget": {
"webpagetest": {
"SpeedIndex": 1000,
"lastVisualChange": 2000,
"render": 800,
"visualComplete": 2000,
"visualComplete95": 2000,
"TTFB": 150,
"fullyLoaded": 3000
}
}
}
If you use Lighthouse you can configure:
{
"budget": {
"lighthouse": {
"performance": 90,
"accessibility": 90,
"best-practices": 90,
"seo": 90,
"pwa": 90
}
}
}
If you use GPSI you can configure:
{
"budget": {
"gpsi": {
"speedscore": 90
}
}
}
And then you can always combine them all.
If you need more metrics for your budget, either create an issue or look below for using the full internal data structure.
All possible metrics you can configure #
Here’s a list of all static metrics you can configure. Remember that you can also use your own metric, either from the User Timing API (marks/measures) or metrics from scripting.
{
"budget": {
"googleWebVitals": {
"timeToFirstByte": limit,
"firstContentfulPaint": limit,
"largestContentfulPaint": limit,
"interactionToNextPaint": limit,
"totalBlockingTime": limit,
"cumulativeLayoutShift": limit,
},
"timings": {
"firstPaint": limit,
"firstContentfulPaint": limit,
"largestContentfulPaint": limit,
"loadEventEnd": limit,
"fullyLoaded": limit,
"serverResponseTime": limit,
"backEndTime": limit,
"pageLoadTime": limit,
"FirstVisualChange": limit,
"LastVisualChange": limit,
"SpeedIndex": limit,
"ContentfulSpeedIndex": limit,
"PerceptualSpeedIndex": limit,
"VisualReadiness": limit,
"VisualComplete95": limit,
"VisualComplete99": limit,
"VisualComplete": limit,
},
"cpu": {
"totalBlockingTime": limit,
"maxPotentialFid": limit,
"longTasks": limit,
"longTasksTotalDuration": limit,
},
"browser": {
"cpuBenchmark": limit,
},
"pageinfo": {
"cumulativeLayoutShift": limit,
"domElements": limit,
"documentHeight": limit,
},
"requests": {
"total": limit,
"html": limit,
"javascript": limit,
"css": limit,
"image": limit,
"font": limit,
"httpErrors": limit,
},
"transferSize": {
"total": limit,
"html": limit,
"javascript": limit,
"css": limit,
"image": limit,
"font": limit,
"favicon": limit,
"json": limit,
"other": limit,
"plain": limit,
"svg": limit,
},
"contentSize": {
"total": limit,
"html": limit,
"javascript": limit,
"css": limit,
"image": limit,
"font": limit,
"favicon": limit,
"json": limit,
"other": limit,
"plain": limit,
"svg": limit,
},
"thirdParty": {
"transferSize": limit,
"requests": limit,
},
"firstParty": {
"requests": limit,
"transferSize": limit,
"contentSize": limit,
},
"score": {
"score": limit,
"bestpractice": limit,
"privacy": limit,
"performance": limit,
},
"lighthouse": {
"performance": limit,
"accessibility": limit,
"best-practices": limit,
"seo": limit,
"pwa": limit,
},
"webpagetest": {
"SpeedIndex": limit,
"lastVisualChange": limit,
"render": limit,
"visualComplete": limit,
"visualComplete95": limit,
"TTFB": limit,
"fullyLoaded": limit,
},
"gpsi": {
"performance": limit,
"accessibility": limit,
"best-practices": limit,
"seo": limit,
"pwa": limit,
},
"sustainable": {
"totalCO2": limit,
"co2PerPageView": limit,
"co2FirstParty": limit,
"co2ThirdParty": limit,
},
"axe": {
"critical": limit,
"serious": limit,
"minor": limit,
"moderate": limit,
},
}
}
Budget configuration using the internal data structure #
There’s also an old version of setting a budget where you can do it for all metrics collected by sitespeed.io and works on the internal data structure.
You can read more about the metrics/data structure in the metrics section.
{
"browsertime.pageSummary": [{
"metric": "statistics.timings.firstPaint.median",
"max": 1000
}, {
"metric": "statistics.visualMetrics.FirstVisualChange.median",
"max": 1000
}],
"coach.pageSummary": [{
"metric": "advice.performance.score",
"min": 75
}, {
"metric": "advice.info.domElements",
"max": 200
}, {
"metric": "advice.info.domDepth.max",
"max": 10
}, {
"metric": "advice.info.iframes",
"max": 0
}, {
"metric": "advice.info.pageCookies.max",
"max": 5
}],
"pagexray.pageSummary": [{
"metric": "transferSize",
"max": 100000
}, {
"metric": "requests",
"max": 20
}, {
"metric": "missingCompression",
"max": 0
}, {
"metric": "contentTypes.css.requests",
"max": 1
}, {
"metric": "contentTypes.image.transferSize",
"max": 100000
},{
"metric": "documentRedirects",
"max":0
}]
}
Then run it like this:
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:33.6.0 https://www.sitespeed.io/ --budget.configPath myBudget.json -b chrome -n 11
And, if the budget fails, the exit status will be > 0. You can also choose to report the budget as JUnitXML (Jenkins) or TAP.
JUnit XML #
You can output a JUnit XML file from the budget result like this:
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:33.6.0 https://www.sitespeed.io/ --budget.configPath myBudget.json --budget.output junit -b chrome -n 5
It will create a junit.xml in the outputFolder.
TAP #
If you would instead like to use TAP, you can do so like this:
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:33.6.0 https://www.sitespeed.io/ --budget.configPath myBudget.json --budget.output tap -b chrome -n 5
It will create a budget.tap in the outputFolder.
JSON #
You can output the result of the budget as JSON:
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:33.6.0 https://www.sitespeed.io/ --budget.configPath myBudget.json --budget.output json -b chrome -n 5
It will create a budgetResult.json in the outputFolder.
Remove working/passing result #
There’s a feature where you can configure sitespeed.io to remove data (the result HTML/videos/screenshots) for all pages that passes your budget. This is useful if you crawl your site and only want to keep the result of the pages that fails, to save space. Use --budget.removeWorkingResult to remove data for pages that works.
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:33.6.0 https://www.sitespeed.io/ --budget.configPath myBudget.json --budget.removeWorkingResult -b chrome -n 5