It's easy to picture Google flipping a switch and sending a new algorithm out to billions of searchers overnight. The reality, as Google describes it, is a long chain of evaluation. Proposed changes are scored by trained human raters, compared side by side with current results, tried on small slices of real traffic and reviewed by senior engineers before they launch.
Understanding that process is useful for anyone who works in SEO. It explains why Google talks about "improvements" rather than "penalties", why you may see results nobody else can reproduce and why core updates take days or weeks to roll out. It also offers a model for testing changes on your own site.
This guide walks through the testing stages Google describes in its "How Search Works" materials, with the figures Google has published, and what they mean for site owners in 2026.
Key Takeaways
- Google describes several stages of testing: search quality tests with raters, side-by-side experiments, live traffic experiments and a launch review.
- For 2023, Google reports 719,326 search quality tests, 124,942 side-by-side experiments, 16,871 live traffic experiments and 4,781 launches.
- Google says that if it can't show a change makes things better for people, it doesn't launch it.
- Rater feedback helps Google benchmark quality, but ratings don't directly affect the ranking of any page.
- Live experiments mean some searchers see results that aren't final, which can explain odd rank-tracker readings.
The scale of testing
Google's How Search Works page on rigorous testing publishes yearly totals. The 2023 figures show how much testing sits behind each launch.
| Activity (2023) | Number reported by Google | What it involves |
|---|---|---|
| Search quality tests | 719,326 | Raters evaluate the quality of results for sample queries |
| Side-by-side experiments | 124,942 | Raters compare two sets of results and say which is better and why |
| Live traffic experiments | 16,871 | A small share of real users sees a change, compared with a control group |
| Launches | 4,781 | Changes approved and released to Search |
Put simply, only a small share of tested ideas end up as launches. Google sums up the principle behind that filter:
“If we can’t show that a change actually makes things better for people, we don’t launch it.”
— Google, How Search Works, Rigorous testing
The testing stages, step by step
Based on Google's descriptions, a change to Search typically moves through these stages:
- Idea and development. Engineers identify a problem, such as low-quality results for a type of query, and build a proposed change.
- Search quality tests. Raters assess results produced with the change for samples of real queries, using Google's Search Quality Rater Guidelines.
- Side-by-side experiments. Raters are shown two sets of results, current and proposed, and indicate which they prefer and why.
- Live traffic experiments. The change is turned on for a small percentage of users. Google compares how they interact with results against a control group.
- Launch review. Google says every proposed change goes through a review by its most experienced engineers and data scientists, who look at the data from all the tests.
- Rollout. Approved changes launch. Larger ones, such as core updates, roll out gradually and are announced on the Search Status Dashboard.
Raters: measuring, not ranking
Google works with about 16,000 external raters, according to its overview of the rater guidelines. Their ratings are aggregated to measure how well systems perform. Google is clear about their limits:
“These ratings do not directly impact ranking, but they do help us benchmark the quality of our results.”
— Google, How Search Works, Rigorous testing
Live experiments: real users, real behaviour
Rater tests show whether results look better on paper. Live experiments show whether people actually find them more useful. Because those experiments run on a small share of traffic, you or your rank tracker may sometimes see results that differ from what most people see. That's one reason we advise against reacting to single-day ranking movements, as explained in why rankings change between updates.
From testing to announced updates
Most launches are small and unannounced. Bigger, broader changes are bundled into updates that Google announces, such as core updates and spam updates. Google's core updates documentation says it gives notice when these happen, while smaller core updates are not announced because they aren't widely noticeable.
| Type of change | Typically announced? | Where to check |
|---|---|---|
| Broad core update | Yes | Search Status Dashboard |
| Spam update | Yes | Search Status Dashboard |
| Smaller core updates | No | Not announced |
| Continuous system improvements (e.g. reviews system) | No | Not announced |
| Widespread ranking or indexing issues | Yes, as incidents | Search Status Dashboard |
| New SERP features and experiments | Sometimes, via Google blogs | Google's product and Search Central blogs |
Our guides to how Google rolls out core updates and the Google Search Status Dashboard cover the announcement side, and SERP changes vs algorithm updates explains how interface experiments differ from ranking changes.
What site owners can learn from Google's approach
Google's process is a good template for your own SEO decisions:
- Define “better” before you test. Google measures against a written quality standard. Decide which metric (clicks, conversions, engagement) defines success for your change.
- Use control groups. Compare changed pages with similar unchanged pages, as in SEO A/B testing.
- Test before rolling out site-wide. Try a template change on a subset of pages first.
- Combine judgement and data. Human review catches problems that metrics miss, and vice versa.
- Don't ship what you can't prove. If a change doesn't improve the metric that matters, roll it back.
The same principles apply beyond SEO. Our A/B testing guide covers experiment design in more depth.
Common mistakes
- Believing a rater visited your site and lowered it. Raters evaluate samples to measure systems. They don't change rankings.
- Treating one odd result as an update. It may be a live experiment you happened to be included in.
- Assuming every launch is announced. Thousands of launches a year are not.
- Testing your own changes without a control. Without one, seasonality and Google changes can be mistaken for your results.
- Reading too much into experiment screenshots. Features seen in tests may never launch.
Related Guides
- Who Names Google Updates? From Florida to Medic
- Why Google Stopped Announcing Some Updates
- 12 Common Myths About Google Algorithm Updates
Frequently Asked Questions
How does Google test changes to Search?
Google describes search quality tests with raters, side-by-side experiments, live traffic experiments on a small share of users, and a launch review by senior engineers and data scientists.
How many experiments does Google run?
For 2023, Google reports 719,326 search quality tests, 124,942 side-by-side experiments and 16,871 live traffic experiments, leading to 4,781 launches.
Do quality raters decide rankings?
No. Google says ratings do not directly impact ranking but help it benchmark the quality of results.
Can I see Google experiments in my own searches?
Possibly. Live traffic experiments show changes to a small share of users, so your results may occasionally differ from what most people see.
Are all tested changes announced?
No. Only larger changes, such as core and spam updates, are typically announced. Most launches ship without notice.
Conclusion
Every Google update you read about has passed through layers of human evaluation and live testing. That's why the best response to change is the same as Google's own: define quality clearly, measure carefully and improve what helps people. For a wider view of how updates fit together, read our hub on Google algorithm updates. If you'd like help building a testing culture into your SEO, request a free quote.
References
- Rigorous testing — How Search Works (Google)
- Search Quality Rater Guidelines: An Overview (Google PDF)
- Google Search's core updates — Google Search Central
- Creating helpful, reliable, people-first content — Google Search Central
- Ranking incident history — Google Search Status Dashboard
- How Google's Search Algorithm Works — Semrush Blog
