Google’s Amazing Domain Diversity Metrics

I have always been intrigued by the organic search ecosystem. You can think of it like a marketplace where different sites come to life, grow, thrive, or die depending upon their performance. Like the natural ecosystem governed by physical laws, or the marketplace governed by financial law, Google’s search results are governed by the algorithm. It is hard to know when features of the ecosystem are a product of natural occurrence given the interaction of participants within the environment, or when they are deliberately controlled. This is the case with domain diversity.

We know that in past Google has shown an interest in maintaining some level of domain diversity. I am not the only one to have complained about Wikipedia dominating the search results at one point or another, and certainly webmasters have felt the squeeze before when domain clustering seemed to fill the search results with just one or two sites at a time. But what is perhaps most remarkable is how steady domain diversity has been in Google over the last 2 years.

Using keywords randomly pulled from GrepWords, and the historical data from SEMRush, I took a look at domain diversity from 4 different standpoints:

1. Unique Domains per Keyword (ie: the total number of unique domains divided by the total number of keywords)
2. Percentage of Ranking Slots Maintained by the top 2.5% of Domains
3. Estimated Traffic Share of Top 2.5% of Domains
4. Estimated Value Share of Top 2.5% of Domains

The results were startlingly steady. It is almost uncanny. The number of unique domains per keyword hovered between 5.46 and 4.93 over the last 2 years, roughly 1/2 of 1 domain. The top 2.5% has regularly maintained market share in terms of ranking slots, estimated traffic and estimated traffic value.

Date	Keywords	DPK	Top 2.5%	Visits	Values
d-20150515	677	5.29	0.3	0.38	0.34
d-20150415	676	5.27	0.31	0.35	0.33
d-20150315	668	5.26	0.3	0.36	0.35
d-20150215	668	5.27	0.3	0.38	0.37
d-20150115	668	5.26	0.3	0.37	0.38
d-20141215	674	5.29	0.31	0.4	0.4
d-20141115	674	4.93	0.27	0.39	0.35
d-20141015	675	5.35	0.3	0.41	0.4
d-20140915	674	5.26	0.3	0.42	0.43
d-20140815	675	5.28	0.31	0.41	0.41
d-20140715	587	5.36	0.3	0.4	0.4
d-20140615	587	5.31	0.3	0.4	0.37
d-20140515	587	5.21	0.32	0.4	0.4
d-20140415	587	5.25	0.31	0.39	0.35
d-20140315	586	5.18	0.32	0.41	0.39
d-20140215	575	5.22	0.32	0.4	0.36
d-20140115	575	5.27	0.31	0.4	0.37
d-20131215	571	5.27	0.31	0.4	0.34
d-20131115	573	5.29	0.31	0.41	0.43
d-20131015	570	5.44	0.31	0.36	0.36
d-20130915	587	5.41	0.3	0.38	0.4
d-20130815	569	5.46	0.3	0.38	0.36
d-20130715	568	5.45	0.31	0.39	0.37

Looking at these numbers makes me believe that Google is actively policing domain diversity as a quality metric. While it is pure conjecture, I think this data suggests that Google looks at the impact of future algorithm updates in terms of their impact on an ideal domain-diversity level. This may likely keep the ecosystem fresh and churning, rather than monopolized by a series of mega sites (even though it certainly may not feel that way).