If you’ve been using Google’s AdWords and/or Analytics products for any length of time, you must be aware that the numbers don’t always add up. In fact, they don’t add up at a sometimes alarming frequency. That is to say, the reported numbers either don’t match up to what you know must be reality, or they are not consistent. Needless to say, this realization can be a little disconcerting, considering the amount and level of detail Google provides.
I ran across yet another curious example of this the other day. One of the features in the new AdWords interface about which I was most excited was the in-line “see search terms” button in the keywords tab of ad groups. The button pops up a quick report that promises to show you the most frequent search entries that actually resulted in displays of your ads for selected keyword(s) or the entire group. This information was available before, but only by going several keystrokes and screens away into a formal report. The ability to pop this information up while still in my keywords list–and even add any newly-discovered keywords right from the list–seemed to me revolutionary.
My thinking was that this listing would be most valuable in discovering exact match keywords I should be bidding on. For example, let’s say I have a phrase match keyword “rugby shirts” that my keywords tab shows as performing at a low but reasonable CTR. The “see search terms” report reveals that this keyword has generated lots of impressions for all kinds of related phrases (“izod rugby shirts,” “boy’s rugby shirts” etc.). But most interestingly, it seems to reveal that the keyword “rugby shirts” as an exact match (i.e., the person searching entered the words “rugby shirts” and only those words) generates a hugely higher CTR than the average for the phrase keyword over all. It seems to be a no-brainer that I should create an exact match for that keyword, and bid it higher as it is more productive (in terms of CTR, but not actual click-quantity, of course).
This I’ve been doing for over a month now, but my new-found joy was tarnished a bit recently by a couple of discoveries. First, I noticed that in some cases, the newly-created exact match didn’t perform anywhere near as well as might be predicted from the “see search terms” report. I chalked this up to the occasional anomaly, and kept in mind that “past performance does not guarantee future results.” But then came the second blow. Yesterday I happened to run the “see search terms” report with one of my recently-created exact matches as the only selection, just out of curiosity. In the ad group keyword list, it showed 2 clicks with 16 impressions (a CTR of 12.50%). But when I went into the “see search terms” report, the exact match row for this keyword showed 2 clicks with 3 impressions, a CTR of 66.67%! Then in the standard “other search terms” row below that, it showed 0 clicks with 13 impressions. Now together those add up to the the 2/16 numbers shown in the ad group keyword list.
OK, but here’s the question: Why would the report relegate a significant percentage of the impressions for the exact match to the “other search terms” row? There is no other search term for an exact match. I put that question to an AdWords rep in a chat, and he told me he’d need to talk to someone in “technical” and get back to me. Today he responded. Seems that the report is “unreliable” for exact matches. He assured me that all of the 16 impressions were legitimately for display of the exact match, yet could not explain why the search terms report dumps 13 of them into “other search terms.”
This would all be academic, were it not for the discouraging implication: the “see search terms” report can’t be relied upon to consistently point out high-performing exact matches. The CTR it reports for exact matches is often (always?) going to be inflated, because a significant number of the impressions have been inexplicably peeled off into the “other search terms” row. I’m still going to use the tool for exact match discovery (many of the ones I have tried have performed very well, if not up to the level of the report), but with a much more jaundiced eye.
The more important–and jarring–lesson is that Google’s reports and stats probably have far more squirmy-wormy room in them than those of us who depend upon them to do our jobs well would like to believe.