Once upon a time me and a few cohorts wondered about just what levels of flux there were in the Google SERPs. Has personalized search really changed the consistency of rankings? It’s an issue that has been spoken about many times in the search world. We set out to see what was up. After two rounds of research we noticed that this was unlikely to be the case. You can learn more in the post; The SEO guide to Google personalized search
It seemed only sensible, given Google’s Psearch expansion, to have another look. And so last December we started a third round. Has anything changed? How much flux is out there? Well, read on and find out what we’re seeing so far…
Now, first things first; the goal wasn’t really to establish how personalized search operates. What we really wanted to know is how much movement is there in the SERPs for a given query type, in a given region (USA in this case). Yes we’re also noting some potential personalized search affects, but movement/flux was the core observation.
We must understand that even though there is some interesting data here, it is by no means a large enough sample to get nitty gritty with IMHO. It is also important to remember this is a specific task from a quasi-transactional query session (task development).
We want to go back and do more from an informational query space and one more suited to respondents (familiar, such as ‘learn SEO’ etc..). This approach, last time out, did show more movement than we saw with this space (we’ve done this round 2x now).
As always, no magic bullets here… Just more links in the chain (pun intended of course).
The Set up
We decided to take a set of queries to build a session (task) in a space that may or may not be familiar to the respondents. The queries we used were;
‘buy antique lamps’
‘buy lamps online’
We also asked them to tell us;
- Where they’re located (region/country)
- Which browser they’re using
- Cleared search history lately?
- Is personalized/search history on?
- Most common Google app used
- If they had searched for ‘lamps’ or furnishings in the last 60 days
The goal being to start understanding what may or may not be influencing the search result flux. While we had some questions to be answered, there was no initial bias (hypothesis) behind the analysis. It should also be noted that the entire collection process only lasted 4 days so that we could try our best to avoid any temporal anomalies.
Today we’re looking at the data from the US. At the end of the test period there was really only enough to do analysis on the US and UK markets. I shall post the UK data in the coming weeks.
US Respondent Stats;
Most used Google app;
Cleared Cache lately?
Related searches in last 60 days?
Is personalized search on?
What have we learned?
It’s a good question, one that I don’t think we have a definitive answer to. Let us remember this is but one query type in a limited set. Personalization is more likely to have an effect in a query space where the user is more active in. That being said, there were some interesting elements that emerged so far.
Much like the original round of testing, a year ago, we can say that there is no massive upheaval where SERPs are vastly different from one user to another. This much has been consistent in each of the (3) rounds in both informational and transactional query spaces.
Here are some notes on other questions we’ve had;
Are the top 3 stable?
Ultimately not much has changed as far as aiming for the sweet spot (above the fold). These are easily the most stable listings and it is unlikely these would be affected more than 95% of the time and even then, rare to drop too far down when they do.
Are top 10 stable?
Where is the biggest re-rankings?
We’re considering running the test again while reversing the order to see if indeed the order of the session queries created the increased movement the further we went into the process. It does seem like this is an important element to watch for in future testing.
What causes SERP flux?
What do strong listings have in common?
Indented listings would often be vulnerable but other than that, the higher the ranking, the more security one would have. Beyond that we’d need to actually dig into each of the sites for further clues.
Is related searches past 60 days making a difference?
Are concurrent tasks creating a difference?
Are browsers making a difference?
So far, I’d say that this doesn’t play into it….or at least isn’t much of a signal of note.
|Is Psearch on making a difference?
This one I still have to say not really. At least not in the traditional sense of having surfing history enabled. Actually, some of the odder SERP flux came from those that weren’t actively browsing with it turned on. If there is a strong Psearch in play, it wasn’t due to past related searches.
|Is regional location creating flux?
While there is some flux, there was no major insertion of listings that could be considered regional. This being said, we’d likely want more regional data and more targted queries to better establish this to be the case.
PPC and Uni results. – it was also interesting to see how the Shopping results popped in and out. While it is beyond the scope if this research, they were interesting and far more irregular than the SERP flux. This does play into one’s placement in the top 10 and is something to look at down the road.
Indented listings came and went (on final query) – another finicky element were the indented listings. It seems those can be more finicky than the average flux/movement in the SERPs. Looking for this in the next round and digging deeper if it persists is another area of interest.
Click data – given the suspicions of query analysis and task related triggers, in the near future introducing random clicks on links (and return/bounce?) might be worth putting into the mix. We haven’t given many signals beyond merely a 3 query task. If we get more evidence of this being an element, deepening the tasks seems a likely next route (past further rounds like this one).
What does it mean for SEOs?
Once upon a time this journey began because I really wanted to know if ‘rankings are dead’ and if ‘everyone gets a different SERP’. A few of the age old, but poorly understood, questions of the search world. So, have they been answered? Well, sort of.
Rankings are dead? – Yes and no. The old days of ranking reports telling us one ranks in X-position, seem to not be a valid approach. Should we be tracking/watching rankings? Most certainly we want to be getting top 10 and then above the fold to ensure the best performance.
I’d also suggest that you might occasionally get people in a targeted region to check SERPs for you now and again.
Everyone sees a different SERP? – Yes and no. The rankings are reasonably consistent as far as the make up of the top 10. Even the movement isn’t massive when there is some.
But are they ALL the same… most certainly not.
We shouldn’t be changing how we approach things ultimately… Above the fold is the real estate that’s prime (what else is new?). It may be the measuring that we will have to adapt. You will need to find ways to check rankings from a few locales and discern a mean average instead of a definitive placement.
We really have only touched the surface so far (with this round) and are going to want to start looking at a few more things including;
As you may have started to notice, there are still a ton of elements which are unknown. This is why I am still not making any major calls on what we’re seeing. Please bear in mind that we’ve only 2 rounds last year and one recent set of data. This really only begins to give us some insight; nothing more. The last thing we need are SEOs running around taking any of this as gospel at this point.
In the short term we’re going to sort the UK data from this round and I will be posting about that in the coming weeks… Stay tuned.