If you’ve been reading some of my blog posts in the past you’ll have noticed that I worked a lot around analytics, experimentation and SEO. Having been able to combine these areas together has led to the point that for both Postmates and The Next Web previously we worked on setting up a framework for SEO Experimentation. In my opinion over time you’d like to learn more about what Google appreciates and how it can help you future wise, to think about what you should be focusing your attention on == prioritization. Lately I read a blog post on the SEMrush blog with the title: SEO Experiments that will blow your mind (clickbait alert!). Hoping there would be a lot of great ideas in that blog post I started reading realizing that over 50% of examples weren’t even close to being an experiment. They were just plain research, from over a year ago (which was alarming too!). Which pushed me to write this blog post to tell you more about what SEO experimentation really is and how you can learn more about it as it’s still a relatively new area in SEO that is only been explored and exposed by a few big companies.
Update: since publishing this post I’ve became even more excited about SEO Experimentation and its possibilities. That’s why I’ve updated it with more information and some frequently asked questions that I get about the subject.
What really is, SEO Experimentation?
You’re testing 2-X variations of a template on your site (all product pages, all category pages) and measure over a period of time what the impact is in organic search traffic of the pages that have seen changes in your experiment. You want to isolate your changes as much as possible, you set a certain level of significance and you calculate in a mathematical way how your results have changed in the period before and after the change.
- Research (I compare data between periods too, but it doesn’t make it an experiment).
- Guess work: I’ve seen my pageviews go up with % after optimizing my title on this page.
I would encourage you to also read this post on Moz by Will Critchlow in which he shared how they built Distilled ODN (we worked with their platform at TNW) and how you can be testing with SEO yourself.
How is SEO Experimentation different?
Measurement: Instead of measuring conversion rates or other business metrics you’re mostly focused on tracking the number of sessions that is coming in from organic search. If the variants that you’ve worked on increase you’ll start calculating the impact on this. On the side you’re still making sure that the conversion rate of these pages doesn’t decline as in the end it will still be the business metrics that will count.
Bucketing: Instead of bucketing users and sending user A into bucket A you’re doing this with pages. In the end you want to make sure that the same pages (just like you do with users) end up in the same bucket. How you usually do that is that you sort by alphabet or category to have some kind of logic. What’s important though is to make sure that these buckets are in balance. Which is harder than you would do with user testing.
What are some examples of SEO Experiments?
Examples of things that you could be testing? Let’s list a few examples that I can think of:
- Updating Title tags: does a certain keyword help, do I need to make them longer/shorter?
- Adding/Removing Schema.org, what is the impact of adding structured data?
- Adding content: does adding more content to a page help with its ranking?
- Testing Layouts, what lay-out does help better for the SEO on this page? Ever noticed why Amazon has so many different product pages 😉
How do I start doing SEO Experimentation?
Let me give you a very much shortened and simplified idea of how an SEO experiment works:
- Think about what you want to be testing (adding descriptions, setting a canonical), write down a hypothesis.
- Figure out if you have enough traffic (500+ per variation per day) would be a minimal I’d say.
- Figure out how you can serve different content/HTML per page. Are you able to somehow serve different areas based on a randomized factor, still making sure that your buckets are just as big?
- Setup the experiment and let it run for as long is needed to see valid results (usually at least 3-4+ weeks).
- Analyze the results after this period and look how these buckets of pages performed before your experimen and after. Are these results valid, you didn’t make any other changes to these pages in the meantime? Are the results significant?
- You have found what the winner + loser is. Time to start another experiment.
Isn’t this cloaking? Doesn’t this hurt for my Google rankings?
Not really, you’re not changing anything based on who’s looking at the page. You’re changing this only on certain pages that are being served and the search engine + user will see the same thing. Ever looked at Amazon’s product pages and wonder why they all have a different layout? Because they’re testing for both user experience as SEO.
You want to learn more about this subject?
Great, when we were setting up our own SEO experimentation framework at Postmates about 6 months ago I tried to find all the articles related to it and talked to as many people as possible on this. These were mostly the articles I would refer you to, if you want to learn more.
Let’s really start innovating in the SEO industry and let’s get rid of terrible clickbait headlines. SEO Experimentation is like I mentioned something that we should be embracing as it’s a more scientific approach that is going to lead to finding new insights. If you want to talk more about this, feel free to reach out, at some point I might write a longer post explaining what you should be thinking off while building your own framework for SEO experimentation.