Announcing my Technical SEO Course on CXL Institute

If there was one thing that I could teach people in SEO, it was always the technical side of SEO that came up first. Mostly, because I think it’s a skill that doesn’t suit too many SEOs and there is already enough (good or bad, you’ll be the judge of that) content about the international, link building or content side of SEO out there. As technical SEO is getting more and more technical and in-depth about the subject itself, I’m excited to announce that I’m launching a new technical course with the folks of CXL institute.

The course will cover everything from structured data to XML sitemaps and back to some more basic on-page optimization. Along the way, I show you my process for auditing a site and coming up with the improvements. I’ll try to teach you about as many different issues and solutions as I could think of.

It’s not going to be ‘the most complete’ course ever on this topic, technical SEO evolves quickly, and likely some things will already be outdated now it’s published, while we have worked on it for months. But I’m going to do my best to inform you here and on CXL Institute about any changes or any improvements that we might be able to make in a future version. If you have any questions about the course or want to cheer me on, reach out via Twitter on @MartijnSch.


Keyword Gap Analysis: Identifying your competitor’s keywords with opportunity (with SEMrush)

Keyword research can provide you with a lot of insights, no matter what tool you’re using they all can provide you a great deal of insight into your own performance but also that of others. But while I was doing some keyword research I thought about writing a bit more about one specific part: gap analysis. In itself an easy to understand the concept, but it can provide a skewed view of your competition (or not). To demonstrate this we’ll take a look at an actual example of some sites while using SEMrush’s data.

What does your competition look like?

You know who your competitors are right? At least your direct ones, but often people who work in SEO or the level above (Marketing/Growth) don’t always know who the actual players are. I worked in the food delivery industry, but more often than not I was facing more competition from totally random sites or some big ones than our competitors (for good reason). So it’s important to know what your overlap is in the search rankings (it’s one of the reasons you should actually be tracking rankings, but that’s a topic for another day) with other sites. This way you know what competitors are rising/declining in your space and what you can learn from their strategies to apply to your own site. But this is exactly where the caveat is, is that actually the case!?

So let’s look at an example, as you can see in this screenshot from SEMrush the playing field for Site A is quite large. They’re ‘ranking’ for tens of thousands of keywords and are placing in a decently sized industry. While they’re ahead of their competition it’s also clear that there are some ‘competitors’ in the space that are behind them in search visibility.

So let’s take the next competitor, we’ll call them ‘Competitor A’. What we see here is that they rank for 250.000 keywords. A significant number still, compared to what we’re ranking for. It doesn’t mean though that all their keywords are what we’re ranking for. So let’s dive into gap analysis.

Keyword Gap Analysis

In short, there are three ways to look at keyword analysis:

  • What keywords am I ranking for, that my competitor is also ranking for (overlap)?
  • What keywords am I ranking for, that my competitor is not ranking for (competitive advantage)?
  • What keywords am I not ranking for, that my competitor is ranking for (opportunity)?

Today we’ll only talk about the last one, what keywords could I be ranking for, as my competitor is already ranking for them, to drive more growth. When using SEMrush you can do this by creating a report like this (within the Keyword Gap Analysis feature):

Screenshot of: Creating a keyword gap analysis report in SEMrush

You always have the three options available to select. In this case, we’ll do the Common Keywords Option. And the result that we should see looks something like this:

Screenshot of the result, with a list of keywords.

What are the keywords with actual opportunity?

So there is apparently xx.xxx keywords that I’m not ranking for (and likely should). That’s significant and almost leads me to believe that we’re not doing a good job. So what the problem often is, which is not a bad thing. Is that the majority of these keywords are being driven by the long tail (specific queries with very low volume). So what ends up happening is that I’m likely looking at tons of keywords that I don’t want to focus on (and hopefully will benefit from by just creating a little bit more generic good content). So when I did this for a competitor and filtered down on keywords that were for them at least ranking position <20 and had a volume >10 monthly I had only 2500 keywords left. That’s just a few % of the keywords that we got started with. It’s required to add that I’m not saying to ignore the other keywords, but now you have the keywords that you have a real opportunity to drive actual results. In the end, you should be able to rank well, as your competitor is already ranking (position: <20), there is actual volume (>10) and you’re not in there at all.

This is just something that I was playing with while exploring some industries, and it’s a topic that I haven’t seen a lot of content about over the last years. While the data is often available it will both help you get new content ideas but also helps you identify the actual value (keyword volume should turn into business results: impressions x CTR x Conversion Rate == $$$) on the revenue side.


Calculating Click Through Rates for SEO, based on Google Search Console Data (in R)

Updated June 4, 2019: Added a YouTube video which will guide you through the setup process in RStudio and how to run the script yourself.


Averages lie & average click-through rates aren’t very helpful! Here I said it. What I do believe in, is that you can calculate click-through rates (CTR) for your own site in a great way though. With data from Google Search Console. Especially while using R, so let’s dive in on how this works and what you can do with it!

Why ‘Averages’ Lie?

Look at the graph below, a great CTR for position 1. But for example, this research shows that the average CTR for position 1 is: 24% (AWR data, Feb 2019). Which one is correct? Neither of them. As it really depends on the industry, what features show up in the search results that might decrease CTR (think rich snippets like local packs, news results). All of this is making it really hard to make a good analysis of what you could expect if you rank higher for a bunch of keywords in your industry. So while I was working at Postmates on ranking certain category pages better we decided to calculate our own CTR and were intrigued by how far CTRs were off from research (the research isn’t wrong! It’s just generalized across industries). Eventually, with the data in hand, we were able to make better estimates/forecasting of how much traffic we could expect when rankings would increase in that segment. In the rest of this post, I’ll go more in-depth on the specific practice on how we calculated this.

Using Google Search Console Data

Visual of Google Search Console Report (this is not RVshare data)

You’ve seen this report in Google Search Console, providing you with a detailed view of the performance of your keywords and the average position for your keywords. In this graph, we see something positive, a CTR & position that go up slightly over time. But what if you would want to know the average CTR for a certain segment of keywords per position. That’s way harder to do in the interface. Because of that, I used the R script from Mark Edmondson that he wrote about here almost three years ago.

It will help you extract the data from Google Search Console in a raw way so you can use it to digest it and create your own visualizations (like the one we’ll talk about next).

Visualizing CTR Curves in R

CTR Curve visualized CTR per Position (note: this is randomized data for an unknown site)

So let’s dive right into how you can do this yourself, I’ll provide you with the full R script and you will need to download RStudio yourself in step 1.

  1. Download and Install RStudio
  2. Download the following .r script from Gist
  3. Run these commands to install the right packages for RStudio:
    1. install.packages(“googleAuthR”)
    2. install.packages(“searchConsoleR”)
    3. install.packages(“dplyr”)
    4. install.packages(“ggplot2”) if necessary
  4. Line #21 – Change this to the property name from Google Search Console
  5. Line #25 – Not neccesary: If you want the CTR curve for positions over 20, change the number.
  6. Line #40 – Recommended: Exclude the word(s) that are part of your brand name. So you get the right CTR curve for non-branded keywords only
  7. Line #41 – Not necessary: This script is taking a ‘sample’ of 50.000 keywords to calculate your CTR curve of. You can increase this limit to more if needed, if you have less than 50.000 keywords it’s not an issue
  8. Run the script! The output should be a visual as shown earlier in this post

Want to take a deep breath and let me help you go over this again? I’ve made a quick screen share video of what to do in RStudio and how to use the R script.

Hopefully, now, you’ve had a better chance to understand what the actual CTR is for your own site and you can use this to visualize CTR curves for specific parts of your site or pages that have a similar META description. Over time you could use this, for example, for measuring the impact on CTR.

Credits where credits are due! There are many use cases for using CTR data by visualizing it with R, and I’m grateful that a while ago Mark Edmondson opened my eyes about this + credits to Tim Wilson’s documentation on using R and improving visualizations.

Want to read this article in Spanish? Read it here.


Love and/or Hate?! – SEOs & Developers

“I hate my developers,” “We deployed the wrong thing(s),” “Somebody put up a Disallow in my robots.txt that wasn’t meant to be there,” “The whole site is deindexed.” These are just some of the quotes that I’ve heard over the years on the relation between developers/engineers. Where usually, the developers are the ones that receive all the hate. It should stop. Usually, I try to reflect on what I would do in a situation like that. This often means that the SEO should have a little more reflection and figure out what they could have done to avoid a situation like that. In most cases, the answers that I get back are that they’re having a hard time working with their development team and don’t have a very good understanding of each other’s work.

How to work together & Creating a better understanding with developers

Speak their language, Learn their language, Use their tools/platforms, Ask for their input. Embrace their new technologies and ideas. Then, have a ton of fun!

  • Github/JIRA/Confluence: How is work assigned to your development team, what tools do you use for this and do you leverage the tools that they like using or do you require them to be stuck in a tool that they don’t like. I’m quite confident that they rather use the tools that are already part of their regular development cycle.
  • Retrospectives, Lunch & Learn: How often have you joined a retrospective of a development team or have been part of their lunch & learn. You don’t always have to know all the details about what they’ve worked on. But it for sure helps build up your own knowledge by learning more about their process/terminology and codebase/technology.
  • Pull Requests: Have you ever gone through the code that was about to be merged into your codebase to see specifically what kind of changes your developer made. It will help you double check if this is what you meant and gives you a good understanding on where certain parts that matter for SEO live.
  • Specs & Technical Requirements: Do you write your developers requirements, are you involved, have you ever read them to better understand what an integration takes? If you’ve always been used to rapid development on platform X doesn’t mean that platform Y is just as fast to have new implementations build for.
  • Write requirements yourself or write a testing plan: An SEO feature didn’t turn out what you expected it to be? Well, did you actually exlain how it was supposed to work and wrote all the edge cases for it? Did you provide the specific for the texts that you wanted to show up. I bet you didn’t. So work with your product managers on writing requirements and help guide your QA engineers on how this could be tested. There should always be a way for you to test business logic in releases besides the functional/technical aspect of it.
  • Embrace new technologies: Often developers have a better insight into what new technologies can help the platform that you work with. When I had no idea how a certain database + API could help us and started to get frustrated our engineering lead was happy to tell me that it would make rollouts of future landing pages way easier and avoided them having to use an old API that sometimes took days to deploy. Their efforts to implement this paid off big time already in the quarter after they had launched that (both in speed of the site and development time).

Educating Developers

Give your developers 1 fact a day about something relevant to them and SEO. Of course, you all care about certain topics already: user experience, speed, etc. But can you really expect them to know about canonical tags or the inner workings of HREF lang integrations? I’m convinced they shouldn’t know anything about that until they’ve met you. In the end, it’s not part of their education/learning curve, and it likely should never be there. On the other hand, if you guide your development team and tell them about why certain improvements need to happen, you’ll likely, over time, build up a valuable resource with your developers. Hopefully, in a way that they can actively contribute to your actual SEO strategy. That’s why I want to share two stories of how SEOs and Developers could also work together:

500 Server Request

A few years ago, when I worked internally with the TwitterCounter (RIP 🙁 ) team at The Next Web (one of their sister companies), the development team was planning to temporarily shut down the service for a few hours (2-6 hours depending on the progress). It was not great as a user experience, but it was inevitable as the servers were about to hit their limits and needed to be upgraded. Specifically for SEO, this meant that tens of millions of pages wouldn’t be accessible (including for crawlers) for the duration of the maintenance. Not great, but obviously, for SEO, you can take the right precautions. As our development had been sitting together with me for months (literally, they sat right next to me), I had the chance to slowly educate them on many SEO topics (including dealing with maintenance shutdowns). So when the time came to shut down the servers, the amazing question that came up was: “So, I imagine that we’ll put up a temporary 50X status code, so we will tell search engines that we are doing server maintenance).” I don’t think there was ever a better moment in time where developers stepped up with the right mindset and understanding about SEO.

Title experiments

At Postmates, we wanted to run experiments on title tags on different kinds of templates. As pages were crawled very frequently and there was enough traffic, it meant that we could run a few dozen experiments potentially yearly just on that. Which meant that it would be a lot of work every few weeks to create/update experiments. Instead, she approached us with the question if it was Ok to build a tiny CMS to run title experiments ourselves. A great case to show that it took a developer a few more days to build this on top of that but that it also provided us with the ability to work without her on this so we could leverage her (or others in our organization) for other (likely more high leverage)  projects.

Velocity == Magic

Most developers are motivated by velocity, they want to see progress just as much as you want to. An easy way on how to achieve this is to break SEO workup. When you come in, I totally get it, you want XML sitemaps across different templates/sections, etc.. It’s not realistic to all put that work together as you just created a project yourself that will takes weeks to months to build. And there is no way that builds a business case and quickly your reputation is gone. Break up the work, start with an easy XML sitemap for a subsection. Show the value and expand from there: show them what an XML Sitemap Index looks like, have them add it to the robots.txt. Next: have them GZIP the files if they’re getting out of hand, after that you have a robust setup and meanwhile a bunch of releases that helped you get out the work.

Celebrating Success

Are you celebrating success with your development team? Have you shown them the results of their ideas and the work that they’ve done? How consistently do you share these learnings? For example, when? When we launched projects at Postmates, we went back to them regularly to show them how a canonical change or an additional category index page had helped our efforts to push another page. In the best cases, we could tell them how much additional revenue this had resulted in because we initiated either new pages or the updates were significant enough to measure impact.


TL;DR: I believe that developers and SEOs should work better together, and that mostly needs to come from the SEO side, so there is less hate on developers. Learn more about your development team’s procedure, work, codebase, programming language, and what they value in their work. That way, you can educate them on important SEO topics over time and celebrate success when you have successfully delivered a project or improvement.

What other ways do you see to improve the relationships between developers and SEO? Leave a tip in the comments.


Specializing as an SEO – Growing as an SEO (4/4)

The previous three blog posts in this series talked about writing better job descriptions for SEO roles and levels and seniority for SEOs but also but growing on a more personal level. In the last blog post in this series, I want to talk about either being a generalist or specializing as an SEO. But also, what does it mean to be a T-shaped marketer and especially in SEO. How does it change your role in the team, what skills are expected, etc.

All in all, these are questions that I ask/I’ve asked when interviewing SEO people. If I’m looking for a very technical SEO I’m likely not going to achieve great results if I hire somebody who’s really passionate about content or link building. It doesn’t mean that they’re not great SEOs, it just means that they’re not the right ones that I’m looking for at that moment.

T-Shape

The topic of T-shaped marketers isn’t a new one, Rand and Joanna have blogged about it before. And for years it’s been a topic that comes back on a regular basis in blogs/podcasts and at conferences.

A (simplified) example of a T-shaped SEO role with a deeper focus on the technical side.

 

Local, Public Relations, Content, Analytics, Technical, International, Social, Link building, UX, Statistics, Paid Acquisition, Partnerships, Business Development, Psychology, Research. Just some of the areas that I could think of in a 2-minute brainstorm when I think about other skills that SEOs could/should/must have. They’re not all as important and it depends on what you’re looking for in an SEO role (from both sides: employee and employer).

Skills

So what other skills do you need as an SEO, I have some ideas on what would be useful. It’s far from the truth but some of the skills that I see always come in handy. These are some of the examples that are on my list and that I usually look for:

  • Web Analytics: Can you use a tool like Google Analytics or something else for analytics to prove the value of your work. Can you show me what works and what doesn’t? I wouldn’t make the GAIQ the centerpiece of your resume, but it at least shows that you have worked with it before.
  • Other Channels: Social Media/PPC: It doesn’t really matter what channel they have worked on, but it shows that they at least.
  • Understanding the funnel: Most SEOs focus on the top of the funnel > acquisition. But do they also realize what happens with their value and users once they convert (or don’t).
  • Research: Can you find the right information about an industry, do you know a ton of shortcuts on finding information (through Google) or not. All of this will help you find more opportunities (for authority building), data, insights, etc.

What isn’t required in my opinion? It’s an unpopular opinion,  likely: writing. I’m not a great writer, but everybody can write (although I realize not at an excellent level). But I’ve also noticed over the years that writers are easy to hire whenever you need them. So this makes it something that I don’t value that important.

Specializations / Disciplines / Areas

So what kind of specializations or disciplines are there in SEO, I think there is about 5. There might be more depending on how specialistic you want to go within a certain area (I know technical SEOs who can go super deep in a particular area). For me, this doesn’t take into account which business model or industry you specialize in (legal, gambling, real estate, marketplaces), but more about what area in SEO you’re good at. For example, I’m likely best at Technical SEO but for sure know how to align Content and Authority Building good enough to really benefit from the work that is done in that area. I know a fair share about internationalization, but likely not good enough to call myself a true expert. Local SEO, well I would advise you to talk to somebody else if you want to optimize 100 local business (at scale, Yes I can ;)).

Authority Building

You can have the best site in the world but at some point, you need to start building authority for it to really get attention and awareness. Do you know how to work with brand marketing and/or teams focused on Public Relations (PR), do you hire a link builder? What links do you really need? What mentions would be great to have? What kind of press would you really like to get: NYT, The Next Web, a local business magazine covering your CEO? And if you’re small how do you stand out, what’s your messaging, how do you scale outreach, etc.

Content

What content works well for a search engine, what keywords do you focus on? How do you select & hire freelancers? What kind of visuals do you need for blog posts, how do you structure your blog posts. What kind of keywords are (not) important?

Over the years I’ve worked a lot with editors, writers, etc. and almost always had somebody on my team to deal with content. It pays off to have somebody work on creating excellent content that also easily gets picked up by publications. It makes authority building easier but also ensures that the content itself can be found, indexed and higher ranks in search engines.

Technical SEO

Do you know how a sitemap works, do you know how to deal with structured data, can you talk me what a log file looks like and the information that you can retrieve from it? These are just some of the questions that SEOs in this area ask themselves on a daily basis. You’re basically working alongside engineers/developers to build out features that can make the site more accessible and easier to understand for a search engine.

Local SEO

“Local SEO”, sometimes this also means dealing with hundreds of stores for an enterprise. Or are you able to deal with a local business that just needs more promotion and they’ve been wanting to grow their SEO traffic as it can help them be the next big store in their city? How do you get more awareness for a local store, how do you add 400 listings to GMB, etc. All kinds of questions that come into play when you’re thinking about local SEO.

Internationalization

How do you deal with different languages, do you use different TLDs/subfolders, etc. These are the questions that these people keep themselves busy with on a daily basis. How do you optimize for the scale of different languages and regions and how do you optimize for that. What language/region needs its own content and how do I link pages together across languages (with hreflang and/or sitemaps). Do I hire local SEO teams/agencies?

Usually, internationalization is a topic that comes up at bigger companies, barely ever do small startups go overseas and have to deal with multiple languages from the start. But these are some of the important topics to think about when you want to specialize yourself in this area.

Growing as an SEO – This series

In this series I’ve also blogged about: