Updated June 4, 2019: Added a YouTube video which will guide you through the setup process in RStudio and how to run the script yourself.
Averages lie & average click-through rates aren’t very helpful! Here I said it. What I do believe in, is that you can calculate click-through rates (CTR) for your own site in a great way though. With data from Google Search Console. Especially while using R, so let’s dive in on how this works and what you can do with it!
Why ‘Averages’ Lie?
Look at the graph below, a great CTR for position 1. But for example, this research shows that the average CTR for position 1 is: 24% (AWR data, Feb 2019). Which one is correct? Neither of them. As it really depends on the industry, what features show up in the search results that might decrease CTR (think rich snippets like local packs, news results). All of this is making it really hard to make a good analysis of what you could expect if you rank higher for a bunch of keywords in your industry. So while I was working at Postmates on ranking certain category pages better we decided to calculate our own CTR and were intrigued by how far CTRs were off from research (the research isn’t wrong! It’s just generalized across industries). Eventually, with the data in hand, we were able to make better estimates/forecasting of how much traffic we could expect when rankings would increase in that segment. In the rest of this post, I’ll go more in-depth on the specific practice on how we calculated this.
Using Google Search Console Data
Visual of Google Search Console Report (this is not RVshare data)
You’ve seen this report in Google Search Console, providing you with a detailed view of the performance of your keywords and the average position for your keywords. In this graph, we see something positive, a CTR & position that go up slightly over time. But what if you would want to know the average CTR for a certain segment of keywords per position. That’s way harder to do in the interface. Because of that, I used the R script from Mark Edmondson that he wrote about here almost three years ago.
It will help you extract the data from Google Search Console in a raw way so you can use it to digest it and create your own visualizations (like the one we’ll talk about next).
Visualizing CTR Curves in R
CTR Curve visualized CTR per Position (note: this is randomized data for an unknown site)
So let’s dive right into how you can do this yourself, I’ll provide you with the full R script and you will need to download RStudio yourself in step 1.
Run these commands to install the right packages for RStudio:
install.packages(“ggplot2”) if necessary
Line #21 – Change this to the property name from Google Search Console
Line #25 – Not neccesary: If you want the CTR curve for positions over 20, change the number.
Line #40 – Recommended: Exclude the word(s) that are part of your brand name. So you get the right CTR curve for non-branded keywords only
Line #41 – Not necessary: This script is taking a ‘sample’ of 50.000 keywords to calculate your CTR curve of. You can increase this limit to more if needed, if you have less than 50.000 keywords it’s not an issue
Run the script! The output should be a visual as shown earlier in this post
Want to take a deep breath and let me help you go over this again? I’ve made a quick screen share video of what to do in RStudio and how to use the R script.
Hopefully, now, you’ve had a better chance to understand what the actual CTR is for your own site and you can use this to visualize CTR curves for specific parts of your site or pages that have a similar META description. Over time you could use this, for example, for measuring the impact on CTR.
Credits where credits are due! There are many use cases for using CTR data by visualizing it with R, and I’m grateful that a while ago Mark Edmondson opened my eyes about this + credits to Tim Wilson’s documentation on using R and improving visualizations.
Want to read this article in Spanish? Read it here.
“I hate my developers,” “We deployed the wrong thing(s),” “Somebody put up a Disallow in my robots.txt that wasn’t meant to be there,” “The whole site is deindexed.” These are just some of the quotes that I’ve heard over the years on the relation between developers/engineers. Where usually, the developers are the ones that receive all the hate. It should stop. Usually, I try to reflect on what I would do in a situation like that. This often means that the SEO should have a little more reflection and figure out what they could have done to avoid a situation like that. In most cases, the answers that I get back are that they’re having a hard time working with their development team and don’t have a very good understanding of each other’s work.
How to work together & Creating a better understanding with developers
Speak their language, Learn their language, Use their tools/platforms, Ask for their input. Embrace their new technologies and ideas. Then, have a ton of fun!
Github/JIRA/Confluence: How is work assigned to your development team, what tools do you use for this and do you leverage the tools that they like using or do you require them to be stuck in a tool that they don’t like. I’m quite confident that they rather use the tools that are already part of their regular development cycle.
Retrospectives, Lunch & Learn: How often have you joined a retrospective of a development team or have been part of their lunch & learn. You don’t always have to know all the details about what they’ve worked on. But it for sure helps build up your own knowledge by learning more about their process/terminology and codebase/technology.
Pull Requests: Have you ever gone through the code that was about to be merged into your codebase to see specifically what kind of changes your developer made. It will help you double check if this is what you meant and gives you a good understanding on where certain parts that matter for SEO live.
Specs & Technical Requirements: Do you write your developers requirements, are you involved, have you ever read them to better understand what an integration takes? If you’ve always been used to rapid development on platform X doesn’t mean that platform Y is just as fast to have new implementations build for.
Write requirements yourself or write a testing plan: An SEO feature didn’t turn out what you expected it to be? Well, did you actually exlain how it was supposed to work and wrote all the edge cases for it? Did you provide the specific for the texts that you wanted to show up. I bet you didn’t. So work with your product managers on writing requirements and help guide your QA engineers on how this could be tested. There should always be a way for you to test business logic in releases besides the functional/technical aspect of it.
Embrace new technologies: Often developers have a better insight into what new technologies can help the platform that you work with. When I had no idea how a certain database + API could help us and started to get frustrated our engineering lead was happy to tell me that it would make rollouts of future landing pages way easier and avoided them having to use an old API that sometimes took days to deploy. Their efforts to implement this paid off big time already in the quarter after they had launched that (both in speed of the site and development time).
Give your developers 1 fact a day about something relevant to them and SEO. Of course, you all care about certain topics already: user experience, speed, etc. But can you really expect them to know about canonical tags or the inner workings of HREF lang integrations? I’m convinced they shouldn’t know anything about that until they’ve met you. In the end, it’s not part of their education/learning curve, and it likely should never be there. On the other hand, if you guide your development team and tell them about why certain improvements need to happen, you’ll likely, over time, build up a valuable resource with your developers. Hopefully, in a way that they can actively contribute to your actual SEO strategy. That’s why I want to share two stories of how SEOs and Developers could also work together:
500 Server Request
A few years ago, when I worked internally with the TwitterCounter (RIP 🙁 ) team at The Next Web (one of their sister companies), the development team was planning to temporarily shut down the service for a few hours (2-6 hours depending on the progress). It was not great as a user experience, but it was inevitable as the servers were about to hit their limits and needed to be upgraded. Specifically for SEO, this meant that tens of millions of pages wouldn’t be accessible (including for crawlers) for the duration of the maintenance. Not great, but obviously, for SEO, you can take the right precautions. As our development had been sitting together with me for months (literally, they sat right next to me), I had the chance to slowly educate them on many SEO topics (including dealing with maintenance shutdowns). So when the time came to shut down the servers, the amazing question that came up was: “So, I imagine that we’ll put up a temporary 50X status code, so we will tell search engines that we are doing server maintenance).” I don’t think there was ever a better moment in time where developers stepped up with the right mindset and understanding about SEO.
At Postmates, we wanted to run experiments on title tags on different kinds of templates. As pages were crawled very frequently and there was enough traffic, it meant that we could run a few dozen experiments potentially yearly just on that. Which meant that it would be a lot of work every few weeks to create/update experiments. Instead, she approached us with the question if it was Ok to build a tiny CMS to run title experiments ourselves. A great case to show that it took a developer a few more days to build this on top of that but that it also provided us with the ability to work without her on this so we could leverage her (or others in our organization) for other (likely more high leverage) projects.
Velocity == Magic
Most developers are motivated by velocity, they want to see progress just as much as you want to. An easy way on how to achieve this is to break SEO workup. When you come in, I totally get it, you want XML sitemaps across different templates/sections, etc.. It’s not realistic to all put that work together as you just created a project yourself that will takes weeks to months to build. And there is no way that builds a business case and quickly your reputation is gone. Break up the work, start with an easy XML sitemap for a subsection. Show the value and expand from there: show them what an XML Sitemap Index looks like, have them add it to the robots.txt. Next: have them GZIP the files if they’re getting out of hand, after that you have a robust setup and meanwhile a bunch of releases that helped you get out the work.
Are you celebrating success with your development team? Have you shown them the results of their ideas and the work that they’ve done? How consistently do you share these learnings? For example, when? When we launched projects at Postmates, we went back to them regularly to show them how a canonical change or an additional category index page had helped our efforts to push another page. In the best cases, we could tell them how much additional revenue this had resulted in because we initiated either new pages or the updates were significant enough to measure impact.
TL;DR: I believe that developers and SEOs should work better together, and that mostly needs to come from the SEO side, so there is less hate on developers. Learn more about your development team’s procedure, work, codebase, programming language, and what they value in their work. That way, you can educate them on important SEO topics over time and celebrate success when you have successfully delivered a project or improvement.
What other ways do you see to improve the relationships between developers and SEO? Leave a tip in the comments.
The previous three blog posts in this series talked about writing better job descriptions for SEO roles and levels and seniority for SEOs but also but growing on a more personal level. In the last blog post in this series, I want to talk about either being a generalist or specializing as an SEO. But also, what does it mean to be a T-shaped marketer and especially in SEO. How does it change your role in the team, what skills are expected, etc.
All in all, these are questions that I ask/I’ve asked when interviewing SEO people. If I’m looking for a very technical SEO I’m likely not going to achieve great results if I hire somebody who’s really passionate about content or link building. It doesn’t mean that they’re not great SEOs, it just means that they’re not the right ones that I’m looking for at that moment.
The topic of T-shaped marketers isn’t a new one, Rand and Joanna have blogged about it before. And for years it’s been a topic that comes back on a regular basis in blogs/podcasts and at conferences.
A (simplified) example of a T-shaped SEO role with a deeper focus on the technical side.
Local, Public Relations, Content, Analytics, Technical, International, Social, Link building, UX, Statistics, Paid Acquisition, Partnerships, Business Development, Psychology, Research. Just some of the areas that I could think of in a 2-minute brainstorm when I think about other skills that SEOs could/should/must have. They’re not all as important and it depends on what you’re looking for in an SEO role (from both sides: employee and employer).
So what other skills do you need as an SEO, I have some ideas on what would be useful. It’s far from the truth but some of the skills that I see always come in handy. These are some of the examples that are on my list and that I usually look for:
Web Analytics: Can you use a tool like Google Analytics or something else for analytics to prove the value of your work. Can you show me what works and what doesn’t? I wouldn’t make the GAIQ the centerpiece of your resume, but it at least shows that you have worked with it before.
Other Channels: Social Media/PPC: It doesn’t really matter what channel they have worked on, but it shows that they at least.
Understanding the funnel: Most SEOs focus on the top of the funnel > acquisition. But do they also realize what happens with their value and users once they convert (or don’t).
Research: Can you find the right information about an industry, do you know a ton of shortcuts on finding information (through Google) or not. All of this will help you find more opportunities (for authority building), data, insights, etc.
What isn’t required in my opinion? It’s an unpopular opinion, likely: writing. I’m not a great writer, but everybody can write (although I realize not at an excellent level). But I’ve also noticed over the years that writers are easy to hire whenever you need them. So this makes it something that I don’t value that important.
Specializations / Disciplines / Areas
So what kind of specializations or disciplines are there in SEO, I think there is about 5. There might be more depending on how specialistic you want to go within a certain area (I know technical SEOs who can go super deep in a particular area). For me, this doesn’t take into account which business model or industry you specialize in (legal, gambling, real estate, marketplaces), but more about what area in SEO you’re good at. For example, I’m likely best at Technical SEO but for sure know how to align Content and Authority Building good enough to really benefit from the work that is done in that area. I know a fair share about internationalization, but likely not good enough to call myself a true expert. Local SEO, well I would advise you to talk to somebody else if you want to optimize 100 local business (at scale, Yes I can ;)).
You can have the best site in the world but at some point, you need to start building authority for it to really get attention and awareness. Do you know how to work with brand marketing and/or teams focused on Public Relations (PR), do you hire a link builder? What links do you really need? What mentions would be great to have? What kind of press would you really like to get: NYT, The Next Web, a local business magazine covering your CEO? And if you’re small how do you stand out, what’s your messaging, how do you scale outreach, etc.
What content works well for a search engine, what keywords do you focus on? How do you select & hire freelancers? What kind of visuals do you need for blog posts, how do you structure your blog posts. What kind of keywords are (not) important?
Over the years I’ve worked a lot with editors, writers, etc. and almost always had somebody on my team to deal with content. It pays off to have somebody work on creating excellent content that also easily gets picked up by publications. It makes authority building easier but also ensures that the content itself can be found, indexed and higher ranks in search engines.
Do you know how a sitemap works, do you know how to deal with structured data, can you talk me what a log file looks like and the information that you can retrieve from it? These are just some of the questions that SEOs in this area ask themselves on a daily basis. You’re basically working alongside engineers/developers to build out features that can make the site more accessible and easier to understand for a search engine.
“Local SEO”, sometimes this also means dealing with hundreds of stores for an enterprise. Or are you able to deal with a local business that just needs more promotion and they’ve been wanting to grow their SEO traffic as it can help them be the next big store in their city? How do you get more awareness for a local store, how do you add 400 listings to GMB, etc. All kinds of questions that come into play when you’re thinking about local SEO.
How do you deal with different languages, do you use different TLDs/subfolders, etc. These are the questions that these people keep themselves busy with on a daily basis. How do you optimize for the scale of different languages and regions and how do you optimize for that. What language/region needs its own content and how do I link pages together across languages (with hreflang and/or sitemaps). Do I hire local SEO teams/agencies?
Usually, internationalization is a topic that comes up at bigger companies, barely ever do small startups go overseas and have to deal with multiple languages from the start. But these are some of the important topics to think about when you want to specialize yourself in this area.
“It’s not about resources, it’s about resourcefulness” – I’ll leave it up to you to Google who this quote is from. It’s not that hard, it’s a quote that hit me a few years ago. I felt I was stalling in a role and needed to move forward. There is so much that you can do yourself to advance your career and learn. That’s why I wanted to focus in this blog post on the things that I’ve used, and in most cases are still using to learn more about SEO.
I’ve tried to list as many different learning options as possible, in the end, everybody learns in a different way. It’s one of my favorite questions to ask in interviews what the best way is for people to learn. If they’re aware of what it is, it for sure adds a bonus point to the candidate.
Conferences & Meetups
While I was still living in Amsterdam, at some point I felt like that I didn’t miss any meetup related to online marketing in a while. I went to a ton of them, and they were great. There is so much to learn at an event: soft skills: networking, talking, socializing, small talk. All skills that are just as important (I’d argue even more important) than the on the job skills (crawling, technical, content, etc.).
So let’s give you a selection of the blogs that I’ve been following over the years that helped me build my SEO knowledge. These are some of the ones on the list (read: it’s far from complete, I’ll keep updating the list with specific SEO blogs).
Follow people, follow experts. You can learn so much from the approach from that other people are taking. Just to get a different insight or to learn a new tool. SEO is a rich field where everybody has their own tactics and I feel strongly that every week I pick up on some new tactics in SEO that I’ve never thought of (mostly around research or authority building). We can’t know everything but it’s a good tactic.
Talk to people, they can tell & learn you more. I’ve asked people at companies that I admire for years if I could have coffee with them. If you’re reading this and one of them, I thank you again! Don’t ask for trade secrets, but if you do your research it will strengthen the conversation. So let’s expand a little bit on that …
After I’ve just mentioned that having coffee is great to learn more, but make sure you come prepared. As we’re talking about SEO, run an audit. Ask them why they’ve done certain things this way. I’ve learned a great deal just analyzing and researching the best sites trying to figure out what their SEO strategies are after which I got that confirmed by their teams while having coffee with them. I’m not telling this to brag about it, but to give you an insight into what you can do to get more out of the meeting too. It will strengthen the conversation. You’re using somebody’s time and she or he will likely appreciate it if you know what they’re talking about in more detail.
Creating Playbooks / Keeping track
Recently I shared for the first time the idea behind building a playbook in a presentation that I gave at a conference. It’s something that I’ve actually been doing for a few years now. For about a five years now I’ve been saving job descriptions, not of jobs that I wanted to be hired for (at least mostly not). But Marketing roles that I thought I was going to hire one day or grow into. They’re a great archive (I have close to 200!) these days for whenever I need to fill a specific role.
But the same methodology applies to most parts of somebodies work, most content teams have a style guide, when you’re working in CRO you have templates to document and hypothesize your experiments. But I felt that most of these ‘standards’ were missing across functions within Marketing and specifically (in-house) SEO teams.
What other tools are out there for others to use as well? What learning options have I missed and should I add to the post? Leave a comment here or on Twitter: @MartijnSch and I’ll make sure to keep this post updated, just like the others.
Since I’ve joined RVshare, I needed to think a lot about these questions (again): what people do I need to hire? What experience level do they need to be at? This made me reflect back on hiring for my teams at Postmates and The Next Web and my views on different levels in certain functions. As my background is, mostly, in SEO I started to think about what levels I would form within a big SEO team and what their differences are. This is my first attempt at this framework and part of the series about growing as an SEO, the previous blog post talked about how to write a proper job description for an SEO role.
In this blog post I want to talk more about the different seniority levels, what do they mean? What kind of role are you looking for: specialist or generalist? What level are they at? And what kind of levels do you need for your own team and what might be the different responsibilities for the different roles and how do they change (over time).
Generalist versus Specialist
Are you a smaller or bigger company and how big is your SEO team? What are you really looking for on your team? What is your own background? Do you know enough about SEO yourself to successfully guide & lead an SEO person?
You’ve probably heard of the idea behind a T-shaped role. Do you expect somebody to know a lot in one specific area (specialist) or do you want that person also to know a lot about the other areas that have a relation with SEO. This visual is just the tip of the iceberg o other skills that you can expect from an SEO.
I started myself as a generalist in my career, like most people. Back then, Springest had a need for traffic acquisition and I worked on their SEO, Affiliate Marketing and later on their Paid Acquisition (mainly Adwords). Next, to that, I worked a lot with Google Analytics to learn more about the keywords that were driving performance (this was before ‘not provided’ got introduced).
Mostly in smaller organizations, I see marketing leaders or founders hire for this type of person. In most companies, you early on need somebody to test the waters for all the channels and need to be able to manage more than just one thing. SEO isn’t usually the fastest growing channel for a company as it takes a while. That has a huge impact on why there aren’t a ton of people with a dedicated focus on SEO in most startups.
Later on, when I left Springest and joined The Next Web I was much more of a specialist. I focused solely on SEO, although later on, I added analytics and CRO (all before I lead their Marketing team). This meant that I needed to be proficient in all the areas that were part of SEO: technical SEO, content (we had tons of editors to work with) and figure out how to build out our authority at a big scale. All this type of work was very much only focused on SEO and didn’t have much impact on other channels.
Most SEO roles these days that I see are similar, they’re usually part of a digital marketing team and/or are the only person on the team with a dedicated focus on SEO to help that channel. They have often contact with a product manager. Marketing manager and the needed people focused on content, design, and development. But they’re the ones driving the specific roadmap for SEO.
Not everybody is a generalist or a specialist, neither is everybody a manager or wants to focus on just one discipline. But for most people, it makes sense to belong to a specific ladder.
Individual Contributor Roles in SEO
With most companies, you’ll start at the bottom of the totem pole when you start your career in SEO. Most people will start right around the title of SEO Associate or SEO Specialist at the beginning of their career and work their way up the ladder. After a while, most of them will need to make the decision to either continue to be an Individual Contributor (IC) or move into the role of manager where they start managing (or better: leading) people.
SEO Internship: We all need to build up experience and what better way to do that than with an internship/apprenticeship. This role will usually get the support of the SEO team while you learn how SEO works. Most people that I’ve seen enter this role have a passion for online marketing and are studying something in a related field (or totally not, sometimes even better). You’re never long in this role (at most 5 months), you either tend to like or not so you can move up on the SEO ladder.
SEO Associate: In some cases, this role comes in between an internship and having the title SEO Specialist. This usually happens within enterprises where you’re dealing with bigger SEO teams. There is not a ton of difference between the role of an SEO Specialist and the SEO Associate. But usually, SEO specialists tend to have a little bit more of experience (1-2 years as a maximum). They’re starter positions and sometimes the titles are intertwined.
(Senior) SEO Specialist: For most people, this is where they’ll start, the SEO Specialist. I’ve been and done there myself when I joined TNW this was my job title. I was the only person on the Marketing & Sales department dealing with SEO and was answering to the (at the time) CMO. This meant that I was working on all the aspects of SEO and was working with a development. When you’re getting more experience and depending on the size of the organization and HR structure it could be that you get the title Senior SEO Specialist after a while to claim the more experience that you have.
(Senior) SEO Manager: You’re growing, you’re basically now sort of managing the SEO process and you’re not answering usually to somebody who’s leading the SEO team anymore. You’re the one in charge of SEO but you’re not leading anybody specifically on the SEO team itself.
(Senior) Head of SEO: The highest level that I usually see on SEO teams as an Individual Contributor. It makes it that you’re not managing other people but work deeply on SEO and have the fundamental knowledge and resources around you to manage the whole process from start to finish. There isn’t a ton of companies that I know that are able to support this role as in most companies they’ll require you to become a manager.
Management Roles in SEO
Some people chose to go the route of the manager, they want to lead a team and be responsible for multiple people. This is where management & leadership skills are becoming more important as they’re not working 100% of their time hands-on on SEO anymore.
SEO Team Lead: This role likely makes sense by reading the job title. You’re part of a small SEO team and you’re the lead. I like to apply this seniority level on a team when it’s small and the ‘manager’ isn’t very experienced yet as a leader. It’s usually the case when they have moved over from the level of SEO Specialist and you decide to hire another SEO Specialist. Somebody has to lead the wolf pack and decide on a strategy. If the 1st SEO person has the ambition to step over to a more managerial role over time, this is a good start.
(Senior) SEO Manager: You’re managing the SEO team and you work with some people outside your own team to get things done. Usually, the case when you’re part of a bigger Growth or Marketing team and you’re the one deciding on what work is important to help the bigger team achieve its goals.
Director of SEO:You can strategically think about SEO and you’re part of a bigger organization. That’s what my last title was at Postmates. Our overall Growth organization of which we were part of was around 50 people and we had multiple Directors of different functions (Growth Product, Growth Engineering, Paid Acquisition, etc.) report into our VP of Growth. You lead a team that can also work cross-functionally with other teams within and outside the same group.
VP of SEO: Likely the highest seniority title that I’ve seen in SEO for in-house was that of VP of SEO. There are a few companies, mainly in the United States, that use that title. They’re enterprise companies (in all cases that I’ve seen at least). Where they differ from a Director of SEO role is that they’re focused on the bigger picture. They lead a team that is usually 1.5-2 times as big as the level lower and are responsible for just more than SEO. A position like this is usually also heavily involved in functions like Public Relations, Brand Marketing, and Content Creation depending on where that might live in the rest of the organization.
‘Global … Head of SEO’ – Global companies & reflecting this on titles
Through Twitter (@micahfk) reached out, with a good point about the title: “Global Head of SEO”. I’ve seen this level a few times myself as well and I agree with his point that this title can in most cases have more weight than a title on a Director level. In companies at scale, there will be a global team managing all of the enterprises’ SEO strategy where on the local level (usually countries or regions) teams will work on the local execution (and often strategy). They’ll have similar titles, but usually, the people who will head up a Global team will rank higher on the organizational chart.
This framework is simplified and not perfect. It’s a first shot at assessing what roles there are in an SEO function from an in-house point of view. As I’ve never worked with/for an agency I’m sure their views on this would be different, I won’t blog about that. It’s up for grabs for somebody who has extensively worked on that side of the fence.
Work in Progress: This blog post is a work in progress. I hope to extend it over the upcoming weeks with more information on the responsibilities and areas that the different roles work on.