Category: Web Analytics

Measuring Content Performance: Content Engagement Metrics

Measuring Content Performance: Content Engagement Metrics

What is the effectiveness of our content, how well does our content work? Who is writing the best content? What should we be writing about next? These were the top X questions I received on a weekly basis while working at The Next Web. And I’m probably not the only one, I hear a lot of companies that are asking the same questions. We were a publisher, with a big editorial team (~10 FTE editors) and publishing around 30-40 blog posts on a daily basis and had so for the past ~10 years. Which meant that there were over 65.000 articles in our archive that we could learn from. One of the hardest projects in the end.

“Tons of page views doesn’t always mean this post is doing well or is of the highest quality/the best journalism”. If one of our post hit Reddit, HackerNews or was a top story in Google News we for sure celebrated this but the impact on the business wasn’t always that big as we were hoping. Traffic from most of these sources has very low engagement and basically only brought in money through display advertising (with incredibly low CPMs). Overall, this meant that we wanted to find a better metric to score the performance of editors and find out what kind of quality really worked well for the business, not just for engagement & visitor metrics. So we got started looking into this…

Usage Metrics, Useful & Useless

What I still see a lot of companies do, and in all honestly you can’t always blame them for that. Is look at the basic usage metrics of content. How much time has a user spend reading this article, what is the bounce rate, etcetera. Which isn’t great as most of the time you don’t know the context. Is a good blog post one that has been read for 2 minutes, or 1 minute. Obviously the length of the blog post and if it has videos/images has an impact on this. All data that is usually easily overseen when analyzing content performance. Even with taking this into account it’s making it hard to come up with recommendations while working for a publisher. Are you really always going to recommend to use at least 2 ¾ images in a blog post and require a video that is exactly 23 seconds long so we know for sure people will watch it? Probably not.

Business results

Most of all, these metrics don’t align with your business goals. How often does the CEO/COO/CMO ask you what the bounce rate is of your articles so they can calculate the return on investment. Never right? That’s why over time we need better metrics that align with the business and that provide useful insight into what content really helps and what content is just good for vanity metrics like page views. You’ll be surprised in the end how many companies are still driven by these.

Capturing Goals in Google Analytics is important to measure impact of performance.

Creating a Content Engagement Metric

It’s going to be about the monetary value that content represents for your business but we won’t be setting it up that way. In the end you need to be able to evaluate the way you calculate this and over time this will likely change with your business. That’s why it’s important that we can make the decisions in an open and honest way so we can make changes later when we think they’re needed.

Setting up Goals

If you want to get started and do this on your own it’s going to be important to know what kind of events are important to your business and site. In our case we looked at, for example: newsletter subscriptions, article shares on social media (they bring in more visits, so useful for more revenue), ticket sales, ecommerce sales, etc. But also in general we wanted to know what the value for us was with a pageview. I’m not going to walk you through the setup of all these goals in this post, but if you’re unfamiliar with setting up goals in Google Analytics, read & watch this.

Custom & Calculated Metrics

So now you have determined what the goals are that matter to you. What is going to be important in this step is making sure that you assign ‘points/value’ to all of the goals. You don’t have to edit the goals for that (probably even better not to, to not interfere with the page value metric). So for example, think of it like this:

  • Pageview: 1 point
  • Article share: 5 points
  • Newsletter subscription: 10 points
  • Ecommerce sale: 150 points (or varying on the product value)
  • Ticket sale: 250 points

Now it’s time to set up a calculated metric to take this into account. Why a calculated metric? By using this we’re able over time to adjust the formula to our needs but also individually assign values to the goals if we wanted to (in that case we’d use goal completions as well).

When you get started with this, know what numbers you are using for the goals that you have setup. You can find that in the Goals interface within Google Analytics.

  1. Name: Content Engagement Score.
  2. Formatting Type: Integer (you can change this to Float or Currency if that’s more applicable)
  3. Formula: {{Goal1 Completions}} * 1 + {{Goal2 Completions}} * 5 + {{Goal3 Completions}} * 10…

In the end that should look something like this:

Reporting & Dashboarding for Content Quality

Now you have successfully created a calculated metric that we can use for reporting and informing other teams in the company about the content performance.

Trial & Error: Formulas should always be up for discussion

This formula isn’t going to be good the first time, you’ll have to tweak it and assign values that make more sense in your case. That’s why I also didn’t share the actual values of TNW (they were different from the example) as it matters to your specific business and goals.

To talk a little bit more about how things worked out for us, we tweaked it 3 to 4 times over a period of two months to get closer to our actual goal of measuring performance. In the end we could have editors who still would write for a ton of pageviews so get hundreds of thousands of points through that but editors who did great on business results weren’t rewarded value through that. So we upped some of the points assigned to our business goals to align them better.

Custom Reporting

Just setting up the goals probably already gets you quite far. It will allow you to create a page + goals report in which you can see how many goals have been hit for certain pages. Very useful if you want to measure the performance.

Dashboarding: Google DataStudio/Chartio versus (Google) Sheets

You need to share the reporting around this in an easy way, depending on your teams it will depend what works best for them. Sharing spreadsheets or making it easy for people to immediately take a look at a dashboard in a tool like Chartio and/or Google DataStudio.

Performance by day/editor/topic

Now you have all this data you can hopefully combine this with the data that you also have gathered through your custom dimensions. Previously I have blogged on The Next Web about all the ideas that I have around custom dimensions. Give it a go, it will be surprising to see how much easier it will become to report & analyze the performance across topics, time of day, team members when you can really align this with your business goals.


This is just an example of how we were measuring the performance of content for the business. There are many other ways to do this and you will have to customize the formula for your own business. What other metrics and ideas would you take into account to analyze and report this? Leave a comment or reach out on Twitter: @MartijnSch

Exporting Amplitude Data to Google BigQuery

Exporting Amplitude Data to Google BigQuery

I’ve written about using Amplitude before on this blog (in Dutch), but what if you want to combine the huge amount of data that you have in Amplitude with your other big data. The Enterprise version gives you the ability to export your data to a Redshift cluster. But a lot of companies these days are also on Google Cloud Platform and want to use Google BigQuery, which is similar in its setup to Redshift.

Amplitude

The Export API from Amplitude lets you download all your events data (regardless of your account plan) for free. There are some limits to the data that they can export but most startups/companies that use them are likely to stay under that on a daily/hourly basis. Which means that you can export the data. So basically everything that you need to do is setup a cronjob that either every hour or day can retrieve the data. It will parse the data and prepare new files that will be temporarily stored in Google Cloud Storage (this will ensure that the data is easier/faster available for upload into Google BigQuery). The next step is loading the data from GCS to GBQ.

Google BigQuery

Over the last years I wanted to do more with Google BigQuery and this project was perfect for it. As BigQuery has many connectors with multiple Google products (Analytics, Data Studio) and other vendors like Tableau it should give companies the ability to analyze their data and connect to other sources.

Schemas

Within Google BigQuery we’re going to save the data in two tables:

  • Events: As everything is an event in Amplitude that’s also one of the tables that you’ll need to use in Google BigQuery, that’s why every event will end up as being it’s own row in Google BigQuery.
  • Properties: Every event can have properties in a few different ways: events, users, groups, group properties and an actual data property. We are connecting them to the data from the events table.

FAQ

  • Do I need to be a paying customer for Amplitude? No, you don’t. The free plans, which I started to use this on will have the support for the Export API as well.
  • What is the cost of the integration? The costs that are associated with this setup are related to Google Cloud Platform. You’ll have to pay for storage in Google Cloud Storage and the storage of Google BigQuery. For the setup that I’m running we’re saving millions of rows monthly and the costs are in total less than 10 dollar.
  • What do I need to do to get this up and running? Check out the README in the repository on Github, it will give you a getting started checklist to ensure that you can run this script.

 

Feedback? / Contribute?

I haven’t been the first person to work on an integration with BigQuery. I’ve seen other solutions for Java and Python, but they all work a bit different. If you have any feedback on the setup, leave an Issue on Github, submit a Pull Request with your proposed changes. In the end, I can code, but I don’t consider myself to be an engineer 😉

Using Amplitude for Product & Web Analytics

I’ve previously published this blog post in Dutch on Webanalisten.nl.

What if you are looking for a product for web analytics but have a lot of events, a complicated product and sending more and more data over time. Sometimes it wouldn’t just work to go with Google Analytics (360), Adobe Analytics and maybe integrating your custom build solution or Snowplow might be too complicated for your organization. In this post I’d like to show you another tool that might be interesting to you: Amplitude. I’ve been working with it for the last year and it provides some great flexibility over other products in the industry.

What is Amplitude?

“Analytics for modern product teams”

Or in normal language, you can track any event and connect that to a user. All the events that you send can have properties, just like the user can have properties. You can filter by all these data points and move your data around to turn it into multiple types of charts: funnels, event hits, revenue, etc. In this article we’ll be running through how you can be using Amplitude and what it’s good for. Let’s go ahead and dive in!

Why would you be using Amplitude?

You want to measure what is happening within your product and what users are doing. Keeping in mind that all this data can help you improve their workflows and measure the impact certain changes have on their behaviour. In that aspect, Amplitude is directly competing with mostly tools outside of web analytics, like: Google Analytics for Firebase, Snowplow, KISSmetrics, Mixpanel, etc. In the next section we’ll explain why, as a lot of features are interpreted differently from regular web analytics but can still help you a lot in your daily work:

What’s the difference?

 

  • Instant Reporting/DIY: While most analytics tools provide you with a lot of pre configured dashboards. Amplitude will let you do this all on your own, which can be a time consuming task but in my opinion it also lets you think a bit more about the way you set-up your analytics infrastructure.
  • No default metrics: Bounce rate doesn’t exist as any event can be triggered to influence it (plus, would that be your most useful metric anyway?)
  • Funnels: Anything can be a  funnel, in my opinion that makes it very powerful as it doesn’t require you to create any predefined goals and also will make sure you can create funnels retroactively (recognize this pain point in Google Analytics?). If you added events a few weeks ago and now is the time to start creating a funnel of it, you’re able to. Want to change the funnel and add more/less events. You can.

 

 

  • User/Session: Sessions (on Web) don’t really exist in Amplitude. While in my opinion this is a metric that has a very loosely defined definition anyway it can come in handy from time to time to measure retention. It will provide this data on mobile where sessions are way easier to be defined (app open/close).
  • Channel/Traffic Source: If you’re looking for an easy way to track where your visitors are coming from with detailed reports that would be associated with costs. That’s just not what Amplitude is for. While it can easily save the UTM data that you’re sending along it won’t offer your great reporting around it. That’s why there focus is mostly on product analytics.
  • Merging Events/Linking Events: At the beginning of this section we talked about the need for setting up all the dashboards yourself. As you won’t have a very defined plan with your tracking for what’s to come over the next few years it can be hard to follow a certain naming convention from scratch. Usually turning your analytics data into an unstructured chaos over time. Within Amplitude you’re able to merge certain event names and link them together. So you can easily change your old event names to something new and still connect the data together. One of the features I really miss sometimes in other tools when I’m trying to redefine naming conventions and clean them up.

Why data governance is even more important

The role of data governance is becoming more important by using tools like this in combination with having the need for good documentation. If you come into an organization that is already sending hundreds of different events it can be really hard to get started with making a more deep analysis as you’re not always familiar with the

    • Naming conventions: You want to make sure that you’re using the right names for the events and that you’re making sure that their logical in order to send the data. It would be good to give this article, on creating measurement plans, that I wrote for online-behavior.com a read. We’ll talk later about how Amplitude can still help you if you would like to make changes to the events you sent.

 

  • Segments/Cohorts: As most of the data for users can be saved in event or user properties this will also mean that you need to make sure that the data in there doesn’t change too often as it might affect how you’ve set up your segments and cohorts.

 

  • Also funnels and many reports can be impacted by the way you save data.

Overview of features

  • Dashboarding/Charts: As we talked about the flexibility that Amplitude can provide you with, this mostly shows in the way you’re working with charts and adding them to dashboards. You can create dozens of different charts and add them to a dashboard. The dashboards will then, for example, give you the ability to change the date range. If you don’t like that you can still make all the changes from the actual chart.
  • A/B Testing – Significance Calculator: Are you running an A/B test on your site and sending the data to Amplitude. Within certain charts you can segment out the results and immediately calculate if they’re significant for what you’re analyzing. Saves time trying to find a significance calculator.

      • Custom Metrics: Just as many other web analytics tools, Amplitude will give the ability to create custom formulas within a chart to calculate other metrics.

     

  • Retroactive reporting: You have added tracking months ago but only today you’ve figured out that an event should be measure as a goal? You can set up a goal funnel really easily with old data and have all the old data being available to you.
  • Realtime: The fact that all of the events that you send to Amplitude are processed in real time makes it very powerful. Basically within minutes of launching a new feature you can start analyzing the data to see what’s going on or if it’s broken. Gone are the times were you need to wait for hours to have all the data that you are collecting be fully available.
  • Unlimited event/user properties & ‘dimensions’: Every event can have properties that can be related to the event. In addition to that a user can have properties that can be used too. So if I want to mark certain users with an action I can easily send that along with an event to update the records for this.
  • CLTV: Measuring the lifetime value of users will obviously require you to start identifying users (relatively easy to set up). But this will enable you to look into how you’re users are performing over time and if you have high retention for what that means for their customer lifetime value. This is an example report that would provide me with the performance of a segment of users over the last 12 weeks and what they’re worth to the business.
Chart for CLTV

What’s missing?

Google integrations? Obviously some things are missing, while the Cohort feature’ abilities are very powerful and Amplitude can provide you with some cool integrations with other software it still can’t make the connection with the audience data from Google. Which is obviously always going to be a big upside of the Google Analytics Suite.

Transactions/Purchase: The way Amplitude is tracking a conversion is a bit weird. You send all the products that you purchase as different revenue events. There is no concept of a purchase, which seems strange. Also it’s really hard to identify what the Xth purchase was, these are events that you need to setup yourself.

UTM/Traffic Source Reporting: It does exist but it isn’t great and definitely not as powerful as you’re used to in your regular web analytics tools. Does it get the job done for product analytics, Yes it does I’d say. If you’re looking to do anything more advanced with the data you should be building additional capabilities on your own end.

Use Cases

  • Funnels: Every event can be part of a funnel and that makes it very flexible and useful if you want to compare user behaviour. For example connecting certain user actions before a purchase funnel can be the case too.
  • Customer Lifetime Value/Retention:
  • Cohorts: Where you would have segments & audiences in Google Analytics you have the ability to also create cohorts of users to measure the impact of certain properties/events on their behaviour over time. For example this is a cohort that we used often at Postmates where we would look at what users that have come in with the sign up referrer that includes google, yahoo, bing (an organic search user). We would use this cohorts either to export them from other marketing purposes (email/push campaigns) or to analyze their behaviour against other cohorts.
    • How do organic search users in the last month behave different if they have used x feature?
    • How do users who have touched feature x find the other features?
Segmenting users with its Cohort feature.

Conclusion

Overall I’m pretty satisfied with Amplitude and how it can help you with its flexibility in adding/creating events and figuring out later what kind of dashboarding/charts you’ll create on top of this. But it’s likely (for now) not going to replace most of the data that you’re used to in web analytics as that would require a lot of additional setup and tracking. You can use it very effectively within organizations to track certain aspects and user behaviour. All in all a great addition to most analytics teams. All in all I would advise most companies to use these tools together as they can be very useful in providing more insights into what the user is doing specifically.

If you’ve worked with Amplitude and want to share more about your experiences, leave a comment! Currently I’m working on exporting Amplitude data to Google BigQuery for ‘big data’ analysis, in a future post I hope to share with you on how you can set that up yourself.

Introducing: the Google Tag Manager & Google Analytics for AMP – WordPress Plugin

Last updated on March 15th, 2017 at 01:00 am

Today it’s time to make it easier for sites running AMP on WordPress to track & measure their web site traffic. Over the past weeks I’ve been working on a WordPress Plugin which supports adding Google Tag Manager and Google Analytics to your AMP pages. As AMP itself is quite a hard new platform to get completely right, I thought it was time to make it easier for developers, marketers and analysts to do more with the platform.

Over the last year AMP had as massive increase in support and their WordPress plugin is rising to the charts of the most downloaded plugins on WordPress but overall the support for web analytics & tag management is lacking. The documentation is available, but as the abilities within WordPress are still limited I thought it would be pretty straight forward to come up with a plugin to enable users to start using GTM & GA on AMP pages.

So that’s what this plugin will help you with, the ability to add Google Analytics or Google Tag Manager to your pages, start tracking more advanced features (outbound click tracking) and provide 10 (for now) custom variables that you can use as custom dimensions.

You’re curious how to start? Check this out:

Moving away from onclick events, data attributes it is

Last updated on May 23rd, 2018 at 07:04 pm

Onclick attributes are amazing to easily sent some data to the dataLayer. It will make sure that on every click the data is being sent, let’s for example look at this:

<a href=”http://example.com” onclick=”dataLayer.push({‘eventCategory’:’Click’, ‘eventAction’:’This Link’, ’eventLabel’:’Anchor Text’});”>Link</a>

Looks valid right? It is and it will work for you. But the problem is that with external links the dataLayer push won’t hit the browser + Google Tag Manager before the browser moves on to the external link (not taken into account links that open in a new tab/window). Which means that you’ll lose the onclick event in Google Tag Manager and won’t be able to record it.

But there is a great alternative, which we switched to for The Next Web when we found out this was happening. Data attributes can be very useful instead. If you’d use them like this:

<a href=”http://example.com” data-eventCategory:’Click’, data-eventAction:’This Link’, data-eventLabel:’Anchor Text’”>Link</a>

Why this works? The link attributes on click will always be sent automatically to Google Tag Manager. So making sure that the data attributes are part of the link and not in a separate dataLayer.push will improve it.

How about React/Angular? With these platforms there is obviously never an actual refresh of the page happening and most likely you can still send a dataLayer push when you’re updating the state of the application. That’s why in most cases you won’t have to go with the technique described in this blog post. Lucky you!

Google Tag Manager

Next step is to create a variable in Google Tag Manager for all three attributes:

Make sure to set the default value to undefined. In this case Google Tag Manager won’t take any actions in the next step if there is no data, which will be defining the Trigger. Repeat this two more times for the Event Action and Event Label variables that we need to use in our Tag.

This trigger will check all incoming links, with a data-event-category attribute that doesn’t equal undefined and that has any values. In the next step for your Tag you can use all three variables that you’ve defined before.

You’re all set! You can now use data-attributes for your links to make sure that you won’t lose any data when the browser isn’t ready to receive dataLayer pushes and you just made your tracking a bit more efficient!

The discovery & benefits of using ga:dataSource in Google Analytics

Last updated on February 26th, 2017 at 11:25 am

As there is always something new to learn in this world I recently came across (by a tip from Simon Vreeman) the dimension: dataSource in Google Analytics. It’s one that isn’t very well known, as it’s not available through any of the default reports, but can help bigger companies with integrations in multiple places in a great way. In our case it was incredibly useful to make reporting on AMP performance more easy.

DataSource is meant to provide an additional field to send where you’re sending the data from. But it’s different from a hostname or page path because it’s more about the source. By default it will always send the value: ‘web’ for sites, ‘app’ for mobile apps and apparently ‘amp’ for AMP tracking. Something that, in our case, was very valuable as traffic on AMP pages can be hard to find as a bulk. Filtering on pages that have ?amp or /amp/ as a pageview ain’t great if you want to see the traffic sources for 1 specific article. With LinkedIn also supporting AMP that won’t make it easier. Discovering dataSource makes it easier for us to kill a custom dimension to set the dataSource and immediately being able to filter down on this and then on referral traffic.
How you can leverage it? Send data from a specific source, for example your CRM, data warehouse or your email campaigns. We’re sending for The Next Web a hit to Google Analytics via the measurement protocol to track email opens. We’ll probably move this over as we would see it as a different dataSource then it being a ‘web’ hit.

How to configure the dataSource

Google Tag Manager: You can easily set it up in Google Tag Manager by creating an additional field that you can set, apparently it’s so unknown that it isn’t covered in the default field that are suggested. But if you add ‘dataSource’ as a field and enter the value (in this case a hardcoded value, but I would recommend going with a GTM variable).

Google Analytics: Setting this up in Google Analytics works in about the same way, you can add it as an additional field in your hit call to GA to set it by adding it like this.

@GoogleAnalytics: I know you’re reading this, please help us and update the documentation to also mention ‘amp’ being a default data source for tracking on AMP pages.

How to report

Creating a custom report

As the dimension isn’t available through a standard report you’ll have to create a custom report to do so. With our data I created a report like this that can be used to report on where traffic is coming from specifically (I’ll talk about the hit referral path in another blog post).

Look at this table ? with information about the dataSource dimension