SEO Site Monitoring

Audits Hero Image
When fruit goes bad it rots, telling you something is wrong and needs throwing. Your website is being worked on by lots of different people, monitoring allows you to similarly know when something is broken.
Let's chat

What is Site Monitoring and what is involved?

Modern, enterprise websites often release changes to their code several times a day, preferring to tolerate mistakes but not taking a long time to find them. My site monitoring platform allows for sites with millions of URLs to be monitored daily and includes a helpful Looker report for stakeholders and all the requisite data to replicate issues.
Crawling

Crawling

Crawling at an hourly, daily or weekly cadence is setup and run so to find and aggregate issues on the site.
Reporting

Reporting

Data from the crawl is aggregated and feed into a Looker report, which showcases both current versus previous crawl numbers and trends for 30 days previous. Additionally all of the data required to fix the indicated issues is included and available in a onedrive folder which updates after every crawl.
Data Gathering

Data Gathering

A secondary but useful side effect of crawling your website so often is I can also pull and aggregate data from across your site. Want a count of pages which have ‘Staff Writer’ instead of an actual author, this can be collected and provided daily.

How the Process Works

Step 1

Setup

Working with your engineering teams I create a route for daily crawling, which typically involves whitelisting my IP, UserAgent and header token. Once whitelisted I setup the crawls and begin a series of tests to confirm we won’t be blocked.
Step 2

Reporting and training

I connect the aggregated data collected from the crawls to a looker template which outlines current vs previous crawl deltas and provides a 30 day lookback of crawl stats. In recognition of the fact people ignore reporting they do not understand, I am happy to provide training.
Step 3

Alerting

Once reporting is being populated we begin looking at alerting. I will work with you to setup thresholds and/or categories that must always result in alerts, pushing them either by email or slack messages.
Have a question about the process?
Let's Chat

FAQ's

There's not much mention of SEO in the above, how comes?
The conventional wisdom here is that if users enjoy being on your website, meander around and complete whatever it is that they came to do – then search engines will favour you in their results. Whether that’s due to bounce rate, pogo-sticking, scroll depth or a long list of other considerations is largely academic, what’s important is that the more people who access and accomplish their task using your website the less you’ll have to worry about in terms of SEO.
How do I know if my site is slow?

Firstly it’s worth clarifying that your website can be slow to load and slow in terms of interactivity (I.e. when you click on a button nothing happens).

Whilst tools exist that I can share which list how well you score on a scale of 1 – 100 or A – F, a more practical and simpler test is simply to disconnect from wifi on your phone and click around your website. If the pages are taking a long time to finish loading, things randomly shift around and when you click on something it takes a noticeably long time for anything to happen – then it’s probably slow enough to warrant having a conversation.

The aim here is to keep people in the flow of what they’re doing, a button not doing anything when I click on it creates rage clicks and is usually a good reason for someone to go back to a search engine and try someone else.

What's the difference between lab testing and RUM testing?

Lab testing, sometimes called synthetic testing, refers to using tools which try to emulate the experiences of users and in doing so measures and scores what they expect will be your scores. Lab testing is integral for measuring whilst working on improvements, it allows engineers to see what the expected difference in timing will be once released to the website.

RUM, or Real User Measurement, testing is when you proactively measure users experiences when using your website. The benefit here is that it is 1st party data, which means it’s more dependable and carries no erroneous assumptions.

A fun bit of trivia is that Google actually provides you with RUM data by discreetly measuring users experiences in the chrome browser. This means that even if you have not been actively measuring your users interactions you can gather some data from Chrome.

How to Contact Me
Drop me an email
hello@tomgregan.co.uk
Book a Meeting
tomgregan.co.uk/calendar
Visit my Contact Page
tomgregan.co.uk/contact