SEO Automation in 2021

SEOSearch engine optimization (SEO) is a branch of web marketing which is aimed to improve the visibility of a website or a web page in a search engine’s organic (meaning un-paid) search results (which are collected in a SERP). What kind of tactics does SEO use? SEO activities are divided into two main categories: On-page SEO:…automation is the process of using software to optimize a website’s performance programmatically. This article focuses on what you can do with the help of artificial intelligence to improve the SEO of your website.
Let’s first remove the elephant in the room: SEO is not a solved problem (yet), and while we, as toolmakers, struggle to alleviate the work of web editors on one side while facilitating the job of search engines on the other, SEO automation is still a continually evolving field, and yes, a consistent amount of tasks can be fully automated, but no, the entire SEO workflow is still way too complicated to be entirely automated. There is more to this: Google is a giant AI, and adding AI to our workflow can help us interact at a deeper level with Google’s giant brain. We see this a lot with structured data; the more we publish structured information about our content, the more Google can improve its results and connect with our audience.

This blog post is also available as Web Story ? “SEO Automation in 2021 Web Story“

An introduction to automatic SEO
Will Artificial Intelligence Solve SEO?
Automating Structured Data Markup
Finding new untapped content ideas with the help of AI
Automating Content Creation
Creating SEO-Driven Article Outlines
Crafting good page titles for SEO
Generating meta descriptions that work
Creating FAQ content on scale
How Does SEO Automation Work?

An introduction to automatic SEO

When it comes to search engine optimization, we are typically overwhelmed by the amount of manual work that we need to do to ensure that our website ranks well in search engines. So, let’s have a closer look at the workflow to see where SEO automation can be a good fit..

Technical SEO: Analysis of the website’s technical factors that impact its rankings, focusing on website speed, UX (Web Vitals), mobile response, and structured data.
- Automation: Here, automation kicks in well already with the various SEO suites like MOZ, SEMRUSH, and WooRank, website crawling software like ScreamingFrog, Sitebulb, etc., and a growing community of SEO professionals (myself included) using Python and JavaScript that are continually sharing their insights and code. If you are on the geeky side and use Python, my favorite library is advertools by @eliasdabbas ? .
On-Page SEO: Title Tag, Meta Descriptions, and Headings.
- Automation: Here is where AI/deep learning brings value. We can train language models specifically designed for any content optimization task (i.e., creating meta descriptions or, as shown here by @hamletBatista, title tag optimization). We can also use natural language processing (like we do with WordLift) to improve our pages’ structured data markup ?.
Off-page SEO: Here, the typical task would be creating and improving backlinks.
- Automation: Ahrefs backlink checker is probably among the top solutions available for this task. Alternatively, you can write your Python or Javascript script to help you claim old links using the Wayback machine (here is the Python Package that you want to use).
SEO strategy: Traffic pattern analysis, A/B testing, and future predictions.
- Automation: here also we can use machine learning for time series forecasting. A good starting point is this blog post by @JR Oaks. We can use machine learning models to predict future trends and highlight the topics for which a website is most likely to succeed. Here we would typically see a good fit with Facebook’s library Prophet or Google’s Causal Impact analysis.

Will Artificial Intelligence Solve SEO?

AI effectively can help us across the entire SEO optimization workflow. Some areas are, though, based on my personal experience, more rewarding than others. Still, again – there is no one size fix all and, depending on the characteristics of your website, the success recipe might be different. Here is what I see most rewarding across various verticals.

Automating Structured Data Markup

Structured data is one of these areas in SEO where automation realistically delivers a scalable and measurable impact on your website’s traffic. Google is also focusing more and more on structured data to drive new features on its result pages. Thanks to this, it is getting simpler to drive additional organic traffic and calculate the investment return.

Here is how we can calculate the ROI of structured data

Here is a concrete example of a website where, by improving the quality of structured data markup (on scale, meaning by updating thousands of blog posts), we could trigger Google’s **Top stories,** to create a newflow of traffic for a news publisher.

Finding new untapped content ideas with the help of AI

There are 3.5 billion searches done every day on Google, and finding the right opportunity is a daunting task that can be alleviated with natural language processing and automation. You can read Hamlet Batista’s blog post on how to classify search intents using Deep Learning or try out Streamsuggest by @DataChaz to get an idea.

Here at WordLift, we have developed our tool for intent discovery that helps our clients gather ideas using Google’s suggestions. The tool ranks queries by combining search volume, keyword competitiveness and if you are already using WordLift, your knowledge graph. This comes in handy as it helps you understand if you are already covering that specific topic with your existing content or not. Having existing content on a given topic might help you create a more engaging experience for your readers. Here is a preview of our new ideas generator – write me to learn more

We give early access to our upcoming tools and features to a selected number of clients. Do you want to join our VIP Program?

Become a VIP

Automating Content Creation

Here is where I expect to see the broadest adoption of AI by marketers and content writers worldwide. With a rapidly growing community of enthusiasts, it is evident that AI will be a vital part of content generation. New tools are coming up to make life easier for content writers, and here are a few examples to help you understand how AI can improve your publishing workflow.

Creating SEO-Driven Article Outlines

We can train autoregressive language models such as GPT-3 that use deep learning to produce human-like text. Creating a full article is possible, but the results might not be what you would expect. Here is an excellent overview by Ben Dickson that demystifies AI in the context of content writing and helps us understand its limitations. https://platform.twitter.com/embed/index.html?dnt=true&embedId=twitter-widget-0&frame=false&hideCard=false&hideThread=false&id=1303340597449822208&lang=en&origin=https%3A%2F%2Fwordlift.io%2Fblog%2Fen%2Fseo-automation%2F&theme=light&widgetsVersion=ed20a2b%3A1601588405575&width=550px

There is still so much that we can do to help writing be more playful and cost-effective. One of the areas where we’re currently experimenting is content outlining. Writing useful outlines helps us structure our thoughts, dictates our articles’ flow, and is crucial in SEO (a good structure will help readers and search engines understand what you are trying to say). Here is an example of what can be done in this area.

I provide a topic such as “SEO automation” and I get the following outline proposals:

What is automation in SEO?
How it is used?
How it is different from other commonly used SEO techniques?

You still have to write the best content piece on the Internet to rank, but using a similar approach can help you structure ideas faster.

Crafting good page titles for SEO

Creating a great title for SEO boils down to:

helping you rank for a query (or search intent);
entice the user to click through to your page from the search results

It’s a magical skill that marketers acquire with time and experience. And yes, this is the right task for SEO automation as we can infuse the machine with learning samples by looking at the best titles on our website. Here is one more example that we’re working on: a trained model that can come up with great title suggestions given a few topics. Let’s try it out. Here I am adding two topics: SEO automation and AI (quite obviously).

The result is valuable, and most importantly, the model is stochastic, so if we try the same combination of topics multiple times each time, the model generates a new title.

Generating meta descriptions that work

Also, we can unleash deep learning and craft the right snippet for our pages or at least provide the editor with a first-draft to start with for meta description. Here is an example of an abstractive summary for this blog post.

Creating FAQ content on scale

The creation of FAQ content can be partially automated by analyzing popular questions from Google and Bing and providing a first draft response using deep learning techniques. Here is the answer that I can generate for “Is SEO important in 2021?”

DISCLAIMER: these tools are not yet part of WordLift but are being tested with a selected number of clients. Do you want to join our VIP Program to automate your SEO? Drop us an email.

How Does SEO Automation Work?

Here is how you can proceed when approaching SEO automation. It is always about finding the right data, identifying the strategy, and running A/B tests to prove your hypothesis before going live on thousands of web pages.

It is also essential to distinguish between:

Deterministic output – where I know what to expect and
Stochastic output – where the machine might generate a different variation every time, and we will need to keep a final human validation step.

I believe that the future of SEO automation and the contribution of machine/deep learning to digital marketing is now. SEOs have been automating their tasks for a while now, but SEO automation tools using AI are just starting to take off and significantly improve traffic and revenues.

Are you Interested in trying out WordLift Content Intelligence solutions to scale up your content production? Book a meeting with one of our experts or drop us a line.

The image we used in this blog post is a series of fantastical objects generated by OpenAI’s DALL·E new model by combining two unrelated ideas (clock and mango).