FALCON AI, our in-house suite of AI for SEO on WordPress products, reached a milestone in January: we analysed 20,000 keywords in one month for the first time.
We produced 40 pieces of content in January. This requires 40 keywords, so for each keyword we select, we’re analysing about 500 possibles.
The numbers are huge: why would you want to consider 500 different keywords, just to get 1 blog post?
If you get your keyword selection wrong, your SEO Content is going to fail. That’s why we put so much work into finding great keywords. Let’s dig into why and how we do it.
SEO Content as a production process
All production flows have a basic characteristic: the material becomes more valuable as it moves through the process… A common rule we should always try to heed is to detect and fix any problem in a production process at the lowest-value stage possible.
Andy Grove, High Output Management
In the legendary book High Output Management, former Intel President Andy Grove describes a diner serving breakfast as an analogy for the production challenges he faced. Just as Intel wants to avoid production issues by catching bad parts early, we want to do the same with SEO Content.
Keyword research is the earliest part of the content production stage where you can make massive changes, with no consequences. If you choose a bad keyword, all of the work on a post will go to waste.
The flip side, obviously, is that if you pick a great keyword, you are set up for success.
In the production process, the keyword selection stage for SEO Content is thus the single point at which you can have a disproportionate impact on the success or failure of your post.
This is why we put so much effort into finding the right keywords: it’s worth evaluating 500 possible keywords to find 1 good one, because the cost of choosing bad keywords is too high.
Legacy keyword research delivers hit-and-miss results
Manual keyword research often starts with a tool like Ahrefs: you enter a sensible “seed” keyword like “WordPress forms” and then start looking at the suggestions:
This generates a list of related keywords, difficulty and volume information, and we can see parent topics too.
If you’re feeling ambitious, you might apply some filters to avoid brand-match searches of competitors. You might also look up a “content gap” to see which keywords competitors are ranking for that you’re missing.
The output of this is typically a keyword tree with a couple of branches. You might decide that “WordPress forms” is too competitive, but “accessible forms” or “WordPress form templates” are keywords you can compete for:
You’d thus be happy with your selection, and you’re off to write an article about how to connect a WordPress form to a CRM.
This is OK, but you’re able to analyse at most 5-10 keyword branches, you’re limited to the obvious related keywords your software finds, and you get no insight into what you need to write in order to rank.
Whilst you might look at keyword difficulties and competition, you don’t get any insight into whether writing a post about “WordPress form CRM” is actually a good idea or not. The assumption will be that if the volume is ok, the difficulty isn’t too high, and you have some sort of WordPress form solution that connects to a CRM, you’re good to go.
Those assumptions on what makes a keyword “good” or “bad” are unqualified and untested. If your goal is to make sales for your product from SEO Content, those criteria tell you very little about whether the post will achieve that goal or not. Some keywords and posts will work if you do enough of them, but it’s your results are hit-and-miss.
This is the source of so much frustration with SEO, and it’s why WordPress businesses find SEO so hard to do despite it obviously being the best marketing channel for bringing in new customers. Hit-and-miss keyword research delivers hit-and-miss SEO results.
Frustration with hit-and-miss results is compounded by the slow feedback loops. It can take 6 months for a new post to rank well even if you’ve worked on publishing great content regularly, so you don’t get any feedback on whether your keyword was good or bad for a very long time after publication. It can take substantially longer if you’re just starting up and your website is still new, or you don’t have the Domain Authority where it should be because of a lack of content around your area of expertise. Even 6 months is enough time to abandon SEO Content because it “doesn’t work”.
6 months to see results is also enough time that it is very difficult to get any insights into why posts worked or didn’t work. Not getting or applying that feedback means you waste massive amounts of resources and time on posts which were never going to work.
Hit-and-miss is a terrible time for everyone. I’ll be honest — this is a problem we’ve faced in the past. Before we solved this problem at Ellipsis, hit-and-miss SEO Content lost us clients. We operate at a scale that lets us solve problems individual WordPress businesses can’t solve, though. Solving this problem is what led us down the path to FALCON AI.
Exploring deep across the long-tail of keywords
With FALCON AI, we can generate seed keywords from the AI. FALCON generates the seed keywords by using AI to analyse the content of a product page and pull out seed keywords (great for finding long-tail keywords that fit with a product’s features), and it automatically looks up what your competitors are ranking for. This gives us a seed keyword list.
We can then lookup long-tail, related keywords, and keyword ideas on those seed keywords. This generates up to 500 possible keywords from each seed. We automatically filter out duplicates and low-value keywords, so this typically evaluates 5,000 keywords in one go.
Instead of manually looking at a couple of keywords, we can go deep down every possible keyword rabbit hole. This is a simple representation: instead of just looking at a couple of angles, we can look at every angle. The actual FALCON outputs, are, of course, much more detailed than this:
A human can’t possibly evaluate this volume of keywords, so we’re using Machine Learning to rapidly evaluate if a keyword fits with our client’s product or not. We do this with a custom Machine Learning model that’s trained to pick out keywords relevant to individual WordPress products.
Using the AI, we can take the 10,000 keywords we’re evaluating and output the ones with good product fit for manual review. That manual review gives us a shortlist, which we’ll bring into the final review stage: predicting if the keyword will be successful, or not.
Will this specific keyword and title combination be successful?
I wrote about this final stage when we first introduced FALCON AI last year. This was the original problem FALCON AI was built to solve: predicting if a specific keyword and title combination would result in a successful post, or not.
We’re uniquely placed to solve this problem, as we’ve got a massive stack of posts we can feed the AI for training data. This is complemented by the actual top ranking results for thousands of WordPress and WooCommerce keywords to give us a Machine Learning model that can confidently predict if a post is going to work.
This is where the keyword research goes far beyond guesswork: instead of looking at a keyword’s data and saying “that looks ok”, we can test if specifically the proposed keyword and title combination will succeed or not. This filters out the bad keywords at the earliest possible stage.
Titles are really important. They’re an outsized ranking factor not just because Google uses them to determine where your content should initially rank, but also because an engaging title will get more clicks — and thus a higher ranking. It’s a virtuous cycle if you get the title right.
FALCON AI being able to predict if a specific keyword and title combination is going to succeed or not is cool, but I don’t fancy writing 10 different title options for 500 keywords a month. If each title takes 30 seconds to write, that’s an entire week writing titles!
The solution is ingenious: FALCON AI uses a custom fine-tuned GPT-3 model to generate title options automatically. The AI is trained using all our best practices fro titles and the current top 10 results for the keyword.
The result of this is we can generate high-performance titles that perfectly reflect the search intent, as they’re based on what’s actually ranking. We can generate multiple titles in one go, and then run those all through the AI to output the title with the highest chance of success.
This is the process we’re using to massively increase the chance of success. So what are the results?
FALCON AI is 28x the size of 6 months ago, and it halves the time to rank in the top 10
The original training data for the machine learning part of FALCON was done in May 2021. Since then, we’ve retrained it five times, and the dataset is now 28x larger. A much larger dataset gives us much higher accuracy overall and better capabilities for edge cases.
We’re ultimately measured on Content Growth by results, and for SEO Content these start with rankings. Preliminary results with the latest version of FALCON AI have seen the time required for an article to rank in the top 10 decrease by half.
Of course, it isn’t perfect. Getting SEO Content right has a lot of variables to it, including your site’s authority on the topic, promotion and backlinks, etc. These mean that even the best-written content sometimes just won’t rank high enough. But FALCON AI means we can eliminate as many variables as possible and significantly increase probability of ranking in the top spots for our clients’ content. Particularly for clients who put in the time and effort to create regular quality content in the right way, FALCON AI adds an unbeatable layer of reliability, predictability and speed to rankings.
Our hypothesis is we’re doing a much better job than competitors of providing relevant, insightful content to the searcher. Google is recognising this and thus rewarding our content.
We’ll continue making improvements at a rapid pace. FALCON AI is available exclusively for our Content Growth clients. Keyword selection is an important part of FALCON AI, but it’s only one of many parts. If you’d like a chat about how we can help with your SEO Content, get in touch with us.
