Keyword Research
Guide

Why keyword research still matters

Some argue that with AI-generated content everywhere, keywords matter less. The opposite is true. Understanding what your audience searches for - the exact language they use, the intent behind each query, the commercial value of each click - is more important than ever as competition for ranking positions intensifies.

The difference is how you use keywords: not as targets to stuff into content, but as signals of intent to shape an entire content strategy.

The three metrics that actually matter

Every keyword research tool shows you dozens of metrics. Most are noise. Focus on these three:

1. Search volume

How many people search for this term per month. Higher is not always better - a 100 monthly search keyword with perfect buyer intent can be worth more than a 10,000-search keyword where 95% of searchers want free content.

2. Keyword difficulty (KD)

How hard it is to rank in the top 10. This is a function of how authoritative the pages currently ranking are. For a new site, target KD 0-30. For a mid-authority site, 20-50 is realistic within 3-6 months of good content and links.

3. Business value

This is the metric the tools don't give you - you have to assess it manually. Ask: if I rank #1 for this keyword, what kind of person arrives? Are they likely to buy? Become a lead? This is more important than search volume for commercial sites.

A practical keyword research process

Step 1: Seed keywords from your audience

Start with 5-10 core topics your business covers. For a project management SaaS: "project management," "task tracking," "team collaboration," "Gantt chart," etc. These are your seeds - you'll expand from here.

Step 2: Expand with a keyword tool

Run each seed through Ahrefs, SEMrush, or Google Keyword Planner. For each seed, collect:

• The seed keyword itself (check volume + KD)
• Questions containing the keyword (excellent for featured snippets)
• Long-tail variations (lower volume, higher intent)
• "Best [X]", "How to [X]", "[X] for [audience]" variations

Step 3: Analyse competitor keywords

Find your top 3 organic competitors (not your business competitors - the sites ranking for your target keywords). In Ahrefs, use "Content Gap" to find keywords they rank for that you don't. These are your highest-priority opportunities - validated demand, proven rankability.

Step 4: Prioritise by opportunity score

Score each keyword: Business Value (1-5) x Search Volume bucket (1-5) / Keyword Difficulty (1-5). This gives you a rough priority order. Combine high-business-value keywords with low-KD for quick wins early.

Step 5: Map to intent, then to pages

Group keywords by intent: informational (guides, definitions, how-tos), commercial (comparisons, reviews, best-of lists), and transactional (pricing, demos, sign-up). Each intent cluster maps to a different type of page - and a different place in your conversion funnel.

A warning about your pre-September 2025 data

In September 2025 Google removed the num=100 search parameter, which had let tools fetch 100 results in a single request. Rank-tracking bots doing exactly that had been inflating Search Console impressions for years. When the parameter vanished, those phantom impressions vanished with it.

The effect was large and it was not a ranking change. Across 319 properties analysed by Search Engine Land, roughly 88 percent lost impressions overnight, and average position appeared to improve - because the impressions being removed were the deep, never-seen ones dragging the average down. Clicks were largely unaffected, which is the tell: nothing about real user behaviour changed.

Three practical consequences for keyword research:

• Any year-over-year impression comparison spanning September 2025 is invalid. Rebaseline from that date rather than explaining a drop that never happened to real traffic.
• Lead with clicks, not impressions or average position. Clicks are the metric that survived the change intact.
• Treat third-party volume and difficulty scores as directional. Deep SERP data got materially more expensive to collect, and several tools cut their default tracking depth in response. Google published guidance in June 2026 making the same point: third-party tools have no access to its ranking data.

None of this makes the tools useless. It means the numbers are estimates that got noisier, and your own Search Console data is the only record of what actually happened to your site.

Measuring keywords in AI search

In June 2026 Google added Search Generative AI performance reports to Search Console - the first official visibility data for AI Overviews and AI Mode. It is genuinely useful and it is also more limited than the announcement suggests. Three caveats to know before you build any reporting on it:

• Data starts 18 May 2026, with no backfill. There is no history before that date and there never will be.
• Impressions only - no clicks. Google has promised click data without giving a date.
• No query dimension. You get impressions, pages, countries, devices and dates. The one thing a keyword researcher actually wants - which queries triggered you - is not there.

It is also rolling out to a subset of properties, so it may not appear in your account yet. Note that AI Overview and AI Mode traffic is also folded into the overall Web search type in the main Performance report, so these reports are additive rather than a separate bucket - do not add the two together.

What this means for how you pick keywords

Click-through expectations for informational terms have genuinely fallen. Ahrefs measured AI Overviews reducing top-result CTR by 34.5 percent in April 2025, revised to 58 percent by February 2026. Pew's behavioural panel of 68,879 searches found users clicked a traditional result 8 percent of the time when an AI Overview was present, against 15 percent when it was not. Attribute those numbers to their source and date if you use them - methodologies vary wildly and the flat "zero-click" percentages circulating online are not comparable.

The practical shift is toward targeting the topic or entity cluster rather than the individual head term, because AI Mode fans a question out into multiple related queries behind the scenes. But do not read that as licence to spin up a page per sub-query - Google explicitly calls that scaled content abuse. Commercial and transactional terms, where someone still needs to reach a site to buy something, have held up considerably better than informational ones.

From keywords to topical authority

Modern SEO rewards depth, not breadth. Instead of writing one-off articles for individual keywords, build topical clusters:

• Pillar page: the comprehensive, definitive resource on a broad topic (e.g., "The Complete Guide to Project Management")
• Cluster pages: 8-12 focused articles on subtopics, all linking to and from the pillar (e.g., "Agile vs Waterfall," "How to Create a Gantt Chart," "Project Management Templates")

This approach builds topical authority faster and more durably than scattering articles across many disconnected topics.