Every SEO has a task they do by hand that they know a computer should be doing. Checking where 400 old URLs redirect after a migration. Exporting Search Console data 1,000 rows at a time because the interface will not give you more. Sorting 3,000 keywords into groups in a spreadsheet, row by row. These jobs are boring, error-prone and repeated every month.
Python is the most common way marketers automate them, not because it is fashionable, but because it reads almost like English, it has well-maintained libraries for web requests and data tables, and an AI assistant can explain any line you do not understand. You do not need to become a developer. You need to be able to run three or four short scripts and adapt them.
This guide is a gentle start for non-developers. It covers a headache-free setup, then three complete scripts you can run today: a bulk status code and redirect checker, a Search Console API export that gets past the interface limit, and a keyword grouper that uses only built-in Python. It ends with how to use AI assistants to write and debug scripts without creating security problems.
Why marketers reach for Python instead of another spreadsheet
Spreadsheets are excellent until the data gets wide, long or repetitive. Python earns its place when a task has one of these traits:
- Volume: thousands of URLs or keywords, where formulas slow down or browser extensions time out.
- Network calls: fetching pages or calling APIs, which spreadsheets do clumsily.
- Repetition: the same export and clean-up every week or month.
- Joining sources: combining a crawl, a Search Console export and an analytics export by URL.
| Task | Manual time (rough) | Scripted time | Best tool |
|---|---|---|---|
| Check status and redirect target of 500 URLs | Hours with a browser | A few minutes | Python or a crawler |
| Export page + query data beyond 1,000 rows | Not possible in the interface | Minutes via API | Python, Sheets add-on or BigQuery |
| Group 3,000 keywords by similarity | A day of sorting | Seconds | Python |
| Monthly report table for one client | An hour | Minutes | Sheets or a dashboard |
| One-off check of a single page | Seconds | Not worth scripting | Browser |
The time estimates are illustrative, but the pattern holds: scripting pays off for anything large or repeated. If your work is mostly light formulas and sharing, our guide to Google Sheets for marketers may be the better first step.
Setting up Python without the usual headaches
Most beginner frustration comes from setup, not code. Follow this sequence once and you will avoid the classic “module not found” and “python is not recognised” errors.
- Install Python 3 from python.org. On Windows, tick the box to add Python to PATH on the first installer screen. On a Mac, use the python.org installer rather than relying on any system copy.
- Create a project folder, for example
seo-scripts, and open a terminal in it (on Windows, typecmdin the folder’s address bar; on a Mac, right-click the folder in Finder and choose New Terminal at Folder). - Create a virtual environment:
python -m venv .venv(usepython3on a Mac). This keeps each project’s packages separate. - Activate it:
.venv\Scripts\activateon Windows orsource .venv/bin/activateon a Mac. Your prompt now shows(.venv). - Install packages:
pip install requests pandas google-api-python-client google-auth. - Use a simple editor, such as VS Code with the Python extension. Save scripts as
.pyfiles in the project folder and run them withpython script_name.py.
If you prefer not to install anything, Google Colab runs Python notebooks in the browser. It is fine for the keyword and API scripts, but keep service-account keys out of shared notebooks.
Script one: bulk status code and redirect checker
This is the script to run before and after every migration, and whenever someone hands you a list of URLs and asks “are these working?”. Put one URL per line in urls.txt, then run:
import csv
import requests
HEADERS = {"User-Agent": "Mozilla/5.0 (compatible; MySEOChecker/1.0)"}
with open("urls.txt", encoding="utf-8") as f:
urls = [line.strip() for line in f if line.strip()]
with open("redirect_report.csv", "w", newline="", encoding="utf-8") as out:
writer = csv.writer(out)
writer.writerow(["url", "first_status", "hops", "final_url", "final_status", "chain"])
for url in urls:
try:
r = requests.get(url, headers=HEADERS, timeout=15, allow_redirects=True)
chain = [resp.status_code for resp in r.history] + [r.status_code]
writer.writerow([url, chain[0], len(r.history), r.url, r.status_code,
" > ".join(str(c) for c in chain)])
except requests.RequestException as e:
writer.writerow([url, "error", "", "", "", type(e).__name__])
print("checked", url)What it does, line by line in plain terms: it reads your URL list; requests each URL and follows redirects; records every status code in the chain (for example 301 > 301 > 200); and writes the first status, number of hops, final URL and final status to redirect_report.csv. Errors such as timeouts are recorded rather than crashing the script.
How to read the output:
| Pattern in the report | Meaning | Action |
|---|---|---|
| first_status 301, hops 1, final 200 | Clean permanent redirect | None |
| hops 2 or more | Redirect chain | Point the first URL straight at the final one |
| first_status 302 or 307 on a permanent move | Temporary redirect | Change to 301 or 308 |
| final_status 404 or 410 | Redirect to a dead page, or deleted page | Fix mapping, or confirm deletion was intended |
| first_status 200 on a URL that should redirect | Missing redirect, or a client-side one | Check the HTML for meta refresh or JavaScript |
| first_status 5xx | Server error | Escalate; Google slows crawling on 5xx |
One limitation matters: the requests library only follows server-side redirects. A 200 response that redirects with a meta refresh or JavaScript will show as 200. Our guide to meta refresh and JavaScript redirects explains how Google treats those, and Google’s redirects documentation lists which methods it treats as permanent.
Script two: pulling Search Console data past the 1,000-row ceiling
The Performance report in Search Console shows at most 1,000 rows in the interface. The Search Analytics API returns far more per request, which used to be 5,000 rows:
“With the GSC API, the maximum page size is 25,000 rows (the default is 1,000).”
— Analytics Edge, Download Over 25,000 Rows From Google Search Console API
Search Engine Land reported the change from 5,000 to 25,000 rows per request when Google introduced it. To get more than 25,000 rows, you request the next “page” of results with the startRow parameter until a page comes back short.
One-time setup (about 15 minutes):
- In Google Cloud Console, create a project and enable the Google Search Console API.
- Create a service account and download its JSON key into your project folder as
service-account.json. Treat this file like a password. - In Search Console, add the service account’s email address as a user on the property (restricted access is enough for reading data). Our Search Console property setup guide covers users and permissions.
- Use
sc-domain:example.comfor a Domain property or the full URL with trailing slash for a URL-prefix property.
import pandas as pd
from google.oauth2 import service_account
from googleapiclient.discovery import build
SITE = "sc-domain:example.com" # or "https://www.example.com/"
KEY_FILE = "service-account.json"
SCOPES = ["https://www.googleapis.com/auth/webmasters.readonly"]
creds = service_account.Credentials.from_service_account_file(KEY_FILE, scopes=SCOPES)
service = build("searchconsole", "v1", credentials=creds)
rows, start_row, page_size = [], 0, 25000
while True:
body = {
"startDate": "2026-07-01",
"endDate": "2026-09-30",
"dimensions": ["page", "query"],
"type": "web",
"rowLimit": page_size,
"startRow": start_row,
}
resp = service.searchanalytics().query(siteUrl=SITE, body=body).execute()
batch = resp.get("rows", [])
rows.extend(batch)
if len(batch) < page_size:
break
start_row += page_size
df = pd.DataFrame([
{"page": r["keys"][0], "query": r["keys"][1], "clicks": r["clicks"],
"impressions": r["impressions"], "ctr": r["ctr"], "position": r["position"]}
for r in rows
])
df.to_csv("gsc_page_query.csv", index=False)
print(len(df), "rows saved")This pulls page-and-query rows for a quarter and saves them to a CSV you can open in Excel or Sheets. Change the dimensions list to ["query"], ["page"], ["date"] or add "country" and "device" as needed.
Know the limits before you trust the totals. Third-party write-ups of Google’s documentation describe a daily cap:
“Google Search Console API does not return more than 50,000 rows per day per search type (web, image, video, etc.).”
— Weld, Google Search Console 50k limit
| Limit | Value | Reported by | Workaround |
|---|---|---|---|
| Rows per API request | 25,000 (default 1,000) | Analytics Edge, Search Engine Land | Page with startRow |
| Rows per day per search type | 50,000 | Weld | Query one day at a time; BigQuery bulk export |
| Data history | 16 months | Windsor.ai | Save exports monthly |
| Data freshness | Typically 2–3 days to settle | Windsor.ai | Exclude the latest days |
| Queries per minute, per site and per user | 1,200 | Windsor.ai | Add a short pause in loops |
Analytics Edge adds that results are sorted by clicks with no secondary sort, which can produce duplicated or missing rows across pages, and that anonymised queries are filtered out. So a common technique is to query one day at a time and combine the days. Windsor.ai’s guide also notes that Search Console keeps 16 months of data and that numbers take two to three days to settle, so avoid the most recent days in reports. For very large sites, Google’s bulk data export to BigQuery, announced in February 2023, provides a daily data dump without the row caps; see our guide to BigQuery exports for the analytics side.
Script three: grouping keywords by similarity with built-in Python
Keyword grouping usually means paid tools or embeddings. For a first pass, Python’s standard library is enough. This script needs no installs beyond Python itself. Put your keywords in keywords.csv with a header row and one keyword per line:
import csv
from difflib import SequenceMatcher
def normalise(kw):
words = kw.lower().replace("-", " ").split()
return " ".join(sorted(words))
def similar(a, b):
return SequenceMatcher(None, a, b).ratio()
keywords = [row[0] for row in csv.reader(open("keywords.csv", encoding="utf-8"))][1:]
groups = [] # each group: [label, [keywords]]
THRESHOLD = 0.75
for kw in keywords:
key = normalise(kw)
for group in groups:
if similar(key, group[0]) >= THRESHOLD:
group[1].append(kw)
break
else:
groups.append([key, [kw]])
with open("keyword_groups.csv", "w", newline="", encoding="utf-8") as f:
writer = csv.writer(f)
writer.writerow(["group", "keyword"])
for label, members in groups:
for kw in members:
writer.writerow([members[0], kw])
print(f"{len(keywords)} keywords grouped into {len(groups)} groups")The trick is in normalise(): lowercasing and sorting the words means “london emergency plumber” and “emergency plumber london” become the same string, so word order no longer matters. SequenceMatcher then scores how similar two strings are between 0 and 1, which catches plurals and small variations.
We tested it on a ten-keyword sample. It produced five groups: the three “emergency plumber london” variants together, the three “boiler repair cost” variants (including “cost of boiler repair” and “boiler repair costs uk”) together, “blocked drain” with “blocked drains”, and “unblock drain” and “plumber near me” on their own.
- Raise
THRESHOLD(towards 0.85) for tighter, smaller groups. - Lower it (towards 0.65) for broader groups, at the risk of merging different intents.
- Remember the limit: this is lexical similarity. “Unblock drain” and “blocked drain” may share intent, but the strings are not similar enough to group. For intent-level clustering, compare search results or use embeddings, as discussed in our keyword clustering guide.
A worked example: one Monday morning, three scripts
A hypothetical scenario. An agency has just migrated a client from an old CMS. It has 640 old URLs from the previous sitemap, a Search Console property and a keyword list of 2,100 terms for the next content plan.
- 09:00: run the redirect checker on the 640 URLs. The report shows 590 clean 301s, 31 chains of two hops, 12 redirects landing on 404s and 7 URLs still returning 200. That is 50 fixes, prioritised by which URLs had organic clicks.
- 09:20: run the Search Console script for the 90 days before launch, with page as the dimension, and join it to the redirect report in Excel or pandas. Two of the 12 broken redirects turn out to be among the top 20 landing pages. They are fixed first.
- 09:40: run the keyword grouper on 2,100 terms. At 0.75 it returns several hundred groups; the writer reviews the 50 largest and maps them to existing or planned pages.
Total hands-on time is under an hour for work that would previously take most of a day. On much bigger sites, the same approach scales with sampling and log data; see large-site SEO.
Using AI assistants to write and debug scripts safely
AI assistants are the reason Python is now realistic for non-developers. They are very good at explaining errors, adding features to working scripts and converting a description into a first draft. They also make confident mistakes. A safe working pattern:
- Describe inputs and outputs precisely: “Read urls.txt, one URL per line; write a CSV with columns url, status, final_url.”
- Ask for comments on every block so you can follow what it does.
- Run on five rows first, and check the output by hand against a browser or Search Console.
- Paste the full error message when something fails, along with the script, and ask what caused it and how to fix it.
- Never paste secrets. Replace keys, passwords and service-account contents with placeholders.
- Check library names and API fields against official documentation, because invented parameter names are a common AI error.
- Keep working versions in a folder or a Git repository so you can roll back.
Our library of ChatGPT and Claude prompts for SEO includes prompts for script writing. When the scripts become part of a team’s release process, share them with developers using the checklist in SEO for developers.
Questions beginners ask about Python for SEO
Do I need to learn programming properly before using Python for SEO?
No. Most useful SEO scripts are 20–50 lines: read a list, call a website or API, write a CSV. Learn to run a script, install a package and read an error message, and you can adapt existing scripts. Deeper skills come from fixing real problems.
Is Python better than Google Sheets or Apps Script for SEO tasks?
They overlap. Sheets is better for sharing and light formulas; Python is better for thousands of URLs, API pagination, joining large files and anything you want to repeat on a schedule. Many teams pull data with Python and present it in Sheets or a dashboard.
Why does the Search Console API return fewer rows than I expect?
Two limits apply. Each request returns at most 25,000 rows, so you must page with startRow. And third-party documentation of Google’s limits reports a cap of 50,000 rows per day per search type, with anonymised queries filtered out, so very large sites will not get every query. For complete data, use the BigQuery bulk export.
Is it safe to paste my scripts into ChatGPT or Claude?
Scripts themselves are usually fine. Never paste API keys, service-account JSON files, passwords or client data you are not allowed to share. Replace them with placeholders before asking for help.
Will running a status code checker harm my site?
Not at modest speed. The scripts here make one request at a time. If you check tens of thousands of URLs, add a short pause between requests and run during quiet hours, especially on smaller servers.
Your first week of SEO scripting
Day one: install Python, create a virtual environment and run the redirect checker on ten URLs. Day two: set up a service account and pull a month of Search Console data. Day three: group a keyword list and compare it with your existing pages. By the end of the week you will have three reusable tools and, more importantly, the confidence to adapt them.
If you would rather have the analysis without the scripting, our SEO team runs these checks as part of audits and migrations. Request a free quote and tell us what you need automated.
Further reading on python for seo
- Should Internal Search Result Pages Be Indexed?
- HTTP/2 and HTTP/3: Do Web Protocols Affect SEO?
- Preload, Preconnect and Prefetch: Resource Hints for Faster Pages
Sources for this Python for SEO guide
- Analytics Edge: Download over 25,000 rows from Google Search Console API
- Weld: The Google Search Console 50k row limit (2025)
- Windsor.ai: Guide to the Google Search Console API (updated September 2026)
- Search Engine Land: Search Analytics API now gives you 25,000 rows per request
- Google Search Central Blog: Bulk data export (February 2023)
- Google Search Central: Redirects and Google Search
- Google Search Central: HTTP status codes and network errors
