Technical SEO Audit for 2026: Crawlability, Schema, AI Search

Technical SEO audit: what to fix first

A technical SEO audit is the quickest way to find why pages are indexed poorly, rendered slowly, or skipped by AI Overviews. In 2026, start with crawlability, canonicals, structured data, Core Web Vitals, and whether your content answers a query clearly enough for Google and AI search to extract it.

If you host client sites, this is also where hosting choices start to matter. A small business on the right platform can recover faster after a migration, and a team on a Hostperl VPS gets more control over headers, caching, logs, and server responses when a search issue needs a real fix.

This tutorial walks through a practical audit you can run on a live site, with commands for server checks, crawl checks, page speed checks, and schema validation. It also shows where to connect the findings to other Hostperl guidance, including DNS and SSL setup, safe hosting migration, and WordPress deployment on VPS.

What you need before you start

Use one test domain or staging site first. For this guide, replace these placeholders with your own values:

• SITE_URL = your full site URL, for example https://www.example.com
• DOMAIN_NAME = your root domain, for example example.com
• SITEMAP_URL = your sitemap, for example https://www.example.com/sitemap.xml
• SERVER_IP = your VPS IP address, for example 203.0.113.10

Run the server-side checks from the VPS and the browser checks from your local computer. Keep the site live while you audit it, then fix one issue at a time.

Confirm the server and OS

On your local computer, connect to the VPS first:

ssh root@203.0.113.10

If Hostperl gave you a default non-root account, use that username instead. Once you are in, check the operating system before you install or edit anything:

cat /etc/os-release

You should see either Ubuntu or Debian on one branch, or AlmaLinux/Rocky Linux on another. The next steps differ by family.

Prepare a non-root admin user for audit work

Do not do routine SEO checks as root. Create a sudo-capable user and keep the original root session open until the new login works.

Ubuntu and Debian

On the VPS as root, create the account, add it to sudo, and prepare SSH access:

adduser seoadmin
usermod -aG sudo seoadmin
mkdir -p /home/seoadmin/.ssh
chmod 700 /home/seoadmin/.ssh

Copy your public key into /home/seoadmin/.ssh/authorized_keys. The safe way is to paste the key from your local computer:

cat > /home/seoadmin/.ssh/authorized_keys <<'EOF'
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExampleKeyHere yourname@laptop
EOF
chown -R seoadmin:seoadmin /home/seoadmin/.ssh
chmod 600 /home/seoadmin/.ssh/authorized_keys

On your local computer, open a second terminal and test:

ssh seoadmin@203.0.113.10
sudo -v

If sudo prompts for the password and returns without error, keep going.

AlmaLinux and Rocky Linux

On the VPS as root, create the account and grant wheel access:

useradd -m seoadmin
passwd seoadmin
usermod -aG wheel seoadmin
mkdir -p /home/seoadmin/.ssh
chmod 700 /home/seoadmin/.ssh

Then install the key and permissions:

cat > /home/seoadmin/.ssh/authorized_keys <<'EOF'
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExampleKeyHere yourname@laptop
EOF
chown -R seoadmin:seoadmin /home/seoadmin/.ssh
chmod 600 /home/seoadmin/.ssh/authorized_keys

Test from a second terminal before making any lockout-prone changes:

ssh seoadmin@203.0.113.10
sudo -v

Check crawlability before page speed

Search engines can only rank what they can fetch and understand. Start with robots rules, sitemap status, and HTTP response codes.

On your local computer, inspect robots.txt and the sitemap:

curl -I https://www.example.com/robots.txt
curl -I https://www.example.com/sitemap.xml
curl -s https://www.example.com/robots.txt

You want 200 OK for both files unless a temporary redirect is expected. A blocked robots file or a broken sitemap is one of the fastest ways to waste crawl budget.

Check a few important URLs for status codes and canonicals:

curl -I https://www.example.com/
curl -I https://www.example.com/services/
curl -I https://www.example.com/contact/

Watch for 200, 301, or 302. A 404 on a page you expect to rank means Google can’t use it.

Measure headers and canonical signals

Canonical mistakes often come from hosting migrations, trailing slashes, and www/non-www mismatches. That is common after moving from shared hosting to VPS, especially when a site goes live before DNS settles.

Inspect the full response headers and the canonical tag in the HTML:

curl -sL https://www.example.com/ | sed -n '1,120p' | grep -iE 'canonical|robots|x-robots-tag|content-security-policy'

You want one clear canonical URL per page. If the page points to the wrong domain, fix the CMS setting, the reverse proxy, or the application base URL. For WordPress sites, this usually belongs in the site URL settings, not only in the web server config.

Run a Core Web Vitals check

Core Web Vitals matter because slow pages rarely hold attention long enough to convert. They also affect whether Google treats your page as a decent candidate for answer snippets or AI summaries.

Install Lighthouse on the VPS or use your browser locally. On a Linux admin shell, this is a quick server-side check for mobile and desktop constraints:

node -v
npm -v
npx lighthouse https://www.example.com/ --output=json --output-path=/tmp/lighthouse-report.json --quiet

Look for LCP, INP, and CLS in the report. On a typical Hostperl VPS, a clean cache and a lean theme usually improve first load time more than cosmetic tweaks. If your site is resource-heavy, compare results before and after enabling page caching or a CDN.

Also check TTFB from the command line:

curl -o /dev/null -s -w 'DNS:%{time_namelookup} Connect:%{time_connect} TLS:%{time_appconnect} TTFB:%{time_starttransfer} Total:%{time_total}\n' https://www.example.com/

High TTFB usually points to backend slowness, overloaded PHP workers, or a database issue.

Validate structured data for AI search visibility

Schema helps search engines understand entities, not just keywords. For most hosting customers, the useful markup is straightforward: Organization, WebSite, BreadcrumbList, Article, Product, or FAQPage where appropriate.

Test a page with Google’s rich result testing approach and confirm the JSON-LD is valid. If you want a quick local check, pull the schema block from the page:

curl -sL https://www.example.com/ | grep -n '<script type="application/ld+json">' -A40 -B2

A healthy page includes one coherent entity profile, matching names, and consistent URLs. Don’t describe your business one way in the schema and another way in the visible page text.

If you are publishing service pages, make sure the page title, H1, schema name, and internal anchor text all refer to the same entity. That makes the page easier for Google and AI systems to interpret.

Fix site-wide issues on Ubuntu and Debian

On the VPS as the non-root sudo user, update the system and install basic audit tools:

sudo apt update
sudo apt -y upgrade
sudo apt -y install curl ca-certificates unzip jq

Check the current web server and PHP packages if the site is running on the box:

systemctl status nginx --no-pager
systemctl status apache2 --no-pager
php -v

If you maintain a WordPress site, read the WordPress VPS tutorial for the same family of deployment checks you will use after fixes go live.

Fix site-wide issues on AlmaLinux and Rocky Linux

On the VPS as the non-root sudo user, refresh packages and install the same audit helpers:

sudo dnf -y update
sudo dnf -y install curl ca-certificates unzip jq

Check the relevant service names, which differ from Debian-based systems:

systemctl status nginx --no-pager
systemctl status httpd --no-pager
php -v

If SELinux is enforcing and a page is failing to load, inspect the denial logs before you disable anything:

sudo ausearch -m AVC,USER_AVC -ts recent
sudo journalctl -t setroubleshoot --no-pager -n 50

Most hosting customers do better by fixing the label, boolean, or file context than by turning SELinux off.

Check logs before you change content

Search problems often look like SEO issues when they are really server errors. Review logs before rewriting page copy.

On the VPS as the non-root sudo user, use the relevant log paths:

sudo tail -n 100 /var/log/nginx/error.log
sudo tail -n 100 /var/log/apache2/error.log
sudo tail -n 100 /var/log/httpd/error_log
sudo journalctl -u nginx -n 50 --no-pager

Look for 5xx errors, upstream timeouts, PHP worker exhaustion, or repeated bot traffic. If you see upstream errors on a VPS plan, that is a sign the site may need more RAM or a better caching layer. Hostperl’s managed VPS hosting is a better fit than a crowded shared environment when search visibility depends on stable response times.

Use content structure that AI systems can quote safely

AI Overviews and answer engines prefer clear definitions, direct answers, and well-labeled sections. Your pages should answer the user’s question early, then expand with examples and limitations.

Use this pattern on important pages:

• One-sentence answer in the first paragraph
• One H2 per major subtopic
• Short lists with concrete values or steps
• Matching titles, meta descriptions, and headings
• Schema that matches the visible content exactly

If you run an agency or manage client sites, this matters even more. You need pages that survive migration, indexing, and support handoff without confusion. Hostperl’s panel comparison guide is useful when you want a clearer operational stack before a rebuild.

Common technical SEO fixes that move the needle

These are the fixes I see most often on sites that have been moved, reskinned, or rushed into production:

• Remove accidental noindex tags from staging templates
• Force one preferred host version, usually www or non-www, with 301 redirects
• Generate and submit a clean XML sitemap
• Compress images and use modern formats where possible
• Reduce render-blocking CSS and scripts on the homepage
• Make sure article pages have a visible author, date, and clear entity name

After each fix, recrawl the affected page and re-check the HTML source. Do not stack six changes at once unless you want to guess which one worked.

Verification from the server and from a client device

On the VPS as the non-root sudo user, confirm the web service is running and listening:

systemctl status nginx --no-pager
ss -ltnp | grep -E ':80|:443'

Then confirm the page returns HTML and the canonical is present:

curl -sL https://www.example.com/ | grep -iE 'canonical|schema|article'

On your local computer, run a browser test and a crawl test:

curl -I https://www.example.com/

Open the page in a private browser window and inspect the rendered title, meta description, headings, and visible content. If the browser shows one thing and the source shows another, your JS rendering or cache layer needs attention.

For a basic smoke test, load the homepage, one service page, and one blog post. If all three return 200, show the same canonical host, and load within a reasonable time, your technical SEO audit has likely found the real issues.

Troubleshooting the most common failures

Issue: robots.txt returns 404.
Diagnostic: curl -I https://www.example.com/robots.txt
Clue: 404 or a redirect loop.
Fix: place the file in the web root and confirm the vhost maps the right document root.

Issue: canonical points to the wrong domain.
Diagnostic: curl -sL https://www.example.com/ | grep -i canonical
Clue: the tag points to staging or a previous host.
Fix: update the application base URL, then purge cache and reload the page.

Issue: TTFB is slow on every page.
Diagnostic: curl -o /dev/null -s -w '%{time_starttransfer}\n' https://www.example.com/
Clue: consistently high first-byte time.
Fix: check PHP-FPM, database latency, and page cache before touching content.

Issue: structured data is invalid.
Diagnostic: copy the JSON-LD block into a validator.
Clue: mixed entity names or broken JSON syntax.
Fix: correct the schema so it matches the visible page and test again.

FAQ

How often should I run a technical SEO audit?

Run a full audit after every migration, redesign, domain change, or major plugin update. For stable sites, a monthly check is enough to catch crawl and performance regressions early.

What matters most for AI search visibility?

Clear answers, consistent entity naming, valid schema, and pages that load fast enough to be fetched reliably. AI systems need clean signals more than keyword repetition.

Should I fix content or server issues first?

Fix server and crawlability issues first. If pages return errors, load too slowly, or block crawlers, content changes will not help much.

Do I need schema on every page?

No. Add structured data where it matches the page purpose. Service pages, articles, FAQs, products, and breadcrumbs usually matter most.

What should I check after a hosting migration?

Test redirects, canonicals, sitemap paths, SSL, robots.txt, and HTTP status codes. Then confirm the site still renders correctly in a browser and from the command line.