AI SEO

Search is a pipe. Most agencies decorate the walls.

Crawling, indexing, serving. Render in the middle. No paid crawl. No ranking guarantee. That is Google's own documentation, not a hot take.

Search is a pipe. Most agencies decorate the walls.

How Google Search actually works

Google Search is a fully automated system. It does not accept payment to crawl a site more often or rank it higher. It also does not guarantee that a page will be crawled, indexed, or served, even when the page follows Search Essentials.

There are three stages. Not every URL survives all three.

  • Crawling. Google discovers URLs, then Googlebot fetches them.
  • Indexing. Google analyzes the fetched (and often rendered) content and stores what it understood.
  • Serving. A person searches. Google returns pages it believes are relevant and useful.

Source: Google Search Central, In-depth guide to how Google Search works (updated December 2025).

Stage 1. Discovery and crawl

There is no central registry of the web. Google finds URLs three ways:

  • It already knows the URL from a previous visit.
  • It extracts a link from a page it already crawled.
  • You give it a list (a sitemap).

Then it may crawl. Crawl rate is algorithmic. HTTP 500s tell it to slow down. `robots.txt` can disallow a fetch. Login walls hide pages. A 2MB HTML fetch cap (March 2026, Inside Googlebot) means oversized shells get truncated.

YOWYO implementation:

  • Real path URLs (`/work/gab44`), never hash routes (`#/work/gab44`). Hash fragments are not sent to the server and are a weak discovery surface.
  • `<a href>` with real paths so the first HTML parse already yields links, before JavaScript runs.
  • `sitemap.xml` of crawlable URLs, named in `robots.txt`.
  • Lean HTML. No 3MB app shell.
  • `robots.txt` Allow all public pages. Disallow the editor.

`robots.txt` controls crawling, not indexing. A page blocked in robots can still appear as a URL-only result if Google found the URL elsewhere. To control indexing, use a robots meta tag or `X-Robots-Tag`.

Stage 2. Render, then index

Googlebot fetches. It parses HTML, extracts links, queues a render. A recent Chromium executes JavaScript. The rendered HTML is what gets indexed, and links found after render go back into the crawl queue.

That is why an empty app-shell SPA can eventually rank, and why it is still a worse idea:

  • Render is a second queue. Delay is real.
  • Not every bot runs JavaScript.
  • Users (and other agents) see a blank first paint.

Google's own JavaScript SEO guide still says server-side or pre-rendering is a great idea.

YOWYO implementation:

  • First HTML includes title, description, Organization + WebSite JSON-LD, and a `<noscript>` map of real links.
  • Unique `<title>` and meta description per route.
  • Canonical per route so similar pages cluster correctly.
  • Visible FAQ on `/faq` with `FAQPage` JSON-LD using the same words.
  • Images get alt. Pages stay under the fetch budget.

Stage 3. Serving (including generative features)

Ranking is not one score. Google describes systems (relevance, quality, spam, freshness, diversity). Helpful, people-first content is the documented requirement.

For generative AI features on Search, Google published a dedicated guide. The point is blunt: there is no separate "AI ranking hack." The same fundamentals apply. Unique titles. Crawlable links. Content a human asked for.

YOWYO will not:

  • Buy links
  • Cloak
  • Spin scaled unhelpful pages
  • Build doorway sites
  • Invent reviews
  • Promise positions

AEO extras (not a Google requirement)

Other agents read the open web too. We ship:

ArtifactWho it helpsGoogle requirement?
`robots.txt` + sitemapGooglebot and othersSitemap is optional; robots is a crawl control
`llms.txt`Other agentsNo. Courtesy map.
FAQPage JSON-LDRich results / extractorsOnly if a visible FAQ exists
Allow GPTBot, ClaudeBot, PerplexityBot, Google-ExtendedAnswer enginesOptional

What a YOWYO AI SEO engagement actually does

  • Map every public URL. Kill hash routers and soft 404s.
  • Put titles, descriptions, and canonicals on real HTML.
  • Make internal links crawlable.
  • Ship sitemap + robots that agree with the live tree.
  • Write people-first pages for the queries the business actually earns.
  • Add FAQ only when the answers are on the page.
  • Keep HTML lean. Measure. Do not decorate the crawl.

That is plumbing. Not blog spam with "AI" in the title.