Search is a pipe. Most agencies decorate the walls.
Crawling, indexing, serving. Render in the middle. No paid crawl. No ranking guarantee. That is Google's own documentation, not a hot take.

How Google Search actually works
Google Search is a fully automated system. It does not accept payment to crawl a site more often or rank it higher. It also does not guarantee that a page will be crawled, indexed, or served, even when the page follows Search Essentials.
There are three stages. Not every URL survives all three.
- Crawling. Google discovers URLs, then Googlebot fetches them.
- Indexing. Google analyzes the fetched (and often rendered) content and stores what it understood.
- Serving. A person searches. Google returns pages it believes are relevant and useful.
Source: Google Search Central, In-depth guide to how Google Search works (updated December 2025).
Stage 1. Discovery and crawl
There is no central registry of the web. Google finds URLs three ways:
- It already knows the URL from a previous visit.
- It extracts a link from a page it already crawled.
- You give it a list (a sitemap).
Then it may crawl. Crawl rate is algorithmic. HTTP 500s tell it to slow down. `robots.txt` can disallow a fetch. Login walls hide pages. A 2MB HTML fetch cap (March 2026, Inside Googlebot) means oversized shells get truncated.
YOWYO implementation:
- Real path URLs (`/work/gab44`), never hash routes (`#/work/gab44`). Hash fragments are not sent to the server and are a weak discovery surface.
- `<a href>` with real paths so the first HTML parse already yields links, before JavaScript runs.
- `sitemap.xml` of crawlable URLs, named in `robots.txt`.
- Lean HTML. No 3MB app shell.
- `robots.txt` Allow all public pages. Disallow the editor.
`robots.txt` controls crawling, not indexing. A page blocked in robots can still appear as a URL-only result if Google found the URL elsewhere. To control indexing, use a robots meta tag or `X-Robots-Tag`.
Stage 2. Render, then index
Googlebot fetches. It parses HTML, extracts links, queues a render. A recent Chromium executes JavaScript. The rendered HTML is what gets indexed, and links found after render go back into the crawl queue.
That is why an empty app-shell SPA can eventually rank, and why it is still a worse idea:
- Render is a second queue. Delay is real.
- Not every bot runs JavaScript.
- Users (and other agents) see a blank first paint.
Google's own JavaScript SEO guide still says server-side or pre-rendering is a great idea.
YOWYO implementation:
- First HTML includes title, description, Organization + WebSite JSON-LD, and a `<noscript>` map of real links.
- Unique `<title>` and meta description per route.
- Canonical per route so similar pages cluster correctly.
- Visible FAQ on `/faq` with `FAQPage` JSON-LD using the same words.
- Images get alt. Pages stay under the fetch budget.
Stage 3. Serving (including generative features)
Ranking is not one score. Google describes systems (relevance, quality, spam, freshness, diversity). Helpful, people-first content is the documented requirement.
For generative AI features on Search, Google published a dedicated guide. The point is blunt: there is no separate "AI ranking hack." The same fundamentals apply. Unique titles. Crawlable links. Content a human asked for.
YOWYO will not:
- Buy links
- Cloak
- Spin scaled unhelpful pages
- Build doorway sites
- Invent reviews
- Promise positions
AEO extras (not a Google requirement)
Other agents read the open web too. We ship:
| Artifact | Who it helps | Google requirement? |
|---|---|---|
| `robots.txt` + sitemap | Googlebot and others | Sitemap is optional; robots is a crawl control |
| `llms.txt` | Other agents | No. Courtesy map. |
| FAQPage JSON-LD | Rich results / extractors | Only if a visible FAQ exists |
| Allow GPTBot, ClaudeBot, PerplexityBot, Google-Extended | Answer engines | Optional |
What a YOWYO AI SEO engagement actually does
- Map every public URL. Kill hash routers and soft 404s.
- Put titles, descriptions, and canonicals on real HTML.
- Make internal links crawlable.
- Ship sitemap + robots that agree with the live tree.
- Write people-first pages for the queries the business actually earns.
- Add FAQ only when the answers are on the page.
- Keep HTML lean. Measure. Do not decorate the crawl.
That is plumbing. Not blog spam with "AI" in the title.