How to Get Google AI Overviews Into Your Search Results

Google's AI Overviews load after the rest of the results page, so a scraper that grabs the page as soon as it arrives can miss them. Massive's Search endpoint fixes that with one parameter. Add awaiting=ai and the request waits up to one minute for the AI Overview to render before returning the page. The same parameter can wait for People Also Ask (awaiting=answers) and sponsored results (awaiting=ads). This spotlight covers how it works, how to combine it with location and language settings, and the cache setting to change when you need a fresh answer.

Key Takeaways

• awaiting=ai makes Massive's Search endpoint wait up to one minute for Google's AI Overview before returning results (Massive Docs).
• awaiting=answers waits for People Also Ask, and awaiting=ads waits for sponsored results. Repeat the key to wait for several.
• 18% of Google searches in Pew's March 2025 data produced an AI summary (Pew Research Center, 2025).
• Results are cached for one day by default. Set expiration=0 to force a fresh search.

Why capture AI Overviews at all?

Because they change what people click. In a Pew Research Center study of 900 U.S. adults' browsing in March 2025, 18% of Google searches produced an AI summary. Users who saw one clicked a traditional result in 8% of visits, against 15% for searches without one, and clicked a link inside the summary in just 1% of visits (Pew Research Center, 2025).

If an AI Overview appears for a query you care about, it is the first thing a searcher reads. SEO teams track whether their pages are cited in it. Generative engine optimization (GEO) platforms measure how often a brand appears. Content teams mine People Also Ask for the questions real users type. All of them need the full results page, including the parts that load last. Our map of the LLM search stack lists 13 products whose main job is measuring AI visibility. Google describes the feature itself as a way to "help people get to the gist of a complicated topic or question more quickly" (Google Search Central), which is exactly why it sits above every organic result.

Why do AI Overviews go missing from scraped results?

Google renders some features after the main results. The documentation calls these "lazy" features. Per Massive's docs, the AI Overview "can take significant time to generate." A request that captures the page the moment the organic results arrive can return a page where the AI Overview hasn't appeared yet. You get a valid response with the most important block missing, and nothing in the response tells you it was ever there. That silent gap is worse than an error, because it reads as "no AI Overview for this query" in your data.

Multiply that across thousands of tracked keywords and the error compounds: a visibility report can show a brand losing AI Overview presence when nothing changed except how fast the capture tool gave up. The fix is to wait for the feature explicitly, and to record that you waited, so a later reader of the data knows an empty result means "none appeared within the window" rather than "we didn't look."

How the awaiting parameter works

The awaiting parameter tells the Search endpoint which lazy features to wait for before it returns. Per the Search documentation:

To wait for more than one, repeat the key. This request waits for both the AI Overview and People Also Ask:

API=render.joinmassive.com

curl -G "https://$API/search" \
  --oauth2-bearer "$TOKEN" \
  -d terms=how+to+make+tea \
  -d awaiting=ai \
  -d awaiting=answers

Export your Massive API token as TOKEN first (export TOKEN=...). --oauth2-bearer sends it as an Authorization: Bearer header, and -G adds each -d value to the URL as a query parameter.

The same request in Python. requests encodes a list value as repeated keys, which is the form the endpoint expects:

import os
import requests

URL = ("https://render."
       "joinmassive.com/search")
TOKEN = os.environ["TOKEN"]

params = {
    "terms": "how to make tea",
    "awaiting": [
        "ai", "answers"
    ],
}
resp = requests.get(
    URL,
    params=params,
    headers={
        "Authorization":
        f"Bearer {TOKEN}"
    },
    timeout=120,
)
resp.raise_for_status()
html = resp.text

Set your client timeout above one minute, as in the example. A short client timeout will cut off a request that is still legitimately waiting for an AI Overview.

We checked these values from a second direction. Orthogonal's x402 gateway, which offers this endpoint to AI agents (see our Orthogonal spotlight), publishes a machine-readable schema for it. When we decoded that schema on October 2, 2026, the awaiting field read: "Lazy features to wait for (ai, answers, ads); repeatable." That matches Massive's own documentation, value for value.

What if no AI Overview appears?

Not every query gets an AI Overview. Google says plainly that AI Overviews "are only shown when our systems determine that it is additive to classic Search, and as such, often don't trigger" (Google Search Central). Massive's documentation doesn't spell out what the response looks like when none renders within the one-minute cap. Don't treat a returned page as proof either way. Check the HTML for the AI Overview block itself, and record "not present" as an observation for that query, location, and time, not as a permanent fact. Running the same query again later, or from another location, is the only way to tell an absent Overview from a slow one.

Two habits make that practical. First, log each capture with the query, location, language, time, and the awaiting values you sent. Second, save one known-positive response, a query where you've seen an AI Overview in a browser, and use it as the test fixture for your parser.

Combine awaiting with location, language, and device

AI Overviews and People Also Ask vary by where and how someone searches. The Search endpoint takes the same parameters that shape a real user's results:

Source: Massive Docs, Search.

Measuring one query from one place tells you one version of the answer. If your buyers are in several markets, run the same query with each market's location and language and compare the AI Overviews side by side. To compare what LLM chat products answer by location, rather than Google, see the AI chat endpoint.

Fresh results: the expiration setting

By default, the Search endpoint caches results for one day (expiration=1). That's useful for repeated lookups. For monitoring AI Overviews, though, you usually want the live answer. Set expiration=0 to disable the cache:

API=render.joinmassive.com

curl -G "https://$API/search" \
  --oauth2-bearer "$TOKEN" \
  -d terms=best+crm+software \
  -d awaiting=ai \
  -d expiration=0

For large batches, use asynchronous mode. The Search endpoint can queue a search and let you retrieve it later, and the callback parameter, a webhook or SQS URL, notifies you when results are ready (Massive Docs). That fits lazy features well, because a batch of queries that each wait up to a minute for an AI Overview shouldn't hold open thousands of client connections.

What the response looks like

Search results come back as rendered HTML. When you request several pages that don't use infinite scrolling, each page's HTML is separated from the next by an empty line, per the docs. You parse the AI Overview, People Also Ask, and ads blocks out of that HTML. Google changes its markup often, so test your parser against saved responses and re-check it regularly.

For how this endpoint compares with search APIs built for agents, see web search APIs for AI agents compared.

AI Overviews and People Also Ask: the bottom line

• AI Overviews render late. Wait for them with awaiting=ai, or your data will show gaps that look like real absences.
• Combine lazy features by repeating the key, and set your client timeout above a minute.
• To see what users in each market see, pair awaiting with uule, language, and device, and use expiration=0 whenever you need today's answer instead of a cached one.

Sources

• Google Search Central, "AI Features and Your Website"
• Massive Docs, "Search"
• Pew Research Center, "Google users are less likely to click on links when an AI summary appears in the results", 2025