# How to Get Google AI Overviews Into Your Search Results

Google's AI Overviews load after the rest of the results page, so a scraper that grabs the page as soon as it arrives can miss them. Massive's Search endpoint fixes that with one parameter. Add `awaiting=ai` and the request waits up to one minute for the AI Overview to render before returning the page. The same parameter can wait for People Also Ask (`awaiting=answers`) and sponsored results (`awaiting=ads`). This spotlight covers how it works, how to combine it with location and language settings, and the cache setting to change when you need a fresh answer.

> **Key Takeaways**
>
> - `awaiting=ai` makes Massive's Search endpoint wait up to one minute for Google's AI Overview before returning results ([Massive Docs](https://docs.joinmassive.com/web-render/search)).
> - `awaiting=answers` waits for People Also Ask, and `awaiting=ads` waits for sponsored results. Repeat the key to wait for several.
> - 18% of Google searches in Pew's March 2025 data produced an AI summary ([Pew Research Center](https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/), 2025).
> - Results are cached for one day by default. Set `expiration=0` to force a fresh search.

## Why capture AI Overviews at all?

Because they change what people click. In a Pew Research Center study of 900 U.S. adults' browsing in March 2025, 18% of Google searches produced an AI summary. Users who saw one clicked a traditional result in 8% of visits, against 15% for searches without one, and clicked a link inside the summary in just 1% of visits ([Pew Research Center](https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/), 2025).

![Bar chart from Pew Research Center, March 2025: users clicked a traditional result in 15% of visits without an AI summary and 8% with one, and clicked a link inside the summary in 1% of visits. 18% of searches produced an AI summary.](assets/pew-ai-summary-clicks.png "Source: Pew Research Center, 900 U.S. adults, March 2025.")

If an AI Overview appears for a query you care about, it is the first thing a searcher reads. SEO teams track whether their pages are cited in it. Generative engine optimization (GEO) platforms measure how often a brand appears. Content teams mine People Also Ask for the questions real users type. All of them need the full results page, including the parts that load last. Our [map of the LLM search stack](https://www.joinmassive.com/blog/llm-search-stack-2026) lists 13 products whose main job is measuring AI visibility. Google describes the feature itself as a way to "help people get to the gist of a complicated topic or question more quickly" ([Google Search Central](https://developers.google.com/search/docs/appearance/ai-features)), which is exactly why it sits above every organic result.

## Why do AI Overviews go missing from scraped results?

Google renders some features after the main results. The documentation calls these "lazy" features. Per Massive's docs, the AI Overview "can take significant time to generate." A request that captures the page the moment the organic results arrive can return a page where the AI Overview hasn't appeared yet. You get a valid response with the most important block missing, and nothing in the response tells you it was ever there. That silent gap is worse than an error, because it reads as "no AI Overview for this query" in your data.

![Two capture paths for the same query. Capturing right away returns the page when organic results arrive, and the AI Overview renders after the capture, so it is missed. With awaiting=ai the request waits up to one minute for the AI Overview and returns the full page.](assets/awaiting-timeline.png "Why AI Overviews go missing, and how awaiting=ai fixes it.")

Multiply that across thousands of tracked keywords and the error compounds: a visibility report can show a brand losing AI Overview presence when nothing changed except how fast the capture tool gave up. The fix is to wait for the feature explicitly, and to record that you waited, so a later reader of the data knows an empty result means "none appeared within the window" rather than "we didn't look."

## How the awaiting parameter works

The `awaiting` parameter tells the Search endpoint which lazy features to wait for before it returns. Per the [Search documentation](https://docs.joinmassive.com/web-render/search):

| Value | Waits for |
|-------|-----------|
| `ai` | The AI Overview, for a maximum of 1 minute |
| `answers` | The People Also Ask results |
| `ads` | Sponsored results |

To wait for more than one, repeat the key. This request waits for both the AI Overview and People Also Ask:

```bash
API=render.joinmassive.com

curl -G "https://$API/search" \
  --oauth2-bearer "$TOKEN" \
  -d terms=how+to+make+tea \
  -d awaiting=ai \
  -d awaiting=answers
```

Export your Massive API token as `TOKEN` first (`export TOKEN=...`). `--oauth2-bearer` sends it as an `Authorization: Bearer` header, and `-G` adds each `-d` value to the URL as a query parameter.

The same request in Python. `requests` encodes a list value as repeated keys, which is the form the endpoint expects:

```python
import os
import requests

URL = ("https://render."
       "joinmassive.com/search")
TOKEN = os.environ["TOKEN"]

params = {
    "terms": "how to make tea",
    "awaiting": [
        "ai", "answers"
    ],
}
resp = requests.get(
    URL,
    params=params,
    headers={
        "Authorization":
        f"Bearer {TOKEN}"
    },
    timeout=120,
)
resp.raise_for_status()
html = resp.text
```

Set your client timeout above one minute, as in the example. A short client timeout will cut off a request that is still legitimately waiting for an AI Overview.

We checked these values from a second direction. Orthogonal's x402 gateway, which offers this endpoint to AI agents (see our [Orthogonal spotlight](https://www.joinmassive.com/blog/orthogonal-spotlight)), publishes a machine-readable schema for it. When we decoded that schema on October 2, 2026, the `awaiting` field read: "Lazy features to wait for (ai, answers, ads); repeatable." That matches Massive's own documentation, value for value.

## What if no AI Overview appears?

Not every query gets an AI Overview. Google says plainly that AI Overviews "are only shown when our systems determine that it is additive to classic Search, and as such, often don't trigger" ([Google Search Central](https://developers.google.com/search/docs/appearance/ai-features)). Massive's documentation doesn't spell out what the response looks like when none renders within the one-minute cap. Don't treat a returned page as proof either way. Check the HTML for the AI Overview block itself, and record "not present" as an observation for that query, location, and time, not as a permanent fact. Running the same query again later, or from another location, is the only way to tell an absent Overview from a slow one.

Two habits make that practical. First, log each capture with the query, location, language, time, and the `awaiting` values you sent. Second, save one known-positive response, a query where you've seen an AI Overview in a browser, and use it as the test fixture for your parser.

## Combine awaiting with location, language, and device

AI Overviews and People Also Ask vary by where and how someone searches. The Search endpoint takes the same parameters that shape a real user's results:

| Parameter | What it controls |
|-----------|------------------|
| `uule` | Google's encoded location string for geotargeting the search |
| `language` | Search language (name, ISO code, or Google code) |
| `display` | Interface display language |
| `device` | Device to emulate |
| `serps` | Number of results pages to retrieve (1 to 10) |
| `size` | Results per page (0 to 100) |
| `offset` | Initial results to skip (0 to 100) |

Source: [Massive Docs, Search](https://docs.joinmassive.com/web-render/search).

Measuring one query from one place tells you one version of the answer. If your buyers are in several markets, run the same query with each market's location and language and compare the AI Overviews side by side. To compare what LLM chat products answer by location, rather than Google, see the [AI chat endpoint](https://www.joinmassive.com/blog/ai-chat-endpoint-geotargeted-llm-completions).

## Fresh results: the expiration setting

By default, the Search endpoint caches results for one day (`expiration=1`). That's useful for repeated lookups. For monitoring AI Overviews, though, you usually want the live answer. Set `expiration=0` to disable the cache:

```bash
API=render.joinmassive.com

curl -G "https://$API/search" \
  --oauth2-bearer "$TOKEN" \
  -d terms=best+crm+software \
  -d awaiting=ai \
  -d expiration=0
```

For large batches, use asynchronous mode. The Search endpoint can queue a search and let you retrieve it later, and the `callback` parameter, a webhook or SQS URL, notifies you when results are ready ([Massive Docs](https://docs.joinmassive.com/web-render/search)). That fits lazy features well, because a batch of queries that each wait up to a minute for an AI Overview shouldn't hold open thousands of client connections.

## What the response looks like

Search results come back as rendered HTML. When you request several pages that don't use infinite scrolling, each page's HTML is separated from the next by an empty line, per the docs. You parse the AI Overview, People Also Ask, and ads blocks out of that HTML. Google changes its markup often, so test your parser against saved responses and re-check it regularly.

For how this endpoint compares with search APIs built for agents, see [web search APIs for AI agents compared](https://www.joinmassive.com/blog/web-search-apis-for-ai-agents-compared-seltz-exa-brave-and-render-apis).

## Frequently Asked Questions

### How do I scrape Google AI Overviews with Massive?

Send a request to the `/search` endpoint on render.joinmassive.com with your query in `terms` and add `awaiting=ai`. The endpoint waits up to one minute for the AI Overview to render, then returns the full results page as HTML.

### How long does the Search endpoint wait for an AI Overview?

Up to one minute. Set your HTTP client's timeout above that so it doesn't cut off a request that is still waiting.

### Can I capture People Also Ask and AI Overviews in one request?

Yes. Repeat the parameter: send `awaiting=ai` and `awaiting=answers` in the same request. Add `awaiting=ads` to also wait for sponsored results.

### Are Search endpoint results cached?

Yes, for one day by default. Set `expiration=0` to disable the cache and get a fresh search.

## AI Overviews and People Also Ask: the bottom line

- AI Overviews render late. Wait for them with `awaiting=ai`, or your data will show gaps that look like real absences.
- Combine lazy features by repeating the key, and set your client timeout above a minute.
- To see what users in each market see, pair `awaiting` with `uule`, `language`, and `device`, and use `expiration=0` whenever you need today's answer instead of a cached one.

## Sources

- Google Search Central, ["AI Features and Your Website"](https://developers.google.com/search/docs/appearance/ai-features)
- Massive Docs, ["Search"](https://docs.joinmassive.com/web-render/search)
- Pew Research Center, ["Google users are less likely to click on links when an AI summary appears in the results"](https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/), 2025