How should failed or empty collections be handled?

Inspect per-target failedRuns and each successful run identity. Empty sources or null fields are valid output; a missing failed run is not an empty successful answer. A completed job can contain failed collections.

Collected data

Google Search: sources

Provider-specific source titles and URLs, with empty lists allowed.

Collection target: google. Field availability depends on the provider output shown below.

Understand the collected data

Chatbots return sources alongside text and markdown. Google returns AI Overview sources within aioverview and search links within serp. Google News returns article links within googleNews.results and googleNews.topNews.

Collected data

The API does not guarantee a complete reference panel, resolve every redirect, or return normalized domains and source frequency. Build those transformations downstream if needed.

The google target collects search modules in serp and an AI Overview in aioverview when present. Organic results, sponsored results, shopping and People Also Ask depend on the returned page. An absent AI Overview is null, not a failed run. Google Search and Google News share one collection task per repeat when requested together. They still return separate response arrays and per-target failed-run counts.

Provider output examples

These are illustrative successful run excerpts, not full result files or calculated metrics. Submit POST /v1/crawl-jobs, track completion, then download the authenticated result. Outputs depend on the target; a chatbot has no news articles or Google position fields.

{
  "runNumber": 1,
  "runTaskId": "7f5c0888-f5ad-4661-b07b-79fbbe1c7b31",
  "status": "completed",
  "capturedAt": "2026-10-09T10:01:00.000Z",
  "pageUrl": "https://www.google.com/search?q=example",
  "aioverview": null,
  "serp": {
    "organic": [
      {
        "position": 1,
        "snippet": "Illustrative search result summary.",
        "title": "Illustrative CRM comparison",
        "url": "https://example.com/crm-comparison"
      }
    ],
    "peopleAlsoAsk": [],
    "shopping": [],
    "sponsored": []
  }
}

Illustrative provider output. Empty lists and null fields are valid; inspect failedRuns in the result file.

Full result format and failed-run handling →

Sources FAQ

Does the API calculate sources?: Chatbots return sources alongside text and markdown. Google returns AI Overview sources within aioverview and search links within serp. Google News returns article links within googleNews.results and googleNews.topNews.
How should failed or empty collections be handled?: Inspect per-target failedRuns and each successful run identity. Empty sources or null fields are valid output; a missing failed run is not an empty successful answer. A completed job can contain failed collections.

Use collected data to build your own reporting and analysis. Our team arranges API access and follows up with next steps.

Request API access Read the docs