AI Answer Sampling
Asks the same prompt several times and returns how the answers differed — per business, how many runs named it, its best, worst and average position, and the text of every run.
When to use this
A client asking "am I showing up in ChatGPT?" is asking for a rate, and one call cannot give one. A model composes its answer per request, so ask twice ten minutes apart and the businesses and their order can differ. This runs the prompt 2-25 times and hands back the distribution: named in 8 of 10 runs, average position 2.1.
/v1/ai/sampleParameters
| Parameter | Type | Required | Description |
|---|---|---|---|
| keyword | string | Yes | The prompt or keyword to ask repeatedly (max 500 chars) |
| platform | string | Yes | "chat_gpt", "gemini" or "google_ai_mode". The mention endpoints are not sampleable — they read stored aggregates, so running one ten times returns the same numbers ten times. |
| samples | number | Yes | How many times to run the prompt, 2-25. Below about 5 the rate is too coarse to act on; 10 is the usual figure for daily tracking. |
| location | string | No | Location (e.g. "Orchard Park, NY"). Required on google_ai_mode, which is a Google search and depends on where it runs from. Gemini supports city-level; ChatGPT resolves to country. |
| force_web_search | boolean | No | Require the platform to search the web before answering. chat_gpt only; gemini returns a 400. |
| language | string | No | Language to ask in, as an ISO 639-1 code or name ("fr", "French"). Defaults to the language of the market you pass. |
| callback_url | string | No | Public https URL to POST the finished sample to, instead of polling. |
| force_new | boolean | No | Run the prompt again although an identical submit is inside the 10-minute replay window. This is how you take a second measurement of the same prompt on purpose; without it the repeat returns the first job. |
| idempotency_key | string | No | Your own key for this submit, max 200 chars. Two submits carrying the same key return the same job for 24 hours, whatever their parameters — use it when runs are driven from a schedule. |
Example Request
curl -X POST https://api.localseodata.com/v1/ai/sample \
-H "Authorization: Bearer sk_live_your_key" \
-H "Content-Type: application/json" \
-d '{
"keyword": "best plumber in Orchard Park NY",
"platform": "chat_gpt",
"samples": 10
}'Example Response
{
"status": "success",
"credits_used": 40,
"data": {
"job_id": "sample_V1StGXR8Z5",
"status": "complete",
"keyword": "best plumber in Orchard Park NY",
"platform": "chat_gpt",
"samples_requested": 10,
"samples_returned": 10,
"samples_failed": 0,
"samples_refused": 0,
"samples_unreadable": 0,
"samples_shortlist_read": 10,
"credits_reserved": 40,
"credits_refunded": 0,
"credits_charged": 40,
"reader_version": 5,
"businesses": [
{
"name": "Bolt Plumbing LLC",
"matched_on": "domain",
"cid": "",
"domain": "boltplumbing.com",
"name_variants": ["Bolt Plumbing", "Bolt Plumbing LLC"],
"cited_domains": [
{ "domain": "boltplumbing.com", "runs": 5 },
{ "domain": "bbb.org", "runs": 3 }
],
"category": "Plumber",
"rating": 4.8,
"reviews_count": 412,
"address": "1200 Southwestern Blvd, Orchard Park, NY",
"phone": "+1 716-555-0142",
"website": "https://boltplumbing.com",
"appearances": 8,
"appearance_rate": 0.8,
"listing_appearances": 8,
"shortlist_appearances": 7,
"shortlist_rate": 0.7,
"mention_appearances": 1,
"top_pick_count": 2,
"best_rank": 1,
"worst_rank": 4,
"mean_rank": 2.13,
"best_shortlist_rank": 1,
"mean_shortlist_rank": 1.86,
"runs_seen": [1, 2, 3, 5, 6, 8, 9, 10]
}
],
"runs": [
{
"run": 1,
"ok": true,
"outcome": "answered",
"shortlist_read": true,
"businesses": [],
"recommended_businesses": [
{
"shortlist_rank": 1,
"listed": true,
"name": "Bolt Plumbing",
"mention": "Bolt Plumbing — Orchard Park-based, 24/7 emergency calls.",
"is_top_pick": false,
"citation": { "url": "https://boltplumbing.com/", "domain": "boltplumbing.com" }
}
],
"response_text": "For plumbing in Orchard Park, the names that come up most are..."
}
]
}
}Response Fields
| Field | Type | Description |
|---|---|---|
| samples_requested | integer | How many runs were asked for |
| samples_returned | integer | How many came back. Every rate below is out of this number |
| samples_failed | integer | Runs that did not complete. These are refunded |
| samples_refused | integer | Runs where the model declined the prompt. Not refunded — the run itself completed. A high count means the platform is declining the prompt, not that the business is invisible |
| samples_unreadable | integer | Answered runs whose prose no shortlist could be read from |
| samples_shortlist_read | integer | Answered runs the shortlist reader could read. Every shortlist_rate is out of this |
| credits_charged | integer | What the job actually cost: the reservation less the refund for runs that never completed |
| credits_reserved | integer | What was held at submit |
| credits_refunded | integer | What came back for undelivered runs |
| reader_version | integer | Which version of the prose reader produced the shortlist. It is our reading of the answer, not data the platform returned — confirm two jobs share a version before comparing them, because across a bump a reader improvement reads as a drop |
| businesses[].name | string | Business name, as the last run that named it wrote it |
| businesses[].matched_on | string | "cid", "domain" or "name" — how runs were tied together into one business |
| businesses[].name_variants | string[] | Every spelling the answers used, when they used more than one. The record of which rows were folded together |
| businesses[].cited_domains | array | Which source the answers cited for this business and in how many runs, most-used first. A business sourced from its BBB listing rather than its own site is telling you which page the model treats as authoritative |
| businesses[].cid | string | Google's identifier in decimal, on google_ai_mode only. Empty elsewhere |
| businesses[].appearances | integer | How many runs named this business |
| businesses[].appearance_rate | float | appearances / samples_returned, to 3 places |
| businesses[].shortlist_appearances | integer | Runs whose prose put this business on its shortlist |
| businesses[].shortlist_rate | float | shortlist_appearances / samples_shortlist_read. This is the answer to "how often does an AI recommend us?" |
| businesses[].mention_appearances | integer | Runs that named the business in a sentence rather than listing it. Those runs carry no shortlist_rank |
| businesses[].listing_appearances | integer | Runs where the platform emitted a structured card for it. Always 0 on gemini, which sends no listing |
| businesses[].top_pick_count | integer | Runs where the answer singled it out as its top pick |
| businesses[].best_shortlist_rank | integer | null | Highest shortlist position it reached. Null if the prose never listed it |
| businesses[].mean_shortlist_rank | float | null | Average shortlist position across the runs that listed it |
| businesses[].best_rank | integer | Highest position it reached in any run |
| businesses[].worst_rank | integer | Lowest position it reached in any run |
| businesses[].mean_rank | float | Average position across the runs that named it. Absent runs are not counted |
| businesses[].runs_seen | array | Which run numbers named it |
| runs[].run | integer | 1-based run number |
| runs[].ok | boolean | Whether that run came back |
| runs[].outcome | string | "answered", "refused" or "error" — a refusal is not an absence, and is counted separately from a run that did not complete |
| runs[].shortlist_read | boolean | Whether a shortlist could be read from that run's prose |
| runs[].recommended_businesses | array | The shortlist read out of that run's own prose, in the order the answer argued for it. A different ranking from businesses, and on gemini the only one there is |
| runs[].recommended_businesses[].shortlist_rank | integer | null | Place in the answer's own list, from 1. Null when the answer named the business outside its list |
| runs[].recommended_businesses[].listed | boolean | Whether this came from a list entry rather than a sentence |
| runs[].recommended_businesses[].mention | string | The answer's own words where it introduced the business |
| runs[].recommended_businesses[].is_top_pick | boolean | Whether the answer singled this one out |
| runs[].recommended_businesses[].citation | object | The url and domain the answer cited for it |
| runs[].businesses | array | Businesses named in that run, in the same shape as /v1/serp/local-pack |
| runs[].response_text | string | The answer text for that run |
| runs[].error | string | The error message, on failed runs only |
Notes
- TWO DIFFERENT RANKINGS, do not merge them. businesses is the platform's LISTING, in the order it emitted cards. recommended_businesses is the SHORTLIST read out of the answer's own prose, in the order the answer argued for it — that is the recommendation, and the two disagree routinely. Report shortlist_rank when a client asks where an AI recommends them.
- GEMINI HAS NO LISTING AT ALL. It sends no structured business block, so businesses is empty in every gemini run and every gemini figure comes from the prose shortlist. An empty businesses array on gemini is not evidence that nobody was named.
- reader_version says which version of the prose reader produced the shortlist. It is our reading of the answer, not data the platform returned, so before comparing this job against an older stored one confirm the versions match — across a bump the numbers are not comparable and a reader improvement reads as a drop.
- A business the answers spelled more than one way is reported as ONE row, with every spelling in name_variants. Only a legal form (LLC, Inc), a bare connector, or a tradeless corporate suffix merges; any word naming a different trade leaves the rows separate. Two rows can therefore still be one business — where one name is the other plus a trade word and both carry the same own-site domain in cited_domains, add the two rates. We leave that seam visible rather than guess, because a chain's branches share a domain the same way and a wrong merge cannot be undone from the payload.
- A refusal is not an absence. samples_refused counts runs where the model declined the prompt; those are excluded from every rate, and they are not refunded because the run itself completed.
- appearance_rate is out of samples_returned, not samples_requested. A run that did not complete is a measurement that was not taken, so it is refunded rather than counted as an absence — samples_failed says how many there were.
- mean_rank averages only the runs that named the business. Named 2nd in 3 of 10 runs reads mean_rank 2 and appearance_rate 0.3; the two say different things and which one is bad is the actionable part.
- matched_on says how runs were tied together: "cid" is Google's own identifier and is exact, "domain" is domain plus name (never domain alone — two branches of a chain share one), "name" is the string alone and is the weakest.
- On google_ai_mode prefer the cid. Google labels cards editorially, so the same business can be named differently between runs and a name-keyed count would report it twice at half the rate each.
- Results are never cached. Serving a sample from cache would answer a later question with an earlier answer, which is the failure this endpoint exists to avoid.