# Data provenance > How GPU Ninja's data is researched, verified and reviewed, and how site owners can identify or block the research bot, GPUNinjaResearchBot. URL: https://gpuninja.com/docs/data-provenance Last updated: 2026-10-05 ## In short GPU Ninja's data changes only through reviewed pull requests to its repository. A daily automated research run proposes updates, every proposed value is checked against the page it cites, and a maintainer reviews and merges the result. Maintainers also make changes by hand, through the same review and the same checks. No path leads from research to the live site without a person agreeing to it. What the published data means (collections, frozen records, provenance fields, prices) is on Data (https://gpuninja.com/docs/data). ## The daily research run Once a day, at 05:17 UTC, an automated job looks for new hardware, changed specifications, and the current price and stock of graphics card offers. - What it covers: chips, boards (graphics cards) and the offers on boards. It can propose a new chip, board or offer, a changed price, stock state or specification, and that a chip or board has been discontinued. Benchmarks, chassis, AI models, platforms, and offers on chassis and platforms are maintained by hand. - What it never touches: discontinued records. They are frozen snapshots, so they are excluded from research and from every proposal. It also never deletes a record and never changes a slug. - How it searches: it asks an OpenAI model with web search, in a few small batches, each limited to the records it was asked about. Every claim the model returns must cite a URL the model actually consulted; a claim citing anything else is thrown away. Which model is used is still awaiting the maintainers' approval. - Merchants and sources: prices are only accepted from merchants in a committed registry of merchants and their domains, which also lists manufacturer domains and blocked domains. Blocked domains are left out of the web search, and a claim citing one is thrown away. A price must be in the merchant's own currency, and a new offer's link must be on the merchant's own site. - Cost caps: each run and each month have a fixed spending limit in code, counting every attempt of every run. A run stops starting new work before it could exceed its limit, and no run starts once the month's limit is spent. ## Verification GPU Ninja doesn't take a model's word for anything. Each claim first passes a set of cheap checks that need no request: the cited URL must be a plain `https` address the model consulted, from a domain that isn't blocked, about a record the run is allowed to change, with a value in the right form and range. A claim that fails any of them is discarded without fetching anything. For every claim that passes, the research run fetches the cited page itself and looks for the value: 1. If the product's name isn't on the page, the claim is marked unverified, whatever else the page says. 2. If the quoted passage is on the page and contains the value, the claim is verified. 3. If the passage is there but shows a different value, the claim is contradicted and discarded. 4. If the passage isn't found, structured product data on the page (schema.org offers whose product name matches) can verify or contradict a price or stock claim. 5. If the page can't be fetched, or doesn't show the value either way, the claim is kept but marked unverified and flagged for the reviewer. The match checks what a number means, not just that it appears. A price after words such as "was", "list price", "MSRP" or "you save" doesn't count, struck-through text is ignored, and a currency symbol next to the price must match the claimed currency. A quote that reads as both in stock and out of stock is marked unverified. Verification has its own time limit and a cap on the number of pages it fetches; a claim whose page isn't reached in time is marked unverified too. When several claims disagree about one value, a verified claim beats an unverified one. Then the manufacturer's own page wins for chip and board specs, and the merchant's own page wins for offers. Otherwise the most recent retrieval wins. Either way the conflict is shown to the reviewer with both sources. Every value that survives carries per-field provenance: its source URL, when it was retrieved, when it was last verified, the run that produced it, and its verification status. Those are published with the data. ## Review The run's output is a single pull request per UTC date, opened by a bot account, with: - the changes to the database dump; - a change record, a structured list of every inserted, updated and deleted value, derived from the dump diff so it can't disagree with the data; - a readable summary, such as "RTX 5090 street price 2099 → 1949", with the source of each change, flagged conflicts and unverified values, and the run's estimated cost. Before the pull request is opened, and again on it as a required check, GPU Ninja's validation check tests the data: formats, plausible ranges, references between records, provenance on every value, the frozen-record rules, slug renames, and that the change record matches the dump. If a proposal fails the check, the run drops it, along with anything that depends on it, and tries again, so one odd value never reaches the pull request. The pull request also builds the whole site as a dry run, so a change that would break a page or the API fails before it is merged. Safeguards keep the daily review readable: - No change, no pull request. A quiet day produces nothing, so maintainers keep reading the ones that do arrive. - A circuit breaker. If a run would change more than a quarter of the active records it covers (active chips, active boards and their offers), it opens no pull request at all and alerts the maintainers instead. - No auto-merge. A maintainer reads and merges every change, and can drop any individual proposal before merging. The bot can't approve its own pull request. - Nothing is skipped silently. If an earlier research pull request is still waiting for review, the run skips rather than piling up a second one. A skip that lasts three runs in a row alerts the maintainers. - Repeat offenders are surfaced. A proposal discarded in three consecutive runs is listed separately, so a maintainer decides about it explicitly. ## Manual changes and corrections Maintainers change the data by hand through the same kind of pull request, with the same change record and the same required checks. A hand edit to an existing value is treated as a correction unless the maintainer marks it as a real-world change. Discontinued records are frozen: changing one needs a correction file in the same pull request, naming the record, each field's old and new value, a reason and, where there is one, a source. The validation check rejects any change to a frozen record without a matching correction. Correction files are kept permanently, so every change to history stays explained. Research never rewrites history either. If a wrong value is merged, a maintainer fixes it with a new pull request, which is reviewed and recorded like any other change. ## From merge to the site A merge starts a deploy, which rebuilds every page, `.txt` file and API document from the database and publishes them together. The merged change record then appears in the changes feed (https://gpuninja.com/docs/api/changes), and the next daily digest (https://gpuninja.com/docs/digest) summarises it for readers. To report a value that is wrong, see Reporting an error (https://gpuninja.com/docs/data#reporting-an-error). ## The research bot When the research run fetches a cited page to check it, it identifies itself with this user agent: ``` GPUNinjaResearchBot/1.0 (+https://gpuninja.com/docs/data-provenance) ``` How it behaves: - It fetches only pages that a research claim cited, each at most once per run, with `GET` requests. - It checks `robots.txt` before every request to a host, including each host a redirect leads to, and obeys the rules for its token, `GPUNinjaResearchBot`. It fetches each host's `robots.txt` at most once per run and reads only its first 512,000 bytes. If your `robots.txt` returns a server error, times out or can't be reached, it treats your whole site as disallowed. If it returns a `4xx`, it treats everything as allowed. - It is gentle: at most one request to your host at a time, at least one second between request starts to the same host, at most 100 pages from one host per run, a 15-second timeout for the whole response, at most 3 MiB per page and at most five redirects. - It uses `https` only, on the standard port, with certificate checks always on. A redirect to `http` ends the fetch. - It only reads HTML (`text/html` or `application/xhtml+xml`). Pages are parsed as text, scripts are never run, and whole pages are never stored, logged or published. The only page text kept is the quote of at most 300 characters that a claim cited, which is shown to the maintainers reviewing the change as evidence and is never added to the data. - It refuses to connect to private, loopback and other internal network addresses, on every redirect. ### Opting out To stop GPU Ninja's research bot fetching your site, add this to your `robots.txt`: ``` User-agent: GPUNinjaResearchBot Disallow: / ``` To block only part of your site, disallow those paths instead. Allow and disallow rules follow the usual `robots.txt` matching, where the longest matching rule wins. The bot reads your `robots.txt` afresh on each daily run, so a change takes effect from the next run. If you opt out, claims citing your pages can no longer be verified by fetching them. They are marked unverified and flagged for a maintainer instead, and a merchant whose site can never be checked may be removed from the registry. > Note: > This user agent covers only GPU Ninja's own verification fetches. Finding pages in the first place is done by OpenAI's web search, not by this bot, so the `GPUNinjaResearchBot` token doesn't govern that step. For anything else about the bot, contact us through Support (https://gpuninja.com/docs/support).