AI ability, measured by the same yardstick

RANKING

Cumulative rankings

These figures are added up across days. It includes 13 runs and 856 trials. Sorted by accuracy, highest first. “—” means that metric has not been measured yet. It does not mean 0%.

The real-source rate is measured only in runs that included a task asking for URLs along with the answer. For Claude, “—” means we have not yet sent that type of task. It does not mean 0%.

You can recount the figures yourself. You can download the CSV (1 row per trial) from the cumulative JSON and from each run's page. The figures in this table match a recount of that file.

METRICS

What we count

We count the same 5 metrics every time. We never add or drop metrics from run to run. The 2 marked ones are metrics that other benchmarks rarely publish.

• Accuracy Number of correct trials divided by the number of valid trials in that run
• Time to answer Seconds from sending until the answer stops. We report the median, minimum and maximum
• ConsistencyUnique metric Share of tasks whose conclusion matched when the same task was sent 3 times
• Real-source rateUnique metric Share of URLs cited in the answers that actually opened
• Instruction compliance Share of trials that followed the rules on character count, format and number of items

The consistency rate puts a number on the fact that AI can change its answer even to the same question. The real-source rate is the share of URLs cited by the AI that we actually opened and confirmed, so made-up sources show up directly in the number.

BY FIELD

Top AI by field

The AI with the highest accuracy for each type of work. When the field changes, so does the ranking.

• Meeting minutes and summaries ChatGPT Accuracy: 91.7% / 2 runs
• Reading images and photos Claude Accuracy: 83.7% / 3 runs
• Reading PDFs and financial statements ChatGPT Accuracy: 100% / 1 run
• Research and fact-checking ChatGPT Accuracy: 100% / 1 run
• English text and translation ChatGPT Accuracy: 83.3% / 1 run
• Spreadsheets and aggregation ChatGPT Accuracy: 100% / 1 run
• Document outlining ChatGPT Accuracy: 83.3% / 1 run
• Excel and spreadsheet formulas ChatGPT Accuracy: 100% / 1 run
• Email and business documents ChatGPT Accuracy: 100% / 1 run
• Data cleanup and name matching ChatGPT Accuracy: 83.3% / 1 run

RUNS

List of runs

In each run, we send 6 tasks to the 4 AIs, 3 times each. Each headline is a one-line summary generated automatically from the results.

• Reading images and photos Reading numbers from images of receipts, charts, and tables (4 AIs, 6 tasks, 3 runs each) We gave 6 tasks to AIs, 3 times each; Claude had the highest accuracy at 100.0% (18/18 trials), with ChatGPT second at 72.2% November 3, 2026 / 72 valid trials

METHOD

How we measure

We describe the steps as they are, so the measurement can be repeated. We do not change the conditions partway through.

What we do in each run

1. We choose the day's field. We leave 7 full days between runs in the same field and never reuse a task.
2. We regenerate the task materials using that day's date as the seed. The data are for a fictional company.
3. We create 6 tasks. Two questions are easy, two medium and two hard. The correct answers are set at this point.
4. We send the same task prompt and the same materials to the 4 AIs. We send each task 3 times, each time in a new chat.
5. We rotate the sending order in each run. This keeps the time of day from skewing the ranking.
6. We record the seconds from sending until the answer stops. The default wait time is 300 seconds.
7. We record the model name shown on screen and the time we checked it. Because AI models change without notice.
8. Scoring is done only by a program. It decides based on whether numbers match, whether required words are present, whether the format fits, whether URLs open and the result of running formulas.
9. We write out, row by row in a CSV, which trials were right or wrong and why, and publish it as is.

How we score

When a task is created, we have already decided which scoring type it uses. Matters of taste are not scored. Tasks that cannot be scored by machine are not in the task catalog in the first place.

exact_number

Whether the number matches the correct answer. The rounding tolerance is set in advance

set_match

How many of the required items were missed. Counts both omissions and made-up additions

contains_all

Whether all the required words and phrases are included

format_check

Whether the answer followed the rules on character count, format and number of items

url_alive

Opens each cited URL to check that it really exists

formula_run

Runs the formula from the answer to check the result

consistency

Whether the conclusion matches when the same task is sent 3 times

How we count trials with no answer

These rules are set in advance. This way we cannot change them later to suit ourselves.

• Refused or timed out

  We keep it in the base and count it as wrong. Because at work, that is the same as getting no answer.

• Hit the plan's usage limit

  We remove it from the base. We treat that trial as not measured.

• A human verification screen appeared

  We stop there. We neither click through it automatically nor try to get around it. We skip that trial.

• Operating error on AInformation's side

  We remove it from the base. We do not report it as the AI's fault.

LIMITS

Limits of this benchmark

So that no one judges by the numbers alone, we set out first what we cannot measure.

• 1 run every 5 days

  Results vary from day to day, even on the same task. Please don't rank the AIs on a single run.

• We use each AI's paid plan

  These are not results from free versions. Plan names are shown on each run's page.

• We use the browser interface

  These are not API results. The same model can behave differently in the web interface and through the API.

• Each run has 6 tasks

  They do not cover all the work in that field.

• We measure only 5 metrics

  We do not measure writing style, readability, or behavior in long conversations.

DATA

Published data

Machines can read them as is. You may modify and redistribute them. The license is CC BY 4.0.

• Cumulative rankings (JSON) Contains this table, the by-field results and the history as they are
• List of runs (JSON) Contains the URLs of each run's JSON and CSV

To download each run's CSV (1 row per trial), go from List of runs to that run's page.

FAQ

FAQ

How do you measure?

We send exactly the same task prompt and the same materials to the 4 AIs on the same day. We open a new chat every time so earlier answers don't carry over. The sending order is rotated in each run. We also record the seconds from sending until the answer stops.

Why do you send the same task 3 times?

Because AI answers vary even for the same question. From a single try, you can't tell whether a result reflects real ability or chance. We send each task 3 times, and the share of tasks whose conclusion matched is reported as the consistency rate.

Who does the scoring?

Scoring is done only by a program. We do not let AI score the answers. The correct answers are set when the tasks are created, and whether an answer is right is decided by whether numbers match, whether required words are present, whether the format fits, whether URLs actually open and the result of running formulas. Matters of taste are not scored.

Is the data in the tasks from real companies?

No. We create new data for a fictional company every time. We do not use real companies, real people or text from other companies. We share the materials we used as they are, so you can repeat the same test.

How do you count cases where the AI couldn't answer?

Refusals and timeouts stay in the base and are counted as wrong. If we hit the plan's usage limit or a human verification screen appears, we remove that trial from the base and treat it as not measured. Operating errors on AInformation's side are also removed from the base, and we do not report them as the AI's fault.

Do you use the free or paid versions?

We use each AI's paid plan through the browser interface. We don't use the API. For each run, we record the plan name, the model name shown on screen and the time we checked it.

Does this ranking show which AI is the smartest?

No. We measure only 5 things on a fixed set of work tasks: accuracy, time to answer, consistency rate, real-source rate and instruction compliance. We do not measure writing style, readability or behavior in long conversations. If the tasks change, the ranking changes too.

Can I cite these results?

Yes. The license is CC BY 4.0. As the source, please write “Source: AInformation, AI Practical Work Benchmark” with the URL of the page you used and the measurement date. The results are also available as JSON and CSV. With the CSV (1 row per trial), you can recount the figures we publish.

CITATION

Cite this benchmark

Source: AInformation, “AI Practical Work Benchmark” (https://ainformation.jp/en/ai-surveys/ai-work-benchmark), as of November 2, 2026, cumulative: 10 fields, 13 runs, 856 trials

Copy citation

When you quote a figure, please include the measurement date and the base (how many of how many trials). Cumulative figures grow with every run, so noting which version you cited makes it easier to check later.

Related pages

• Original AI SurveysResults from asking people directly in questionnaires
• AI Model BenchmarkMeasured by sending the same questions to each API