{"meta":{"name":"Feature-by-feature comparison of 4 generative AIs (source: AInformation)","description":"Data on accuracy by feature, correctness on each task, the top-ranked AI, the test date and the source article, from giving ChatGPT, Gemini, Claude and Copilot the same tasks worded the same way. AInformation gave the same tasks, worded the same way, to the paid plans of 4 AIs. Rows from the AI Practical Work Benchmark show accuracy from 6 tasks per field, each given 3 times and scored only by a program. Rows from the 4-AI testing lab show the results of 3 tasks (easy, medium and hard), each given once, with every answer checked one by one in the article. AIs we could not measure are left out of the denominator, and refusals and timeouts are counted as incorrect. The figures are copied exactly as they appear in the articles and the published data.","language":"en","publisher":"AInformation","source_url":"https://ainformation.jp/en/ai-comparison","methodology_url":"https://ainformation.jp/en/ai-comparison#ai-about","original_url":"https://ainformation.jp/data/ai-compare.json","legacy_url":"https://ainformation.jp/wp-content/themes/swell_child/assets/aicompare/aicompare.json","canonical_url":"https://ainformation.jp/en/data/ai-compare.json","license":"CC BY 4.0","license_url":"https://creativecommons.org/licenses/by/4.0/","attribution":"Source: AInformation, “Generative AI Comparison” (https://ainformation.jp/en/ai-comparison), as of October 29, 2026","updated_at":"2026-10-29","count":21,"unit":" features","note":"The contents are in data, in the same format as the original Japanese JSON (original_url), with only the text translated. Numbers, prices, dates and URLs are the same as in the Japanese version."},"data":{"meta":{"count":21,"updated_at":"2026-10-29","generated_at":"2026-10-29T13:54:07+09:00","trials_total":807,"trials_bench":675,"trials_lab":132,"articles":19,"count_lab":11,"count_bench":10,"ais":["ChatGPT","Gemini","Claude","Copilot"],"by":"AInformation","license":"CC BY 4.0","description":"This is a scoring table by feature, built by giving ChatGPT, Gemini, Claude, and Copilot the same tasks in the same wording and comparing the results. It combines the AI Practical Work Benchmark (scored by program) and the 4AI Verification Lab (measured article by article), each with the date it was tested."},"uses":{"write":"Writing, translation, and documents","image":"Image","voice":"Voice","file":"Files and spreadsheets","search":"Research","code":"Code & automation","other":"Other"},"items":[{"key":"bench-gazou","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Reading images and photos","what":"Reading numbers from images of receipts, graphs, and tables","use":"image","date":"2026-10-24","winner":"ChatGPT","tie":[],"reason":"ChatGPT had the highest accuracy at 77.8%, followed by Gemini in second place at 66.7%","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"77.8%","s":"14/18 times","na":false,"win":true},"Gemini":{"v":"66.7%","s":"12 of 18","na":false,"win":false},"Claude":{"v":"55.6%","s":"10/18 times","na":false,"win":false},"Copilot":{"v":"66.7%","s":"12 of 18","na":false,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"77.8% (14/18 times)","Gemini":"66.7% (12 of 18)","Claude":"55.6% (10/18 times)","Copilot":"66.7% (12 of 18)"},"best":["ChatGPT"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"36 seconds","Gemini":"183.7 seconds","Claude":"13.9 seconds","Copilot":"14.2 seconds"},"best":["Claude"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"88.9%","Copilot":"88.9%"},"best":["ChatGPT","Gemini"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"16.7%","Copilot":"100%"},"best":["ChatGPT","Gemini","Copilot"]},{"item":"Timed out (300 seconds)","vals":{"ChatGPT":"1 time","Gemini":"3 times","Claude":"0 times","Copilot":"1 time"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":16180,"url":"https://ainformation.jp/article/16180","title":"Comparing 4 image-reading AIs: 3 prompts for reading numbers and how to choose"},"data_url":"https://ainformation.jp/research/bench/gazou-2026-10-24","demand":4348,"order":0},{"key":"lab-image_gen","src":"lab","src_label":"4-AI Test Lab","feature":"Image generation","what":"Gave the same 3 tasks (apple, banner, 4-panel comic; easy, medium, hard) to each AI in the same wording","use":"image","date":"2026-09-27","winner":"Gemini","tie":[],"reason":"Returned an image matching the instructions for all 3 tasks (time could not be measured; ChatGPT could not be tested)","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"Could not test","s":"","na":true,"win":false},"Gemini":{"v":"○3","s":"3 tasks","na":false,"win":true},"Claude":{"v":"○1 △2","s":"3 tasks","na":false,"win":false},"Copilot":{"v":"○1 △2","s":"3 tasks","na":false,"win":false}},"rows":[{"item":"Time taken (easy, apple), seconds","vals":{"ChatGPT":"× (could not test)","Gemini":"Could not measure","Claude":"60.7 seconds","Copilot":"45.2 seconds"},"best":["Copilot"]},{"item":"Time taken (medium, banner), seconds","vals":{"ChatGPT":"× (could not test)","Gemini":"Could not measure","Claude":"45.0 seconds","Copilot":"57.6 seconds"},"best":["Claude"]},{"item":"Time taken (hard, 4-panel comic), seconds","vals":{"ChatGPT":"× (could not test)","Gemini":"Could not measure","Claude":"126.2 seconds","Copilot":"Could not measure"},"best":[]},{"item":"Answer length in characters (easy, medium, hard)","vals":{"ChatGPT":"× (could not test)","Gemini":"Could not measure","Claude":"398, 498, and 514 characters","Copilot":"7 characters, 6 characters, could not measure"},"best":[]},{"item":"Correctness (apple)","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Met the conditions","Claude":"△ Looked like an illustration, not a photo","Copilot":"△ The bottom of the background was gray"},"best":["Gemini"]},{"item":"Correctness (banner)","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Text was also correct","Claude":"○ 1920×1080 with the right font","Copilot":"○ Had text that was not requested"},"best":[]},{"item":"Correctness (4-panel comic)","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Included even the panel numbers and scene names","Claude":"△ The character in panel 4 was not looking outside","Copilot":"△ The panel number was cut off"},"best":["Gemini"]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Scene names in the image were also correct","Claude":"○ Even the instructions were in everyday language","Copilot":"○ Just one sentence"},"best":[]},{"item":"Limitations","vals":{"ChatGPT":"× (could not test)","Gemini":"No display of remaining uses","Claude":"Does not create photos or illustrations (official)","Copilot":"Creating images requires an eligible Microsoft 365 plan (official)"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":5,"models":{"Claude":"Opus 5.5 medium"},"article":{"id":11932,"url":"https://ainformation.jp/article/11932","title":"Comparing 3 image-generation AIs: Gemini was best at photorealistic images"},"data_url":null,"demand":1960,"order":1},{"key":"lab-image_input","src":"lab","src_label":"4-AI Test Lab","feature":"Sending a photo or image and asking about it (image recognition)","what":"Gave the same three tasks (receipt, bar chart, table — easy, medium, hard) to each tool","use":"image","date":"2026-09-25","winner":"Gemini","tie":[],"reason":"Claude was fastest on every task, but Gemini got all three right","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"Could not test","s":"","na":true,"win":false},"Gemini":{"v":"○3","s":"3 tasks","na":false,"win":true},"Claude":{"v":"○2 △1","s":"3 tasks","na":false,"win":false},"Copilot":{"v":"Could not test","s":"","na":true,"win":false}},"rows":[{"item":"Time taken (receipt, easy)","vals":{"ChatGPT":"× (could not test)","Gemini":"26.3 seconds","Claude":"17.3 seconds","Copilot":"× (could not test)"},"best":["Claude"]},{"item":"Time taken (bar chart, medium)","vals":{"ChatGPT":"× (could not test)","Gemini":"31.8 seconds","Claude":"13.9 seconds","Copilot":"× (could not test)"},"best":["Claude"]},{"item":"Time taken (table, hard)","vals":{"ChatGPT":"× (could not test)","Gemini":"25.7 seconds","Claude":"15.3 seconds","Copilot":"× (could not test)"},"best":["Claude"]},{"item":"Accuracy (receipt)","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Correct","Claude":"○ Correct","Copilot":"× (could not test)"},"best":[]},{"item":"Accuracy (bar chart)","vals":{"ChatGPT":"× (could not test)","Gemini":"○ All 5 values correct","Claude":"△ Returned 6 values","Copilot":"× (could not test)"},"best":["Gemini"]},{"item":"Accuracy (table)","vals":{"ChatGPT":"× (could not test)","Gemini":"○ All 20 cells matched","Claude":"○ All 20 cells matched","Copilot":"× (could not test)"},"best":[]},{"item":"Character count (receipt)","vals":{"ChatGPT":"× (could not test)","Gemini":"80 characters","Claude":"195 characters","Copilot":"× (could not test)"},"best":[]},{"item":"Character count (bar chart)","vals":{"ChatGPT":"× (could not test)","Gemini":"45 characters","Claude":"57 characters","Copilot":"× (could not test)"},"best":[]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Natural","Claude":"○ Natural","Copilot":"× (could not test)"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":6,"models":{"Gemini":"Pro, extended","Claude":"Opus 5.5 medium"},"article":{"id":11544,"url":"https://ainformation.jp/article/11544","title":"Comparing image-recognition AI: differences between Gemini and Claude, and how to choose"},"data_url":null,"demand":1434,"order":2},{"key":"bench-gijiroku","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Meeting minutes and summaries","what":"Pulls decisions and action items from meeting minutes","use":"write","date":"2026-10-29","winner":"ChatGPT","tie":[],"reason":"ChatGPT had the highest accuracy at 83.3%, followed by Gemini in second place at 55.6%","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"83.3%","s":"15 of 18 times","na":false,"win":true},"Gemini":{"v":"55.6%","s":"10/18 times","na":false,"win":false},"Claude":{"v":"55.6%","s":"10/18 times","na":false,"win":false},"Copilot":{"v":"55.6%","s":"10/18 times","na":false,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"83.3% (15 of 18 times)","Gemini":"55.6% (10/18 times)","Claude":"55.6% (10/18 times)","Copilot":"55.6% (10/18 times)"},"best":["ChatGPT"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"25.1 seconds","Gemini":"19.1 seconds","Claude":"16.9 seconds","Copilot":"18.6 seconds"},"best":["Claude"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"83.3%","Claude":"88.9%","Copilot":"66.7%"},"best":["ChatGPT"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"83.3%","Gemini":"83.3%","Claude":"22.2%","Copilot":"56.2%"},"best":["ChatGPT","Gemini"]},{"item":"Timed out (300 seconds)","vals":{"ChatGPT":"0 times","Gemini":"0 times","Claude":"0 times","Copilot":"2 times"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash-Lite","Claude":"Opus 5.5, medium","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash-Lite","Claude":"Opus 5.5, medium"},"article":{"id":16680,"url":"https://ainformation.jp/article/16680","title":"Comparing the accuracy of 4 meeting-minutes summary AIs: 3 ready-to-use prompts"},"data_url":"https://ainformation.jp/research/bench/gijiroku-2026-10-29","demand":714,"order":3},{"key":"lab-transcript_minutes","src":"lab","src_label":"4-AI Test Lab","feature":"Creating meeting minutes from audio","what":"Gave the same three tasks (transcription, decisions made, and minutes format — easy, medium, hard) to each tool","use":"voice","date":"2026-09-29","winner":"Gemini","tie":[],"reason":"From a recording, it pulled together 4 decisions, 2 action items, and the next meeting in 28.8 seconds (it got one product name wrong; ChatGPT and Copilot could not be tested)","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"Could not test","s":"","na":true,"win":false},"Gemini":{"v":"○1","s":"1 task","na":false,"win":true},"Claude":{"v":"×1","s":"1 task","na":false,"win":false},"Copilot":{"v":"Could not test","s":"","na":true,"win":false}},"rows":[{"item":"Time taken (easy, transcription)","vals":{"ChatGPT":"× (could not test)","Gemini":"25.5 seconds (declined)","Claude":"144.5 seconds (could not run)","Copilot":"× (could not test)"},"best":[]},{"item":"Time taken (medium, decisions made)","vals":{"ChatGPT":"× (could not test)","Gemini":"28.8 seconds","Claude":"42 seconds (could not read it)","Copilot":"× (could not test)"},"best":["Gemini"]},{"item":"Time taken (hard, minutes format)","vals":{"ChatGPT":"× (could not test)","Gemini":"28.8 seconds","Claude":"Still running at 363.1 seconds","Copilot":"× (could not test)"},"best":["Gemini"]},{"item":"Answer length in characters (easy, medium, hard)","vals":{"ChatGPT":"× (could not test)","Gemini":"27 characters, 367 characters, 480 characters","Claude":"964 characters, 469 characters, 202 characters","Copilot":"× (could not test)"},"best":[]},{"item":"Accuracy (4 decisions, 2 action items, next meeting)","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Got everything (wrote Product E as Product B)","Claude":"× Could not return the content","Copilot":"× (could not test)"},"best":["Gemini"]},{"item":"Full transcription","vals":{"ChatGPT":"× (could not test)","Gemini":"× Declined","Claude":"× Could not run","Copilot":"× (could not test)"},"best":[]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"× (could not test)","Gemini":"○ Short, like real minutes","Claude":"○ Polite, but talks a lot about commands","Copilot":"× (could not test)"},"best":[]},{"item":"Limits (official help)","vals":{"ChatGPT":"× (could not test)","Gemini":"10 minutes free for audio; 3 hours with AI Pro","Claude":"Audio is not listed among the supported formats","Copilot":"No format listed"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":9,"models":{"Gemini":"Flash extended","Claude":"Opus 4.8, high"},"article":{"id":12327,"url":"https://ainformation.jp/article/12327","title":"How to create meeting minutes with Gemini, and what to do when transcription fails"},"data_url":null,"demand":471,"order":4},{"key":"bench-shirabemono","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Research and fact-checking","what":"Look up a fact that has one definite answer","use":"search","date":"2026-09-29","winner":"ChatGPT","tie":[],"reason":"ChatGPT had the highest accuracy rate at 100%, followed by Copilot in second place at 72.2%. Gemini refused to answer 8 out of 18 times.","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"100%","s":"18 of 18 times","na":false,"win":true},"Gemini":{"v":"22.2%","s":"4 of 18 times","na":false,"win":false},"Claude":{"v":"Could not measure","s":"Stopped for human confirmation","na":true,"win":false},"Copilot":{"v":"72.2%","s":"13 of 18 times","na":false,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"100% (18 of 18 times)","Gemini":"22.2% (4 of 18 times)","Claude":"Could not measure (stopped for human confirmation)","Copilot":"72.2% (13 of 18 times)"},"best":["ChatGPT"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"20.2 seconds","Gemini":"33 seconds","Claude":"—","Copilot":"20.1 seconds"},"best":["Copilot"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"33.3%","Claude":"—","Copilot":"66.7%"},"best":["ChatGPT"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"100%","Gemini":"50%","Claude":"—","Copilot":"31.2%"},"best":["ChatGPT"]},{"item":"Share of source URLs that could be opened","vals":{"ChatGPT":"100%","Gemini":"83.3%","Claude":"—","Copilot":"93.3%"},"best":["ChatGPT"]},{"item":"Number of refusals","vals":{"ChatGPT":"0 times","Gemini":"8 times","Claude":"—","Copilot":"0 times"},"best":[]},{"item":"Timed out (300 seconds)","vals":{"ChatGPT":"0 times","Gemini":"0 times","Claude":"—","Copilot":"2 times"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"Medium","Gemini":"Flash extended","Claude":"—","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":54,"models":{"ChatGPT":"Medium","Gemini":"Flash extended"},"article":null,"data_url":"https://ainformation.jp/research/bench/shirabemono-2026-09-29","demand":300,"order":5},{"key":"bench-kessan","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Reading PDFs and financial statements","what":"Pulls numbers from an earnings PDF","use":"file","date":"2026-09-28","winner":null,"tie":["ChatGPT","Claude"],"reason":"ChatGPT and Claude tied at 100% accuracy","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"100%","s":"15 of 15 times","na":false,"win":true},"Gemini":{"v":"73.3%","s":"11 of 15 times","na":false,"win":false},"Claude":{"v":"100%","s":"15 of 15 times","na":false,"win":true},"Copilot":{"v":"Not measured","s":"","na":true,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"100% (15 of 15 times)","Gemini":"73.3% (11 of 15 times)","Claude":"100% (15 of 15 times)","Copilot":"Not measured"},"best":["ChatGPT","Claude"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"17.2 seconds","Gemini":"28.6 seconds","Claude":"13.9 seconds","Copilot":"—"},"best":["Claude"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"80%","Claude":"100%","Copilot":"—"},"best":["ChatGPT","Claude"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"86.7%","Gemini":"16.7%","Claude":"60%","Copilot":"—"},"best":["ChatGPT"]},{"item":"Number of refusals","vals":{"ChatGPT":"0 times","Gemini":"3 times","Claude":"0 times","Copilot":"—"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"Medium","Gemini":"Flash extended","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":45,"models":{"ChatGPT":"Medium","Gemini":"Flash extended","Claude":"Opus 4.8, high"},"article":null,"data_url":"https://ainformation.jp/research/bench/kessan-2026-09-28","demand":284,"order":6},{"key":"lab-file_reading","src":"lab","src_label":"4-AI Test Lab","feature":"File (PDF) reading","what":"Sent the same three prompts (easy, medium, hard) asking for revenue and operating profit, which expense items increased in SG&A, and a profit margin calculation","use":"file","date":"2026-10-04","winner":"Claude","tie":[],"reason":"It answered all three questions correctly, and answered the two easy and medium questions in 14.0 seconds and 13.9 seconds, the fastest among the four.","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"○3","s":"3 tasks","na":false,"win":false},"Gemini":{"v":"○3","s":"3 tasks","na":false,"win":false},"Claude":{"v":"○3","s":"3 tasks","na":false,"win":true},"Copilot":{"v":"○3","s":"3 tasks","na":false,"win":false}},"rows":[{"item":"Seconds taken (easy: revenue and operating profit)","vals":{"ChatGPT":"17.4 seconds","Gemini":"25.8 seconds","Claude":"14.0 seconds","Copilot":"20.4 seconds"},"best":["Claude"]},{"item":"Seconds taken (medium: expense items that increased in SG&A)","vals":{"ChatGPT":"14.3 seconds","Gemini":"22.4 seconds","Claude":"13.9 seconds","Copilot":"20.2 seconds"},"best":["Claude"]},{"item":"Seconds taken (hard: profit margin calculation)","vals":{"ChatGPT":"17.3 seconds","Gemini":"32.0 seconds","Claude":"20.1 seconds","Copilot":"20.1 seconds"},"best":["ChatGPT"]},{"item":"Character count (easy)","vals":{"ChatGPT":"279 characters","Gemini":"140 characters","Claude":"121 characters","Copilot":"107 characters"},"best":[]},{"item":"Character count (medium)","vals":{"ChatGPT":"454 characters","Gemini":"160 characters","Claude":"248 characters","Copilot":"180 characters"},"best":[]},{"item":"Character count (hard)","vals":{"ChatGPT":"794 characters","Gemini":"329 characters","Claude":"571 characters","Copilot":"543 characters"},"best":[]},{"item":"Accuracy (3 questions)","vals":{"ChatGPT":"○ Got all 3 questions right","Gemini":"○ Got all 3 questions right","Claude":"○ Got all 3 questions right","Copilot":"○ Got all 3 questions right"},"best":[]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"Natural (the lead-in to the summary is a bit stiff)","Gemini":"Natural (memo style, ending lines with “up”)","Claude":"Natural, stated clearly in one sentence (shows an English status message while working on question 3)","Copilot":"Natural (marks decreases with a ▲)"},"best":["Claude"]},{"item":"Restrictions (official conditions)","vals":{"ChatGPT":"80 files in 3 hours; free plan is 3 files a day","Gemini":"10 files per session; up to 100MB except for video","Claude":"500MB per file; 20 files per conversation","Copilot":"Uploads allowed even on the free plan, within size and limit caps"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":12,"models":{"ChatGPT":"Medium","Gemini":"Pro","Claude":"Fable 5.1 Medium"},"article":{"id":12975,"url":"https://ainformation.jp/article/12975","title":"Comparing the best PDF-reading AI tools: 3 prompts for reading financial statements"},"data_url":null,"demand":209,"order":7},{"key":"lab-code_execution","src":"lab","src_label":"4-AI Test Lab","feature":"Code execution (data analysis and charts)","what":"Sent the same prompt for 3 tasks (easy, medium, hard): a bar chart, an annual total table, and a regression line","use":"code","date":"2026-10-22","winner":"Claude","tie":[],"reason":"It got all 3 tasks right and was the fastest among the tools that answered every task correctly","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"○3","s":"3 tasks","na":false,"win":false},"Gemini":{"v":"○2 ×1","s":"3 tasks","na":false,"win":false},"Claude":{"v":"○3","s":"3 tasks","na":false,"win":true},"Copilot":{"v":"○1 △1","s":"2 tasks","na":false,"win":false}},"rows":[{"item":"Time taken (easy: bar chart)","vals":{"ChatGPT":"92.9 seconds","Gemini":"147.3 seconds","Claude":"45.6 seconds","Copilot":"— (could not be compared this time)"},"best":["Claude"]},{"item":"Time taken (medium: annual total table)","vals":{"ChatGPT":"46.1 seconds","Gemini":"68.5 seconds (declined)","Claude":"26.7 seconds","Copilot":"21.0 seconds (1 error)"},"best":["Claude"]},{"item":"Time taken (hard: regression line)","vals":{"ChatGPT":"43.0 seconds","Gemini":"135.3 seconds","Claude":"23.8 seconds","Copilot":"27.6 seconds"},"best":["Claude"]},{"item":"Character count (easy, medium, hard)","vals":{"ChatGPT":"20 characters, 753 characters, 131 characters","Gemini":"358 characters, 26 characters, 750 characters","Claude":"416 characters, 897 characters, 346 characters","Copilot":"—, 628 characters, 654 characters"},"best":[]},{"item":"Accuracy (12 bars on the bar chart)","vals":{"ChatGPT":"○ Matched the CSV","Gemini":"○ Matched the CSV","Claude":"○ Matched the CSV","Copilot":"—"},"best":[]},{"item":"Accuracy (annual total: Product A at 146,400 ranked first)","vals":{"ChatGPT":"○ All 5 products matched","Gemini":"× Replied that it could not handle the task","Claude":"○ All 5 products matched","Copilot":"△ Product C came out as 82,400"},"best":["Claude"]},{"item":"Accuracy (slope 288.8, intercept 9839.4)","vals":{"ChatGPT":"○","Gemini":"○","Claude":"○","Copilot":"○"},"best":[]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"Short, but the chart labels were in English","Gemini":"Thorough, and added extra statistics","Claude":"Short and to the point","Copilot":"Focused on formulas and code"},"best":["Claude"]},{"item":"Limitations","vals":{"ChatGPT":"Not checked this time","Gemini":"10 files per session, 100 MB per file","Claude":"Turned on via settings in Free, Pro, and Max; 30 MB per file","Copilot":"Attachments allowed on the free plan, with a limit"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":12,"models":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":16059,"url":"https://ainformation.jp/article/16059","title":"Comparing 4 AI data analysis tools: 3 prompts for drawing charts"},"data_url":null,"demand":158,"order":8},{"key":"lab-canvas_docs","src":"lab","src_label":"4-AI Test Lab","feature":"Creating and editing documents together (Canvas, Docs, Pages)","what":"We sent the same three prompts (easy, medium, hard) for creating a document, fixing only the second paragraph, and making three slides","use":"write","date":"2026-10-03","winner":"Claude","tie":[],"reason":"Returned results in the requested format for all three tasks, and noted on its own how it counted the second paragraph and where it filled in placeholder content","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"Could not test","s":"","na":true,"win":false},"Gemini":{"v":"Slides only","s":"","na":false,"win":false},"Claude":{"v":"All three tasks","s":"","na":false,"win":true},"Copilot":{"v":"Tasks 2 and 3","s":"","na":false,"win":false}},"rows":[{"item":"Time taken (easy: create a document)","vals":{"ChatGPT":"× (could not test)","Gemini":"255 seconds (no document was created)","Claude":"57 seconds","Copilot":"× (could not be recorded)"},"best":["Claude"]},{"item":"Time taken (medium: fix only the second paragraph)","vals":{"ChatGPT":"× (could not test)","Gemini":"96.4 seconds","Claude":"23.1 seconds","Copilot":"26.4 seconds"},"best":["Claude"]},{"item":"Time taken (hard: three slides)","vals":{"ChatGPT":"× (could not test)","Gemini":"88.4 seconds","Claude":"54.1 seconds","Copilot":"29.7 seconds"},"best":["Copilot"]},{"item":"Character count (easy)","vals":{"ChatGPT":"× (could not test)","Gemini":"27 characters","Claude":"459 characters","Copilot":"× (could not be recorded)"},"best":[]},{"item":"Character count (medium)","vals":{"ChatGPT":"× (could not test)","Gemini":"70 characters","Claude":"289 characters","Copilot":"228 characters"},"best":[]},{"item":"Character count (hard)","vals":{"ChatGPT":"× (could not test)","Gemini":"202 characters","Claude":"426 characters","Copilot":"135 characters"},"best":[]},{"item":"Accuracy (whether the result matched the requested format)","vals":{"ChatGPT":"× (could not test)","Gemini":"Slides only (the document was refused, and the screen showed Slide 1 of 4)","Claude":"All three tasks (the diff for task 2 was not confirmed)","Copilot":"Tasks 2 and 3 (task 1 could not be recorded; the diff was not confirmed)"},"best":["Claude"]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"× (could not test)","Gemini":"Short, natural polite language","Claude":"Quite formal, polite language","Copilot":"Somewhat stiff, with very formal phrasing such as “it is indeed possible”"},"best":[]},{"item":"Restrictions (official conditions)","vals":{"ChatGPT":"Rolled out to paid accounts in stages","Gemini":"Requires logging in to the Gemini app","Claude":"Beta for Pro, Max, Team, and Enterprise; counts toward usage limits","Copilot":"For individuals: Microsoft 365 Personal, Family, Premium, and Pro"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":10,"models":{"Gemini":"Pro","Claude":"Fable 5.1 Medium","ChatGPT":"Medium"},"article":{"id":12876,"url":"https://ainformation.jp/article/12876","title":"Comparing Canvas: 3 prompts for asking AI to create documents and slides"},"data_url":null,"demand":117,"order":9},{"key":"bench-hyokeisan","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Spreadsheets and aggregation","what":"Calculate totals and growth rates from a sales table","use":"file","date":"2026-10-02","winner":null,"tie":["ChatGPT","Copilot"],"reason":"ChatGPT and Copilot tied for the highest accuracy rate at 100%. Gemini refused 6 of 18 times","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"100%","s":"18 of 18 times","na":false,"win":true},"Gemini":{"v":"50%","s":"9 of 18 times","na":false,"win":false},"Claude":{"v":"83.3%","s":"15 of 18 times","na":false,"win":false},"Copilot":{"v":"100%","s":"18 of 18 times","na":false,"win":true}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"100% (18 of 18 times)","Gemini":"50% (9 of 18 times)","Claude":"83.3% (15 of 18 times)","Copilot":"100% (18 of 18 times)"},"best":["ChatGPT","Copilot"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"20.4 seconds","Gemini":"36.1 seconds","Claude":"26.6 seconds","Copilot":"17.1 seconds"},"best":["Copilot"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"83.3%","Claude":"100%","Copilot":"100%"},"best":["ChatGPT","Claude","Copilot"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"100%","Gemini":"91.7%","Claude":"16.7%","Copilot":"88.9%"},"best":["ChatGPT"]},{"item":"Number of refusals","vals":{"ChatGPT":"0 times","Gemini":"6 times","Claude":"0 times","Copilot":"0 times"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"Medium","Gemini":"Flash","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"Medium","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":12628,"url":"https://ainformation.jp/article/12628","title":"Comparing 4 data analysis AI tools: 3 prompts for summarizing sales tables and how to choose one"},"data_url":"https://ainformation.jp/research/bench/hyokeisan-2026-10-02","demand":84,"order":10},{"key":"bench-eibun","src":"bench","src_label":"AI Practical Work Benchmark","feature":"English text and translation","what":"Translate an English press release","use":"write","date":"2026-09-30","winner":"ChatGPT","tie":[],"reason":"ChatGPT had the highest accuracy rate at 83.3%, followed by Copilot at 55.6% in second place. Gemini refused 12 out of 18 times","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"83.3%","s":"15 of 18 times","na":false,"win":true},"Gemini":{"v":"16.7%","s":"3/18 times","na":false,"win":false},"Claude":{"v":"50%","s":"9 of 18 times","na":false,"win":false},"Copilot":{"v":"55.6%","s":"10/18 times","na":false,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"83.3% (15 of 18 times)","Gemini":"16.7% (3/18 times)","Claude":"50% (9 of 18 times)","Copilot":"55.6% (10/18 times)"},"best":["ChatGPT"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"18.7 seconds","Gemini":"39.4 seconds","Claude":"23.1 seconds","Copilot":"17.1 seconds"},"best":["Copilot"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"88.9%","Copilot":"83.3%"},"best":["ChatGPT","Gemini"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"100%","Copilot":"55.6%"},"best":["ChatGPT","Gemini","Claude"]},{"item":"Number of refusals","vals":{"ChatGPT":"0 times","Gemini":"12 times","Claude":"0 times","Copilot":"0 times"},"best":[]},{"item":"Timed out (300 seconds)","vals":{"ChatGPT":"0 times","Gemini":"0 times","Claude":"1 time","Copilot":"0 times"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"Medium","Gemini":"Flash extended","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"Medium","Gemini":"Flash extended","Claude":"Opus 4.8, high"},"article":{"id":12367,"url":"https://ainformation.jp/article/12367","title":"4 AI translation tools compared for accuracy: 2 ready-to-use translation prompts"},"data_url":"https://ainformation.jp/research/bench/eibun-2026-09-30","demand":54,"order":11},{"key":"bench-kossi","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Document outline creation","what":"Create a document outline with a set page count and character count","use":"write","date":"2026-10-03","winner":"ChatGPT","tie":[],"reason":"ChatGPT had the highest accuracy rate at 83.3%, followed by Gemini in second place at 77.8%","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"83.3%","s":"15 of 18 times","na":false,"win":true},"Gemini":{"v":"77.8%","s":"14/18 times","na":false,"win":false},"Claude":{"v":"61.1%","s":"11/18 times","na":false,"win":false},"Copilot":{"v":"27.8%","s":"5/18 times","na":false,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"83.3% (15 of 18 times)","Gemini":"77.8% (14/18 times)","Claude":"61.1% (11/18 times)","Copilot":"27.8% (5/18 times)"},"best":["ChatGPT"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"20.4 seconds","Gemini":"19.3 seconds","Claude":"35.5 seconds","Copilot":"17.1 seconds"},"best":["Copilot"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"88.9%","Copilot":"88.9%"},"best":["ChatGPT","Gemini"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"100%","Copilot":"94.4%"},"best":["ChatGPT","Gemini","Claude"]},{"item":"Timed out (300 seconds)","vals":{"ChatGPT":"0 times","Gemini":"1 time","Claude":"0 times","Copilot":"0 times"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"Medium","Gemini":"Flash-Lite","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"Medium","Gemini":"Flash-Lite","Claude":"Opus 4.8, high"},"article":{"id":12749,"url":"https://ainformation.jp/article/12749","title":"4 document-creation AIs compared: 3 prompts for building an outline"},"data_url":"https://ainformation.jp/research/bench/kossi-2026-10-03","demand":46,"order":12},{"key":"bench-suushiki","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Excel/spreadsheet formulas","what":"Write a working spreadsheet formula","use":"file","date":"2026-10-08","winner":null,"tie":["ChatGPT","Gemini","Claude"],"reason":"ChatGPT, Gemini, and Claude tied with a 100% accuracy rate","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"100%","s":"18 of 18 times","na":false,"win":true},"Gemini":{"v":"100%","s":"18 of 18 times","na":false,"win":true},"Claude":{"v":"100%","s":"18 of 18 times","na":false,"win":true},"Copilot":{"v":"94.4%","s":"17/18 times","na":false,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"100% (18 of 18 times)","Gemini":"100% (18 of 18 times)","Claude":"100% (18 of 18 times)","Copilot":"94.4% (17/18 times)"},"best":["ChatGPT","Gemini","Claude"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"14.2 seconds","Gemini":"19.1 seconds","Claude":"13.9 seconds","Copilot":"14.1 seconds"},"best":["Claude"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"100%","Copilot":"88.9%"},"best":["ChatGPT","Gemini","Claude"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"100%","Gemini":"100%","Claude":"0%","Copilot":"94.4%"},"best":["ChatGPT","Gemini"]},{"item":"Model shown on screen","vals":{"ChatGPT":"Medium","Gemini":"Flash","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"Medium","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":14599,"url":"https://ainformation.jp/article/14599","title":"Comparing 4 Excel function AIs: 3 ready-to-use formula prompts"},"data_url":"https://ainformation.jp/research/bench/suushiki-2026-10-08","demand":43,"order":13},{"key":"lab-long_summary","src":"lab","src_label":"4-AI Test Lab","feature":"Long-text summarization","what":"We sent the same prompt for three tasks (easy, medium, hard): a 3-line summary, a list of decisions, and a table of owners and deadlines","use":"write","date":"2026-10-05","winner":"Gemini","tie":[],"reason":"It met the criteria on all 3 tasks and listed all 9 items in the owners-and-deadlines table in 35.4 seconds (Claude also listed 9 items, but took 95.2 seconds)","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"Pass · 8 of 8 · 7 of 9","s":"","na":false,"win":false},"Gemini":{"v":"Pass · 8 of 8 · 9 of 9","s":"","na":false,"win":true},"Claude":{"v":"Pass · 8 of 8 · 9 of 9","s":"","na":false,"win":false},"Copilot":{"v":"Pass · 8 of 8 · 8 of 9","s":"","na":false,"win":false}},"rows":[{"item":"Time taken (easy: 3-line summary)","vals":{"ChatGPT":"23.5 seconds","Gemini":"28.8 seconds","Claude":"20.5 seconds","Copilot":"21.6 seconds"},"best":["Claude"]},{"item":"Time taken (medium, decisions made)","vals":{"ChatGPT":"37.9 seconds","Gemini":"38.0 seconds","Claude":"48.5 seconds","Copilot":"27.0 seconds"},"best":["Copilot"]},{"item":"Time taken (hard: owners-and-deadlines table)","vals":{"ChatGPT":"26.9 seconds","Gemini":"35.4 seconds","Claude":"95.2 seconds","Copilot":"21.1 seconds"},"best":["Copilot"]},{"item":"Character count (easy)","vals":{"ChatGPT":"345 characters","Gemini":"173 characters","Claude":"353 characters","Copilot":"220 characters"},"best":[]},{"item":"Character count (medium)","vals":{"ChatGPT":"1,726 characters","Gemini":"1,178 characters","Claude":"1,056 characters","Copilot":"1,056 characters"},"best":[]},{"item":"Character count (hard)","vals":{"ChatGPT":"522 characters","Gemini":"822 characters","Claude":"905 characters","Copilot":"459 characters"},"best":[]},{"item":"Accuracy (easy, medium, hard)","vals":{"ChatGPT":"Pass · 8 of 8 · 7 of 9","Gemini":"Pass · 8 of 8 · 9 of 9","Claude":"Pass · 8 of 8 · 9 of 9","Copilot":"Pass · 8 of 8 · 8 of 9"},"best":["Gemini"]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"Natural report-style wording, with a file name tag on each line","Gemini":"Passive sentences throughout, with a TXT tag on each line","Claude":"Memo-style with headings and colons, declines to guess","Copilot":"Natural plain declarative style (da/de aru)"},"best":["Claude"]},{"item":"Limitations","vals":{"ChatGPT":"Did not hit the usage limit","Gemini":"Did not hit the usage limit","Claude":"Did not hit the usage limit (shows the percentage of the weekly limit used)","Copilot":"Did not hit the usage limit"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":12,"models":{"ChatGPT":"Medium","Gemini":"Pro","Claude":"Opus 4.8, high"},"article":{"id":14374,"url":"https://ainformation.jp/article/14374","title":"4 long-text summarization AIs compared: 3 prompts for capturing decisions"},"data_url":null,"demand":37,"order":14},{"key":"lab-web_search","src":"lab","src_label":"4-AI Test Lab","feature":"Web search (answers with sources)","what":"Sent the same three prompts (easy, medium, hard) asking about sunrise and sunset times, announcements from four companies, and the consumer price index","use":"search","date":"2026-10-16","winner":"ChatGPT","tie":[],"reason":"The easy and hard questions were answered correctly with a URL to an official source that showed the value, and for the medium question, 3 of the 4 companies matched their official pages down to the date","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"○2 △1","s":"3 tasks","na":false,"win":true},"Gemini":{"v":"○2 ×1","s":"3 tasks","na":false,"win":false},"Claude":{"v":"○1","s":"1 task","na":false,"win":false},"Copilot":{"v":"○2 △1","s":"3 tasks","na":false,"win":false}},"rows":[{"item":"Seconds taken (easy: sunrise and sunset times)","vals":{"ChatGPT":"48.4 seconds","Gemini":"95.5 seconds","Claude":"26.2 seconds (up to the permission check)","Copilot":"20.2 seconds"},"best":["Copilot"]},{"item":"Seconds taken (medium: announcements from 4 companies)","vals":{"ChatGPT":"82.5 seconds","Gemini":"218.2 seconds (a declining response)","Claude":"41.6 seconds (up to the permission check)","Copilot":"26.2 seconds"},"best":["Copilot"]},{"item":"Seconds taken (hard: consumer price index)","vals":{"ChatGPT":"51.5 seconds","Gemini":"205.6 seconds","Claude":"26.2 seconds","Copilot":"20.2 seconds"},"best":["Copilot"]},{"item":"Character count (easy, medium, hard)","vals":{"ChatGPT":"186 characters, 639 characters, 184 characters","Gemini":"170 characters, 63 characters, 447 characters","Claude":"132 characters, 1,317 characters, 502 characters","Copilot":"244 characters, 846 characters, 327 characters"},"best":[]},{"item":"Accuracy (easy: sunrise at 5:48 a.m. and sunset at 5:05 p.m.)","vals":{"ChatGPT":"○ Included a URL showing the value","Gemini":"○ URL was a gateway page","Claude":"– Stopped at the permission check","Copilot":"○"},"best":["ChatGPT"]},{"item":"Accuracy (medium: 4 companies' announcements within 7 days)","vals":{"ChatGPT":"△ Matched official sources for 3 companies; couldn't find the fourth","Gemini":"× Declined, saying it was still learning","Claude":"– Stopped at the permission check","Copilot":"△ Opened 4 URLs, but for 2 companies the announcements weren't about features"},"best":["ChatGPT"]},{"item":"Accuracy (hard: national overall CPI of 1.9%, for August)","vals":{"ChatGPT":"○","Gemini":"○ Cited a source with a different site's label","Claude":"○","Copilot":"○"},"best":[]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"Short, clear, and natural","Gemini":"The declining response was a stock phrase","Claude":"Only the first line of the price answer was in English","Copilot":"Natural, but repeats the conclusion twice"},"best":["ChatGPT"]},{"item":"Limitations","vals":{"ChatGPT":"Turn on web search from the + menu; no limit shown","Gemini":"No toggle; no limit shown","Claude":"Asks for permission before opening a page","Copilot":"Worked on a free account; no limit shown"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":12,"models":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":15474,"url":"https://ainformation.jp/article/15474","title":"Comparing 4 AI web search tools: 3 prompts for getting answers with sources"},"data_url":null,"demand":16,"order":15},{"key":"lab-memory","src":"lab","src_label":"4-AI Test Lab","feature":"Memory (remembering across conversations)","what":"We gave the same text for 3 tasks (easy, medium, hard): making it remember how to address you, making it remember a writing style, and making it remember 3 facts","use":"other","date":"2026-10-09","winner":"ChatGPT","tie":[],"reason":"It was the only AI that answered the name and all 3 facts correctly in a new conversation, and said it remembered none of the 4 after the memory was deleted","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"○1","s":"1 task","na":false,"win":true},"Gemini":{"v":"×1","s":"1 task","na":false,"win":false},"Claude":{"v":"×1","s":"1 task","na":false,"win":false},"Copilot":{"v":"○1","s":"1 task","na":false,"win":false}},"rows":[{"item":"Seconds taken (easy: remembering how to address you)","vals":{"ChatGPT":"17.5 seconds","Gemini":"19.1 seconds","Claude":"17.2 seconds","Copilot":"17.7 seconds"},"best":["Claude"]},{"item":"Seconds taken (medium: remembering a writing style)","vals":{"ChatGPT":"22.2 seconds","Gemini":"19.5 seconds","Claude":"17.9 seconds","Copilot":"17.3 seconds"},"best":["Copilot"]},{"item":"Seconds taken (hard: remembering 3 facts)","vals":{"ChatGPT":"23.8 seconds","Gemini":"37.4 seconds","Claude":"14.1 seconds","Copilot":"18.3 seconds"},"best":["Claude"]},{"item":"Character count (easy, medium, hard)","vals":{"ChatGPT":"48 characters, 130 characters, 133 characters","Gemini":"82 characters, 72 characters, 128 characters","Claude":"368 characters, 204 characters, 103 characters","Copilot":"74 characters, 76 characters, 83 characters"},"best":[]},{"item":"Accuracy (how to address you and 3 facts, in a new conversation)","vals":{"ChatGPT":"Correct — answered everything (14.5 seconds, 14.4 seconds)","Gemini":"Incorrect — did not remember (28.4 seconds, 19.2 seconds)","Claude":"Incorrect — did not remember (14.3 seconds, 14.0 seconds)","Copilot":"Correct — answered everything (14.1 seconds, 14.0 seconds)"},"best":["Copilot"]},{"item":"Writing style rule (200-character explanation in a new conversation)","vals":{"ChatGPT":"Incorrect — 204 characters, included sentences with many commas","Gemini":"Incorrect — 194 characters, included sentences with many commas","Claude":"Incorrect — 280 characters, included sentences with many commas","Copilot":"Incorrect — 127 characters, included sentences with many commas"},"best":[]},{"item":"Asking again after deleting the memory","vals":{"ChatGPT":"Correct — said it remembered none of the 4 (14.3 seconds)","Gemini":"— No answer even before deletion (31.4 seconds)","Claude":"Incorrect — answered all 4 from a past chat (57.3 seconds)","Copilot":"Incorrect — answered all 4 from the history (23.4 seconds)"},"best":["ChatGPT"]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"Short, direct, one-sentence, and natural","Gemini":"Overly polite, with phrases like “I'm very sorry”","Claude":"Long, with a partial task output appearing in response to the deletion request","Copilot":"Short and natural, in one sentence"},"best":["ChatGPT"]},{"item":"Limitations","vals":{"ChatGPT":"No stated limit; says that fully deleting a memory also requires deleting it in settings and deleting the original conversation","Gemini":"No stated limit; officially, memory of past conversations can be turned on or off in settings","Claude":"No stated limit; officially on by default for Free, Pro, and Max","Copilot":"No stated limit; officially, conversation history remains even after deleting everything"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":32,"models":{"ChatGPT":"Medium","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":14764,"url":"https://ainformation.jp/article/14764","title":"Comparing 4 AI memory features: the risk of past conversations resurfacing after deletion"},"data_url":null,"demand":7,"order":16},{"key":"bench-mail","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Email and business documents","what":"Write an apology email for a delayed delivery","use":"write","date":"2026-10-14","winner":null,"tie":["ChatGPT","Copilot"],"reason":"ChatGPT and Copilot tied with a 100% accuracy rate","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"100%","s":"18 of 18 times","na":false,"win":true},"Gemini":{"v":"38.9%","s":"7/18 times","na":false,"win":false},"Claude":{"v":"83.3%","s":"15 of 18 times","na":false,"win":false},"Copilot":{"v":"100%","s":"18 of 18 times","na":false,"win":true}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"100% (18 of 18 times)","Gemini":"38.9% (7/18 times)","Claude":"83.3% (15 of 18 times)","Copilot":"100% (18 of 18 times)"},"best":["ChatGPT","Copilot"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"17.3 seconds","Gemini":"98.6 seconds","Claude":"18.7 seconds","Copilot":"17.1 seconds"},"best":["Copilot"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"75%","Claude":"88.9%","Copilot":"100%"},"best":["ChatGPT","Copilot"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"100%","Gemini":"90%","Claude":"50%","Copilot":"83.3%"},"best":["ChatGPT"]},{"item":"Number of refusals","vals":{"ChatGPT":"0 times","Gemini":"4 times","Claude":"0 times","Copilot":"0 times"},"best":[]},{"item":"Timed out (300 seconds)","vals":{"ChatGPT":"0 times","Gemini":"4 times","Claude":"0 times","Copilot":"0 times"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"Medium","Gemini":"Flash, high","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"Medium","Gemini":"Flash, high","Claude":"Opus 4.8, high"},"article":{"id":15169,"url":"https://ainformation.jp/article/15169","title":"Comparing 4 recommended AI email writers: 3 apology email prompts"},"data_url":"https://ainformation.jp/research/bench/mail-2026-10-14","demand":3,"order":17},{"key":"bench-meibo","src":"bench","src_label":"AI Practical Work Benchmark","feature":"Data formatting & name matching","what":"Standardize inconsistent spellings of business partner names","use":"file","date":"2026-10-19","winner":"ChatGPT","tie":[],"reason":"ChatGPT had the highest accuracy rate at 83.3%, followed by Gemini in second place at 77.8%","metric":"Accuracy rate (same task run 3 times each)","cells":{"ChatGPT":{"v":"83.3%","s":"15 of 18 times","na":false,"win":true},"Gemini":{"v":"77.8%","s":"14/18 times","na":false,"win":false},"Claude":{"v":"66.7%","s":"12 of 18","na":false,"win":false},"Copilot":{"v":"66.7%","s":"12 of 18","na":false,"win":false}},"rows":[{"item":"Accuracy rate","vals":{"ChatGPT":"83.3% (15 of 18 times)","Gemini":"77.8% (14/18 times)","Claude":"66.7% (12 of 18)","Copilot":"66.7% (12 of 18)"},"best":["ChatGPT"]},{"item":"Time taken (median, in seconds)","vals":{"ChatGPT":"38.7 seconds","Gemini":"63.5 seconds","Claude":"20.2 seconds","Copilot":"17.1 seconds"},"best":["Copilot"]},{"item":"Same answer all 3 times (reproducibility rate)","vals":{"ChatGPT":"100%","Gemini":"88.9%","Claude":"100%","Copilot":"100%"},"best":["ChatGPT","Claude","Copilot"]},{"item":"Returned the answer in the requested format (compliance rate)","vals":{"ChatGPT":"83.3%","Gemini":"83.3%","Claude":"33.3%","Copilot":"85.7%"},"best":["Copilot"]},{"item":"Timed out (300 seconds)","vals":{"ChatGPT":"0 times","Gemini":"0 times","Claude":"0 times","Copilot":"4 times"},"best":[]},{"item":"Model shown on screen","vals":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash","Claude":"Opus 4.8, high","Copilot":"—"},"best":[]}],"method":"6 tasks run 3 times each, each one in a new chat, scored only by program","trials":72,"models":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":15660,"url":"https://ainformation.jp/article/15660","title":"Comparing 4 AIs for checking inconsistent spellings: 3 name-matching prompts and how to choose"},"data_url":"https://ainformation.jp/research/bench/meibo-2026-10-19","demand":2,"order":18},{"key":"lab-spreadsheet_reading","src":"lab","src_label":"4-AI Test Lab","feature":"Reading spreadsheets (Excel)","what":"Sent the same prompt for 3 tasks (easy, medium, hard): an annual total, a growth rate, and a quarterly summary table","use":"file","date":"2026-10-18","winner":"Gemini","tie":[],"reason":"It answered all 3 questions, and both the conclusion and the numbers matched the correct answers","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"○2","s":"2 tasks","na":false,"win":false},"Gemini":{"v":"○3","s":"3 tasks","na":false,"win":true},"Claude":{"v":"○2 △1","s":"3 tasks","na":false,"win":false},"Copilot":{"v":"○2 ×1","s":"3 tasks","na":false,"win":false}},"rows":[{"item":"Time taken (easy: annual total)","vals":{"ChatGPT":"— (could not be compared this time)","Gemini":"31.7 seconds","Claude":"35.6 seconds","Copilot":"17.4 seconds"},"best":["Copilot"]},{"item":"Time taken (medium: growth rate)","vals":{"ChatGPT":"36.0 seconds","Gemini":"28.4 seconds","Claude":"23.2 seconds","Copilot":"20.2 seconds"},"best":["Copilot"]},{"item":"Time taken (hard: quarterly summary table)","vals":{"ChatGPT":"48.6 seconds","Gemini":"34.6 seconds","Claude":"60.2 seconds","Copilot":"× (420.2 seconds, no answer)"},"best":["Gemini"]},{"item":"Character count (easy, medium, hard)","vals":{"ChatGPT":"—, 79 characters, 345 characters","Gemini":"229 characters, 302 characters, 665 characters","Claude":"341 characters, 258 characters, 802 characters","Copilot":"43 characters, 172 characters, —"},"best":[]},{"item":"Accuracy (easy: 558,900)","vals":{"ChatGPT":"—","Gemini":"○ The breakdown also matched","Claude":"△ The total matched, but 2 items in the breakdown were off","Copilot":"○"},"best":["Gemini"]},{"item":"Accuracy (medium: Product B at 1.32x)","vals":{"ChatGPT":"○","Gemini":"○","Claude":"○","Copilot":"○"},"best":[]},{"item":"Accuracy (hard: 24 cells)","vals":{"ChatGPT":"○ All matched","Gemini":"○ All matched","Claude":"○ All matched, plus xlsx","Copilot":"×"},"best":[]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"Short and to the point","Gemini":"Added a table and a trend summary","Claude":"Included a disclaimer, slightly formal tone","Copilot":"Shortest and most direct, at 43 characters"},"best":[]},{"item":"Limitations","vals":{"ChatGPT":"Sent via the Work screen","Gemini":"10 files per session, 100 MB per file","Claude":"On via settings; 30 MB per file","Copilot":"Free, with limits on volume and size"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":10,"models":{"Gemini":"Flash","Claude":"Opus 4.8, high","ChatGPT":"GPT-6.1 Sol Light"},"article":{"id":15626,"url":"https://ainformation.jp/article/15626","title":"Comparing 4 AIs for reading Excel files: 3 prompts for tallying data"},"data_url":null,"demand":2,"order":19},{"key":"lab-translation","src":"lab","src_label":"4-AI Test Lab","feature":"Translation (English to Japanese, Japanese to English)","what":"Sent the same prompt for 3 tasks (easy, medium, hard): English to Japanese, Japanese to English, and plain Japanese","use":"write","date":"2026-10-27","winner":"Claude","tie":[],"reason":"Across the three tasks, it kept every number and produced the most polished Japanese for English-to-Japanese and the best-handled honorifics for Japanese-to-English.","metric":"Correctness (○/△/× for each task)","cells":{"ChatGPT":{"v":"14 of 14, 5 of 5","s":"","na":false,"win":false},"Gemini":{"v":"×1","s":"1 task","na":false,"win":false},"Claude":{"v":"14 of 14, 5 of 5","s":"","na":false,"win":true},"Copilot":{"v":"14 of 14, 5 of 5","s":"","na":false,"win":false}},"rows":[{"item":"Seconds taken (easy: English to Japanese)","vals":{"ChatGPT":"51.6 seconds","Gemini":"31.4 seconds (declined)","Claude":"44.7 seconds","Copilot":"34.5 seconds"},"best":["Copilot"]},{"item":"Seconds taken (medium: Japanese to English)","vals":{"ChatGPT":"45.0 seconds","Gemini":"19.2 seconds","Claude":"23.2 seconds","Copilot":"23.3 seconds"},"best":["Gemini"]},{"item":"Seconds taken (hard: Japanese to plain Japanese)","vals":{"ChatGPT":"49.0 seconds","Gemini":"217.9 seconds","Claude":"29.3 seconds","Copilot":"23.4 seconds"},"best":["Copilot"]},{"item":"Character count (easy)","vals":{"ChatGPT":"970 characters","Gemini":"35 characters","Claude":"1,210 characters","Copilot":"1,085 characters"},"best":[]},{"item":"Character count (medium)","vals":{"ChatGPT":"949 characters","Gemini":"1,105 characters","Claude":"1,319 characters","Copilot":"1,064 characters"},"best":[]},{"item":"Character count (hard)","vals":{"ChatGPT":"606 characters","Gemini":"632 characters","Claude":"797 characters","Copilot":"447 characters"},"best":[]},{"item":"Accuracy (number match)","vals":{"ChatGPT":"14 of 14, 5 of 5","Gemini":"× (no translation for English to Japanese); 5 of 5","Claude":"14 of 14, 5 of 5","Copilot":"14 of 14, 5 of 5"},"best":[]},{"item":"Naturalness of the Japanese","vals":{"ChatGPT":"○ No honorific on the addressee in the English translation","Gemini":"△ Declined the English-to-Japanese task","Claude":"◎ Formatted as a press release","Copilot":"△ Left English text in the English-to-Japanese translation"},"best":["Claude"]},{"item":"Limitations","vals":{"ChatGPT":"None","Gemini":"Declined the English-to-Japanese press release","Claude":"None","Copilot":"None (tested on a free account)"},"best":[]}],"method":"Each of the 4 AIs was given the same prompt once (a new chat each time)","trials":12,"models":{"ChatGPT":"GPT-6.1 Sol Light","Gemini":"Flash","Claude":"Opus 4.8, high"},"article":{"id":16485,"url":"https://ainformation.jp/article/16485","title":"Comparing 4 recommended AI translation tools: 3 ready-to-use prompts"},"data_url":null,"demand":0,"order":20}],"wins":{"win":{"ChatGPT":8,"Gemini":5,"Claude":4,"Copilot":0},"tie":{"ChatGPT":4,"Gemini":1,"Claude":2,"Copilot":2}},"board":{"rows":[{"ai":"ChatGPT","correct":191,"trials":207,"acc":92.2999999999999971578290569595992565155029296875,"sec":21.88329999999999841975295566953718662261962890625,"cons":100,"inst":96.099999999999994315658113919198513031005859375,"src":100,"runs":12},{"ai":"Claude","correct":146,"trials":190,"acc":76.7999999999999971578290569595992565155029296875,"sec":19.2727000000000003865352482534945011138916015625,"cons":95,"inst":53.7999999999999971578290569595992565155029296875,"src":null,"runs":12},{"ai":"Copilot","correct":130,"trials":180,"acc":72.2000000000000028421709430404007434844970703125,"sec":16.629999999999999005240169935859739780426025390625,"cons":87.2000000000000028421709430404007434844970703125,"inst":76.7999999999999971578290569595992565155029296875,"src":93.2999999999999971578290569595992565155029296875,"runs":11},{"ai":"Gemini","correct":127,"trials":207,"acc":61.39999999999999857891452847979962825775146484375,"sec":49.00829999999999841975295566953718662261962890625,"cons":86.099999999999994315658113919198513031005859375,"inst":83.099999999999994315658113919198513031005859375,"src":83.2999999999999971578290569595992565155029296875,"runs":12}],"fields":10,"trials_total":784,"updated":"2026-10-28","url":"https://ainformation.jp/research/bench"},"hub":{"faq":[{"q":"Which AI is the most accurate when comparing generative AI tools?","a":"In the cumulative results of the AI Practical Work Benchmark (10 fields, 784 trials in total), ChatGPT had the highest accuracy rate at 92.3% (191 of 207 trials). Claude came in second at 76.8%, followed by Copilot at 72.2% and Gemini at 61.4%. That said, the rankings change by feature, so choose based on the row for the feature you want to use."},{"q":"How did you compare ChatGPT, Gemini, Claude, and Copilot?","a":"AInformation gave the same text for the same tasks to the paid-plan screens of all 4 AI tools. The AI Practical Work Benchmark runs 6 tasks per category, 3 times each, and a program alone judges whether each answer is correct. The 4-AI Verification Lab runs 3 tasks (easy, medium, hard) once each, and we check each answer ourselves in the article. Both start a new conversation every time."},{"q":"Will I get the same results with the free versions?","a":"Not necessarily. We used ChatGPT Pro, Google AI Pro, Claude's Max plan, and Microsoft 365 Personal, and recorded the model names shown on screen in the scoring table. Free versions offer different models and usage limits, so both response speed and accuracy may differ."},{"q":"Why do some entries say “could not test” or “could not measure”?","a":"There are 3 reasons we could not measure a result: the editorial team's operation was interrupted, the plan's usage limit was reached, or a human verification check appeared and we stopped. These are not cases where the AI gave a wrong answer, so we do not include them in the accuracy rate's denominator. Cases where the AI refused or timed out are counted as AI results, so we include them in the denominator as incorrect answers."},{"q":"When is this comparison table updated?","a":"Comparison articles from the 4-AI test lab are added to the table on the day they are published. AI Practical Work Benchmark results are added the day after testing. When we retest the same feature, we replace the old result with the new one. The table currently covers 21 features, and the most recent test was run on October 29, 2026. The data is published under CC BY 4.0, so you can quote it as long as you cite the source."}],"about":["This table shows how AInformation compared ChatGPT, Gemini, Claude, and Copilot on the same tasks. The data comes from 10 fields in the AI Practical Work Benchmark (675 trials in total) and 11 comparison articles from the 4-AI test lab (132 submissions that produced an answer).","The AI Practical Work Benchmark recreates a fictional company's documents each time and sends the same task to all four AIs. A program checks whether each answer is correct by matching numbers, checking for specific words, checking the format, and confirming URLs open. The AIs never grade their own or each other's answers.","The 4-AI Verification Lab tests one feature on four AIs using three tasks: easy, medium, and hard. Each article checks the time taken and the content of every answer one by one, and its verification conditions describe the criteria used.","In the benchmark, the win mark goes to the AI with the highest accuracy rate (ties are marked as equal). In the lab, it goes to whichever AI the editorial team concluded was best in the article. Since each result comes from a single round of testing, rankings can change with a different day or task."]}}}