Job 10 · Not yet tested
Reliable, cited answer to a research question?
Research tools on three questions whose answers are known before the run. Citations have to point at real sources that support the claim.
People who need a sourced answer, not a confident paragraph.
The standard test
Pack 0.1.0 · specified · Three known questions
3 questions with known answers
Setup
Three questions. The reviewer knows the answers and does not paste them into the prompt. We check whether the answer is right and whether each citation is a real page that supports the claim. A wrong answer with confident citations scores poorly on output quality.
Inputs
The three questions. No source pack, so the tool has to find its own.
Time limit
One wall-clock cap for every tool in the job. The number is published with the pack before the first run and does not move between tools.
What we record
- Answers correct
- Citations that support the claim
- Invented sources
- Time
- Cost
- Version
- Plan
How the FitScore is built
FitScore = weighted average × 20, rounded to the nearest integer. The weighted average is the sum of (criterion score × weight) divided by 100.
- Output quality30
- Reliability15
- Cost and free tier15
- Speed10
- Setup friction10
- Agent, API, and MCP support10
- Privacy and data10
5 excellent · 4 good · 3 usable · 2 poor · 1 fails. Full method.
Candidates
Candidates for Research assistants. This order is the launch list, not a ranking. No FitScores are published.
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Questions this page answers
- Reliable, cited answer to a research question?
- FitForTool has not published a FitScore for this job. The launch candidates are Perplexity, ChatGPT deep research, Gemini Deep Research, Claude Research, Grok DeepSearch, and Elicit. Each one is listed as not yet tested. A ranking is published only after every tool in the comparison has had the same test pack run, and a person has reviewed the result. Until that page shows a FitScore, there is no winner.
- How will FitForTool test research assistants?
- 3 questions with known answers. Three questions. The reviewer knows the answers and does not paste them into the prompt. We check whether the answer is right and whether each citation is a real page that supports the claim. A wrong answer with confident citations scores poorly on output quality. We record answers correct, citations that support the claim, invented sources, time, cost, version, and plan. Those notes are then scored from 1 to 5 on seven shared criteria. The pack has been specified and has not been run.
- Can a vendor pay to rank higher on this job?
- No. FitForTool does not sell position or score. A sponsorship, if we run one later, is labelled and sits outside the ranked list. An affiliate link is disclosed and does not change order. A vendor may ask for a retest, and we rerun the same pack. Sponsors do not see the result before it is public.
- What does "Not yet tested" mean?
- The tool is a candidate for this job. There is no published hands-on run, so there is no FitScore, no criterion score, and no rank. Research-only is a different label, used when we have read the docs and still have not run the pack. Neither label is a score.