Review AI Outputs

Human reviewers can assess writing, code, images, research, or other AI-generated content against a question you define. Use their feedback and optional AI ratings to improve the next output. Your team makes the final decision.

Rating AI reviews supported tasks by default once the owner allows processing, alone or alongside invited reviewers. Its ratings are advisory and cannot replace required human review. Supported AI tasks.

1. Scope the review policy

Choose the human-review percentage in Agents → Review setup → Human review using Impact of an AI mistake. For example, 50 starts with 50% human review and can fall to 25% after sufficient comparable evidence. If adaptive review is unavailable, it stays at 50%.

At 0, independent Rating AI rates eligible outputs without initial human calibration. At 100, every eligible output needs human review and no AI evaluation is created. Previously saved Stakes policies remain human-only until the owner explicitly chooses a new balance.

RateLoop keeps review evidence separate by agent version, policy version, workflow, risk tier, and reviewer audience. A new or changed scope begins with frequent review. Stable agreement can lower baseline coverage, while critical risk, missing context, a long unreviewed gap, or weaker agreement can require another check.

The workspace owner controls the question, response window, invited reviewer group and permitted material in Review setup. Evidence from a different scope cannot silently lower review coverage.

2. The agent requests a review

After connection, the agent reads its owner-approved context and evaluates each eligible output. If review is required, it requests one review and waits for the same operation instead of creating a duplicate. Generic MCP integrations are advisory; only a verified host integration that controls delivery can establish that the output stayed blocked while it waited. No host currently holds that tier.

See Agents & MCP for the connection and tool sequence.

3. Eligible reviewers answer without seeing other responses

Only people invited by your workspace can receive its reviews. An assigned reviewer sees the material needed for that case, selects an answer, and can add a reason before the deadline. Other responses and AI ratings remain hidden while the reviewer answers.

Humans has three tabs: To review, Profile, and Settings. Profile holds your account details, workspace access, and Download my reviews (CSV). The download contains your own authorized reviews; blinded policy reviews provide a submission receipt without revealing concealed answers or labels.

Every public-Alpha review option accepts only public, synthetic, or safely redacted material. Workspace-scoped access is not permission to submit confidential customer information, secrets, regulated data, or personal data you are not authorized to share.

4. RateLoop returns a result and evidence

The result keeps the review question, policy scope, verdict, reasons, agreement or disagreement, and available evidence together. Workspace members can inspect completed work and supporting records together in Results.

The evidence records what the review process observed. It does not prove that the source material was correct, that every agent output was intercepted, or that a later customer decision was compliant.

5. Your team decides what happens next

The accountable owner decides whether to approve, revise, retest, escalate, or stop. RateLoop supplies human review evidence; it does not issue an automatic production, safety, legal, medical, or compliance approval.

Use the same feedback to choose models

Enable Optimize model cost for text generated in Chat or through the API. Existing human ratings can improve model selection with your private-learning permission. Qualification requires human review of every answer in the evaluation period; no second review is requested.

RateLoop may try permitted models on a small share of new requests. Once enough human feedback supports a change, it selects the lowest estimated-cost model validated for your saved question and request type. Until then, it keeps your default apart from permitted trials.

Keep control of permitted models and spending, and turn optimization off at any time. Savings depend on your workload. Review connections alone do not change the model used by ChatGPT, Codex or other external hosts.

In Claude Code, the RateLoop plugin can apply the same evidence to agent sessions. When a cheaper Claude model met your review standard for a workflow, it runs new sessions of that workflow on that model, never above the model the user chose and on the user's own Claude Code credentials. Other hosts receive the recommendation from the read-only guidance tool, and owners see each workflow's model fit under Agents → Setup → Evaluate and improve.

Continue with the Evidence reference for record boundaries or Use Cases for example review questions.