Aristo 评分标准
版本 rubric-2.0
Aristo 给出的每个分数都依据这份评分标准计算。它是公开的,方便您提出异议;它有版本记录,让您的历史记录保持可比。
提示词如何评分
每项标准依据下方的评分锚点给出 0 到 4 分;如果某项与任务无关,则标记为不适用。总分为加权平均值,以 0 到 100 分呈现。系统会进行三次独立的评审采样并取中位数;如果结果不一致,会再加入由更强模型完成的两次采样,并显示置信度说明。
目标与成功标准
是否说明了期望的结果,以及如何判断结果是好的?
评分锚点
- 0
No discernible outcome.
示例: Tell me about marketing.
- 1
Topic or activity stated, outcome implicit.
示例: Write about our new product.
- 2
Outcome stated, no success criteria.
示例: Write a launch email for our product.
- 3
Outcome plus at least one testable criterion.
示例: …that gets existing customers to book a demo; under 150 words.
- 4
Outcome, testable criteria, and priority among them when they conflict.
示例: …Success = clear single call to action and a demo link above the fold; brevity matters more than covering every feature.
背景与输入
模型是否获得了所需的事实、材料、数据和情境,并且界定清晰?
评分锚点
- 0
No context at all; the model must guess the situation, the subject and the material.
示例: Summarise this.
- 1
Situation named in passing; the material or facts the task needs are missing or only alluded to.
示例: Summarise the meeting we had about the budget.
- 2
The needed material is present but undelimited or incomplete, so the model cannot tell input from instruction.
示例: Summarise the meeting notes below for the finance team. Budget review 14 May: Ana raised…
- 3
Material is clearly delimited (tags, fences or headings) and the situation is explained; small gaps remain.
示例: <notes>…</notes> These are notes from the 14 May budget review; summarise them for the finance team.
- 4
Complete and delimited material, background the model could not infer, and a statement of what it does not have.
示例: <notes>…</notes> Background: this is the third review this quarter; last time we overspent on travel. Figures for June are not yet available, so do not estimate them.
受众与用途
输出是给谁看的,他们会用它做什么?
评分锚点
- 0
No reader or purpose can be inferred.
示例: Explain photosynthesis.
- 1
Reader implied by the topic or the register, but never stated.
示例: Explain photosynthesis simply.
- 2
Reader named; what they will do with the output is not.
示例: Explain photosynthesis to a 12-year-old.
- 3
Reader and purpose of use both stated.
示例: Explain photosynthesis to a 12-year-old preparing for a science quiz next week.
- 4
Reader, purpose and what they already know or need, so the level, emphasis and register follow from it.
示例: …for a 12-year-old who has covered cells but not chemistry; she needs to answer short written questions, so favour precise terms over analogies.
约束与边界
范围、必须与禁止事项、篇幅、来源、安全,以及不确定时该怎么做。
评分锚点
- 0
No boundaries of any kind.
示例: Write a blog post about remote work.
- 1
One vague boundary, usually length or tone, nothing on scope or sources.
示例: Write a short blog post about remote work.
- 2
Several concrete boundaries (length, scope, sources) but silent on what to do when uncertain.
示例: …about 600 words, for managers, no product recommendations, cite at least two studies.
- 3
Concrete boundaries plus explicit must-nots and an instruction for the uncertain case.
示例: …if you cannot find a study published after 2023, say so rather than inventing one.
- 4
Boundaries are complete, consistent and prioritised, stated as what to do rather than only what to avoid.
示例: …Word count may stretch to 800 if needed to keep both studies; accuracy outranks length; mark any claim you are unsure of with “(verify)”.
任务指令
清晰、有序、无歧义的步骤;单一任务,或明确排好顺序的多项任务。
评分锚点
- 0
No actionable instruction; a topic or a fragment.
示例: Customer churn Q3.
- 1
One ambiguous instruction, or several tasks tangled together with no order.
示例: Look at the churn data and also fix the dashboard and tell me what to do.
- 2
A clear single task, or multiple tasks listed but their order or dependencies unstated.
示例: Analyse the Q3 churn data and recommend three actions.
- 3
Clear, ordered steps or an explicit decomposition; each step has one verb and one object.
示例: 1. Compute churn by plan. 2. Identify the plan with the biggest rise. 3. Propose three actions for that plan.
- 4
Ordered steps with the reasoning behind them, what to do if a step cannot be completed, and where to stop.
示例: …Step 2 matters because we can only fund one intervention. If the data for a plan is missing, skip it and say so. Stop after the three actions; do not draft the rollout plan.
输出约定
结果的格式、结构、模式(schema)、篇幅、语言和语气。
评分锚点
- 0
Nothing said about the form of the result.
示例: Compare the two proposals.
- 1
A single loose hint such as “briefly” or “in a table”.
示例: Compare the two proposals in a table.
- 2
Format and rough length stated; structure, language or tone left open.
示例: Compare the two proposals in a table of at most ten rows, then a one-paragraph recommendation.
- 3
Format, structure, length and language or tone all stated and consistent with the goal.
示例: …columns: criterion, proposal A, proposal B, winner. Write in Portuguese, neutral tone, no preamble.
- 4
A complete contract: schema or template, length, language, tone, and how edge cases appear in the output.
示例: …Return JSON {"rows":[{"criterion","a","b","winner"}],"recommendation"}; if a criterion cannot be judged, set winner to "tie" and explain in recommendation.
示例与参考
在真正有帮助时提供示例或参考材料;要多样,避免过度拟合。
评分锚点
- 0
Examples would clearly help (a specific format, style or classification) but none is given.
示例: Classify each support ticket by urgency.
- 1
A reference is mentioned but not provided, or one example that is incomplete.
示例: Classify each ticket by urgency, like we did last month.
- 2
One complete example, or several that all show the same case, so the model may over-fit it.
示例: <example>Ticket: “Site down for all users” → urgent</example>
- 3
Two to five varied examples in a consistent, delimited format.
示例: <examples><example>…→ urgent</example><example>…→ normal</example><example>…→ low</example></examples>
- 4
Varied examples that include an edge case or a counter-example, each with a short note on why.
示例: …<example>“Invoice typo” → low, because nothing is blocked</example><example>“Cannot log in (one user)” → normal, not urgent: single user</example>
角色与口吻
能改变输出的视角或专业身份;语气指导。
评分锚点
- 0
A perspective would change the output and none is given, or the role given contradicts the task.
示例: Review this contract clause.
- 1
A generic label with no bearing on the output (“you are a helpful assistant”).
示例: You are a helpful assistant. Review this contract clause.
- 2
A relevant role named, but its perspective is not used in the instructions.
示例: You are a commercial lawyer. Review this contract clause.
- 3
A role whose expertise shapes the task, plus guidance on voice or stance.
示例: As a commercial lawyer acting for the buyer, review this clause and flag risks to the buyer; be direct, no hedging.
- 4
Role, stance, voice and the standard to judge by, applied only where it changes the output.
示例: …Judge against standard Canadian commercial practice; where the clause is market-standard say so in one line and move on.
模型适配与规范
适合目标模型系列;没有矛盾或全大写式的强调;长材料放在问题之前;不使用过时技巧。
评分锚点
- 0
Contradictory instructions, or shouting in capitals (“CRITICAL”, “MUST”), or a trick that harms current models (forced step-by-step, prefill-style openings).
示例: CRITICAL: you MUST think step by step and NEVER make mistakes. Also be brief but cover everything.
- 1
No contradictions, but obsolete tricks or heavy emphasis remain, or long material comes after the question.
示例: Think step by step. What are the risks? [20 pages of contract follow]
- 2
Plain, calm instructions; structure is weak (no delimiters) or the request ignores the target family’s conventions.
示例: Here is the contract and here are my questions, mixed together in one paragraph.
- 3
Calm, consistent instructions; long material first and the question last; delimiters or tags separate parts; explains the why where it matters.
示例: <contract>…</contract> Review the clause above. Keep the answer short because it goes into a client email.
- 4
Everything at 3, written for the chosen model family: structure and emphasis in the vendor’s recommended form, effort left to the model, nothing redundant.
示例: Same prompt with <contract> and <questions> tags, a one-line role, no “think carefully” padding, and success criteria the model can check itself.
任务类型
权重随任务类型而变化。例如,代码转换不需要考虑受众,而图像提示词的大部分权重在输出约定上。
| 任务类型 | 目标与成功标准 | 背景与输入 | 受众与用途 | 约束与边界 | 任务指令 | 输出约定 | 示例与参考 | 角色与口吻 | 模型适配与规范 |
|---|---|---|---|---|---|---|---|---|---|
| 写作 | 3 | 3 | 3 | 2 | 2 | 2 | 1 | 1 | 1 |
| 分析 | 3 | 3 | 2 | 2 | 2 | 2 | 不适用 | 1 | 1 |
| 编程 | 3 | 3 | 不适用 | 3 | 2 | 2 | 1 | 不适用 | 1 |
| 研究 | 3 | 3 | 2 | 3 | 2 | 2 | 不适用 | 1 | 1 |
| 数据提取 | 3 | 3 | 不适用 | 2 | 2 | 3 | 1 | 不适用 | 1 |
| 创意 | 2 | 3 | 2 | 2 | 2 | 2 | 1 | 2 | 1 |
| 辅导 | 3 | 3 | 3 | 2 | 2 | 2 | 1 | 1 | 1 |
| 智能体系统提示词 | 3 | 3 | 2 | 3 | 3 | 2 | 1 | 2 | 2 |
| 图像生成 | 3 | 2 | 不适用 | 2 | 1 | 3 | 不适用 | 不适用 | 1 |
| 其他 | 3 | 3 | 2 | 2 | 2 | 2 | 1 | 1 | 1 |
本评分标准依据有关提示词训练的研究证据(先明确需求,再斟酌措辞)以及各厂商针对推理模型的最新指南;来源列于项目文档中。