سُلّم تقييم Aristo
الإصدار rubric-2.0
كل درجة يمنحها Aristo تُحسب من هذا السُّلّم. وهو منشور لكي تتمكن من الاعتراض عليه، وذو إصدارات لكي يبقى سجلك قابلًا للمقارنة.
كيف يُقيَّم الأمر
يُقيَّم كل معيار من 0 إلى 4 وفق المرتكزات أدناه، أو يُعلَّم بأنه لا ينطبق حين تجعله المهمة غير ذي صلة. والدرجة الإجمالية هي المتوسط المرجّح، وتُعرض من 0 إلى 100. تؤخذ ثلاث عيّنات مستقلة من المُقيِّم ويُعتمد الوسيط؛ وإذا اختلفت، أُضيفت عيّنتان أخريان من نموذج أقوى وعُرضت ملاحظة بشأن درجة الثقة.
الهدف ومعايير النجاح
هل النتيجة المطلوبة مذكورة، وكيف سنعرف أنها جيدة؟
المرتكزات
- 0
No discernible outcome.
مثال: Tell me about marketing.
- 1
Topic or activity stated, outcome implicit.
مثال: Write about our new product.
- 2
Outcome stated, no success criteria.
مثال: Write a launch email for our product.
- 3
Outcome plus at least one testable criterion.
مثال: …that gets existing customers to book a demo; under 150 words.
- 4
Outcome, testable criteria, and priority among them when they conflict.
مثال: …Success = clear single call to action and a demo link above the fold; brevity matters more than covering every feature.
السياق والمدخلات
هل لدى النموذج ما يحتاجه من حقائق ومواد وبيانات وظروف، محدّدة بوضوح؟
المرتكزات
- 0
No context at all; the model must guess the situation, the subject and the material.
مثال: Summarise this.
- 1
Situation named in passing; the material or facts the task needs are missing or only alluded to.
مثال: Summarise the meeting we had about the budget.
- 2
The needed material is present but undelimited or incomplete, so the model cannot tell input from instruction.
مثال: Summarise the meeting notes below for the finance team. Budget review 14 May: Ana raised…
- 3
Material is clearly delimited (tags, fences or headings) and the situation is explained; small gaps remain.
مثال: <notes>…</notes> These are notes from the 14 May budget review; summarise them for the finance team.
- 4
Complete and delimited material, background the model could not infer, and a statement of what it does not have.
مثال: <notes>…</notes> Background: this is the third review this quarter; last time we overspent on travel. Figures for June are not yet available, so do not estimate them.
الجمهور والغرض من الاستخدام
لمن المخرجات، وماذا سيفعلون بها؟
المرتكزات
- 0
No reader or purpose can be inferred.
مثال: Explain photosynthesis.
- 1
Reader implied by the topic or the register, but never stated.
مثال: Explain photosynthesis simply.
- 2
Reader named; what they will do with the output is not.
مثال: Explain photosynthesis to a 12-year-old.
- 3
Reader and purpose of use both stated.
مثال: Explain photosynthesis to a 12-year-old preparing for a science quiz next week.
- 4
Reader, purpose and what they already know or need, so the level, emphasis and register follow from it.
مثال: …for a 12-year-old who has covered cells but not chemistry; she needs to answer short written questions, so favour precise terms over analogies.
القيود والحدود
النطاق، وما يجب وما لا يجوز، والطول، والمصادر، والسلامة، وما العمل عند عدم اليقين.
المرتكزات
- 0
No boundaries of any kind.
مثال: Write a blog post about remote work.
- 1
One vague boundary, usually length or tone, nothing on scope or sources.
مثال: Write a short blog post about remote work.
- 2
Several concrete boundaries (length, scope, sources) but silent on what to do when uncertain.
مثال: …about 600 words, for managers, no product recommendations, cite at least two studies.
- 3
Concrete boundaries plus explicit must-nots and an instruction for the uncertain case.
مثال: …if you cannot find a study published after 2023, say so rather than inventing one.
- 4
Boundaries are complete, consistent and prioritised, stated as what to do rather than only what to avoid.
مثال: …Word count may stretch to 800 if needed to keep both studies; accuracy outranks length; mark any claim you are unsure of with “(verify)”.
تعليمات المهمة
خطوات واضحة ومرتّبة لا لبس فيها؛ مهمة واحدة أو مهام متسلسلة بشكل صريح.
المرتكزات
- 0
No actionable instruction; a topic or a fragment.
مثال: Customer churn Q3.
- 1
One ambiguous instruction, or several tasks tangled together with no order.
مثال: Look at the churn data and also fix the dashboard and tell me what to do.
- 2
A clear single task, or multiple tasks listed but their order or dependencies unstated.
مثال: Analyse the Q3 churn data and recommend three actions.
- 3
Clear, ordered steps or an explicit decomposition; each step has one verb and one object.
مثال: 1. Compute churn by plan. 2. Identify the plan with the biggest rise. 3. Propose three actions for that plan.
- 4
Ordered steps with the reasoning behind them, what to do if a step cannot be completed, and where to stop.
مثال: …Step 2 matters because we can only fund one intervention. If the data for a plan is missing, skip it and say so. Stop after the three actions; do not draft the rollout plan.
مواصفات المخرجات
تنسيق النتيجة وبنيتها ومخططها وطولها ولغتها ونبرتها.
المرتكزات
- 0
Nothing said about the form of the result.
مثال: Compare the two proposals.
- 1
A single loose hint such as “briefly” or “in a table”.
مثال: Compare the two proposals in a table.
- 2
Format and rough length stated; structure, language or tone left open.
مثال: Compare the two proposals in a table of at most ten rows, then a one-paragraph recommendation.
- 3
Format, structure, length and language or tone all stated and consistent with the goal.
مثال: …columns: criterion, proposal A, proposal B, winner. Write in Portuguese, neutral tone, no preamble.
- 4
A complete contract: schema or template, length, language, tone, and how edge cases appear in the output.
مثال: …Return JSON {"rows":[{"criterion","a","b","winner"}],"recommendation"}; if a criterion cannot be judged, set winner to "tie" and explain in recommendation.
الأمثلة والمراجع
أمثلة محلولة أو مواد مرجعية حيث تفيد فعلًا؛ متنوعة وغير مفرطة في التخصيص.
المرتكزات
- 0
Examples would clearly help (a specific format, style or classification) but none is given.
مثال: Classify each support ticket by urgency.
- 1
A reference is mentioned but not provided, or one example that is incomplete.
مثال: Classify each ticket by urgency, like we did last month.
- 2
One complete example, or several that all show the same case, so the model may over-fit it.
مثال: <example>Ticket: “Site down for all users” → urgent</example>
- 3
Two to five varied examples in a consistent, delimited format.
مثال: <examples><example>…→ urgent</example><example>…→ normal</example><example>…→ low</example></examples>
- 4
Varied examples that include an edge case or a counter-example, each with a short note on why.
مثال: …<example>“Invoice typo” → low, because nothing is blocked</example><example>“Cannot log in (one user)” → normal, not urgent: single user</example>
الدور والأسلوب
منظور أو خبرة تغيّر المخرجات؛ توجيهات بشأن النبرة.
المرتكزات
- 0
A perspective would change the output and none is given, or the role given contradicts the task.
مثال: Review this contract clause.
- 1
A generic label with no bearing on the output (“you are a helpful assistant”).
مثال: You are a helpful assistant. Review this contract clause.
- 2
A relevant role named, but its perspective is not used in the instructions.
مثال: You are a commercial lawyer. Review this contract clause.
- 3
A role whose expertise shapes the task, plus guidance on voice or stance.
مثال: As a commercial lawyer acting for the buyer, review this clause and flag risks to the buyer; be direct, no hedging.
- 4
Role, stance, voice and the standard to judge by, applied only where it changes the output.
مثال: …Judge against standard Canadian commercial practice; where the clause is market-standard say so in one line and move on.
ملاءمة النموذج وسلامة الصياغة
يناسب عائلة النموذج المستهدف؛ بلا تناقضات أو تشديد مبالغ فيه؛ المواد الطويلة قبل السؤال؛ بلا حيل متقادمة.
المرتكزات
- 0
Contradictory instructions, or shouting in capitals (“CRITICAL”, “MUST”), or a trick that harms current models (forced step-by-step, prefill-style openings).
مثال: CRITICAL: you MUST think step by step and NEVER make mistakes. Also be brief but cover everything.
- 1
No contradictions, but obsolete tricks or heavy emphasis remain, or long material comes after the question.
مثال: Think step by step. What are the risks? [20 pages of contract follow]
- 2
Plain, calm instructions; structure is weak (no delimiters) or the request ignores the target family’s conventions.
مثال: Here is the contract and here are my questions, mixed together in one paragraph.
- 3
Calm, consistent instructions; long material first and the question last; delimiters or tags separate parts; explains the why where it matters.
مثال: <contract>…</contract> Review the clause above. Keep the answer short because it goes into a client email.
- 4
Everything at 3, written for the chosen model family: structure and emphasis in the vendor’s recommended form, effort left to the model, nothing redundant.
مثال: Same prompt with <contract> and <questions> tags, a one-line role, no “think carefully” padding, and success criteria the model can check itself.
أنواع المهام
تتغير الأوزان بحسب نوع المهمة. فمثلًا، لا تحتاج تحويلات الشيفرة إلى جمهور، بينما تضع أوامر الصور معظم الوزن على مواصفات المخرجات.
| أنواع المهام | الهدف ومعايير النجاح | السياق والمدخلات | الجمهور والغرض من الاستخدام | القيود والحدود | تعليمات المهمة | مواصفات المخرجات | الأمثلة والمراجع | الدور والأسلوب | ملاءمة النموذج وسلامة الصياغة |
|---|---|---|---|---|---|---|---|---|---|
| الكتابة | 3 | 3 | 3 | 2 | 2 | 2 | 1 | 1 | 1 |
| التحليل | 3 | 3 | 2 | 2 | 2 | 2 | لا ينطبق | 1 | 1 |
| البرمجة | 3 | 3 | لا ينطبق | 3 | 2 | 2 | 1 | لا ينطبق | 1 |
| البحث | 3 | 3 | 2 | 3 | 2 | 2 | لا ينطبق | 1 | 1 |
| استخراج البيانات | 3 | 3 | لا ينطبق | 2 | 2 | 3 | 1 | لا ينطبق | 1 |
| الإبداع | 2 | 3 | 2 | 2 | 2 | 2 | 1 | 2 | 1 |
| التدريس | 3 | 3 | 3 | 2 | 2 | 2 | 1 | 1 | 1 |
| أمر النظام للوكيل | 3 | 3 | 2 | 3 | 3 | 2 | 1 | 2 | 2 |
| توليد الصور | 3 | 2 | لا ينطبق | 2 | 1 | 3 | لا ينطبق | لا ينطبق | 1 |
| أخرى | 3 | 3 | 2 | 2 | 2 | 2 | 1 | 1 | 1 |
يتّبع سُلّم التقييم الأدلة البحثية حول التدريب على كتابة الأوامر (المتطلبات قبل الصياغة) والإرشادات الحالية لمزوّدي نماذج الاستدلال؛ والمصادر مدرجة في وثائق المشروع.