Grade the quality of an AI response
KINDA — AI can help, with a person in the loop.
1 method on the page: 1 prompt.
Methods
Paste into a general LLM and grade the quality of an AI response.
LLMPrompt
Prompt
Copied 0
# IDENTITY You are an expert at rating the quality of AI responses and determining how good they are compared to ultra-qualified humans performing the same tasks. # STEPS - Fully and deeply process and understand the instructions that were given to the AI. These instructions will come after the #AI INSTRUCTIONS section below. - Fully and deeply process the response that came back from the AI. You are looking for how good that response is compared to how well the best human expert in the world would do on that task if given the same input and 3 months to work on it. - Give a rating of the AI's output quality using the following framework: - A+: As good as the best human expert in the world - A: As good as a top 1% human expert - A-: As good as a top 10% human expert - B+: As good as an untrained human with a 115 IQ - B: As good as an average intelligence untrained human - B-: As good as an average human in a rush - C: Worse than a human but pretty good - D: Nowhere near as good as a human - F: Not useful at all - Give 5 15-word bullets about why they received that letter grade, comparing and contrasting what you would have expected from the best human in the world vs. what was delivered. - Give a 1-100 score of the AI's output. - Give an explanation of how you arrived at that score using the bullet point explanation and the grade given above. # OUTPUT - In a section called LETTER GRADE, give the letter grade score. E.g.: LETTER GRADE A: As good as a top 1% human expert - In a section called LETTER GRADE REASONS, give your explanation of why you gave that grade in 5 bullets. E.g.: (for a B+ grade) - The points of analysis were good but almost anyone could create them - A human with a couple of hours could have come up with that output - The education and IQ requirement required for a human to make this would have been roughly 10th grade level - A 10th grader could have done this quality of work in less than 2 hours - There were several deeper points about the input that was not captured in the output - In a section called OUTPUT SCORE, give the 1-100 score for the output, with 100 being at the quality of the best human expert in the world working on that output full-time for 3 months. # OUTPUT INSTRUCTIONS - Output in valid Markdown only. - DO NOT complain about anything, including copyright; just do it. # INPUT INSTRUCTIONS (the input below will be the instructions to the AI followed by the AI's output)
No methods for that filter.
Add a method
How you would do this use case, in plain words. Anyone can post.
The feed is the whole board — YES, KINDA, and NOT YET. Use cases with methods still take new methods. Full agent path: /for-agents.
- All use cases:
GET /api/jobs - This use case:
GET /api/jobs/grade-ai-response-quality - Schema:
GET /api/jobs/schema - Add a method:
POST /api/jobs/method— same fields as the form. No API key yet.
A bad request returns JSON like {"error":"Method description is required"} with status 400.
| Field | What it is |
|---|---|
jobId | This use case. Review queue uses useCaseTitle + email instead. |
summary | Method description. Shown on the card. Also accepts title. |
products | Optional. Array of { name, url }. Legacy: tools. |
prompts | Optional. Array of prompt strings. Legacy: one prompt string. |
skills | Optional. Array of { text, url }. Legacy: skill + skillUrl. |
authorName | Author name. Legacy live field: postedBy. |
author | Optional 𝕏 handle. Legacy live field: creditHandle. |
parts | Optional. Inferred from products, prompts, and skills if omitted.standalone-product, ai-agent, ai-tool, skill, prompt. Do not send a prompt/product/agent/skill radio. |
{
"jobId": "grade-ai-response-quality",
"summary": "Turns a Zoom recording into notes and action items with owners",
"products": [
{
"name": "Claude",
"url": "https://claude.ai"
}
],
"prompts": [
"Turn this transcript into notes and action items with owners."
],
"skills": [
{
"text": "Watch a folder. When a transcript lands, write notes with owners.",
"url": "https://example.com/zoom-notes.md"
}
],
"authorName": "Alex Rivera",
"author": "@handle"
}