Case studyProduct Strategy
Product Vision
Enable developers to evaluate and demonstrate their understanding of a codebase through repository-specific assessments, with an emphasis on understanding rather than simply producing code.
Product Principles
- Assess understanding, not authorship. Assessment results indicate performance on a specific quiz; they must not be presented as proof of who wrote the code.
- Ground assessments in the repository. Questions should test understanding of the submitted codebase rather than rely on generic programming trivia.
- Make results actionable. Feedback should help users identify gaps in their understanding and find useful next steps.
- Be transparent about uncertainty. Results should communicate the limits of the assessment and avoid implying more certainty than the evidence supports.
- Keep sharing optional. Users decide whether to share their assessment results publicly.
- Protect trust through clear boundaries. Explain what the assessment can and cannot establish, especially when displaying a shareable badge or result page.
Success Metrics
SlopTeacher's success can be evaluated across three areas:
Activation
The proportion of users who start and complete a repository assessment.
Assessment quality
Whether questions and diagnostic reports are relevant to the selected repository and useful to the person taking the assessment.
Continued value
Repeat assessment usage and voluntary sharing of assessment results.
Quantitative targets and internal performance data are not disclosed in this case study. These categories should be described as measurement areas, not established results, unless they have been instrumented and reviewed.
| Area | What it measures | Status in this case study |
|---|---|---|
| Activation | Start and complete a repository assessment | Measurement area |
| Assessment quality | Relevance and usefulness of questions and diagnostic reports | Measurement area |
| Continued value | Repeat usage and voluntary sharing of results | Measurement area |
Product Requirements
| # | Requirement |
|---|---|
| 1 | Accept public GitHub repository URLs and prepare a compact repository snapshot for assessment. |
| 2 | Allow users to choose a quiz length of 10, 15, or 20 questions. |
| 3 | Support easy, medium, and hard difficulty levels. |
| 4 | Provide a text-based interview with a pass, pending, or slop result. |
| 5 | Generate a diagnostic report with notes and relevant learning resources. |
| 6 | Offer an optional signed README badge and a shareable result page. |
| 7 | Require authentication for app sessions. |
Constraints
| Constraint | Detail |
|---|---|
| Repository access | Only public GitHub repositories are supported; private repositories are not supported. |
| Authentication | Authentication is required for app sessions. |
| Usage limits | Daily usage limits apply to sessions and model calls to help keep the free product available. |
| Badge meaning | A badge records the result of an assessment session. It does not prove authorship or establish overall developer competence. |
Prioritization
Detailed backlog priorities and internal implementation plans are omitted from this case study.
MVP focus
The MVP prioritized the core assessment journey: repository submission, configurable questioning, diagnostic feedback, and optional result sharing.
What was deferred
Features outside this journey were deferred in favor of focusing on the core experience and keeping the product manageable to operate.
MVP Definition
Shipped MVP: v1.1.0
What the MVP includes
The MVP includes a marketing site, authenticated assessments on public GitHub repositories, a diagnostic report, and an optional shareable badge and result page.
Status and disclosure
The product is live.
Internal technology choices, infrastructure details, and implementation-specific information are intentionally excluded from this public case study.
