Create a Small AI Test Set With Cases You Can Explain
A useful test set contains examples whose expected results you can explain. It does not need hundreds of rows to reveal a basic instruction problem. Start with the behaviour you need to trust and write the expected answer before inspecting the model's output. Set up the example For a fictional task extractor, include a clear assignment, a suggestion with no owner, a cancelled task and a sentence with no deadline. Explain how each should be labelled. If you cannot agree on the expected result yourself, clarify the task definition first. Keep the input wording fixed while comparing prompt versions. Record the date, the visible product settings and the output. A correct result on the ordinary assignment should not hide a failure on the cancelled task. Report the cases separately before adding a total score. Review before using the result Add examples when a real failure reveals a new category, but do not remove a hard case merely because it lowers the score. Keep private or protected ...