Compare AI Assistants With a Rubric Before Picking a Winner
A single impressive reply is a weak basis for choosing an assistant for recurring work. First decide what a usable answer must contain. Then compare outputs against the same rubric, using tasks that resemble your actual workload. Define a small evaluation set Choose three different tasks: a reference-based summary, a rewrite with firm constraints, and a structured extraction. Use fictional or public material that you are authorised to share. Include one awkward case, such as a missing field, rather than testing only clean examples. For a fictional meeting note, ask every assistant to extract decisions, owners and dates, marking absent values as unknown. Provide identical instructions and inputs. Record product settings you can observe, the date, and whether browsing or other tools were enabled. Score observable outcomes A simple rubric can award one point each for complete required fields, faithful use of the source, correct handling of unknowns, and usable formatting. Add a separate c...