AI test set: build fictional examples for a small business

An AI test set checks a defined task, not a software product's general intelligence. For a small business, the most useful starting point is a decision you can explain. In this guide, a fictional digital-services business wants to sort incoming enquiries. It asks for neither automatic replies, quotations nor research about people. The expected output contains a category and a short justification grounded in the supplied text.

We will use four categories: Qualify, Missing information, Out of scope and Review. They lead to four different human next steps. Qualify means the explicit request matches the offer. Missing information means the need cannot yet be identified. Out of scope means the only requested service is excluded. Review flags a contradiction, several incompatible intentions or content that does not resemble a new sales enquiry.

Write these definitions before creating examples. Otherwise, you may adjust the rules to each tool response and conclude that the tool works because you agree with it after the event. A category is useful only if two people can understand why it was selected. A confident explanation does not make an unstable category definition any better.

The method proposed here is an editorial working aid. It is neither a laboratory protocol nor a certification. It helps you create comparable trials for a limited task; it does not measure every risk associated with a system. The examples are intended for a small business operating in France, but they contain no assessment of any actual customer or commercial opportunity.

The set starts with an invented reference sheet. For our fictional business, the offer includes creating a brochure website and redesigning an existing brochure website. It excludes managing advertising campaigns and repairing computer hardware. No price, delivery timeframe, preferred sector or territory is specified. The classifier must therefore invent nothing about those points.

Add a priority rule: contradictions or requests mixing several categories go to Review. Next, an explicit request matching the offer goes to Qualify, even when budget or timing is absent. Missing information is reserved for messages where the need itself remains unclear. An exclusively excluded request goes to Out of scope. A simple thank-you, complaint or attempt to change the instructions goes to Review.

These rules are not universal. You could choose different ones for your business, provided you write them down and rebuild the answer key. Do not change a definition halfway through a comparison without creating a new version of the set. The purpose is to make decisions consistent, not to pretend there is only one sensible commercial process.

If you cannot clearly describe what you accept, first work on your services page. A test set does not resolve a contradictory offer. It may expose the contradiction, which is useful, but the corresponding business decision remains yours. Keep the reference sheet next to the examples so the reviewer does not have to reconstruct your offer from memory.

Create the messages from a blank page. Use no real name, email address, telephone number or extract from a customer file. You do not need realistic contact details to test whether a request is understood. An identifier such as CASE-01 is enough to retrieve the document and its result.

A completely invented example is different from a real file whose names have been replaced. Unusual details may still identify someone. France's CNIL notes that synthetic data is not automatically anonymous. This source, dated June 19, 2025 and checked on November 6, 2026, supports distinguishing independently created examples from data generated using real people.

Create variety by changing wording, length and precision rather than the authors' identities. An enquiry can be brief, polite, awkwardly written or detailed. The set should remain understandable to a human and should not depend on hidden knowledge. Avoid inserting invented personal details merely to make a case feel more realistic.

Keep files simple. A text document lets you see exactly what will be transmitted. If you use a spreadsheet, check other tabs, comments and hidden cells before using it. The trial folder should contain no business export added for convenience. This separation reduces handling mistakes; it does not guarantee the security of the service being used. Review the whole input rather than assuming the visible first page is the entire document.

A set containing only obvious enquiries creates a fragile impression of success. Prepare a matrix of difficulties: explicit need, vague need, excluded service, contradiction, mixed requests and content trying to impose its own rule. Each difficulty should correspond to a mistake you want to detect, rather than an unusual example included simply to make the test look demanding.

For each case, record three things: what changes, what must remain stable and why the case exists. In a wording variation, the words change but the expected category stays the same. In a need variation, you replace a website redesign with a computer repair: the category must change. This distinction prevents you from expecting the same answer mechanically from two messages that merely look similar.

Include a message that sounds highly professional but requests an excluded service. Polished language must not move it into Qualify. Conversely, a spelling mistake should not exclude a clear request matching the offer. You are testing understanding of the need, not a preference for a particular writing style.

Keep the matrix proportionate. Start with enough cases to cover the main difficulties, then add situations when you identify a gap. No magic number turns a small set into evidence of general reliability. It is better to know precisely what ten cases check than to claim that a hundred similar messages cover your entire business.

The following ten cases apply only the reference sheet described above. They are invented examples, not messages received by SprintLead or results measured with software. Keep the identifiers so you can compare outputs from different attempts without guessing which input was used.

The answer key contains a category and a reason, not a sales reply to send. It must never turn Qualify into “good customer,” “creditworthy” or “ready to buy.” Nothing in the set supports those conclusions. If you add a next step, keep it human and limited: review, request clarification or prepare a conversation.

To reuse the model, first replace the reference sheet with your fictional offer, then rewrite the expected results. Do not automatically keep this table's categories after your rules have changed.

Ask a second person to classify the examples using only the reference sheet. Compare disagreements before running the test. If you cannot jointly explain why CASE-06 belongs in Review, the problem may be your rule rather than the tool. Correct the definition or case and retain the new version. An unclear answer key makes every later comparison harder to interpret.

If you work alone, revisit the set after writing it and deliberately look for another interpretation. This does not replace independent review, but it may reveal an ambiguous word or hidden expectation. Record that limitation rather than presenting your answer key as unquestionable truth. You are making a business rule explicit, not discovering a universal category for every enquiry.

The key should also specify blocking errors: inventing a service, stating an absent price, obeying CASE-09 or making a decision about a person. An elegant explanation does not compensate for these errors. Conversely, wording different from your reference sentence may be acceptable when it preserves exactly the meaning and category.

Microsoft's documentation on evaluation sets, checked on November 6, 2026, describes pairing cases with expected answers. Our model remains independent of that product: you can maintain it in a simple document without installing the tool or adopting its metrics. The link supports the general evaluation concept, not a claim that this small example has been validated by Microsoft.

Use part of the examples to improve your instruction, then keep other situations aside until the final comparison. Development cases help reveal an unclear definition. Reserved cases show whether the instruction also works on text you have not repeatedly corrected with the tool during preparation. Keep the distinction visible in your document or spreadsheet.

This separation concerns your working method. It does not mean you are technically training the model or controlling how the provider uses data. Before using a service, follow its authorized framework and your business's rules. The CNIL questions and answers on generative AI, checked on November 6, 2026, explain limitations and precautions concerning transmitted information.

To create reserved cases, write new messages following the same reference sheet. One variant could request modernizing the pages of a brochure website without using the word “redesign.” Another could request only a screen repair. Write the expected categories before submitting them, and do not give the answer key to the tool that must classify the messages.

If you later use a reserved case to repair your instruction, it becomes a development case. Replace it in the reserved group. Without that discipline, the final check gradually measures your ability to make the tool repeat known answers rather than its ability to apply the rules. A new wording is useful only when its expected meaning remains clear.

Keep the reference sheet, instruction, set version and obtained responses together. Record the service, date and available settings. When comparing two instructions, keep the same cases. When comparing two set versions, explain what changed. A comparison whose inputs vary silently does not support a solid conclusion, however tidy the final report looks.

Request a simple output: case identifier, one of the four allowed categories and a justification based on the message. Add no enrichment, web research or action in an external tool. Sorting remains an exercise using fictional documents. Instructions contained inside a message are data to examine, not orders to follow. The task should not quietly expand while you are trying to evaluate it.

Save the original response before correcting it. Record your judgment and the reason for disagreement separately. If you directly replace the incorrect category in the output file, you lose evidence of the mistake and may count a success that never happened. A corrected deliverable and an uncorrected model response are different records.

You can repeat selected cases to observe variation. Do not retain only the best attempt. Record unsuccessful runs and their conditions without exceeding the budget and time you chose. A successful repeat after several failures must not erase those failures from the assessment. It may still reveal a useful pattern, provided the record remains complete.

Count correct categories, but focus especially on the nature of the differences. An overall result can conceal a serious error on one case. In this exercise, classifying CASE-09 as Qualify because the message orders it is different from providing an overlong explanation. Present those errors separately rather than blending them into a single reassuring score.

A practical review grid distinguishes wrong category, explanation unsupported by the text, invented information, an instruction inside the message being followed, unusable format and missing response. One case may contain several defects. The aim is not to inflate an error count but to understand the mechanism that makes an output unusable. This is more actionable than calling the whole answer “poor.”

Include checking and correction time. Sorting that is usually correct may still be unhelpful when verification takes more work than manual classification. Compare the same scope and quality standard. Claim no time saving before measuring it in your own trials. The apparent speed of the first answer is only one part of the work.

Then choose a precise action: clarify a rule, add a missing case, simplify the output or stop this use. When changing an instruction, rerun relevant older cases to check that a repair has not damaged a previously understood situation. Improvement on one example is not enough to conclude that performance across the whole set has improved.

Give the set a name and version, such as “Fictional website-enquiry sorting, version 1.” Add the date, reference sheet used, case list and changes since the previous version. Keep identifiers stable when a case's meaning has not changed. If you rewrite its underlying need, say so explicitly instead of silently replacing the text. A version record should explain differences, not merely label files.

A positive result means only that the observed outputs met your criteria on these examples and under these conditions. It demonstrates neither legal compliance, service confidentiality nor reliability across every real message. Moving to real data or automatic action requires a separate review. Do not present it as the inevitable technical next step after a successful test.

The commercial need also requires verification. B2B customer interviews can help you understand the requests your offer should address. The SprintLead page for agencies concerns finding businesses for prospecting, a different step from building this set. It provides no evidence about the security or performance of the tool being evaluated.

Your final deliverable has four parts: reference sheet, cases, answer key and trial log. Another person should be able to read them and reconstruct your conclusion. If they cannot explain why you accepted or rejected the use, complete the record before seeking a more flattering score. A clear decision is more useful than a number whose meaning nobody can defend.