Six AI projects worth testing before you scale anything.

Decision: Which source-supported information should a reviewer use to answer a question, prepare a case, or continue an investigation?

Best when

The organization has a bounded, permissioned body of current material and reviewers can judge whether an answer is supported, complete, and appropriate for the situation.

Inputs

Approved source documents, source owners, access rules, version and freshness data, known gaps, representative questions, expected evidence, and explicit subjects the system must not answer.

Workflow

Retrieve candidate passages, show their provenance, compose a bounded answer, expose uncertainty or missing evidence, and route the result to a person before any consequential use or writeback.

Human authority

People approve external or consequential answers, resolve conflicting sources, change access, accept exceptions, and decide whether the underlying record is authoritative.

First evidence

A protected set of real questions with expected evidence, citation correctness and coverage, unsupported-claim review, abstention behavior, permission tests, reviewer corrections, and review time.

Failure modes

Stale or contradictory documents, retrieval misses, access leakage, hidden instructions in source material, fabricated support, omitted context, over-reliance, and a feedback queue nobody owns.

Operating change

The source library becomes a maintained product. Someone must own freshness, permissions, disputed answers, feedback triage, incident response, and the decision to remove a source or suspend the assistant.

Decision: How should an incoming document be represented, which fields are sufficiently supported, and which exceptions need a person?

Best when

Documents are frequent enough to justify a pipeline, the target record and business rules are explicit, and field-level evidence can remain attached to every accepted or rejected value.

Inputs

Representative files and images, document and field taxonomies, target schemas, field definitions, deterministic rules, duplicate policy, labeled exceptions, privacy constraints, and downstream acceptance criteria.

Workflow

Identify the document, extract candidate fields with page or region provenance, validate type and business rules deterministically, detect duplicates, and send ambiguity or contradiction to an exception queue.

Human authority

People resolve ambiguous evidence, approve material records, correct classifications, authorize downstream actions, and decide how to treat handwriting, missing pages, conflicting values, and unsupported fields.

First evidence

Field-level accuracy by document and field type, false acceptance, false rejection, provenance coverage, duplicate behavior, exception volume, correction patterns, latency, and downstream reconciliation.

Failure modes

Image quality, layout drift, handwriting, copied identifiers, unit mistakes, prompt injection inside documents, personal-data exposure, silent default values, and apparent confidence without usable evidence.

Operating change

Intake becomes a monitored queue rather than a one-time upload. Schema versions, source retention, reviewer workload, correction capture, reprocessing, and downstream reconciliation all need explicit owners.

Decision: Which queue, category, priority, or next review should receive a request without allowing a model to grant access or make the underlying business decision?

Best when

Categories have operational meaning, owners and service paths are stable enough to maintain, and a wrong route can be detected and corrected before material harm occurs.

Inputs

A current taxonomy, queue ownership, labeled historical requests, rare and high-consequence cases, priority rules, prohibited actions, identity and permission context, and an escalation path.

Workflow

Normalize the request, suggest labels and a route, apply deterministic permission and priority policy, expose uncertainty, send protected classes to review, and record the final route and any override.

Human authority

People define categories and priority policy, approve sensitive routing, handle conflicts, change ownership, and retain authority over entitlement, eligibility, discipline, credit, care, and other consequential outcomes.

First evidence

Confusion by class, rare-case recall, high-consequence misroutes, time to correction, override reasons, queue balance, policy-block behavior, drift in category volume, and end-to-end resolution rather than label accuracy alone.

Failure modes

Historical labels encode inconsistent practice, category drift, urgency hidden in free text, strategic manipulation, biased routing, permissions inferred from content, automation loops, and unstaffed queues.

Operating change

The taxonomy becomes an operational contract. Queue owners need a way to correct routes, update definitions, review drift, identify new classes, and stop automation when ownership or policy changes.

Decision: Which draft, evidence, and unresolved questions should an accountable person review before communicating or committing anything?

Best when

The final author has real review time and subject authority, source and tone rules can be stated, and the draft can be withheld when evidence or context is missing.

Inputs

Approved facts and sources, recipient context, purpose, tone and policy rules, required disclosures, prohibited claims, examples with usage rights, privacy constraints, and a review rubric.

Workflow

Assemble permitted context, produce a draft with visible source support and unresolved items, run deterministic policy and data checks, then require review before sending, publishing, or changing a system of record.

Human authority

A person owns factual accuracy, professional judgment, promises, tone, recipients, disclosure, and the send or publish action. Approval is a real decision, not a button pressed after a cursory glance.

First evidence

Factual support, prohibited-claim detection, reviewer corrections, rejected drafts, unresolved-item capture, inappropriate disclosure, time spent reviewing, and incidents. Edit distance can describe workload, not quality by itself.

Failure modes

Plausible unsupported statements, private data in context or output, copied phrasing, policy drift, fabricated references, wrong recipients, hidden reviewer fatigue, and fluency that suppresses skepticism.

Operating change

Source, policy, and disclosure rules need versioning. Review responsibility, correction feedback, incident handling, retained records, and the ability to disable a template or channel become part of the communication process.

Decision: Which future range or unusual condition deserves investigation, and what should an accountable planner decide after seeing it?

Best when

The outcome and forecast horizon are measurable, a simple baseline exists, error costs can be discussed, and planners can act on exceptions without treating a forecast as an order or commitment.

Inputs

Time-stamped outcomes, known drivers, horizon and refresh cadence, segment definitions, lead times, decision constraints, missing-data rules, naive baselines, error costs, and recorded interventions.

Workflow

Create a time-correct comparison, produce ranges and exception signals, compare them with the baseline, expose missing or changed conditions, send exceptions to a planner, and return actual outcomes for review.

Human authority

People approve purchases, staffing, pricing, capacity, financial treatment, safety actions, and other commitments. They can reject a signal, record outside knowledge, and halt use after a regime change.

First evidence

Backtesting by horizon and segment, error against a simple baseline, interval calibration, false alarms, missed costly events, stability after interventions, data delay, decision use, and realized outcomes after enough time passes.

Failure modes

Future information leaking into training, broken timestamps, regime shifts, unrecorded interventions, sparse segments, correlated errors, misleading averages, false precision, and feedback from prior model-influenced decisions.

Operating change

Planning moves toward exception review with recorded reasons. Data delays, overrides, external events, model changes, error cost, and actual results need a recurring review rather than a one-time accuracy report.

Decision: Which image, frame, region, or item should a trained person inspect, and what evidence should accompany that review?

Best when

Capture conditions can be controlled or measured, the target condition is defined, representative images can be lawfully retained, and a false negative has a safe review or sampling path.

Inputs

A defect or condition taxonomy, representative images across cameras and environments, region labels, capture requirements, acceptance tolerances, rare and borderline cases, privacy rules, and the downstream disposition process.

Workflow

Check capture quality, preserve the source image, produce a candidate region and score, apply a bounded threshold, route uncertain and sampled normal cases to inspection, and record the final human disposition.

Human authority

Qualified people define the inspected condition, approve disposition, stop production or service when required, set sampling policy, resolve disputed labels, and decide whether capture or model changes are acceptable.

First evidence

Performance by condition, product, site, camera, lighting, angle, subgroup, and time; false negatives; false positives; image-quality rejection; localization quality; reviewer agreement; latency; drift; and sampled misses.

Failure modes

Lighting and camera shifts, shortcut learning, mislabeled or unrepresentative images, rare unseen conditions, privacy exposure, unsuitable compression, latency, reviewer over-trust, and a score used beyond its evaluated context.

Operating change

Cameras, capture instructions, sampling, label review, privacy, model versions, false-negative investigation, maintenance, and safe fallback become part of the inspection process, not background technical details.