Enterprise AI Project Wiki · Testing and acceptance

Acceptance criteria

Also calledAcceptance conditions · Pass criteria · Criteria for acceptance

Definition

Acceptance criteria are pre-agreed, verifiable conditions used by project stakeholders to decide whether a specified deliverable or version can be accepted. They identify the object, applicable conditions, expected result, tolerance, verification method, evidence, and decision authority so that pass or fail follows recorded results rather than an improvised judgment.

What acceptance criteria are for

A requirement describes the intended capability or constraint. Acceptance criteria turn the results that matter into decision conditions. They give the customer, delivery team, and testers a shared view of completion and connect scope, pricing, scheduling, testing, and issue resolution.

Criteria should be agreed before implementation and formal acceptance. They may change with an approved requirement change, but the reason, impact, approver, and effective version must be recorded. A team should not revise the rule after seeing an inconvenient result.

The object can be a feature, dataset, integration, deliverable version, or full project phase. The broader it is, the more important it becomes to split the decision into individually verifiable conditions and identify any criteria that act as hard gates.

Anatomy of an executable criterion

Not every criterion needs eight visible fields, but this information must be available in the criterion or the acceptance material it references.

Stable ID and requirement source

Give the criterion an identifier and link it to a requirement, scope item, business rule, or risk so that changes and results remain traceable.

Defined acceptance object

Name the feature, interface, data, document, model configuration, and exact version. Results for an earlier object cannot prove a changed object passed.

Preconditions and scenario

State the role, permissions, environment, data state, dependency state, and whether the scenario is normal, boundary, exceptional, or knowledge-limited.

Observable expected result

Describe the user-visible, data, system-action, business-state, or human-handoff outcome rather than only an internal implementation.

Threshold and tolerance

For time, count, error, success rate, or quality level, define units, population, calculation, sample, and permitted variance. Words such as fast or accurate are insufficient.

Verification method

Specify test, demonstration, inspection, analysis, or document review, and whether it is performed by people, automation, or both parties together.

Evidence and record location

Identify the report, log, screenshot, sample version, issue record, or approval artifact needed to reproduce and review the conclusion.

Decision owner and exceptions

State who may accept the result, which failures block acceptance, which deviations may be conditionally accepted, and who approves an exception.

Dimensions that usually need criteria

Business and functional outcome

Can each role complete the intended journey, and do rules, state changes, calculations, and exception messages match the confirmed requirements?

Data and integration outcome

Are fields, formats, mappings, deduplication, synchronization, retries, and reconciliation correct, and are discrepancies detectable and manageable?

Performance, capacity, and availability

Under agreed workloads and conditions, measure response, throughput, concurrency, resource use, sustained operation, and recovery rather than quoting a context-free number.

Security, access, and audit

Verify permitted and prohibited actions for each role, controls for sensitive operations, traceability, and disposition of known risks.

Operations and maintainability

Deployment, configuration, monitoring, alerting, backup, recovery, and support contacts must work well enough for the receiving team to continue operation.

Deliverable completeness

Source, configuration inventory, deployment instructions, operating guides, licenses, accounts, and training are checked against scope and tied to the accepted version.

From requirement to acceptance criterion

Fix the object and decision owner
Identify the scope, formal version, and person or group authorized to conclude acceptance.
Start from outcomes and risk
List the work users must complete, the failures that must not occur, and the business impact of failure.
Separate the scenarios
Write normal, boundary, exceptional, unauthorized, insufficient-information, and dependency-failure outcomes separately.
Choose measurement and thresholds
Set samples, environment, repetitions, calculation, and tolerance. Where quantification is unsuitable, define a clear human-review rubric.
Define evidence and stop conditions
Specify what proves each result and whether security, data corruption, unauthorized action, or another critical failure stops acceptance.
Review and version the agreement
Business, engineering, test, and project owners review understandability, feasibility, and cost before implementation.
Preserve traceability through change
Update requirements, criteria, tests, and issues together. Old results cannot prove a changed criterion or version passed.

Ambiguous wording versus testable wording

The right-hand side illustrates structure only. Project numbers must come from actual requirements, risks, and defensible measurements.

Not directly decidable	Ready to develop into tests
The system must be fast	Under the agreed environment and workload, named critical operations meet the target response time under the agreed calculation, with the test report retained.
AI answers must be accurate	On a frozen, stratified sample set, evaluate factuality, citations, insufficient information, sensitive cases, and high-risk actions separately, each with its own measure, threshold, and critical-failure rule.
Permissions work correctly	List allowed and prohibited actions for every role. Unauthorized access is blocked and audited; authorized actions update the correct business state.
The integration is complete	Test agreed fields and states for success, duplicate requests, timeout, invalid data, and dependency failure; mapping, retry, idempotency, and alert behavior follow the interface agreement.
Project documents were delivered	Check every item for name, version, readability, required access, and recipient; a designated receiving person uses the deployment guide in one controlled verification.

Turning results into an acceptance decision

Record each criterion, not only a total

Keep pass, fail, blocked, or not-applicable status with actual result and evidence. A total score must not hide a security, data, or core-business failure.

Give critical criteria hard gates

Unauthorized action, data corruption, incorrect business execution, or unrecoverable failure can be immediate stop or fail conditions instead of being averaged with low-risk results.

Close the issue loop

For failures, record severity, impact, owner, and fix version. Retest an identifiable object and cover the affected regression area before closing.

Make conditional acceptance explicit

If acceptance proceeds with open issues, record deviations, risks, temporary controls, owner, due date, and approver. Silence is not acceptance.

Bind the conclusion to a version

The record names the version, environment, sample, criteria version, execution date, and signatories. Material subsequent change triggers a new impact decision.

Additional criteria for enterprise AI

Does the sample represent real use and high-risk cases?
Stratify by business task, role, source quality, and risk, and record provenance, version, and limitations.
Inspect: sample inventory, labels, sampling method, and domain review.
Are different outcomes measured separately?
Factuality, citation, completeness, refusal, tool execution, and human handoff need separate measures and gates; one average accuracy figure cannot represent them all.
Inspect: scenario metrics, human rubric, and critical-failure list.
Is insufficient information an accepted test case?
When evidence is missing, conflicting, or out of scope, the system follows the agreed clarification, refusal, or handoff behavior instead of inventing an answer.
Inspect: insufficient-information set, expected actions, citations, and refusal records.
Do side-effecting actions have hard controls?
Order changes, messages, writes, approvals, or payments need criteria for parameters, access, confirmation, idempotency, audit, and failure recovery.
Inspect: tool permission matrix, action logs, duplicate and failure tests.
Are variability and version change handled?
Repeat runs where necessary, record model, prompt, knowledge, and parameter versions, and define which changes trigger reevaluation.
Inspect: run configuration, repeated results, version differences, and retest rules.
Is there human judgment and recourse?
High-risk or subjective-quality cases need named domain reviewers and a path for users to correct, appeal, or hand off an outcome.
Inspect: reviewer identity, rubric guidance, handoff records, and feedback closure.

Concepts commonly confused with acceptance criteria

Related concept	Difference
Requirement	A requirement states a needed capability or constraint. Acceptance criteria state how to decide whether a specified result satisfies it.
Acceptance plan	The plan organizes objects, people, environment, timing, samples, process, and decision. Criteria are the conditions the plan uses to judge results.
Test case	A test case specifies preconditions, input, and execution steps. Several cases may verify one criterion, and one case may provide evidence for several criteria.
Definition of Done	A Definition of Done is normally a team's general quality bar for every increment. Acceptance criteria are specific to a feature, deliverable, or project result.
Acceptance testing	Acceptance testing is the verification activity. Acceptance criteria are the decision basis against which its results are evaluated.
Service level objective	An SLO describes service performance over continued operation. Acceptance may reference it, but a one-time test cannot prove long-term performance.

Sources and scope

This entry explains acceptance criteria for enterprise software and AI projects. It does not replace a contract, procurement policy, sector regulation, or legal advice. The number of criteria, thresholds, priorities, sign-off method, and exception process must be agreed for the project. Criteria for one user story are not automatically the final acceptance terms for an entire project.