Enterprise AI Project Wiki · Requirements and business scenarios
Data requirement
Also calledData requirement · Data requirements · Data specification requirement · Information data requirement
A data requirement is a normative statement of what data a specified system or process must use or produce for an explicit business purpose, actor, state, time, and operating condition, and what definition, source, structure, quality, provenance, access, protection, exchange, and lifecycle conditions that data must satisfy. It turns business, system, risk, and compliance needs into designable, verifiable, and traceable data obligations.
A data requirement states what data must satisfy; it does not begin with table design
Data requirements first establish why a business or system needs data, who produces it, which source is authoritative, in what state it is used, and what quality and protection conditions apply. Modelling, storage, and transport choices follow. Starting with existing database columns can turn an implementation into assumed business truth and omit documents, messages, external master data, human evidence, and derived results.
Data requirements may be a class within system or software requirements or a controlled set of their own. NASA software-requirements guidance includes internal data, databases, inputs and outputs, validation, and error handling; its software data dictionary captures variables, formats, units, ranges, and structures. The requirement states the obligation; the dictionary is one information item for definitions.
Reliable creation of a purchase request needs functional and interface behaviour, but also an authoritative supplier identity, amount and currency semantics, missing and conflicting-data handling, state history, audit provenance, retention, and disposal. Data requirements link to those requirements without replacing them.
Establish data objects, sources, and responsibility boundaries first
The same customer status may appear in CRM, contracts, and a manual note, each with a different owner and update time. Establish the business object, authoritative source, and correction authority before choosing fields or storage. A tidy schema can otherwise preserve the wrong meaning very reliably.
Object and granularity
Inventory entities, events, states, documents, messages, reference data, configuration, logs, and derived results. State what one record represents, how it is identified, and how it may be split or aggregated.
Producer, authority, custodian, and consumer
Separate the supplier, the business role authorised to decide definition and quality, the technical custodian, and authorised consumers. Database administration does not confer business-data authority.
Authoritative source and copies
Identify which system, document, or authorised decision governs each attribute or state, when copies synchronise, how conflicts are decided, and the point in time at which caches and reports are valid.
Raw, transformed, and derived states
Preserve relationships from original input through cleansing, normalisation, aggregation, scoring, or human correction, and state which layer is permitted for a decision.
Boundary and external dependency
Distinguish customer-supplied, third-party, system-generated, and exported data. Rights, availability, format change, and supplier exit cannot remain implicit.
Sensitivity and applicability
Under authorised policy, classify personal, commercial, credential, security, restricted, and public data and its territory, tenant, role, and purpose, then link each class to controls.
Dimensions normally covered by a complete data-requirement set
A field type says what data looks like, not whether it can be obtained, is fresh enough, may be used, must be deleted, or can be traced after an error. Complete requirements follow purpose and lifecycle so data serves the task without escaping rights and accountability.
Definition and business semantics
Name, meaning, rule, enumeration, unit, precision, time zone, language, code list, validity period, and synonyms; split definitions where one word has different contextual meanings.
Structure and relationships
Type, length, cardinality, conditional presence, keys, entity relationships, hierarchy, order, file or message layout, and integrity constraints without prematurely mandating a database product.
Source, collection, and validation
Channel, identity, authority, time, evidence, accepted format, validation rules, and treatment of invalid, duplicate, late, conflicting, or insufficient data.
Quality and fitness for use
For each purpose, specify applicable accuracy, completeness, consistency, timeliness, uniqueness, validity, or other characteristics, with scope, sample, measure, threshold, severity, and action.
Time and state
Event and recording times, effective and expiry dates, version, current and historical state, late correction, and temporal order. One updated-at field cannot carry every time meaning.
Provenance and traceability
Record source, acquisition, transformations, processing version, human changes, and destinations so a material result can be explained, reproduced, corrected, and impact-assessed.
Access, use, and protection
State who may read, create, amend, approve, export, or delete for what purpose, plus isolation, minimisation, masking, encryption, audit, and unauthorised outcomes.
Exchange and interoperability
Direction, trigger, format, semantics, units, version, batch, acknowledgement, error, duplicate, and compatibility conditions, with transport responsibility traced to interface requirements.
Volume and performance conditions
Volume, growth, peaks, file size, history depth, query and batch windows, and the capacity-review trigger when assumptions change.
Retention, archive, migration, and disposal
Use authorised policy to define retention start, period, legal hold, archive readability, migration reconciliation, deletion propagation, destruction evidence, and exceptions; do not default to indefinite storage.
How to make one data requirement decidable
“Ensure accurate and complete data” is not actionable because accuracy and completeness differ by use. A decidable statement identifies the object, scenario, source, quality rule, time, exception treatment, and the person and evidence that confirm it.
Stable ID, subject, object, and purpose
Identify the obligated system or role, data object, and approved purpose. The same data may need different quality, access, and retention for different purposes.
Condition, action, and normative result
State trigger, state, inputs, and required collection, creation, validation, preservation, rejection, transformation, or disposal outcome; avoid support data management.
Controlled-definition reference
Reference an explicit dictionary, code-list, schema, interface, classification, or rule version. An unapproved reference is an open item with owner and due date.
Quality and time criterion
Give population, window, statistic, threshold, exception, and severity. Record unknown numbers as controlled parameters such as Q1 or T1 instead of inventing defaults.
Exception, conflict, and safe outcome
Define whether null, duplicate, invalid, stale, conflicting, migrated, or unavailable-source data is blocked, quarantined, labelled, rolled back, or referred to a person.
Source, rationale, authority, and verification
Trace to a business rule, system requirement, risk, contract, or authorised policy; identify definition and approval ownership and inspection, analysis, demonstration, or test evidence.
Traceability and change impact
Link functions, interfaces, reports, models, tests, migrations, and controls producing or consuming the data, so definition, source, or purpose changes expose affected objects.
Quality is purpose-specific; provenance must support accountability and correction
An address used to deliver a contract needs different precision from one used for city-level statistics. Define quality around the decision and consequence of error. Provenance then shows where a value came from, how it changed, and which results it influenced so correction can propagate.
ISO/IEC 25012 treats data quality as the degree to which characteristics satisfy stated and implied needs under specified conditions. High-quality data is not a criterion by itself: an address used for delivery, deduplication, analytics, or identity checking has different needs. State the use and consequence before selecting characteristics and thresholds.
The NIST Research Data Framework emphasises metadata, provenance, lifecycle, and fitness for purpose. For material data, provenance should answer who or what acquired it, when, under which version and transformation, what human and automated changes occurred, and which downstream outputs used it.
Findings need operational treatment: block, quarantine, preserve the original, initiate correction, notify an accountable role, reprocess downstream consumers, accept an authorised exception, or revoke a result. A detached dashboard does not demonstrate conformance.
- Does every metric serve a defined use?
Identify object, scenario, population, window, and error consequence.
Check: purpose–characteristic–criterion mapping.
- Can a result be traced to source and processing version?
Retain sufficient source, rule, time, and operator information.
Check: sample reconstruction and impact analysis.
- Does failure trigger the right business action?
Map severity to blocking, quarantine, alert, remediation, or acceptance.
Check: exception-path tests and closure records.
Every lifecycle stage needs accountable ownership and evidence
Data acquired lawfully is not automatically reusable for every purpose or retainable forever. Acquisition, validation, sharing, archive, correction, and deletion have different responsibilities, and caches, indexes, backups, and downstream copies must follow relevant changes.
Acquisition and authority
Confirm source, method, purpose, notice or rights, minimum fields, initial quality, and receipt evidence. Technical access does not grant a new use.
Storage and use
Define the authoritative copy, isolation, access, amendment, concurrency, consistency, audit, and permitted analysis, including test, support, and reporting environments.
Sharing and publication
Limit recipient, fields, purpose, territory, format, and onward distribution; record transfer, acknowledgement, rejection, and withdrawal. Authority determines openness.
Retention and archive
Let authorised business, records, contract, and legal sources set period, start event, holds, and format; prove information remains findable, readable, and interpretable.
Migration and consolidation
Define mappings, scope, cleansing, duplicate and conflict rules, count and amount reconciliation, sampling, rollback, approval, and source freeze.
Deletion and retirement
Cover production, replicas, caches, indexes, exports, logs, and controllable third parties. Explain delayed backup or record deletion, access restriction, expiry, and evidence.
Derive, verify, and change data requirements under control
Business decisions, rules, functions, risks, and interfaces all generate data requirements. Profile representative records to test assumptions and feed discovered problems back into rules and process. When source or purpose changes, assess reports, models, interfaces, and retention—not only one table.
Work backwards from decisions
For each decision, transaction, compliance claim, and operation, identify necessary facts, authorised confirmation, and the consequence of error, absence, or staleness.
Derive from process and interfaces
Follow normal, exception, cancellation, and recovery paths to find every input, output, history, and reconciliation item. Exchanging parties share semantics and version.
Derive from risk and policy
Translate fraud, access, privacy, security, records, and continuity needs into minimisation, protection, provenance, retention, stopping, and evidence duties confirmed by authorised roles.
Verify definition and structure
Use review, schema checks, code-list reconciliation, examples, and boundary values to prove definitions unambiguous, relationships consistent, and consumers aligned.
Verify quality and transformation
Use approved representative samples, known answers, reconciliation, statistics, and replay, separating source defects from processing defects.
Verify access and lifecycle
Test allowed and forbidden paths by role, purpose, tenant, region, and state, and prove archive, migration, hold, deletion, and third-party propagation.
Control change
Before approval, analyse definition, source, schema, threshold, purpose, access, or period changes across producers, consumers, history, interfaces, models, reports, tests, and migration.
Constructed procurement-request data requirements
This customer-independent procurement data shows purpose, authority, quality, retention, and deletion combined into verifiable requirements. Q1, R1, M1, and similar values remain open because business, risk, and data owners must approve real thresholds from evidence.
DR-ID-201 — request identity
A formal request has a globally unique, non-reusable identifier generated by the request service and a separate user-facing number. Draft, formal, and ERP IDs remain traceable; display numbers do not establish internal identity.
DR-MD-202 — supplier authority
At submission, supplier identity and active state come from approved master-data version M1; the name is a display snapshot. With no unique active match or unavailable master data, retain the draft and reason.
DR-SEM-203 — amount semantics
Each amount retains currency, tax inclusion, precision and rounding, plus exchange-rate source and time where applicable. Controlled detail rules recalculate totals; a difference above pending tolerance Q1 blocks submission.
DR-STATE-204 — state history
Every transition records old and new state, event and record times, authorised actor, reason, and rule version. A correction is a linked new event, not an invisible overwrite.
DR-PROV-205 — evidence provenance
Attachments and extracted fields retain source identity, content digest, channel, extraction or entry method, processing version, and confirmation state so decisions lead back to evidence.
DR-MIG-206 — migration reconciliation
Batches retain source IDs and transformation outcomes and reconcile counts, states, amount totals, and risk samples. Authority does not switch until discrepancies close.
DR-LIFE-207 — retention and disposal
Requests and evidence follow the approved record category and R1 start rule; holds suspend deletion. At expiry, the process covers controllable copies and keeps evidence without reproducing content.
Enterprise AI needs separate requirements for training, retrieval, inference, evaluation, feedback, and log data
The label “AI data” hides changes of purpose. A conversation may provide context only for one inference, then enter logs, human review, evaluation, or later training. Every move into another use requires renewed decisions about authority, period, minimisation, and exit.
Separate data by purpose
Define source, use, access, period, and prohibited reuse separately for training corpora, retrieval knowledge, live context, tool I/O, evaluation sets, human feedback, and logs.
Provenance, rights, and version
Record source, rights or authority, acquisition time, processing, version, and scope. Constrain and evidence whether an external model retains or trains on data.
Representativeness and task fitness
Tie criteria to task, language, role, time, and risk. Manage weak representation, label disagreement, and coverage gaps as limitations; record count is not fitness evidence.
Retrieval evidence and freshness
Knowledge retains authority, version, effective dates, access, and indexing state. Used passages remain traceable; stale, conflicting, or insufficient material triggers limitation or human handling.
Minimisation, isolation, and prompt injection
Pass task-required fields only and enforce access by user, tenant, and purpose. Treat external content and model output as untrusted data whose instructions cannot alter authority.
A useful conversation or correction does not automatically become training material
Production logs first support operations, security, and incident investigation and may contain personal information, contracts, or temporary context. Reusing conversations, corrections, or failures for evaluation or training needs a new purpose, authority, screening, minimisation, and version. Possible future improvement does not justify indefinite retention.
Label synthetic and generated data
Record generation method, source relationship, model and prompt version, review, and permitted uses so generated content is not treated as independent fact or unknowingly recycled.
Correction, deletion, and propagation
Define how withdrawal, individual requests, rights expiry, or corrections update indexes, caches, evaluations, and controllable derivatives; document irreversible effects and compensating decisions.
Combination version and monitoring
Link dataset, split, labels, knowledge snapshot, model, prompt, and rules to evaluation and production. Distribution, source, or purpose changes trigger review.
How data requirements differ from neighbouring concepts
A data model, database schema, quality rule, and privacy policy may implement or constrain data requirements, but they are not the same level. Requirements first state what business data is needed and which conditions it must satisfy; design then chooses organisation and storage, preserving responsibility through platform change.
| Concept | Boundary from a data requirement |
|---|---|
| Business data | Facts and records used or produced in business; data requirements state obligations for a specified system and purpose. |
| Data model or database design | Organises entities, schemas, tables, indexes, and physical implementation. It implements requirements but cannot supply business semantics, quality, access, or lifecycle authority by itself. |
| Data dictionary | A controlled information item for names, definitions, types, formats, units, ranges, and relationships. Its existence does not complete source, purpose, or lifecycle duties. |
| Database requirement | Focuses on database information, access, capacity, integrity, and retention. Data requirements also cover files, messages, human evidence, external sources, and non-persistent I/O. |
| Data-quality requirement | One class specifying purpose-specific quality criteria. The full set also covers semantics, sources, relationships, access, exchange, and lifecycle. |
| Metadata and lineage | Metadata describes data and lineage records origins and processing. They support discovery and traceability but are not the whole requirement set. |
| Master data | Core entities consistently identified across processes. Governance sets authority; a project requirement references identity, version, quality, and use rules. |
| Interface requirement | Defines exchange and bilateral boundary responsibility. A data requirement defines content meaning, quality, authority, and lifecycle; they cross-reference. |
| Test data | Inputs, states, and expected results for verification. It represents requirements and needs protection, but is not the production requirement or automatically authorised production copy. |
| Training data | Data used to train or adjust model parameters, one AI-data purpose. Requirements also cover retrieval, inference, evaluation, feedback, logs, and business data. |
| Privacy and security requirement | States rights, risks, and controls for protected data. Data requirements apply these to objects and stages but do not establish legal conclusions or a full security design. |
| Backup | A replica mechanism for recovery. Requirements state recovery, integrity, access, retention, and deletion boundaries; backup is not authority, archive definition, or migration plan. |
Sources and scope
This entry explains data requirements in enterprise system, software, and AI projects: verifiable provisions for the data a project must receive, create, identify, validate, transform, exchange, store, protect, retain, migrate, archive, and dispose of, and for the semantics, source, quality, time, provenance, access, and lifecycle conditions that data must satisfy. It does not fully define business data, data models or database design, data dictionaries, master-data governance, metadata management, interface requirements, a data-quality management system, privacy or security policy, backup design, test data, training data, document templates, or particular legal duties. Authorised project roles must confirm the actual data objects, authoritative sources, quality thresholds, legal basis, locations, retention periods, and approval rights.
- ISO/IEC/IEEE 29148:2018 — requirements engineering, data in system and software specifications, requirement quality, verification, traceability, and change; confirmed current in 2024
- NASA Software Requirements Specification — internal and external data, databases, I/O, structures, access, integrity, retention, validation, and error handling
- NASA SWE-110 Software Data Dictionary — I/O variables, formats, units, raw and calibrated data, telemetry layouts, commands, and operating limits
- NASA Software Data Dictionary guidance — tables, fields, types, lengths, defaults, nulls, constraints, entry, transformation, output, update, and distribution
- ISO/IEC 25012:2008 — a general structured-data quality model and basis for requirements, measures, and evaluation
- NIST SP 1500-18r2 Research Data Framework — metadata, provenance, lifecycle, fitness for purpose, quality, retention, and disposition
- NIST SP 1500-6r2 — quality curation, updates, provenance, validation, annotation, preservation, persistent identity, standardisation, and lifecycle
- NIST data-provenance definition — the process and record of origins, transmission, and methods from data generation onward
- NIST Privacy Framework — managing data processing, purpose, roles, risk, protection, and lifecycle privacy activities
- NIST AI RMF Core — AI data and metadata, provenance, representativeness, quality, privacy, security, third parties, monitoring, and change