Inputs Are Infrastructure: Writing Data Contracts That Stop Silent Text Pipeline Decay
Photo: Andrej Cedilnika, Jeffrey Baumesb, Luis Ibanezb, Sean Megasonc, and Brian Wylied, Public domain, via Wikimedia Commons
Ask an NLP team what they monitor and the answer usually starts downstream: output validation, hallucination rates, latency percentiles, drift scores on embeddings. Ask what they monitor about the inputs and the answer is often a schema check on required fields — sometimes not even that.
This asymmetry is expensive. A large share of the production incidents that get filed as model regressions begin upstream: a field renamed without notice, a feed that stopped refreshing over a holiday weekend, an encoding change that turned quotation marks into mojibake, an upstream team that quietly widened a column and doubled average input length. The model behaved exactly as configured. The inputs did not.
Data contracts are the missing layer. Not a document that lives in a wiki, but an enforceable set of assertions about what arrives, when, in what shape, from whom, and what happens when the promise breaks.
What a Text Data Contract Should Specify
A workable contract has seven clauses. Teams that skip clauses rarely fail loudly; they fail in the middle of a quarter, in a specific customer's workflow.
1. Ownership. A named team, a named individual, and a backup. Shared ownership is the most common cause of unnoticed degradation, because everyone assumes the other party is watching the feed.
2. Structural schema. Field names, types, nullability, nesting depth, and maximum payload size. For text specifically, this includes encoding guarantees — UTF-8, normalized to a documented Unicode form — and an explicit statement about whether markup, HTML entities, or escaped characters may appear.
3. Semantic invariants. The properties a schema cannot express: expected language mix, median and 95th-percentile character length, expected proportion of empty or boilerplate strings, and acceptable duplicate rates. These are the checks that catch a broken scraper before it poisons a retrieval index.
4. Freshness. A maximum acceptable age for each record and a maximum acceptable lag for the feed as a whole, expressed as an SLO rather than a hope. Distinguish event time from ingestion time; conflating them hides backfills that overwrite recent data with old data.
5. Volume expectations. Expected range per hour and per day, with seasonal bands. A feed that drops to 40 percent of normal volume is often a partial outage rather than a quiet day.
6. Lineage. The upstream systems, transformations, and joins that produced the record, carried as metadata rather than reconstructed during an incident. If a downstream embedding index is rebuilt, lineage is what tells you which of six upstream changes is responsible.
7. Escalation path. Who is paged, at what threshold, through which channel, and what the consuming team is authorized to do in the meantime — fall back to a cached snapshot, degrade to a smaller corpus, or halt processing entirely.
Format Guarantees Are Not Pedantry
Text pipelines break in ways that numeric pipelines do not. Character encoding shifts produce invisible corruption that tokenizers handle inconsistently, inflating token counts and shifting embeddings into regions the model never saw during training. HTML that was previously stripped begins arriving raw after an upstream template change. Line endings shift from Unix to Windows conventions and split records unexpectedly. None of these raise exceptions. All of them degrade quality gradually, which is precisely why they are hard to attribute.
Write these guarantees into the contract and test them on arrival, not on a weekly report.
Decide Where to Fail
A contract without a defined failure response is documentation, not enforcement. Classify each check by severity.
- Blocking violations stop ingestion: schema breaks, encoding failures, missing required fields. Processing corrupt data usually costs more than pausing.
- Warning violations admit the data but raise a ticket and annotate the batch: volume dips within tolerance, mild freshness lag, elevated duplicate rate.
- Observational checks log only, and exist to build a baseline you can compare against later.
The severity map should be agreed upon by producer and consumer together. A consumer that unilaterally blocks ingestion will find itself in an argument during an outage; a producer that never sees the checks will keep breaking them.
The Ownership Gap Is the Real Risk
Most teams can write a schema. Fewer can answer the question that determines whether the contract survives contact with an organization: who is accountable when it fails at 2 a.m. on a Saturday?
Establish a lightweight cadence. Review contract violations monthly with both producer and consumer present. Track violation counts as a health metric for the relationship, not as a scorecard for blame. Give the consuming team a documented right to fall back, so that a broken contract degrades service gracefully instead of taking the product down.
Start With the Three Feeds That Scare You
Do not attempt a comprehensive data contract program across every pipeline at once. Pick the three feeds whose failure would be most embarrassing or most expensive, write contracts for those, and instrument them end to end. Add the freshness SLO and the escalation path first — those two clauses alone prevent the majority of silent outages.
Then expand deliberately. A text pipeline is only as trustworthy as the least specified input it accepts. Data contracts turn that vague trust into something a reviewer, an auditor, or an on-call engineer can actually verify.