Data pipeline prompts should explain what one output row represents and how results are reconciled with the source. A job can finish successfully while duplicating a business event or silently excluding late data. These prompts help define a data contract, test meaningful invariants and plan reruns after partial failure. They apply to warehouse transformations and scheduled processing when the underlying tools and ownership are supplied.
Define grain, time and ownership first
Provide source and destination schemas, transformation code, schedule and the meaning of each timestamp. Explain whether a row represents an event, entity or aggregated period. Include the authoritative source and who owns data quality decisions. Without a clear grain, an agent may remove apparent duplicates that are actually distinct business events.
The dbt data-test documentation describes assertions about datasets and tests that identify failing records. It is one concrete implementation reference, not a requirement to adopt dbt. Use equivalent checks in your current stack and keep business meaning explicit.
Turn the dataset contract into checks
Ask which keys should be unique, which relationships must hold and which values are allowed to be missing. Include exceptions and their reason. A useful check identifies offending records and an owner, rather than simply producing a red dashboard. Keep high-volume failure output bounded while preserving enough information to investigate.
Define a data contract for {{output_dataset}} from {{schemas_and_code}}.
State the grain, keys, time semantics, null rules and business invariants.
For each rule propose a check, expected failure output and owner role.
Use {{sanitized_examples}} to show valid and invalid records. Separate
confirmed rules from assumptions. Implement checks in {{existing_tool}}
without silently discarding records that violate the contract.
Handle late and corrected data deliberately
A scheduled run needs a rule for events that arrive after its original window. Ask how the pipeline distinguishes event time from processing time and whether corrected source records replace earlier output. A larger lookback window may help but still needs a deduplication and reconciliation policy. Make the cost and remaining gap visible.
Review late-data handling in {{pipeline_paths}}.
Inputs arrive according to {{arrival_and_correction_pattern}}.
Trace event time, processing time, watermarks and output updates.
Identify records missed or double-counted by the current window.
Propose a policy with reconciliation checks and a small test dataset that
includes out-of-order events and a correction to an already processed row.
Plan reruns and backfills around partial output
A retry should not assume the previous attempt wrote nothing. Ask the agent to identify write boundaries and checkpoints. Define whether output is replaced, merged or appended, and how a failed batch is recognized. Include a way to stop and resume a long backfill while normal scheduled processing continues.
Plan a rerun or backfill for {{affected_window}} in {{pipeline}}.
Explain checkpointing, output write semantics and duplicate prevention.
Cover failure before output, partial output and failure after output but
before acknowledgement. Include concurrent scheduled runs, progress checks
and a stop/resume procedure. Reconcile the result against {{source_of_truth}}
without assuming equal row counts alone prove correctness.
Make completion an evidence-backed statement
After a run, check the expected grain and invariants alongside counts, timing and source reconciliation. Record the input window and code version so another person can reproduce the result. If an upstream gap remains, explain which output is affected instead of marking the pipeline healthy because its process exited successfully.
Save the approved pipeline prompt in Kipt with variables for dataset, window and write semantics. Use the database migration prompts when the task also changes the schema consumed by the pipeline.