Skip to content

title: "Bulk import validation" description: "Bulk import validation is performed in multiple phases before and during execution." Validation is designed to detect structural and data-level issues witho...

Bulk import validation

Bulk import validation is performed in multiple phases before and during execution.
Validation is designed to detect structural and data-level issues without blocking valid rows from being written.

Bulk import does not use the same validation model as interactive writes.

When validation occurs

Validation occurs in two distinct phases:

Structural validation

Structural validation is performed immediately after the ZIP file is uploaded.

It validates:

  • ZIP integrity
  • CSV format (RFC 4180)
  • file naming against table API names
  • presence of mandatory columns
  • column names against the active schema

Structural validation failures block the import from continuing.

Dry run validation

After structural validation passes, a dry run is executed on temporary tables.

The dry run validates:

  • identifier resolution
  • relation resolution
  • data type conversion
  • row-level mutation outcomes

No data is written to live tables during the dry run.

The same validation logic is applied again during execution.

How violations are reported

Validation violations are reported at the row level.

Characteristics:

  • each error is associated with a specific row
  • errors are collected independently per row
  • errors do not block evaluation of other rows
  • error reports are downloadable after dry run and after execution

Violations detected during dry run do not prevent scheduling.

Violations detected during execution do not roll back successful rows.

Partial vs full failure semantics

Bulk import uses partial failure semantics.

This means:

  • structural validation failures stop the import entirely
  • row-level validation failures affect only the failing rows
  • successful rows are written even if other rows fail
  • execution is non-transactional across rows, files, and tables

There is no automatic retry or rollback mechanism.

Failed rows must be corrected and re-imported manually in a new import.

Common mistakes

Common validation-related mistakes include:

  • assuming write rules are enforced
  • expecting dry run success to guarantee execution success
  • assuming the import is transactional
  • expecting failed rows to block execution
  • attempting to reuse or modify an existing import

Bulk import validation is intentionally permissive to allow large, imperfect data sets to be ingested safely.

Related concepts