Long Django import: choose all-or-nothing or checkpointed batches explicitly

Agree what failure should leave behind

A business owner needs a large import to stop blocking ordinary work and to recover after a bad row. First decide whether every row must succeed together or earlier validated batches may remain committed. These are different business rules. Moving work out of an HTTP request can avoid a request timeout, but it does not automatically shorten database transactions or define recovery.

• Use a permitted synthetic import with a known invalid row.
• Record whether external notifications, files or cache changes accompany imported rows.

Keep slow preparation outside writes when possible

Django 5.2 advises keeping transactions short because open transactions have a database cost. Where consistency permits, fetch, parse and validate data before entering the write transaction. If the whole import truly must be atomic, retain that boundary and assess its operational cost rather than silently weakening the rule. No universal batch size or performance improvement is established by this guide.

• Measure ordinary-page responsiveness during the agreed sample import.
• Do not hold a write transaction open while waiting on an avoidable remote request.

Nested atomic is not a checkpoint

An inner atomic block normally creates a savepoint inside the outer transaction. Exiting that inner block does not commit a batch independently; a later outer rollback can still undo it. Real checkpointed batches need separate committed transaction boundaries and an explicit record of completed work. Agree stable row identity and restart behaviour so reprocessing a batch does not create unintended duplicates.

• If partial completion is allowed, show exactly which batches remain after a controlled failure.
• Database rollback does not restore in-memory objects or cache; reconcile or defer those effects separately.

Acceptance includes interruption and restart

Test a successful synthetic import, a failure at the agreed boundary and a resumed attempt. Compare final records with expected counts and content, and check ordinary pages during processing. This acceptance proposal is not a benchmark or evidence that any client import was delivered. Send framework/database versions, approximate size and redacted symptom initially; private data, worker execution and scope remain separately gated.