Skip to main content

23. Data Availability & State Compression

23.1 The Data-Availability Problem​

The repository contains erasure-coding, batching, compression, snapshot and pruning-oriented modules intended to improve data availability and state management. Their integration, reconstruction thresholds and failure guarantees depend on the active node path and network configuration.

23.2 Erasure-Coded Availability​

Each shard maintains a DA layer in which block data is erasure-coded into blobs. Erasure coding lets the full data be reconstructed from any sufficiently large subset of fragments, so data remains available even if some validators are offline. Cross-shard transactions (Sharding section) carry a Merkle proof of availability that the destination shard verifies without downloading the full blob — it checks that the data is committed and retrievable, deferring the actual fetch until needed. Large payloads travel as blob transactions (Mempool section) through this layer rather than inline in vertices.

23.3 Transaction Batching and Compression​

The batching and compression layers (quantos/src/batching/, quantos/src/compression/) reduce the bytes that must be propagated and stored. Transactions are grouped into batches for amortised verification and propagation, and payloads are compressed before storage. Combined with the two-tier signature aggregation (Cryptographic Primitives section), which shrinks committee signatures from megabytes to ~130 bytes, these layers attack the dominant bandwidth and storage costs of a post-quantum chain directly.

23.4 Quantum-Resistant State Compression (QRSC)​

The state-compression engine (state/compression.rs) applies several techniques specific to a PQC blockchain:

  • Temporal signature aggregation — aggregating signatures across time windows (epochs), not only across signers in one block, exploiting redundancy in a validator's repeated participation.
  • Semantic state-diff encoding — compressing state transitions by encoding their semantic delta rather than raw before/after bytes.
  • Quantum-resistant Merkle Mountain Range (QR-MMR) — an append-friendly accumulator with hash-based (and lattice) commitments for compact, quantum-safe history proofs.
  • Incremental compression — compression runs continuously without stop-the-world pauses, so it does not interrupt block production.

The compression code contains design targets for reducing state size and synchronization cost. No universal reduction percentage, sync speedup, CPU overhead, or patent status should be treated as a measured protocol guarantee without a reproducible benchmark and a verified legal/project statement.

23.5 Snapshot Sync and Pruning​

Compression dovetails with the storage layer: compact snapshots feed snapshot-sync so new nodes join quickly (Storage section), and archival pruning plus state rent (State Model section) bound the live working set. The net effect is that a validator's hot data — what it must serve at consensus speed — stays small even as the chain's total history grows without limit.