Skip to main content

​
Durability

Every write to S2 is durable on object storage before it is acknowledged. Appends execute atomically — either all the records in a batch will become durable, or none. depends on the storage class for the stream:
  • Standard — 400ms
  • Express — 40ms
In either case, the data will be durable on disk in multiple availability zones of the cloud region.

​
Ordering

Records within streams are totally ordered. Every record gets a monotonically increasing sequence number. With an AppendSession (available via SDKs), you can pipeline batches with an ordering guarantee, and receive acknowledgements back in the corresponding order. If any batch fails, subsequent batches will not become durable.

​
Consistency

Stream operations are linearizable. This means that if an append has been acknowledged, any subsequent read or check-tail operation must reflect that write.
We test for this property using the Porcupine model checker. Read more →

​
Correctness

We are huge believers in deterministic simulation testing. S2 involves many moving parts, and investing in DST allows us to be confident about invariants that the system needs to always uphold. In short, deterministic testing allows us to run the different processes that comprise S2 over a simulated network. We can inject faults, crash processes randomly, trigger recovery procedures, and make sure that we never violate any of our commitments to durability and strong consistency. Our setup involves both a custom Rust framework that makes use of Turmoil, which we can run both as part of CI/CD, and in much larger “nightly” searches for subtle or rare regressions. We additionally use the Antithesis platform as a second form of DST on S2.