Streams are always durable and totally ordered.
Every stream has a URI: s2://{basin}/{stream}. Basins can hold unlimited streams.
Appends always land at the tail, atomically – either all the records in a batch or none.
Reads can start from any position and optionally follow for live updates.
Positions
We talk about the head and tail of a stream.
The head is the beginning, and the tail is where the next record to be appended will go.
Each record on a stream has an associated sequence number and timestamp.
- sequence number increases strictly monotonically
- timestamp increases monotonically, and may be identical between records
The tail position is (next_seq_num, last_record_timestamp).Both will be 0 for an empty stream!
Lifecycle automation
You can automate the creation and deletion of streams. This is very useful for patterns with many dynamic streams, like one per session or job.
Creation
Streams can be created explicitly, or automatically on first use.
If a basin has create_stream_on_append or create_stream_on_read configured, streams will be created with the basin’s default config the first time they’re accessed.
Deletion
Streams can be configured to automatically delete once they become empty. The min_age_secs threshold requires the stream to have been empty for a minimum duration before auto-deletion kicks in.
See also
