wal

The wal connector provides a write-ahead log implementation for use with transactional streams to deliver guaranteed delivery between a source and a downstream connected target system.

The connector detects and responds to circuit breaker and event acknowledgements to provide the guarantee of delivery through a persistent write ahead log it maintains and updates for each user event it sees.

Configuration

define connector my_wal from wal
with
  "config": {
    # required - directory where WAL storage is configured
    "path" = "state",

    # Maximum chunk size in bytes
    "chunk_size" = 1024,

    # Retention based on maximum number of chunks
    "max_chunks" = 100,
  }
end;