Content-Addressable Storage

Design

Content in Chronicle is stored by its BLAKE3 hash. This provides:

• Deduplication: Identical content is stored once, regardless of how many files reference it
• Integrity: Content is verified on read (hash mismatch = corruption detected)
• Efficient sync: Only transfer content that the other side doesn't have

Storage Tiers

┌─────────────────────┐
│   Local CAS         │  Filesystem-backed, fastest
│   (~/.chronicle/cas) │
└──────────┬──────────┘
           │ miss
           ▼
┌─────────────────────┐
│      S3 CAS         │  Cloud-backed, shared across agents
│   (s3://bucket/cas)  │
└─────────────────────┘

TieredCAS

The TieredCAS combines local and S3 storage:

• Write: Store locally + async upload to S3
• Read: Check local first → fetch from S3 on miss → promote to local cache
• Pooled CAS: Optional shared local cache across agents (~/.chronicle/shared_cas/)

SocketCAS

For sandbox replication, content can be fetched over the replication socket:

• Sandbox requests content by hash from the agent
• Agent retrieves from its CAS and sends over the wire
• Avoids sandbox needing direct S3 access

Chunking

Large files are split into chunks for efficient transfer:

struct ChunkManifest {
    total_size: u64,
    chunk_size: u64,
    chunks: Vec<ChunkInfo>,  // hash + offset + size per chunk
}

When a ContentFetchResponse includes a chunk_manifest:

• The receiver fetches individual chunks in parallel by their hashes
• Chunks are reassembled on the receiving side
• Each chunk is independently content-addressed

Compression

All CAS content is compressed with Zstd before storage:

• Typical compression ratio: 3-5x for source code
• Transparent — compression/decompression happens at the CAS layer
• Both local and S3 storage use compressed content

ContentStorage Trait

class="token comment">#[async_trait]
pub trait ContentStorage: Send + Sync {
    async fn get(&self, hash: &str) -> Result<Option<Vec<u8>>>;
    async fn put(&self, hash: &str, data: &[u8]) -> Result<()>;
    async fn exists(&self, hash: &str) -> Result<bool>;
    async fn delete(&self, hash: &str) -> Result<()>;
}

Implementations: LocalFileCAS, S3CAS, SocketCAS, TieredCAS