weevil

A tiny toy implementation of core TigerBeetle concepts in Rust. Built as a learning exercise in systems programming — zero-copy I/O, non-blocking network, append-only durable storage, and static memory allocation.

What it is

Weevil is a single-threaded TCP server that accepts financial transfers from concurrent clients, appends them to a single append-only WAL file, and responds with account balance information. Each transfer atomically debits one account and credits another — both sides land in the WAL as a single 64-byte record. It is not a real financial system.

Concepts explored

• Zero-copy wire protocol — all messages are fixed 64-byte #[repr(C)] structs cast directly from network buffers using bytemuck. No serialization layer. Responses are the same: a fixed 64-byte AccountResponse struct cast directly to the wire.
• Non-blocking I/O — a single-threaded mio event loop handles multiple concurrent connections without threads.
• Append-only WAL — all transactions are written as raw bytes to a single wal.log file. One fdatasync per event loop batch regardless of how many accounts were touched. Periodically checkpointed: account balances are snapshotted to a checkpoint file via atomic temp-file rename, then the WAL is truncated. On startup, the checkpoint is loaded first and the WAL tail is replayed on top.
• Batch commit with response ordering — transactions accumulate across one poll iteration. At the end of each iteration, all pending transactions are written to the WAL and flushed with a single fdatasync. Only after the flush completes are sessions promoted from AwaitingCommit to Writing, guaranteeing no client receives a response before its transaction is durable on disk.
• Separate debit and credit accumulators — AccountEntry tracks debit_balance: u128 and credit_balance: u128 as independent unsigned accumulators rather than a single signed balance. Unsigned types cannot go negative, transaction volume is preserved in both directions, and the net balance is derived by comparison and safe subtraction at display time. Matches the TigerBeetle model. (64-Bit Bank Balances 'Ought to be Enough for Anybody'?)
• Batched disk writes — all pending transactions for an account are written in a single write_all(bytemuck::cast_slice(...)) call rather than one syscall per transaction. bytemuck::cast_slice reinterprets the contiguous [Transfer; N] array as a flat &[u8] with no copying.
• Balance replay — on startup, account balances are restored from the checkpoint file (if present), then the WAL is replayed 64 bytes at a time to recover any transactions that postdate the last checkpoint.
• Static connection table — connections are stored in a fixed [Option<Session>; MAX_CONNECTIONS] array. The mio Token is a direct array index. No HashMap, no hashing, no pointer chasing — O(1) lookup by design.
• Static account cache with open addressing — accounts are stored in a fixed [Option<AccountEntry>; MAX_ACCOUNTS] array. Slot selection uses modulo hashing with linear probing and full wrap-around — no HashMap, no heap allocation. MAX_ACCOUNTS is prime (257) to reduce probe clustering.
• Atomic double-entry transfers — each Transfer carries both a debit_account_id and credit_account_id. Both sides of the transfer are written to the WAL as a single 64-byte record and applied together on flush. There is no moment where one account has been debited but the other has not been credited — the WAL record is either fully replayed or not at all.
• Cache-level pending transfer buffer — AccountEntryCache holds a single [Transfer; MAX_BATCH] array shared across all accounts. All in-flight transfers accumulate in one buffer regardless of which accounts they touch. One buffer, one write_all, one fdatasync per event loop batch.
• Type-state response buffer — SessionStatus::AwaitingCommit([u8; 64]) and Writing([u8; 64]) carry the response payload inside the state. The type system enforces that a session cannot be in Writing state without a response ready to send. No separate write_buf field, no Option to unwrap.
• CRC32 checksums — every wire message and every log record carries a CRC32 checksum in repurposed padding bytes. Computed in new() with the checksum field zeroed, verified on network ingress and during startup log replay. Table-free, no dependencies

Protocol

All messages are 64 bytes. Client-to-server messages are distinguished by the final byte (message_kind). Server-to-client responses are always AccountResponse.

Client → Server

Byte offset	Field	Type
Transfer (message_kind = 1)
0–15	amount	u128
16–23	debit_account_id	u64
24–31	credit_account_id	u64
32–35	checksum	u32
36–62	padding	[u8; 27]
63	message_kind = 1	u8
Account (message_kind = 0)
0–7	account_id	u64
8–11	checksum	u32
12–62	padding	[u8; 51]
63	message_kind = 0	u8

Server → Client

Byte offset	Field	Type
0–15	debit_balance	u128
16–31	credit_balance	u128
32–39	account_id	u64
40–43	checksum	u32
44–62	padding	[u8; 19]
63	status	u8

The response reflects the committed balances at the previous flush boundary — the pending transfer has been accepted into the batch but balances are updated when the batch is written to disk, not at enqueue time. For transfers the response reflects the credit account's balance.

status	Meaning
0	Success
1	Account not found
2	Account cache full

Running

# start the server (creates ./data_files/ automatically on first run)
cargo run --bin server

# run the test client (NUM_THREADS concurrent connections, NUM_TRANSACTIONS each)
cargo run --bin client <NUM_THREADS> <NUM_TRANSACTIONS>

The client registers each account, sends a series of random transfers, then queries the final balance. ./data_files/wal.log and ./data_files/checkpoint persist across restarts.

Performance

Measured on a MacBook Pro (Apple Silicon) with NUM_THREADS varied and total transfers fixed at 10,000. All transfers are fully durable — each batch is flushed with fdatasync before any client receives a response.

Threads	Transfers/thread	Total transfers	TPS (avg 3 runs)
10	1,000	10,000	~1,252
20	500	10,000	~2,531
40	250	10,000	~4,976
100	100	10,000	~10,787
200	50	10,000	~11,793
250	40	10,000	~12,227
251	1000	251,000	~13,058

TPS scales with concurrency because fdatasync cost is amortized across all transfers that arrive during a single event loop iteration. At low concurrency (10 threads), transfer batches range between 1 and 9 transfers per sync. At ~200 threads transfer batches grow to about 56-72.

Assuming the following configuration (I landed on these arbitrarily), the peak memory footprint is 1.2MB regardless of the throughput.

// Maximum number of connections the server will accept
pub const MAX_CONNECTIONS: usize = 256;

// Maximum number of accounts to handle
pub const MAX_ACCOUNTS: usize = 257; // prime number to reduce probe clustering

// Maximum size of the WAL file before a snapshot
pub const MAX_WAL_SIZE: u64 = 1024 * 1024; // 1MB max

// Maximum number of transfers to store in memory before flushing to disk
pub const MAX_BATCH: usize = 1000;

What it is not

Weevil omits most of what makes TigerBeetle production-worthy: O_DIRECT, cluster replication, and anything resembling fault tolerance. It is a learning artifact.

Transaction history is not preserved — the WAL is truncated after each checkpoint. Only current account balances survive a restart. There is no audit log, no way to replay individual transactions, and no mechanism to answer "what happened to this account."

Next Steps

• Transfer TimeStamps — add a timestamp: u64 field to Transfer, repurposed from padding. Allows temporal reconstruction of account balance change history. This won't matter much with current checkpointing behavior, but once a storage engine is in place it will be useful.

• Transfer IDs for idempotency — add a txid: u64 field to Transfer, repurposed from padding. Clients assign an ID to each transfer; the server echoes it back in AccountResponse. Duplicate submissions with the same ID can be detected and rejected, making retries safe.

• Testing — coverage is thin. Missing: checksum rejection on corrupt data; WAL-only replay (no checkpoint); checkpoint + WAL tail replay; partial checkpoint.tmp doesn't corrupt state on restart; MAX_BATCH boundary (100th transaction succeeds, 101st returns PendingTransactionsFull); account cache full response; open-addressing correctness under hash collisions.