Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@
.DS_Store
Cargo.lock
.idea/
mdbx-sys/target/
mdbx-sys/target/
docs/
fuzz/corpus/
fuzz/artifacts/
fuzz/target/
47 changes: 42 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,31 @@ DUP_SORT/DUP_FIXED methods validate flags at runtime:

- `require_dup_sort()` returns `MdbxError::RequiresDupSort`
- `require_dup_fixed()` returns `MdbxError::RequiresDupFixed`
- `debug_assert_integer_key()` validates key length (4 or 8 bytes) in debug builds

Methods requiring DUP_SORT: `first_dup`, `last_dup`, `next_dup`, `prev_dup`, `get_both`, `get_both_range`
Methods requiring DUP_FIXED: `get_multiple`, `next_multiple`, `prev_multiple`

### Input Validation Model

MDBX's C layer aborts the process (via `cASSERT`) on certain constraint
violations — notably INTEGER_KEY size mismatches and oversized keys/values.
These aborts cannot be caught.

Our validation model (in `src/tx/assertions.rs`):

- **Debug builds:** `debug_assert` checks catch constraint violations in
Rust before they reach FFI. This includes key/value size limits, INTEGER_KEY
length (must be 4 or 8 bytes), INTEGER_DUP length, and append ordering.
- **Release builds:** No checks are performed. Invalid input passes through
to MDBX, which may abort the process.
- **Benchmarks and fuzz targets:** MUST constrain inputs to valid ranges.
Do not feed arbitrary-length keys to INTEGER_KEY databases or oversized
keys/values to any database. The fuzz/bench harness is responsible for
generating valid input, not the library.

This is intentional. The library trusts callers in release mode for
performance. The debug assertions exist to catch bugs during development.

### Error Types

- `MdbxError` - FFI/database errors (in `src/error.rs`)
Expand All @@ -68,6 +88,7 @@ src/
codec.rs - TableObject trait
tx/
mod.rs
assertions.rs - Debug assertions for key/value constraints
cursor.rs - Cursor impl
database.rs - Database struct
sync.rs - Transaction impl
Expand All @@ -76,14 +97,29 @@ src/
sys/
environment.rs - Environment impl
tests/
cursor.rs - Cursor tests
transaction.rs - Transaction tests
environment.rs - Environment tests
cursor.rs - Cursor tests
transaction.rs - Transaction tests
environment.rs - Environment tests
proptest_kv.rs - Property tests: key/value operations
proptest_cursor.rs - Property tests: cursor operations
proptest_dupsort.rs - Property tests: DUPSORT operations
proptest_dupfixed.rs - Property tests: DUPFIXED operations
proptest_iter.rs - Property tests: iterator operations
proptest_nested.rs - Property tests: nested transactions
benches/
cursor.rs - Cursor benchmarks
cursor.rs - Cursor read benchmarks
cursor_write.rs - Cursor write benchmarks
transaction.rs - Transaction benchmarks
db_open.rs - Database open benchmarks
reserve.rs - Reserve vs put benchmarks
nested_txn.rs - Nested transaction benchmarks
concurrent.rs - Concurrency benchmarks
scaling.rs - Scaling benchmarks
deletion.rs - Deletion benchmarks
iter.rs - Iterator benchmarks
utils.rs - Benchmark utilities
fuzz/
fuzz_targets/ - cargo-fuzz targets (FFI/unsafe boundary hardening)
```

## Testing
Expand All @@ -109,3 +145,4 @@ This SHOULD be run alongside local tests and linting, especially for changes tha
- Modify build configuration
- Add new dependencies
- Change platform-specific code

23 changes: 22 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "signet-libmdbx"
description = "Idiomatic and safe MDBX wrapper"
version = "0.8.1"
version = "0.8.2"
edition = "2024"
rust-version = "1.92"
license = "MIT OR Apache-2.0"
Expand Down Expand Up @@ -45,6 +45,7 @@ tracing = "0.1.44"

[dev-dependencies]
criterion = "0.8.1"
libc = "0.2"
proptest = "1"
rand = "0.9.2"
tempfile = "3.20.0"
Expand All @@ -68,3 +69,23 @@ harness = false
[[bench]]
name = "deletion"
harness = false

[[bench]]
name = "cursor_write"
harness = false

[[bench]]
name = "reserve"
harness = false

[[bench]]
name = "nested_txn"
harness = false

[[bench]]
name = "concurrent"
harness = false

[[bench]]
name = "scaling"
harness = false
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,24 @@ NOTE: Most of the repo came from [lmdb-rs bindings].
- `sys` - Environment and transaction management.
- `tx` - module contains transactions, cursors, and iterators

## Input Validation

MDBX's C layer **aborts the process** on certain constraint violations,
such as passing an invalid key size to an `INTEGER_KEY` database or
exceeding the maximum key/value size. These aborts cannot be caught.

This crate uses a **debug-only validation model**:

- **Debug builds** (`cfg(debug_assertions)`): Rust-side assertions check
key/value constraints before they reach FFI. Violations panic with a
descriptive message, catching bugs during development.
- **Release builds**: No validation is performed. Invalid input passes
directly to MDBX for maximum performance.

**Callers are responsible for ensuring inputs are valid in release
builds.** The debug assertions exist to catch bugs during development,
not to provide runtime safety guarantees.

## Updating the libmdbx Version

To update the libmdbx version you must clone it and copy the `dist/` folder in
Expand Down
183 changes: 183 additions & 0 deletions benches/concurrent.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
#![allow(missing_docs, dead_code)]
mod utils;

use criterion::{BatchSize, BenchmarkId, Criterion, criterion_group, criterion_main};
use signet_libmdbx::{ObjectLength, WriteFlags};
use std::{
borrow::Cow,
hint::black_box,
sync::{Arc, Barrier},
thread,
};
use utils::{bench_key, bench_value, quick_config, setup_bench_env_with_max_readers};

const N_ROWS: u32 = 1_000;
const READER_COUNTS: &[usize] = &[1, 4, 8, 32, 128];

/// Max readers set high enough for the largest reader count plus criterion
/// overhead threads.
const MAX_READERS: u64 = 256;

fn bench_n_readers_no_writer(c: &mut Criterion) {
let mut group = c.benchmark_group("concurrent::readers_no_writer");

for &n_readers in READER_COUNTS {
let (_dir, env) = setup_bench_env_with_max_readers(N_ROWS, Some(MAX_READERS));
let env = Arc::new(env);
let keys: Arc<Vec<[u8; 32]>> = Arc::new((0..N_ROWS).map(bench_key).collect());
// Open the db handle once — dbi is stable for the environment lifetime.
let db = {
let txn = env.begin_ro_sync().unwrap();
txn.open_db(None).unwrap()
};

group.bench_with_input(
BenchmarkId::from_parameter(n_readers),
&n_readers,
|b, &n_readers| {
b.iter_batched(
|| Arc::new(Barrier::new(n_readers + 1)),
|barrier| {
let handles: Vec<_> = (0..n_readers)
.map(|_| {
let env = Arc::clone(&env);
let keys = Arc::clone(&keys);
let barrier = Arc::clone(&barrier);
thread::spawn(move || {
let txn = env.begin_ro_sync().unwrap();
barrier.wait();
let mut total = 0usize;
for key in keys.iter() {
let val: Cow<'_, [u8]> =
txn.get(db.dbi(), key.as_slice()).unwrap().unwrap();
total += val.len();
}
black_box(total)
})
})
.collect();
barrier.wait();
handles.into_iter().for_each(|h| {
h.join().unwrap();
});
},
BatchSize::PerIteration,
)
},
);
}
group.finish();
}

fn bench_n_readers_one_writer(c: &mut Criterion) {
let mut group = c.benchmark_group("concurrent::readers_one_writer");

for &n_readers in READER_COUNTS {
let (_dir, env) = setup_bench_env_with_max_readers(N_ROWS, Some(MAX_READERS));
let env = Arc::new(env);
let keys: Arc<Vec<[u8; 32]>> = Arc::new((0..N_ROWS).map(bench_key).collect());
// Open the db handle once — dbi is stable for the environment lifetime.
let db = {
let txn = env.begin_ro_sync().unwrap();
txn.open_db(None).unwrap()
};

group.bench_with_input(
BenchmarkId::from_parameter(n_readers),
&n_readers,
|b, &n_readers| {
b.iter_batched(
|| Arc::new(Barrier::new(n_readers + 2)),
|barrier| {
// Spawn readers.
let reader_handles: Vec<_> = (0..n_readers)
.map(|_| {
let env = Arc::clone(&env);
let keys = Arc::clone(&keys);
let barrier = Arc::clone(&barrier);
thread::spawn(move || {
let txn = env.begin_ro_sync().unwrap();
barrier.wait();
let mut total = 0usize;
for key in keys.iter() {
let val: Cow<'_, [u8]> =
txn.get(db.dbi(), key.as_slice()).unwrap().unwrap();
total += val.len();
}
black_box(total)
})
})
.collect();

// Spawn one writer inserting one extra entry.
let writer = {
let env = Arc::clone(&env);
let barrier = Arc::clone(&barrier);
thread::spawn(move || {
barrier.wait();
let txn = env.begin_rw_sync().unwrap();
txn.put(
db,
bench_key(N_ROWS + 1),
bench_value(N_ROWS + 1),
WriteFlags::empty(),
)
.unwrap();
txn.commit().unwrap();
})
};

barrier.wait();
writer.join().unwrap();
reader_handles.into_iter().for_each(|h| {
h.join().unwrap();
});
},
BatchSize::PerIteration,
)
},
);
}
group.finish();
}

/// Single-thread comparison: sync vs unsync transaction creation.
fn bench_single_thread_sync_vs_unsync(c: &mut Criterion) {
let (_dir, env) = setup_bench_env_with_max_readers(N_ROWS, None);
let keys: Arc<Vec<[u8; 32]>> = Arc::new((0..N_ROWS).map(bench_key).collect());

c.bench_function("concurrent::single_thread::sync", |b| {
b.iter(|| {
let txn = env.begin_ro_sync().unwrap();
let db = txn.open_db(None).unwrap();
let mut total = 0usize;
for key in keys.iter() {
total += *txn.get::<ObjectLength>(db.dbi(), key.as_slice()).unwrap().unwrap();
}
black_box(total)
})
});

c.bench_function("concurrent::single_thread::unsync", |b| {
b.iter(|| {
let txn = env.begin_ro_unsync().unwrap();
let db = txn.open_db(None).unwrap();
let mut total = 0usize;
for key in keys.iter() {
total += *txn.get::<ObjectLength>(db.dbi(), key.as_slice()).unwrap().unwrap();
}
black_box(total)
})
});
}

criterion_group! {
name = benches;
config = quick_config();
targets =
bench_n_readers_no_writer,
bench_n_readers_one_writer,
bench_single_thread_sync_vs_unsync,
}

criterion_main!(benches);
Loading