-
-
Notifications
You must be signed in to change notification settings - Fork 60
Description
Context
I'm using fjall as the storage backend for an embedded database project. During benchmarking, I noticed that Keyspace::drop() takes ~240ms on Windows even when using .temporary(true).
The Problem
When running benchmarks or tests that create/destroy many short-lived database instances, the synchronous journal flush on drop becomes a significant bottleneck.
Example scenario:
// Each iteration takes ~240ms just for the drop
for _ in 0..100 {
let ks = Config::new(&temp_path).temporary(true).open()?;
// do one operation
// drop happens here - ~240ms
}In comparison, RocksDB's equivalent drop takes ~8ms.
How RocksDB handles this
RocksDB provides avoid_flush_during_shutdown option:
By default RocksDB will flush all memtables on DB close if there are unpersisted data. The flush can be skipped to speedup DB close. Unpersisted data WILL BE LOST. DEFAULT: false
They also have manual_wal_flush for finer control over when WAL is flushed.
Reference: RocksDB Basic Operations Wiki
Request
Would it be possible to add a configuration option like skip_flush_on_drop(true) or avoid_flush_during_shutdown(true) that skips the synchronous journal persist when the keyspace is dropped?
This would be useful for:
- Test suites that create many temporary databases
- Benchmarks measuring operation throughput (not lifecycle)
- Scenarios where data loss on crash is acceptable
For production use cases with .temporary(false), the current behavior (flush on drop) is absolutely correct and should remain the default.
Current Workaround
For benchmarks, I restructured to use a shared database instance so the drop happens outside the timed section:
// Before: drop included in timing (~240ms per iteration)
b.iter_batched(
|| Config::new(&path).temporary(true).open().unwrap(),
|ks| {
// one operation
}, // ks dropped here - 240ms overhead!
BatchSize::SmallInput,
)
// After: drop outside timing (~40µs per iteration)
let shared_ks = Config::new(&path).temporary(true).open().unwrap();
b.iter(|| {
// operations on shared_ks
});
// shared_ks dropped once at the end, not per iterationThis works for benchmarks, but doesn't help for integration tests that need isolated database instances per test.
Thanks for the great library!