Skip to content

Feature request: Option to skip flush on drop for temporary/test databases #227

@lveillard

Description

@lveillard

Context

I'm using fjall as the storage backend for an embedded database project. During benchmarking, I noticed that Keyspace::drop() takes ~240ms on Windows even when using .temporary(true).

The Problem

When running benchmarks or tests that create/destroy many short-lived database instances, the synchronous journal flush on drop becomes a significant bottleneck.

Example scenario:

// Each iteration takes ~240ms just for the drop
for _ in 0..100 {
    let ks = Config::new(&temp_path).temporary(true).open()?;
    // do one operation
    // drop happens here - ~240ms
}

In comparison, RocksDB's equivalent drop takes ~8ms.

How RocksDB handles this

RocksDB provides avoid_flush_during_shutdown option:

By default RocksDB will flush all memtables on DB close if there are unpersisted data. The flush can be skipped to speedup DB close. Unpersisted data WILL BE LOST. DEFAULT: false

They also have manual_wal_flush for finer control over when WAL is flushed.

Reference: RocksDB Basic Operations Wiki

Request

Would it be possible to add a configuration option like skip_flush_on_drop(true) or avoid_flush_during_shutdown(true) that skips the synchronous journal persist when the keyspace is dropped?

This would be useful for:

  • Test suites that create many temporary databases
  • Benchmarks measuring operation throughput (not lifecycle)
  • Scenarios where data loss on crash is acceptable

For production use cases with .temporary(false), the current behavior (flush on drop) is absolutely correct and should remain the default.

Current Workaround

For benchmarks, I restructured to use a shared database instance so the drop happens outside the timed section:

// Before: drop included in timing (~240ms per iteration)
b.iter_batched(
    || Config::new(&path).temporary(true).open().unwrap(),
    |ks| {
        // one operation
    }, // ks dropped here - 240ms overhead!
    BatchSize::SmallInput,
)

// After: drop outside timing (~40µs per iteration)  
let shared_ks = Config::new(&path).temporary(true).open().unwrap();
b.iter(|| {
    // operations on shared_ks
}); 
// shared_ks dropped once at the end, not per iteration

This works for benchmarks, but doesn't help for integration tests that need isolated database instances per test.

Thanks for the great library!

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions