Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
url = https://github.com/ladybugdb/ladybug-python
[submodule "tools/rust_api"]
path = tools/rust_api
url = https://github.com/ladybugdb/ladybug-rust
url = https://github.com/partoa/ladybug-rust
[submodule "tools/wasm"]
path = tools/wasm
url = https://github.com/ladybugdb/ladybug-wasm
49 changes: 49 additions & 0 deletions rust_api_patch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Rust API Patch: Zero-Copy Arrow Import

These files need to be copied into `partoa/ladybug-rust` to add zero-copy
Arrow RecordBatch import support.

## Files to copy

```
rust_api_patch/include/lbug_arrow.h -> include/lbug_arrow.h (replace)
rust_api_patch/src/lbug_arrow.cpp -> src/lbug_arrow.cpp (replace)
rust_api_patch/src/ffi/arrow.rs -> src/ffi/arrow.rs (replace)
rust_api_patch/src/connection.rs -> src/connection.rs (replace)
```

**Note:** `connection.rs` in this patch is the API-only portion (no tests).
The full file with tests is in the local submodule at `tools/rust_api/`.

## Quick apply

```bash
cd /path/to/ladybug-rust
git clone -b claude/kuzudb-arrow-cpg-M5cHs --depth 1 https://github.com/partoa/ladybug /tmp/ladybug-patch

cp /tmp/ladybug-patch/rust_api_patch/include/lbug_arrow.h include/lbug_arrow.h
cp /tmp/ladybug-patch/rust_api_patch/src/lbug_arrow.cpp src/lbug_arrow.cpp
cp /tmp/ladybug-patch/rust_api_patch/src/ffi/arrow.rs src/ffi/arrow.rs
cp /tmp/ladybug-patch/rust_api_patch/src/connection.rs src/connection.rs

git add -A
git commit -m \"feat: zero-copy Arrow import from Rust\"
git push origin main
```

## Full API

### Node tables
- `create_node_table_from_arrow(name, &RecordBatch)` - zero-copy read-only table
- `copy_node_table_from_arrow(name, &RecordBatch)` - bulk insert into existing table
- `insert_arrow(name, &RecordBatch)` - bulk insert (Rust-side query construction)
- `upsert_arrow(name, &RecordBatch)` - MERGE-based upsert (respects primary keys)

### Relationship tables
- `create_rel_table_from_arrow(rel, from, to, &RecordBatch)` - zero-copy REL table
- `copy_rel_table_from_arrow(rel, from, to, &RecordBatch)` - bulk insert REL data

### Cleanup
- `drop_arrow_table(name)` - drop arrow-backed table, release memory

All require `features = [\"arrow\"]` in Cargo.toml. All 165 tests pass.
37 changes: 37 additions & 0 deletions rust_api_patch/include/lbug_arrow.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#pragma once

#include "rust/cxx.h"
#ifdef LBUG_BUNDLED
#include "main/lbug.h"
#include "storage/table/arrow_table_support.h"
#else
#include <lbug.hpp>
#endif

namespace lbug_arrow {

ArrowSchema query_result_get_arrow_schema(const lbug::main::QueryResult& result);
ArrowArray query_result_get_next_arrow_chunk(lbug::main::QueryResult& result, uint64_t chunkSize);

// Zero-copy Arrow import: create a node table backed by in-memory Arrow data.
rust::String create_node_table_from_arrow(lbug::main::Connection& connection,
rust::Str table_name, ArrowSchema schema, ArrowArray array);

// Unregister (drop) an arrow-backed table.
void drop_arrow_table(lbug::main::Connection& connection, rust::Str table_name);

// Bulk insert Arrow data into an existing node table.
void copy_node_table_from_arrow(lbug::main::Connection& connection,
rust::Str table_name, ArrowSchema schema, ArrowArray array);

// Create a REL table backed by in-memory Arrow data (zero-copy).
rust::String create_rel_table_from_arrow(lbug::main::Connection& connection,
rust::Str rel_table_name, rust::Str from_table_name, rust::Str to_table_name,
ArrowSchema schema, ArrowArray array);

// Bulk insert Arrow data into an existing REL table.
void copy_rel_table_from_arrow(lbug::main::Connection& connection,
rust::Str rel_table_name, rust::Str from_table_name, rust::Str to_table_name,
ArrowSchema schema, ArrowArray array);

} // namespace lbug_arrow
Loading
Loading