Show HN: Minarrow – Fast Rust columnar engine with Apache Arrow compatibility
I built it for use cases like:
- Structured tick/quote/event ingestion in HFT - Native async streaming (Tokio, etc.) - Edge and embedded systems that need zero-copy structured memory - SIMD-native kernels (all buffers are 64-byte aligned) - polyglot apps
Minarrow gives you:
- Arrow-compatible layout and schema support - <1.5s build times - Always-typed data access (no downcasting, no Any) - Async IPC and memory-mapped file support (via a sibling crate) - .to_arrow() and .to_polars() for interop
It doesn’t try to replace the Arrow ecosystem—but for Rust-first, performance-sensitive systems, it offers Arrow compatibility without the abstraction tax.
Repo: https://github.com/pbower/minarrow Docs: https://docs.rs/minarrow/latest/minarrow/
Benchmarks (full results in repo): Minarrow sums 1k integers in 88 ns (direct), within a few ns of raw Vec (64 ns) and ~2× faster than Arrow-rs struct (162 ns). On 1B integers with SIMD + Rayon: ~114 ms.
No comments yet