Rendezvous-hashed partitioning
HRW hashing over a fixed ring of virtual buckets — deterministic placement, minimal data movement when membership changes.
OPEN SUBSTRATE · BSL 1.1
stplr.
A self-managing, horizontally-scaling distributed key/value + set-ops store.
Rendezvous-hashed. Replicated. Zero-touch. The open data plane other systems build on.
Source available now — the full engine, BSL 1.1, on GitHub.
Everything in the box
Replicated, survives node loss
Top-R replication (R≥2 by default). Reads and writes fail over to a surviving replica automatically — no client retries to wire up.
Zero-touch operations
Online rebalance, graceful drain, auto-heal, crash-resume, and live membership — no manual reshard, no downtime, no operator babysitting.
KV + server-side set-ops
An object store (collection, key) → value plus posting-list set operations — set_add / set_remove / set_members executed on the shard, not pulled to the client.
Durable by default
LMDB-backed — the disk is the database. Survives restarts with no separate persistence config; or run fully in-memory for dev.
Replayable change feed
A durable, value-level log you can tail and replay — recovery and downstream sync without bolt-on CDC.
Recently shipped: per-key TTL · atomic CAS + counters · bearer auth · direct-to-shard writes · coordinator leader election · durable ingest queue · rack/zone-aware replicas · Prometheus /metrics · scheduled backups
On the roadmap: streaming + point-in-time restore · log retention & compaction · in-process TLS · aggregate multi-load-gen benchmark
One binary. Composable roles.
A node runs any combination of shard, coordinator, and api — chosen per process, changeable at runtime. Scale the data plane and the query plane independently.
- shard owns and serves a partition; the horizontal-scale unit, replicated R-ways.
- coordinator drives rebalancing, membership, and leader election; clients route direct to the shards (or through it).
- api stateless protocol endpoint; run N replicas behind a load balancer.
Availability today: a highly available data plane — replicated shards with auto-heal, crash-resume, and rack/zone-aware placement. Clients route reads and writes direct to the owning shards; the coordinator holds a leader-election lease and drives rebalancing, off the data path. Writes can run through a durable ingest queue for crash-safe, at-least-once delivery.
# stplr node — run any combination of roles node_id = "node-a" roles = ["shard", "coordinator"] replication = 2 store = "lmdb" # generic, replicated, online-rebalanced set-ops set_add(coll, key, member) → append to a posting list set_members(coll, key) → read it back, R-way safe
BENCHMARKED · DURABLE TIER
Throughput where durability usually costs it.
On matched single nodes, stplr led the open durable field — Cassandra, ScyllaDB, etcd, TiKV, FoundationDB — on both read and write throughput, while staying one self-managing binary you start in seconds. In-memory caches win raw point ops by design; nothing else durable keeps up at this operational cost.
215kdurable reads / sec
150kdurable writes / sec
~2sone binary, to first query
Per-shard, on a 4-vCPU node: 500k ops, 64-byte values, LMDB-durable, binary protocol — reported as the median across several fresh instances (measured ~153k write / ~221k read; the headline is a conservative floor under that). That's the per-shard floor — scale out by adding shards, not cores. Full methodology and the load harness available on request.
The durable field — single node, each on its native protocol
| store | SET / s | GET / s | set ops | stand‑up |
|---|---|---|---|---|
| stplr | 150k | 215k | native | 1 binary · 2s |
| ScyllaDB | 86k | 88k | — | container · 12s |
| FoundationDB | 32k | 50k | — | server+file · 10s |
| TiKV | 35k | 46k | — | PD+TiKV · 30s |
| etcd | 29k | 38k | — | container · 3s |
| Cassandra | 29k | 38k | — | JVM · 30s |
Dedicated 4‑vCPU node, 500k ops, 64‑byte values, zero errors; each store on its native client (stplr binary, CQL, gRPC, …). stplr’s figure is the median across several fresh instances; peer numbers are their original same‑hardware runs. In-memory caches are a separate class — for reference Valkey 436k/446k and Memcached 123k/131k win raw point ops but persist nothing.
Scaled to 3 nodes — the whole field, identical hardware
| durable, 3‑node clusters | SET / s | GET / s | set ops | vs stplr |
|---|---|---|---|---|
| stplr 3 shards | 226k | 265k | native | — |
| ScyllaDB | 103k | 102k | — | 2.6× |
| Cassandra | 81k | 84k | — | 3.2× |
| TiKV | 50k | 75k | — | 3.5× |
| FoundationDB see note | 32k | 50k | — | 5.3× |
| etcd see note | 31k | 38k | — | 7× |
| Valkey in-memory, not durable | 246k | 262k | native | ≈1× |
Each store clustered across the same three dedicated 4‑vCPU nodes, 2M ops, 64‑byte values, 256 concurrent, zero errors on every store; RF = 1 / R = 1 (pure scale-out). “vs stplr” = stplr’s read throughput over that store’s. etcd replicates rather than shards — three nodes buy it HA, not throughput (its figure ≈ its single‑node number), shown for completeness. FoundationDB’s multi‑node cluster wouldn’t come up cleanly in the harness, so its single‑node number stands (consistent with its highest stand‑up cost). Valkey is an in‑memory cache (not durable), shown as the speed ceiling — stplr edged its reads and nearly matched its writes while persisting every op.
FEATURE-FOR-FEATURE · DURABLE / DISTRIBUTED
One binary. The job of a whole stack.
Most of these capabilities exist somewhere in the field. What's rare is having them in one self-managing store — durable, sharded, and zero-touch — instead of stitched across a database, a cache, a coordinator, and a queue.
| capability | stplr | ScyllaDB | Cassandra | TiKV | FoundationDB | etcd | Valkey |
|---|---|---|---|---|---|---|---|
| Durable to disk | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ~ |
| Horizontal sharding (scale-out) | ✓ | ✓ | ✓ | ✓ | ✓ | — | ✓ |
| Replication / HA | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Online rebalance + self-heal | ✓ | ✓ | ✓ | ✓ | ✓ | ~ | ~ |
| Server-side set ops | ✓ | ~ | ~ | — | — | — | ✓ |
| Per-key TTL | ✓ | ✓ | ✓ | ~ | — | ✓ | ✓ |
| Atomic CAS + counters | ✓ | ~ | ~ | ✓ | ✓ | ✓ | ✓ |
| Change feed / watch | ✓ | ~ | ~ | — | ~ | ✓ | ~ |
| Built-in leader election | ✓ | — | — | — | — | ✓ | — |
| Auth + TLS | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Prometheus metrics | ✓ | ✓ | ~ | ✓ | ~ | ✓ | ~ |
| Snapshot / backup | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Single binary, no extra services | ✓ | — | — | — | — | ✓ | ✓ |
| License | BSL→Apache | source-avail. | Apache 2 | Apache 2 | Apache 2 | Apache 2 | BSD‑3 |
✓ native · ~ partial or via a slower/limited path (hover for detail) · — not built in. etcd is a consensus store — it replicates rather than shards, so “no” on sharding is by design, not a defect. Valkey is in-memory-first; durability is asynchronous. The point isn't that stplr invents any one row — it's the only column that’s green down the functional rows and ships as one zero-config binary.
Two things stplr is not — by design
Not a consensus coordinator
stplr is a high-availability data plane, not a Raft/Paxos config store. Every shard is strongly consistent under its own lock — atomic CAS, counters, single-shard transactions — while the cluster keeps serving through node loss and network partitions instead of stalling for a quorum. Cross-DC replication is asynchronous, last-writer-wins by design. Need a strongly-consistent lock service for a handful of keys? That's what etcd is for — and stplr runs leader election on top when you want it. This store is built to keep millions of keys moving at throughput, not to arbitrate a five-node quorum.
Not a vector database
Vectors find what's similar. stplr does the half of retrieval vectors can't: exact, structured correlation — membership, set algebra (AND / OR / NOT), and deterministic joins across sources, with provenance you can audit. It's the precise, explainable complement to fuzzy similarity search — the structured-retrieval layer a correlation engine builds on, not a re-ranked nearest-neighbor index. Pair stplr with your vector store; don't expect either one to be the other.
BUILD ON TOP · ENTERPRISE
stplr is the substrate. Stitch is the engine.
stplr gives you the distributed data plane. Stitch — the proprietary correlation engine (US 11,151,112) — runs on the same cluster and does what raw key/value can't.
- Multi-table correlation from a single query — join entities across many sources
- AND / OR / NOT set logic over the results, evaluated at scale
- Index / stitch / map construction built and maintained for you
- Production-ready: metering, licensing, SSO / RBAC / audit
- Runs on the stplr cluster you already operate — same ops, same scaling
MEET THE MASCOT
Every serious substrate deserves a face with fangs.
Say hello to Vypr — the staple-fanged viper. Coiled, fast, and unreasonably good at holding your data together.
Merch is coming.
Stickers, patches, and the hoodie you'll actually wear to the on-call shift. Open substrate, open swag drawer.
Tell me when there's swagSource-available, becoming open.
stplr is licensed under the Business Source License 1.1. Use, copy, modify, and run it in production for any purpose — the one limit is offering it to third parties as a hosted key-value, storage, or set-operation service.
Each version converts to the Apache License 2.0 four years after it is first published. Source-available today, true open source on a clock.
Browse the source on GitHub