$ memista
crate  ·  experimental  ·  GPL-3.0

A lightweight vector search library for Rust.

memista is a single-crate vector search service. SQLite for metadata, USearch for the index, Actix-web for the HTTP layer. Embed it as a library, or run the bundled binary as a standalone server.

cargo install memista read the docs

experimental v0.1.x rust 1.56+

What it actually is

memista pairs two well-known pieces: USearch for approximate nearest-neighbour search and SQLite for the rows that describe each vector. A single binary starts an HTTP server on 127.0.0.1:8083 with three endpoints: insert, search, drop. That is the whole surface. 

curl -X POST http://localhost:8083/v1/insert \
  -H "Content-Type: application/json" \
  -d '{
    "database_id": "my_app",
    "chunks": [{
      "embedding": [0.1, 0.2],
      "text": "Hello world",
      "metadata": "{\"source\": \"readme\"}"
    }]
  }'

Why a library, not a database

One process

No Docker container, no sidecar, no managed cluster. The crate compiles into your binary or runs as one of its own.

SQLite for the boring parts

Metadata lives in a chunks_<db_id> table you can open in sqlite3 and inspect. Backups are cp.

USearch for the fast parts

Vectors are indexed with USearch (HNSW under the hood), with simsimd and fp16lib features compiled in.

Multi-tenant by partition

Each database_id gets its own SQLite table and its own <id>.usearch index file. Isolation is by name.

How the pieces fit

┌─────────────────────────────────────────────────────────┐
│  POST /v1/insert    POST /v1/search    DELETE /v1/drop  │
│             (actix-web + apistos OpenAPI)               │
├─────────────────────────────────────────────────────────┤
│                                                         │
│   ┌──────────────────┐         ┌─────────────────────┐  │
│   │   SQLite (WAL)   │◀───────▶│   USearch index     │  │
│   │ chunks_<db_id>   │  chunk  │  <db_id>.usearch    │  │
│   │ (text, metadata) │   id    │  HNSW · IP · f32    │  │
│   └──────────────────┘         └─────────────────────┘  │
│                                                         │
└─────────────────────────────────────────────────────────┘

On insert, the row goes into SQLite first; the returned chunk_id becomes the USearch key. On search, USearch returns ranked keys and we hydrate the text/metadata back from SQLite. The index file is saved to disk after every insert batch.

Honest about the corners

limitations
  • Embedding dimensions are hardcoded to 2 in the current crate. This is a demo default; you will be editing IndexOptions::dimensions before shipping.
  • Not tested past ~100k vectors. Skelf calls this experimental for a reason.
  • No auth. The server binds to 127.0.0.1 and trusts everything on the wire. Put a reverse proxy in front if it leaves the box.
  • Index dimension changes force a rebuild. The on-disk format is fixed at creation.

Embed as a library

[dependencies]
memista = "0.1"
use memista::{AppState, Config, create_app};
use async_sqlite::{PoolBuilder, JournalMode};
use actix_web::HttpServer;
use std::sync::Arc;

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    let config = Config::from_env().expect("load config");
    let db_pool = PoolBuilder::new()
        .path(&config.database_path)
        .journal_mode(JournalMode::Wal)
        .open().await.expect("db pool");
    let app_state = Arc::new(AppState { db_pool });

    HttpServer::new(move || create_app(app_state.clone()))
        .bind((config.server_host.as_str(), config.server_port))?
        .run().await
}

From the blog

Compare