$ memista

Insert vectors over HTTP

How the POST /v1/insert endpoint works: the chunk shape, how SQLite row ids become USearch keys, and when the index is written to disk.

Difficulty: beginnerTime: 6 min

POST /v1/insert is how data gets into memista. This guide covers the chunk shape and what happens under the hood.

The request

curl -X POST http://localhost:8083/v1/insert \
  -H "Content-Type: application/json" \
  -d '{
    "database_id": "my_app",
    "chunks": [{
      "embedding": [0.1, 0.2],
      "text": "Hello world",
      "metadata": "{\"source\": \"readme\"}"
    }]
  }'

Each chunk carries three things:

  • embedding — the vector. In the stock crate this is length 2; see changing embedding dimensions.
  • text — the content the vector represents.
  • metadata — a string (commonly JSON) stored verbatim and returned on search.

What happens on insert

  1. The row (text + metadata) is written to the SQLite table chunks_<database_id> first.
  2. The returned chunk_id becomes the USearch key for the vector.
  3. After the batch, the <database_id>.usearch index file is saved to disk.

Because the SQLite id and the USearch key are the same integer, search can return ranked keys and hydrate the text/metadata straight back from SQLite. See how it works for the full picture, then run a search.

Frequently Asked Questions

What goes into a chunk?

An embedding (a vector of f32), the text it represents, and a metadata string (typically JSON you store and get back verbatim).

When is the index saved to disk?

The .usearch index file is saved after each insert batch, so a crash after a successful insert does not lose the batch.

Related

Try memista

A single crate, GPL-3.0. Two files on disk and three endpoints.