Database

In-process databases usable directly from Python.

Inside your process, a Python database library runs without a server. DuckDB handles analytical SQL there; to search vectors, start with Chroma.

How to choose:

  • Analytical SQL over Parquet, CSV, or DataFrames: DuckDB
  • Vector search in a prototype or notebook: Chroma
  • ClickHouse's SQL dialect, or pandas code that runs on ClickHouse: chDB
  • Vectors stored with their raw images and text, on disk: LanceDB
  • Vector search embedded in an app or an edge device: Zvec
  • A small JSON document store for one script: TinyDB

Listed in editorial order, grouped by use case. Click a column header to re-sort the whole list.

Press / to search. Tags open their category. Click a row for details.

Search and filter

Results

Row number Tags

Analytical 2 projects

An in-process SQL OLAP database management system, optimized for analytics and fast queries, similar to SQLite but for analytical workloads.
duckdb/github.com/duckdb/duckdb /42,674 stars /
In-process OLAP SQL engine with the full ClickHouse dialect, zero-copy pandas/Arrow interop, and federation to remote ClickHouse clusters via remoteSecure().
chdb-io/github.com/chdb-io/chdb /2,921 stars /

Vector 3 projects

A developer-friendly embedded retrieval database for multimodal AI.
lancedb/github.com/lancedb/lancedb /11,811 stars /
An open-source embedding database for building AI applications with semantic search.
chroma-core/github.com/chroma-core/chroma /29,604 stars /
A lightweight, in-process vector database that embeds directly into applications.
alibaba/github.com/alibaba/zvec /16,161 stars /

Key-Value & Document 1 project

A tiny, document-oriented database.
msiemens/github.com/msiemens/tinydb /7,576 stars /

Database guide

Start DuckDB with duckdb.sql(), which runs on an in-memory database kept inside the module. It queries Parquet, CSV, and JSON files by name, and pandas DataFrames, Polars DataFrames, and Arrow tables by variable name, with no import step. When the data should stay, pass a file name to duckdb.connect().

chDB runs ClickHouse inside your process, so you get its SQL dialect without a ClickHouse server. If you come from pandas, start with the DataStore API: change the import, and your pandas code works unchanged. For SQL, use the connection-based API, and query DataFrames with the Python() table engine.

Chroma's in-memory client is the quick way to try it in a notebook, and PersistentClient keeps the data on disk. Add text documents, and Chroma embeds and indexes them for you, so you can query with text too.

LanceDB starts as an embedded library pointed at a local directory. Its indexes live on disk, and a table stores the raw data, metadata, and embeddings together, which suits images and other multimodal data. Its embedding function registry generates vectors as you insert rows, and Python code can then query with text.

Zvec runs entirely in-process, with no server or daemon, and its docs pitch it from prototypes to embedded apps and edge devices. Its quickstart defines a schema of scalar and vector fields first, then combines vector search with filters on those scalar fields.

TinyDB is pure Python with no dependencies, and stores Python dicts in a JSON file: open it with TinyDB('db.json') and search with Query(). Its own docs say to pick something else when you need access from several processes or threads, indexes, or ACID guarantees.

Know a project that belongs here?

Tell us what it does and why it stands out.