Try SpiralDB

Ready to take Spiral for a spin? This guide walks through the typical workflow:

1. Create a project
2. Connect a bucket
3. Model your table(s)
4. Ingest & enrich data

Use uv (or your favorite package manager) to install the Python client and CLI.

uv add pyspiral

pip install pyspiral

Run spiral walkthrough for an interactive tutorial of Spiral CLI.

1. Create a project

Create a project first. Projects are the top-level unit used for access control and storage configuration.

spiral projects create --description my-first-project

The CLI prints a <project-id> you can reference from Python:

from spiral import Spiral
 
sp = Spiral()
project = sp.project("<project-id>")

Learn more: Projects, Command Line

2. Connect a bucket

Before ingesting, connect your project to a backing file system (for example S3, GCS, or Azure Blob).
This is how Spiral reads and writes table data in object storage.

spiral fs update --type s3 --bucket <bucket-name> --region <region> <project-id>

After this is configured, table writes and enrichments can read/write through that file system.

Learn more: File Systems, CLI: spiral fs update

3. Data modeling

Spiral tables are sorted and unique by a primary key, and support nested column groups for multimodal data. Start by designing a stable key schema and a column layout that matches how you query.

import pyarrow as pa
 
key_schema = pa.schema([
    ("created_at", pa.timestamp("ms")),
    ("id", pa.string()),
])

Learn more: Data Model, Tables Overview, Best Practices

4. Ingest & Enrich

Create a table, then write Arrow tables or python objects. Every write must include key columns.

from datetime import datetime
 
events = project.create_table("getting-started.events", key_schema=key_schema, exist_ok=True)
 
data = pa.table({
    "created_at": pa.array(
        [datetime(2024, 1, 1, 0, 0), datetime(2024, 1, 1, 0, 1), datetime(2024, 1, 1, 0, 2)],
        type=pa.timestamp("ms"),
    ),
    "id": ["evt-1", "evt-2", "evt-3"],
    "type": ["PullRequestEvent", "PushEvent", "PullRequestEvent"],
    "actor": ["alice", "bob", "carol"],
    "url": [
        "https://picsum.photos/seed/alice/64",
        "https://picsum.photos/seed/bob/64",
        "https://picsum.photos/seed/carol/64",
    ],
})
 
events.write(data)

Learn more: Write Tables

Enrichment lets you append new columns without rewriting existing data, including media fetched from URLs/S3/files.

from spiral import expressions as se
 
enrichment = events.enrich(
    se.pack({
        "thumbnail": se.http.get(events["url"])
    })
)
enrichment.run()

Learn more: Enrichment