Skip to Content
New EngineOverview

Spiral is a local SQL engine for AI/ML datasets where reading the value is the expensive part: compressed columns, images, video, audio, documents, tensors, features, and scientific arrays. The current client is a command-line SQL REPL over Vortex files, in-memory tables, and table functions registered in the active Spiral session.

Use Spiral to filter and align metadata first, then materialize only the rows, bytes, or tensors a research workload needs.

What works today

  • Start spiral sql from source.
  • Register Vortex files as session-local tables.
  • Create small in-memory tables for examples and smoke tests.
  • Run a focused analytical SQL subset: projection, filtering, grouping, aggregates, ordering, limits, VALUES, EXPLAIN, and EXPLAIN ANALYZE.
  • Query local files through built-in table functions: spql.vortex.read, spql.arrow.read_ipc, and spql.parquet.read.
  • Export query results with COPY TO in Vortex or Arrow IPC format.
  • Query the catalog with spql.meta.table_functions(), spql.meta.source_functions(), and spql.meta.extension_types().
  • Use implemented Blob, JPEG, EXIF, and image preview extension types in sessions that register the default CLI plugins.

What this documentation reserves

Spiral is being shaped around extension packages for images, video, audio, documents, tensors, and custom data systems. Those pages are marked Experimental: Contact Us until the public packages are implemented. They describe the intended user contract so examples and product language stay consistent as the surface grows.

Researcher path

Start Here

Build the client and run the smallest complete Spiral session.

Load Data

Open local Vortex files, query Arrow IPC and Parquet files, and create scratch tables.

Query Data

Select, filter, group, sort, inspect plans, and discover registered table functions.

Extensions

Use Blob and JPEG primitives today and track the experimental image, video, audio, document, tensor, and custom extension surfaces.

Research Workflows

Follow the query-to-batch model for training pipelines and reproducible samples.

Run and Debug

Configure the REPL, tune local queries, troubleshoot errors, and check limits.

Reference

Find exact CLI, SQL, function, table-function, copy-format, and support details.

Working model

Vortex files, in-memory tables, or table functions
  -> SQL
  -> Spiral relation plan
  -> terminal rows, plan report, or execution report
Last updated on
TroubleshootingOverview