Lakerunner

Lakerunner transforms an S3-compatible bucket into a production-grade Observability stack in minutes.

What is Lakerunner?

Lakerunner is an open-source observability data lake that stores logs, metrics, and traces in cloud object storage (S3, GCS, Azure Blob). It provides:

Cost-effective storage - Store petabytes of observability data at object storage prices
Fast queries - Columnar format with intelligent indexing for sub-second queries
Native Grafana integration - Query your data lake directly from Grafana
Kubernetes-native - Deploy with Helm, auto-scales with demand

Architecture

Lakerunner separates ingest and query into independent, horizontally scalable paths that share only object storage and a lightweight metadata index in PostgreSQL.

ingest

Collectors / Agents

OpenTelemetry, vendor exporters

OTel logs / metrics / traces

Object Storage — raw

otel-raw/{logs,metrics,traces}/

object-created notifications

PubSub Adapter

SQS / GCP Pub/Sub / Azure Event Grid / HTTP

process-{logs,metrics,traces}

normalize raw telemetry → Parquet

compact small segments into larger ones

produce time-aggregated rollups (metrics)

writes & registers

Object Storage — cooked

db/{org}/{collector}/

tbl_{segment_id}.parquet

PostgreSQL (lrdb)

segment metadata index

query

Cardinal Agent

LakeQL or PromQL / LogQL

query-api

parse + plan

fan out, merge, stream via SSE

segment lookup

PostgreSQL (lrdb)

segment metadata index

fan out work units

N query-workers

DuckDB on Parquet

stateless, scale horizontally

read parquet

Object Storage — cooked

parquet segments

partial results merged + streamed via SSE

Cardinal Agent

For more detail, see the dedicated Ingestion and Query architecture pages.

Deployment Options

Lakerunner supports two deployment modes:

Mode	Use Case	Infrastructure
Kubernetes POC	Proof of concept with real cloud resources	Any Kubernetes cluster
Production	Full HA deployment	Production Kubernetes with autoscaling

Getting Started

Ready to deploy? Head to the Installation Guide to get started with our interactive setup wizard.

Prerequisites

Before installing Lakerunner, ensure you have:

kubectl - Kubernetes CLI
Helm 3.14+ - Package manager for Kubernetes
Kubernetes cluster 1.28+ (local or cloud)

For POC and Production deployments, you’ll also need:

S3-compatible object storage with notification capability (S3, GCS, or Azure Blob)
PostgreSQL 16+ database

Reach out to support@cardinalhq.io for support or to ask questions not answered in our documentation.