If your Datadog bill is creeping past the comfort line — or if your Loki + Mimir + Tempo + Grafana + Alertmanager stack has become a part-time job you didn't sign up for — there is a third option.

This is the practical setup guide for running Pharlux on a single sub-$25/month VPS: one binary, one config file, one systemd unit, no Docker, no external databases beyond embedded SQLite. By the end you will have an OpenTelemetry-native ingestion endpoint (OTLP gRPC and HTTP/protobuf), durable storage, and SQL across metrics and logs through Apache DataFusion. Setup is about thirty minutes.

The honest math​

Three options for a small team running 5 services:

(Datadog ranges are intentionally that — ranges. Their published pricing for Infrastructure Pro, Log Management, and APM is the source; numbers move with host count, retention, and event volume. Check their pricing page for current specifics.)

The cost gap is real. The operational gap is harder to prove — and that one comes down to design choices.

Why a 4 GB / 2 vCPU VPS is enough​

Pharlux's design centre is 1–10 services on a single VPS. The architecture pays for that:

• A single statically-linked Rust binary, ~85 MB on disk. No Docker daemon, no init system inside a container, no orchestrator. systemd starts it.
• Embedded SQLite for metadata. No Postgres, no Kafka, no ClickHouse.
• A custom write-ahead log (WAL) followed by per-signal Apache Parquet files on local disk. Frozen formats: WAL framing per ADR-0018, Parquet schemas per ADR-0003.
• Apache DataFusion as the in-process query engine, capped at a 256 MB MemoryPool in V1 (ADR-0011) so a runaway query cannot OOM the box.
• A custom DataFusion TableProvider that unions the live WAL with on-disk Parquet (ADR-0002) — freshly-ingested data is queryable without delay.
• Memory-safe TLS via rustls. Zero OpenSSL in the dependency tree. The binary is genuinely static musl — no glibc surprise on the target VPS.

Sustained load testing on a 4 vCPU / 8 GB VPS produced 577,000 metric points/sec over 17.36 million points with zero errors and 7 ms average request latency. The 4 GB / 2 vCPU tier handles considerably less than that — call it the working envelope for a small team's actual production traffic, with headroom — but the architectural ceiling sits well above the small-team workload.

What you would outgrow this for: 250+ hosts, multi-region deployments, dedicated SaaS-grade isolation with regulatory attestations. Pharlux is not pretending to be those things in V1.

Picking a provider​

Any reputable provider with a 4 GB / 2 vCPU tier will run this fine. Approximate monthly pricing for that tier as of writing:

This is not a recommendation — Pharlux runs the same way on any of them. (Disclosure: Veltara Works dogfoods on BinaryLane.) Pick whichever you already trust, in the region closest to whatever you are observing.

What to check when choosing:

• Network egress pricing — Pharlux is the receiving side of OTLP traffic, so ingress is free almost everywhere. Outbound is mostly your dashboard sessions.
• Snapshot or backup pricing — see the FAQ section below.
• IPv6 availability — useful for OTLP from IPv6-only services.
• Region proximity — keep the box close to your services to keep OTLP latency low.

30-minute setup​

The canonical reference is the Getting Started guide — this section is the abbreviated version.

1. Provision the VPS​

Ubuntu 24.04 LTS is the supported baseline. Other recent systemd-based distributions work too — the binary is statically-linked musl, so glibc version is not a constraint.

2. Download the v1.0.0 binary​

sudo curl -fSL -o /usr/local/bin/pharlux \
  https://github.com/Veltara-Works/pharlux/releases/download/v1.0.0/pharlux-linux-amd64
sudo chmod +x /usr/local/bin/pharlux

The download is one file, ~85 MB. Verify the checksum from the release page:

curl -fSL https://github.com/Veltara-Works/pharlux/releases/download/v1.0.0/pharlux-linux-amd64.sha256 \
  | (cd /usr/local/bin && sha256sum -c -)

3. Install the systemd unit​

sudo pharlux install

That single command writes /etc/systemd/system/pharlux.service with MemoryMax=1G, Restart=always, DynamicUser=yes, StateDirectory=pharlux, and ConfigurationDirectory=pharlux. It also creates /etc/pharlux/jwt.secret (64 random bytes, mode 0640) and prepares /var/lib/pharlux/ for the systemd-managed dynamic UID. No host-level pharlux user is needed.

4. (Optional) Configure​

A zero-length /etc/pharlux/pharlux.toml is a valid configuration — every key has a sensible default, including binding 0.0.0.0 on ports 3100 (HTTP/UI), 4317 (OTLP gRPC), and 4318 (OTLP HTTP). Override only what you need to change. The full key reference is at /docs/getting-started/ and /docs/sizing-guide/.

5. Open the OTLP and UI ports​

Pharlux trusts the reverse proxy for TLS termination. For public-facing OTLP, run it behind nginx, Caddy, or Cloudflare Tunnel. On a private network, internal-only is fine.

sudo ufw allow 3100/tcp comment 'Pharlux UI/API'
sudo ufw allow 4317/tcp comment 'OTLP gRPC'
sudo ufw allow 4318/tcp comment 'OTLP HTTP'

6. Start it and create the bootstrap admin​

sudo systemctl daemon-reload
sudo systemctl enable --now pharlux

# Confirm the service is healthy (no auth required for /health):
curl http://localhost:3100/api/v1/health

The bootstrap admin is created via the host CLI (the operator-trust path); subsequent users are added through the API by an existing admin.

sudo systemctl stop pharlux
read -rs PHARLUX_PW
sudo pharlux user add --username admin --password "$PHARLUX_PW" --admin
sudo systemctl start pharlux
unset PHARLUX_PW

7. Point your OpenTelemetry Collector at it​

Sample otel-collector.yaml exporter section:

exporters:
  otlp/pharlux:
    endpoint: "your-vps-ip:4317"
    tls:
      insecure: true   # only on a trusted network; use TLS in production

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [otlp/pharlux]
    logs:
      receivers: [otlp]
      exporters: [otlp/pharlux]

8. Verify it is working​

# Log in as the bootstrap admin to get a JWT.
TOKEN=$(curl -s -X POST http://your-vps-ip:3100/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"your-password"}' | jq -r .token)

# Run a query.
curl -s -X POST http://your-vps-ip:3100/api/v1/query \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"sql":"SELECT count(*) FROM metrics WHERE timestamp > now() - INTERVAL '\''5 minutes'\''"}' | jq .

If you have been sending OTLP for at least a minute, you should see a non-zero count.

What you will see​

The shipped UI handles ad-hoc SQL, saved queries, and basic dashboards. Cross-signal queries — for example, joining metrics and logs by trace_id — are one-line SQL through the DataFusion TableProvider. The same data is queryable through the HTTP API by any tool that can speak HTTP and SQL.

12-month cost summary​

For a small team running 5 services on a single 4 GB / 2 vCPU VPS:

For comparison, a Datadog Pro deployment for the same 5 services with logs and APM lands in the low-hundreds-of-dollars-per-month range, depending on host count, log volume, and APM coverage. The cost-reduction ratio for the small-team scenario typically falls between 10× and 25×.

The point of citing ranges rather than a single Datadog number is that those numbers move. The point of citing the Pharlux number exactly is that it does not — $0 is $0, and the VPS line is whatever your provider charges.

Honest limitations​

What you give up at this scale, in V1:

• No traces yet. Traces ship in V1.1 (ADR-0005). If you need distributed tracing today, run Pharlux for metrics + logs and a separate trace store in parallel.
• No PromQL yet. Queries are SQL via DataFusion in V1; PromQL ships in V1.1 (ADR-0005). If your team has months of PromQL muscle memory, that is real switching cost.
• Single-VPS architecture. Multi-VPS clustering and S3 cold tier are V1.1+ features at the commercial-tier level. The Scale tier ceiling is 250 hosts. If you require more hosts, send us a message.
• No managed cloud option. Pharlux is self-hosted-first. If you need a fully-managed SaaS with credit-card sign-up, that is not the V1 product.
• No SAML / OIDC / LDAP in Community. Those are commercial-tier features. JWT and admin/read-only auth are in V1 Community.

There are also scenarios where Pharlux is genuinely the wrong choice. If your team has months of Grafana dashboards-as-code investment and a dedicated SRE who enjoys operating the LGTM stack, the migration cost outweighs the operational savings. If you need traces today, see above. If your shop is large enough to have a Datadog budget and uses every feature of the platform, the cost framing of this post does not apply to you.

Frequently asked questions​

Will a $20/month VPS actually handle my workload?​

For 1–10 services with typical metric and log volume, yes. The 4 GB / 2 vCPU tier is the documented minimum spec. Teams operating closer to the upper end of that range — 8–10 active services with high-cardinality metrics — should consider the next tier up (8 GB / 4 vCPU, around $30–50/month depending on provider).

Can I run other things on the same VPS?​

Yes. Pharlux's memory budget is bounded — DataFusion is capped at a 256 MB MemoryPool in V1 (ADR-0011), and the rest of the binary's working set is small. The original design target was deliberately co-tenancy-friendly. If the VPS already runs a small web service or two, Pharlux fits alongside them.

What is the backup story?​

Pharlux's state lives in two places: the Parquet directory on disk, and the embedded SQLite metadata file. A nightly filesystem snapshot at the provider level (most VPS providers offer this for a few dollars a month) is the cheapest workable backup. For finer-grained backups, the Parquet files are append-only after rotation — rsync to off-box storage works well.

How do I upgrade Pharlux?​

systemctl stop pharlux, replace the binary, systemctl start pharlux. The WAL format is frozen (ADR-0018) and Parquet schemas are frozen (ADR-0003), so V1.x patch upgrades and the V1.0 → V1.1 transition are binary swaps, not data migrations.

What if I outgrow the $20/month tier?​

Move to the 8 GB / 4 vCPU tier on the same provider. The systemd unit and config carry over; the data directory is rsync-able. If you outgrow that, the commercial tiers — Team ($49/month, 10 hosts), Business ($199/month, 50 hosts), Scale ($899/month, 250 hosts) — extend retention, add SAML / OIDC / LDAP, audit log, white-label, and S3 cold tier as you grow.

Can I migrate from Grafana + Loki gradually?​

Yes. Point your OpenTelemetry Collector at both Pharlux and your existing stack in parallel. Verify Pharlux is seeing the data and the queries you care about work. Cut over alerting last. There is no big-bang migration.

Is the Community tier really free forever?​

Yes. The Community edition is AGPL-3.0 and unmetered — run it at any scale. The commercial tiers exist for organisations that want SAML, audit log, white-label, S3 cold tier, or support; they are dual-licensed (ADR-0022) and remove the AGPL terms for organisations that need that.

Get Pharlux​

• Download v1.0.0 — github.com/Veltara-Works/pharlux/releases/tag/v1.0.0
• Documentation — pharlux.com/docs/getting-started/
• Source — github.com/Veltara-Works/pharlux

Pharlux is one of several developer tools built by Veltara Works — alongside email hosting, cloud infrastructure, and software license management. See veltaraworks.com for the full portfolio.

The thing most observability platforms don't talk about: you cannot query data you have not persisted, and persisting hurts throughput. ClickHouse-style batch writers compress beautifully and scan fast, but typical batch intervals introduce tens of seconds of staleness before new data is queryable. In-memory buffers are immediate but lose data on crash. The two requirements — durability and freshness — pull in opposite directions, and most platforms pick one and document around the other.

Pharlux ships a different design. A custom Apache DataFusion TableProvider unions an in-memory WAL buffer with on-disk Apache Parquet files into one consistent view at query time. Freshly-ingested data is queryable as soon as it lands in the WAL — no flush wait, no buffer-window staleness — and it is queryable through the same SQL surface as historical data on disk.

This is the load-bearing piece behind Pharlux's freshness story, and the riskiest decision we made in the whole architecture — risky enough that we held the proof-of-concept as a mandatory gate before writing any production code. It lives in pharlux-store/src/table_provider.rs. This post walks through the design, the runtime semantics, and the trade-offs.

The trade-off, concretely​

Three specific scenarios drive the design:

1. Acknowledged-but-not-flushed data must survive a crash. When OTel Collectors send a batch and Pharlux returns 200 OK, that data is persisted. A SIGKILL immediately afterwards must not lose the batch.
2. Just-acknowledged data must be queryable. Operators run pharlux user add, restart a service, hit a dashboard. The dashboard query lands within seconds. The metric points just emitted by the restarted service must appear in that query — not "in 30 seconds when the next flush completes."
3. Compressed historical data must scan fast. Six months of metrics needs to live in a format the query engine can scan efficiently — column-pruned, predicate-pushed, page-skipped. That is what Parquet was designed for.

A single storage format that satisfies all three is hard. Parquet is column-stored, optimised for analytical scans, and not designed for high-frequency small writes. An append-only WAL is fast to write and easy to crash-recover, but Parquet's compression ratios and scan performance assume larger, well-organised files. Pharlux uses both, with the WAL as the durable buffer and Parquet as the compressed steady state — which puts the unification problem at query time.

The naive options that don't work​

A few approaches the design considered and rejected:

• Query Parquet only, accept N-second staleness. Loses requirement #2. ClickHouse, SigNoz, and most batch-oriented platforms accept this. Pharlux's design centre — small teams debugging incidents in real time — does not.
• Flush WAL → Parquet on every batch. Parquet writers have meaningful per-file overhead (footer encoding, schema serialisation, dictionary pages). Flushing on every batch destroys throughput and produces thousands of tiny Parquet files that are slow to scan.
• In-memory buffer only, no on-disk WAL. Loses requirement #1. Any crash between ingest and the next flush loses data. Rejected explicitly in ADR-0018.
• Two storage layers, two query paths. Some systems route "recent" queries to a hot store and "historical" queries to cold storage, requiring the user to know which is which. The user does not want to know which is which. They want one SQL surface.
• A real database (PostgreSQL, ClickHouse, etc.). Violates the single-binary constraint. Pharlux is one statically-linked Rust binary; there is no PostgreSQL or ClickHouse process to run alongside it.

The design that satisfies all three requirements is: keep the WAL durable on disk, keep recently-acknowledged records in an in-memory snapshot of that WAL, write Parquet files when the WAL exceeds a size or time threshold, and let the query engine see both at once.

The split​

Pharlux runs two storage layers in the same process:

The WAL is an append-only file on disk with a strict frame format (length-prefixed prost-encoded protobuf records, each followed by a CRC32 — see ADR-0018). Every accepted batch is written to the WAL with a configurable fsync policy before the API returns 200 OK. On crash, replay reads the WAL forward, validates each record's CRC, truncates at the first invalid record (a partial write at the tail), and rebuilds the in-memory state. Crash recovery is gated by a hard test: 10 consecutive crash-recovery test runs with zero flakes is one of the V1 release gates.

Parquet is the on-disk steady state. Per-signal schemas (metrics, logs, V1.1 traces) are frozen (ADR-0003). Files live under /var/lib/pharlux/{metrics,logs}/{tenant_id}/YYYY/MM/DD/HH/ so retention and compaction can operate on time partitions without scanning the whole tree. Compression is column-stored, dictionary-encoded for high-repeat fields like name and scope_name, with timestamp-sorted row groups for predicate pushdown.

The link between them is the PharluxMetricsTable (and its PharluxLogsTable sibling), which holds:

struct TableState {
    wal_records: Vec<WalRecord>,      // in-memory snapshot of the WAL tail
    parquet_files: Vec<PathBuf>,      // committed files on disk
}

pub struct PharluxMetricsTable {
    schema: SchemaRef,
    state: Arc<RwLock<TableState>>,
}

The RwLock is the load-bearing primitive. Ingest takes the write lock briefly to push a WalRecord into the buffer. Query takes the read lock briefly to snapshot both halves. Flush takes the write lock briefly to drain WAL records and register newly-written Parquet paths.

How a query executes​

The implementation lives in impl TableProvider for PharluxMetricsTable. The DataFusion query planner calls scan() with the projection, filters, and limit; Pharlux returns an ExecutionPlan that yields Arrow RecordBatches covering both layers.

The interesting bit is the snapshot:

async fn scan(&self, ...) -> Result<Arc<dyn ExecutionPlan>> {
    // Snapshot WAL records and open Parquet file handles under read lock.
    // Opening handles inside the lock ensures compaction cannot delete
    // files between path observation and handle acquisition.
    let (wal_records, parquet_handles) = {
        let state = self.state.read().unwrap();
        let records = state.wal_records.clone();
        let handles: Vec<File> = state.parquet_files.iter()
            .filter_map(|p| File::open(p).ok())
            .collect();
        (records, handles)
    };
    // ... build RecordBatches, delegate to MemTable for projection/filter/limit
}

Two subtle properties matter here:

• Open handles inside the lock. A naive implementation would clone the path list under the read lock and open files later. Compaction or retention could delete one of those files in the gap between the path snapshot and the open call, causing a query to fail with ENOENT mid-flight. Opening the handles inside the read lock means each query holds a kernel-level reference to its set of files for the duration of the scan, even if the on-disk path is unlinked.
• Clone the WAL records. The Vec<WalRecord> clone is a real cost (shallow copy of the slice plus protobuf payloads), but it removes the WAL data from the lock's lifetime. The query proceeds on a private copy; ingest and flush can continue concurrently on the original buffer without the query holding the read lock for the full scan duration.

After the snapshot, WAL records are converted into a single Arrow RecordBatch matching the production Parquet schema (timestamp as Timestamp(Nanosecond, UTC); name as a Dictionary(Int32, Utf8); tenant_id as a non-null Utf8; etc.). Parquet files are read through ParquetRecordBatchReaderBuilder, which yields the same Arrow shape. All batches are collected into a MemTable, and MemTable::scan() applies projection pushdown, filter pushdown, and the optional LIMIT — the same DataFusion optimisations that run for any other table.

The user-side SQL surface is just SQL:

SELECT name, count(*) AS cnt
FROM metrics
WHERE timestamp > now() - INTERVAL '5 minutes'
GROUP BY name
ORDER BY cnt DESC;

There is no metrics_recent vs metrics_historical distinction in the schema. The query engine sees one logical table; the TableProvider produces the union; the user does not need to know.

Atomic transitions​

The other moving piece is what happens when WAL records become Parquet files. The flush path:

// Step 1: read-lock to clone the records we'll flush
let to_flush = {
    let state = self.state.read().unwrap();
    let n = count.min(state.wal_records.len());
    state.wal_records[..n].to_vec()
};

// Step 2: write Parquet files — no lock held during I/O
let written = parquet_writer::flush_records_to_parquet(data_dir, &to_flush, ...)?;
let paths: Vec<PathBuf> = written.iter().map(|w| w.path.clone()).collect();

// Step 3: write-lock to drain flushed records and register new files
{
    let mut state = self.state.write().unwrap();
    let n = to_flush.len().min(state.wal_records.len());
    state.wal_records.drain(..n);
    state.parquet_files.extend(paths);
}

The relevant invariant is in the doc comment on this function: "records transition from WAL to Parquet atomically under the write lock — a concurrent query sees records in either WAL or Parquet, never both and never missing."

That property is what makes the union query correct. Step 2 (the actual Parquet write) happens with no lock held — disk I/O cannot block ingest or queries. Step 3 takes the write lock just long enough to swap pointers: drain the flushed prefix from wal_records, push the new PathBufs into parquet_files. A query running on a snapshot taken before this step sees the records as WAL entries; a query on a snapshot taken after sees them as Parquet entries. There is no in-between state where a record is missing or duplicated.

Performance characteristics​

The design's runtime profile, on a 4 GB / 2 vCPU VPS:

• Ingest path: OTLP request → bounded tokio::sync::mpsc channel (default 1,000 batches) → WAL writer task → fsync. The channel bound is the primary backpressure signal; when full, ingest returns HTTP 429 / gRPC RESOURCE_EXHAUSTED after a 100 ms send timeout (per ADR-0015). OTel Collectors retry 429 with exponential backoff out of the box.
• WAL ceiling: 64 MB. Above this, a flush is forced regardless of the configured flush interval. The 64 MB figure is a deliberate cap on memory-resident WAL state, since the in-memory Vec<WalRecord> is what the query path snapshots.
• Query path: scan() snapshots state under read lock (microseconds), reads Parquet (column-pruned per the projection, predicate-pushed per the filter), unions with the WAL batch, and streams back through DataFusion. Memory is bounded by an explicit MemoryPool cap of 256 MB (per ADR-0011) — pathological queries fail with a clear error rather than OOM-killing the box.
• Process ceiling: 1 GB hard MemoryLimit enforced by systemd. If the process ever exceeds 1 GB, the kernel kills it; Restart=always brings it back; WAL replay rebuilds the in-memory state from disk; no acknowledged data is lost.

Sustained load testing on a 4 vCPU / 8 GB VPS produced 577,000 metric points/sec over 17.36 million points with zero errors and 7 ms average request latency. The 4 GB / 2 vCPU tier handles considerably less than that in absolute throughput — but the architectural ceiling sits well above small-team production traffic.

What we give up​

The design is honest about its limits:

• No per-record delete. Parquet is append-only. GDPR-style erasure runs at the partition level via retention plus targeted deletes, not row-level.
• No incremental Parquet. Flushed Parquet files are immutable; updates to old data require a rewrite of the affected file by the compaction job.
• DataFusion has no sparse indexes. Full-text log search via LIKE on a large logs table is a Parquet full scan. ADR-0005 documents the V1 threshold (~10 GB/day) above which Tantivy indexing is the V1.1 scaling story.
• MemoryPool ceiling is real. Unbounded GROUP BY over a long time range can fail with a 256 MB MemoryPool error. The error message names the cap and recommends narrowing the time range or adding LIMIT. We chose this over silent OOM.

These trade-offs are listed in the ADRs. They are not surprises; they are the cost paid for the single-binary, embedded-execution architecture.

Why this design over alternatives​

As I said up top, this was the highest-risk decision we made — risky enough that we treated the proof-of-concept as a mandatory gate. The PoC ran the full path (WAL crash recovery, concurrent ingest + query, compaction, two-tenant isolation) before any production code was written. It passed. The fallback — embedded DuckDB behind the QueryEngine trait (ADR-0014) — remains documented but unused.

The reason the design is preferred over a real database engine is alignment, not novelty. ClickHouse, embedded or otherwise, would deliver excellent compression and scan performance — and would also bring its own internal WAL, its own page cache, its own memory accounting, and 1-2 GB of resident memory in its own right. On a 4 GB VPS that is most of the budget. The DataFusion + Parquet path lives within Pharlux's memory budget (200-430 MB realistic, 1 GB hard ceiling) because there is no second engine to feed.

The reason it is preferred over an in-memory-only design is durability. Acknowledged data must survive a crash; the WAL is the contract.

The reason it is preferred over a flush-on-every-batch design is throughput and Parquet hygiene. Many small Parquet files are slow to scan and burn footer encoding overhead; the WAL absorbs the small writes and Parquet sees larger, well-organised files.

Frequently asked questions​

Why an in-memory WAL snapshot if the WAL is on disk?​

The on-disk WAL is the durable record. The Vec<WalRecord> in memory is a snapshot of the same records, kept as Arrow-shaped objects to avoid re-parsing protobuf on every query. The on-disk WAL is the source of truth on crash recovery; the in-memory copy is the source of truth on hot-path queries.

What happens if a query is running when the WAL flushes?​

The query took its snapshot under the read lock before the flush ran. The query sees those records as WAL entries. A query that starts after the flush completes sees them as Parquet entries. There is no race window where a record is missing or duplicated — that is the property the doc comment on flush() calls out explicitly.

How is tenant_id enforced across the union?​

tenant_id is a non-null Utf8 column in every Parquet schema and a field on every WalRecord. Every query goes through TenantScopedQueryBuilder with a mandatory WHERE tenant_id = ? predicate before reaching the TableProvider. Community deployments use the constant "default" tenant — the code path is identical. Multi-tenant from day one (per Pharlux's hard invariants) is not a retrofit.

Can DataFusion push my filter into the Parquet reader?​

Yes for predicate pushdown on standard column types — DataFusion uses Parquet page indexes and row group statistics to skip pages that cannot match. Projection pushdown applies on every scan. The WAL side is in-memory Arrow, where filters are applied at scan time without an indexing pass — at typical WAL sizes (well under 64 MB) this is fast.

What happens if the Parquet file is deleted by retention while my query is reading it?​

The query holds an open File handle taken inside the read lock. On Linux, deleting an open file unlinks the directory entry but leaves the file content readable through any open handle until the last handle closes. The query completes against the original content; the disk space is reclaimed when the query finishes. This is documented in the scan() comment: "opening handles inside the lock ensures compaction cannot delete files between path observation and handle acquisition."

Why not just use embedded DuckDB?​

DuckDB was the documented Phase 0 fallback (ADR-0013) and would also have worked — DuckDB has a battle-hardened Parquet reader and real production mileage in embedded mode. The reason DataFusion is the preferred choice is forward compatibility: Pharlux writes plain Apache Parquet files, readable directly by DuckDB, Trino, Ballista, Polars, Spark, and any other engine that speaks Parquet. Operators who outgrow Pharlux take their data with them in an open interchange format that every analytical tool understands. DataFusion's Arrow-native API also lets Pharlux compose query results as zero-copy Arrow streams from Parquet through the engine to the API response, where DuckDB's API exposes SQL strings — Arrow composition matters for the dashboard latency budget.

Get Pharlux​

• Download v1.0.0 — github.com/Veltara-Works/pharlux/releases/tag/v1.0.0
• Documentation — pharlux.com/docs/getting-started/
• Source — github.com/Veltara-Works/pharlux
• The TableProvider source — pharlux-store/src/table_provider.rs

Pharlux is one of several developer tools built by Veltara Works — alongside email hosting, cloud infrastructure, and software license management. See veltaraworks.com for the full portfolio.