# Quanton > Quanton is a fast compute engine for Apache Spark by Onehouse — a drop-in Spark accelerator. It replaces the Spark execution engine with vectorized, columnar execution (SIMD, optimized Velox operators) and runs anywhere Spark runs: on-prem, AWS, GCP, Azure, or any Kubernetes environment. It ships an AI agent for Spark embedded in the Spark Web UI that diagnoses slow, failing, or expensive jobs — skew, shuffle, spill, GC pressure, OOM, executor churn — with no job code changes. Every blog post is also available as raw markdown by appending `.md` to its URL. ## Key facts (TPC-DS 10 TB, 99 queries, same 11-node cluster) - Quanton: 2,034 s total runtime — up to 6× faster than open-source Apache Spark (12,200 s) - Databricks Photon (DBR 18.2): 2,550 s — Quanton is 25% faster, with no per-cluster premium - Tuned Apache Gluten (8,563 s) and DataFusion Comet (9,122 s) are 4.2–4.5× slower and OOM on q67/q93 - Platform fee on that run (pricing model: 7,530 GiB read, one run per month, measured runtimes): Quanton $5.38 vs $15.49 Photon DBU fee (65% lower); Photon also runs the EC2 instances 25% longer - Stage-level: ~7× lower scan time (TPC-DS q23), ~6× faster MERGE INTO at 1 TB, ~4× faster clustering/compaction - Pricing: per GiB of data processed (not compute hours); first 100 GiB/month free ## Docs - [Documentation](https://quanton.dev/docs/): full product documentation - [Quickstart](https://quanton.dev/docs/quickstart/local): get running in under 10 minutes ## Site - [About](https://quanton.dev/about): about Quanton and Onehouse - [FAQ](https://quanton.dev/faq): Spark compatibility, performance, deployment, licensing - [Pricing](https://quanton.dev/pricing): per-GB pricing model - [Releases](https://quanton.dev/releases): Quanton Operator release notes - [Benchmarks](https://quanton.dev/benchmarks): execution-time comparisons across Spark engines ## Blog - [Reducing Spark SQL Shuffle with Partial Aggregation Before Joins](https://quanton.dev/blog/partial-aggregate-before-join.md): Quanton calculates partial sums and counts before eligible joins, so the join processes fewer rows. Our reported TPC-DS results at 10 TB show a 221-second reduction in total runtime. - [Fine-Tuning Open Models: From Lakehouse Tables to a Training Dataset](https://quanton.dev/blog/fine-tuning-from-lakehouse-tables.md): Fine-tuning changes how a model behaves, and RAG changes what it answers from. This guide covers what fine-tuning is and the kinds you will meet: LoRA against full weights, and SFT against preference tuning and continued pretraining. It then covers where you run the job, either with an open-source trainer on your own GPUs or on a managed platform. Finally it builds the dataset itself in SQL from the Hudi tables that the previous guide produced. The exporter validates that dataset against the trainer's schema before anything is written, versions it so that every run stays reproducible, and writes a manifest naming the exact command that loads it. - [RAG on Your Lakehouse: Serve Context From Structured and Unstructured Data in SQL](https://quanton.dev/blog/rag-on-documents.md): Your object storage holds structured data such as fact tables, exports and CDC feeds, and unstructured data such as PDF, TXT, MD and HTML files. Many teams keep those files on a NAS and read them through an object-storage client. This guide reads both kinds with one engine, and it parses contracts from the CUAD dataset on Hugging Face into Hudi tables that hold the chunks and the embeddings. One SQL query then answers a question, mixing filter, join and aggregate with vector similarity in the same plan. - [Index-Aware MERGE: Reading Only the Row Groups That Match](https://quanton.dev/blog/index-aware-merge.md): A JOIN or MERGE has to find the target rows that match your source keys. Spark tries to prune the target with dynamic partition pruning, but when updates land randomly across a large fact table it barely helps, so it falls back to scanning and shuffling most of the table. Quanton asks an index instead, mapping each key to the exact file and row position so the engine reads only the row groups that actually match. - [Why Native Spark Accelerators Get OOMKilled Under Memory Pressure](https://quanton.dev/blog/native-accelerator-oom.md): Native Spark accelerators move most of a query's working set out of the JVM heap and into off-heap native memory — and that is exactly where they tend to get OOMKilled. The surprising part is that the engine often frees its memory correctly; the kill happens anyway. Quanton is built to track live memory accurately, avoid fragmentation and spill under pressure instead of dying. - [Runtime Bloom Filters in Spark Joins: How Probe-Side Pruning Works](https://quanton.dev/blog/bloom-filters-before-joins.md): Spark applies Bloom filters to large fact tables before joining them with dimension tables, reducing the amount of data that must be shuffled and joined. Quanton preserves Spark's Bloom filter injection strategy while accelerating filter evaluation through vectorized execution. - [GROUP BY ROLLUP Without the Row Explosion](https://quanton.dev/blog/native-rollup-operator.md): ROLLUP is how SQL computes subtotals and grand totals in one query. Spark executes it by exploding every input row into N+1 copies before aggregating — a tax that grows with the depth of the rollup. Quanton replaces that with a rollup operator that re-aggregates already-collapsed state, removing the input-side explosion. - [Running Apache Spark on Snowflake Container Services (SPCS)](https://quanton.dev/blog/quanton-on-snowflake-spcs.md): We talked with 1,200+ data engineers at Snowflake Summit and Databricks Data+AI Summit. Only 15% of Snowflake users are on Iceberg. We think that's about to change. Today we're shipping Quanton on SPCS: real Apache Spark inside your Snowflake account, 2-5x better price/performance than Databricks with Photon, 63% fewer credits burned. - [Shuffle Hash Join vs. Sort-Merge Join: How Modern Engines Execute Equi-Joins](https://quanton.dev/blog/shuffle-hash-join-vs-sort-merge-join.md): SHJ and SMJ produce identical results but make opposite bets on CPU, memory, and data layout. A deep dive into how vectorized engines implement both — adaptive hash-table layouts, SIMD tag probing, normalized keys — and why Quanton defaults to a vectorized SHJ. - [Speeding Up Lakehouse Reads: Object-Store Latency, Parquet Decoding and Shuffle](https://quanton.dev/blog/accelerating-lakehouse-reads.md): High and unpredictable object-store access latencies, compute-intensive Parquet decoding, and shuffle preparation can dominate Spark job runtimes. Quanton is built to optimize this critical first mile. - [Native Clustering for Hudi and Iceberg Tables](https://quanton.dev/blog/clustering-acceleration.md): Clustering fixes the two file-layout problems every data team hits: too many files scanned, and small files piling up. The catch is that the sort-and-rewrite isn't free. Quanton ships a native execution path for clustering on Hudi and Iceberg. - [Low-Shuffle MERGE INTO for Apache Iceberg](https://quanton.dev/blog/low-shuffle-merge.md): A typical MERGE updates a tiny fraction of rows, yet OSS Apache Spark + Apache Iceberg shuffles the entire target table. Here's the shape of that pain — and the low-shuffle columnar MERGE path Quanton uses instead. - [Introducing Quanton: Spark at Full Speed](https://quanton.dev/blog/quanton-launch.md): Today we're launching Quanton, a drop-in compute engine that makes your Apache Spark workloads dramatically faster — no rewrites required.