# What is Databricks Photon? A plain-English guide to the engine (and when it's worth turning on)

> A plain-English guide to Databricks Photon: how the vectorized C++ engine works, which workloads it speeds up, what the DBU premium costs, and when to turn it on.

Published 2026-09-30 by Quanton Team (Quanton, by Onehouse).
Canonical: https://quanton.dev/blog/what-is-databricks-photon

---
Databricks Photon is a [native execution engine](https://docs.databricks.com/aws/en/compute/photon) that can make Spark queries run significantly faster without any code changes. On SQL warehouses it is included by default. On job and all-purpose clusters it raises the DBU rate, so a faster job is not automatically a cheaper one.

That trade-off is the question this guide answers: when does Photon actually lower your bill, and when does it just make an expensive job finish sooner? Photon accelerates some workloads substantially and barely touches others.

This guide starts with what Photon is and how it works. It then covers the workloads it accelerates and what it costs on a job cluster. The last sections show how to turn it on and how to decide whether it earns the premium.

## What is Databricks Photon?

Photon is Databricks' native query engine, written in C++. It replaces parts of Apache Spark's JVM-based execution layer, running the same query you wrote but executing it on faster machinery underneath. Databricks announced it in 2020 and made it [generally available in 2022](https://www.databricks.com/product/photon).

It is designed to be a drop-in replacement for Apache Spark's JVM-based engine. Photon is ANSI SQL-compliant and works with existing SQL and DataFrame code, so there are no rewrites and no new API to learn. The engine takes over the parts of the query plan it supports.

It does not support every operation. When a query reaches an operation Photon cannot run, that portion falls back to standard Spark automatically, with no error or warning. This silent fallback is convenient, and it is also where a significant amount of wasted spend can hide.

Where Photon runs affects its economics. On Databricks SQL warehouses and serverless compute it is enabled by default and included at no separate charge. On classic job and all-purpose clusters it is a toggle, and there it carries a DBU premium.

## How Photon works

Photon's performance comes from two design choices: it processes data in vectorized batches rather than row by row, and it executes in C++ rather than on the JVM.

### Vectorized, columnar execution

Standard Spark execution is row-oriented: whole-stage codegen fuses operators, but it does not exploit batch-level SIMD. Photon uses columnar, vectorized execution instead, operating on batches of column values rather than individual records. This matches how analytical queries access data: a few columns across many rows.

Processing columns in batches lets Photon use the CPU's SIMD instructions (AVX-512 on x86, NEON/SVE on ARM), which apply one operation across many values at once. Filters, aggregations, and join-key comparisons run across an entire batch in a single step rather than value by value.

The result is more efficient CPU-bound work with no code changes. The largest gains appear on the operations that dominate analytics: scans, filters, aggregations, and joins over large tables. On the other hand, Python or Java code inside Spark jobs remains unchanged and does not run on Photon.

### C++ over the JVM

The second design choice is the runtime language. Executing natively in C++ avoids garbage-collection pauses and gives the engine direct control over memory layout. On memory-intensive jobs, garbage collection alone can consume a meaningful share of runtime. C++ compilers also have better support for SIMD instructions than the JVM.

Photon operates within Databricks Spark rather than replacing it entirely. Spark's Catalyst optimizer still plans each query, and Photon connects to execution through JNI, handling the physical operators it supports while the rest of the stack is unchanged.

Because of this design, Photon requires no change to how you submit work. The same job runs against the same catalog, and only the execution speed of the supported stages changes.

## Databricks Photon performance: how fast, really?

Databricks publishes strong performance figures, and they should be read as vendor benchmarks rather than guarantees. The main claims are up to roughly 12x faster on certain SQL queries, about 2x over the previous Databricks Runtime generation, and up to 80% lower TCO in [Databricks' published benchmarks](https://www.databricks.com/product/photon). These figures reflect Databricks' own test workloads. Inside Databricks, Photon clusters consume DBUs [2.9x](https://azure.microsoft.com/en-us/pricing/details/databricks/) faster than non-Photon clusters (known as DBU emission rate).

In practice, Photon's gains depend on data volume and query type. Large scans, wide aggregations, and multi-table joins over big tables benefit the most. Independent analyses reach the same conclusion, from [Unravel's Photon guide](https://www.unraveldata.com/resources/when-to-use-databricks-photon) to [LakeSentry's cost breakdown](https://lakesentry.io/blog/databricks-photon-cost/): Photon delivers large gains on suitable workloads and little on others.

Small and I/O-bound jobs benefit the least. A query that spends most of its time waiting on object storage, or that completes in under two seconds, offers little CPU work for Photon to accelerate. Photon accelerates computation, so jobs bound by storage or network see little change.

Treat every published figure as a starting point. The number that decides anything is the one you get by running a representative job on the same cluster twice, once with Photon and once without.

## When Photon helps, and when it doesn't

Photon does not offer uniform speedup. Some jobs run dramatically faster, and others see almost no gain.

| Photon accelerates | Photon skips |
| :---- | :---- |
| Large scans, joins, and aggregations | UDFs and RDD/Dataset-API code |
| Delta/Parquet vectorized reads and writes | pandas and driver-side work |
| BI on SQL warehouses, wide tables | Model training and stateful streaming |
| High-volume nightly ETL |  |

### Workloads Photon accelerates: large scans and joins

Photon performs best on large-scale ETL and SQL-heavy analytics: large scans, multi-table joins, and heavy aggregations over sizeable Delta or Parquet tables. These are the CPU-bound, columnar operations that vectorized execution and SIMD are designed for.

It also performs well on vectorized reads and writes against Delta and Parquet, on wide tables with many columns, and on BI queries served from SQL warehouses. High-volume dashboard traffic and large nightly transforms are typical cases where the premium is justified.

The common factor is volume and compute intensity. The more of a job's runtime goes to processing large batches of data, the more Photon accelerates it.

### Workloads Photon skips: UDFs, streaming, and the fallback trap

Photon provides little or no benefit on several common patterns. Code written against the RDD or Dataset APIs, custom UDFs, single-node or driver-side work (including pandas), model training, and stateful streaming all fall outside what Photon accelerates. Queries that complete in under two seconds also see essentially no gain.

The risk is the silent fallback. Unsupported operations revert to standard Spark without any error, so a job can run partly or entirely on standard Spark while the cluster is still billed at the Photon rate. In such cases, you pay the premium DBU emission rate and get none of the speedup. In fact, with partial fallback, row-to-columnar conversions at each boundary can make the job slower than plain Spark.

To avoid this, confirm that Photon is materially faster before relying on the cost. Check the Spark UI query plan for Photon operators and verify that the stages you care about ran natively rather than falling back. A job dominated by UDF logic will not benefit from Photon regardless of the cluster configuration.

The same cluster can run a Photon-friendly transform in the morning and a UDF-heavy job in the afternoon, and only the first justifies the higher rate.

## Databricks Photon cost: the DBU multiplier

On job clusters, Photon is not free, and whether it lowers your bill comes down to a single calculation.

### How the DBU premium works

On classic job and all-purpose clusters, enabling Photon raises the DBUs consumed per hour by up to about 2.9x, depending on SKU and cloud. Independent analyses, including [Miles Cole's TCO study](https://milescole.dev/data-engineering/2024/04/30/Is-Databricks-Photon-A-NoBrainer.html), LakeSentry, and the broader Databricks community, report about 2x on all-purpose compute and, for jobs compute, about 2.9x on AWS and 2.5x on Azure and GCP.

This premium applies only to clusters where Photon is a toggle. On SQL warehouses and serverless, Photon is enabled by default and priced into the compute rate rather than charged as a separate multiplier.

Rates and multipliers vary by compute tier, cloud, and region, and they change over time. Calculate against [current Databricks pricing](https://www.databricks.com/product/pricing) for your tier rather than assuming a fixed number.

### Does it actually lower your bill?

Databricks bills DBUs as cluster size times hours times rate, so a job that finishes in half the time on the same cluster costs roughly half as much. A faster engine is therefore a direct cost lever.

The premium changes that. On classic compute you pay for EC2 plus DBUs, and the multiplier applies only to the DBU part, while a speedup cuts both. Since instance cost is the same with or without Photon, the fair comparison is the platform fee, and break-even lands below the DBU multiplier. Example: at $1.00/node-hr EC2, $0.60/node-hr DBUs, and a 2.9x multiplier, Photon breaks even at about a 1.7x speedup. Photon lowers the bill only when the speedup exceeds the premium; when the speedup is small, the bill goes up.

Some factors can shift the result. A faster engine can sometimes allow smaller or fewer nodes, or shorten runtime enough that the reduced hours offset the higher rate. In other cases it does not, and the job simply costs more for the same output. This has to be evaluated per workload.

For this reason, the more effective first step is often to remove compute waste rather than enable Photon everywhere. Idle executors, spill, garbage collection, stragglers, and retries inflate the bill regardless of engine. Our guide to [Databricks cost optimization](/blog/databricks-cost-optimization) explains how to find that waste in the Spark UI first.

## Enabling Photon

Enabling Photon is straightforward. On SQL warehouses and serverless it is already the default. On job and interactive clusters running Databricks Runtime 9.1 LTS or later, enable it with the [Use Photon Acceleration](https://docs.databricks.com/aws/en/compute/configure) checkbox in the cluster configuration, or by setting `"runtime_engine": "PHOTON"` in the clusters API or your infrastructure-as-code.

Enable it per workload rather than as a global default. Apply it to large, CPU-bound ETL and analytics jobs where it pays off, and leave it off for short, UDF-heavy, or otherwise unsuitable jobs. Enabling it on every cluster is how the premium can quietly exceed the savings.

Photon is a Databricks feature, so using it requires remaining on the Databricks platform. It is tied to DBUs and Databricks Runtime, which is appropriate if you are committed to Databricks and worth stating plainly if you are evaluating alternatives. The engine cannot be extracted and run elsewhere.

## Quanton: a Photon alternative that bills differently

If the concern with Photon is cost rather than performance, [the Quanton engine](/) is a Photon-class alternative with a different price structure. It is open core, includes a free tier, and reverts to open-source Spark with a single configuration line.

### A drop-in Spark engine you run yourself

Quanton fills the same role as Photon: it replaces the Spark execution engine, with no code changes. Built on optimized and reimplemented Meta’s Velox engine operators, it runs in your own cloud account (BYOC) on your existing Kubernetes or Spark platform and makes existing jobs up to 6x faster than open-source Spark. Similar to Photon, it replaces only the execution layer, so data sources, job definitions, orchestration, and catalogs stay in place.

The practical difference is where it runs and how it bills. Quanton runs in your own account and bills per GiB of data processed instead of per compute-hour. Per-hour billing gives a vendor reason not to make the engine faster. Per-GiB does not, so Quanton can keep getting faster while your cost stays flat.

### Photon vs Quanton on identical hardware

On performance, Quanton and Photon are in the same class. On a full TPC-DS 10 TB run (all 99 queries, the same 11-node cluster), Quanton finished in 2,034 seconds and Photon in 2,550 seconds, so Quanton was 25% faster. Current results are published on the [Quanton benchmarks page](/benchmarks).

Cost is where the two differ. On that same identical-hardware run, Quanton's platform fee was $5.38 against $15.49 in Photon DBU fees, 65% lower for the same work. Measured as total cost of ownership, the gap holds: Quanton's [TCO calculator](/pricing) puts one TPC-DS 10 TB run at roughly the following.

| Setup (one TPC-DS 10 TB run) | Cost per run | vs Quanton |
| :---- | :---- | :---- |
| Quanton (BYOC) | ~$9.84 | baseline |
| Databricks Classic + Photon | ~$21.08 | 2.1x |
| Databricks Serverless + Photon | ~$36.13 | 3.7x |

Figures: TPC-DS 10 TB, all 99 queries, identical hardware. Estimates only, and rates move, so treat them as a checkpoint.

There is no DBU markup. And unlike Databricks Serverless, where instance discounts are bundled into the platform rate, Quanton runs in your own account, so your reserved and spot discounts remain yours. Because billing is per GiB rather than per hour, each future speedup lowers your cloud bill instead of a vendor's revenue.

If Photon's economics work for you inside Databricks, keep it enabled. If they do not, run a side-by-side comparison on the same jobs at the same time.

## Should you turn Photon on?

Photon is a capable engine. It is a native, vectorized, C++ execution layer that accelerates large, CPU-bound SQL and ETL work without code changes, and on SQL warehouses and serverless it is enabled by default and included in the price. Where it fits, enabling it is a clear decision.

The judgment required is on the DBU premium on job clusters. The general rule is the 2–3x DBU multiplier: Photon pays off when the speedup on your workload clearly exceeds it, breaks even when they are comparable, and costs more when the job is short, UDF-heavy, or falling back to standard Spark. Enable it per workload, and confirm in the Spark UI that it engaged before relying on the result.

Enable Photon where the economics work inside Databricks, and where they do not, consider a per-GiB engine that runs in your own account and bills for data processed rather than compute hours. In both cases, base the decision on your own measurements.

Want a second number to compare against your Databricks bill? [**Create an account**](https://cloud.onehouse.ai/signup) and run Quanton side by side with Photon on your own jobs, in your own VPC.

## Frequently asked questions

### Is Photon enabled by default?

This depends on the compute type. On Databricks SQL warehouses and serverless compute, Photon is enabled by default. On classic job and all-purpose clusters (Databricks Runtime 9.1 LTS and later), the Use Photon Acceleration checkbox controls it, and via the Clusters or Jobs API you set runtime_engine to PHOTON. Enabling it on these clusters is what adds the DBU premium.

### Does Photon cost extra, and how much more?

On job and interactive clusters, yes. Enabling Photon raises the DBUs consumed per hour by about 2.9x, depending on tier, cloud, and region. On SQL warehouses and serverless it is priced into the compute rate rather than charged separately. Because rates change, calculate against current Databricks pricing for your tier, and note that the premium is only worthwhile when the speedup exceeds it.

### Does Photon support UDFs and streaming?

Not fully. Photon does not accelerate custom UDFs or RDD and Dataset-API code, and it supports stateless streaming only rather than stateful streaming. When a query includes an operation Photon cannot run, it falls back to standard Spark silently, so check the Spark UI query plan to confirm the engine engaged on the stages that matter.

### Can I get Photon-class speed without Databricks?

Yes. Quanton is a drop-in replacement for the Spark execution engine that finished a full TPC-DS 10 TB run 25% faster than Photon on identical hardware, while running in your own cloud account and billing per GiB of data processed rather than per compute-hour. It is open core with a free tier and reverts to open-source Spark with one configuration line, so it can run side by side against your current setup without a migration. Current benchmarks and pricing are published on the Quanton site.