Gate fidelity is the central performance metric for a quantum computing platform. It is also one of the more easily misrepresented numbers in the field, because the same gate can produce very different fidelity estimates depending on which measurement technique you use and what assumptions you make during data analysis. This post describes the three main benchmarking techniques we use, what each one measures, where each one gives misleadingly optimistic results, and how we combine them for calibration feedback and for reporting to early-access partners.
Randomized benchmarking
Standard randomized benchmarking (RB) estimates the average single-qubit gate error rate by applying random sequences of Clifford gates of increasing length and measuring the survival probability of the initial state. The survival probability decays exponentially with sequence length; the decay rate gives the average error per Clifford gate. Because Clifford gates form a group, each random Clifford sequence can be inverted exactly, making the final state known and allowing clean survival probability measurement without process tomography.
RB has two important properties that make it valuable for calibration feedback. First, it is highly efficient: you only need to measure a small number of random sequences at each length to get a good estimate of the decay rate, and the total number of shots scales as approximately 200 sequences times 20 lengths, which takes a few minutes on our system. Second, it is insensitive to SPAM (state preparation and measurement) errors, because SPAM errors appear as a constant offset in the survival probability that cancels out of the exponential decay rate.
RB also has important limitations. The number it reports is the average Clifford gate error rate, not the error rate for any specific gate. And crucially, it assumes the error channel is gate-independent (the same depolarizing channel for every gate in the sequence). If some gates in the native gate set are much worse than others, RB will average over them and may not detect the outlier. For our platform, the single-qubit RB number tracks overall calibration quality well; it does not identify which specific rotation angles have calibration errors without additional per-gate analysis.
For two-qubit gates, we run interleaved RB: alternating random single-qubit Clifford sequences with a fixed two-qubit CZ gate. The ratio of the interleaved decay rate to the reference (CZ-free) decay rate gives the isolated CZ error rate. Interleaved RB is reliable when the gate being characterized is close to ideal; when the gate has large coherent errors, the interleaved ratio can underestimate the true error rate because the coherent errors partially cancel in the random interleaving.
Cycle benchmarking
Cycle benchmarking (CB) is an extension of RB designed for characterizing the error properties of a complete circuit layer (a "cycle" of gates applied simultaneously to multiple qubits). Where standard RB characterizes a single-qubit gate in isolation, CB characterizes the dressed noise of an entire parallel gate layer, including crosstalk between simultaneously driven qubits.
The technique uses simultaneous randomized sequences on all qubits in the array, interleaved with the cycle being characterized. The measured decay provides an estimate of the dressed error rate for the full cycle, capturing effects that RB on individual qubits would miss: off-resonant excitation of neighboring qubits during a gate, global phase errors from multiple simultaneous AOMs, and coherent crosstalk through the Rydberg interaction channel.
We run CB regularly because our AI calibration system uses the per-cycle error rate as one of its observation inputs. A sudden increase in CB error rate that is not reflected in single-qubit RB indicates that crosstalk has increased, which triggers the calibration system to examine the correlation between simultaneously activated sites. This diagnostic capability would be invisible from single-qubit benchmarks alone.
CB's main practical limitation is cost: it requires running the full array simultaneously, which consumes the entire system for the duration of the benchmark and cannot be interleaved with science experiments the way single-qubit RB can. We run full CB at the start of each experimental session rather than continuously.
Gate set tomography
Gate set tomography (GST) provides the most complete characterization of gate errors, producing a full process matrix for each gate in the native gate set as well as the state preparation and measurement operations. Unlike RB, which reports a single error rate number, GST reveals the full error channel: which specific Pauli error components are present, whether errors are coherent (systematic rotations) or incoherent (stochastic noise), and whether the gate is performing a correct unitary perturbed by noise or a systematically incorrect unitary.
The distinction between coherent and incoherent errors matters for different reasons. Coherent errors (like a 1% miscalibration in the rotation angle of a Z gate) can be corrected by re-calibrating the gate pulse duration. They cause fidelity loss that grows quadratically with the number of gates applied (because coherent errors can add constructively), which makes them disproportionately harmful for deep circuits. Incoherent errors cause fidelity loss that grows linearly and are irreducible without improving the physical hardware.
We run GST on a per-experiment basis when we observe unexplained fidelity degradation and want to identify the root cause. It is too time-intensive for continuous monitoring: a complete GST run on a single qubit pair requires several thousand circuit executions and takes approximately 2 hours. Running GST on all 100 qubits would be prohibitively slow. Our practice is to maintain an always-on RB monitor and trigger a targeted GST run on a subset of qubits when RB detects a calibration regression.
What we report and why
When we report gate fidelity numbers to early-access partners or in internal documentation, we use interleaved RB for single and two-qubit gates in normal monitoring, and GST for detailed characterization of specific gate implementations. We do not report process fidelity from process tomography without simultaneously noting the SPAM error contribution, because uncorrected SPAM can inflate apparent gate fidelity significantly.
We also do not report the best fidelity we have ever achieved; we report typical operational fidelity from the most recent 30-day period. There is a significant difference between a peak fidelity achieved under ideal conditions on the best-performing qubit pair and the fidelity that research partners will routinely experience when running their own circuits. Reporting the former while implying the latter is a practice we see in the field and actively try to avoid.
One thing all three techniques agree on: the current limiting factor for two-qubit gate fidelity on our platform is thermal motion, specifically the finite temperature of atoms in the trap contributing a distribution of interaction strengths to the Rydberg blockade gate. This is visible in GST as a coherent error that varies slowly across an experimental session as the atom temperature changes, visible in RB as a slow drift in the per-gate error rate on the scale of tens of minutes, and visible in CB as a correlated error that is larger for pairs with higher sensitivity to the interaction strength. All three diagnostic tools are pointing at the same underlying physics, which gives us confidence in the diagnosis and in the path to address it through sideband cooling.