Changelog

Release notes for Quantum Pipeline, starting from version 1.4.0.

For earlier versions, see the GitHub releases.

---

2.1.0

This version includes fixes for issue #52 reported by @avnerbensoussan.

ExcitationPreserving ansatz

The ExcitationPreserving ansatz now works instead of getting stuck near the Hartree-Fock energy. It prepends the Hartree-Fock state as the circuit's initial state.

Full entanglement is used and optimization starts from a small jitter off zero parameters. The jitter matters because zero parameters sit exactly on the HF reference, a stationary point where the optimizer stalls; noise lets it descend into the correlation energy.

Full entanglement is required because adjacent-only gates can only slide electrons between neighbouring orbitals (a Slater determinant in, a Slater determinant out). It leaves the correlated states unreachable. With these changes it reaches chemical accuracy and improves with more reps.

Error is now raised when Hartree-Fock data is not provided, rather than silently producing a meaningless result.

Reproducible seeding

A fixed --seed now reaches the Aer simulator (seed_simulator) and the transpiler (seed_transpiler). Two runs with the same seed now produce identical results.

Exact statevector mode

A new --exact flag computes exact expectation values through the Aer EstimatorV2 (default_precision=0.0) with no shot noise. It overrides --shots.

Minimum energy reporting

The reported minimum is now the best evaluated iteration: the energy and the parameters come from the same step, rather than pairing the lowest observed energy with whatever parameters happened to be stored alongside it. This applies to every run, not only exact mode.

Noise model fix

--noise now reaches the backend. The flag was previously dropped before the backend configuration was built, so noisy runs silently ran noiseless and were recorded as if they were not.

Spark incremental processing

Incremental processing now derives a deterministic experiment_id from the experiment's own fields instead of a fresh value on each run, so re-processing no longer duplicates the dataset every day.

Batch generation

The batch run key now includes the basis set.

Tiers differ by basis set while sharing molecule, optimizer, init strategy, seed, and ansatz, so without the basis in the key a finished Tier 1 (sto3g) run marked the higher-basis tiers as done and they were silently skipped.

Kafka delivery

A failed Kafka send is no longer dropped silently. Undelivered results are spooled to disk (gen/undelivered/) for later replay and the run exits non-zero, so a downed broker cannot discard computed output while the job still reports success.

---

2.0.0

Complete rewrite of the simulation module, DAGs, tests, and infrastructure. Adds batch generation, multiple ansatze and optimizers, a full observability stack, and a storage migration from MinIO to Garage. Removes legacy code paths and simplifies metrics.

Removals

• quantum-pipeline.py root script removed. The package is now invoked via quantum_pipeline.cli:main entry point.
• Chemical accuracy metric removed from simulation output and tests.
• Scientific literature references replaced with simpler PySCF-derived metrics (HF energy, nuclear repulsion) throughout the codebase.

Batch generation

A batch system for generating ML training data across many configurations. Four tiers sweep combinations of basis set, optimizer, init strategy, seed, and ansatz. Three hardware lanes (two GPU, one CPU) run concurrently via ThreadPoolExecutor, and state is persisted to gen/ml_batch_state.json for idempotent resume. Progress metrics (qp_batch_*) are pushed to the PushGateway per tier and lane.

Multiple ansatze

The --ansatz flag selects between EfficientSU2 (default), RealAmplitudes, and ExcitationPreserving.

Extended optimizer support

Eight optimizers are used in batch generation: L-BFGS-B, COBYLA, SLSQP, Nelder-Mead, Powell, BFGS, CG, and TNC. Additional scipy optimizers (Newton-CG, COBYQA, trust-constr, dogleg, trust-ncg, trust-exact, trust-krylov) are available but less tested.

Observability

PerformanceMonitor collects system and experiment metrics and exports them to the Prometheus PushGateway. Monitoring environment variables renamed from QUANTUM_PERFORMANCE_* to shorter names: MONITORING_ENABLED, MONITORING_INTERVAL, PUSHGATEWAY_URL, MONITORING_EXPORT_FORMAT. Metrics use the qp_ prefix (qp_vqe_* for experiment metrics, qp_sys_* for system metrics, qp_batch_* for batch progress).

Grafana dashboards added for VQE experiment tracking, batch progress, and system resource usage (monitoring/grafana/dashboards/).

Full exporter stack in the Docker Compose environment: statsd-exporter, postgres-exporter, redis-exporter, nvidia-gpu-exporter, and Garage S3 metrics. Prometheus scrape configuration covers all exporters and the PushGateway.

Storage migration

MinIO replaced by Garage (dxflrs/garage) as the S3-compatible object store. Environment variables: S3_ENDPOINT (default http://garage:3901), S3_REGION, S3_ACCESS_KEY, S3_SECRET_KEY. Feature tables are synced to Cloudflare R2 via the R2 sync DAG.

Docker

• GPU image updated to CUDA 12.6.3, builds qiskit-aer from source with cuQuantum. CUDA_ARCH build argument controls the target GPU architecture (default 8.6).
• DOCKER_GID build argument for Airflow container to access the Docker socket (default 970).

Simulation module

Runner, Solver, and Mapper base classes now use proper ABCs. Hardcoded constants (HF fidelity threshold, per-optimizer default iterations) were moved to configs/constants.py.

The L-BFGS-B optimizer config had a workaround that set ftol and gtol to 1e-15 when max_iterations was active, to prevent early convergence. This was redundant - compute_energy() already enforces the iteration limit by raising MaxFunctionEvalsReachedError. The hack also prevented natural convergence, which is arguably worse for data collection. Removed.

DAGs

Configuration that was duplicated across 4 DAG files and 2 Spark scripts (S3 paths, default args, Spark session setup) now lives in docker/airflow/common/. Retries use exponential backoff. Spark tasks have SLA targets. A dedicated batch generation DAG orchestrates ML data collection runs, including container image builds. Airflow logic simplified by removing overly complex scheduling and retry paths.

Tests

Solver tests were spread across 8 files with overlapping coverage (4 files testing VQESolver, 3 testing optimizer config). Consolidated to 4 files. Deleted 6 tests that were testing scipy.optimize.minimize behavior rather than project itself.

Added pytest-xdist for parallel test execution and @pytest.mark.slow for a fast inner loop (pytest -m "not slow" runs in ~38s).

Verification

Ran the refactored simulation across a range of configurations. All combinations completed correctly on CPU (Aer statevector, seed 42, EfficientSU2 2 reps unless noted). The table below is a partial sample; more combinations are still being verified.

Optimizer	Basis	Init	Iters	H\(_2\) (Ha)	HeH\(^+\) (Ha)	Notes
COBYLA	sto-3g	random	30	-1.555	-4.004	Completed naturally
L-BFGS-B	sto-3g	random	30	-0.895	-3.283	Hard limit
SLSQP	sto-3g	random	50	-1.111	-3.562	RealAmplitudes 3r
Nelder-Mead	sto-3g	random	50	-0.553	-3.200
Powell	sto-3g	random	30	-0.005	-0.011	ExcitationPreserving
COBYLA	sto-3g	random	235/221	-1.785	-4.338	Convergence 1e-4
COBYLA	sto-3g	hf	50	-1.836	-4.267
BFGS	sto-3g	hf	40	-1.838	-4.224
L-BFGS-B	sto-3g	hf	30	-1.838	-4.218
L-BFGS-B	6-31g	random	1029/1372	+2.101	-0.956	H\(_2\) stuck in local min
L-BFGS-B	6-31g	hf	50	-1.857	-4.294	HF avoids local min

The last two rows are a good example of why initialization matters. L-BFGS-B with random init on 6-31g spent 1029 iterations to arrive at a positive energy for H\(_2\) - a barren plateau. The same setup with HF init reached -1.857 Ha in 50 iterations.

---

1.4.0

Hartree-Fock Initialization

The random parameter initialization used in prior versions (uniform \([0, 2\pi)\)) was identified as a major contributor to poor convergence (see Experimental Observations).

Version 1.4.0 adds --init-strategy hf, which attempts to start VQE from the classical Hartree-Fock solution instead.

A naive implementation - prepending a HartreeFock circuit to EfficientSU2 and setting all parameters to zero - turned out not to work. The fixed CX entangling gates in EfficientSU2 are not parameterized and always act, regardless of rotation angles. At zero parameters the rotation gates become identity, but the CX gates still scramble the HF state.

The current approach runs a short classical pre-optimization that finds EfficientSU2 parameters which directly prepare the HF state through the ansatz, by maximizing state fidelity via Statevector simulation. This partially resolves the initialization problem - for H\(_2\) (reps=1, 16 parameters) the pre-optimization reliably achieves fidelity of 1.0, while for H\(_2\)O (reps=2, 72 parameters) it reaches approximately 0.993.

Early results comparing v1.3.x (random init) against v1.4.0 (HF init) under the same conditions (sto-3g, COBYLA, reps=2, statevector, GPU, max 300 iterations):

H\(_2\) (4 qubits, 32 parameters, HF reference: -1.117 Ha):

Version	Init	Best electronic	Total energy	Iterations	Time
v1.3.x	Random	-1.73 Ha	n/a	300	23 s
v1.4.0	HF	-1.837 Ha	-1.122 Ha	187	15 s

The random run oscillated around -1.68 to -1.73 Ha without settling, while HF init converged in 187 iterations (below the 300 limit) and found energy within 5 mHa of the HF reference. Pre-optimization fidelity: 1.000.

H\(_2\)O (12 qubits, 72 parameters, HF reference: -74.963 Ha):

Version	Init	Best electronic	Total energy	Iterations	Time
v1.3.x	Random	~-75.95 Ha	n/a	300	965 s
v1.4.0	HF	-84.209 Ha	-75.016 Ha	300	660 s

The random run showed large oscillations at the end (costs jumping between -73 and -76 Ha between iterations), indicating the optimizer was struggling with the landscape. The HF run produced a stable result 53 mHa below the HF reference, capturing some correlation energy. Pre-optimization fidelity: 0.993.

Note

v1.3.x did not account for nuclear repulsion energy, so total energy is not available for those runs. The electronic energies are not directly comparable between versions - the v1.4.0 total energy column is what should be compared against literature values.

These results are encouraging but there is a lot more work to be done - particularly around scaling the pre-optimization to larger molecules and higher ansatz reps, where fidelity drops and the approach becomes slower. Alternative strategies such as ADAPT-VQE or layer-by-layer parameter fitting have not been explored yet.

Nuclear Repulsion Energy

VQE optimizes the electronic Hamiltonian, which does not include nuclear repulsion energy. Previous versions reported only the raw VQE minimum, making it difficult to compare results against literature values directly.

The pipeline now captures nuclear repulsion energy from PySCF and adds it to the VQE result. VQEResult exposes a total_energy property (electronic + nuclear repulsion), and both components are logged:

Best energy: -1.83805736 Ha
Total energy (electronic + nuclear repulsion): -1.12295302 Ha
  (nuclear repulsion: 0.71510434 Ha)

Code Organization

• quantum_pipeline/circuits/ package - HFData and build_hf_initial_state moved from solvers/ into a dedicated module, since initial state construction is circuit-level work rather than solver logic.
• Mapper wrapper - Mapper.get_qiskit_mapper() provides access to the underlying qiskit-nature mapper, so that circuit construction does not depend on direct qiskit imports.

Bug Fixes

• Timer context manager - accessing Timer.elapsed inside a with block before __exit__ ran caused ValueError when MaxFunctionEvalsReachedError was raised. The exception is now handled after the timer block exits.
• Convergence vs max-iterations - the --convergence flag no longer conflicts with the default max_iterations=100. When convergence mode is enabled, max_iterations is cleared unless explicitly set.
• Simulation method validation - GPU-only methods now produce a clear error when --gpu is not enabled.
• Ansatz reps parsing - --ansatz-reps is correctly parsed as an integer.

Known Limitations

The items below are either known to be incomplete or are being actively worked on:

• HF pre-optimization scaling - for molecules with many qubits and high ansatz reps, the fidelity pre-optimization may not converge to a satisfactory starting state. This is a known weakness of the current approach.
• EfficientSU2 ansatz - the hardware-efficient ansatz still does not preserve particle number or spin symmetry. The anomalies documented in the thesis (e.g. HeH\(^+\) sub-FCI energy) remain possible.