Environment Variables¶

The project is configured through environment variables defined in a .env file at the project root. Docker Compose reads this file and injects the variables into each service container.

Overview¶

An example configuration file is provided as .env.ml.example. The recommended setup path is:

just setup

This generates secrets and creates the .env file. You can also do it manually:

cp .env.ml.example .env
chmod 600 .env

Complete Reference¶

Quantum Pipeline Settings¶

Variable	Default	Description
`LOG_LEVEL`	`INFO`	Logging verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR`.
`SIMULATION_METHOD`	`statevector`	Qiskit Aer simulation method: `statevector`, `density_matrix`, `automatic`, `matrix_product_state`, `tensor_network`.
`CONTAINER_TYPE`	`unknown`	Label identifying the container hardware config (e.g. `CPU`, `GPU_GTX1060_6GB`). Set in Docker Compose per service, read by the performance monitor and VQE runner for tagging results.

IBM Quantum (Optional)¶

Environment variables for IBM Quantum cloud backend. Only needed if running VQE on real hardware or IBM cloud simulators via QiskitRuntimeService.

Variable	Default	Description
`IBM_RUNTIME_CHANNEL`	-	IBM Quantum channel: `ibm_quantum`, `ibm_cloud`, or `local`.
`IBM_RUNTIME_INSTANCE`	-	IBM Quantum service instance identifier.
`IBM_RUNTIME_TOKEN`	-	IBM Quantum API token.

Kafka Configuration¶

Variable	Default	Description
`KAFKA_VERSION`	`4.2.0`	Apache Kafka Docker image tag.
`KAFKA_SERVERS`	`kafka:9092`	Kafka bootstrap servers. Also read by the CLI entrypoint at runtime - when set, overrides the `--servers` flag.
`KAFKA_EXTERNAL_HOST_IP`	`localhost`	IP/hostname for external Kafka access.
`KAFKA_EXTERNAL_PORT`	`9094`	External Kafka listener port.
`KAFKA_INTERNAL_PORT`	`9092`	Internal Kafka listener port.
`KAFKA_NODE_ID`	`1`	KRaft node ID.

Schema Registry¶

Variable	Default	Description
`SCHEMA_REGISTRY_VERSION`	`8.2.0`	Docker image tag.
`SCHEMA_REGISTRY_TOPIC`	`_schemas`	Internal Kafka topic for schema storage.
`SCHEMA_REGISTRY_HOSTNAME`	`schema-registry`	Schema Registry hostname.
`SCHEMA_REGISTRY_PORT`	`8081`	HTTP listener port.
`SCHEMA_REGISTRY_URL`	`http://schema-registry:8081`	Full URL for Schema Registry. Used by batch simulation containers in Docker Compose. Not in `.env.ml.example` (defaults in compose).

Garage Configuration (S3-compatible Storage)¶

Variable	Default	Description
`GARAGE_VERSION`	`v2.2.0`	Garage Docker image tag.
`GARAGE_S3_API_PORT`	`3901`	S3 API port.
`GARAGE_RPC_PORT`	`3900`	Internal RPC port.
`GARAGE_ADMIN_PORT`	`3903`	Admin API port.
`GARAGE_WEB_PORT`	`3902`	Web endpoint port.
`GARAGE_RPC_SECRET`	-	32-byte hex RPC secret (generated by `just setup`).
`GARAGE_ADMIN_TOKEN`	-	32-byte hex admin token (generated by `just setup`).
`S3_ACCESS_KEY`	-	S3 access key (generated by `just setup` after first start).
`S3_SECRET_KEY`	-	S3 secret key (generated by `just setup` after first start).
`S3_REGION`	`garage`	S3 region identifier.
`S3_ENDPOINT`	`http://garage:3901`	S3 endpoint URL.
`S3_RAW_BUCKET`	`raw-results`	Bucket for raw VQE results.
`S3_FEATURES_BUCKET`	`features`	Bucket for processed feature datasets.
`S3_ICEBERG_BUCKET`	`warehouse`	Bucket for Iceberg warehouse data.

Airflow Configuration¶

Variable	Default	Description
`AIRFLOW_VERSION`	`3.1.8`	Airflow base image tag.
`AIRFLOW_POSTGRES_USER`	`airflow`	PostgreSQL username.
`AIRFLOW_POSTGRES_PASSWORD`	`airflow-password`	PostgreSQL password.
`AIRFLOW_POSTGRES_DB`	`airflow`	PostgreSQL database name.
`AIRFLOW_FERNET_KEY`	-	Fernet encryption key for connection credentials. See Security Considerations.
`AIRFLOW_WEBSERVER_SECRET_KEY`	-	Secret key for webserver session signing. See Security Considerations.
`AIRFLOW_JWT_SECRET`	-	JWT secret for Airflow 3.x API authentication. See Security Considerations.
`AIRFLOW_WEBSERVER_PORT`	`8084`	Host port for the Airflow web UI.
`AIRFLOW_DAGS_PAUSED_AT_CREATION`	`True`	Whether new DAGs start paused.
`AIRFLOW_LOAD_EXAMPLES`	`False`	Load Airflow example DAGs.
`AIRFLOW_ADMIN_USERNAME`	`admin`	Initial admin username.
`AIRFLOW_ADMIN_PASSWORD`	`admin`	Initial admin password.
`AIRFLOW_ADMIN_FIRSTNAME`	`Admin`	Admin user first name.
`AIRFLOW_ADMIN_LASTNAME`	`User`	Admin user last name.
`AIRFLOW_ADMIN_EMAIL`	`admin@example.com`	Admin user email.
`POSTGRES_VERSION`	`16-alpine`	PostgreSQL Docker image tag.

Spark Configuration¶

Variable	Default	Description
`SPARK_VERSION`	`4.0.2`	Spark version (used in image tags).
`SPARK_MASTER_HOST`	`spark-master`	Spark master hostname.
`SPARK_MASTER_PORT`	`7077`	Spark master RPC port.
`SPARK_WORKER_MEMORY`	`2G`	Memory allocated per Spark worker.
`SPARK_WORKER_CORES`	`2`	CPU cores allocated per Spark worker.
`SPARK_DEFAULT_QUEUE`	`default`	Default resource queue for Spark jobs.

MLflow Configuration¶

Variable	Default	Description
`MLFLOW_VERSION`	`v3.10.1`	MLflow Docker image tag.
`MLFLOW_PORT`	`5000`	MLflow tracking server port.
`MLFLOW_TRACKING_URI`	`http://localhost:5000`	MLflow tracking server URI. Used by the `quantum_pipeline.ml.tracking` module to locate the tracking server. Not set in `.env.ml.example` (defaults to `http://localhost:5000` in code).

Monitoring Configuration¶

These variables control the performance monitoring subsystem. The code default for MONITORING_ENABLED is false (in settings.py), but .env.ml.example sets it to true so monitoring is on by default in Docker Compose deployments.

Variable	Default	Description
`MONITORING_ENABLED`	`false` (code) / `true` (.env example)	Enable performance metric collection. Accepts `true`, `1`, `yes`, `on`.
`MONITORING_INTERVAL`	`10`	Metric collection interval in seconds.
`PUSHGATEWAY_URL`	`http://localhost:9091`	Prometheus PushGateway URL for pushing performance metrics.
`MONITORING_EXPORT_FORMAT`	`prometheus`	Export format: `json`, `prometheus`, or both (comma-separated, e.g. `json,prometheus`).

Docker Build Configuration¶

Variable	Default	Description
`DOCKER_GID`	`970`	GID of the `docker` group on the host. Used to build the Airflow image so it can access the Docker socket for batch generation DAGs. Check with `stat -c '%g' /var/run/docker.sock`.
`CUDA_ARCH`	`6.1 7.0 8.0 9.0+PTX` (Dockerfile) / `6.1` (.env.ml.example)	CUDA compute capabilities for the GPU image build. The Dockerfile.gpu default covers Pascal to Hopper and Blackwell (via PTX). A single value builds about 4x faster and is only useful for local builds; `.env.ml.example` sets `6.1` (Pascal) to match the project's test hardware.
`QUANTUM_PIPELINE_HOST_ROOT`	`$HOME/code/quantum-pipeline`	Absolute path to the quantum-pipeline repo on the host. Used by `docker-compose.ml.yaml` and the batch generation script (`scripts/generate_ml_batch.py`) to resolve volume mount paths. The compose file defaults to the maintainer's local path; override this for your own system.

R2 Sync Configuration¶

Optional - only needed if syncing data to Cloudflare R2.

Variable	Default	Description
`R2_ACCOUNT_ID`	-	Cloudflare account ID.
`R2_ACCESS_KEY_ID`	-	R2 access key.
`R2_SECRET_ACCESS_KEY`	-	R2 secret key.
`R2_BUCKET`	`qp-data`	R2 bucket name.

Security Considerations¶

Never commit .env to version control and set restrictive permissions (chmod 600 .env). Generate strong keys for Airflow:

# Fernet key
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"

# Webserver secret key
python -c "import secrets; print(secrets.token_hex(32))"

# JWT secret (Airflow 3.x)
python -c "import secrets; print(secrets.token_hex(32))"

The just setup script generates these automatically.

For tighter security, restrict service port exposure by binding to 127.0.0.1 (e.g., "127.0.0.1:8081:8081") and use unique passwords for each service.