Skip to content

Environment and credentials

The engine takes per-invocation choices as CLI flags and a deployment’s standing configuration as environment variables. This page lists every variable the engine reads. If a variable is missing from this page, the engine does not read it.

Where the published dataset lives

A real build reads the published dataset to detect what is already there and to seed history appends. There is no default root. Configure exactly one of the two modes, or the build exits with PublisherConfigurationError.

VariableMeaning
METEO_DATA_BASEThe published dataset’s public HTTPS root (for example https://forecasts.your.club/data). On day one, before anything is published there, the URL returns 404 and the engine correctly reads that as “nothing published yet”.
METEO_R2_BUCKETS3 mode: the dataset bucket’s name.
METEO_S3_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEYS3 mode: the endpoint and credential pair. R2_ENDPOINT is honored as an alias for deployments that configured it before the vendor-neutral name existed.

S3 mode is active only when METEO_DATA_BASE is unset and all three of the endpoint (METEO_S3_ENDPOINT or its R2_ENDPOINT alias), AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are present. The engine then reads the bucket directly, so no CDN sits between the engine and its own reads. When METEO_DATA_BASE is set, it always wins.

Inputs

VariableMeaning
METEO_SITESDefault path to the site catalogue when --sites is not passed. A dispatch with neither fails with a usage error.

Provider host overrides

VariableMeaning
METEO_DATAMART_BASEPoints every ECCC Datamart fetch at another host. The default is https://dd.weather.gc.ca, and the research mirror is https://hpfx.collab.science.gc.ca. Tune the wire has the measured difference.
METEO_RRFS_BASEPoints the RRFS builder at another object-store base without a release. The default is https://noaa-rrfs-pds.s3.amazonaws.com/rrfs_public.

Observation windows

VariableMeaning
METEO_GOES_BACKFILL_HOURSHow far back a GOES observation build re-reads granules into its rolling window. The default is 6. A value that is not a number fails loudly.

Debug switches

VariableMeaning
METEO_TRACEBACKWhen set, CLI errors print their full stack trace to stderr instead of the one-line message.

--max-steps reaches the builders in-process and does not pass through the environment, so there is no variable to export for it.