Go: egadev

e2e-egadev runs the six-stage EGA_DEV pipeline (upload through the proxy, ingest, accession, mapping, release, download via a FEGA export request; no C1, finalize or inbox-cleanup stage) against the live egadev.uio.no environment, and needs real credentials and key paths.

1. Build the binaries

   cd e2e && go build -o build/ ./cmd/...

2. Export credentials, key paths and the real endpoints

   export E2E_TESTS_LSAAI_TOKEN=<token>
   export E2E_TESTS_EGA_DEV_BASE_DIRECTORY=<dir>
   export E2E_TESTS_EGA_DEV_ARCHIVE_PUB_KEYPATH=<path>
   export E2E_TESTS_EGA_DEV_JWT_PRIV_KEYPATH=<path>
   export E2E_TESTS_PROXY_HOST=<host> E2E_TESTS_PROXY_PORT=<port>
   export E2E_TESTS_CEGAMQ_CONN_STR=<amqps-url>
   export E2E_TESTS_CEGAAUTH_USERNAME=<user> E2E_TESTS_CEGAAUTH_PASSWORD=<password>
   export E2E_TESTS_EXPORT_REQUEST_MAX_RETRIES=5
   export E2E_TESTS_EXPORT_REQUEST_INTERVAL_IN_SECONDS=10

   The env.sh defaults point into the compose stack (the proxy at proxy, the CEGA broker at cegamq:5673), so a host run sets the real egadev endpoints itself; env.sh deliberately does not carry them. The CEGA credentials need real values too: both upload PATCH calls send them as HTTP basic auth, and env.sh defaults them to the mock stack’s dummy, so an unset host run sends a credential egadev never issued rather than failing on a missing one.

   The two export-request values drive the outbox poll in the download stage. env.sh sets them to 5 and 10, but only a container run sources that file: the Go config falls back to 0 for both, and a retry count of 0 skips the poll entirely, so the stage reports outbox listing never ran instead of the result of a listing.

3. Run it

   ./build/e2e-egadev

Switching between the two suites

E2E_SUITE is read when dev.sh sources env.sh and re-renders the compose file, so changing it and running ./dev.sh start again rebuilds and recreates the e2e-tests service.

An earlier run’s leftovers do not interfere. Every run namespaces itself: a UUID-named fixture, an accession id of EGAF5 plus ten random digits, a dataset id of EGAD plus eleven. Every assertion matches on those, so a stale file in the inbox or outbox is invisible to the next run rather than fatal to it.

what do I have to configure?

Variable	What it is for
E2E_TESTS_LSAAI_TOKEN	a real LS-AAI access token. It supplies the subject and audience for the visas the runner mints, and is itself sent as the bearer on upload, outbox listing and download
E2E_TESTS_EGA_DEV_BASE_DIRECTORY	where the 10 MiB test fixture and its .enc are created, and where the downloaded file is written (<base>/out/)
E2E_TESTS_EGA_DEV_ARCHIVE_PUB_KEYPATH	the Crypt4GH public key the fixture is encrypted to
E2E_TESTS_EGA_DEV_JWT_PRIV_KEYPATH	the RSA key that signs the visa carried by the export request
E2E_TESTS_EGA_DEV_JWT_PUB_KEYPATH	used by the JUnit runner’s export test; no Go stage reads it. The Go config still carries the field because the JUnit runner reads it, so both retire together with that suite (#851)
E2E_TESTS_CEGAAUTH_USERNAME	CEGA HTTP basic auth, sent on both upload PATCH calls alongside the LS-AAI bearer. env.sh supplies the mock value dummy, so a host run must override it with a value the live proxy accepts
E2E_TESTS_CEGAAUTH_PASSWORD	the password half of that basic auth, with the same dummy default and the same need to override
E2E_TESTS_EXPORT_REQUEST_MAX_RETRIES	how many times the download stage polls the outbox for the exported file before giving up. Defaults to 0 in the Go config, which skips the poll, so a host run must set it. env.sh uses 5
E2E_TESTS_EXPORT_REQUEST_INTERVAL_IN_SECONDS	seconds between those polls, and the initial wait before the first one. Same 0 default and the same need to set it. env.sh uses 10

The key paths are absolute and resolve anywhere on disk; the fega distribution instead resolves bare key filenames under the certs directory. The env.sh EGA_DEV block is left commented for the same reason: per-operator values, like the endpoints and CEGA credentials in step 2.

why does it run on a host, and can I run it in the container instead?

The EGA_DEV pipeline reads no staged certificates at all: its keys come from the absolute paths above, the CEGA broker connection verifies against the system roots (the egadev broker certificate chains to a public CA that the local stack’s mkcert pool cannot verify), the HTTP client validates the live certificate, and the direct Postgres check runs only in the FEGA pipeline. The container form, E2E_SUITE=go E2E_ENV=egadev ./dev.sh start, needs wiring the operator does: the EGA_DEV variables are commented placeholders in the compose template’s environment block, and the e2e-tests service mounts no volume through which host-side key files could reach the container’s filesystem.

why is there more than one selector?

E2E_SUITE picks the runner (go or java) and E2E_ENV picks which Go binary runs, so the pair selects one of the six distributions. The retiring JUnit runner uses its own E2E_TESTS_INTEGRATION selector instead and ignores E2E_ENV. Neither runner selects an environment at runtime: choosing the environment is choosing the program.

does the runner validate my token?

It never verifies the signature: it base64url-decodes the payload, requires non-empty sub and aud claims, and refuses the run at setup otherwise. Any well-formed token clears the runner itself; the live environment validates the token when the runner presents it. That is also why the mocked stack needs no token at all: there, the runner signs its own visa with a key the stack already trusts. The proxy-side rejection of forged tokens has its own logic-level unit tests; an endpoint-level check against a running proxy is tracked in #854.

what should I watch out for?

A misconfigured run fails at startup rather than deep inside a stage. The config is checked right after loading, every missing variable is aggregated into a single error naming them all, ports are range-checked to 1-65535, and the export polling knobs have to be positive.

What is left to watch for comes from the source itself: the export request encodes the recipient key as base64 SPKI DER, and the code marks that encoding as not yet verified against live egadev.