Testing suite

Continuous Integration (CI)

The entire test suite is run with GitHub Actions using the latest stable and nightly Rust. On nightly Rust (with the nightly feature both enabled and not), all tests (except for doctests) are run with AddressSanitizer(with ODR violation detection disabled) and LeakSanitizer.

For LeakSanitizer to be more effective, the opt-level for tests is set to 1 in Cargo.toml via. [profile.dev]. MemorySanitizer and ThreadSanitizer are not included when testing. See rust-san for more information.

Orion is also tested with targets:

• thumbv7em-none-eabihf (to ensure building as no_std works)
• i686-unknown-linux-gnu (32-bit)
• x86_64-apple-darwin (OSX)
• mips64-unknown-linux-gnuabi64 (Big-endian)
• armv7-unknown-linux-gnueabihf
• x86_64-pc-windows-msvc (Windows, Visual Studio 2019)
• wasm32-unknown-unknown (WebAssembly)

On stable Rust, cargo-audit is used to check for dependencies with reported vulnerabilities.

Automated code coverage is provided by tarpaulin and Codecov.

Official test vectors

• HMAC: RFC 4231. NIST CAVP test vectors. Wycheproof test vectors.
• ChaCha20: RFC 8439.
• Poly1305: RFC 8439.
• ChaCha20Poly1305: RFC 8439.
• BLAKE2b: KAT test vectors. OpenSSL test vectors.
• SHA512: NIST CAVP test vectors.

Custom test vectors

• PBKDF2: These test vectors have been generated with the cryptography.io Python package. More information here.
• HKDF: Test vectors from here. Other test vectors from here. Wycheproof test vectors.

Other test vectors

• Poly1305: Monocypher test vectors. BoringSSL test vectors.
• ChaCha20: BoringSSL test vectors.
• ChaCha20Poly1305: BoringSSL test vectors. Wycheproof test vectors.
• HChaCha20: Monocypher generated test vectors.
• XChaCha20: draft RFC test vectors.
• XChaCha20Poly1305: sodiumoxide test vectors. BoringSSL test vectors. Wycheproof test vectors.
• Argon2i: Monocypher generated test vectors. PyNaCl test vectors. Reference implementation generated test vectors.

Fuzzing

Orion is fuzzed using honggfuzz-rs in orion-fuzz.

Fuzzing is done both to detect unwanted behavior but also verify results. Below is a list of other libraries that are used to cross-check the results that Orion produces:

• HMAC: ring.
• HKDF: ring.
• PBKDF2: ring.
• Poly1305: sodiumoxide (binding to libsodium).
• ChaCha20: chacha.
• ChaCha20Poly1305: sodiumoxide (binding to libsodium).
• HChaCha20: None currently.
• XChaCha20: chacha.
• XChaCha20Poly1305: sodiumoxide (binding to libsodium).
• BLAKE2b: blake2-rfc.
• SHA512: ring.
• Argon2i: rust-argon2.

Testing for timing-based side-channel vulnerabilities

Orion is tested in CI, to ensure that relevant parts execute in constant-time. These tests use dudect and can be found in orion-dudect. Orion is also tested for constant-time execution with WASM at orion-sidefuzz.