Skip to main content

​
Synopsis

crucihil run --suite <path> --rig <path> [options]

​
Description

crucihil run loads a YAML suite manifest and a rig TOML, connects all configured backends, and executes the test functions. Results are printed to stdout. Optionally writes JUnit XML or an HTML report. The rig TOML determines which backends are used — switching from virtual to real hardware is a one-line change in the TOML. The test code and suite YAML are identical in both cases.

​
Options

​
--suite
path
required
Path to the YAML suite manifest (.yaml). The suite file must exist. Short form: -s.
​
--rig
path
required
Path to the rig TOML config (.toml). The file must exist. Short form: -r.
​
--output
path
Write JUnit XML results to this path. Creates or overwrites the file. Short form: -o.
​
--html
path
Write a self-contained HTML test report to this path. The HTML file has no external dependencies and can be opened in any browser.
​
--tags
string
Comma-separated list of tags. Only tests that match at least one tag are executed. Example: --tags smoke,engine.
​
--suite-type
string
Comma-separated suite types. Only tests that match at least one type are executed. Common values: smoke, regression. Example: --suite-type smoke.
​
--verbose
boolean
default:"false"
Show per-test status lines during the run. Also enables debug logging from backends. Short form: -v.

​
Exit codes

CodeMeaning
0All executed tests passed
1One or more tests failed or errored
2Framework error (bad config, missing files, backend setup failure)
blocked tests (where a rig precondition was not met) do not count as failures and do not affect the exit code. Only fail and error statuses trigger exit code 1.

​
Output format

Suite  : engine_validation v1.0.0  (5 tests)
Rig    : rigs/virtual.toml

  ✓ [PASS   ] engine_startup      (0.012s)
  ✓ [PASS   ] can_heartbeat       (0.008s)
  ✗ [FAIL   ] brake_response      (1.203s)
             Expected BrakeStatus.Pressure >= 75.0, got 0.0
  ⊘ [BLOCKED] fault_injection     (0.001s)
             Rig precondition: power_rail 'ecu_main' not available in virtual mode
  – [SKIP   ] vehicle_speed       (0.000s)

4 run · 2 passed · 1 failed · 1 blocked · 0 errored  (1.224s total)

JUnit  : results/run.xml
HTML   : results/report.html

​
Status icons

IconStatusMeaning
✓PASSTest assertion passed
✗FAILTest assertion failed — firmware is not behaving as expected
⊘BLOCKEDRig precondition not met — does not count against pass rate
!ERRORUnexpected exception in test code
–SKIPTest was disabled (enabled: false in YAML)

​
JUnit XML output

When --output is provided, CruciHiL writes a JUnit-compatible XML file. This integrates with:
  • GitHub Actions (actions/upload-artifact + test summary)
  • GitLab CI (junit artifact report type)
  • Jenkins (JUnit Plugin)
  • Any CI system that understands JUnit XML
Example GitHub Actions step: 
- name: Run HiL tests
  run: |
    crucihil run \
      --suite suites/regression.yaml \
      --rig rigs/virtual.toml \
      --output results/junit.xml

- name: Publish test results
  uses: mikepenz/action-junit-report@v4
  if: always()
  with:
    report_paths: results/junit.xml

​
HTML report

The HTML report is a single self-contained file. It includes:
  • Pass/fail summary with counts and duration
  • Per-test expandable rows with error messages
  • Signal trace data (if recorded)
  • Rig config metadata

​
Filtering tests

Run only smoke tests: 
crucihil run --suite suites/regression.yaml --rig rigs/virtual.toml \
  --suite-type smoke
Run only tests tagged engine: 
crucihil run --suite suites/regression.yaml --rig rigs/virtual.toml \
  --tags engine
Combine filters (must match at least one tag AND at least one suite type): 
crucihil run --suite suites/regression.yaml --rig rigs/virtual.toml \
  --tags engine,startup --suite-type smoke

​
Examples

​
Run against virtual backend (no hardware)

crucihil run --suite suites/smoke.yaml --rig rigs/virtual.toml

​
Run with HTML and JUnit output

crucihil run \
  --suite suites/regression.yaml \
  --rig rigs/bench.toml \
  --output results/run.xml \
  --html results/report.html

​
Verbose mode (see per-backend debug logs)

crucihil run --suite suites/smoke.yaml --rig rigs/virtual.toml --verbose

​
See also