AI Test Generation

Overview

generate_test_suite is an MCP tool that scaffolds a complete test suite — both the YAML manifest and the Python implementation file — from a description. When signal context is available (from a DBC or rig TOML) and an AI key is present, the Python stubs contain real assertions rather than pass placeholders. This is exposed as:

An MCP tool callable from Claude, Copilot, or Cursor
A function in crucihil.mcp.tools.suites for programmatic use

How it works

Context sources

The tool builds context from up to three sources, merged in priority order:

Source	Flag	What it contributes
DBC file	`dbc_path`	Every `MessageName.SignalName` pair — exact names, no hallucination
Rig TOML	`rig_toml_path`	ECU names, power rail names, GPIO availability
Manual items	`context_items`	Fault scenarios, integration flows, sensor feeds, anything the DBC/TOML can’t express

When all three are provided, the AI has a complete picture of the hardware and can generate test functions that reference real signal names.

AI generation

When context is available and an AI key is detected:

A prompt is built: suite name, description, rig name, and all context items
The AI generates async Python test functions using the rig.can.expect/rig.can.send API
Function names are extracted from the AI output and used to populate the YAML manifest

When no AI key is present, a static placeholder stub is written instead.

Output files

The tool writes two files:

<output_dir>/<suite_name>.yaml — YAML v2 manifest with test IDs, tags, and module paths
<output_dir>/<suite_name>.py — Python file with typed async test functions

Using from an AI assistant

With the MCP server connected to Claude Desktop, Copilot, or Cursor:

User: Generate a test suite for the BrakeController that validates its
      CAN signal interface. Use the bench rig TOML and the DBC at
      defs/powertrain.dbc.

Claude: [calls generate_test_suite]

The tool call:

{
  "suite_name": "brake_validation",
  "description": "Validate BrakeController signal interface via CAN",
  "rig_name": "bench_01",
  "output_dir": "tests/suites",
  "rig_toml_path": "rigs/bench.toml",
  "dbc_path": "defs/powertrain.dbc",
  "context_items": [
    "can_dropout on BrakeDemand",
    "power glitch during active braking",
    "ECU startup sequence"
  ]
}

Example generated YAML

# tests/suites/brake_validation.yaml
suite:
  name: brake_validation
  version: "1.0.0"
  description: "Validate BrakeController signal interface via CAN"

hardware:
  required:
    - can0
  optional:
    - eth0

definitions:
  can_dbc: "defs/vehicle_can.dbc"

defaults:
  hw_variants: [bench_01]
  suite_types: [smoke, regression]
  timeout: 10.0
  enabled: true

tests:

  - id: brake_pressure_response
    name: "brake pressure response"
    suite_types: [smoke, regression]
    priority: high
    tags: [todo]
    requirements: []
    module: tests.suites.brake_validation
    function: test_brake_pressure_response

  - id: brake_demand_dropout
    name: "brake demand dropout"
    suite_types: [smoke, regression]
    priority: high
    tags: [todo]
    requirements: []
    module: tests.suites.brake_validation
    function: test_brake_demand_dropout

Example generated Python

"""Test functions for the brake_validation suite.

Generated by CruciHiL MCP — fill in assertions below.
Run with:
  crucihil run --suite tests/suites/brake_validation.yaml \
               --rig rigs/virtual.toml
"""

from __future__ import annotations

import asyncio

from crucihil.hal.rig import Rig


async def test_brake_pressure_response(rig: Rig) -> None:
    """Verify BrakeStatus.Pressure increases within 200ms of BrakeDemand.Value."""
    await rig.can.send(message="BrakeDemand", fields={"Value": 80.0})
    result = await rig.can.expect(
        signal="BrakeStatus.Pressure",
        condition=lambda v: v >= 75.0,
        timeout=0.2,
    )
    assert result.passed, result.fail_msg


async def test_brake_demand_dropout(rig: Rig) -> None:
    """Verify ECU enters safe state when BrakeDemand frames drop out for 2s."""
    async with rig.fault.inject(
        rig.fault.can_dropout(arb_id=0x200, duration=2.0)
    ):
        await asyncio.sleep(2.0)
    result = await rig.can.expect(
        signal="BrakeStatus.Active",
        condition=lambda v: v == 0,
        timeout=1.0,
    )
    assert result.passed, result.fail_msg

Parameters

string

required

Snake-case name for the suite (e.g. brake_validation). Used as the filename and YAML suite.name.

string

required

One-sentence description of what the suite validates. Included in the YAML and the Python file header.

string

default:"Virtual_Sim"

Hardware variant to target. Written to defaults.hw_variants in the YAML.

string

default:"tests/suites"

Directory to write both files. Created if it does not exist.

list[string]

Manual context items merged on top of auto-extracted DBC/TOML context. Can contain anything: signal names, fault scenarios, integration flows, sensor feeds, or plain descriptions. Examples: ["can_dropout on EngineData", "ECU startup sequence", "verify camera latency under 50ms"].

string

Path to a rig TOML. Auto-extracts ECU names, power rail names, and GPIO availability. Interface names and IPs are never extracted (hardware details stay in TOML).

string

Path to a DBC file. Auto-extracts every MessageName.SignalName pair with exact DBC names — no hallucination.

string

AI provider override: anthropic, openai, or gemini. Auto-detected from env vars if not set.

Return value

{
  "suite_name": "brake_validation",
  "yaml_path": "tests/suites/brake_validation.yaml",
  "py_path": "tests/suites/brake_validation.py",
  "module_path": "tests.suites.brake_validation",
  "yaml_content": "...",
  "py_content": "...",
  "auto_context_count": 26,
  "context_items_used": ["BrakeDemand.Value", "BrakeStatus.Pressure", ...],
  "next_steps": [
    "Edit tests/suites/brake_validation.py — replace stubs with real assertions.",
    "Run against virtual rig: crucihil run --suite ... --rig rigs/virtual.toml",
    "Add more test IDs to the YAML and matching functions to the Python file."
  ]
}

After generation

The generated files work immediately with crucihil run:

crucihil run \
  --suite tests/suites/brake_validation.yaml \
  --rig rigs/virtual.toml

Edit the Python file to sharpen assertions, add signal tolerance checks, or wire up fault injection scenarios. Then run against real hardware by swapping the rig TOML — the Python file stays the same.

AI Features

AI Test Generation

Overview

How it works

Context sources

AI generation

Output files

Using from an AI assistant

Example generated YAML

Example generated Python

Parameters

Return value

After generation

See also

​Overview

​How it works

​Context sources

​AI generation

​Output files

​Using from an AI assistant

​Example generated YAML

​Example generated Python

​Parameters

​Return value

​After generation

​See also

Overview

How it works

Context sources

AI generation

Output files

Using from an AI assistant

Example generated YAML

Example generated Python

Parameters

Return value

After generation

See also