Testing

31 tests across 7 suites — integration tests for all 5 AI providers, end-to-end GNU tools validation, and Thunderbolt 4 benchmarks. Built-in C test runner with interactive TUI.

31 Total Tests
7 Test Suites
5 Providers Tested
3 Test Types

Overview

scorpiox code ships with a comprehensive test suite covering all AI provider integrations, end-to-end GNU tool validation on Windows, and hardware transport benchmarks. Tests are written as Python scripts executed via pytest and driven by scorpiox-runtest, a purpose-built C test runner with an interactive TUI.

The test runner (scorpiox/scorpiox-runtest.c) scans the tests/ directory for test_*.py files, copies them to /tmp, and executes them via pytest. It supports three modes: interactive TUI browsing, pattern-match filtering, and run-all batch mode.

Test Framework

Interactive TUI

Browse discovered tests in a terminal interface. Arrow keys to navigate, Enter to run a single test, or press 'a' to run all.

Run All

Execute every test_*.py in sequence with scorpiox-runtest -a. Ideal for CI pipelines and pre-release verification.

Pattern Match

Filter tests by name pattern: scorpiox-runtest -p claude runs only claude_code provider tests.

CLI Options

scorpiox-runtest # interactive TUI browser scorpiox-runtest -a # run all tests scorpiox-runtest -p gemini # pattern match — run tests matching "gemini" scorpiox-runtest -v # verbose output

CI Integration

For automated pipelines, run tests directly via pytest:

# Run all Python tests pytest tests/ -v # Run only provider hello tests pytest tests/test_*_hello.py -v # Run a specific provider's full suite pytest tests/test_claude_code_*.py -v # GNU tools acceptance (PowerShell on Windows) pwsh tests/test_gnu_tools.ps1 # Thunderbolt 4 loopback (requires two machines or loopback adapter) sudo tb4_test server [iface] # terminal 1 sudo tb4_test client [iface] # terminal 2

Test Suites

Each AI provider has three integration tests: a basic hello response test, a ReadImage tool call test, and a multi-turn conversation with Bash tool execution. All provider tests use scorpiox-sdk --print for non-interactive invocation.

Claude Code
integration 3 tests
Provider integration — hello response, ReadImage tool call, multi-turn Bash + context awareness
tests/test_claude_code_hello.py tests/test_claude_code_image.py tests/test_claude_code_multiturn.py
Codex
integration 3 tests
Provider integration — hello response, ReadImage tool call, multi-turn Bash + context awareness
tests/test_codex_hello.py tests/test_codex_image.py tests/test_codex_multiturn.py
Gemini
integration 3 tests
Provider integration — hello response, ReadImage tool call, multi-turn Bash + context awareness
tests/test_gemini_hello.py tests/test_gemini_image.py tests/test_gemini_multiturn.py
OpenAI
integration 3 tests
Provider integration — hello response, ReadImage tool call, multi-turn Bash + context awareness
tests/test_openai_hello.py tests/test_openai_image.py tests/test_openai_multiturn.py
Scorpiox
integration 3 tests
Provider integration — hello response, ReadImage tool call, multi-turn Bash + context awareness
tests/test_scorpiox_hello.py tests/test_scorpiox_image.py tests/test_scorpiox_multiturn.py
Thunderbolt 4 Transport
benchmark 3 tests
libtb4 loopback tests — small RPC message (15 bytes), 1 MB tensor transfer, 16 MB large tensor transfer with throughput measurement
scorpiox/libtb4/tb4_test.c
GNU Tools Acceptance
e2e 13 tests
GNU coreutils Windows acceptance — grep (--include, --exclude, -P PCRE), find (-printf, -delete), sed -z, diff (-y, --color), tar (--exclude, --strip-components), xargs -d, gawk gensub()
tests/test_gnu_tools.ps1

Test Helpers & Fixtures

Fixtures

Test image fixtures are stored in tests/fixtures/ and used by ReadImage integration tests:

tests/fixtures/test_1x1.png # minimal 1×1 pixel PNG tests/fixtures/test_red.png # solid red test image

The shared helper module tests/helpers.py provides:

FunctionDescription
run_sdk()Wrapper for scorpiox-sdk --print — invokes the SDK in non-interactive mode and captures output
has_event()Assert that a specific event type appears in the SDK output stream
has_tool_call()Assert that a tool call (Bash, ReadImage, etc.) was made during the conversation
has_error()Assert that an error event is present — used for negative testing

The tests/code_generator_models.py file is a build-time utility, not a test. It fetches the Claude model list via scorpiox-claudecode-models and generates the C header sx_models_generated.h.

Running Tests

Run All Tests

scorpiox-runtest -a # or equivalently: pytest tests/ -v

Run a Single Provider

# Claude Code provider tests only pytest tests/test_claude_code_hello.py tests/test_claude_code_image.py tests/test_claude_code_multiturn.py -v # Or use pattern matching scorpiox-runtest -p claude_code

Verbose Mode

scorpiox-runtest -v # shows full pytest output pytest tests/ -v --tb=long # verbose with full tracebacks

Interactive Browser

Launch the TUI test browser with no arguments. Navigate with arrow keys, press Enter to run the highlighted test, or 'a' to run all:

scorpiox-runtest

Coverage

Tested Modules

✓ sx_provider_claude_code
✓ sx_provider_codex
✓ sx_provider_google_gemini
✓ sx_provider_openai
✓ sx_provider_scorpiox
✓ sx_agent (tool calls, multi-turn)
✓ sx_tools (Bash, ReadImage)
✓ libtb4 (tb4_transport)

Untested Modules

The following modules have no dedicated unit tests. They are exercised indirectly through integration tests but lack isolated coverage:

✗ libsxnet/sx_json
✗ libsxnet/sx_http
✗ libsxnet/sx_mcp
✗ libsxnet/sx_api
✗ libsxnet/sx_request
✗ libsxnet/sx_state
✗ libsxnet/sx_frp
✗ libsxutil/sx_config
✗ libsxutil/sx_exec
✗ libsxutil/sx_pty
✗ libsxutil/sx_conversation
✗ libsxutil/sx_metrics
✗ libsxutil/sx_rewind
✗ libsxutil/sx_trace
✗ libsxutil/sx_log
✗ libsxutil/sx_bgtask
✗ libsxutil/sx_html_strip
✗ libsxutil/sx_kql
✗ libsxutil/sx_slashcmd
✗ libsxutil/sx_systemprompt
✗ libsxutil/sx_platform
✗ libsxutil/sx_tcp_fetch
✗ libsxutil/sx_procmon
✗ libsxutil/sx_wsl
✗ libsxui/sxui_display
✗ libsxui/sxui_input
✗ libsxui/sxui_bashterm
✗ libsxui/sxui_fileexplorer
✗ libsxui/sxui_infobox
✗ libsxui/sxui_resume
✗ libsxui/sxui_state
✗ libsxbridge/sx_bridge
✗ bridge/ws2tcp
✗ scorpiox-email/sxmail_*
✗ sxmux_*
✗ scorpiox-mcp
✗ scorpiox-unshare
✗ scorpiox-vm
✗ scorpiox-voice
✗ scorpiox-whatsapp
✗ sx_cache_keepalive