📄 03-data-handling.md
⬇ Download Raw

Data Handling, File I/O, Privacy & Encryption Audit

Audit Date: 2026-04-28 Codebase: scorpiox/clang Scope: 222 source files (142,256 LOC excluding vendor code), plus vendored mbedTLS and yyjson Auditor: Security Audit Agent

1. Data Flow Summary

The scorpiox codebase is a multi-component C application comprising:

ComponentPurposeData Sensitivity
sx (main CLI)AI coding assistant TUIHIGH — handles API keys, user conversations, system prompts
scorpiox-serverHTTP server executing scriptsHIGH — JWT secrets, POST bodies, auth headers
scorpiox-server-emailSMTP/IMAP email serverCRITICAL — email content, user passwords, TLS keys, DKIM keys
scorpiox-server-fetchtokenTCP→HTTP token proxyHIGH — API keys, auth tokens
scorpiox-server-http2tcpHTTP relayMEDIUM — relay keys, proxied traffic
scorpiox-sshpassSSH password wrapperCRITICAL — plaintext passwords passed via CLI -p flag
scorpiox-otpTOTP generatorHIGH — TOTP secrets passed via CLI -s flag
scorpiox-beamFile transferLOW — file data (no encryption)
scorpiox-whatsappWhatsApp bridgeHIGH — WhatsApp auth credentials, message content
scorpiox-tmuxTerminal multiplexer launcherMEDIUM — env vars, container configs
libsxutilShared utilitiesHIGH — config cascade, logging, session management
libsxnetNetwork/API layerHIGH — API keys, FRP crypto, HTTP requests/responses
Data flow path:
User Input → sx TUI → libsxnet (API provider) → HTTP/TLS → Remote API

.scorpiox/sessions/<id>/ (conversations, traffic, logs, config snapshots)

.scorpiox/conversations/ (persistent conversation JSON)

.scorpiox/traces/ (debug trace JSONL)


2. File I/O Inventory

2.1 Configuration Files (Read)

FileLocationContains Sensitive DataNotes
scorpiox-env.txt/YES — API keys, passwords, secretsGlobal config
scorpiox-env.txt.scorpiox/ (CWD)YES — project-level overridesHigher priority cascade
scorpiox-env.txt~/.config/scorpiox/YES — user-level configUser override

Config cascade: Built-in defaults → exe_dir config → CWD project config

2.2 Session Data (Read/Write)

File PatternLocationContentSensitivity
conversation.json.scorpiox/sessions//Full conversation historyHIGH — may contain user code, secrets
session.log.scorpiox/sessions//Session log with timestampsMEDIUM — debug data, API calls
agent.log.scorpiox/sessions//Agent-level logMEDIUM
events.jsonl.scorpiox/sessions//Event streamMEDIUM
trace.jsonl.scorpiox/sessions//Data flow tracesHIGH — API request/response data
config-snapshot.txt.scorpiox/sessions//Resolved config at session startMEDIUM — sensitive keys are redacted (*)
traffic/.scorpiox/sessions//Raw HTTP request/response bodiesCRITICAL — full API payloads
messages/.scorpiox/sessions//Per-message filesHIGH
meta.json.scorpiox/sessions//Session metadata (timestamps)LOW
required_skills.txt.scorpiox/sessions//Skill listLOW*

2.3 Conversation Persistence (Read/Write)

File PatternLocationContentSensitivity
.json.scorpiox/conversations/Complete conversation JSONHIGH
latest.scorpiox/conversations/Symlink to latest conversationLOW

2.4 Email Server Files

FilePurposeSensitivity
TLS cert/keyEMAIL_TLS_CERT, EMAIL_TLS_KEY configCRITICAL — TLS private key
DKIM keyEMAIL_DKIM_KEY configCRITICAL — DKIM RSA private key
Accounts fileEMAIL_ACCOUNTS_FILE configCRITICAL — user:hash pairs
MaildirEMAIL_MAILDIR configCRITICAL — email content
Aliases fileEMAIL_ALIASES_FILE configLOW
Mail queue.scorpiox/mail-queue/HIGH — queued email content

2.5 HTTP Server Temporary Files

File PatternLocationCreated ByCleaned Up
sx_post__/tmp/ (Linux) or %TEMP% (Windows)scorpiox-serverYESunlink() after script execution

2.6 Socket/PID Files (sxmux)

File PatternLocationPurpose
.sockSXMUX_SOCKET_DIRUnix domain socket
.pidSocket dirPID file for stale detection

Socket directory created with mkdir(..., 0700) — ✅ Correct permissions.

2.7 WhatsApp Auth Data

File PatternLocationSensitivity
creds.json.scorpiox/whatsapp/auth/CRITICAL — WhatsApp session credentials
Message filesWA_INBOX_DIRHIGH — message content

3. PII Assessment

3.1 Directly Handled PII

PII TypeWhereContext
Email addressesscorpiox-server-emailSMTP FROM/TO, JWT user_email claim
User identifiersscorpiox-server JWTuser_id, user_email in JWT payload
Conversation contentsx main, session filesUser prompts may contain any data
WhatsApp messagesscorpiox-whatsappMessage body, sender info

3.2 Indirect PII Exposure

3.3 PII Mitigation


4. Credential Management

4.1 Built-in Default Credentials

FINDING — MEDIUM RISK: The config defaults table in sx_config.c contains:
KeyDefault ValueRisk
OPENAI_API_KEY"xxx"Low — placeholder, not a real key
ANTHROPIC_ANTIGRAVITY_URLhttp://192.168.1.12:8045/v1/messagesMEDIUM — hardcoded internal IP leaks network topology
ANTHROPIC_ZAI_URLhttps://api.z.ai/api/anthropic/v1/messagesLow — public URL
TCP_HOSTproxy.scorpiox.netLow — public hostname
All _SSH_PASS, _SSH_USER, _SSH_HOST"" (empty)✅ Correctly defaulted to empty
All _API_KEY (except OpenAI)"" (empty)✅ Correctly defaulted to empty

4.2 Config File Credential Storage

Credentials are stored in plaintext scorpiox-env.txt files as KEY=VALUE pairs. Sensitive keys include:

FINDING — HIGH RISK: All credentials stored as plaintext in config files. No encryption, no OS keychain integration, no vault support.

4.3 Credential Redaction

GOOD: sx_config_is_sensitive() correctly identifies sensitive keys by checking for substrings: KEY, PASS, SECRET, TOKEN, CREDENTIAL (case-insensitive).

GOOD: Config snapshot in session files redacts sensitive values to **.

GOOD: Startup logging masks keys containing "KEY" as *.

⚠️ GAP: Logging only masks keys matching strstr(keys[k], "KEY") — this misses _PASS and _SECRET keys in the startup log (though snapshot_write_entry uses the comprehensive sx_config_is_sensitive()).

4.4 Runtime Credential Handling

MechanismImplementationRisk
JWT validationHMAC-SHA256 with constant-time compare✅ Secure
JWT secretLoaded from SERVER_JWT_SECRET configPlaintext in config
API keys → HTTP headersPassed as Authorization/x-api-key headers✅ Normal pattern
FRP tokenPBKDF2-derived AES key✅ Good KDF usage
SSH passwords (scorpiox-sshpass)Passed via -p CLI argumentCRITICAL — visible in ps output
TOTP secrets (scorpiox-otp)Passed via -s CLI argumentHIGH — visible in ps output
SMTP auth (email server)SHA-256 hashed passwords in accounts file⚠️ No salt, no KDF
Authorization headerPassed through to CGI scripts via HTTP_AUTHORIZATION env var⚠️ Exposed to child processes

4.5 Token Proxy Security

scorpiox-server-fetchtoken:

5. Encryption Usage

5.1 In-Transit Encryption

ComponentProtocolLibraryVerification
sxmail_tls.cTLS 1.2 (server)mbedTLS✅ Proper certificate loading, entropy seeding
sx_frp.cAES-128-CFB stream ciphermbedTLS✅ PBKDF2-SHA1 key derivation
sx_http.c (via providers)HTTPSmbedTLS TLS clientConfig: CLAUDE_CODE_PROXY_STRICT controls cert verification
scorpiox-beamNONEN/ARISK — file transfer over raw TCP with only xxHash64 integrity

5.2 Crypto Primitives Used

AlgorithmLocationPurposeAssessment
AES-128-CFBsx_frp.cFRP control stream encryption✅ Appropriate for stream cipher
PBKDF2-SHA1sx_frp.cFRP key derivation⚠️ Only 64 iterations — very low; SHA-1 is deprecated for new designs
SHA-256scorpiox-server.cJWT HMAC-SHA256 signature✅ Industry standard
SHA-256sxmail_auth.cPassword hashing⚠️ No salt, no KDF — vulnerable to rainbow tables
SHA-1scorpiox-otp.cHMAC-SHA1 for TOTP (RFC 6238)✅ Required by TOTP spec
MD5sx_frp.cFRP login hash⚠️ MD5 is cryptographically broken
RSAsxmail_dkim.cDKIM signing✅ Via mbedTLS
TLS 1.2sxmail_tls.c, sx_frp.cTransport security✅ mbedTLS implementation

5.3 Encryption at Rest

FINDING — HIGH RISK: No encryption at rest is used anywhere in the codebase. All persistent data (conversations, sessions, config files, email maildir, WhatsApp credentials) is stored in plaintext on the filesystem.

5.4 mbedTLS Configuration

sx_mbedtls_config.h enables: Note: TLS 1.3 is NOT enabled. Only TLS 1.2 is configured.

6. Temporary File Security

6.1 POST Body Temp Files (scorpiox-server)

// Linux path construction:

snprintf(tmppath, sizeof(tmppath), "/tmp/sx_post_%d_%ld.txt",

sx_getpid(), (long long)time(NULL));

FINDINGS:

6.2 Other Temporary Usage


7. Logging Analysis

7.1 Log Architecture

7.2 Sensitive Data in Logs

Log EntryRiskMitigation
Config values at startupKeys with "KEY" substring masked as ⚠️ Incomplete — _PASS, _SECRET not masked in startup log
SMTP auth success/failureLogs username only✅ Password not logged
JWT validation failureLogs "signature verification failed"✅ No secret/token logged
FRP tokenNot logged✅ Good
API request/response bodiesWritten to traffic/ dir (separate from logs)⚠️ Contains full API payloads
DKIM key pathLogged at INFO level✅ Path only, not key content
fetchtoken API keyLogged as ✅ Good

7.3 Trace System

The trace system (sx_trace.c) writes detailed data flow checkpoints to .scorpiox/traces/.jsonl and/or .scorpiox/sessions//trace.jsonl:

FINDING — HIGH RISK: Trace data may contain API keys in HTTP headers and full conversation content. Trace files have no special access controls.

7.4 Log File Locations

LogPathCreated By
Session log.scorpiox/sessions//session.logsx main
Agent log.scorpiox/sessions//agent.logsx main
Fallback log.scorpiox/logs/sx-session.logsx main
Email server log~/.scorpiox/server-email.logscorpiox-server-email
Trace.scorpiox/sessions//trace.jsonlsx_trace.c

8. Environment Variable Usage

8.1 Environment Variables Read

VariableRead BySensitivity
HOME / USERPROFILEMultipleLow — path discovery
SHELLsxmux_pane.cLow — default shell
EDITOR / VISUALsx.cLow — editor selection
ENABLE_TERM_BACKGROUNDsx_term.cLow — UI config
SXMUX_INIT_CMDsxmux_pane.cLow — pane init command

8.2 Environment Variables Set (for child processes)

VariableSet ByContainsRisk
HTTP_AUTHORIZATIONscorpiox-server.cAuth header valueHIGH — full Authorization header passed to CGI scripts
POST_BODY_FILEscorpiox-server.cTemp file pathLow — path only
Various CGI varsscorpiox-server.cQuery params, headersMedium — depends on content

8.3 Config System vs Environment

The codebase uses its own sx_config_get() system rather than raw getenv() for most configuration. The config cascade (defaults → file → file) avoids polluting the process environment with secrets. This is a good pattern.

Exception: scorpiox-sshpass passes password via write() to PTY (not env var), which is better than SSHPASS env var but the password is still visible in /proc//cmdline via the -p argument.

9. Data Retention

9.1 Session Retention

9.2 Conversation Retention

9.3 Email Retention

9.4 Trace/Log Retention


10. Risk Assessment

Critical Risks

IDFindingLocationImpact
C-1SSH passwords visible in process listingscorpiox-sshpass -p flagPassword exposure to any user who can run ps
C-2Email password hashing uses unsalted SHA-256sxmail_auth.cRainbow table attacks on stolen accounts file
C-3No encryption at rest for any persistent dataAll componentsData breach exposure if filesystem is compromised

High Risks

IDFindingLocationImpact
H-1All credentials stored as plaintext in config filesscorpiox-env.txtCredential theft via file access
H-2TOTP secrets visible in process listingscorpiox-otp -s flagTOTP secret exposure
H-3Session traffic dir contains full API payloads.scorpiox/sessions//traffic/API keys in headers, conversation content
H-4Trace files contain unredacted API data.scorpiox/sessions//trace.jsonlSame as H-3
H-5FRP PBKDF2 uses only 64 iterationssx_frp.cBrute-force key derivation feasible
H-6FRP login uses MD5 hashsx_frp.cMD5 is cryptographically broken
H-7Predictable temp file names (no mkstemp)scorpiox-server.cSymlink / TOCTOU attacks on /tmp/sx_post_*

Medium Risks

IDFindingLocationImpact
M-1Hardcoded internal IP 192.168.1.12 in defaultssx_config.cNetwork topology leak
M-2fetchtoken API key comparison not constant-timescorpiox-server-fetchtoken.cTiming side-channel
M-3Startup log masks only "KEY"-containing config namessx.c_PASS, _SECRET values logged in cleartext
M-4TLS 1.2 only (no TLS 1.3)sx_mbedtls_config.hMissing modern TLS features
M-5scorpiox-beam file transfer has no encryptionscorpiox-beam.cData in transit is unprotected
M-6Conversations dir not in .gitignore auto-managementsx_conversation.cAccidental commit of conversation data
M-7Authorization header passed to CGI via env varscorpiox-server.cCredential leakage to untrusted scripts
M-8No conversation retention/cleanup mechanismsx_conversation.cData hoarding risk

Positive Findings

IDFindingLocation
P-1Config snapshot correctly redacts sensitive keys via sx_config_is_sensitive()sx_session.c, sx_config.c
P-2Session data auto-added to .gitignoresx_session.c
P-3JWT uses constant-time signature comparisonscorpiox-server.c
P-4Socket directory created with 0700 permissionssxmux_session.c
P-5POST body temp files cleaned up after usescorpiox-server.c
P-6No hardcoded real API keys or passwords in sourcesx_config.c
P-7Config system avoids env var pollution (uses own cascade)sx_config.c
P-8FRP uses proper AES-128-CFB with random IVsx_frp.c
P-9TLS implementation via well-maintained mbedTLS librarysxmail_tls.c

11. Recommendations

  • Replace unsalted SHA-256 password hashing with Argon2id or bcrypt (C-2)
  • Use mkstemp() for temp files instead of predictable name construction (H-7)
  • Increase PBKDF2 iterations from 64 to at least 100,000 (H-5)
  • Add encryption at rest for session/conversation data or at minimum restrict file permissions to 0600 (C-3)
  • Use environment variable or stdin for scorpiox-sshpass password instead of CLI -p (C-1)
  • Add constant-time comparison for fetchtoken API key validation (M-2)
  • Extend startup log masking to cover _PASS and _SECRET patterns (M-3)
  • Enable TLS 1.3 in mbedTLS configuration (M-4)
  • Add encryption to scorpiox-beam file transfers (M-5)
  • Add conversation cleanup mechanism similar to session retention (M-8)
  • Remove hardcoded internal IP from config defaults (M-1)
  • Consider redacting Authorization headers in traffic/trace files (H-3, H-4)