FEATURE: Lexical Boundaries
| Field | Value |
|---|---|
| Version | 20260816 V1 |
| Description | Defines TOML comment, whitespace, line-ending, encoding, and control-character handling. |
| Depends On | ARCHITECTURE.md |
| Provides | TOML lexical boundary validation |
| Consumes | internal/toml |
Scope
Handle spaces and tabs, LF and CRLF line endings, comments outside strings, multiline lexical boundaries, valid UTF-8 documents, and rejection of prohibited control characters in comments and applicable value contexts.
Programmatic Acceptance
=== AC lexical-comments-valid-suite ===
Intent: The authoritative conformance suite passes every valid comment and lexical-boundary case.
import subprocess
result = subprocess.run( ["sh", "sources/stage_test.sh", "valid/comment/**"], capture_output=True, text=True, ) print(result.stdout) print(result.stderr) assert result.returncode == 0 === END AC lexical-comments-valid-suite ===
=== AC lexical-boundaries-invalid-suite ===
Intent: The authoritative conformance suite rejects every invalid control-character and encoding case.
import subprocess
result = subprocess.run( [ "sh", "sources/stage_test.sh", "invalid/control/|invalid/encoding/", ], capture_output=True, text=True, ) print(result.stdout) print(result.stderr) assert result.returncode == 0 === END AC lexical-boundaries-invalid-suite ===
=== AC lexical-boundaries-suites ===
Intent: The valid comment and invalid lexical-boundary conformance slices complete successfully.
import subprocess
patterns = [ "valid/comment/", "invalid/control/|invalid/encoding/**", ] for pattern in patterns: result = subprocess.run( ["sh", "sources/stage_test.sh", pattern], capture_output=True, text=True, ) print(result.stdout) print(result.stderr) assert result.returncode == 0 === END AC lexical-boundaries-suites ===
User Acceptance
- None.
Guardrails
- Comments must not consume content inside strings.
- Only TOML-permitted whitespace and line endings may be accepted.
- Invalid UTF-8 must be rejected.
- Prohibited control characters must be rejected in every applicable lexical context.