Last active
March 2, 2026 18:07
-
-
Save satishbabariya/365e4d4c09b91523b18ac9a36fc53426 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ | |
| │ Plan to implement │ | |
| │ │ | |
| │ Roadmap Assessment: Where We Are & What's Next │ | |
| │ │ | |
| │ Context │ | |
| │ │ | |
| │ The user asked for a roadmap status check — where the cmod project stands relative to its implementation phases, and │ | |
| │ what the next phase of work should be. This assessment is based on a thorough audit of all 7 crates, 28 CLI │ | |
| │ commands, 559 tests, 21 RFCs, and the 5 example projects. │ | |
| │ │ | |
| │ --- │ | |
| │ Current Status: Phases 0-2 Structurally Complete, Phase 1 Has Critical Gaps │ | |
| │ │ | |
| │ Phases 0-2 are structurally complete — all scaffolding exists for dependency resolution, builds, workspaces, and │ | |
| │ caching. All 28 CLI commands have real working logic (no stubs). 559 tests pass. However, the build system (Phase 1) │ | |
| │ has critical design flaws that prevent cmod from building most real-world C++ module projects. │ | |
| │ │ | |
| │ What's Working │ | |
| │ │ | |
| │ ┌──────────────────────────────────────────┬───────────────────────────┬──────────────────────────────────────────┐ │ | |
| │ │ Area │ Crate │ Status │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ Manifest parsing & config │ cmod-core (71 tests) │ Done │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ Git dependency resolver + semver │ cmod-resolver (47 tests) │ Done │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ Build orchestration (clang++ invocation) │ cmod-build (73 tests) │ Partial — critical gaps below │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ Local artifact cache (SHA-256) │ cmod-cache (40 tests) │ Done │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ Workspace management │ cmod-workspace (11 tests) │ Done │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ Security foundations (TOFU trust) │ cmod-security (34 tests) │ Partial — signatures stubbed │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ CLI (28 commands) │ cmod-cli (283 tests) │ Done │ │ | |
| │ ├──────────────────────────────────────────┼───────────────────────────┼──────────────────────────────────────────┤ │ | |
| │ │ Example projects │ examples/ │ Done — 3 of 5 compile with Homebrew LLVM │ │ | |
| │ └──────────────────────────────────────────┴───────────────────────────┴──────────────────────────────────────────┘ │ | |
| │ │ | |
| │ --- │ | |
| │ Critical Gaps in Phase 1 Build System │ | |
| │ │ | |
| │ 1. Module Graph Name Collision — CRITICAL │ | |
| │ │ | |
| │ crates/cmod-build/src/graph.rs:28 │ | |
| │ │ | |
| │ ModuleGraph uses BTreeMap<String, ModuleNode> keyed by module name only. A module with both interface (lib.cppm) and │ | |
| │ implementation (lib.cpp) units collide — the second overwrites the first. Blocks ~80% of real C++ projects. │ | |
| │ │ | |
| │ Fix: Key nodes by unique ID (source path). Support multiple translation units per logical module. │ | |
| │ │ | |
| │ 2. Module Partitions Not Fully Supported — CRITICAL │ | |
| │ │ | |
| │ graph.rs, plan.rs, compiler.rs │ | |
| │ │ | |
| │ PartitionUnit enum variant exists but partitions are treated identically to regular interfaces. No │ | |
| │ partition-to-module ownership tracking, no internal vs exported partition distinction, no partition-specific PCM │ | |
| │ passing via -fmodule-file=. │ | |
| │ │ | |
| │ 3. Source Classification Fragile — HIGH │ | |
| │ │ | |
| │ crates/cmod-build/src/runner.rs:852 │ | |
| │ │ | |
| │ Regex-based on first ~50 lines. Misses declarations after long comments, global module fragments, or preprocessor │ | |
| │ blocks. Should prefer clang-scan-deps P1689 output. │ | |
| │ │ | |
| │ 4. Workspace Cross-Member PCM Propagation — HIGH │ | |
| │ │ | |
| │ crates/cmod-cli/src/commands/build.rs:244 │ | |
| │ │ | |
| │ Upstream member PCMs collected by filename convention. No transitive dependency tracking — A→B→C may miss C's PCMs │ | |
| │ for A. │ | |
| │ │ | |
| │ 5. Lockfile Content Hashes Not Computed — MEDIUM │ | |
| │ │ | |
| │ crates/cmod-resolver/src/resolver.rs │ | |
| │ │ | |
| │ ResolvedDep.hash field exists but tree hashing is not implemented. Makes --verify unable to detect tampering. │ | |
| │ │ | |
| │ 6. No End-to-End C++ Compilation Tests — MEDIUM │ | |
| │ │ | |
| │ All 559 tests validate Rust logic only. None invoke clang++ to compile C++ code. │ | |
| │ │ | |
| │ --- │ | |
| │ Phase Map │ | |
| │ │ | |
| │ Phase 0 — Foundations (MVP) ✅ COMPLETE │ | |
| │ Phase 1 — Module-Aware Builds ⚠️ STRUCTURAL but CRITICAL GAPS (above) │ | |
| │ Phase 2 — Workspaces & Caching ✅ COMPLETE │ | |
| │ Phase 3 — Distributed Builds 🔲 NOT STARTED │ | |
| │ Phase 4 — Security & Verification 🔲 FOUNDATION LAID (trust store, verify stubs) │ | |
| │ Phase 5 — Tooling & Ecosystem 🔲 NOT STARTED │ | |
| │ │ | |
| │ --- │ | |
| │ Recommended Next Phase: "Phase 1.5 — Build System Hardening" │ | |
| │ │ | |
| │ Before Phase 3 (distributed builds), the build system must work correctly for real C++ projects. Advancing to │ | |
| │ distributed caching without a reliable local build is putting the cart before the horse. │ | |
| │ │ | |
| │ Priority 1: Redesign Module Graph │ | |
| │ │ | |
| │ - Change BTreeMap<String, ModuleNode> to support multiple TUs per module │ | |
| │ - Key by source path, group by logical module name │ | |
| │ - Add edges: partition → owning module, implementation → interface │ | |
| │ - Update BuildPlan::from_graph() for multi-unit modules │ | |
| │ - Files: graph.rs, plan.rs, runner.rs │ | |
| │ │ | |
| │ Priority 2: Full Partition Support │ | |
| │ │ | |
| │ - Track partition ownership (:ops belongs to local.math) │ | |
| │ - Generate correct -fmodule-file=local.math:ops=ops.pcm flags │ | |
| │ - Validate partition declarations match owning module │ | |
| │ - Files: graph.rs, plan.rs, compiler.rs │ | |
| │ │ | |
| │ Priority 3: Robust Source Classification │ | |
| │ │ | |
| │ - Use clang-scan-deps P1689 as primary (already partially implemented) │ | |
| │ - Improve regex fallback for global module fragment pattern │ | |
| │ - Remove 50-line scan limit │ | |
| │ - Files: runner.rs, compiler.rs │ | |
| │ │ | |
| │ Priority 4: Workspace PCM Propagation │ | |
| │ │ | |
| │ - Build workspace-level dependency graph from member [dependencies] │ | |
| │ - Propagate PCMs transitively based on resolved deps │ | |
| │ - Move logic from CLI command into cmod-build or cmod-workspace crate │ | |
| │ - Files: workspace.rs, commands/build.rs │ | |
| │ │ | |
| │ Priority 5: End-to-End Build Tests │ | |
| │ │ | |
| │ - Integration tests that compile example projects with real clang++ │ | |
| │ - Skip when clang++ unavailable │ | |
| │ - Optionally add LLVM to GitHub Actions CI matrix │ | |
| │ - Files: tests/real_projects.rs │ | |
| │ │ | |
| │ Priority 6: Lockfile Content Hashes │ | |
| │ │ | |
| │ - Compute SHA-256 tree hash during cmod resolve │ | |
| │ - Store in lockfile, validate in cmod verify │ | |
| │ - Files: resolver.rs, verify.rs │ | |
| │ │ | |
| │ --- │ | |
| │ After Phase 1.5 │ | |
| │ │ | |
| │ ┌──────────────────────────┬──────────────────────────────────────────────────────────┬──────────────────────────┐ │ | |
| │ │ Phase │ Focus │ Key RFCs │ │ | |
| │ ├──────────────────────────┼──────────────────────────────────────────────────────────┼──────────────────────────┤ │ | |
| │ │ Phase 3 — Distributed │ Remote cache protocol, artifact upload/download, CI │ RFC-0005, RFC-0011, │ │ | |
| │ │ Builds │ workflows │ RFC-0013 │ │ | |
| │ ├──────────────────────────┼──────────────────────────────────────────────────────────┼──────────────────────────┤ │ | |
| │ │ Phase 4 — Security │ Full GPG/SSH signature verification, --locked --verify │ RFC-0009 │ │ | |
| │ │ │ enforcement │ │ │ | |
| │ ├──────────────────────────┼──────────────────────────────────────────────────────────┼──────────────────────────┤ │ | |
| │ │ Phase 5 — Ecosystem │ LSP server, plugin SDK, CMake interop │ RFC-0010, RFC-0016, │ │ | |
| │ │ │ │ RFC-0018 │ │ | |
| │ └──────────────────────────┴──────────────────────────────────────────────────────────┴──────────────────────────┘ │ | |
| │ │ | |
| │ --- │ | |
| │ Verification │ | |
| │ │ | |
| │ After Phase 1.5, all 5 example projects should compile and run with Homebrew LLVM: │ | |
| │ - examples/hello — binary with module interface + implementation unit (currently works but only because impl is │ | |
| │ inlined) │ | |
| │ - examples/library — static lib with partitions :ops and :stats │ | |
| │ - examples/with-deps — git dependencies (fmt + json) │ | |
| │ - examples/workspace — monorepo with 3 members and cross-member deps │ | |
| │ - examples/path-deps — local path dependencies with partitions │ | |
| ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ | |
| ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ | |
| │ Plan to implement │ | |
| │ │ | |
| │ Next Steps: Phase 2.5 — Production Hardening & Phase 3 Foundation │ | |
| │ │ | |
| │ Context │ | |
| │ │ | |
| │ Phase 1.5 (Build System Hardening) is fully complete. All 6 priorities were implemented: │ | |
| │ - Module graph redesigned for multi-TU support │ | |
| │ - Full partition support with ownership tracking │ | |
| │ - Robust source classification (no scan limit, handles comments/preprocessor) │ | |
| │ - Workspace PCM propagation with transitive dependency resolution │ | |
| │ - End-to-end C++ build tests (9 tests compiling real example projects) │ | |
| │ - Lockfile content hash computation and verification │ | |
| │ - Path dependency building and PCM propagation added as a bonus fix │ | |
| │ │ | |
| │ Current state: 291 lib tests + 9 E2E tests passing. All 5 example projects compile. Zero clippy warnings. The build │ | |
| │ system works for real C++ module projects. │ | |
| │ │ | |
| │ What's next: Before jumping to Phase 3 (distributed builds), there's a practical gap — the remote cache uses curl │ | |
| │ subprocesses, signature verification only checks presence (not validity), and several CLI commands have thin │ | |
| │ implementations. A focused "Phase 2.5" hardens what exists, then we lay the Phase 3 foundation. │ | |
| │ │ | |
| │ --- │ | |
| │ Phase 2.5: Production Hardening (8 priorities) │ | |
| │ │ | |
| │ Priority 1: Replace curl-based Remote Cache with Native HTTP │ | |
| │ │ | |
| │ Why: remote.rs shells out to curl for every cache operation. No auth, no retries, no timeouts. This is the #1 │ | |
| │ blocker for real team usage. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Add ureq (or reqwest with blocking) to cmod-cache/Cargo.toml │ | |
| │ - Rewrite HttpRemoteCache in crates/cmod-cache/src/remote.rs to use native HTTP │ | |
| │ - Add Authorization: Bearer <token> support via RemoteCacheConfig.auth_token │ | |
| │ - Add retry with exponential backoff (3 attempts) │ | |
| │ - Add configurable timeout (default 30s) │ | |
| │ - Add [cache] manifest fields: auth_token, timeout, retries │ | |
| │ │ | |
| │ Files: crates/cmod-cache/src/remote.rs, crates/cmod-cache/Cargo.toml, crates/cmod-core/src/manifest.rs │ | |
| │ │ | |
| │ Priority 2: Real GPG/SSH Signature Verification │ | |
| │ │ | |
| │ Why: check_commit_signature() in verify.rs:76 only checks if a PGP/SSH block exists — it doesn't validate it. This │ | |
| │ makes cmod verify --signatures a false sense of security. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Call gpg --verify for PGP signatures (parse exit code + stderr) │ | |
| │ - Call ssh-keygen -Y verify for SSH signatures │ | |
| │ - Map results to SignatureStatus::Valid/Untrusted/Invalid with real reason strings │ | |
| │ - Add SignatureStatus::Unsigned when no signature found (unchanged) │ | |
| │ - Gracefully degrade if gpg/ssh-keygen not available (warn, return Untrusted) │ | |
| │ │ | |
| │ Files: crates/cmod-security/src/verify.rs │ | |
| │ │ | |
| │ Priority 3: Security Policy Enforcement (policy.rs) │ | |
| │ │ | |
| │ Why: The security crate's policy.rs is referenced in the architecture but doesn't exist yet. The manifest has a │ | |
| │ [security] section with require_signatures and trusted_publishers but enforcement is minimal. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Create crates/cmod-security/src/policy.rs │ | |
| │ - SecurityPolicy struct: require_signatures: bool, allowed_sources: Vec<String>, min_trust_level: TrustLevel │ | |
| │ - enforce() method that validates lockfile packages against policy │ | |
| │ - Integrate with cmod build --verify and cmod verify │ | |
| │ │ | |
| │ Files: crates/cmod-security/src/policy.rs, crates/cmod-security/src/lib.rs, crates/cmod-cli/src/commands/verify.rs │ | |
| │ │ | |
| │ Priority 4: Improve cmod add Workflow │ | |
| │ │ | |
| │ Why: add.rs (86 LOC) doesn't validate the dep exists, doesn't check for conflicts, and doesn't auto-resolve. Users │ | |
| │ have to manually run cmod resolve after every cmod add. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Check if dependency already exists (warn if version differs) │ | |
| │ - Validate Git URL is reachable (unless --offline) │ | |
| │ - Auto-run cmod resolve after adding (unless --no-resolve) │ | |
| │ - Support --features flag for optional feature selection │ | |
| │ │ | |
| │ Files: crates/cmod-cli/src/commands/add.rs │ | |
| │ │ | |
| │ Priority 5: Integrate cmod tidy with Module Graph │ | |
| │ │ | |
| │ Why: tidy.rs uses naive text scanning for import statements. It doesn't use the module graph we just redesigned in │ | |
| │ Phase 1.5. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Build actual ModuleGraph from sources (reuse build_module_graph() from build.rs) │ | |
| │ - Check which modules are actually imported in the graph │ | |
| │ - Cross-reference with declared [dependencies] in manifest │ | |
| │ - Report unused deps with their source (Git URL, version) │ | |
| │ │ | |
| │ Files: crates/cmod-cli/src/commands/tidy.rs │ | |
| │ │ | |
| │ Priority 6: Extend cmod check with Semantic Validation │ | |
| │ │ | |
| │ Why: check.rs (180 LOC) only validates names and structure. It doesn't check if the module graph is valid or if │ | |
| │ sources can be classified. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Build module graph and run graph.validate() │ | |
| │ - Classify all sources and report any classification failures │ | |
| │ - Check partition ownership consistency │ | |
| │ - Validate that all imports can be resolved (within graph or as external deps) │ | |
| │ - Optional: run clang-scan-deps to verify P1689 output matches graph │ | |
| │ │ | |
| │ Files: crates/cmod-cli/src/commands/check.rs │ | |
| │ │ | |
| │ Priority 7: Cache Compression & Metadata │ | |
| │ │ | |
| │ Why: Cache artifacts are stored uncompressed. For remote transfer, this wastes bandwidth. No metadata about what's │ | |
| │ cached. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Add zstd compression for cache artifacts (compress on store, decompress on get) │ | |
| │ - Add cache_metadata.json per entry with: timestamp, size, compiler version, target │ | |
| │ - Add cmod cache status --json for machine-readable output │ | |
| │ - Add cmod cache inspect <key> to show entry metadata │ | |
| │ │ | |
| │ Files: crates/cmod-cache/src/cache.rs, crates/cmod-cli/src/commands/cache.rs, crates/cmod-cache/Cargo.toml │ | |
| │ │ | |
| │ Priority 8: CI Workflow Improvements │ | |
| │ │ | |
| │ Why: No Windows CI. No LLVM integration test step. The E2E tests we just added are never run in CI. │ | |
| │ │ | |
| │ Changes: │ | |
| │ - Add Windows runner to .github/workflows/ci.yml matrix │ | |
| │ - Add LLVM step that runs cargo test --test example_projects on macOS │ | |
| │ - Add cargo test --test real_projects and cargo test --test e2e_validation to CI │ | |
| │ - Pin LLVM version in CI for reproducibility │ | |
| │ │ | |
| │ Files: .github/workflows/ci.yml │ | |
| │ │ | |
| │ --- │ | |
| │ Verification │ | |
| │ │ | |
| │ After Phase 2.5: │ | |
| │ - Remote cache works with a real HTTP server (can test with python -m http.server) │ | |
| │ - cmod verify --signatures actually validates GPG/SSH signatures │ | |
| │ - cmod add <dep> resolves automatically │ | |
| │ - cmod tidy uses the module graph for accurate unused detection │ | |
| │ - cmod check catches graph validation errors │ | |
| │ - CI runs on Windows + LLVM E2E tests on macOS │ | |
| │ - Cache artifacts are compressed │ | |
| │ │ | |
| │ --- │ | |
| │ After Phase 2.5 → Phase 3 │ | |
| │ │ | |
| │ Phase 3 (Distributed Builds) becomes tractable once the remote cache is production-grade: │ | |
| │ │ | |
| │ ┌──────┬────────────────────────────────────────────┬──────────┐ │ | |
| │ │ Step │ Focus │ Key RFC │ │ | |
| │ ├──────┼────────────────────────────────────────────┼──────────┤ │ | |
| │ │ 3a │ Remote cache server reference impl │ RFC-0005 │ │ | |
| │ ├──────┼────────────────────────────────────────────┼──────────┤ │ | |
| │ │ 3b │ Cache push/pull in CI pipelines │ RFC-0005 │ │ | |
| │ ├──────┼────────────────────────────────────────────┼──────────┤ │ | |
| │ │ 3c │ Distributed build execution (worker nodes) │ RFC-0013 │ │ | |
| │ ├──────┼────────────────────────────────────────────┼──────────┤ │ | |
| │ │ 3d │ Precompiled module distribution │ RFC-0011 │ │ | |
| │ └──────┴────────────────────────────────────────────┴──────────┘ │ | |
| ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment