From edeb770ff4ad5feab925a47b4ad6ad25c834a6bc Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sat, 29 Aug 2026 20:42:52 +0800 Subject: [PATCH 01/17] =?UTF-8?q?=E5=AE=8C=E6=88=90=E5=8D=8F=E7=A8=8B?= =?UTF-8?q?=E8=BF=90=E8=A1=8C=E6=97=B6=E7=AC=AC=E5=9B=9B=E9=98=B6=E6=AE=B5?= =?UTF-8?q?=E4=B8=8E=E5=8E=8B=E6=B5=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 322 ++++------- README.md | 65 ++- README.zh.hant.md | 57 +- README.zh.md | 57 +- benchmarks/v1-readiness/mcpp.toml | 9 + benchmarks/v1-readiness/src/main.cpp | 500 ++++++++++++++++++ docs/architecture.md | 96 ++-- docs/architecture.zh.hant.md | 94 ++-- docs/architecture.zh.md | 93 ++-- .../benchmarks/2026-08-29-cmp-v1-readiness.md | 75 +++ .../2026-08-29-cmp-phase4-v1-completion.md | 36 ++ ...6-08-29-cmp-phase4-v1-completion-design.md | 118 +++++ examples/basic/src/main.cpp | 80 ++- src/async_manual_reset_event.cppm | 263 +++++++++ src/cancellation.cppm | 14 + src/cmp.cppm | 2 + src/run_loop.cppm | 191 +++++-- src/task_group.cppm | 13 +- tests/async_manual_reset_event_test.cpp | 336 ++++++++++++ tests/run_loop_test.cpp | 204 +++++++ tests/task_group_test.cpp | 98 +++- 21 files changed, 2274 insertions(+), 449 deletions(-) create mode 100644 benchmarks/v1-readiness/mcpp.toml create mode 100644 benchmarks/v1-readiness/src/main.cpp create mode 100644 docs/benchmarks/2026-08-29-cmp-v1-readiness.md create mode 100644 docs/superpowers/plans/2026-08-29-cmp-phase4-v1-completion.md create mode 100644 docs/superpowers/specs/2026-08-29-cmp-phase4-v1-completion-design.md create mode 100644 src/async_manual_reset_event.cppm create mode 100644 src/cancellation.cppm create mode 100644 tests/async_manual_reset_event_test.cpp diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index fd72b93..93e0819 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -2,217 +2,129 @@ ## 项目概览 -CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库。当前公开核心包括惰性、唯一所有权的 -`Task`,支持变参与同类型动态集合的结构化 `when_all()`,eager 可变结构化 `TaskGroup`, -无分配一次性 `OneShotEvent`、RAII `AsyncMutex`,单消费者 `RunLoop`,以及可复制的 -`RunLoop::Scheduler`。Scheduler 支持普通 -`schedule()`、相对期限 `schedule_after()` 和绝对期限 `schedule_at()`;定时等待可显式接收 -`std::stop_token`,取消获胜时抛出 `OperationCancelled`。由 Scheduler 入队的 continuation -均由调用 `RunLoop::run()` 的线程恢复。 - -项目使用 `mcpplibs.cmp` 模块;`.xlings.json` 固定 mcpp 2026.8.11.2,`mcpp.toml` 固定 -LLVM 22.1.8,测试依赖为 `compat.gtest` 1.15.2。实现位于 `src/`,契约测试位于 `tests/`, -`examples/basic` 是独立的 path-dependency 消费示例。 +CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 `mcpplibs.cmp`。当前 v1 +核心包括: + +- 懒启动、唯一所有权的 `Task`; +- 变参和 `std::vector` 结构化 `when_all()`; +- eager、可递归接纳至静止点的 `TaskGroup`; +- `OneShotEvent`、可取消的 `AsyncManualResetEvent`、RAII `AsyncMutex`; +- 单消费者 `RunLoop` 及可复制 `Scheduler`; +- 普通、相对期限、绝对期限调度,以及显式 `std::stop_token` 协作式取消。 + +`.xlings.json` 固定 mcpp 2026.8.11.2,工具链解析为 LLVM 22.1.8,测试依赖是 +`compat.gtest` 1.15.2。`examples/basic` 是独立 path-dependency consumer。 ## 当前目标与状态 -当前分支为 `main`,已与远端同步到 `7b13ddc`。AsyncMutex v1 的设计、实现、5 项契约测试、 -独立示例和三语文档均已完成;PR #9 及其 squash 合并提交的 Linux、macOS、Windows CI 全部 -通过,当前模块已交付。本阶段完成后读取的本机时间为 2026-08-24 03:07:04 CST,已进入用户 -指定的 03:00–04:00 停止窗口,本轮连续开发目标到此完成。 - -## 已完成工作 - -- 导出 `OperationCancelled`,并为相对、绝对定时等待增加 `std::stop_token` 重载;原有无 token - 重载保持不变。 -- 使用标准 `std::vector` 堆替换私有 `std::priority_queue`,保留普通 Timer 的 O(log n) - 插入和到期路径。 -- 在 awaiter 协程帧内保存最小取消状态,以同步 stop callback 构造竞态、deadline 和取消终态。 -- stop callback 只在 RunLoop 互斥量下标记取消、重排 Timer 并通知,不向 ready 队列分配, - 不在 `request_stop()` 线程 inline 恢复用户协程。 -- deadline 和取消只有一个终态;已经进入 ready 队列的 deadline 完成不会被较晚停止请求替换。 -- 增加预取消、跨线程、RunLoop 线程、非最早 Timer、晚取消、竞态恰好一次、无效 Scheduler、 - 无 stop-state token、清理复用和十万次预取消栈安全等测试。 -- 独立示例现在从协程内依次打印正常定时、并发汇合和取消结果。 -- 已同步 README、架构说明、设计文档和实施计划的英文、简体中文、繁体中文事实。 -- 已完成 `when_all` v1 设计:输入 Task 由汇合 awaiter 结构化持有,全部结束后按参数顺序 - 返回结果或抛出第一个异常;`void` 映射为 `std::monostate`。 -- 已导出 variadic `when_all(Task...)`;支持零输入、异构与 move-only 结果、void - 占位、同步完成和跨线程完成。 -- 新增 8 项汇合契约测试、10 万次同步汇合栈安全压力,以及独立 consumer 的并发结果打印。 -- README 与架构说明的英文、简体中文、繁体中文事实已同步到 `when_all` v1。 -- 已导出 `when_all(std::vector>)`;支持空集合、move-only 与 void 结果、按索引保序、 - 全部收尾后按索引传播首个异常,以及同步和跨线程完成。 -- 范围 awaiter 复用现有 `JoinTask` / `JoinCounter`,在任何子任务启动前预留全部 wrapper 空间, - 不增加依赖、任意 range 抽象或公开生命周期接口。 -- 新增 7 项动态集合契约测试和 5 万次立即汇合栈安全压力;独立示例改为在协程内构造并汇合 - `std::vector>`。 -- README、架构说明、设计文档和实施计划已同步动态集合 API 与 53 项测试基线。 -- 已导出不可移动的 `TaskGroup`;`spawn(Task)` 接管并 eager 启动子任务,单次 `join()` - 关闭接纳、等待全部终态,再按接纳顺序传播第一个异常。 -- TaskGroup 使用一个 mutex 保护状态、计数、wrapper 存储和 join continuation;最后一个子任务 - 在锁外通过对称转移恢复 join,不引入后台线程或 detached 生命周期。 -- TaskGroup 直接复用 `std::stop_source` / `std::stop_token` 提供显式共享取消通道,不自动向 - Task 注入 token。 -- 新增 9 项 TaskGroup 契约测试,覆盖 eager/lazy 边界、并发接纳、join 后关闭、异常顺序、 - 取消、跨线程发布和 5 万次即时完成栈安全。 -- 独立示例和英文、简体中文、繁体中文 README / 架构文档已同步 TaskGroup v1。 -- 已导出不可移动的 `OneShotEvent`;`co_await event` 无分配注册等待者,首次 `set()` 永久设置 - 事件并在 setter 线程恢复全部等待者,后续 set 幂等。 -- event 使用一个原子 sentinel/list 状态和协程帧内侵入节点,注册/set 竞态不会丢失唤醒; - acquire-release 顺序发布 set 前写入。 -- 新增 7 项事件契约测试,覆盖预先 set、多个等待者、setter 线程、1,000 次注册竞态、显式 - 返回 RunLoop 和 5 万等待者栈安全。 -- 已导出不可移动的 `AsyncMutex`;`lock_async()` 无分配返回 move-only RAII Guard,竞争等待者 - 按 FIFO 顺序交接,ownership 不绑定线程。 -- 一个标准 mutex 保护 locked/head/tail;无分配 thread-local trampoline 将立即释放形成的嵌套 - resume 展平成循环,5 万次竞争交接不增长原生栈。 -- 新增 5 项 AsyncMutex 契约测试,覆盖 Guard move、异常释放、FIFO、跨线程 ownership 和栈安全。 - -## 重要决策 - -- 直接复用 `std::stop_source`、`std::stop_token` 和 `std::stop_callback`,不自造 token/source。 -- 仅定时等待增加 token 重载;普通 `schedule()`、Task 隐式传播和结构化并发不属于 v1。 -- token 重载即使收到预先请求停止的 token 也始终挂起,并通过 RunLoop 队列恢复。 -- 取消通过现有 Task 异常路径传播;调用方可以在协程内捕获 `OperationCancelled`。 -- Timer entry 只增加一个非 owning 取消状态指针;外部销毁已发布协程帧仍属于无效使用。 -- 取消当前用 O(n) 扫描并重建堆;只有实测为瓶颈后才引入可删除或带索引的堆。 -- 不增加后台线程、依赖、公开 Timer 句柄、detached 或测试专用公共接口。 -- `when_all` v1 使用“子任务数 + 1”原子计数哨兵,覆盖同步完成和跨线程完成竞态。 -- `when_all` 等待全部子任务收尾后再传播异常,不允许因 fail-fast 提前销毁仍在运行的帧。 -- `when_all` 负责一次性结果汇合;TaskGroup v1 只接纳 `Task`,需要结果时继续使用 - `when_all()` 或由调用方持有至 join 结束的状态。 -- 范围阶段只接受标准 `std::vector>`,不为尚无需求的任意 range 增加模板层。 -- TaskGroup 的 join 仅能使用一次;join 开始后不允许继续或递归接纳,未 join 的 open/joining - group 在析构时终止进程,避免销毁仍被发布的子协程帧。 -- TaskGroup 取消保持显式:`request_stop()` 不会自动取消不接收 token 的子任务,也不提供 - `cancel_and_join()`、结果 future、scheduler 选择或 completed-wrapper 回收。 -- OneShotEvent 只有单向 unset→set 状态,不提供 reset、代际、取消注销、值或隐式 Scheduler - 转移;需要复用事件或 channel 时再设计独立状态机。 -- AsyncMutex v1 只暴露 RAII acquisition,不提供手工 unlock、try-lock、取消注销、递归锁或 - scheduler 选择;只有基准证明短状态锁成为瓶颈后才考虑原子双队列。 - -## 修改 / 重要文件 - -- `src/run_loop.cppm`:公开异常、token 重载、取消状态、回调和显式 Timer 堆。 -- `src/when_all.cppm`:join coroutine、原子计数哨兵和 variadic/vector 公开 API。 -- `src/task_group.cppm`:eager void 子任务作用域、单次 join、显式 stop 通道和生命周期状态机。 -- `src/one_shot_event.cppm`:无分配一次性事件和原子等待者链表。 -- `src/async_mutex.cppm`:RAII Guard、FIFO 等待队列和栈安全交接 trampoline。 -- `src/cmp.cppm`:导出 `:when_all`、`:task_group`、`:one_shot_event` 与 `:async_mutex` 分区。 -- `tests/run_loop_test.cpp`:Cancellation v1 契约、边界、竞态和栈安全测试。 -- `tests/when_all_test.cpp`:变参/vector 的结构化所有权、结果、异常、线程和栈安全测试。 -- `tests/task_group_test.cpp`:TaskGroup 接纳、join、异常、取消、线程和栈安全测试。 -- `tests/one_shot_event_test.cpp`:事件发布、注册竞态、线程和栈安全测试。 -- `tests/async_mutex_test.cpp`:Guard、FIFO、异常、跨线程和交接栈安全测试。 -- `examples/basic/src/main.cpp`:协程内正常、并发、TaskGroup、event、mutex 和预取消输出示例。 -- `README.md`、`README.zh.md`、`README.zh.hant.md`:当前 API、示例和路线图。 -- `docs/architecture.md`、`docs/architecture.zh.md`、`docs/architecture.zh.hant.md`:实现契约、 - 边界与验证基线。 -- `docs/superpowers/specs/2026-08-23-cmp-cancellation-v1-design.md`:已实现设计。 -- `docs/superpowers/plans/2026-08-23-cmp-cancellation-v1.md`:已完成实施清单和本地结果。 -- `docs/superpowers/specs/2026-08-24-cmp-when-all-v1-design.md`:当前汇合 API 设计。 -- `docs/superpowers/plans/2026-08-24-cmp-when-all-v1.md`:当前实施与交付清单。 -- `docs/superpowers/specs/2026-08-24-cmp-when-all-range-design.md`:当前动态集合设计。 -- `docs/superpowers/plans/2026-08-24-cmp-when-all-range.md`:当前动态集合实施清单。 -- `docs/superpowers/specs/2026-08-24-cmp-task-group-v1-design.md`:当前 TaskGroup v1 设计。 -- `docs/superpowers/plans/2026-08-24-cmp-task-group-v1.md`:当前 TaskGroup v1 实施清单。 -- `docs/superpowers/specs/2026-08-24-cmp-one-shot-event-v1-design.md`:当前 OneShotEvent v1 设计。 -- `docs/superpowers/plans/2026-08-24-cmp-one-shot-event-v1.md`:当前 OneShotEvent v1 实施清单。 -- `docs/superpowers/specs/2026-08-24-cmp-async-mutex-v1-design.md`:当前 AsyncMutex v1 设计。 -- `docs/superpowers/plans/2026-08-24-cmp-async-mutex-v1.md`:当前 AsyncMutex v1 实施清单。 -- `.agent/HANDOFF.md`:本文件。 - -## 验证情况 - -- 新测试先因缺少 `OperationCancelled` 和 token 重载按预期编译失败,完成实现后转绿。 -- Dev 严格无缓存构建通过,完整测试 38/38(8 个 Task、30 个 RunLoop)。 -- Cancellation v1 合并前的 Release 严格无缓存构建通过,完整测试 38/38。 -- Release 的完整 RunLoop 30 项套件额外连续执行 10 次,全部通过;未见双恢复、死锁或丢唤醒。 -- `examples/basic` 的 `mcpp run` 状态为 0,依次输出 `Coroutine result: 42` 和 - `Concurrent result: 42`、`Coroutine cancelled`。 -- 功能提交 `3a997ab` 的 Linux x86_64、macOS arm64、Windows x86_64 手动 CI 均通过构建、 - 38 项测试和独立示例。 -- PR #4 最终 head `67d0efc` 的三平台检查全部通过;合并提交 `ad80fb0` 触发的 `main` 三平台 - CI 也全部通过。 -- mcpp 仍提示本机 SubOS 缺少 `subos_info`,但工具链解析为 LLVM 22.1.8,未影响构建或运行。 -- 已按用户授权提交、推送、创建 PR #4 并 squash 合并;未删除远端功能分支或修改其他远端 - 资源。 -- 新测试先因缺少 `when_all` 导出按预期编译失败;实现后聚焦测试 8/8 通过。 -- Dev 与 Release 严格无缓存构建通过;两个 profile 的完整测试均为 46/46(8 个 Task、 - 30 个 RunLoop、8 个 when_all)。 -- Release 的完整 `when_all` 8 项套件额外连续执行 50 次,全部通过。 -- 更新后的独立示例构建运行通过并输出三行预期结果。 -- PR #5 head `d6ea2a8` 的 Linux x86_64、macOS arm64、Windows x86_64 CI 均通过构建、 - 46 项测试和独立示例;已 squash 合并为 `d7dd418`。 -- vector 新测试先因缺少匹配重载按预期编译失败;实现后聚焦测试 15/15 通过。 -- Dev 与 Release 严格无缓存构建通过;两个 profile 的完整测试均为 53/53(8 个 Task、 - 30 个 RunLoop、15 个 when_all)。 -- Release 的完整 `when_all` 15 项套件额外连续执行 50 次,全部通过。 -- 动态集合独立示例运行通过,依次输出 `Coroutine result: 42`、`Concurrent result: 42` 和 - `Coroutine cancelled`。 -- 本阶段本地验证完成于 2026-08-24 01:27:08 CST;远端三平台 CI 尚待提交与 PR 后验证。 -- PR #6 首轮 macOS 在 vector 跨线程测试中以 exit 139 失败;根因是测试辅助 awaiter 启动 - 新线程后才读取自身 `worker_` 成员,新线程可能先恢复并销毁 awaiter。修复为发布前把指针 - 复制到当前栈;库公开实现未因此改变,三平台复验待新提交触发。 -- 竞态修复后的 Dev/Release 聚焦套件均为 15/15;两个跨线程汇合测试在本机 Release 下额外 - 连续执行 5,000 轮全部通过。 -- PR #6 最终 head `ee378aa` 的 Linux x86_64、macOS arm64、Windows x86_64 CI 均通过构建、 - 53 项测试和独立示例;已 squash 合并为 `5c202c8`,功能分支保留。 -- 本地与远端 `main` 已同步到交接提交 `4467ab0`;该提交的 Linux、macOS、Windows 最终 CI - 均成功。 -- TaskGroup 新测试先因缺少公开类型按预期编译失败;实现后聚焦 Dev 测试 9/9 通过。 -- TaskGroup 阶段 Dev/Release 严格无缓存构建通过;两个 profile 的完整测试均为 62/62(8 个 - Task、30 个 RunLoop、15 个 when_all、9 个 TaskGroup)。 -- Release 的完整 TaskGroup 套件额外连续执行 50 轮,跨线程完成测试 5,000 轮、并发接纳测试 - 1,000 轮,全部通过。 -- 更新后的独立示例运行成功,依次输出 `Coroutine result: 42`、`Concurrent result: 42`、 - `Task group result: 42` 和 `Coroutine cancelled`。 -- TaskGroup v1 本地实现与验证阶段完成于 2026-08-24 02:14:36 CST。 -- PR #7 head `c074250` 的 Linux x86_64、macOS arm64、Windows x86_64 CI 均通过 62 项测试和 - 独立示例;已 squash 合并为 `c3883ab`,功能分支保留。 -- 本地与远端 `main` 已同步到交接提交 `a9183fc`;该提交的 Linux run `32657710516`、Windows - run `32657710523`、macOS run `32657710496` 均成功。 -- OneShotEvent 新测试先因缺少公开类型按预期编译失败;实现后聚焦 Dev 测试 7/7 通过。 -- 更新后的独立示例运行成功,并在取消输出前增加 `Event signalled`。 -- OneShotEvent 阶段 Dev/Release 严格无缓存构建通过;两个 profile 的完整测试均为 69/69。 -- Release 的完整事件套件额外连续执行 50 轮,覆盖 50,000 次注册/set 竞态;跨线程恢复测试 - 额外连续执行 5,000 轮,全部通过。 -- OneShotEvent v1 本地实现与验证阶段完成于 2026-08-24 02:37:41 CST。 -- PR #8 head `02efa37` 的 Linux x86_64、macOS arm64、Windows x86_64 CI 均通过 69 项测试和 - 独立示例;已 squash 合并为 `26ae032`,功能分支保留。 -- 本地与远端 `main` 已同步到交接提交 `f8ae3e5`;该提交的 Linux run `32658884971`、macOS - run `32658884957`、Windows run `32658884967` 均成功。 -- AsyncMutex 新测试先因缺少公开类型按预期编译失败;实现后聚焦 Dev 测试 5/5 通过。 -- AsyncMutex 阶段 Dev/Release 严格无缓存构建通过;两个 profile 的完整测试均为 74/74。 -- Release 的完整 mutex 套件额外连续执行 50 轮,跨线程 ownership 测试 5,000 轮,全部通过。 -- 更新后的独立示例运行成功,并在取消输出前增加 `Mutex result: 42`。 -- PR #9 head `6fa1bf3` 的 Linux run `32659868054`、macOS run `32659868121`、Windows run - `32659868088` 均成功;已 squash 合并为 `7b13ddc`,功能分支保留。 -- 合并提交 `7b13ddc` 的 Linux run `32660022056`、macOS run `32660022046`、Windows run - `32660022050` 均成功,本地 `main` 已同步。 - -## 已知问题 / 风险 - -- 三套 Actions 均提示 `actions/checkout@v4` 的 Node.js 20 已弃用并被强制使用 Node.js 24; - 当前不影响 CI 结果,workflow 升级应作为后续独立修改。 -- 取消定位为 O(n),适合当前最小实现;大量并发取消的性能上限尚未基准测试。 -- 外部销毁已经发布到 RunLoop 的协程帧仍不受支持;Cancellation v1 不提供 detached 安全。 -- 取消只覆盖 Scheduler 定时等待,不会中断阻塞调用,也不会自动传播到任意子 Task 或外部 - awaiter。 -- pending `when_all`(包括 vector 重载)仍遵循现有 Task 边界:子任务发布 continuation 后,外部提前销毁聚合 - 帧属于无效使用。 -- TaskGroup 会保留所有 wrapper 帧至析构,空间复杂度为 O(n);v1 不为尚无实测需求增加完成后 - 回收机制。 -- TaskGroup 遗漏 join 会确定性终止,且取消不会自动注入子任务;这两项均是已记录的公开契约。 -- OneShotEvent 会在带 pending 等待者析构时终止;等待者帧必须存活到 set,v1 不提供取消注销。 -- AsyncMutex 会在仍持有或有 pending 等待者时析构终止;排队协程帧必须存活到取得 Guard。 +用户要求补齐第四阶段,增加成功/失败/竞态测试,并对计算、文件 I/O、网络 I/O 做压力验证, +确认 CMP v1 可用于开发。 + +第四阶段实现、测试、示例和本机压测已经完成,交付前最终一致性检查已通过。Git 交付分支为 +`feature/phase4-v1-readiness`;用户已明确授权提交并推送本阶段全部内容(包括压测),但没有 +授权创建 PR 或合并。 + +## 本阶段实现 + +### TaskGroup 静止点 join + +- `join()` 等待活动计数归零,不再在开始等待时立即关闭接纳。 +- 已经属于 group 的活动子任务可在 join 等待期间继续 `spawn()`;静止点到达后永久关闭。 +- 外部线程与最后完成瞬间竞态接纳时不保证哪方获胜;join 后接纳抛出 `std::logic_error`。 +- 新增 lazy `cancel_and_join()`:真正被等待时先 `request_stop()`,再执行同一个单次 join。 + +### 更广泛的显式取消 + +- `OperationCancelled` 移到独立 `:cancellation` 分区。 +- 新增 `Scheduler::schedule(std::stop_token)`;预取消也先排队,消费前取消获胜则抛出异常。 +- ready 与 timer 共用取消状态和 stop callback 协议;终态在 RunLoop 状态锁下确定,避免双恢复。 +- 普通调度和定时调度的取消查找仍为 O(n),只有压测证明瓶颈后才升级数据结构。 + +### AsyncManualResetEvent + +- 新增不可移动、无 waiter 分配的 `AsyncManualResetEvent`。 +- 支持初始状态、`is_set()`、`set()`、`reset()`、`wait(stop_token)` 和 `co_await event`。 +- pending 等待者按 FIFO 恢复;双向侵入队列支持 O(1) 取消移除。 +- set/cancel 竞态只有一个终态;预取消确定由取消获胜。 +- setter 或取消请求线程直接恢复等待者;thread-local trampoline 保证嵌套唤醒不增长原生栈。 +- 带 pending 等待者析构会终止,事件必须比等待协程帧活得更久。 + +## 测试与压测 + +- 全量套件现为 7 个测试二进制、90 项测试。 +- 新增/扩展测试覆盖正常、预取消、晚取消、跨线程、异常、拒绝、静止点、递归接纳、FIFO、 + publication、set/cancel 竞态、生命周期和原生栈安全。 +- `AsyncManualResetEvent` 10 项套件在 Release 下连续 30 轮通过;每轮包含 1,000 次 set/cancel + 竞态、50,000 个等待者和 20,000 个嵌套信号。 +- RunLoop 三项关键取消竞态在 Release 下连续 100 轮通过。 +- TaskGroup 四项递归、并发和取消关键用例在 Release 下连续 100 轮通过。 +- 最终文件状态下,Dev / Release `--strict --cache=off` 构建均通过;两个 profile 的全量 + 测试均为 90/90,7 个测试二进制、0 失败。 + +独立 POSIX Release 压测位于 `benchmarks/v1-readiness`。五轮均为 PASS: + +| 场景 | 每轮成功 | 每轮预期失败 | 每轮非预期失败 | 中位耗时 | 中位吞吐 | +| --- | ---: | ---: | ---: | ---: | ---: | +| compute | 49,000 | 1,000 | 0 | 28.172 ms | 1,774,824.8 ops/s | +| file_io | 1,000 | 100 | 0 | 185.222 ms | 5,938.8 ops/s | +| network_loopback | 20,000 | 100 | 0 | 2,455.883 ms | 8,184.4 ops/s | + +原始五轮数据、环境、命令和边界见 +`docs/benchmarks/2026-08-29-cmp-v1-readiness.md`。文件和网络工作由外部 `std::jthread` +适配,再通过事件和 Scheduler 返回 RunLoop;这验证 v1 互操作及生命周期,不宣称原生异步 +I/O 性能。 + +## 示例输出 + +`examples/basic` 已运行成功,输出: + +```text +Coroutine result: 42 +Concurrent result: 42 +Task group result: 42 +Recursive group result: 3 +Event signalled +Reusable event cycles: 2 +Mutex result: 42 +Coroutine cancelled +``` + +## 本阶段重要文件 + +- `src/cancellation.cppm` +- `src/run_loop.cppm` +- `src/task_group.cppm` +- `src/async_manual_reset_event.cppm` +- `src/cmp.cppm` +- `tests/run_loop_test.cpp` +- `tests/task_group_test.cpp` +- `tests/async_manual_reset_event_test.cpp` +- `examples/basic/src/main.cpp` +- `benchmarks/v1-readiness/mcpp.toml` +- `benchmarks/v1-readiness/src/main.cpp` +- `docs/benchmarks/2026-08-29-cmp-v1-readiness.md` +- `docs/superpowers/specs/2026-08-29-cmp-phase4-v1-completion-design.md` +- `docs/superpowers/plans/2026-08-29-cmp-phase4-v1-completion.md` +- 三份 README、三份架构说明和本文件。 + +## 已知边界与风险 + +- v1 没有原生异步 I/O、blocking pool、多 worker 调度、work stealing 或 detached 所有权。 +- 阻塞调用仍会阻塞其所在执行线程;当前 I/O consumer 使用外部线程适配。 +- 压测的 socket 实现仅支持 POSIX,不进入 Windows/macOS/Linux 通用 CI;库和根测试仍是跨平台目标。 +- Scheduler 大量并发取消时 O(n) 查找可能成为瓶颈,目前没有数据要求升级。 +- TaskGroup 保留 wrapper 帧至析构,空间复杂度 O(n);外部接纳与最终静止点竞态不提供保证。 +- AsyncManualResetEvent 在 set/cancel 调用线程恢复,不隐式保证 RunLoop 亲和;开发者需显式 + `co_await scheduler.schedule()` 返回目标循环。 +- 本机 mcpp 仍提示 SubOS 缺少 `subos_info`;工具链解析和本次构建、测试、运行未受影响。 +- 当前已完成本机验证,尚未进行 PR 三平台 CI 验证。 ## 剩余工作 -当前连续开发目标无剩余模块工作;不要在本轮继续设计或实现下一模块。 +本地实现与验证没有剩余项。本阶段 Git 交付以 `feature/phase4-v1-readiness` 上的第四阶段 +提交为边界;PR、合并和对应三平台 CI 仍需用户另行决定。 ## 推荐下一步 -后续会话如要继续开发,先重新读取本文件、路线图和本机时间,再由同步后的 `main` 创建新 -分支;不要向 AsyncMutex v1 追加手工 unlock、取消注销或 lock-free 双队列。 +确认功能分支远端同步后,由用户决定是否创建 PR 并进行三平台 CI。第四阶段合并后,再单独 +设计第五阶段多 worker 调度,不把第六阶段 I/O 后端提前混入。 diff --git a/README.md b/README.md index f839ba4..1c4a921 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ **English** · [简体中文](README.zh.md) · [繁體中文](README.zh.hant.md) [mcpp](https://github.com/mcpp-community/mcpp) · [Architecture](docs/architecture.md) · +[v1 readiness benchmark](docs/benchmarks/2026-08-29-cmp-v1-readiness.md) · [Issues](https://github.com/mcpplibs/cmp/issues) [![ci-linux](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml) @@ -17,10 +18,10 @@ > [!IMPORTANT] > CMP provides a lazy, single-consumer `Task` / `Task`, structured variadic and vector -> `when_all()`, an eager structured `TaskGroup`, a one-way `OneShotEvent`, an RAII `AsyncMutex`, and -> a caller-thread `RunLoop` with explicit and monotonic timed scheduling. Timed waits and TaskGroup children can -> use explicit cooperative cancellation with `std::stop_token`; asynchronous I/O and detached -> execution are not implemented. +> `when_all()`, an eager structured `TaskGroup`, one-shot and reusable events, an RAII `AsyncMutex`, +> and a caller-thread `RunLoop` with explicit and monotonic timed scheduling. Ready scheduling, +> timed waits, reusable-event waits, and TaskGroup children can use explicit cooperative +> cancellation with `std::stop_token`; asynchronous I/O and detached execution are not implemented. CMP is being built as a modern coroutine runtime and library on standard stackless C++ coroutines. Its explicit `co_await` model now covers fixed and incremental structured concurrency, @@ -74,11 +75,10 @@ cd examples/basic mcpp run ``` -The example prints `Coroutine result: 42`, joins two timed Tasks and prints -`Concurrent result: 42`, eagerly spawns two scoped Tasks and prints `Task group result: 42`, awaits -a one-time notification and prints `Event signalled`, runs two guarded Tasks and prints -`Mutex result: 42`, then prints `Coroutine cancelled` from a pre-cancelled timed wait. All messages -come from `Task` coroutines. +The example prints `Coroutine result: 42`, `Concurrent result: 42`, `Task group result: 42`, and +`Recursive group result: 3`; it then demonstrates one-shot and reusable notifications with +`Event signalled` and `Reusable event cycles: 2`, prints `Mutex result: 42`, and finishes with +`Coroutine cancelled`. All messages come from `Task` coroutines. ## Current API @@ -219,18 +219,23 @@ parameter order is rethrown. Named Tasks must be moved into `when_all()`. A runt homogeneous collection can be passed as `std::vector>`; it returns a result vector in the same index order, and a named input vector must also be moved. -`TaskGroup::spawn()` takes ownership of a `Task` and starts it immediately. Awaiting the -single-use `join()` closes admission and waits for every accepted child before rethrowing the first -exception in admission order. The group must be joined before destruction. `get_stop_token()` and -`request_stop()` provide one explicit standard cancellation channel; the token is not injected -automatically, so developers pass it to children that support cancellation. Use `when_all()` when -child results must be returned. +`TaskGroup::spawn()` takes ownership of a `Task` and starts it immediately. The single-use +`join()` waits until the group reaches quiescence; an active child may recursively add work while +join is waiting. Admission closes permanently when the active count reaches zero. The group must +be joined before destruction. `get_stop_token()` and `request_stop()` provide one explicit +standard cancellation channel, while `cancel_and_join()` requests that channel before joining. +The token is not injected automatically. Use `when_all()` when child results must be returned. `co_await event` suspends on an unset `OneShotEvent` without allocating. The first thread-safe `set()` permanently sets it and resumes every registered waiter exactly once on the setter thread; later waits continue inline and later sets do nothing. The event is immovable and must outlive its waiters. It deliberately has no reset or implicit Scheduler transfer. +`AsyncManualResetEvent` adds reusable `set()` / `reset()` cycles. `wait(stop_token)` supports +cooperative cancellation, pending waiters are resumed in FIFO order, and set-versus-cancel has one +stable winner. Resumption occurs on the thread calling `set()` or requesting cancellation; await a +Scheduler explicitly when RunLoop affinity is required. The event must outlive every waiter. + `auto guard = co_await mutex.lock_async()` acquires `AsyncMutex` without allocating a waiter and releases it through RAII. Contended waiters acquire in FIFO order and resume on the releasing thread; ownership is not tied to a thread. The mutex must outlive every guard and queued waiter. @@ -242,16 +247,18 @@ library. `RunLoop::run()` consumes one root Task, executes ready coroutines on the calling thread, returns its value, and rethrows its exception. `Scheduler::schedule()` always suspends and queues the -continuation. `schedule_after()` waits for a relative `steady_clock` duration, while +continuation; its `std::stop_token` overload can cancel that queued wait. `schedule_after()` waits +for a relative `steady_clock` duration, while `schedule_at()` waits for an absolute steady-clock time point; expiry makes work eligible and never resumes it inline. Scheduler handles are copyable, but remain tied to their originating RunLoop. Sequential `run()` calls are supported; nested or concurrent calls are rejected. A moved-from Task must not be awaited. -The timed overloads accepting `std::stop_token` also always suspend. If cancellation wins before -the deadline, awaiting throws `OperationCancelled`; a late stop request cannot replace an already -queued deadline completion. The stop callback only wakes the RunLoop, so user coroutine code is -still resumed by the thread driving `run()`. Cancellation currently performs an O(n) timer lookup. +All Scheduler overloads accepting `std::stop_token` still queue, including pre-cancelled waits. If +cancellation wins before consumption, awaiting throws `OperationCancelled`; a late stop request +cannot replace a completion that already won. The stop callback only wakes the RunLoop, so user +coroutine code is resumed by the thread driving `run()`. Cancellation currently performs an O(n) +queue lookup. RunLoop is not a background thread and does not make blocking code asynchronous. A Task that suspends without arranging a future resume can leave `run()` waiting indefinitely. CMP does not @@ -266,18 +273,22 @@ await the desired Scheduler to return to its RunLoop. ├── mcpp.toml # package identity and test dependency ├── src/cmp.cppm # root module interface ├── src/task.cppm # Task module partition +├── src/cancellation.cppm # shared cooperative-cancellation exception ├── src/run_loop.cppm # RunLoop and Scheduler partition ├── src/when_all.cppm # structured concurrent Task join ├── src/task_group.cppm # eager mutable structured Task scope ├── src/one_shot_event.cppm # allocation-free one-time notification +├── src/async_manual_reset_event.cppm # reusable cancellable notification ├── src/async_mutex.cppm # FIFO coroutine-aware RAII mutex ├── tests/cmp_test.cpp # Task contract and lifetime tests ├── tests/run_loop_test.cpp # scheduler, boundary, and threading tests ├── tests/when_all_test.cpp # join ownership, result, and race tests ├── tests/task_group_test.cpp # mutable scope lifetime and race tests ├── tests/one_shot_event_test.cpp # event publication and race tests +├── tests/async_manual_reset_event_test.cpp # reusable event race tests ├── tests/async_mutex_test.cpp # mutex ownership and hand-off tests ├── examples/basic/ # standalone path-dependency consumer +├── benchmarks/v1-readiness/ # local compute, file, and loopback baseline ├── docs/architecture.md # current structure, boundaries, and evolution └── .github/workflows/ # Linux, macOS, and Windows CI ``` @@ -290,8 +301,10 @@ after CMP has a stable runtime API worth demonstrating. The local verification path is: ```bash -mcpp build --cache=off -mcpp test --cache=off +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off cd examples/basic && mcpp run ``` @@ -301,6 +314,9 @@ global mcpp installation. CMP does not track `mcpp.lock`; `.gitignore` enforces that repository policy. Runtime dependencies belong in `[dependencies]`; gtest is declared explicitly under `[dev-dependencies.compat]`. +The current local suite contains 90 tests across seven binaries. The POSIX-only Release pressure +consumer and its recorded success/failure data are documented in the +[v1 readiness benchmark](docs/benchmarks/2026-08-29-cmp-v1-readiness.md). ## Roadmap @@ -309,9 +325,8 @@ Runtime work is split into independently reviewable phases: 1. package identity and importable-module bootstrap — implemented; 2. coroutine task and lifetime semantics — initial `Task` implemented; 3. a root runner and minimal single-thread scheduler — initially implemented; -4. monotonic Timer v1, cancellable timed waits, variadic/vector joins, TaskGroup v1, OneShotEvent - v1, and AsyncMutex v1 — implemented; recursive scope admission, broader cancellation - propagation, and reusable wake-up paths remain; +4. monotonic Timer v1, cancellable ready/timed waits, variadic/vector joins, quiescent TaskGroup, + OneShotEvent, AsyncManualResetEvent, and AsyncMutex — implemented and pressure-tested; 5. multi-worker scheduling and work stealing; 6. asynchronous I/O integration and a blocking pool. diff --git a/README.zh.hant.md b/README.zh.hant.md index cbe9b50..eeae0e8 100644 --- a/README.zh.hant.md +++ b/README.zh.hant.md @@ -9,6 +9,7 @@ [English](README.md) · [简体中文](README.zh.md) · **繁體中文** [mcpp](https://github.com/mcpp-community/mcpp) · [架構](docs/architecture.zh.hant.md) · +[v1 可開發性壓測](docs/benchmarks/2026-08-29-cmp-v1-readiness.md) · [Issues](https://github.com/mcpplibs/cmp/issues) [![ci-linux](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml) @@ -17,9 +18,10 @@ > [!IMPORTANT] > CMP 已提供延遲啟動、單一消費者的 `Task` / `Task`、支援變參和 vector 的結構化 -> `when_all()`、eager 結構化 `TaskGroup`、單向 `OneShotEvent`、RAII `AsyncMutex`,以及在呼叫 -> 執行緒運行、支援明確排程和單調時鐘定時排程的 `RunLoop`。定時等待和 TaskGroup 子任務可明確使用基於 -> `std::stop_token` 的協作式取消;非同步 I/O 和 detached 執行尚未實作。 +> `when_all()`、eager 結構化 `TaskGroup`、一次性與可複用事件、RAII `AsyncMutex`,以及在呼叫 +> 執行緒運行、支援明確排程和單調時鐘定時排程的 `RunLoop`。就緒排程、定時等待、可複用事件 +> 等待和 TaskGroup 子任務可明確使用基於 `std::stop_token` 的協作式取消;非同步 I/O 和 +> detached 執行尚未實作。 CMP 計畫以標準無堆疊 C++ 協程建構現代協程執行期與函式庫。明確的 `co_await` 模型現已 涵蓋固定與增量結構化並行、一次性事件通知、排程、單調時鐘計時器和可取消定時等待,並將 @@ -70,10 +72,10 @@ cd examples/basic mcpp run ``` -範例會印出 `Coroutine result: 42`,匯合兩個定時 Task 後印出 `Concurrent result: 42`, -eager 啟動兩個作用域 Task 後印出 `Task group result: 42`,等待一次性通知後印出 -`Event signalled`,執行兩個受保護 Task 後印出 `Mutex result: 42`,最後從預先取消的定時等待 -印出 `Coroutine cancelled`;所有輸出都在 `Task` 協程內部。 +範例會印出 `Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42` 和 +`Recursive group result: 3`,接著透過 `Event signalled` 與 `Reusable event cycles: 2` +展示一次性及可複用通知,再印出 `Mutex result: 42` 和 `Coroutine cancelled`;所有輸出都在 +`Task` 協程內部。 ## 目前 API @@ -212,15 +214,21 @@ continuation 之間直接轉移,並透過 RAII 銷毀未消費的協程框架 作為 `std::vector>` 傳入,並依相同索引順序回傳結果 vector;具名輸入 vector 同樣 必須移動。 -`TaskGroup::spawn()` 接管一個 `Task` 並立即啟動它。等待僅能使用一次的 `join()` 會關閉 -接納,等待所有已接納子任務,再依接納順序重新拋出第一個例外。TaskGroup 必須在解構前完成 -join。`get_stop_token()` 和 `request_stop()` 提供一個明確的標準取消通道;token 不會自動 -注入,開發者需要將它傳給支援取消的子任務。需要傳回子任務結果時應使用 `when_all()`。 +`TaskGroup::spawn()` 接管一個 `Task` 並立即啟動它。僅能使用一次的 `join()` 會等待 group +到達靜止點;join 等待期間,仍在執行的子任務可以遞迴增加工作。活動計數歸零時接納永久關閉。 +TaskGroup 必須在解構前完成 join。`get_stop_token()` 和 `request_stop()` 提供明確的標準取消 +通道,`cancel_and_join()` 會先要求該通道再 join。token 不會自動注入。需要傳回子任務結果時 +應使用 `when_all()`。 在未 set 的 `OneShotEvent` 上執行 `co_await event` 會無分配地暫停。第一次執行緒安全的 `set()` 會永久設定事件,並在 setter 執行緒恰好恢復每個已註冊等待者一次;後續等待 inline 繼續,後續 set 不做任何事。事件不可移動且必須比等待者活得更久;v1 不提供 reset 或隱式 Scheduler 轉移。 +`AsyncManualResetEvent` 提供可複用的 `set()` / `reset()` 週期。`wait(stop_token)` 支援協作式 +取消,pending 等待者依 FIFO 順序恢復,set 與 cancel 競態只有一個穩定獲勝方。協程會在呼叫 +`set()` 或要求取消的執行緒恢復;需要 RunLoop 親和時應明確等待 Scheduler。事件必須比所有 +等待者活得更久。 + `auto guard = co_await mutex.lock_async()` 無分配地取得 `AsyncMutex`,並透過 RAII 釋放。競爭 等待者依 FIFO 順序取得所有權並在釋放執行緒恢復;所有權不綁定執行緒。mutex 必須比所有 Guard 和排隊等待者活得更久。v1 不提供手工 unlock、try-lock、取消或隱式 Scheduler 轉移。 @@ -229,14 +237,15 @@ set 不做任何事。事件不可移動且必須比等待者活得更久;v1 `std`,不會向使用端重新匯出整個標準函式庫。 `RunLoop::run()` 消費一個根 Task,在呼叫執行緒執行就緒協程,回傳結果並重新拋出例外。 -`Scheduler::schedule()` 始終暫停目前協程並把 continuation 放入佇列;`schedule_after()` -等待相對的 `steady_clock` 時長,`schedule_at()` 等待絕對的單調時鐘時間點。期限到達只讓 +`Scheduler::schedule()` 始終暫停目前協程並把 continuation 放入佇列;它的 +`std::stop_token` 多載可以取消該排隊等待。`schedule_after()` 等待相對的 `steady_clock` +時長,`schedule_at()` 等待絕對的單調時鐘時間點。期限到達只讓 任務具備執行資格,不會 inline 恢復。Scheduler 可以複製,但始終屬於建立它的 RunLoop。 支援依序多次呼叫 `run()`,巢狀或並行呼叫會被拒絕。已經被移動的 Task 不得再次等待。 -接受 `std::stop_token` 的定時多載也始終暫停。如果取消先於期限獲勝,等待會拋出 -`OperationCancelled`;較晚的停止要求不能取代已經進入就緒佇列的期限完成。停止回呼只喚醒 -RunLoop,使用者協程仍由執行 `run()` 的執行緒恢復。目前取消透過 O(n) 掃描定位計時器。 +所有接受 `std::stop_token` 的 Scheduler 多載仍會排隊,包括預先取消的等待。如果取消先於 +消費獲勝,等待會拋出 `OperationCancelled`;較晚的停止要求不能取代已經獲勝的完成。停止回呼 +只喚醒 RunLoop,使用者協程仍由執行 `run()` 的執行緒恢復。目前取消透過 O(n) 掃描定位佇列項。 RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步程式碼。如果 Task 暫停後沒有 安排未來的恢復動作,`run()` 可能一直等待。CMP 不提供隱式執行緒親和:外部 awaiter 在其他 @@ -250,18 +259,22 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ├── mcpp.toml # 套件識別與測試相依 ├── src/cmp.cppm # 根模組介面 ├── src/task.cppm # Task 模組分割區 +├── src/cancellation.cppm # 共用的協作式取消例外 ├── src/run_loop.cppm # RunLoop 與 Scheduler 分割區 ├── src/when_all.cppm # 結構化並行 Task 匯合 ├── src/task_group.cppm # eager 可變結構化 Task 作用域 ├── src/one_shot_event.cppm # 無分配一次性通知 +├── src/async_manual_reset_event.cppm # 可複用、可取消通知 ├── src/async_mutex.cppm # FIFO 協程感知 RAII 互斥鎖 ├── tests/cmp_test.cpp # Task 契約和生命週期測試 ├── tests/run_loop_test.cpp # 排程、邊界和執行緒測試 ├── tests/when_all_test.cpp # 匯合所有權、結果和競態測試 ├── tests/task_group_test.cpp # 可變作用域生命週期和競態測試 ├── tests/one_shot_event_test.cpp # 事件發布和競態測試 +├── tests/async_manual_reset_event_test.cpp # 可複用事件競態測試 ├── tests/async_mutex_test.cpp # mutex 所有權和交接測試 ├── examples/basic/ # 獨立的路徑相依 consumer +├── benchmarks/v1-readiness/ # 本機計算、檔案和回環網路基線 ├── docs/architecture.zh.hant.md # 目前結構、邊界與演進方向 └── .github/workflows/ # Linux、macOS 和 Windows CI ``` @@ -273,8 +286,10 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 本機驗證路徑: ```bash -mcpp build --cache=off -mcpp test --cache=off +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off cd examples/basic && mcpp run ``` @@ -283,6 +298,8 @@ CI 在 Linux、macOS 和 Windows 上執行等價的建構、測試與獨立範 CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定。執行期相依放在 `[dependencies]`,gtest 明確宣告在 `[dev-dependencies.compat]` 中。 +目前本機套件包含 7 個測試二進位檔、90 項測試。僅用於 POSIX 的 Release 壓測 consumer 及其 +成功/失敗資料記錄在 [v1 可開發性壓測](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 ## 路線圖 @@ -291,8 +308,8 @@ CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定 1. 套件識別與可匯入模組 bootstrap——已完成; 2. 協程 task 與生命週期語意——已實作初始 `Task`; 3. 根任務驅動器和最小單執行緒排程器——已完成初始實作; -4. 單調時鐘 Timer v1、可取消定時等待、變參/vector 匯合、TaskGroup v1、OneShotEvent v1 和 - AsyncMutex v1——已實作;遞迴作用域接納、更廣泛的取消傳播和可複用喚醒路徑仍待開發; +4. 單調時鐘 Timer v1、可取消就緒/定時等待、變參/vector 匯合、靜止點 TaskGroup、 + OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已實作並完成壓力驗證; 5. 多 worker 排程與 work stealing; 6. 非同步 I/O 整合和 blocking pool。 diff --git a/README.zh.md b/README.zh.md index 7da6ade..246b849 100644 --- a/README.zh.md +++ b/README.zh.md @@ -9,6 +9,7 @@ [English](README.md) · **简体中文** · [繁體中文](README.zh.hant.md) [mcpp](https://github.com/mcpp-community/mcpp) · [架构](docs/architecture.zh.md) · +[v1 可开发性压测](docs/benchmarks/2026-08-29-cmp-v1-readiness.md) · [Issues](https://github.com/mcpplibs/cmp/issues) [![ci-linux](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml) @@ -17,9 +18,10 @@ > [!IMPORTANT] > CMP 已提供懒启动、单消费者的 `Task` / `Task`、支持变参和 vector 的结构化 -> `when_all()`、eager 结构化 `TaskGroup`、单向 `OneShotEvent`、RAII `AsyncMutex`,以及在调用 -> 线程运行、支持显式调度和单调时钟定时调度的 `RunLoop`。定时等待和 TaskGroup 子任务可显式使用基于 -> `std::stop_token` 的协作式取消;异步 I/O 和 detached 执行尚未实现。 +> `when_all()`、eager 结构化 `TaskGroup`、一次性与可复用事件、RAII `AsyncMutex`,以及在调用 +> 线程运行、支持显式调度和单调时钟定时调度的 `RunLoop`。就绪调度、定时等待、可复用事件等待 +> 和 TaskGroup 子任务可显式使用基于 `std::stop_token` 的协作式取消;异步 I/O 和 detached +> 执行尚未实现。 CMP 计划基于标准无栈 C++ 协程构建现代协程运行时和库。显式 `co_await` 模型现已覆盖固定与 增量结构化并发、一次性事件通知、调度、单调时钟定时器和可取消定时等待,并将通过经过验证 @@ -70,10 +72,10 @@ cd examples/basic mcpp run ``` -示例会打印 `Coroutine result: 42`,汇合两个定时 Task 后打印 `Concurrent result: 42`, -eager 启动两个作用域 Task 后打印 `Task group result: 42`,等待一次性通知后打印 -`Event signalled`,运行两个受保护 Task 后打印 `Mutex result: 42`,最后从预先取消的定时等待 -打印 `Coroutine cancelled`;所有输出都在 `Task` 协程内部。 +示例会打印 `Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42` 和 +`Recursive group result: 3`,然后通过 `Event signalled` 与 `Reusable event cycles: 2` +演示一次性及可复用通知,再打印 `Mutex result: 42` 和 `Coroutine cancelled`;所有输出都在 +`Task` 协程内部。 ## 当前 API @@ -212,15 +214,21 @@ continuation 之间直接转移,并通过 RAII 销毁未消费的协程帧。 作为 `std::vector>` 传入,并按相同索引顺序返回结果 vector;命名输入 vector 同样 必须移动。 -`TaskGroup::spawn()` 接管一个 `Task` 并立即启动它。等待单次使用的 `join()` 会关闭接纳, -等待所有已接纳子任务,再按接纳顺序重新抛出第一个异常。TaskGroup 必须在析构前完成 join。 -`get_stop_token()` 和 `request_stop()` 提供一个显式标准取消通道;token 不会自动注入,开发者 -需要将它传给支持取消的子任务。需要返回子任务结果时应使用 `when_all()`。 +`TaskGroup::spawn()` 接管一个 `Task` 并立即启动它。单次使用的 `join()` 会等待 group +到达静止点;join 等待期间,仍在运行的子任务可以递归增加工作。活动计数归零时接纳永久关闭。 +TaskGroup 必须在析构前完成 join。`get_stop_token()` 和 `request_stop()` 提供显式标准取消通道, +`cancel_and_join()` 会先请求该通道再 join。token 不会自动注入。需要返回子任务结果时应使用 +`when_all()`。 在未 set 的 `OneShotEvent` 上执行 `co_await event` 会无分配地挂起。第一次线程安全的 `set()` 会永久设置事件,并在 setter 线程恰好恢复每个已注册等待者一次;后续等待 inline 继续,后续 set 不做任何事。事件不可移动且必须比等待者活得更久;v1 不提供 reset 或隐式 Scheduler 转移。 +`AsyncManualResetEvent` 提供可复用的 `set()` / `reset()` 周期。`wait(stop_token)` 支持协作式 +取消,pending 等待者按 FIFO 顺序恢复,set 与 cancel 竞态只有一个稳定获胜方。协程会在调用 +`set()` 或请求取消的线程恢复;需要 RunLoop 亲和时应显式等待 Scheduler。事件必须比所有等待者 +活得更久。 + `auto guard = co_await mutex.lock_async()` 无分配地取得 `AsyncMutex`,并通过 RAII 释放。竞争等待者 按 FIFO 顺序取得所有权并在释放线程恢复;所有权不绑定线程。mutex 必须比所有 Guard 和排队 等待者活得更久。v1 不提供手工 unlock、try-lock、取消或隐式 Scheduler 转移。 @@ -229,14 +237,15 @@ set 不做任何事。事件不可移动且必须比等待者活得更久;v1 `std`,不会向使用方重新导出整个标准库。 `RunLoop::run()` 消费一个根 Task,在调用线程执行就绪协程,返回结果并重新抛出异常。 -`Scheduler::schedule()` 始终挂起当前协程并把 continuation 放入队列;`schedule_after()` -等待相对的 `steady_clock` 时长,`schedule_at()` 等待绝对的单调时钟时间点。期限到达只让 +`Scheduler::schedule()` 始终挂起当前协程并把 continuation 放入队列;它的 +`std::stop_token` 重载可以取消该排队等待。`schedule_after()` 等待相对的 `steady_clock` +时长,`schedule_at()` 等待绝对的单调时钟时间点。期限到达只让 任务具备运行资格,不会 inline 恢复。Scheduler 可以复制,但始终属于创建它的 RunLoop。 支持顺序多次调用 `run()`,嵌套或并发调用会被拒绝。已经被移动的 Task 不得再次等待。 -接受 `std::stop_token` 的定时重载也始终挂起。如果取消先于期限获胜,等待会抛出 -`OperationCancelled`;较晚的停止请求不能替换已经进入就绪队列的期限完成。停止回调只唤醒 -RunLoop,用户协程仍由执行 `run()` 的线程恢复。当前取消通过 O(n) 扫描定位定时器。 +所有接受 `std::stop_token` 的 Scheduler 重载仍会排队,包括预先取消的等待。如果取消先于 +消费获胜,等待会抛出 `OperationCancelled`;较晚的停止请求不能替换已经获胜的完成。停止回调 +只唤醒 RunLoop,用户协程仍由执行 `run()` 的线程恢复。当前取消通过 O(n) 扫描定位队列项。 RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。如果 Task 挂起后没有安排未来的 恢复动作,`run()` 可能一直等待。CMP 不提供隐式线程亲和:外部 awaiter 在其他线程恢复协程 @@ -250,18 +259,22 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ├── mcpp.toml # 包身份和测试依赖 ├── src/cmp.cppm # 根模块接口 ├── src/task.cppm # Task 模块分区 +├── src/cancellation.cppm # 共享的协作式取消异常 ├── src/run_loop.cppm # RunLoop 与 Scheduler 分区 ├── src/when_all.cppm # 结构化并发 Task 汇合 ├── src/task_group.cppm # eager 可变结构化 Task 作用域 ├── src/one_shot_event.cppm # 无分配一次性通知 +├── src/async_manual_reset_event.cppm # 可复用、可取消通知 ├── src/async_mutex.cppm # FIFO 协程感知 RAII 互斥锁 ├── tests/cmp_test.cpp # Task 契约和生命周期测试 ├── tests/run_loop_test.cpp # 调度、边界和线程测试 ├── tests/when_all_test.cpp # 汇合所有权、结果和竞态测试 ├── tests/task_group_test.cpp # 可变作用域生命周期和竞态测试 ├── tests/one_shot_event_test.cpp # 事件发布和竞态测试 +├── tests/async_manual_reset_event_test.cpp # 可复用事件竞态测试 ├── tests/async_mutex_test.cpp # mutex 所有权和交接测试 ├── examples/basic/ # 独立的路径依赖 consumer +├── benchmarks/v1-readiness/ # 本地计算、文件和回环网络基线 ├── docs/architecture.zh.md # 当前结构、边界和演进方向 └── .github/workflows/ # Linux、macOS 和 Windows CI ``` @@ -273,8 +286,10 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 本地验证路径: ```bash -mcpp build --cache=off -mcpp test --cache=off +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off cd examples/basic && mcpp run ``` @@ -283,6 +298,8 @@ CI 在 Linux、macOS 和 Windows 上执行等价的构建、测试和独立示 CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。运行时依赖放在 `[dependencies]`,gtest 明确声明在 `[dev-dependencies.compat]` 中。 +当前本地套件包含 7 个测试二进制、90 项测试。仅用于 POSIX 的 Release 压测 consumer 及其 +成功/失败数据记录在 [v1 可开发性压测](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 ## 路线图 @@ -291,8 +308,8 @@ CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。 1. 包身份和可导入模块 bootstrap——已完成; 2. 协程 task 与生命周期语义——已实现初始 `Task`; 3. 根任务驱动器和最小单线程调度器——已完成初始实现; -4. 单调时钟 Timer v1、可取消定时等待、变参/vector 汇合、TaskGroup v1、OneShotEvent v1 和 - AsyncMutex v1——已实现;递归作用域接纳、更广泛的取消传播和可复用唤醒路径仍待开发; +4. 单调时钟 Timer v1、可取消就绪/定时等待、变参/vector 汇合、静止点 TaskGroup、 + OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已实现并完成压力验证; 5. 多 worker 调度与 work stealing; 6. 异步 I/O 集成和 blocking pool。 diff --git a/benchmarks/v1-readiness/mcpp.toml b/benchmarks/v1-readiness/mcpp.toml new file mode 100644 index 0000000..b0381c8 --- /dev/null +++ b/benchmarks/v1-readiness/mcpp.toml @@ -0,0 +1,9 @@ +[package] +name = "cmp-v1-readiness" +version = "0.1.0" +standard = "c++23" +description = "Standalone CMP v1 workload readiness benchmark" +license = "Apache-2.0" + +[dependencies.mcpplibs] +cmp = { path = "../.." } diff --git a/benchmarks/v1-readiness/src/main.cpp b/benchmarks/v1-readiness/src/main.cpp new file mode 100644 index 0000000..a63a936 --- /dev/null +++ b/benchmarks/v1-readiness/src/main.cpp @@ -0,0 +1,500 @@ +#if defined(_WIN32) +#error "The loopback benchmark currently requires POSIX sockets" +#endif + +#include +#include +#include +#include +#include + +import std; +import mcpplibs.cmp; + +namespace { + +using mcpplibs::cmp::AsyncManualResetEvent; +using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::Task; +using mcpplibs::cmp::TaskGroup; + +using Scheduler = RunLoop::Scheduler; +using Clock = std::chrono::steady_clock; + +constexpr int COMPUTE_TASKS { 50'000 }; +constexpr int COMPUTE_STEPS { 128 }; +constexpr int FILE_WORKERS { 4 }; +constexpr int FILE_ROUNDS { 250 }; +constexpr int FILE_FAILURES { 25 }; +constexpr int NETWORK_WORKERS { 4 }; +constexpr int NETWORK_ROUNDS { 5'000 }; +constexpr int NETWORK_FAILURES { 25 }; +constexpr std::size_t FILE_PAYLOAD_SIZE { 64 * 1024 }; +constexpr std::size_t NETWORK_PAYLOAD_SIZE { 256 }; + +struct Counters final { + std::atomic successes_ {}; + std::atomic expectedFailures_ {}; + std::atomic unexpectedFailures_ {}; +}; + +struct Metrics final { + std::string scenario_ {}; + std::size_t operations_ {}; + std::size_t successes_ {}; + std::size_t expectedFailures_ {}; + std::size_t unexpectedFailures_ {}; + double elapsedMilliseconds_ {}; + + [[nodiscard]] bool passed() const noexcept { + return unexpectedFailures_ == 0 && + successes_ + expectedFailures_ == operations_; + } +}; + +class Socket final { +private: + int handle_ { -1 }; + +public: + Socket() noexcept = default; + + explicit Socket(int handle) noexcept + : handle_ { handle } {} + + Socket(const Socket&) = delete; + Socket& operator=(const Socket&) = delete; + + Socket(Socket&& other) noexcept + : handle_ { std::exchange(other.handle_, -1) } {} + + Socket& operator=(Socket&& other) noexcept { + if (this != &other) { + reset(); + handle_ = std::exchange(other.handle_, -1); + } + return *this; + } + + ~Socket() { + reset(); + } + + [[nodiscard]] int get() const noexcept { + return handle_; + } + + void reset() noexcept { + if (handle_ >= 0) { + ::close(handle_); + handle_ = -1; + } + } +}; + +class TemporaryDirectory final { +private: + std::filesystem::path path_ {}; + +public: + TemporaryDirectory() { + const auto suffix = Clock::now().time_since_epoch().count(); + path_ = std::filesystem::temp_directory_path() / + std::format("cmp-v1-readiness-{}", suffix); + + if (!std::filesystem::create_directory(path_)) { + throw std::runtime_error { "cannot create benchmark directory" }; + } + } + + TemporaryDirectory(const TemporaryDirectory&) = delete; + TemporaryDirectory& operator=(const TemporaryDirectory&) = delete; + TemporaryDirectory(TemporaryDirectory&&) = delete; + TemporaryDirectory& operator=(TemporaryDirectory&&) = delete; + + ~TemporaryDirectory() { + std::error_code error {}; + std::filesystem::remove_all(path_, error); + } + + [[nodiscard]] const std::filesystem::path& path() const noexcept { + return path_; + } +}; + +[[noreturn]] void throw_socket_error(std::string_view operation) { + throw std::system_error { + errno, + std::generic_category(), + std::string { operation } + }; +} + +[[nodiscard]] Socket make_socket() { + const int handle = ::socket(AF_INET, SOCK_STREAM, 0); + if (handle < 0) { + throw_socket_error("socket"); + } + return Socket { handle }; +} + +[[nodiscard]] std::pair make_loopback_pair() { + auto listener = make_socket(); + sockaddr_in address {}; + address.sin_family = AF_INET; + address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + + if (::bind( + listener.get(), + reinterpret_cast(&address), + sizeof(address)) != 0) { + throw_socket_error("bind"); + } + + if (::listen(listener.get(), 1) != 0) { + throw_socket_error("listen"); + } + + socklen_t addressLength { sizeof(address) }; + if (::getsockname( + listener.get(), + reinterpret_cast(&address), + &addressLength) != 0) { + throw_socket_error("getsockname"); + } + + auto client = make_socket(); + if (::connect( + client.get(), + reinterpret_cast(&address), + sizeof(address)) != 0) { + throw_socket_error("connect"); + } + + const int accepted = ::accept(listener.get(), nullptr, nullptr); + if (accepted < 0) { + throw_socket_error("accept"); + } + + return { std::move(client), Socket { accepted } }; +} + +void send_all(int socket, std::string_view data) { + std::size_t offset { 0 }; + + while (offset < data.size()) { + const auto sent = ::send( + socket, + data.data() + offset, + data.size() - offset, + 0); + if (sent <= 0) { + throw_socket_error("send"); + } + offset += static_cast(sent); + } +} + +[[nodiscard]] std::string receive_exactly(int socket, std::size_t size) { + std::string data(size, '\0'); + std::size_t offset { 0 }; + + while (offset < size) { + const auto received = ::recv( + socket, + data.data() + offset, + size - offset, + 0); + if (received <= 0) { + throw_socket_error("recv"); + } + offset += static_cast(received); + } + + return data; +} + +[[nodiscard]] bool non_listening_loopback_connect_is_refused() { + auto reservation = make_socket(); + sockaddr_in address {}; + address.sin_family = AF_INET; + address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + + if (::bind( + reservation.get(), + reinterpret_cast(&address), + sizeof(address)) != 0) { + throw_socket_error("failure bind"); + } + + socklen_t addressLength { sizeof(address) }; + if (::getsockname( + reservation.get(), + reinterpret_cast(&address), + &addressLength) != 0) { + throw_socket_error("failure getsockname"); + } + auto probe = make_socket(); + const int result = ::connect( + probe.get(), + reinterpret_cast(&address), + sizeof(address)); + return result != 0 && errno == ECONNREFUSED; +} + +template +Task run_external(Scheduler scheduler, Work work) { + AsyncManualResetEvent completed {}; + std::exception_ptr exception {}; + + std::jthread worker { [&] { + try { + work(); + } catch (...) { + exception = std::current_exception(); + } + completed.set(); + } }; + + co_await completed; + co_await scheduler.schedule(); + worker.join(); + + if (exception) { + std::rethrow_exception(exception); + } +} + +Task failing_compute(Scheduler scheduler) { + co_await scheduler.schedule(); + throw std::invalid_argument { "expected compute failure" }; +} + +Task compute_operation( + Scheduler scheduler, + int index, + Counters& counters, + std::atomic& checksum) { + if (index % 50 == 0) { + try { + co_await failing_compute(scheduler); + } catch (const std::invalid_argument&) { + counters.expectedFailures_.fetch_add(1); + } catch (...) { + counters.unexpectedFailures_.fetch_add(1); + } + co_return; + } + + co_await scheduler.schedule(); + std::uint64_t value = static_cast(index) + 1; + for (int step { 0 }; step < COMPUTE_STEPS; ++step) { + value ^= value << 13; + value ^= value >> 7; + value ^= value << 17; + } + + checksum.fetch_xor(value, std::memory_order_relaxed); + counters.successes_.fetch_add(1); +} + +Task run_compute(Scheduler scheduler, Counters& counters) { + std::atomic checksum {}; + TaskGroup group {}; + + for (int index { 0 }; index < COMPUTE_TASKS; ++index) { + group.spawn(compute_operation( + scheduler, + index, + counters, + checksum)); + } + + co_await group.join(); + if (checksum.load(std::memory_order_relaxed) == 0) { + counters.unexpectedFailures_.fetch_add(1); + } +} + +Task run_file_io( + Scheduler scheduler, + const std::filesystem::path& directory, + Counters& counters) { + const std::string payload(FILE_PAYLOAD_SIZE, 'f'); + TaskGroup group {}; + + for (int workerIndex { 0 }; workerIndex < FILE_WORKERS; ++workerIndex) { + group.spawn(run_external(scheduler, [&, workerIndex] { + const auto file = directory / + std::format("worker-{}.bin", workerIndex); + + for (int round { 0 }; round < FILE_ROUNDS; ++round) { + try { + { + std::ofstream output { + file, + std::ios::binary | std::ios::trunc + }; + output.write(payload.data(), payload.size()); + if (!output) { + throw std::runtime_error { "file write failed" }; + } + } + + std::ifstream input { file, std::ios::binary }; + std::string actual( + std::istreambuf_iterator { input }, + std::istreambuf_iterator {}); + if (input.bad() || actual != payload) { + throw std::runtime_error { "file verification failed" }; + } + counters.successes_.fetch_add(1); + } catch (...) { + counters.unexpectedFailures_.fetch_add(1); + } + } + + const auto missing = directory / "missing" / "input.bin"; + for (int failure { 0 }; failure < FILE_FAILURES; ++failure) { + std::ifstream input { missing, std::ios::binary }; + if (!input) { + counters.expectedFailures_.fetch_add(1); + } else { + counters.unexpectedFailures_.fetch_add(1); + } + } + })); + } + + co_await group.join(); +} + +Task run_network_io(Scheduler scheduler, Counters& counters) { + const std::string payload(NETWORK_PAYLOAD_SIZE, 'n'); + TaskGroup group {}; + + for (int workerIndex { 0 }; workerIndex < NETWORK_WORKERS; ++workerIndex) { + static_cast(workerIndex); + group.spawn(run_external(scheduler, [&] { + int completed { 0 }; + try { + auto [client, server] = make_loopback_pair(); + for (; completed < NETWORK_ROUNDS; ++completed) { + send_all(client.get(), payload); + if (receive_exactly(server.get(), payload.size()) != payload) { + throw std::runtime_error { "server payload mismatch" }; + } + + send_all(server.get(), payload); + if (receive_exactly(client.get(), payload.size()) != payload) { + throw std::runtime_error { "client payload mismatch" }; + } + counters.successes_.fetch_add(1); + } + } catch (...) { + counters.unexpectedFailures_.fetch_add( + NETWORK_ROUNDS - completed); + } + + for (int failure { 0 }; failure < NETWORK_FAILURES; ++failure) { + try { + if (non_listening_loopback_connect_is_refused()) { + counters.expectedFailures_.fetch_add(1); + } else { + counters.unexpectedFailures_.fetch_add(1); + } + } catch (...) { + counters.unexpectedFailures_.fetch_add(1); + } + } + })); + } + + co_await group.join(); +} + +template +[[nodiscard]] Metrics measure( + std::string scenario, + std::size_t operations, + Scenario scenarioTask) { + Counters counters {}; + RunLoop loop {}; + const auto start = Clock::now(); + + try { + loop.run(scenarioTask(loop.get_scheduler(), counters)); + } catch (...) { + counters.unexpectedFailures_.fetch_add(1); + } + + const auto elapsed = Clock::now() - start; + return Metrics { + std::move(scenario), + operations, + counters.successes_.load(), + counters.expectedFailures_.load(), + counters.unexpectedFailures_.load(), + std::chrono::duration { elapsed }.count() + }; +} + +void print_metrics(const Metrics& metrics) { + const double throughput = metrics.elapsedMilliseconds_ > 0.0 + ? static_cast(metrics.operations_) * 1'000.0 / + metrics.elapsedMilliseconds_ + : 0.0; + + std::println( + "{},{},{},{},{},{:.3f},{:.1f},{}", + metrics.scenario_, + metrics.operations_, + metrics.successes_, + metrics.expectedFailures_, + metrics.unexpectedFailures_, + metrics.elapsedMilliseconds_, + throughput, + metrics.passed() ? "PASS" : "FAIL"); +} + +} // namespace + +int main() { + TemporaryDirectory directory {}; + std::vector results {}; + results.reserve(3); + + results.emplace_back(measure( + "compute", + COMPUTE_TASKS, + [](Scheduler scheduler, Counters& counters) { + return run_compute(scheduler, counters); + })); + + results.emplace_back(measure( + "file_io", + FILE_WORKERS * (FILE_ROUNDS + FILE_FAILURES), + [&](Scheduler scheduler, Counters& counters) { + return run_file_io(scheduler, directory.path(), counters); + })); + + results.emplace_back(measure( + "network_loopback", + NETWORK_WORKERS * (NETWORK_ROUNDS + NETWORK_FAILURES), + [](Scheduler scheduler, Counters& counters) { + return run_network_io(scheduler, counters); + })); + + std::println( + "environment,hardware_threads={} free_space_bytes={}", + std::thread::hardware_concurrency(), + std::filesystem::space(directory.path()).available); + std::println( + "scenario,operations,successes,expected_failures,unexpected_failures,elapsed_ms,ops_per_second,status"); + + for (const auto& result : results) { + print_metrics(result); + } + + return std::ranges::all_of(results, &Metrics::passed) ? 0 : 1; +} diff --git a/docs/architecture.md b/docs/architecture.md index 66bed7a..4753e15 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -6,19 +6,21 @@ CMP is a C++23 module project with a small coroutine execution core. The root module exports a lazy, single-consumer `mcpplibs::cmp::Task`, structured variadic/vector `when_all()`, eager -`TaskGroup`, allocation-free `OneShotEvent`, RAII `AsyncMutex`, `RunLoop`, and its copyable `Scheduler` handle. The -join primitives own every child until completion, while the event publishes one external signal. +`TaskGroup`, allocation-free `OneShotEvent`, reusable `AsyncManualResetEvent`, RAII `AsyncMutex`, +`RunLoop`, and its copyable `Scheduler` handle. Join primitives own every child until completion, +while the events publish external signals. `RunLoop::run()` is the public root execution boundary, while `Scheduler::schedule()` explicitly -returns a suspended coroutine to that loop. `schedule_after()` and `schedule_at()` add relative -and absolute `steady_clock` deadlines without a timer thread; overloads accepting -`std::stop_token` make those waits cooperatively cancellable. +returns a suspended coroutine to that loop. `schedule()`, `schedule_after()`, and `schedule_at()` +have `std::stop_token` overloads for cooperative cancellation; timed scheduling uses relative and +absolute `steady_clock` deadlines without a timer thread. The repository contains: - one mcpp package manifest; -- the root module `mcpplibs.cmp` with Task, RunLoop, join, event, and mutex partitions; +- the root module `mcpplibs.cmp` with Task, cancellation, RunLoop, join, event, and mutex partitions; - gtest contract, lifetime, exception, scheduling, and threading tests; - one standalone path-dependency example; +- one local POSIX pressure consumer for compute, file, and loopback network integration; - Linux, macOS, and Windows CI workflows. ## Package and module identity @@ -65,10 +67,12 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. ├── src/ │ ├── cmp.cppm │ ├── task.cppm +│ ├── cancellation.cppm │ ├── run_loop.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm +│ ├── async_manual_reset_event.cppm │ └── async_mutex.cppm ├── tests/ │ ├── cmp_test.cpp @@ -76,14 +80,18 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp +│ ├── async_manual_reset_event_test.cpp │ └── async_mutex_test.cpp +├── benchmarks/v1-readiness/ +│ ├── mcpp.toml +│ └── src/main.cpp └── mcpp.toml ``` ## Build and tests `.xlings.json` pins the mcpp version used by the project. `mcpp build` builds the inferred library -target. `mcpp test` discovers six test files and links a gtest entry point for each. The tests +target. `mcpp test` discovers seven test files and links a gtest entry point for each. The 90 tests verify Task ownership and symmetric transfer together with structured joins, root execution, scheduling, exception propagation, timed and cross-thread wake-up, cancellation races, invalid scheduler use, loop reuse, and stack-safe repeated completion. @@ -114,7 +122,7 @@ standard = "c++23" cmp = { path = "../.." } ``` -Its program uses the same import path as an external package: +This abridged API sample uses the same import path as an external package: ```cpp import std; @@ -244,9 +252,11 @@ compilation, and the public root runner independently of the root test targets. `print_answer()` on the main thread; a short monotonic timer expires before the coroutine prints `Coroutine result: 42`. The next root Task concurrently joins two timed values and prints `Concurrent result: 42`. The next coroutine eagerly spawns and joins two void Tasks before printing -`Task group result: 42`. Another coroutine awaits a one-time signal and prints `Event signalled`. -Two guarded Tasks produce `Mutex result: 42`. A final coroutine catches `OperationCancelled` from -a pre-cancelled timed wait and prints `Coroutine cancelled`. +`Task group result: 42`; a recursively growing group then prints `Recursive group result: 3`. +Other coroutines print `Event signalled`, exercise two reusable-event cycles, and print +`Reusable event cycles: 2`. Two guarded Tasks produce `Mutex result: 42`. A final structured group +uses `cancel_and_join()` and catches `OperationCancelled` from cancellable ready scheduling before +printing `Coroutine cancelled`. Any translation unit that defines a coroutine imports `std` itself so `std::coroutine_traits` and the standard coroutine protocol types participate in compilation. The CMP module imports `std` @@ -282,12 +292,16 @@ contract violation. `TaskGroup` has the following contract: - `spawn(Task)` accepts ownership and starts the child inline before returning; -- concurrent admission is serialized, while awaiting the single-use `join()` closes admission; +- concurrent admission is serialized; an active child may recursively admit work while the + single-use `join()` is waiting; +- admission closes permanently when the active count reaches zero; external admission racing the + last completion has no guaranteed winner; - every accepted child reaches a terminal state before join resumes; - the first exception in admission order is rethrown only after all children finish; - the group is immovable and must be unused or joined when destroyed; otherwise it terminates; - `get_stop_token()` and `request_stop()` expose an explicit standard cancellation channel but do not inject it into Tasks; +- `cancel_and_join()` lazily requests that channel and then performs the same join; - the final child resumes join on its completion thread; no scheduler affinity is implicit. `OneShotEvent` has the following contract: @@ -300,6 +314,16 @@ contract violation. - the event is immovable and must outlive every waiter; pending destruction terminates; - reset, cancellation-aware removal, values, and implicit Scheduler transfer are not provided. +`AsyncManualResetEvent` has the following contract: + +- a default event is unset; `set()` resumes pending waiters in FIFO order and keeps later waits ready; +- `reset()` changes only future waits, while waiters already selected by set still complete; +- `wait(stop_token)` removes one cancelled pending waiter in O(1), and a pre-cancelled wait wins; +- set versus cancellation has exactly one winner and resumes the waiter exactly once; +- waiters resume on the setter or cancellation-request thread, with no implicit Scheduler transfer; +- a thread-local dispatch trampoline keeps nested signal chains stack-safe; +- the event is immovable and must outlive every waiter; pending destruction terminates. + `AsyncMutex` has the following contract: - `lock_async()` returns an allocation-free operation whose awaited result is a move-only RAII Guard; @@ -316,15 +340,16 @@ contract violation. - a root value, including a move-only value, is returned; a root exception is rethrown; - a RunLoop can be reused sequentially, but nested and concurrent `run()` calls throw `std::logic_error`; -- `schedule()` always suspends and appends its continuation to a thread-safe FIFO ready queue; +- `schedule()` always suspends and appends its continuation to a thread-safe FIFO ready queue; its + token-taking overload can cancel the queued wait before consumption; - `schedule_after()` measures a native `steady_clock` duration at suspension, while `schedule_at()` accepts an absolute steady-clock time point; -- token-taking timed overloads always suspend; cancellation wins by throwing - `OperationCancelled`, while a late stop request cannot replace a deadline already queued; +- all token-taking overloads still queue, including pre-cancelled waits; cancellation wins by + throwing `OperationCancelled`, while a late stop request cannot replace a completion that won; - elapsed deadlines remain asynchronous; future deadlines use a minimum timer heap, and expiry only makes the continuation eligible for FIFO dispatch; -- stop callbacks only change timer state and wake the RunLoop; they never resume user coroutine - code inline, and cancellation currently locates its timer with an O(n) scan; +- stop callbacks only change ready/timer state and wake the RunLoop; they never resume user + coroutine code inline, and cancellation currently locates its queue entry with an O(n) scan; - producers may enqueue from other threads, but only the thread inside `run()` consumes the queue; - using a Scheduler after its RunLoop is destroyed, or while its own RunLoop is not active, throws `std::logic_error` from the await expression; @@ -339,9 +364,9 @@ thread on which the coroutine currently executes. There is no public free-standing `sync_wait`, detached execution, standalone Timer handle, asynchronous I/O backend, custom frame allocator, or blocking-work pool. Cancellation remains -explicit: timed waits accept tokens and TaskGroup owns an optional shared stop channel, but the -module provides neither implicit propagation nor a token overload for plain `schedule()`, and no -compatibility alias for the old scaffold module. +explicit: Scheduler waits and `AsyncManualResetEvent` accept tokens, and TaskGroup owns an optional +shared stop channel, but the module provides no implicit propagation and no compatibility alias for +the old scaffold module. Capturing coroutine lambdas require particular care: invoking a temporary capturing lambda can leave the lazy coroutine referring to a destroyed closure. CMP does not yet provide a helper that @@ -356,35 +381,38 @@ scheduler and do not make a blocking operation asynchronous. The following areas may be considered in separate designs. They are not part of the current package contract: -1. recursive scope admission, result handles, and broader cancellation propagation; -2. reusable events, channels, and additional structured wake-up paths; +1. TaskGroup result handles and additional cancellation-aware primitives; +2. channels and additional structured wake-up paths; 3. multi-worker scheduling and work stealing; 4. asynchronous I/O integrations; 5. a dedicated pool for unavoidable blocking work; 6. result adapters and optional coroutine-frame allocation strategies. -Task, RunLoop, `when_all`, TaskGroup, OneShotEvent, and AsyncMutex occupy separate module partitions because -they are implemented public boundaries. Further partitions or implementation units are added only -when another implemented API needs them. +Task, cancellation, RunLoop, `when_all`, TaskGroup, OneShotEvent, AsyncManualResetEvent, and +AsyncMutex occupy separate module partitions because they are implemented public boundaries. +Further partitions or implementation units are added only when another implemented API needs them. ## Verification Run from the repository root: ```text -mcpp build --cache=off -mcpp test --cache=off +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off cd examples/basic mcpp run ``` -The expected result is a successful library build, 74 passing tests across six binaries, and an +The expected result is a successful library build, 90 passing tests across seven binaries, and an example that prints `Coroutine result: 42`, `Concurrent result: 42`, `Task group result: 42`, -`Event signalled`, `Mutex result: 42`, then `Coroutine cancelled` and exits with status 0. Tests perform one million -immediate Task completions, 100,000 immediate two-Task joins, 50,000 immediate two-Task vector -joins, 50,000 eager TaskGroup completions, 50,000 event waiters, 50,000 mutex hand-offs, 100,000 explicit schedules, -100,000 immediate timers, and 100,000 pre-cancelled timed waits. These check that symmetric -transfer and all joined or queued paths do -not grow the native call stack. The current Windows LLVM toolchain +`Recursive group result: 3`, `Event signalled`, `Reusable event cycles: 2`, `Mutex result: 42`, then +`Coroutine cancelled` and exits with status 0. Tests retain the existing high-volume stack checks +and add 20,000 recursive TaskGroup admissions, 100,000 pre-cancelled ready schedules, 50,000 manual +event waiters, 20,000 nested reusable-event signals, and set/cancel races. Focused phase-4 race +suites pass repeated Release runs. Compute, temporary-file, and loopback-network counts and +throughput are recorded in the +[v1 readiness benchmark](benchmarks/2026-08-29-cmp-v1-readiness.md). The current Windows LLVM toolchain does not emit GNU depfiles. If a file included by a module interface changes, an incremental build can reuse an older BMI or object; `--cache=off` is used for a full local verification. diff --git a/docs/architecture.zh.hant.md b/docs/architecture.zh.hant.md index 6cae878..aff3fa2 100644 --- a/docs/architecture.zh.hant.md +++ b/docs/architecture.zh.hant.md @@ -6,18 +6,20 @@ CMP 是一個具備小型協程執行核心的 C++23 模組專案。根模組匯出延遲啟動、單一消費者的 `mcpplibs::cmp::Task`、結構化變參/vector `when_all()`、eager `TaskGroup`、`RunLoop` 及其 -可複製的 `Scheduler` 控制代碼、無分配 `OneShotEvent` 和 RAII `AsyncMutex`。匯合原語會持有每個子任務直到結束, -一次性事件負責發布一個外部訊號。`RunLoop::run()` 是公開根任務執行邊界, -`Scheduler::schedule()` 用於明確地把暫停協程送回對應執行迴圈。 -`schedule_after()` 和 `schedule_at()` 在沒有計時執行緒的情況下提供相對和絕對的 -`steady_clock` 期限;接受 `std::stop_token` 的多載使這些等待可以協作式取消。 +可複製的 `Scheduler` 控制代碼、無分配 `OneShotEvent`、可複用 +`AsyncManualResetEvent` 和 RAII `AsyncMutex`。匯合原語會持有每個子任務直到結束,事件負責 +發布外部訊號。`RunLoop::run()` 是公開根任務執行邊界,`Scheduler::schedule()` 用於明確地 +把暫停協程送回對應執行迴圈。`schedule()`、`schedule_after()` 和 `schedule_at()` 都有接受 +`std::stop_token` 的協作式取消多載;定時排程使用相對和絕對的 `steady_clock` 期限,且不建立 +計時執行緒。 儲存庫現有內容包括: - 一份 mcpp 套件清單; -- 根模組 `mcpplibs.cmp` 及 Task、RunLoop、join、event、mutex 模組分割區; +- 根模組 `mcpplibs.cmp` 及 Task、cancellation、RunLoop、join、event、mutex 模組分割區; - 涵蓋契約、生命週期、例外、排程和執行緒行為的 gtest 測試; - 一個透過路徑相依使用根套件的獨立範例; +- 一個驗證計算、檔案與回環網路整合的本機 POSIX 壓測 consumer; - Linux、macOS 和 Windows 三套 CI 工作流程。 ## 套件和模組識別 @@ -62,10 +64,12 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m ├── src/ │ ├── cmp.cppm │ ├── task.cppm +│ ├── cancellation.cppm │ ├── run_loop.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm +│ ├── async_manual_reset_event.cppm │ └── async_mutex.cppm ├── tests/ │ ├── cmp_test.cpp @@ -73,14 +77,18 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp +│ ├── async_manual_reset_event_test.cpp │ └── async_mutex_test.cpp +├── benchmarks/v1-readiness/ +│ ├── mcpp.toml +│ └── src/main.cpp └── mcpp.toml ``` ## 建置與測試 `.xlings.json` 固定專案使用的 mcpp 版本。`mcpp build` 建置自動推斷的函式庫目標。 -`mcpp test` 會找到六個測試檔案,並為每個檔案連結 gtest 進入點。測試同時驗證 Task 所有權 +`mcpp test` 會找到七個測試檔案,並為每個檔案連結 gtest 進入點。90 項測試同時驗證 Task 所有權 和對稱轉移、結構化匯合,以及根任務執行、普通與定時排程、例外傳播、跨執行緒期限喚醒、 無效 Scheduler、取消競態、RunLoop 重複使用和不會增長呼叫堆疊的重複完成。 @@ -108,7 +116,7 @@ standard = "c++23" cmp = { path = "../.." } ``` -範例程式使用與外部專案相同的匯入路徑: +下面的精簡 API 範例使用與外部專案相同的匯入路徑: ```cpp import std; @@ -234,12 +242,11 @@ int main() { ``` 這個範例在根測試目標之外,單獨檢查路徑相依解析、模組使用、外部協程編譯和公開根任務 -驅動器。RunLoop 在主執行緒驅動 `print_answer()`;短單調時鐘計時器到期後,協程輸出 -`Coroutine result: 42`。下一個根任務並行匯合兩個定時結果並輸出 -`Concurrent result: 42`。隨後一個協程 eager 啟動並匯合兩個 void Task,再輸出 -`Task group result: 42`。另一個協程等待一次性訊號並輸出 `Event signalled`,兩個受保護 Task -隨後輸出 `Mutex result: 42`。最後一個協程從預先取消的定時等待捕捉 `OperationCancelled`,並 -輸出 `Coroutine cancelled`。 +驅動器。RunLoop 依序輸出 `Coroutine result: 42`、`Concurrent result: 42` 和 +`Task group result: 42`;遞迴增長的 group 輸出 `Recursive group result: 3`。其他協程展示 +一次性及兩輪可複用事件並輸出 `Event signalled`、`Reusable event cycles: 2`,兩個受保護 +Task 輸出 `Mutex result: 42`。最後一個結構化 group 使用 `cancel_and_join()`,從可取消就緒 +排程捕捉 `OperationCancelled` 並輸出 `Coroutine cancelled`。 任何定義協程的轉譯單元都要自行匯入 `std`,使 `std::coroutine_traits` 和標準協程協定型別 參與編譯。CMP 模組私下匯入 `std`,而不是向使用端重新匯出整個標準函式庫。 @@ -273,11 +280,13 @@ int main() { `TaskGroup` 遵循以下契約: - `spawn(Task)` 接管所有權並在傳回前 inline 啟動子任務; -- 並行接納會序列化,等待僅能使用一次的 `join()` 會關閉接納; +- 並行接納會序列化;等待僅能使用一次的 `join()` 期間,仍在執行的子任務可以遞迴接納工作; +- 活動計數歸零時接納永久關閉;外部接納與最後一個完成競態時不保證哪方獲勝; - join 恢復前,每個已接納子任務都必須到達終態; - 全部子任務完成後,依接納順序重新拋出第一個例外; - group 不可移動,解構時必須未使用或已經 join,否則終止程序; - `get_stop_token()` 和 `request_stop()` 提供明確的標準取消通道,但不會向 Task 注入 token; +- `cancel_and_join()` 會在真正被等待時要求該通道,然後執行相同的 join; - 最後一個子任務在其完成執行緒恢復 join,不隱式增加排程器親和。 `OneShotEvent` 遵循以下契約: @@ -290,6 +299,16 @@ int main() { - 事件不可移動且必須比所有等待者活得更久;帶 pending 等待者解構會終止程序; - 不提供 reset、可取消註銷、值或隱式 Scheduler 轉移。 +`AsyncManualResetEvent` 遵循以下契約: + +- 預設事件為 unset;`set()` 依 FIFO 恢復 pending 等待者,並讓後續等待保持 ready; +- `reset()` 只影響未來等待,已被 set 選中的等待者仍會完成; +- `wait(stop_token)` 以 O(1) 移除一個取消的 pending 等待者,預先取消的等待確定由取消獲勝; +- set 與取消競態恰有一個獲勝方,等待者只恢復一次; +- 等待者在 setter 或取消要求執行緒恢復,不隱式轉移 Scheduler; +- thread-local dispatch trampoline 保證巢狀訊號鏈不增長原生呼叫堆疊; +- 事件不可移動且必須比所有等待者活得更久;帶 pending 等待者解構會終止程序。 + `AsyncMutex` 遵循以下契約: - `lock_async()` 傳回無分配操作,等待結果是 move-only RAII Guard; @@ -305,15 +324,16 @@ int main() { - `run(Task)` 消費一個根 Task,並在呼叫執行緒執行就緒 continuation; - 回傳根任務結果,包括 move-only 結果;根任務例外會重新拋出; - RunLoop 可以依序重複使用,但巢狀或並行呼叫 `run()` 會拋出 `std::logic_error`; -- `schedule()` 始終暫停,並把 continuation 追加到執行緒安全的 FIFO 就緒佇列; +- `schedule()` 始終暫停,並把 continuation 追加到執行緒安全的 FIFO 就緒佇列;接受 token 的 + 多載可以在消費前取消該排隊等待; - `schedule_after()` 在暫停時測量原生 `steady_clock` 時長,`schedule_at()` 接受絕對的 單調時鐘時間點; -- 接受 token 的定時多載始終暫停;取消獲勝時拋出 `OperationCancelled`,較晚的停止要求 - 不能取代已經進入就緒佇列的期限完成; +- 所有接受 token 的多載仍會排隊,包括預先取消的等待;取消獲勝時拋出 + `OperationCancelled`,較晚的停止要求不能取代已經獲勝的完成; - 已到期的期限仍非同步排隊;未來期限進入最小 Timer 堆,到期只讓 continuation 具備進入 FIFO 排程的資格; -- 停止回呼只改變計時器狀態並喚醒 RunLoop,不會 inline 恢復使用者協程;目前取消透過 O(n) - 掃描定位對應計時器; +- 停止回呼只改變就緒項/計時器狀態並喚醒 RunLoop,不會 inline 恢復使用者協程;目前取消 + 透過 O(n) 掃描定位佇列項; - 其他執行緒可以入列,但只有正在執行 `run()` 的執行緒會消費佇列; - RunLoop 銷毀後繼續使用其 Scheduler,或在其所屬 RunLoop 未運行時使用 Scheduler,皆會 從 await 運算式拋出 `std::logic_error`; @@ -326,9 +346,9 @@ Task;明確等待原 Scheduler 才會把 continuation 送回對應 RunLoop。 執行緒。 目前沒有公開自由函式 `sync_wait`、detached 執行、獨立 Timer 控制代碼、非同步 I/O 後端、 -自訂協程框架 allocator 或阻塞工作執行緒池。取消仍是明確的:定時等待接受 token,TaskGroup -可持有共享 stop 通道,但模組既不提供隱式傳播,也沒有普通 `schedule()` 的 token 多載, -並且沒有保留舊骨架模組的相容別名。 +自訂協程框架 allocator 或阻塞工作執行緒池。取消仍是明確的:Scheduler 等待和 +`AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模組不提供隱式 +傳播,並且沒有保留舊骨架模組的相容別名。 捕捉變數的協程 lambda 需要特別小心:立即呼叫一個暫時的捕捉 lambda,可能使延遲協程參考 已經銷毀的閉包。CMP 尚未提供延長該閉包生命週期的輔助函式。 @@ -341,33 +361,37 @@ CMP 名稱中的 `C` 與 Go 執行期中的 `G` 相呼應,但這只說明命 以下方向可以分別設計和審查,目前都不是套件的既有約定: -1. 遞迴作用域接納、結果控制代碼和更廣泛的取消傳播; -2. 可複用事件、channel 和更多結構化喚醒路徑; +1. TaskGroup 結果控制代碼及更多支援取消的原語; +2. channel 和更多結構化喚醒路徑; 3. 多工作執行緒排程和工作竊取; 4. 非同步 I/O 整合; 5. 處理無法避免之阻塞工作的專用執行緒池; 6. 結果適配器和可選的協程框架配置策略。 -Task、RunLoop、`when_all`、TaskGroup、OneShotEvent 與 AsyncMutex 已形成真實的公開邊界,因此分別位於 -模組分割區中。只有其他已實作 API 確實需要新邊界時,才繼續增加模組分割區或實作單元。 +Task、cancellation、RunLoop、`when_all`、TaskGroup、OneShotEvent、AsyncManualResetEvent 與 +AsyncMutex 已形成真實的公開邊界,因此分別位於模組分割區中。只有其他已實作 API 確實需要 +新邊界時,才繼續增加模組分割區或實作單元。 ## 驗證 在儲存庫根目錄執行: ```text -mcpp build --cache=off -mcpp test --cache=off +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off cd examples/basic mcpp run ``` -預期結果是函式庫建置成功、六個二進位檔中的 74 項測試全部通過,而且範例依序輸出 +預期結果是函式庫建置成功、七個二進位檔中的 90 項測試全部通過,而且範例依序輸出 `Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42`、 -`Event signalled`、`Mutex result: 42` 和 `Coroutine cancelled` 後以狀態 0 結束。測試分別執行一百萬次立即完成 -的 Task、十萬次立即雙 Task 變參匯合、五萬次立即雙 Task vector 匯合、五萬次 eager TaskGroup -完成、五萬次事件等待者、五萬次 mutex 交接、十萬次明確 -排程、十萬次立即 Timer 和十萬次預先取消的定時等待,用於檢查對稱轉移以及所有匯合或佇列 -路徑都不會增長原生呼叫堆疊。目前 Windows LLVM 工具鏈 +`Recursive group result: 3`、`Event signalled`、`Reusable event cycles: 2`、 +`Mutex result: 42` 和 `Coroutine cancelled` 後以狀態 0 結束。測試保留原有高容量堆疊安全 +檢查,並增加兩萬次 TaskGroup 遞迴接納、十萬次預取消就緒排程、五萬個 manual event 等待者、 +兩萬次巢狀可複用事件訊號及 set/cancel 競態;第四階段重點競態套件已連續執行多輪 Release +測試。計算、臨時檔案和回環網路的成功/失敗計數及吞吐記錄在 +[v1 可開發性壓測](benchmarks/2026-08-29-cmp-v1-readiness.md)。目前 Windows LLVM 工具鏈 不會產生 GNU depfile;如果模組介面包含的檔案發生變更,增量建置可能沿用舊的 BMI 或 目的檔。完整複驗時使用 `--cache=off`。 diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index b474f60..6856091 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -6,18 +6,20 @@ CMP 是一个具备小型协程执行核心的 C++23 模块项目。根模块导出懒启动、单消费者的 `mcpplibs::cmp::Task`、结构化变参/vector `when_all()`、eager `TaskGroup`、`RunLoop` 及其 -可复制的 `Scheduler` 句柄、无分配 `OneShotEvent` 和 RAII `AsyncMutex`。汇合原语会持有每个子任务直到结束,一次性 -事件负责发布一个外部信号。`RunLoop::run()` 是公共根任务 -执行边界,`Scheduler::schedule()` 用于显式地把挂起协程送回对应运行循环。 -`schedule_after()` 和 `schedule_at()` 在没有定时线程的情况下提供相对和绝对的 -`steady_clock` 期限;接受 `std::stop_token` 的重载使这些等待可以协作式取消。 +可复制的 `Scheduler` 句柄、无分配 `OneShotEvent`、可复用 `AsyncManualResetEvent` 和 RAII +`AsyncMutex`。汇合原语会持有每个子任务直到结束,事件负责发布外部信号。 +`RunLoop::run()` 是公共根任务执行边界,`Scheduler::schedule()` 用于显式地把挂起协程送回 +对应运行循环。`schedule()`、`schedule_after()` 和 `schedule_at()` 都有接受 +`std::stop_token` 的协作式取消重载;定时调度使用相对和绝对的 `steady_clock` 期限,且不创建 +定时线程。 仓库现有内容包括: - 一份 mcpp 包清单; -- 根模块 `mcpplibs.cmp` 及 Task、RunLoop、join、event、mutex 模块分区; +- 根模块 `mcpplibs.cmp` 及 Task、cancellation、RunLoop、join、event、mutex 模块分区; - 覆盖契约、生命周期、异常、调度和线程行为的 gtest 测试; - 一个通过路径依赖使用根包的独立示例; +- 一个验证计算、文件与回环网络集成的本地 POSIX 压测 consumer; - Linux、macOS 和 Windows 三套 CI 工作流。 ## 包和模块标识 @@ -62,10 +64,12 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp ├── src/ │ ├── cmp.cppm │ ├── task.cppm +│ ├── cancellation.cppm │ ├── run_loop.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm +│ ├── async_manual_reset_event.cppm │ └── async_mutex.cppm ├── tests/ │ ├── cmp_test.cpp @@ -73,14 +77,18 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp +│ ├── async_manual_reset_event_test.cpp │ └── async_mutex_test.cpp +├── benchmarks/v1-readiness/ +│ ├── mcpp.toml +│ └── src/main.cpp └── mcpp.toml ``` ## 构建与测试 `.xlings.json` 固定项目使用的 mcpp 版本。`mcpp build` 构建自动推断的库目标。 -`mcpp test` 发现六个测试文件,并为每个文件链接 gtest 入口。测试同时验证 Task 所有权和 +`mcpp test` 发现七个测试文件,并为每个文件链接 gtest 入口。90 项测试同时验证 Task 所有权和 对称转移、结构化汇合,以及根任务执行、普通与定时调度、异常传播、跨线程期限唤醒、无效 Scheduler、取消竞态、RunLoop 复用和不会增长调用栈的重复完成。 @@ -108,7 +116,7 @@ standard = "c++23" cmp = { path = "../.." } ``` -示例程序使用与外部项目相同的导入路径: +下面的精简 API 示例使用与外部项目相同的导入路径: ```cpp import std; @@ -234,11 +242,11 @@ int main() { ``` 该示例在根测试目标之外,单独检查路径依赖解析、模块使用、外部协程编译和公共根任务驱动器。 -RunLoop 在主线程驱动 `print_answer()`;短单调时钟定时器到期后,协程输出 -`Coroutine result: 42`。下一个根任务并发汇合两个定时结果并输出 `Concurrent result: 42`。 -随后一个协程 eager 启动并汇合两个 void Task,再输出 `Task group result: 42`。另一个协程 -等待一次性信号并输出 `Event signalled`,两个受保护 Task 随后输出 `Mutex result: 42`。最后一个 -协程从预先取消的定时等待捕获 `OperationCancelled`,并输出 `Coroutine cancelled`。 +RunLoop 依次输出 `Coroutine result: 42`、`Concurrent result: 42` 和 `Task group result: 42`; +递归增长的 group 输出 `Recursive group result: 3`。其他协程演示一次性及两轮可复用事件并 +输出 `Event signalled`、`Reusable event cycles: 2`,两个受保护 Task 输出 +`Mutex result: 42`。最后一个结构化 group 使用 `cancel_and_join()`,从可取消就绪调度捕获 +`OperationCancelled` 并输出 `Coroutine cancelled`。 任何定义协程的翻译单元都要自行导入 `std`,使 `std::coroutine_traits` 和标准协程协议类型 参与编译。CMP 模块私有导入 `std`,而不是向使用方重新导出整个标准库。 @@ -272,11 +280,13 @@ RunLoop 在主线程驱动 `print_answer()`;短单调时钟定时器到期后 `TaskGroup` 遵循以下契约: - `spawn(Task)` 接管所有权并在返回前 inline 启动子任务; -- 并发接纳会串行化,等待单次使用的 `join()` 会关闭接纳; +- 并发接纳会串行化;等待单次使用的 `join()` 期间,仍在运行的子任务可以递归接纳工作; +- 活动计数归零时接纳永久关闭;外部接纳与最后一个完成竞态时不保证哪方获胜; - join 恢复前,每个已接纳子任务都必须到达终态; - 全部子任务完成后,按接纳顺序重新抛出第一个异常; - group 不可移动,析构时必须未使用或已经 join,否则终止进程; - `get_stop_token()` 和 `request_stop()` 提供显式标准取消通道,但不会向 Task 注入 token; +- `cancel_and_join()` 会在真正被等待时请求该通道,然后执行相同的 join; - 最后一个子任务在其完成线程恢复 join,不隐式增加调度器亲和。 `OneShotEvent` 遵循以下契约: @@ -289,6 +299,16 @@ RunLoop 在主线程驱动 `print_answer()`;短单调时钟定时器到期后 - 事件不可移动且必须比所有等待者活得更久;带 pending 等待者析构会终止进程; - 不提供 reset、可取消注销、值或隐式 Scheduler 转移。 +`AsyncManualResetEvent` 遵循以下契约: + +- 默认事件为 unset;`set()` 按 FIFO 恢复 pending 等待者,并让后续等待保持 ready; +- `reset()` 只影响未来等待,已经被 set 选中的等待者仍会完成; +- `wait(stop_token)` 以 O(1) 移除一个取消的 pending 等待者,预先取消的等待确定由取消获胜; +- set 与取消竞态恰有一个获胜方,等待者只恢复一次; +- 等待者在 setter 或取消请求线程恢复,不隐式转移 Scheduler; +- thread-local dispatch trampoline 保证嵌套信号链不增长原生调用栈; +- 事件不可移动且必须比所有等待者活得更久;带 pending 等待者析构会终止进程。 + `AsyncMutex` 遵循以下契约: - `lock_async()` 返回无分配操作,等待结果是 move-only RAII Guard; @@ -304,15 +324,16 @@ RunLoop 在主线程驱动 `print_answer()`;短单调时钟定时器到期后 - `run(Task)` 消费一个根 Task,并在调用线程执行就绪 continuation; - 返回根任务结果,包括 move-only 结果;根任务异常会重新抛出; - RunLoop 可以顺序复用,但嵌套或并发调用 `run()` 会抛出 `std::logic_error`; -- `schedule()` 始终挂起,并把 continuation 追加到线程安全的 FIFO 就绪队列; +- `schedule()` 始终挂起,并把 continuation 追加到线程安全的 FIFO 就绪队列;接受 token 的 + 重载可以在消费前取消该排队等待; - `schedule_after()` 在挂起时测量原生 `steady_clock` 时长,`schedule_at()` 接受绝对的 单调时钟时间点; -- 接受 token 的定时重载始终挂起;取消获胜时抛出 `OperationCancelled`,较晚的停止请求 - 不能替换已经进入就绪队列的期限完成; +- 所有接受 token 的重载仍会排队,包括预先取消的等待;取消获胜时抛出 + `OperationCancelled`,较晚的停止请求不能替换已经获胜的完成; - 已到期的期限仍异步排队;未来期限进入最小 Timer 堆,到期只让 continuation 具备进入 FIFO 调度的资格; -- 停止回调只改变定时器状态并唤醒 RunLoop,不会 inline 恢复用户协程;当前取消通过 O(n) - 扫描定位对应定时器; +- 停止回调只改变就绪项/定时器状态并唤醒 RunLoop,不会 inline 恢复用户协程;当前取消通过 + O(n) 扫描定位队列项; - 其他线程可以入队,但只有正在执行 `run()` 的线程会消费队列; - RunLoop 销毁后继续使用其 Scheduler,或在其所属 RunLoop 未运行时使用 Scheduler,都会 从 await 表达式抛出 `std::logic_error`; @@ -324,9 +345,9 @@ RunLoop 不拥有工作线程,也不提供自动线程亲和。外部 awaiter 其他线程或事件源恢复它,`run()` 可能无限等待。阻塞函数仍会阻塞协程当前所在的线程。 目前没有公共自由函数 `sync_wait`、detached 执行、独立 Timer 句柄、异步 I/O 后端、自定义 -协程帧 allocator 或阻塞任务线程池。取消仍是显式的:定时等待接受 token,TaskGroup 可持有 -共享 stop 通道,但模块既不提供隐式传播,也没有普通 `schedule()` 的 token 重载,并且没有 -保留旧脚手架模块的兼容别名。 +协程帧 allocator 或阻塞任务线程池。取消仍是显式的:Scheduler 等待和 +`AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模块不提供隐式传播, +并且没有保留旧脚手架模块的兼容别名。 捕获变量的协程 lambda 需要特别小心:立即调用一个临时的捕获 lambda,可能使懒协程引用 已经销毁的闭包。CMP 尚未提供延长该闭包生命周期的辅助函数。 @@ -338,33 +359,37 @@ CMP 名称中的 `C` 与 Go 运行时中的 `G` 相呼应,但这只说明命 以下方向可以分别设计和评审,目前都不是包的既有约定: -1. 递归作用域接纳、结果句柄和更广泛的取消传播; -2. 可复用事件、channel 和更多结构化唤醒路径; +1. TaskGroup 结果句柄及更多支持取消的原语; +2. channel 和更多结构化唤醒路径; 3. 多工作线程调度和工作窃取; 4. 异步 I/O 集成; 5. 处理不可避免的阻塞工作的专用线程池; 6. 结果适配器和可选的协程帧分配策略。 -Task、RunLoop、`when_all`、TaskGroup、OneShotEvent 与 AsyncMutex 已经形成真实的公共边界,因此分别位于 -模块分区中。只有其他已实现 API 确实需要新边界时,才继续增加模块分区或实现单元。 +Task、cancellation、RunLoop、`when_all`、TaskGroup、OneShotEvent、AsyncManualResetEvent 与 +AsyncMutex 已经形成真实的公共边界,因此分别位于模块分区中。只有其他已实现 API 确实需要 +新边界时,才继续增加模块分区或实现单元。 ## 验证 在仓库根目录执行: ```text -mcpp build --cache=off -mcpp test --cache=off +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off cd examples/basic mcpp run ``` -预期结果是库构建成功、六个二进制中的 74 项测试全部通过,并且示例依次输出 +预期结果是库构建成功、七个二进制中的 90 项测试全部通过,并且示例依次输出 `Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42`、 -`Event signalled`、`Mutex result: 42` 和 `Coroutine cancelled` 后以状态 0 退出。测试分别执行一百万次立即完成 -的 Task、十万次立即双 Task 变参汇合、五万次立即双 Task vector 汇合、五万次 eager TaskGroup -完成、五万次事件等待者、五万次 mutex 交接、十万次显式 -调度、十万次立即 Timer 和十万次预先取消的定时等待,用于检查对称转移以及所有汇合或队列 -路径都不会增长原生调用栈。当前 Windows LLVM 工具链不会 +`Recursive group result: 3`、`Event signalled`、`Reusable event cycles: 2`、 +`Mutex result: 42` 和 `Coroutine cancelled` 后以状态 0 退出。测试保留原有高容量栈安全检查, +并增加两万次 TaskGroup 递归接纳、十万次预取消就绪调度、五万个 manual event 等待者、两万次 +嵌套可复用事件信号及 set/cancel 竞态;第四阶段重点竞态套件已连续执行多轮 Release 测试。 +计算、临时文件和回环网络的成功/失败计数及吞吐记录在 +[v1 可开发性压测](benchmarks/2026-08-29-cmp-v1-readiness.md)。当前 Windows LLVM 工具链不会 生成 GNU depfile;如果模块接口包含的文件发生变化,增量构建可能复用旧的 BMI 或目标文件。 完整复验时使用 `--cache=off`。 diff --git a/docs/benchmarks/2026-08-29-cmp-v1-readiness.md b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md new file mode 100644 index 0000000..453fd9c --- /dev/null +++ b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md @@ -0,0 +1,75 @@ +# CMP v1 可开发性与压力基线 + +**日期:** 2026-08-29 + +**结论:** 本机 5 轮 Release 压测全部通过;三类场景均无非预期失败。 + +## 测试边界 + +本报告验证 CMP v1 的结构化生命周期、调度、跨线程唤醒和失败传播能够承载计算任务, +以及由外部工作线程适配的文件与网络 I/O。CMP v1 尚无原生异步 I/O 后端,因此 I/O 数据 +不是 epoll、io_uring、kqueue 或 IOCP 性能数据,也不代表多核协程调度吞吐。 + +压测程序位于 `benchmarks/v1-readiness`,只使用标准库、CMP 和 POSIX 回环 socket;它不增加 +项目依赖,也不进入跨平台 CI。任何计数不匹配或非预期失败都会让进程以非零状态退出。 + +## 环境与复现 + +- 系统:WSL2 Linux 5.15.153.1,x86_64; +- 处理器:AMD Ryzen 7 7735H,8 核 16 线程; +- 内存:约 11.7 GiB; +- 临时目录文件系统:ext4; +- mcpp:2026.8.11.2; +- 编译器:LLVM 22.1.8; +- 配置:Release,关闭 mcpp 构建缓存。 + +```bash +cd benchmarks/v1-readiness +mcpp build --profile release --strict --cache=off +./target/x86_64-linux-gnu/f0cc001df3795148/bin/cmp-v1-readiness +``` + +目标目录中的平台哈希可能随环境变化;如果路径不同,使用构建输出给出的可执行文件路径。 + +## 负载 + +- `compute`:50,000 个经 `Scheduler` 调度的 Task,每个成功任务执行 128 轮整数运算;每 50 个 + 任务注入一个预期异常并在结构化子任务内捕获。 +- `file_io`:4 个适配线程,各执行 250 次 64 KiB 临时文件写入、读取和内容校验,再执行 + 25 次缺失文件读取以验证预期失败路径。 +- `network_loopback`:4 个适配线程,各执行 5,000 次 256 字节 TCP 回环请求/响应,再执行 + 25 次未监听端口连接,并且只把 `ECONNREFUSED` 计为预期失败。 + +吞吐量按“成功操作 + 预期失败操作”的总操作数计算。 + +## 五轮原始结果 + +| 轮次 | 场景 | 操作数 | 成功 | 预期失败 | 非预期失败 | 耗时 ms | ops/s | 状态 | +| ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | +| 1 | compute | 50,000 | 49,000 | 1,000 | 0 | 30.127 | 1,659,624.3 | PASS | +| 1 | file_io | 1,100 | 1,000 | 100 | 0 | 185.222 | 5,938.8 | PASS | +| 1 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,453.289 | 8,193.1 | PASS | +| 2 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.295 | 1,831,847.8 | PASS | +| 2 | file_io | 1,100 | 1,000 | 100 | 0 | 183.017 | 6,010.4 | PASS | +| 2 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,455.883 | 8,184.4 | PASS | +| 3 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.182 | 1,839,425.5 | PASS | +| 3 | file_io | 1,100 | 1,000 | 100 | 0 | 219.769 | 5,005.2 | PASS | +| 3 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,424.811 | 8,289.3 | PASS | +| 4 | compute | 50,000 | 49,000 | 1,000 | 0 | 28.172 | 1,774,824.8 | PASS | +| 4 | file_io | 1,100 | 1,000 | 100 | 0 | 184.978 | 5,946.7 | PASS | +| 4 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,660.766 | 7,554.2 | PASS | +| 5 | compute | 50,000 | 49,000 | 1,000 | 0 | 30.833 | 1,621,635.6 | PASS | +| 5 | file_io | 1,100 | 1,000 | 100 | 0 | 185.683 | 5,924.1 | PASS | +| 5 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,560.138 | 7,851.1 | PASS | + +## 汇总 + +| 场景 | 最短 / 中位 / 最长耗时 ms | 最高 / 中位 / 最低 ops/s | 五轮非预期失败 | +| --- | ---: | ---: | ---: | +| compute | 27.182 / 28.172 / 30.833 | 1,839,425.5 / 1,774,824.8 / 1,621,635.6 | 0 | +| file_io | 183.017 / 185.222 / 219.769 | 6,010.4 / 5,938.8 / 5,005.2 | 0 | +| network_loopback | 2,424.811 / 2,455.883 / 2,660.766 | 8,289.3 / 8,184.4 / 7,554.2 | 0 | + +这些结果说明 v1 核心可以用于有明确所有权的协程编排、定时、显式取消、可复用通知,以及 +通过外部适配线程接入本地文件和网络工作。它们是当前机器上的回归基线,不是跨机器 SLA。 +原生异步 I/O、多 worker 调度、work stealing 和 blocking pool 仍属于后续阶段。 diff --git a/docs/superpowers/plans/2026-08-29-cmp-phase4-v1-completion.md b/docs/superpowers/plans/2026-08-29-cmp-phase4-v1-completion.md new file mode 100644 index 0000000..9693314 --- /dev/null +++ b/docs/superpowers/plans/2026-08-29-cmp-phase4-v1-completion.md @@ -0,0 +1,36 @@ +# CMP Phase 4 v1 Completion Plan + +**Date:** 2026-08-29 +**Design:** `docs/superpowers/specs/2026-08-29-cmp-phase4-v1-completion-design.md` +**Status:** Complete + +## 1. Preserve the verified baseline + +**Complete.** mcpp 2026.8.11.2 resolved LLVM 22.1.8 and all existing 74 tests passed with cache +disabled before implementation. + +## 2. Complete phase 4 contracts + +**Complete.** RunLoop now shares cancellation state between ready and timed waits and exposes +`schedule(stop_token)`. TaskGroup admits recursive work until quiescence and provides lazy +`cancel_and_join()`. The root exports the focused cancellation and reusable-event partitions. + +## 3. Add contract and failure tests + +**Complete.** TaskGroup and RunLoop success, rejection, cancellation, race, lifecycle, and stack +boundaries are covered. The manual-reset-event suite covers reuse, FIFO, publication, cancellation, +thread affinity, 1,000-way races, 50,000 waiters, and 20,000 nested signals. The complete Dev and +Release strict suites contain 90 passing tests across seven binaries. + +## 4. Add representative v1 readiness pressure tests + +**Complete.** The standalone Release consumer uses no new package dependency and fails its process +on any count mismatch. Five measured runs of compute scheduling, temporary-file adapter work, and +loopback TCP adapter work all passed with zero unexpected failures. Exact data and scope are in +`docs/benchmarks/2026-08-29-cmp-v1-readiness.md`. + +## 5. Synchronize consumer facts + +**Complete.** The basic consumer, three README variants, three architecture variants, benchmark +report, and HANDOFF describe the implemented API, reproducible validation, I/O-adapter boundary, +and remaining phase-5/6 work. No Git or GitHub operation was performed without separate authority. diff --git a/docs/superpowers/specs/2026-08-29-cmp-phase4-v1-completion-design.md b/docs/superpowers/specs/2026-08-29-cmp-phase4-v1-completion-design.md new file mode 100644 index 0000000..4bc1419 --- /dev/null +++ b/docs/superpowers/specs/2026-08-29-cmp-phase4-v1-completion-design.md @@ -0,0 +1,118 @@ +# CMP Phase 4 v1 Completion Design + +**Date:** 2026-08-29 +**Status:** Implemented and locally verified +**Baseline:** 74/74 tests pass with mcpp 2026.8.11.2 and LLVM 22.1.8 + +## Purpose + +Finish the three capabilities that the public roadmap leaves in phase 4: + +1. recursive TaskGroup admission while a live child keeps the scope open; +2. explicit cancellation beyond timed waits; +3. one reusable, structured wake-up primitive. + +This phase does not implement a multi-worker scheduler, an operating-system I/O backend, detached +ownership, or a blocking pool. Network and file readiness are measured through external adapters, +which is the interoperability boundary supported by v1. + +## TaskGroup Quiescent Join + +`TaskGroup::join()` still starts only when awaited and still succeeds once. Starting join no longer +closes admission immediately. A child already owned by the group may call `spawn()` while join is +waiting. The new child increments the same active counter before it starts, so join observes the +transitive closure of recursively spawned work. + +The group becomes permanently joined when the active count reaches zero under the state mutex. +Admission after that point fails with `std::logic_error`. External concurrent admission racing the +last completion is deliberately not promised: callers may only rely on recursive admission from a +currently active child, matching the lifetime rule used by established async scopes. + +`cancel_and_join()` is a lazy convenience Task. When awaited, it requests the existing standard +stop source and then performs the same single join. Cancellation remains cooperative and children +receive the token explicitly through `get_stop_token()`. + +## General Scheduler Cancellation + +Add `Scheduler::schedule(std::stop_token)`. Like every Scheduler operation it always queues before +resuming, including a pre-cancelled token. If cancellation wins before the ready entry is consumed, +`await_resume()` throws `OperationCancelled`; once consumption wins, a later stop request cannot +replace successful scheduling. + +The RunLoop ready queue stores a continuation plus an optional cancellation-state pointer. Timer +and ordinary scheduling cancellation share the existing stop-callback protocol and one state lock. +Cancellation lookup remains O(n); a different data structure requires benchmark evidence. + +Move `OperationCancelled` into a small `:cancellation` partition so reusable primitives can depend +on the exception without depending on RunLoop. + +## AsyncManualResetEvent + +Add an immovable `AsyncManualResetEvent` with this public surface: + +```cpp +class AsyncManualResetEvent final { +public: + explicit AsyncManualResetEvent(bool initiallySet = false) noexcept; + + [[nodiscard]] bool is_set() const noexcept; + void set() noexcept; + void reset() noexcept; + + [[nodiscard]] WaitOperation wait(std::stop_token token = {}) noexcept; + [[nodiscard]] WaitOperation operator co_await() noexcept; +}; +``` + +An unset wait registers an intrusive coroutine-frame node in FIFO order. `set()` publishes prior +writes, marks the event set, and resumes every registered waiter exactly once on the setter thread. +Later waits continue without suspension until `reset()`. Reset affects only future waits; waiters +selected by an earlier set still complete. + +`wait(token)` uses a doubly linked queue so cancellation removes one pending waiter in O(1). +Registration, set, reset, and cancellation serialize under one short standard mutex. Set versus +cancellation has exactly one winner. A no-allocation thread-local dispatch trampoline flattens +nested wake-up chains. + +Destroying the event with pending waiters terminates. Awaiting frames must remain alive until set or +cancellation completes. CMP does not add scheduler affinity: set and cancellation resume inline, +and a coroutine explicitly awaits its Scheduler to return to a RunLoop. + +## Validation Contract + +Functional and race tests cover: + +- recursive admission before and during join, quiescent closure, exception order, and cancellation; +- pre-cancelled, queued-cancelled, and late-cancelled ordinary scheduling; +- reusable set/reset cycles, multiple waiters, publication, cancellation races, cross-thread wake, + FIFO dispatch, and native-stack safety; +- all pre-existing public contracts. + +A standalone Release benchmark records operation count, elapsed time, throughput, successes, and +failures for computation, temporary-file I/O, and loopback network I/O. I/O work runs in adapter +threads and resumes structured CMP tasks; results therefore validate integration and lifetime +safety, not native non-blocking I/O performance. + +## Deliberately Excluded + +- implicit cancellation hidden inside Task promises; +- cancellable AsyncMutex acquisition or mutation of OneShotEvent semantics; +- result handles, detached work, multi-worker scheduling, work stealing, I/O polling, or a pool; +- third-party dependencies or a benchmark framework. + +## Reference Direction + +- Folly `AsyncScope` permits adding work during join only when the caller is known to be inside + already-added work and uses a barrier sentinel for quiescence. +- Folly `CancellableAsyncScope` documents cooperative token propagation and cancel-then-join. +- cppcoro `async_manual_reset_event` and Folly `Baton` use a set sentinel plus waiter collection; + CMP keeps the observable reusable-event contract but chooses a standard mutex for cancellable + O(1) removal and direct race auditing. + +## Implementation Result + +The three contracts are implemented without new dependencies. Dev and Release strict builds pass, +the complete suite contains 90 passing tests across seven binaries, and focused cancellation, +recursive-admission, and reusable-event races pass repeated Release runs. The standalone example +exercises every new public path. Compute, file-adapter, and loopback-adapter counts are recorded in +`docs/benchmarks/2026-08-29-cmp-v1-readiness.md`; every measured run had zero unexpected failures. diff --git a/examples/basic/src/main.cpp b/examples/basic/src/main.cpp index 93a986d..919dbcf 100644 --- a/examples/basic/src/main.cpp +++ b/examples/basic/src/main.cpp @@ -3,6 +3,7 @@ import mcpplibs.cmp; using mcpplibs::cmp::Task; using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::AsyncManualResetEvent; using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; @@ -59,6 +60,33 @@ Task print_task_group(RunLoop::Scheduler scheduler) { co_return; } +Task add_recursively( + RunLoop::Scheduler scheduler, + TaskGroup& group, + int remaining, + int& total) { + co_await scheduler.schedule(); + ++total; + + if (remaining > 1) { + group.spawn(add_recursively( + scheduler, + group, + remaining - 1, + total)); + } + co_return; +} + +Task print_recursive_group(RunLoop::Scheduler scheduler) { + int total { 0 }; + TaskGroup group {}; + group.spawn(add_recursively(scheduler, group, 3, total)); + co_await group.join(); + std::println("Recursive group result: {}", total); + co_return; +} + Task set_event( RunLoop::Scheduler scheduler, OneShotEvent& event) { @@ -77,6 +105,29 @@ Task print_event(RunLoop::Scheduler scheduler) { co_return; } +Task set_manual_event( + RunLoop::Scheduler scheduler, + AsyncManualResetEvent& event) { + co_await scheduler.schedule(); + event.set(); + co_return; +} + +Task print_manual_reset_event(RunLoop::Scheduler scheduler) { + AsyncManualResetEvent event {}; + + for (int cycle { 0 }; cycle < 2; ++cycle) { + TaskGroup group {}; + group.spawn(set_manual_event(scheduler, event)); + co_await event; + co_await group.join(); + event.reset(); + } + + std::println("Reusable event cycles: 2"); + co_return; +} + Task add_locked( RunLoop::Scheduler scheduler, AsyncMutex& mutex, @@ -99,10 +150,28 @@ Task print_mutex(RunLoop::Scheduler scheduler) { co_return; } -Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token token) { +Task observe_group_cancellation( + RunLoop::Scheduler scheduler, + std::stop_token token, + bool& cancelled) { try { - co_await scheduler.schedule_after(1s, token); + co_await scheduler.schedule(token); } catch (const OperationCancelled&) { + cancelled = true; + } + co_return; +} + +Task print_cancellation(RunLoop::Scheduler scheduler) { + bool cancelled { false }; + TaskGroup group {}; + group.spawn(observe_group_cancellation( + scheduler, + group.get_stop_token(), + cancelled)); + co_await group.cancel_and_join(); + + if (cancelled) { std::println("Coroutine cancelled"); } co_return; @@ -113,11 +182,10 @@ int main() { loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); + loop.run(print_recursive_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); + loop.run(print_manual_reset_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); - - std::stop_source source {}; - source.request_stop(); - loop.run(print_cancellation(loop.get_scheduler(), source.get_token())); + loop.run(print_cancellation(loop.get_scheduler())); return 0; } diff --git a/src/async_manual_reset_event.cppm b/src/async_manual_reset_event.cppm new file mode 100644 index 0000000..62e84da --- /dev/null +++ b/src/async_manual_reset_event.cppm @@ -0,0 +1,263 @@ +export module mcpplibs.cmp:async_manual_reset_event; + +import std; +import :cancellation; + +export namespace mcpplibs::cmp { + +class AsyncManualResetEvent final { +private: + enum class WaitOutcome { + pending, + signalled, + cancelled + }; + +public: + class WaitOperation final { + private: + friend class AsyncManualResetEvent; + + struct CancelCallback final { + WaitOperation* operation_ {}; + + void operator()() const noexcept { + operation_->request_cancel_(); + } + }; + + using StopCallback = std::stop_callback; + + AsyncManualResetEvent* event_ {}; + WaitOperation* previous_ {}; + WaitOperation* next_ {}; + WaitOperation* dispatchNext_ {}; + std::coroutine_handle<> continuation_ {}; + std::stop_token stopToken_ {}; + std::atomic stopRequested_ { false }; + WaitOutcome outcome_ { WaitOutcome::pending }; + bool queued_ { false }; + // 最先析构,阻止回调继续访问当前协程帧。 + std::optional stopCallback_ {}; + + WaitOperation( + AsyncManualResetEvent& event, + std::stop_token stopToken) noexcept + : event_ { &event }, + stopToken_ { std::move(stopToken) } {} + + void request_cancel_() noexcept { + auto* const event = event_; + stopRequested_.store(true, std::memory_order_release); + event->cancel_waiter_(*this); + } + + public: + WaitOperation(const WaitOperation&) = delete; + WaitOperation& operator=(const WaitOperation&) = delete; + WaitOperation(WaitOperation&&) = delete; + WaitOperation& operator=(WaitOperation&&) = delete; + + [[nodiscard]] constexpr bool await_ready() const noexcept { + return false; + } + + [[nodiscard]] bool await_suspend( + std::coroutine_handle<> continuation) { + auto* const event = event_; + continuation_ = continuation; + + if (stopToken_.stop_possible()) { + stopCallback_.emplace( + stopToken_, + CancelCallback { this }); + } + + // 发布后 set() 或取消可能立即恢复并销毁当前 operation。 + return event->register_waiter_(*this); + } + + void await_resume() { + stopCallback_.reset(); + + if (outcome_ == WaitOutcome::cancelled) { + throw OperationCancelled {}; + } + } + }; + +private: + // ponytail: 标准状态锁换取可审查的 reset/cancel 竞态;实测瓶颈后再考虑原子代际状态。 + mutable std::mutex mutex_ {}; + bool set_ { false }; + WaitOperation* head_ {}; + WaitOperation* tail_ {}; + + inline static thread_local bool dispatching_ { false }; + inline static thread_local WaitOperation* dispatchHead_ {}; + inline static thread_local WaitOperation* dispatchTail_ {}; + + [[nodiscard]] bool register_waiter_(WaitOperation& operation) noexcept; + void cancel_waiter_(WaitOperation& operation) noexcept; + static void dispatch_(WaitOperation* operations) noexcept; + +public: + explicit AsyncManualResetEvent(bool initiallySet = false) noexcept + : set_ { initiallySet } {} + + AsyncManualResetEvent(const AsyncManualResetEvent&) = delete; + AsyncManualResetEvent& operator=(const AsyncManualResetEvent&) = delete; + AsyncManualResetEvent(AsyncManualResetEvent&&) = delete; + AsyncManualResetEvent& operator=(AsyncManualResetEvent&&) = delete; + ~AsyncManualResetEvent(); + + [[nodiscard]] bool is_set() const noexcept { + const std::lock_guard lock { mutex_ }; + return set_; + } + + void set() noexcept; + void reset() noexcept; + + [[nodiscard]] WaitOperation wait( + std::stop_token stopToken = {}) noexcept { + return WaitOperation { *this, std::move(stopToken) }; + } + + [[nodiscard]] WaitOperation operator co_await() noexcept { + return wait(); + } +}; + +inline AsyncManualResetEvent::~AsyncManualResetEvent() { + const std::lock_guard lock { mutex_ }; + + if (head_ != nullptr || tail_ != nullptr) { + std::terminate(); + } +} + +inline bool AsyncManualResetEvent::register_waiter_( + WaitOperation& operation) noexcept { + const std::lock_guard lock { mutex_ }; + + if (operation.stopRequested_.load(std::memory_order_acquire)) { + operation.outcome_ = WaitOutcome::cancelled; + return false; + } + + if (set_) { + operation.outcome_ = WaitOutcome::signalled; + return false; + } + + operation.previous_ = tail_; + operation.next_ = nullptr; + operation.queued_ = true; + + if (tail_) { + tail_->next_ = &operation; + } else { + head_ = &operation; + } + + tail_ = &operation; + return true; +} + +inline void AsyncManualResetEvent::cancel_waiter_( + WaitOperation& operation) noexcept { + { + const std::lock_guard lock { mutex_ }; + + if (!operation.queued_ || + operation.outcome_ != WaitOutcome::pending) { + return; + } + + if (operation.previous_) { + operation.previous_->next_ = operation.next_; + } else { + head_ = operation.next_; + } + + if (operation.next_) { + operation.next_->previous_ = operation.previous_; + } else { + tail_ = operation.previous_; + } + + operation.previous_ = nullptr; + operation.next_ = nullptr; + operation.queued_ = false; + operation.outcome_ = WaitOutcome::cancelled; + } + + dispatch_(&operation); +} + +inline void AsyncManualResetEvent::dispatch_( + WaitOperation* operations) noexcept { + while (operations) { + auto* const current = operations; + operations = current->next_; + current->previous_ = nullptr; + current->next_ = nullptr; + current->dispatchNext_ = nullptr; + + if (dispatchTail_) { + dispatchTail_->dispatchNext_ = current; + } else { + dispatchHead_ = current; + } + dispatchTail_ = current; + } + + if (dispatching_) { + return; + } + + dispatching_ = true; + while (dispatchHead_) { + auto* const current = dispatchHead_; + dispatchHead_ = current->dispatchNext_; + if (!dispatchHead_) { + dispatchTail_ = nullptr; + } + + // 恢复后 operation 可能销毁,循环不再访问 current。 + const auto continuation = current->continuation_; + continuation.resume(); + } + dispatching_ = false; +} + +inline void AsyncManualResetEvent::set() noexcept { + WaitOperation* waiters {}; + + { + const std::lock_guard lock { mutex_ }; + + if (set_) { + return; + } + + set_ = true; + waiters = std::exchange(head_, nullptr); + tail_ = nullptr; + + for (auto* waiter = waiters; waiter; waiter = waiter->next_) { + waiter->queued_ = false; + waiter->outcome_ = WaitOutcome::signalled; + } + } + + dispatch_(waiters); +} + +inline void AsyncManualResetEvent::reset() noexcept { + const std::lock_guard lock { mutex_ }; + set_ = false; +} + +} // namespace mcpplibs::cmp diff --git a/src/cancellation.cppm b/src/cancellation.cppm new file mode 100644 index 0000000..db4e075 --- /dev/null +++ b/src/cancellation.cppm @@ -0,0 +1,14 @@ +export module mcpplibs.cmp:cancellation; + +import std; + +export namespace mcpplibs::cmp { + +class OperationCancelled final : public std::exception { +public: + [[nodiscard]] const char* what() const noexcept override { + return "operation cancelled"; + } +}; + +} // namespace mcpplibs::cmp diff --git a/src/cmp.cppm b/src/cmp.cppm index afc931e..1bf4292 100644 --- a/src/cmp.cppm +++ b/src/cmp.cppm @@ -1,8 +1,10 @@ export module mcpplibs.cmp; +export import :cancellation; export import :task; export import :run_loop; export import :when_all; export import :task_group; export import :one_shot_event; +export import :async_manual_reset_event; export import :async_mutex; diff --git a/src/run_loop.cppm b/src/run_loop.cppm index a0e27e5..c1810f5 100644 --- a/src/run_loop.cppm +++ b/src/run_loop.cppm @@ -1,6 +1,7 @@ export module mcpplibs.cmp:run_loop; import std; +import :cancellation; import :task; namespace mcpplibs::cmp::detail { @@ -20,30 +21,35 @@ using RunLoopClock = std::chrono::steady_clock; : now + delay; } -enum class TimedWaitOutcome { +enum class WaitOutcome { pending, - deadline, + completed, cancelled }; -struct TimedCancellationState final { +struct CancellationState final { std::atomic stopRequested_ { false }; - TimedWaitOutcome outcome_ { TimedWaitOutcome::pending }; + WaitOutcome outcome_ { WaitOutcome::pending }; }; class RunLoopState; -struct TimedCancelCallback final { +struct CancelCallback final { std::weak_ptr state_ {}; - TimedCancellationState* cancellation_ {}; + CancellationState* cancellation_ {}; void operator()() const noexcept; }; +struct ReadyEntry final { + std::coroutine_handle<> continuation_ {}; + CancellationState* cancellation_ {}; +}; + struct TimerEntry final { RunLoopClock::time_point deadline_ {}; std::coroutine_handle<> continuation_ {}; - TimedCancellationState* cancellation_ {}; + CancellationState* cancellation_ {}; }; struct TimerEntryLater final { @@ -71,7 +77,7 @@ class RunLoopState final { private: std::mutex mutex_ {}; std::condition_variable condition_ {}; - std::deque> ready_ {}; + std::deque ready_ {}; // ponytail: v1 取消为 O(n) 重排;测得瓶颈后再换可删除堆。 std::vector timers_ {}; bool running_ { false }; @@ -101,7 +107,9 @@ public: running_ = true; } - void enqueue(std::coroutine_handle<> coroutine) { + void enqueue( + std::coroutine_handle<> coroutine, + CancellationState* cancellation = nullptr) { if (!coroutine) { throw std::invalid_argument { "cannot schedule an empty coroutine" }; } @@ -113,7 +121,12 @@ public: throw std::logic_error { "scheduler has no active run" }; } - ready_.push_back(coroutine); + if (cancellation && + cancellation->stopRequested_.load(std::memory_order_acquire)) { + cancellation->outcome_ = WaitOutcome::cancelled; + } + + ready_.push_back(ReadyEntry { coroutine, cancellation }); } condition_.notify_one(); @@ -128,7 +141,7 @@ public: void enqueue_at( RunLoopClock::time_point deadline, std::coroutine_handle<> coroutine, - TimedCancellationState* cancellation = nullptr) { + CancellationState* cancellation = nullptr) { if (!coroutine) { throw std::invalid_argument { "cannot schedule an empty coroutine" }; } @@ -142,15 +155,16 @@ public: throw std::logic_error { "scheduler has no active run" }; } - if (cancellation && cancellation->stopRequested_.load()) { - ready_.push_back(coroutine); - cancellation->outcome_ = TimedWaitOutcome::cancelled; + if (cancellation && cancellation->stopRequested_.load( + std::memory_order_acquire)) { + ready_.push_back(ReadyEntry { coroutine, cancellation }); + cancellation->outcome_ = WaitOutcome::cancelled; shouldNotify = true; } else if (deadline <= RunLoopClock::now()) { - ready_.push_back(coroutine); + ready_.push_back(ReadyEntry { coroutine, cancellation }); if (cancellation) { - cancellation->outcome_ = TimedWaitOutcome::deadline; + cancellation->outcome_ = WaitOutcome::completed; } shouldNotify = true; @@ -170,27 +184,41 @@ public: } } - void request_timer_cancellation( - TimedCancellationState& cancellation) noexcept { + void request_cancellation(CancellationState& cancellation) noexcept { bool shouldNotify { false }; { const std::lock_guard lock { mutex_ }; - const auto timer = std::ranges::find_if( - timers_, - [&](const TimerEntry& entry) { - return entry.cancellation_ == &cancellation; - }); - if (timer == timers_.end() || - cancellation.outcome_ != TimedWaitOutcome::pending) { + if (cancellation.outcome_ != WaitOutcome::pending) { return; } - cancellation.outcome_ = TimedWaitOutcome::cancelled; - timer->deadline_ = RunLoopClock::time_point::min(); - std::ranges::make_heap(timers_, TimerEntryLater {}); - shouldNotify = true; + const auto ready = std::ranges::find_if( + ready_, + [&](const ReadyEntry& entry) { + return entry.cancellation_ == &cancellation; + }); + + if (ready != ready_.end()) { + cancellation.outcome_ = WaitOutcome::cancelled; + shouldNotify = true; + } else { + const auto timer = std::ranges::find_if( + timers_, + [&](const TimerEntry& entry) { + return entry.cancellation_ == &cancellation; + }); + + if (timer == timers_.end()) { + return; + } + + cancellation.outcome_ = WaitOutcome::cancelled; + timer->deadline_ = RunLoopClock::time_point::min(); + std::ranges::make_heap(timers_, TimerEntryLater {}); + shouldNotify = true; + } } if (shouldNotify) { @@ -231,21 +259,36 @@ public: timers_.front().deadline_ <= RunLoopClock::now()) { // 先入 FIFO 再出队,避免 ready 或 Timer 任一侧长期饥饿。 auto& timer = timers_.front(); - ready_.push_back(timer.continuation_); + ready_.push_back(ReadyEntry { + timer.continuation_, + timer.cancellation_ + }); if (timer.cancellation_ && timer.cancellation_->outcome_ == - TimedWaitOutcome::pending) { + WaitOutcome::pending) { timer.cancellation_->outcome_ = - TimedWaitOutcome::deadline; + WaitOutcome::completed; } pop_timer_(); } if (!ready_.empty()) { - coroutine = ready_.front(); + const auto ready = ready_.front(); ready_.pop_front(); + + if (ready.cancellation_ && + ready.cancellation_->outcome_ == + WaitOutcome::pending) { + ready.cancellation_->outcome_ = + ready.cancellation_->stopRequested_.load( + std::memory_order_acquire) + ? WaitOutcome::cancelled + : WaitOutcome::completed; + } + + coroutine = ready.continuation_; break; } @@ -276,11 +319,11 @@ public: } }; -inline void TimedCancelCallback::operator()() const noexcept { - cancellation_->stopRequested_.store(true); +inline void CancelCallback::operator()() const noexcept { + cancellation_->stopRequested_.store(true, std::memory_order_release); if (const auto state = state_.lock()) { - state->request_timer_cancellation(*cancellation_); + state->request_cancellation(*cancellation_); } } @@ -402,13 +445,6 @@ RootOperation make_root_operation(Task task, RootCompletion& completion) { export namespace mcpplibs::cmp { -class OperationCancelled final : public std::exception { -public: - [[nodiscard]] const char* what() const noexcept override { - return "operation cancelled"; - } -}; - class RunLoop final { public: class Scheduler final { @@ -453,6 +489,51 @@ public: constexpr void await_resume() const noexcept {} }; + class CancellableScheduleAwaiter final { + private: + using StopCallback = std::stop_callback; + + std::weak_ptr state_ {}; + std::stop_token stopToken_ {}; + detail::CancellationState cancellation_ {}; + // 最先析构,阻止回调继续访问 awaiter 内状态。 + std::optional stopCallback_ {}; + + public: + CancellableScheduleAwaiter( + std::weak_ptr state, + std::stop_token stopToken) noexcept + : state_ { std::move(state) }, + stopToken_ { std::move(stopToken) } {} + + [[nodiscard]] constexpr bool await_ready() const noexcept { + return false; + } + + void await_suspend(std::coroutine_handle<> continuation) { + const auto state = Scheduler::lock_state_(state_); + + stopCallback_.emplace( + stopToken_, + detail::CancelCallback { + state, + &cancellation_ + }); + + // 发布后 awaiter 可能立即销毁,此后不得再读取成员。 + state->enqueue(continuation, &cancellation_); + } + + void await_resume() { + stopCallback_.reset(); + + if (cancellation_.outcome_ == + detail::WaitOutcome::cancelled) { + throw OperationCancelled {}; + } + } + }; + class ScheduleAfterAwaiter final { private: std::weak_ptr state_ {}; @@ -504,12 +585,12 @@ public: class CancellableScheduleAfterAwaiter final { private: - using StopCallback = std::stop_callback; + using StopCallback = std::stop_callback; std::weak_ptr state_ {}; Duration delay_ {}; std::stop_token stopToken_ {}; - detail::TimedCancellationState cancellation_ {}; + detail::CancellationState cancellation_ {}; // 必须最先析构,阻止回调继续访问 awaiter 内状态。 std::optional stopCallback_ {}; @@ -532,7 +613,7 @@ public: stopCallback_.emplace( stopToken_, - detail::TimedCancelCallback { + detail::CancelCallback { state, &cancellation_ }); @@ -548,7 +629,7 @@ public: stopCallback_.reset(); if (cancellation_.outcome_ == - detail::TimedWaitOutcome::cancelled) { + detail::WaitOutcome::cancelled) { throw OperationCancelled {}; } } @@ -556,12 +637,12 @@ public: class CancellableScheduleAtAwaiter final { private: - using StopCallback = std::stop_callback; + using StopCallback = std::stop_callback; std::weak_ptr state_ {}; TimePoint deadline_ {}; std::stop_token stopToken_ {}; - detail::TimedCancellationState cancellation_ {}; + detail::CancellationState cancellation_ {}; // 必须最先析构,阻止回调继续访问 awaiter 内状态。 std::optional stopCallback_ {}; @@ -584,7 +665,7 @@ public: stopCallback_.emplace( stopToken_, - detail::TimedCancelCallback { + detail::CancelCallback { state, &cancellation_ }); @@ -600,7 +681,7 @@ public: stopCallback_.reset(); if (cancellation_.outcome_ == - detail::TimedWaitOutcome::cancelled) { + detail::WaitOutcome::cancelled) { throw OperationCancelled {}; } } @@ -626,6 +707,14 @@ public: return ScheduleAwaiter { state_ }; } + [[nodiscard]] auto schedule( + std::stop_token stopToken) const noexcept { + return CancellableScheduleAwaiter { + state_, + std::move(stopToken) + }; + } + [[nodiscard]] auto schedule_after(Duration delay) const noexcept { return ScheduleAfterAwaiter { state_, delay }; } diff --git a/src/task_group.cppm b/src/task_group.cppm index 0c68aa8..c3fbe99 100644 --- a/src/task_group.cppm +++ b/src/task_group.cppm @@ -139,6 +139,8 @@ public: [[nodiscard]] Task join(); + [[nodiscard]] Task cancel_and_join(); + [[nodiscard]] std::stop_token get_stop_token() const noexcept { return stopSource_.get_token(); } @@ -182,13 +184,15 @@ inline void TaskGroup::spawn(Task task) { { const std::lock_guard lock { mutex_ }; - if (state_ != State::unused && state_ != State::open) { + if (state_ == State::joined) { throw std::logic_error { "task group is closed" }; } tasks_.push_back(std::move(child)); ++active_; - state_ = State::open; + if (state_ == State::unused) { + state_ = State::open; + } } // 锁外启动,立即完成的子任务会回调当前 group。 @@ -254,4 +258,9 @@ inline Task TaskGroup::join() { co_await JoinAwaiter { *this }; } +inline Task TaskGroup::cancel_and_join() { + request_stop(); + co_await join(); +} + } // namespace mcpplibs::cmp diff --git a/tests/async_manual_reset_event_test.cpp b/tests/async_manual_reset_event_test.cpp new file mode 100644 index 0000000..5ed918c --- /dev/null +++ b/tests/async_manual_reset_event_test.cpp @@ -0,0 +1,336 @@ +#include + +import std; +import mcpplibs.cmp; + +namespace { + +using mcpplibs::cmp::AsyncManualResetEvent; +using mcpplibs::cmp::OperationCancelled; +using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::Task; +using mcpplibs::cmp::TaskGroup; + +Task increment_after_event( + AsyncManualResetEvent& event, + int& completed) { + co_await event; + ++completed; +} + +Task record_after_event( + AsyncManualResetEvent& event, + int value, + std::vector& order) { + co_await event; + order.push_back(value); +} + +Task observe_published_value( + AsyncManualResetEvent& event, + const int& published, + int& observed, + std::thread::id& resumedThread) { + co_await event; + observed = published; + resumedThread = std::this_thread::get_id(); +} + +Task wait_with_cancellation( + AsyncManualResetEvent& event, + std::stop_token stopToken, + bool& signalled, + bool& cancelled, + int& resumeCount, + std::thread::id& resumedThread) { + try { + co_await event.wait(stopToken); + signalled = true; + } catch (const OperationCancelled&) { + cancelled = true; + } + + ++resumeCount; + resumedThread = std::this_thread::get_id(); +} + +Task signal_next( + AsyncManualResetEvent& current, + AsyncManualResetEvent& next, + int& completed) { + co_await current; + ++completed; + next.set(); +} + +Task finish_chain( + AsyncManualResetEvent& event, + int& completed) { + co_await event; + ++completed; +} + +Task run_signal_chain(int count) { + auto events = std::make_unique(count); + int completed { 0 }; + TaskGroup group {}; + + for (int index { 0 }; index + 1 < count; ++index) { + group.spawn(signal_next( + events[index], + events[index + 1], + completed)); + } + group.spawn(finish_chain(events[count - 1], completed)); + + events[0].set(); + co_await group.join(); + co_return completed; +} + +Task run_many_waiters(int count) { + AsyncManualResetEvent event {}; + int completed { 0 }; + TaskGroup group {}; + + for (int index { 0 }; index < count; ++index) { + group.spawn(increment_after_event(event, completed)); + } + + event.set(); + co_await group.join(); + co_return completed; +} + +TEST(CmpAsyncManualResetEventTest, SupportsInitialStateAndReset) { + static_assert(!std::copy_constructible); + static_assert(!std::move_constructible); + + AsyncManualResetEvent unset {}; + AsyncManualResetEvent set { true }; + + EXPECT_FALSE(unset.is_set()); + EXPECT_TRUE(set.is_set()); + + unset.reset(); + EXPECT_FALSE(unset.is_set()); + unset.set(); + EXPECT_TRUE(unset.is_set()); + unset.set(); + EXPECT_TRUE(unset.is_set()); + unset.reset(); + EXPECT_FALSE(unset.is_set()); +} + +TEST(CmpAsyncManualResetEventTest, SetStateContinuesInlineUntilReset) { + RunLoop loop {}; + AsyncManualResetEvent event { true }; + int completed { 0 }; + + loop.run(increment_after_event(event, completed)); + loop.run(increment_after_event(event, completed)); + EXPECT_EQ(completed, 2); + + event.reset(); + TaskGroup group {}; + group.spawn(increment_after_event(event, completed)); + EXPECT_EQ(completed, 2); + + event.set(); + loop.run(group.join()); + EXPECT_EQ(completed, 3); +} + +TEST(CmpAsyncManualResetEventTest, SetPublishesAndResumesOnSetterThread) { + RunLoop loop {}; + AsyncManualResetEvent event {}; + TaskGroup group {}; + int published { 0 }; + int observed { 0 }; + std::thread::id resumedThread {}; + std::thread::id setterThread {}; + + group.spawn(observe_published_value( + event, + published, + observed, + resumedThread)); + + std::jthread setter { [&] { + setterThread = std::this_thread::get_id(); + published = 42; + event.set(); + } }; + setter.join(); + loop.run(group.join()); + + EXPECT_EQ(observed, 42); + EXPECT_EQ(resumedThread, setterThread); +} + +TEST(CmpAsyncManualResetEventTest, SetResumesWaitersInFifoOrder) { + RunLoop loop {}; + AsyncManualResetEvent event {}; + TaskGroup group {}; + std::vector order {}; + + group.spawn(record_after_event(event, 1, order)); + group.spawn(record_after_event(event, 2, order)); + group.spawn(record_after_event(event, 3, order)); + + event.set(); + loop.run(group.join()); + + EXPECT_EQ(order, (std::vector { 1, 2, 3 })); +} + +TEST(CmpAsyncManualResetEventTest, PreCancelledWaitWinsDeterministically) { + RunLoop loop {}; + AsyncManualResetEvent event { true }; + std::stop_source stopSource {}; + bool signalled { false }; + bool cancelled { false }; + int resumeCount { 0 }; + std::thread::id resumedThread {}; + stopSource.request_stop(); + + loop.run(wait_with_cancellation( + event, + stopSource.get_token(), + signalled, + cancelled, + resumeCount, + resumedThread)); + + EXPECT_FALSE(signalled); + EXPECT_TRUE(cancelled); + EXPECT_EQ(resumeCount, 1); + EXPECT_EQ(resumedThread, std::this_thread::get_id()); +} + +TEST(CmpAsyncManualResetEventTest, PendingCancellationRemovesOnlyItsWaiter) { + RunLoop loop {}; + AsyncManualResetEvent event {}; + TaskGroup group {}; + std::stop_source stopSource {}; + std::vector order {}; + bool signalled { false }; + bool cancelled { false }; + int resumeCount { 0 }; + std::thread::id resumedThread {}; + const auto cancellationThread = std::this_thread::get_id(); + + group.spawn(record_after_event(event, 1, order)); + group.spawn(wait_with_cancellation( + event, + stopSource.get_token(), + signalled, + cancelled, + resumeCount, + resumedThread)); + group.spawn(record_after_event(event, 3, order)); + + stopSource.request_stop(); + event.set(); + loop.run(group.join()); + + EXPECT_FALSE(signalled); + EXPECT_TRUE(cancelled); + EXPECT_EQ(resumeCount, 1); + EXPECT_EQ(resumedThread, cancellationThread); + EXPECT_EQ(order, (std::vector { 1, 3 })); +} + +TEST(CmpAsyncManualResetEventTest, CancellationResumesOnRequestThread) { + RunLoop loop {}; + AsyncManualResetEvent event {}; + TaskGroup group {}; + std::stop_source stopSource {}; + bool signalled { false }; + bool cancelled { false }; + int resumeCount { 0 }; + std::thread::id resumedThread {}; + std::thread::id cancellationThread {}; + + group.spawn(wait_with_cancellation( + event, + stopSource.get_token(), + signalled, + cancelled, + resumeCount, + resumedThread)); + + std::jthread canceller { [&] { + cancellationThread = std::this_thread::get_id(); + stopSource.request_stop(); + } }; + canceller.join(); + loop.run(group.join()); + + EXPECT_FALSE(signalled); + EXPECT_TRUE(cancelled); + EXPECT_EQ(resumeCount, 1); + EXPECT_EQ(resumedThread, cancellationThread); +} + +TEST(CmpAsyncManualResetEventTest, SetCancellationRaceResumesExactlyOnce) { + constexpr int ITERATIONS { 1'000 }; + RunLoop loop {}; + int setWins { 0 }; + int cancellationWins { 0 }; + + for (int iteration { 0 }; iteration < ITERATIONS; ++iteration) { + AsyncManualResetEvent event {}; + TaskGroup group {}; + std::stop_source stopSource {}; + std::latch start { 1 }; + bool signalled { false }; + bool cancelled { false }; + int resumeCount { 0 }; + std::thread::id resumedThread {}; + + group.spawn(wait_with_cancellation( + event, + stopSource.get_token(), + signalled, + cancelled, + resumeCount, + resumedThread)); + + std::jthread setter { [&] { + start.wait(); + event.set(); + } }; + std::jthread canceller { [&] { + start.wait(); + stopSource.request_stop(); + } }; + start.count_down(); + setter.join(); + canceller.join(); + loop.run(group.join()); + + ASSERT_EQ(resumeCount, 1); + ASSERT_NE(signalled, cancelled); + setWins += signalled ? 1 : 0; + cancellationWins += cancelled ? 1 : 0; + } + + EXPECT_EQ(setWins + cancellationWins, ITERATIONS); +} + +TEST(CmpAsyncManualResetEventTest, ManyWaitersDoNotGrowTheNativeStack) { + constexpr int WAITER_COUNT { 50'000 }; + RunLoop loop {}; + + EXPECT_EQ(loop.run(run_many_waiters(WAITER_COUNT)), WAITER_COUNT); +} + +TEST(CmpAsyncManualResetEventTest, NestedSignalsDoNotGrowTheNativeStack) { + constexpr int EVENT_COUNT { 20'000 }; + RunLoop loop {}; + + EXPECT_EQ(loop.run(run_signal_chain(EVENT_COUNT)), EVENT_COUNT); +} + +} // namespace diff --git a/tests/run_loop_test.cpp b/tests/run_loop_test.cpp index c5701bf..9c2eb8d 100644 --- a/tests/run_loop_test.cpp +++ b/tests/run_loop_test.cpp @@ -75,6 +75,48 @@ struct CancellationRecord final { std::thread::id thread_ {}; }; +Task schedule_once( + Scheduler scheduler, + std::stop_token stopToken) { + try { + co_await scheduler.schedule(stopToken); + co_return CancellationRecord { + false, + Clock::now(), + std::this_thread::get_id() + }; + } catch (const OperationCancelled&) { + co_return CancellationRecord { + true, + Clock::now(), + std::this_thread::get_id() + }; + } +} + +Task schedule_uncaught( + Scheduler scheduler, + std::stop_token stopToken) { + co_await scheduler.schedule(stopToken); +} + +Task schedule_pre_cancelled_ready_many_times( + Scheduler scheduler, + std::stop_token stopToken, + int count) { + int cancellationCount { 0 }; + + for (int index { 0 }; index < count; ++index) { + try { + co_await scheduler.schedule(stopToken); + } catch (const OperationCancelled&) { + ++cancellationCount; + } + } + + co_return cancellationCount; +} + Task schedule_after_once( Scheduler scheduler, Duration delay) { @@ -382,6 +424,64 @@ EagerOperation record_cancellable_after( } } +EagerOperation record_cancellable_schedule( + Scheduler scheduler, + std::stop_token stopToken, + int& phase, + int& resumedPhase, + bool& cancelled, + int& resumeCount) { + try { + co_await scheduler.schedule(stopToken); + } catch (const OperationCancelled&) { + cancelled = true; + } + + resumedPhase = phase; + ++resumeCount; +} + +Task cancel_ready_before_consumption( + Scheduler scheduler, + std::stop_source& stopSource, + std::optional& operation, + int& resumedPhase, + bool& cancelled, + int& resumeCount) { + int phase { 1 }; + operation.emplace(record_cancellable_schedule( + scheduler, + stopSource.get_token(), + phase, + resumedPhase, + cancelled, + resumeCount)); + + stopSource.request_stop(); + phase = 2; + co_await scheduler.schedule(); +} + +Task stop_after_schedule_is_consumed( + Scheduler scheduler, + std::stop_source& stopSource, + std::optional& operation, + bool& cancelled, + int& resumeCount) { + int phase { 1 }; + int resumedPhase { 0 }; + operation.emplace(record_cancellable_schedule( + scheduler, + stopSource.get_token(), + phase, + resumedPhase, + cancelled, + resumeCount)); + + co_await scheduler.schedule(); + stopSource.request_stop(); +} + Task run_timer_group( Scheduler scheduler, std::vector& operations, @@ -544,6 +644,79 @@ TEST(CmpRunLoopTest, SchedulingReturnsFromAnotherThread) { EXPECT_EQ(resumedThread, callingThread); } +TEST(CmpRunLoopTest, CancellableSchedulingCompletesAndDeregisters) { + RunLoop loop {}; + std::stop_source stopSource {}; + const auto callingThread = std::this_thread::get_id(); + + const auto result = loop.run(schedule_once( + loop.get_scheduler(), + stopSource.get_token())); + + EXPECT_FALSE(result.cancelled_); + EXPECT_EQ(result.thread_, callingThread); + EXPECT_TRUE(stopSource.request_stop()); + loop.run(do_nothing()); +} + +TEST(CmpRunLoopTest, PreRequestedSchedulingCancellationStillQueues) { + RunLoop loop {}; + std::stop_source stopSource {}; + const auto callingThread = std::this_thread::get_id(); + stopSource.request_stop(); + + EXPECT_FALSE(loop.get_scheduler().schedule( + stopSource.get_token()).await_ready()); + + const auto result = loop.run(schedule_once( + loop.get_scheduler(), + stopSource.get_token())); + + EXPECT_TRUE(result.cancelled_); + EXPECT_EQ(result.thread_, callingThread); +} + +TEST(CmpRunLoopTest, ReadyCancellationHasOneStableWinner) { + RunLoop loop {}; + + { + std::stop_source stopSource {}; + std::optional operation {}; + int resumedPhase { 0 }; + bool cancelled { false }; + int resumeCount { 0 }; + + loop.run(cancel_ready_before_consumption( + loop.get_scheduler(), + stopSource, + operation, + resumedPhase, + cancelled, + resumeCount)); + + EXPECT_TRUE(cancelled); + EXPECT_EQ(resumedPhase, 2); + EXPECT_EQ(resumeCount, 1); + } + + { + std::stop_source stopSource {}; + std::optional operation {}; + bool cancelled { false }; + int resumeCount { 0 }; + + loop.run(stop_after_schedule_is_consumed( + loop.get_scheduler(), + stopSource, + operation, + cancelled, + resumeCount)); + + EXPECT_FALSE(cancelled); + EXPECT_EQ(resumeCount, 1); + } +} + TEST(CmpRunLoopTest, TimedSchedulingNeverResumesBeforeItsDeadline) { constexpr auto DELAY = 15ms; RunLoop loop {}; @@ -867,6 +1040,17 @@ TEST(CmpRunLoopTest, RejectsInvalidSchedulersBeforeCancellation) { const auto expired = make_expired_scheduler(); stopSource.request_stop(); + EXPECT_THROW( + driver.run(schedule_uncaught( + owner.get_scheduler(), + stopSource.get_token())), + std::logic_error); + EXPECT_THROW( + driver.run(schedule_uncaught( + expired, + stopSource.get_token())), + std::logic_error); + EXPECT_THROW( driver.run(schedule_after_uncaught( owner.get_scheduler(), @@ -900,6 +1084,12 @@ TEST(CmpRunLoopTest, UncaughtCancellationCleansAndKeepsLoopReusable) { std::stop_source stopSource {}; stopSource.request_stop(); + EXPECT_THROW( + loop.run(schedule_uncaught( + loop.get_scheduler(), + stopSource.get_token())), + OperationCancelled); + EXPECT_THROW( loop.run(schedule_after_uncaught( loop.get_scheduler(), @@ -1065,4 +1255,18 @@ TEST(CmpRunLoopTest, RepeatedPreCancelledTimersDoNotGrowTheStack) { TIMER_COUNT); } +TEST(CmpRunLoopTest, RepeatedPreCancelledSchedulingDoesNotGrowTheStack) { + constexpr int SCHEDULE_COUNT { 100'000 }; + RunLoop loop {}; + std::stop_source stopSource {}; + stopSource.request_stop(); + + EXPECT_EQ( + loop.run(schedule_pre_cancelled_ready_many_times( + loop.get_scheduler(), + stopSource.get_token(), + SCHEDULE_COUNT)), + SCHEDULE_COUNT); +} + } // namespace diff --git a/tests/task_group_test.cpp b/tests/task_group_test.cpp index 7b66fe3..3ec7e78 100644 --- a/tests/task_group_test.cpp +++ b/tests/task_group_test.cpp @@ -79,31 +79,28 @@ Task mark_started(bool& started) { co_return; } -Task try_spawn_after_join_starts( +Task spawn_after_join_starts( Scheduler scheduler, TaskGroup& group, - bool& rejected, + bool& admitted, bool& candidateStarted) { co_await scheduler.schedule(); - try { - group.spawn(mark_started(candidateStarted)); - } catch (const std::logic_error&) { - rejected = true; - } + group.spawn(mark_started(candidateStarted)); + admitted = true; co_return; } -Task verify_join_closes_admission( +Task verify_join_allows_recursive_admission( Scheduler scheduler, - bool& rejected, + bool& admitted, bool& candidateStarted) { TaskGroup group {}; - group.spawn(try_spawn_after_join_starts( + group.spawn(spawn_after_join_starts( scheduler, group, - rejected, + admitted, candidateStarted)); co_await group.join(); } @@ -243,6 +240,48 @@ Task run_immediate_group(int count) { co_return completed; } +Task spawn_recursive_after_schedule( + Scheduler scheduler, + TaskGroup& group, + int remaining, + int& completed) { + co_await scheduler.schedule(); + + if (remaining > 1) { + group.spawn(spawn_recursive_after_schedule( + scheduler, + group, + remaining - 1, + completed)); + } + + ++completed; + co_return; +} + +Task run_recursive_group(Scheduler scheduler, int count) { + int completed { 0 }; + TaskGroup group {}; + group.spawn(spawn_recursive_after_schedule( + scheduler, + group, + count, + completed)); + co_await group.join(); + co_return completed; +} + +Task run_cancel_and_join_group( + Scheduler scheduler, + bool& cancelled, + bool& stopRequested) { + TaskGroup group {}; + const auto token = group.get_stop_token(); + group.spawn(observe_cancellation(scheduler, token, cancelled)); + co_await group.cancel_and_join(); + stopRequested = token.stop_requested(); +} + TEST(CmpTaskGroupTest, EmptyGroupJoinsAndTypeIsImmovable) { static_assert(!std::copy_constructible); static_assert(!std::move_constructible); @@ -300,18 +339,18 @@ TEST(CmpTaskGroupTest, SupportsConcurrentAdmission) { EXPECT_EQ(completed.load(), THREAD_COUNT * TASKS_PER_THREAD); } -TEST(CmpTaskGroupTest, RejectsAdmissionAfterJoinStartsAndRepeatedUse) { +TEST(CmpTaskGroupTest, AllowsRecursiveAdmissionUntilJoinReachesQuiescence) { RunLoop loop {}; - bool rejected { false }; + bool admitted { false }; bool candidateStarted { false }; - loop.run(verify_join_closes_admission( + loop.run(verify_join_allows_recursive_admission( loop.get_scheduler(), - rejected, + admitted, candidateStarted)); - EXPECT_TRUE(rejected); - EXPECT_FALSE(candidateStarted); + EXPECT_TRUE(admitted); + EXPECT_TRUE(candidateStarted); TaskGroup group {}; loop.run(group.join()); @@ -324,6 +363,17 @@ TEST(CmpTaskGroupTest, RejectsAdmissionAfterJoinStartsAndRepeatedUse) { EXPECT_THROW(loop.run(group.join()), std::logic_error); } +TEST(CmpTaskGroupTest, RecursiveAdmissionDoesNotGrowTheNativeStack) { + constexpr int TASK_COUNT { 20'000 }; + RunLoop loop {}; + + EXPECT_EQ( + loop.run(run_recursive_group( + loop.get_scheduler(), + TASK_COUNT)), + TASK_COUNT); +} + TEST(CmpTaskGroupTest, WaitsForAllAndRethrowsFirstExceptionByAdmissionOrder) { RunLoop loop {}; std::vector completions {}; @@ -361,6 +411,20 @@ TEST(CmpTaskGroupTest, StopTokenReachesExistingAndFutureChildren) { EXPECT_FALSE(secondRequestWon); } +TEST(CmpTaskGroupTest, CancelAndJoinRequestsStopBeforeWaiting) { + RunLoop loop {}; + bool cancelled { false }; + bool stopRequested { false }; + + loop.run(run_cancel_and_join_group( + loop.get_scheduler(), + cancelled, + stopRequested)); + + EXPECT_TRUE(cancelled); + EXPECT_TRUE(stopRequested); +} + TEST(CmpTaskGroupTest, PublishesCrossThreadCompletion) { RunLoop loop {}; std::atomic total { 0 }; From ab8446a7b8319c6ea2cd1c1ccb732bfdb6bb4ba8 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sat, 29 Aug 2026 22:05:29 +0800 Subject: [PATCH 02/17] =?UTF-8?q?=E5=AE=9E=E7=8E=B0=E7=AC=AC=E4=BA=94?= =?UTF-8?q?=E9=98=B6=E6=AE=B5=E5=8D=8F=E7=A8=8B=E7=BA=BF=E7=A8=8B=E6=B1=A0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 203 +++---- .agents/skills/mcpp-style-ref/SKILL.md | 12 +- README.md | 58 +- README.zh.hant.md | 51 +- README.zh.md | 51 +- benchmarks/thread-pool/mcpp.toml | 9 + benchmarks/thread-pool/src/main.cpp | 291 +++++++++ docs/architecture.md | 69 ++- docs/architecture.zh.hant.md | 70 ++- docs/architecture.zh.md | 68 ++- docs/benchmarks/2026-08-29-cmp-thread-pool.md | 67 ++ .../2026-08-29-cmp-phase5-thread-pool-v1.md | 82 +++ ...-08-29-cmp-phase5-thread-pool-v1-design.md | 321 ++++++++++ examples/basic/src/main.cpp | 26 + src/cmp.cppm | 1 + src/thread_pool.cppm | 301 +++++++++ tests/thread_pool_test.cpp | 572 ++++++++++++++++++ 17 files changed, 2056 insertions(+), 196 deletions(-) create mode 100644 benchmarks/thread-pool/mcpp.toml create mode 100644 benchmarks/thread-pool/src/main.cpp create mode 100644 docs/benchmarks/2026-08-29-cmp-thread-pool.md create mode 100644 docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md create mode 100644 docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md create mode 100644 src/thread_pool.cppm create mode 100644 tests/thread_pool_test.cpp diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 93e0819..ed3f70f 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -2,129 +2,102 @@ ## 项目概览 -CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 `mcpplibs.cmp`。当前 v1 -核心包括: +CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 `mcpplibs.cmp`。当前已实现 +懒启动唯一所有权 `Task`、变参/vector `when_all()`、静止点 `TaskGroup`、一次性与可复用 +事件、RAII `AsyncMutex`、带定时和取消的调用线程 `RunLoop`,以及固定大小的 CPU +`ThreadPool`。 -- 懒启动、唯一所有权的 `Task`; -- 变参和 `std::vector` 结构化 `when_all()`; -- eager、可递归接纳至静止点的 `TaskGroup`; -- `OneShotEvent`、可取消的 `AsyncManualResetEvent`、RAII `AsyncMutex`; -- 单消费者 `RunLoop` 及可复制 `Scheduler`; -- 普通、相对期限、绝对期限调度,以及显式 `std::stop_token` 协作式取消。 - -`.xlings.json` 固定 mcpp 2026.8.11.2,工具链解析为 LLVM 22.1.8,测试依赖是 +`.xlings.json` 固定 mcpp 2026.8.11.2;当前工具链为 LLVM 22.1.8,测试依赖为 `compat.gtest` 1.15.2。`examples/basic` 是独立 path-dependency consumer。 ## 当前目标与状态 -用户要求补齐第四阶段,增加成功/失败/竞态测试,并对计算、文件 I/O、网络 I/O 做压力验证, -确认 CMP v1 可用于开发。 - -第四阶段实现、测试、示例和本机压测已经完成,交付前最终一致性检查已通过。Git 交付分支为 -`feature/phase4-v1-readiness`;用户已明确授权提交并推送本阶段全部内容(包括压测),但没有 -授权创建 PR 或合并。 - -## 本阶段实现 - -### TaskGroup 静止点 join - -- `join()` 等待活动计数归零,不再在开始等待时立即关闭接纳。 -- 已经属于 group 的活动子任务可在 join 等待期间继续 `spawn()`;静止点到达后永久关闭。 -- 外部线程与最后完成瞬间竞态接纳时不保证哪方获胜;join 后接纳抛出 `std::logic_error`。 -- 新增 lazy `cancel_and_join()`:真正被等待时先 `request_stop()`,再执行同一个单次 join。 - -### 更广泛的显式取消 - -- `OperationCancelled` 移到独立 `:cancellation` 分区。 -- 新增 `Scheduler::schedule(std::stop_token)`;预取消也先排队,消费前取消获胜则抛出异常。 -- ready 与 timer 共用取消状态和 stop callback 协议;终态在 RunLoop 状态锁下确定,避免双恢复。 -- 普通调度和定时调度的取消查找仍为 O(n),只有压测证明瓶颈后才升级数据结构。 - -### AsyncManualResetEvent - -- 新增不可移动、无 waiter 分配的 `AsyncManualResetEvent`。 -- 支持初始状态、`is_set()`、`set()`、`reset()`、`wait(stop_token)` 和 `co_await event`。 -- pending 等待者按 FIFO 恢复;双向侵入队列支持 O(1) 取消移除。 -- set/cancel 竞态只有一个终态;预取消确定由取消获胜。 -- setter 或取消请求线程直接恢复等待者;thread-local trampoline 保证嵌套唤醒不增长原生栈。 -- 带 pending 等待者析构会终止,事件必须比等待协程帧活得更久。 - -## 测试与压测 - -- 全量套件现为 7 个测试二进制、90 项测试。 -- 新增/扩展测试覆盖正常、预取消、晚取消、跨线程、异常、拒绝、静止点、递归接纳、FIFO、 - publication、set/cancel 竞态、生命周期和原生栈安全。 -- `AsyncManualResetEvent` 10 项套件在 Release 下连续 30 轮通过;每轮包含 1,000 次 set/cancel - 竞态、50,000 个等待者和 20,000 个嵌套信号。 -- RunLoop 三项关键取消竞态在 Release 下连续 100 轮通过。 -- TaskGroup 四项递归、并发和取消关键用例在 Release 下连续 100 轮通过。 -- 最终文件状态下,Dev / Release `--strict --cache=off` 构建均通过;两个 profile 的全量 - 测试均为 90/90,7 个测试二进制、0 失败。 - -独立 POSIX Release 压测位于 `benchmarks/v1-readiness`。五轮均为 PASS: - -| 场景 | 每轮成功 | 每轮预期失败 | 每轮非预期失败 | 中位耗时 | 中位吞吐 | -| --- | ---: | ---: | ---: | ---: | ---: | -| compute | 49,000 | 1,000 | 0 | 28.172 ms | 1,774,824.8 ops/s | -| file_io | 1,000 | 100 | 0 | 185.222 ms | 5,938.8 ops/s | -| network_loopback | 20,000 | 100 | 0 | 2,455.883 ms | 8,184.4 ops/s | - -原始五轮数据、环境、命令和边界见 -`docs/benchmarks/2026-08-29-cmp-v1-readiness.md`。文件和网络工作由外部 `std::jthread` -适配,再通过事件和 Scheduler 返回 RunLoop;这验证 v1 互操作及生命周期,不宣称原生异步 -I/O 性能。 - -## 示例输出 - -`examples/basic` 已运行成功,输出: - -```text -Coroutine result: 42 -Concurrent result: 42 -Task group result: 42 -Recursive group result: 3 -Event signalled -Reusable event cycles: 2 -Mutex result: 42 -Coroutine cancelled -``` - -## 本阶段重要文件 - -- `src/cancellation.cppm` -- `src/run_loop.cppm` -- `src/task_group.cppm` -- `src/async_manual_reset_event.cppm` -- `src/cmp.cppm` -- `tests/run_loop_test.cpp` -- `tests/task_group_test.cpp` -- `tests/async_manual_reset_event_test.cpp` -- `examples/basic/src/main.cpp` -- `benchmarks/v1-readiness/mcpp.toml` -- `benchmarks/v1-readiness/src/main.cpp` -- `docs/benchmarks/2026-08-29-cmp-v1-readiness.md` -- `docs/superpowers/specs/2026-08-29-cmp-phase4-v1-completion-design.md` -- `docs/superpowers/plans/2026-08-29-cmp-phase4-v1-completion.md` -- 三份 README、三份架构说明和本文件。 - -## 已知边界与风险 - -- v1 没有原生异步 I/O、blocking pool、多 worker 调度、work stealing 或 detached 所有权。 -- 阻塞调用仍会阻塞其所在执行线程;当前 I/O consumer 使用外部线程适配。 -- 压测的 socket 实现仅支持 POSIX,不进入 Windows/macOS/Linux 通用 CI;库和根测试仍是跨平台目标。 -- Scheduler 大量并发取消时 O(n) 查找可能成为瓶颈,目前没有数据要求升级。 -- TaskGroup 保留 wrapper 帧至析构,空间复杂度 O(n);外部接纳与最终静止点竞态不提供保证。 -- AsyncManualResetEvent 在 set/cancel 调用线程恢复,不隐式保证 RunLoop 亲和;开发者需显式 - `co_await scheduler.schedule()` 返回目标循环。 -- 本机 mcpp 仍提示 SubOS 缺少 `subos_info`;工具链解析和本次构建、测试、运行未受影响。 -- 当前已完成本机验证,尚未进行 PR 三平台 CI 验证。 +第五阶段 ThreadPool v1 已完成本地实现、测试、示例、压测、三语文档同步及复审修复,当前可 +进入用户审查及远程 CI 交付。本轮未执行任何 Git 或 GitHub 操作;提交、推送、PR 和 CI 均不 +应被视为已经完成。 + +## 已完成工作 + +- 新增不可移动的固定大小 `ThreadPool` 和弱引用、可复制的 `ThreadPool::Scheduler`;根模块已 + 导出该分区。 +- `schedule()` 始终挂起并转移到任意 worker;`schedule(stop_token)` 支持 completion/cancel + 原子竞态获胜语义。 +- 实现共享 FIFO、休眠 worker、逐项唤醒、锁外恢复、构造失败清理、关闭/入队线性化、排空并 + join,以及 worker 内自析构终止保护。 +- 新增 15 项确定性 ThreadPool 测试,覆盖 API、FIFO、多 worker 唤醒、结构化组合、取消竞态、 + 高容量/栈安全、关闭与过期 Scheduler。 +- 复审后把默认 worker 数断言收紧为精确契约,并补齐同源与异源过期 Scheduler 的身份比较 + 测试。 +- `examples/basic` 增加协程内 worker 计算、显式回到 RunLoop 并打印 + `Worker pool result: 42` 的示例。 +- 新增标准库限定的 `benchmarks/thread-pool` consumer 和五轮数据报告;每轮覆盖粗粒度 CPU、 + 连续调度、worker 内 fan-out 和并发重调度。 +- README、架构文档、第五阶段设计/计划及压测报告已同步到实现事实。 +- `mcpp-style-ref` 不再重复维护易过期的分区清单,改以架构文档为当前事实来源;本轮涉及的 + 混合换行文件也已统一。 + +## 重要决策 + +- v1 只公开 `ThreadPool`、`get_scheduler()`、`thread_count()` 和两个 `schedule()` 重载;不增加 + executor 基类、submit/detach、resize、优先级、亲和或公共 shutdown API。 +- 默认 worker 数使用 `hardware_concurrency()` 并把未知的零归一为一;显式传零抛出 + `std::invalid_argument`。 +- 队列是无界共享 `std::deque`。有界同步接纳可能让递归提交的全部 worker 互相等待,因此 + backpressure 必须另行设计。 +- 取消项留在 FIFO,由 worker 消费并恢复;取消回调不扫描队列,也不在请求取消的线程恢复用户 + 协程。 +- 析构关闭接纳并排空已经接纳的 continuation;ThreadPool 只拥有执行线程,Task 生命周期仍由 + `Task`、`when_all()` 和 `TaskGroup` 管理。 +- 本机数据证明粗粒度 CPU 并行有效,也证明微任务会争用共享队列;work stealing 只有在代表性 + workload 的 profiler 证明共享队列是主要瓶颈后,才进入单独设计。 + +## 修改 / 重要文件 + +- 核心:`src/thread_pool.cppm`、`src/cmp.cppm` +- 测试:`tests/thread_pool_test.cpp` +- 示例:`examples/basic/src/main.cpp` +- 压测:`benchmarks/thread-pool/`、`docs/benchmarks/2026-08-29-cmp-thread-pool.md` +- 方案:`docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md`、 + `docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md` +- 公共文档:三份 README 与三份 `docs/architecture*` 文档 +- Agent 技能:`.agents/skills/mcpp-style-ref/SKILL.md` + +## 验证情况 + +- `mcpp build --profile dev --strict --cache=off`:通过。 +- `mcpp test --profile dev --strict --cache=off`:8 个二进制、105/105 通过。 +- `mcpp build --profile release --strict --cache=off`:通过。 +- `mcpp test --profile release --strict --cache=off`:8 个二进制、105/105 通过。 +- Dev `thread_pool_test` 定向复验:15/15 通过。 +- Release 关键竞态测试子集连续执行 100 轮:100/100 通过。 +- `examples/basic` 的 `mcpp run`:通过,包含 `Worker pool result: 42`,退出码 0。 +- `mcpp-style-ref` 通过 `quick_validate.py` 校验;架构文档链接有效。 +- 项目文本文件换行复查未发现混合 CRLF/LF 文件。 +- ThreadPool Release benchmark:构建通过;五轮记录共 8,410,240 次操作全部成功、非预期失败 + 为 0;最终可执行性复验也全部 PASS。 +- 五轮中位数显示粗粒度 CPU 任务从 1 到 8 worker 约 7.56 倍加速;完整原始数据见压测报告。 +- 以上均为本机 Linux/WSL2 结果;本轮没有运行 GitHub 三平台 CI。 + +## 已知问题 / 风险 + +- 共享 FIFO 在大量微小 continuation 与较多 worker 时存在显著 mutex/通知争用;这是已记录的 + v1 性能上限,不是丢任务或执行失败。 +- ThreadPool 不抢占;长计算或阻塞调用会占住一个 worker。异步 I/O 与专用 blocking pool 尚未 + 实现。 +- 析构会等待已接纳 continuation 返回;用户代码永久阻塞时析构也会永久等待。在自身 worker + 内析构属于硬生命周期错误并立即终止。 +- 三平台兼容性尚需远程 CI 确认。 +- 本机 mcpp 仍输出 SubOS 缺少 `subos_info` 的环境警告,但本轮所有构建和运行均成功。 ## 剩余工作 -本地实现与验证没有剩余项。本阶段 Git 交付以 `feature/phase4-v1-readiness` 上的第四阶段 -提交为边界;PR、合并和对应三平台 CI 仍需用户另行决定。 +1. 用户审查第五阶段实现与压测结论。 +2. 获得明确授权后,才可执行对应范围的 commit、push、PR 或远程 CI 操作。 +3. 第五阶段合并并通过三平台 CI 后,再开始第六阶段异步 I/O / blocking pool 设计;不要把该 + 能力补进当前 ThreadPool。 ## 推荐下一步 -确认功能分支远端同步后,由用户决定是否创建 PR 并进行三平台 CI。第四阶段合并后,再单独 -设计第五阶段多 worker 调度,不把第六阶段 I/O 后端提前混入。 +先审查 `src/thread_pool.cppm`、`tests/thread_pool_test.cpp` 和线程池压测报告。若认可当前共享 +FIFO v1 边界,再明确授权所需的 Git/GitHub 步骤完成交付;合并后从第六阶段的 I/O 契约与平台 +边界开始设计。 diff --git a/.agents/skills/mcpp-style-ref/SKILL.md b/.agents/skills/mcpp-style-ref/SKILL.md index 6bad24b..f7347c7 100644 --- a/.agents/skills/mcpp-style-ref/SKILL.md +++ b/.agents/skills/mcpp-style-ref/SKILL.md @@ -145,15 +145,9 @@ mcpp --version ## 当前项目结构 -参考本仓库 `src/` 目录结构: - -- `.xlings.json`:声明项目工具环境 -- `mcpp.toml`:声明 `[package]` 与测试依赖;简单库目标可由 mcpp 从 `src/*.cppm` 自动推断 -- `src/cmp.cppm`:库主模块接口,导出 `:task` 与 `:run_loop` 分区 -- `src/task.cppm`:`Task` 与 `Task` 分区 -- `src/run_loop.cppm`:`RunLoop` 与 `Scheduler` 分区 -- `tests/cmp_test.cpp`、`tests/run_loop_test.cpp`:`mcpp test` 自动发现的 gtest 测试;不要定义 `main()` -- `examples/basic/`:独立 mcpp consumer 包,通过 path 依赖引用根库 +开始审查 CMP 前先读取[架构文档](../../../docs/architecture.zh.md),以其中的公共边界和目录结构 +为当前事实来源。模块根接口位于 `src/cmp.cppm`,`tests/**/*.cpp` 由 `mcpp test` 自动发现, +`examples/basic/` 是独立的 path-dependency consumer;不要在技能内重复维护完整分区清单。 构建: diff --git a/README.md b/README.md index 1c4a921..83ebd74 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,7 @@ [mcpp](https://github.com/mcpp-community/mcpp) · [Architecture](docs/architecture.md) · [v1 readiness benchmark](docs/benchmarks/2026-08-29-cmp-v1-readiness.md) · +[thread-pool benchmark](docs/benchmarks/2026-08-29-cmp-thread-pool.md) · [Issues](https://github.com/mcpplibs/cmp/issues) [![ci-linux](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml) @@ -19,14 +20,16 @@ > [!IMPORTANT] > CMP provides a lazy, single-consumer `Task` / `Task`, structured variadic and vector > `when_all()`, an eager structured `TaskGroup`, one-shot and reusable events, an RAII `AsyncMutex`, -> and a caller-thread `RunLoop` with explicit and monotonic timed scheduling. Ready scheduling, -> timed waits, reusable-event waits, and TaskGroup children can use explicit cooperative -> cancellation with `std::stop_token`; asynchronous I/O and detached execution are not implemented. +> a caller-thread `RunLoop` with explicit and monotonic timed scheduling, and a fixed-size CPU +> `ThreadPool`. Ready scheduling on either executor, timed waits, reusable-event waits, and +> TaskGroup children can use explicit cooperative cancellation with `std::stop_token`; +> asynchronous I/O and detached execution are not implemented. CMP is being built as a modern coroutine runtime and library on standard stackless C++ coroutines. Its explicit `co_await` model now covers fixed and incremental structured concurrency, -one-time event notification, scheduling, monotonic timers, and cancellable timed waits and can -grow, in small verified steps, toward asynchronous I/O and safe handling of blocking work. +one-time event notification, caller-thread and multi-worker scheduling, monotonic timers, and +cancellable waits and can grow, in small verified steps, toward asynchronous I/O and safe handling +of blocking work. ## Why CMP? @@ -51,7 +54,7 @@ promise that: - a task is automatically equivalent to a Go goroutine; - an arbitrary blocking call becomes non-blocking; - coroutine switching is safe directly inside a signal handler; -- M:N scheduling, work stealing, general cancellation propagation, or async I/O already exist. +- task migration is implicit, work stealing is already enabled, or arbitrary async I/O exists. Those capabilities must be designed and verified individually. The expected direction is explicit async I/O awaiters, a dedicated blocking pool, and cooperative safe points. @@ -75,8 +78,8 @@ cd examples/basic mcpp run ``` -The example prints `Coroutine result: 42`, `Concurrent result: 42`, `Task group result: 42`, and -`Recursive group result: 3`; it then demonstrates one-shot and reusable notifications with +The example prints `Coroutine result: 42`, `Concurrent result: 42`, `Worker pool result: 42`, +`Task group result: 42`, and `Recursive group result: 3`; it then demonstrates notifications with `Event signalled` and `Reusable event cycles: 2`, prints `Mutex result: 42`, and finishes with `Coroutine cancelled`. All messages come from `Task` coroutines. @@ -92,6 +95,7 @@ using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -124,6 +128,23 @@ Task print_concurrent_results(RunLoop::Scheduler scheduler) { co_return; } +Task calculate_on_workers( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const int result = 21 * 2; + co_await caller.schedule(); + co_return result; +} + +Task print_worker_result( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + const auto result = co_await calculate_on_workers(workers, caller); + std::println("Worker pool result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -192,9 +213,13 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke } int main() { + ThreadPool workers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); + loop.run(print_worker_result( + workers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -260,6 +285,13 @@ cannot replace a completion that already won. The stop callback only wakes the R coroutine code is resumed by the thread driving `run()`. Cancellation currently performs an O(n) queue lookup. +`ThreadPool` starts a fixed number of CPU workers. Its copyable Scheduler always suspends and may +resume a continuation on any worker; explicitly await a `RunLoop::Scheduler` to return to the +caller thread. `schedule(stop_token)` has the same completion-versus-cancellation winner rule as +RunLoop ready scheduling. The v1 implementation uses one work-conserving shared FIFO and sleeping +workers. Destruction closes admission, drains every accepted entry, and joins the workers. It does +not own higher-level Tasks, and blocking calls still block their current worker. + RunLoop is not a background thread and does not make blocking code asynchronous. A Task that suspends without arranging a future resume can leave `run()` waiting indefinitely. CMP does not provide automatic thread affinity: after an external awaiter resumes on another thread, explicitly @@ -275,6 +307,7 @@ await the desired Scheduler to return to its RunLoop. ├── src/task.cppm # Task module partition ├── src/cancellation.cppm # shared cooperative-cancellation exception ├── src/run_loop.cppm # RunLoop and Scheduler partition +├── src/thread_pool.cppm # fixed-size CPU worker scheduler ├── src/when_all.cppm # structured concurrent Task join ├── src/task_group.cppm # eager mutable structured Task scope ├── src/one_shot_event.cppm # allocation-free one-time notification @@ -282,6 +315,7 @@ await the desired Scheduler to return to its RunLoop. ├── src/async_mutex.cppm # FIFO coroutine-aware RAII mutex ├── tests/cmp_test.cpp # Task contract and lifetime tests ├── tests/run_loop_test.cpp # scheduler, boundary, and threading tests +├── tests/thread_pool_test.cpp # worker, cancellation, and shutdown tests ├── tests/when_all_test.cpp # join ownership, result, and race tests ├── tests/task_group_test.cpp # mutable scope lifetime and race tests ├── tests/one_shot_event_test.cpp # event publication and race tests @@ -289,6 +323,7 @@ await the desired Scheduler to return to its RunLoop. ├── tests/async_mutex_test.cpp # mutex ownership and hand-off tests ├── examples/basic/ # standalone path-dependency consumer ├── benchmarks/v1-readiness/ # local compute, file, and loopback baseline +├── benchmarks/thread-pool/ # local multi-worker scheduling baseline ├── docs/architecture.md # current structure, boundaries, and evolution └── .github/workflows/ # Linux, macOS, and Windows CI ``` @@ -314,9 +349,11 @@ global mcpp installation. CMP does not track `mcpp.lock`; `.gitignore` enforces that repository policy. Runtime dependencies belong in `[dependencies]`; gtest is declared explicitly under `[dev-dependencies.compat]`. -The current local suite contains 90 tests across seven binaries. The POSIX-only Release pressure +The current local suite contains 105 tests across eight binaries. The POSIX-only Release pressure consumer and its recorded success/failure data are documented in the [v1 readiness benchmark](docs/benchmarks/2026-08-29-cmp-v1-readiness.md). +Multi-worker correctness and performance data are documented in the +[thread-pool benchmark](docs/benchmarks/2026-08-29-cmp-thread-pool.md). ## Roadmap @@ -327,7 +364,8 @@ Runtime work is split into independently reviewable phases: 3. a root runner and minimal single-thread scheduler — initially implemented; 4. monotonic Timer v1, cancellable ready/timed waits, variadic/vector joins, quiescent TaskGroup, OneShotEvent, AsyncManualResetEvent, and AsyncMutex — implemented and pressure-tested; -5. multi-worker scheduling and work stealing; +5. fixed-size multi-worker scheduling — implemented and benchmarked; work stealing remains gated + by profiling evidence; 6. asynchronous I/O integration and a blocking pool. The remaining order is directional, not a promise that a listed feature is already implemented. diff --git a/README.zh.hant.md b/README.zh.hant.md index eeae0e8..9add5ae 100644 --- a/README.zh.hant.md +++ b/README.zh.hant.md @@ -10,6 +10,7 @@ [mcpp](https://github.com/mcpp-community/mcpp) · [架構](docs/architecture.zh.hant.md) · [v1 可開發性壓測](docs/benchmarks/2026-08-29-cmp-v1-readiness.md) · +[執行緒池壓測](docs/benchmarks/2026-08-29-cmp-thread-pool.md) · [Issues](https://github.com/mcpplibs/cmp/issues) [![ci-linux](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml) @@ -19,13 +20,13 @@ > [!IMPORTANT] > CMP 已提供延遲啟動、單一消費者的 `Task` / `Task`、支援變參和 vector 的結構化 > `when_all()`、eager 結構化 `TaskGroup`、一次性與可複用事件、RAII `AsyncMutex`,以及在呼叫 -> 執行緒運行、支援明確排程和單調時鐘定時排程的 `RunLoop`。就緒排程、定時等待、可複用事件 -> 等待和 TaskGroup 子任務可明確使用基於 `std::stop_token` 的協作式取消;非同步 I/O 和 -> detached 執行尚未實作。 +> 執行緒運行、支援明確排程和單調時鐘定時排程的 `RunLoop`,以及固定大小的 CPU +> `ThreadPool`。兩種執行器的就緒排程、定時等待、可複用事件等待和 TaskGroup 子任務可明確 +> 使用基於 `std::stop_token` 的協作式取消;非同步 I/O 和 detached 執行尚未實作。 CMP 計畫以標準無堆疊 C++ 協程建構現代協程執行期與函式庫。明確的 `co_await` 模型現已 -涵蓋固定與增量結構化並行、一次性事件通知、排程、單調時鐘計時器和可取消定時等待,並將 -透過經過驗證的小步驟繼續探索非同步 I/O,以及阻塞工作的安全隔離。 +涵蓋固定與增量結構化並行、一次性事件通知、呼叫執行緒與多 worker 排程、單調時鐘計時器和 +可取消等待,並將透過經過驗證的小步驟繼續探索非同步 I/O,以及阻塞工作的安全隔離。 ## 為什麼叫 CMP? @@ -48,7 +49,7 @@ C++ 標準協程是語言機制,不是完整執行期。因此 CMP 不會宣 - task 自動等同於 Go goroutine; - 任意阻塞呼叫會自動成為非阻塞呼叫; - 可以直接在訊號處理器中安全切換協程; -- M:N 排程、work stealing、通用取消傳播或非同步 I/O 已經實作。 +- task 會隱式遷移、work stealing 已啟用,或任意非同步 I/O 已經實作。 這些能力必須分別設計和驗證。預期方向是明確的非同步 I/O awaiter、專用 blocking pool 以及協作式安全點。 @@ -72,7 +73,7 @@ cd examples/basic mcpp run ``` -範例會印出 `Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42` 和 +範例會印出 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42` 和 `Recursive group result: 3`,接著透過 `Event signalled` 與 `Reusable event cycles: 2` 展示一次性及可複用通知,再印出 `Mutex result: 42` 和 `Coroutine cancelled`;所有輸出都在 `Task` 協程內部。 @@ -89,6 +90,7 @@ using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -121,6 +123,23 @@ Task print_concurrent_results(RunLoop::Scheduler scheduler) { co_return; } +Task calculate_on_workers( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const int result = 21 * 2; + co_await caller.schedule(); + co_return result; +} + +Task print_worker_result( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + const auto result = co_await calculate_on_workers(workers, caller); + std::println("Worker pool result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -189,9 +208,13 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke } int main() { + ThreadPool workers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); + loop.run(print_worker_result( + workers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -247,6 +270,12 @@ set 不做任何事。事件不可移動且必須比等待者活得更久;v1 消費獲勝,等待會拋出 `OperationCancelled`;較晚的停止要求不能取代已經獲勝的完成。停止回呼 只喚醒 RunLoop,使用者協程仍由執行 `run()` 的執行緒恢復。目前取消透過 O(n) 掃描定位佇列項。 +`ThreadPool` 啟動固定數量的 CPU worker。其可複製 Scheduler 始終暫停,並可在任意 worker 上 +恢復 continuation;需要返回呼叫執行緒時明確等待 `RunLoop::Scheduler`。 +`schedule(stop_token)` 與 RunLoop 就緒排程採用相同的完成/取消勝者規則。v1 使用一個 +work-conserving 共用 FIFO 和休眠 worker;解構關閉接納、排空全部已接納項並 join worker。 +ThreadPool 不擁有上層 Task,阻塞呼叫仍會阻塞目前 worker。 + RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步程式碼。如果 Task 暫停後沒有 安排未來的恢復動作,`run()` 可能一直等待。CMP 不提供隱式執行緒親和:外部 awaiter 在其他 執行緒恢復協程後,需要明確等待目標 Scheduler 才會返回對應 RunLoop。 @@ -261,6 +290,7 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ├── src/task.cppm # Task 模組分割區 ├── src/cancellation.cppm # 共用的協作式取消例外 ├── src/run_loop.cppm # RunLoop 與 Scheduler 分割區 +├── src/thread_pool.cppm # 固定大小的 CPU worker 排程器 ├── src/when_all.cppm # 結構化並行 Task 匯合 ├── src/task_group.cppm # eager 可變結構化 Task 作用域 ├── src/one_shot_event.cppm # 無分配一次性通知 @@ -268,6 +298,7 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ├── src/async_mutex.cppm # FIFO 協程感知 RAII 互斥鎖 ├── tests/cmp_test.cpp # Task 契約和生命週期測試 ├── tests/run_loop_test.cpp # 排程、邊界和執行緒測試 +├── tests/thread_pool_test.cpp # worker、取消和關閉測試 ├── tests/when_all_test.cpp # 匯合所有權、結果和競態測試 ├── tests/task_group_test.cpp # 可變作用域生命週期和競態測試 ├── tests/one_shot_event_test.cpp # 事件發布和競態測試 @@ -275,6 +306,7 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ├── tests/async_mutex_test.cpp # mutex 所有權和交接測試 ├── examples/basic/ # 獨立的路徑相依 consumer ├── benchmarks/v1-readiness/ # 本機計算、檔案和回環網路基線 +├── benchmarks/thread-pool/ # 本機多 worker 排程基線 ├── docs/architecture.zh.hant.md # 目前結構、邊界與演進方向 └── .github/workflows/ # Linux、macOS 和 Windows CI ``` @@ -298,8 +330,9 @@ CI 在 Linux、macOS 和 Windows 上執行等價的建構、測試與獨立範 CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定。執行期相依放在 `[dependencies]`,gtest 明確宣告在 `[dev-dependencies.compat]` 中。 -目前本機套件包含 7 個測試二進位檔、90 項測試。僅用於 POSIX 的 Release 壓測 consumer 及其 +目前本機套件包含 8 個測試二進位檔、105 項測試。僅用於 POSIX 的 Release 壓測 consumer 及其 成功/失敗資料記錄在 [v1 可開發性壓測](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 +多 worker 正確性和效能資料記錄在[執行緒池壓測](docs/benchmarks/2026-08-29-cmp-thread-pool.md)。 ## 路線圖 @@ -310,7 +343,7 @@ CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定 3. 根任務驅動器和最小單執行緒排程器——已完成初始實作; 4. 單調時鐘 Timer v1、可取消就緒/定時等待、變參/vector 匯合、靜止點 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已實作並完成壓力驗證; -5. 多 worker 排程與 work stealing; +5. 固定大小的多 worker 排程——已實作並完成壓測;work stealing 仍需 profiling 證據; 6. 非同步 I/O 整合和 blocking pool。 剩餘順序只是方向,不代表列出的能力已經實作。 diff --git a/README.zh.md b/README.zh.md index 246b849..f368520 100644 --- a/README.zh.md +++ b/README.zh.md @@ -10,6 +10,7 @@ [mcpp](https://github.com/mcpp-community/mcpp) · [架构](docs/architecture.zh.md) · [v1 可开发性压测](docs/benchmarks/2026-08-29-cmp-v1-readiness.md) · +[线程池压测](docs/benchmarks/2026-08-29-cmp-thread-pool.md) · [Issues](https://github.com/mcpplibs/cmp/issues) [![ci-linux](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpplibs/cmp/actions/workflows/ci-linux.yml) @@ -19,13 +20,13 @@ > [!IMPORTANT] > CMP 已提供懒启动、单消费者的 `Task` / `Task`、支持变参和 vector 的结构化 > `when_all()`、eager 结构化 `TaskGroup`、一次性与可复用事件、RAII `AsyncMutex`,以及在调用 -> 线程运行、支持显式调度和单调时钟定时调度的 `RunLoop`。就绪调度、定时等待、可复用事件等待 -> 和 TaskGroup 子任务可显式使用基于 `std::stop_token` 的协作式取消;异步 I/O 和 detached -> 执行尚未实现。 +> 线程运行、支持显式调度和单调时钟定时调度的 `RunLoop`,以及固定大小的 CPU `ThreadPool`。 +> 两种执行器的就绪调度、定时等待、可复用事件等待和 TaskGroup 子任务可显式使用基于 +> `std::stop_token` 的协作式取消;异步 I/O 和 detached 执行尚未实现。 CMP 计划基于标准无栈 C++ 协程构建现代协程运行时和库。显式 `co_await` 模型现已覆盖固定与 -增量结构化并发、一次性事件通知、调度、单调时钟定时器和可取消定时等待,并将通过经过验证 -的小步骤继续探索异步 I/O 以及阻塞工作的安全隔离。 +增量结构化并发、一次性事件通知、调用线程与多 worker 调度、单调时钟定时器和可取消等待, +并将通过经过验证的小步骤继续探索异步 I/O 以及阻塞工作的安全隔离。 ## 为什么叫 CMP? @@ -48,7 +49,7 @@ C++ 标准协程是语言机制,不是完整运行时。因此 CMP 不会宣 - task 自动等同于 Go goroutine; - 任意阻塞调用会自动变成非阻塞调用; - 可以直接在信号处理器中安全切换协程; -- M:N 调度、work stealing、通用取消传播或异步 I/O 已经实现。 +- task 会隐式迁移、work stealing 已启用,或任意异步 I/O 已经实现。 这些能力必须分别设计和验证。预期方向是显式异步 I/O awaiter、专用 blocking pool 以及协作式安全点。 @@ -72,7 +73,7 @@ cd examples/basic mcpp run ``` -示例会打印 `Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42` 和 +示例会打印 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42` 和 `Recursive group result: 3`,然后通过 `Event signalled` 与 `Reusable event cycles: 2` 演示一次性及可复用通知,再打印 `Mutex result: 42` 和 `Coroutine cancelled`;所有输出都在 `Task` 协程内部。 @@ -89,6 +90,7 @@ using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -121,6 +123,23 @@ Task print_concurrent_results(RunLoop::Scheduler scheduler) { co_return; } +Task calculate_on_workers( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const int result = 21 * 2; + co_await caller.schedule(); + co_return result; +} + +Task print_worker_result( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + const auto result = co_await calculate_on_workers(workers, caller); + std::println("Worker pool result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -189,9 +208,13 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke } int main() { + ThreadPool workers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); + loop.run(print_worker_result( + workers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -247,6 +270,12 @@ set 不做任何事。事件不可移动且必须比等待者活得更久;v1 消费获胜,等待会抛出 `OperationCancelled`;较晚的停止请求不能替换已经获胜的完成。停止回调 只唤醒 RunLoop,用户协程仍由执行 `run()` 的线程恢复。当前取消通过 O(n) 扫描定位队列项。 +`ThreadPool` 启动固定数量的 CPU worker。其可复制 Scheduler 始终挂起,并可在任意 worker 上 +恢复 continuation;需要回到调用线程时显式等待 `RunLoop::Scheduler`。 +`schedule(stop_token)` 与 RunLoop 就绪调度采用相同的完成/取消胜者规则。v1 使用一个 +work-conserving 共享 FIFO 和休眠 worker;析构关闭接纳、排空全部已接纳项并 join worker。 +ThreadPool 不拥有上层 Task,阻塞调用仍会阻塞当前 worker。 + RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。如果 Task 挂起后没有安排未来的 恢复动作,`run()` 可能一直等待。CMP 不提供隐式线程亲和:外部 awaiter 在其他线程恢复协程 后,需要显式等待目标 Scheduler 才会返回对应 RunLoop。 @@ -261,6 +290,7 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ├── src/task.cppm # Task 模块分区 ├── src/cancellation.cppm # 共享的协作式取消异常 ├── src/run_loop.cppm # RunLoop 与 Scheduler 分区 +├── src/thread_pool.cppm # 固定大小的 CPU worker 调度器 ├── src/when_all.cppm # 结构化并发 Task 汇合 ├── src/task_group.cppm # eager 可变结构化 Task 作用域 ├── src/one_shot_event.cppm # 无分配一次性通知 @@ -268,6 +298,7 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ├── src/async_mutex.cppm # FIFO 协程感知 RAII 互斥锁 ├── tests/cmp_test.cpp # Task 契约和生命周期测试 ├── tests/run_loop_test.cpp # 调度、边界和线程测试 +├── tests/thread_pool_test.cpp # worker、取消和关闭测试 ├── tests/when_all_test.cpp # 汇合所有权、结果和竞态测试 ├── tests/task_group_test.cpp # 可变作用域生命周期和竞态测试 ├── tests/one_shot_event_test.cpp # 事件发布和竞态测试 @@ -275,6 +306,7 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ├── tests/async_mutex_test.cpp # mutex 所有权和交接测试 ├── examples/basic/ # 独立的路径依赖 consumer ├── benchmarks/v1-readiness/ # 本地计算、文件和回环网络基线 +├── benchmarks/thread-pool/ # 本地多 worker 调度基线 ├── docs/architecture.zh.md # 当前结构、边界和演进方向 └── .github/workflows/ # Linux、macOS 和 Windows CI ``` @@ -298,8 +330,9 @@ CI 在 Linux、macOS 和 Windows 上执行等价的构建、测试和独立示 CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。运行时依赖放在 `[dependencies]`,gtest 明确声明在 `[dev-dependencies.compat]` 中。 -当前本地套件包含 7 个测试二进制、90 项测试。仅用于 POSIX 的 Release 压测 consumer 及其 +当前本地套件包含 8 个测试二进制、105 项测试。仅用于 POSIX 的 Release 压测 consumer 及其 成功/失败数据记录在 [v1 可开发性压测](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 +多 worker 正确性和性能数据记录在[线程池压测](docs/benchmarks/2026-08-29-cmp-thread-pool.md)。 ## 路线图 @@ -310,7 +343,7 @@ CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。 3. 根任务驱动器和最小单线程调度器——已完成初始实现; 4. 单调时钟 Timer v1、可取消就绪/定时等待、变参/vector 汇合、静止点 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已实现并完成压力验证; -5. 多 worker 调度与 work stealing; +5. 固定大小的多 worker 调度——已实现并完成压测;work stealing 仍需 profiling 证据; 6. 异步 I/O 集成和 blocking pool。 剩余顺序只是方向,不代表列出的能力已经实现。 diff --git a/benchmarks/thread-pool/mcpp.toml b/benchmarks/thread-pool/mcpp.toml new file mode 100644 index 0000000..9513216 --- /dev/null +++ b/benchmarks/thread-pool/mcpp.toml @@ -0,0 +1,9 @@ +[package] +name = "cmp-thread-pool-benchmark" +version = "0.1.0" +standard = "c++23" +description = "Standalone CMP thread-pool scheduling benchmark" +license = "Apache-2.0" + +[dependencies.mcpplibs] +cmp = { path = "../.." } diff --git a/benchmarks/thread-pool/src/main.cpp b/benchmarks/thread-pool/src/main.cpp new file mode 100644 index 0000000..6aa6586 --- /dev/null +++ b/benchmarks/thread-pool/src/main.cpp @@ -0,0 +1,291 @@ +import std; +import mcpplibs.cmp; + +namespace { + +using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::Task; +using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::when_all; + +using Scheduler = ThreadPool::Scheduler; +using Clock = std::chrono::steady_clock; + +constexpr int CPU_TASK_COUNT { 512 }; +constexpr int CPU_STEPS { 200'000 }; +constexpr int SCHEDULE_HOPS { 200'000 }; +constexpr int NESTED_TASKS { 20'000 }; +constexpr int PRODUCER_COUNT { 8 }; +constexpr int HOPS_PER_PRODUCER { 25'000 }; + +struct Counters final { + std::atomic successes_ {}; + std::atomic unexpectedFailures_ {}; + std::atomic active_ {}; + std::atomic maxConcurrency_ {}; +}; + +struct Metrics final { + std::string scenario_ {}; + std::size_t workerCount_ {}; + std::size_t operations_ {}; + std::size_t successes_ {}; + std::size_t unexpectedFailures_ {}; + std::size_t maxConcurrency_ {}; + double elapsedMilliseconds_ {}; + + [[nodiscard]] bool passed() const noexcept { + return successes_ == operations_ && unexpectedFailures_ == 0; + } +}; + +void record_completion(Counters& counters) noexcept { + const auto active = counters.active_.fetch_add( + 1, + std::memory_order_relaxed) + 1; + auto maximum = counters.maxConcurrency_.load(std::memory_order_relaxed); + + while (maximum < active && + !counters.maxConcurrency_.compare_exchange_weak( + maximum, + active, + std::memory_order_relaxed)) { + } + + counters.successes_.fetch_add(1, std::memory_order_relaxed); + counters.active_.fetch_sub(1, std::memory_order_relaxed); +} + +[[nodiscard]] std::uint64_t compute_value(int index) noexcept { + std::uint64_t value = static_cast(index) + 1; + + for (int step { 0 }; step < CPU_STEPS; ++step) { + value ^= value << 13; + value ^= value >> 7; + value ^= value << 17; + } + + return value; +} + +Task compute_operation( + Scheduler scheduler, + int index, + Counters& counters, + std::atomic& checksum) { + co_await scheduler.schedule(); + + const auto active = counters.active_.fetch_add( + 1, + std::memory_order_relaxed) + 1; + auto maximum = counters.maxConcurrency_.load(std::memory_order_relaxed); + while (maximum < active && + !counters.maxConcurrency_.compare_exchange_weak( + maximum, + active, + std::memory_order_relaxed)) { + } + + checksum.fetch_xor(compute_value(index), std::memory_order_relaxed); + counters.successes_.fetch_add(1, std::memory_order_relaxed); + counters.active_.fetch_sub(1, std::memory_order_relaxed); +} + +Task run_compute( + Scheduler scheduler, + Counters& counters, + std::uint64_t expectedChecksum) { + std::atomic checksum {}; + TaskGroup group {}; + + for (int index { 0 }; index < CPU_TASK_COUNT; ++index) { + group.spawn(compute_operation( + scheduler, + index, + counters, + checksum)); + } + + co_await group.join(); + if (checksum.load(std::memory_order_relaxed) != expectedChecksum) { + counters.unexpectedFailures_.fetch_add(1); + } +} + +Task run_schedule_hops( + Scheduler scheduler, + Counters& counters) { + for (int index { 0 }; index < SCHEDULE_HOPS; ++index) { + co_await scheduler.schedule(); + record_completion(counters); + } +} + +Task run_one_shot( + Scheduler scheduler, + Counters& counters) { + co_await scheduler.schedule(); + record_completion(counters); +} + +Task run_nested_fanout( + Scheduler scheduler, + Counters& counters) { + co_await scheduler.schedule(); + std::vector> tasks {}; + tasks.reserve(NESTED_TASKS); + + for (int index { 0 }; index < NESTED_TASKS; ++index) { + tasks.emplace_back(run_one_shot(scheduler, counters)); + } + + static_cast(co_await when_all(std::move(tasks))); +} + +Task run_producer( + Scheduler scheduler, + Counters& counters) { + for (int index { 0 }; index < HOPS_PER_PRODUCER; ++index) { + co_await scheduler.schedule(); + record_completion(counters); + } +} + +Task run_concurrent_producers( + Scheduler scheduler, + Counters& counters) { + TaskGroup group {}; + + for (int index { 0 }; index < PRODUCER_COUNT; ++index) { + group.spawn(run_producer(scheduler, counters)); + } + + co_await group.join(); +} + +template +[[nodiscard]] Metrics measure( + std::string scenario, + ThreadPool& pool, + std::size_t operations, + Scenario scenarioTask) { + Counters counters {}; + RunLoop loop {}; + const auto start = Clock::now(); + + try { + loop.run(scenarioTask(pool.get_scheduler(), counters)); + } catch (...) { + counters.unexpectedFailures_.fetch_add(1); + } + + const auto elapsed = Clock::now() - start; + return Metrics { + std::move(scenario), + pool.thread_count(), + operations, + counters.successes_.load(), + counters.unexpectedFailures_.load(), + counters.maxConcurrency_.load(), + std::chrono::duration { elapsed }.count() + }; +} + +void print_metrics(const Metrics& metrics) { + const double throughput = metrics.elapsedMilliseconds_ > 0.0 + ? static_cast(metrics.operations_) * 1'000.0 / + metrics.elapsedMilliseconds_ + : 0.0; + + std::println( + "{},{},{},{},{},{},{:.3f},{:.1f},{}", + metrics.scenario_, + metrics.workerCount_, + metrics.operations_, + metrics.successes_, + metrics.unexpectedFailures_, + metrics.maxConcurrency_, + metrics.elapsedMilliseconds_, + throughput, + metrics.passed() ? "PASS" : "FAIL"); +} + +[[nodiscard]] std::vector worker_counts() { + const auto hardware = std::max( + 1, + std::thread::hardware_concurrency()); + const auto maximum = std::min(hardware, 8); + std::vector counts {}; + + for (const std::size_t candidate : { 1U, 2U, 4U, 8U }) { + if (candidate <= maximum) { + counts.push_back(candidate); + } + } + + if (counts.back() != maximum) { + counts.push_back(maximum); + } + + return counts; +} + +[[nodiscard]] std::uint64_t expected_checksum() noexcept { + std::uint64_t checksum {}; + + for (int index { 0 }; index < CPU_TASK_COUNT; ++index) { + checksum ^= compute_value(index); + } + + return checksum; +} + +} // namespace + +int main() { + const auto expectedChecksum = expected_checksum(); + std::vector results {}; + + for (const auto workerCount : worker_counts()) { + ThreadPool pool { workerCount }; + + results.emplace_back(measure( + "cpu_chunks", + pool, + CPU_TASK_COUNT, + [&](Scheduler scheduler, Counters& counters) { + return run_compute( + scheduler, + counters, + expectedChecksum); + })); + results.emplace_back(measure( + "schedule_hops", + pool, + SCHEDULE_HOPS, + run_schedule_hops)); + results.emplace_back(measure( + "nested_fanout", + pool, + NESTED_TASKS, + run_nested_fanout)); + results.emplace_back(measure( + "concurrent_reschedule", + pool, + PRODUCER_COUNT * HOPS_PER_PRODUCER, + run_concurrent_producers)); + } + + std::println( + "environment,hardware_threads={}", + std::thread::hardware_concurrency()); + std::println( + "scenario,workers,operations,successes,unexpected_failures,max_concurrency,elapsed_ms,ops_per_second,status"); + + for (const auto& result : results) { + print_metrics(result); + } + + return std::ranges::all_of(results, &Metrics::passed) ? 0 : 1; +} diff --git a/docs/architecture.md b/docs/architecture.md index 4753e15..d456c99 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -7,20 +7,21 @@ CMP is a C++23 module project with a small coroutine execution core. The root module exports a lazy, single-consumer `mcpplibs::cmp::Task`, structured variadic/vector `when_all()`, eager `TaskGroup`, allocation-free `OneShotEvent`, reusable `AsyncManualResetEvent`, RAII `AsyncMutex`, -`RunLoop`, and its copyable `Scheduler` handle. Join primitives own every child until completion, -while the events publish external signals. +caller-thread `RunLoop`, fixed-size CPU `ThreadPool`, and their copyable Scheduler handles. Join +primitives own every child until completion, while the events publish external signals. `RunLoop::run()` is the public root execution boundary, while `Scheduler::schedule()` explicitly returns a suspended coroutine to that loop. `schedule()`, `schedule_after()`, and `schedule_at()` have `std::stop_token` overloads for cooperative cancellation; timed scheduling uses relative and -absolute `steady_clock` deadlines without a timer thread. +absolute `steady_clock` deadlines without a timer thread. `ThreadPool::Scheduler::schedule()` +explicitly transfers a continuation to any fixed worker and has the same cancellation-winner rule. The repository contains: - one mcpp package manifest; -- the root module `mcpplibs.cmp` with Task, cancellation, RunLoop, join, event, and mutex partitions; +- the root module `mcpplibs.cmp` with Task, cancellation, executor, join, event, and mutex partitions; - gtest contract, lifetime, exception, scheduling, and threading tests; - one standalone path-dependency example; -- one local POSIX pressure consumer for compute, file, and loopback network integration; +- local v1-readiness and cross-platform ThreadPool benchmark consumers; - Linux, macOS, and Windows CI workflows. ## Package and module identity @@ -69,6 +70,7 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. │ ├── task.cppm │ ├── cancellation.cppm │ ├── run_loop.cppm +│ ├── thread_pool.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -77,6 +79,7 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. ├── tests/ │ ├── cmp_test.cpp │ ├── run_loop_test.cpp +│ ├── thread_pool_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -85,13 +88,16 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. ├── benchmarks/v1-readiness/ │ ├── mcpp.toml │ └── src/main.cpp +├── benchmarks/thread-pool/ +│ ├── mcpp.toml +│ └── src/main.cpp └── mcpp.toml ``` ## Build and tests `.xlings.json` pins the mcpp version used by the project. `mcpp build` builds the inferred library -target. `mcpp test` discovers seven test files and links a gtest entry point for each. The 90 tests +target. `mcpp test` discovers eight test files and links a gtest entry point for each. The 105 tests verify Task ownership and symmetric transfer together with structured joins, root execution, scheduling, exception propagation, timed and cross-thread wake-up, cancellation races, invalid scheduler use, loop reuse, and stack-safe repeated completion. @@ -134,6 +140,7 @@ using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -166,6 +173,16 @@ Task print_concurrent_results(RunLoop::Scheduler scheduler) { co_return; } +Task print_worker_result( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const int result = 21 * 2; + co_await caller.schedule(); + std::println("Worker pool result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -234,9 +251,13 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke } int main() { + ThreadPool workers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); + loop.run(print_worker_result( + workers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -251,7 +272,9 @@ This example checks path dependency resolution, module consumption, external cor compilation, and the public root runner independently of the root test targets. The RunLoop drives `print_answer()` on the main thread; a short monotonic timer expires before the coroutine prints `Coroutine result: 42`. The next root Task concurrently joins two timed values and prints -`Concurrent result: 42`. The next coroutine eagerly spawns and joins two void Tasks before printing +`Concurrent result: 42`. A worker-pool coroutine computes away from the caller, explicitly returns +to the RunLoop, and prints `Worker pool result: 42`. The next coroutine eagerly spawns and joins two +void Tasks before printing `Task group result: 42`; a recursively growing group then prints `Recursive group result: 3`. Other coroutines print `Event signalled`, exercise two reusable-event cycles, and print `Reusable event cycles: 2`. Two guarded Tasks produce `Mutex result: 42`. A final structured group @@ -362,6 +385,23 @@ the continuation to its RunLoop. A Task that suspends without arranging another source to resume it can leave `run()` blocked indefinitely. Blocking functions still block the thread on which the coroutine currently executes. +`ThreadPool` and its Scheduler have the following contract: + +- construction starts a fixed worker count; the default normalizes unknown hardware concurrency + to one, while an explicit zero throws `std::invalid_argument`; +- the pool is immovable and its weak, copyable Scheduler identifies that pool without extending its + lifetime; +- `schedule()` always suspends and enqueues in one shared FIFO; any worker may resume it, and idle + workers sleep instead of spinning; +- `schedule(stop_token)` queues even when pre-cancelled; worker claim and cancellation select one + atomic outcome, and user code is resumed only by a worker; +- user continuations are resumed outside the queue mutex, and every accepted entry wakes a worker; +- destruction closes admission, drains accepted entries, joins all workers, and rejects later + scheduling with `std::logic_error`; +- destroying the pool on one of its own workers is an invalid lifetime arrangement and terminates; +- the pool owns threads, not Tasks, and provides no timer, blocking-I/O adaptation, detached work, + resize, priority, or affinity API. + There is no public free-standing `sync_wait`, detached execution, standalone Timer handle, asynchronous I/O backend, custom frame allocator, or blocking-work pool. Cancellation remains explicit: Scheduler waits and `AsyncManualResetEvent` accept tokens, and TaskGroup owns an optional @@ -383,13 +423,14 @@ package contract: 1. TaskGroup result handles and additional cancellation-aware primitives; 2. channels and additional structured wake-up paths; -3. multi-worker scheduling and work stealing; +3. profile-guided work stealing if representative workloads justify it; 4. asynchronous I/O integrations; 5. a dedicated pool for unavoidable blocking work; 6. result adapters and optional coroutine-frame allocation strategies. -Task, cancellation, RunLoop, `when_all`, TaskGroup, OneShotEvent, AsyncManualResetEvent, and -AsyncMutex occupy separate module partitions because they are implemented public boundaries. +Task, cancellation, RunLoop, ThreadPool, `when_all`, TaskGroup, OneShotEvent, +AsyncManualResetEvent, and AsyncMutex occupy separate module partitions because they are +implemented public boundaries. Further partitions or implementation units are added only when another implemented API needs them. ## Verification @@ -405,14 +446,16 @@ cd examples/basic mcpp run ``` -The expected result is a successful library build, 90 passing tests across seven binaries, and an -example that prints `Coroutine result: 42`, `Concurrent result: 42`, `Task group result: 42`, +The expected result is a successful library build, 105 passing tests across eight binaries, and an +example that prints `Coroutine result: 42`, `Concurrent result: 42`, `Worker pool result: 42`, `Task group result: 42`, `Recursive group result: 3`, `Event signalled`, `Reusable event cycles: 2`, `Mutex result: 42`, then `Coroutine cancelled` and exits with status 0. Tests retain the existing high-volume stack checks and add 20,000 recursive TaskGroup admissions, 100,000 pre-cancelled ready schedules, 50,000 manual event waiters, 20,000 nested reusable-event signals, and set/cancel races. Focused phase-4 race suites pass repeated Release runs. Compute, temporary-file, and loopback-network counts and throughput are recorded in the -[v1 readiness benchmark](benchmarks/2026-08-29-cmp-v1-readiness.md). The current Windows LLVM toolchain +[v1 readiness benchmark](benchmarks/2026-08-29-cmp-v1-readiness.md). ThreadPool counts, concurrency, +and five-round Release measurements are recorded in the +[ThreadPool benchmark](benchmarks/2026-08-29-cmp-thread-pool.md). The current Windows LLVM toolchain does not emit GNU depfiles. If a file included by a module interface changes, an incremental build can reuse an older BMI or object; `--cache=off` is used for a full local verification. diff --git a/docs/architecture.zh.hant.md b/docs/architecture.zh.hant.md index aff3fa2..5001f20 100644 --- a/docs/architecture.zh.hant.md +++ b/docs/architecture.zh.hant.md @@ -5,21 +5,23 @@ ## 目前狀態 CMP 是一個具備小型協程執行核心的 C++23 模組專案。根模組匯出延遲啟動、單一消費者的 -`mcpplibs::cmp::Task`、結構化變參/vector `when_all()`、eager `TaskGroup`、`RunLoop` 及其 -可複製的 `Scheduler` 控制代碼、無分配 `OneShotEvent`、可複用 -`AsyncManualResetEvent` 和 RAII `AsyncMutex`。匯合原語會持有每個子任務直到結束,事件負責 -發布外部訊號。`RunLoop::run()` 是公開根任務執行邊界,`Scheduler::schedule()` 用於明確地 +`mcpplibs::cmp::Task`、結構化變參/vector `when_all()`、eager `TaskGroup`、呼叫執行緒 +`RunLoop`、固定大小的 CPU `ThreadPool` 及其可複製 Scheduler 控制代碼、無分配 +`OneShotEvent`、可複用 `AsyncManualResetEvent` 和 RAII `AsyncMutex`。匯合原語會持有每個 +子任務直到結束,事件負責發布外部訊號。`RunLoop::run()` 是公開根任務執行邊界, +`Scheduler::schedule()` 用於明確地 把暫停協程送回對應執行迴圈。`schedule()`、`schedule_after()` 和 `schedule_at()` 都有接受 `std::stop_token` 的協作式取消多載;定時排程使用相對和絕對的 `steady_clock` 期限,且不建立 -計時執行緒。 +計時執行緒。`ThreadPool::Scheduler::schedule()` 會把 continuation 明確轉移到任意固定 +worker,並採用相同的取消獲勝規則。 儲存庫現有內容包括: - 一份 mcpp 套件清單; -- 根模組 `mcpplibs.cmp` 及 Task、cancellation、RunLoop、join、event、mutex 模組分割區; +- 根模組 `mcpplibs.cmp` 及 Task、cancellation、執行器、join、event、mutex 模組分割區; - 涵蓋契約、生命週期、例外、排程和執行緒行為的 gtest 測試; - 一個透過路徑相依使用根套件的獨立範例; -- 一個驗證計算、檔案與回環網路整合的本機 POSIX 壓測 consumer; +- v1 可開發性壓測和跨平台 ThreadPool 壓測 consumer; - Linux、macOS 和 Windows 三套 CI 工作流程。 ## 套件和模組識別 @@ -66,6 +68,7 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m │ ├── task.cppm │ ├── cancellation.cppm │ ├── run_loop.cppm +│ ├── thread_pool.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -74,6 +77,7 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m ├── tests/ │ ├── cmp_test.cpp │ ├── run_loop_test.cpp +│ ├── thread_pool_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -82,13 +86,16 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m ├── benchmarks/v1-readiness/ │ ├── mcpp.toml │ └── src/main.cpp +├── benchmarks/thread-pool/ +│ ├── mcpp.toml +│ └── src/main.cpp └── mcpp.toml ``` ## 建置與測試 `.xlings.json` 固定專案使用的 mcpp 版本。`mcpp build` 建置自動推斷的函式庫目標。 -`mcpp test` 會找到七個測試檔案,並為每個檔案連結 gtest 進入點。90 項測試同時驗證 Task 所有權 +`mcpp test` 會找到八個測試檔案,並為每個檔案連結 gtest 進入點。105 項測試同時驗證 Task 所有權 和對稱轉移、結構化匯合,以及根任務執行、普通與定時排程、例外傳播、跨執行緒期限喚醒、 無效 Scheduler、取消競態、RunLoop 重複使用和不會增長呼叫堆疊的重複完成。 @@ -128,6 +135,7 @@ using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -160,6 +168,16 @@ Task print_concurrent_results(RunLoop::Scheduler scheduler) { co_return; } +Task print_worker_result( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const int result = 21 * 2; + co_await caller.schedule(); + std::println("Worker pool result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -228,9 +246,13 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke } int main() { + ThreadPool workers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); + loop.run(print_worker_result( + workers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -242,7 +264,8 @@ int main() { ``` 這個範例在根測試目標之外,單獨檢查路徑相依解析、模組使用、外部協程編譯和公開根任務 -驅動器。RunLoop 依序輸出 `Coroutine result: 42`、`Concurrent result: 42` 和 +驅動器。RunLoop 依序輸出 `Coroutine result: 42` 和 `Concurrent result: 42`;worker pool +協程離開呼叫執行緒完成計算,明確回到 RunLoop 後輸出 `Worker pool result: 42`,隨後輸出 `Task group result: 42`;遞迴增長的 group 輸出 `Recursive group result: 3`。其他協程展示 一次性及兩輪可複用事件並輸出 `Event signalled`、`Reusable event cycles: 2`,兩個受保護 Task 輸出 `Mutex result: 42`。最後一個結構化 group 使用 `cancel_and_join()`,從可取消就緒 @@ -345,6 +368,20 @@ Task;明確等待原 Scheduler 才會把 continuation 送回對應 RunLoop。 安排其他執行緒或事件來源恢復它,`run()` 可能無限等待。阻塞函式仍會阻塞協程目前所在的 執行緒。 +`ThreadPool` 及其 Scheduler 遵循以下契約: + +- 建構時啟動固定 worker 數;預設值在硬體並行數未知時正規化為一,明確傳零拋出 + `std::invalid_argument`; +- pool 不可移動;弱參考、可複製的 Scheduler 識別原 pool,但不延長其生命週期; +- `schedule()` 始終暫停並進入一個共享 FIFO,任意 worker 都可恢復,閒置 worker 會休眠; +- `schedule(stop_token)` 即使預先取消也會排隊;worker claim 與取消以原子方式選擇一個終態, + 使用者程式碼只由 worker 恢復; +- continuation 在佇列鎖外恢復,每個已接納項目都會喚醒一個 worker; +- 解構關閉接納、排空已接納項目並 join 所有 worker;之後排程拋出 `std::logic_error`; +- 在 pool 自己的 worker 中解構是無效生命週期安排,會立即終止; +- pool 只擁有執行緒、不擁有 Task,也不提供 timer、阻塞 I/O 適配、detached、resize、優先級 + 或 affinity API。 + 目前沒有公開自由函式 `sync_wait`、detached 執行、獨立 Timer 控制代碼、非同步 I/O 後端、 自訂協程框架 allocator 或阻塞工作執行緒池。取消仍是明確的:Scheduler 等待和 `AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模組不提供隱式 @@ -363,14 +400,14 @@ CMP 名稱中的 `C` 與 Go 執行期中的 `G` 相呼應,但這只說明命 1. TaskGroup 結果控制代碼及更多支援取消的原語; 2. channel 和更多結構化喚醒路徑; -3. 多工作執行緒排程和工作竊取; +3. 代表性負載證明有必要時再加入 profiling 驅動的工作竊取; 4. 非同步 I/O 整合; 5. 處理無法避免之阻塞工作的專用執行緒池; 6. 結果適配器和可選的協程框架配置策略。 -Task、cancellation、RunLoop、`when_all`、TaskGroup、OneShotEvent、AsyncManualResetEvent 與 -AsyncMutex 已形成真實的公開邊界,因此分別位於模組分割區中。只有其他已實作 API 確實需要 -新邊界時,才繼續增加模組分割區或實作單元。 +Task、cancellation、RunLoop、ThreadPool、`when_all`、TaskGroup、OneShotEvent、 +AsyncManualResetEvent 與 AsyncMutex 已形成真實的公開邊界,因此分別位於模組分割區中。 +只有其他已實作 API 確實需要新邊界時,才繼續增加模組分割區或實作單元。 ## 驗證 @@ -385,13 +422,14 @@ cd examples/basic mcpp run ``` -預期結果是函式庫建置成功、七個二進位檔中的 90 項測試全部通過,而且範例依序輸出 -`Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42`、 +預期結果是函式庫建置成功、八個二進位檔中的 105 項測試全部通過,而且範例依序輸出 +`Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42`、 `Recursive group result: 3`、`Event signalled`、`Reusable event cycles: 2`、 `Mutex result: 42` 和 `Coroutine cancelled` 後以狀態 0 結束。測試保留原有高容量堆疊安全 檢查,並增加兩萬次 TaskGroup 遞迴接納、十萬次預取消就緒排程、五萬個 manual event 等待者、 兩萬次巢狀可複用事件訊號及 set/cancel 競態;第四階段重點競態套件已連續執行多輪 Release 測試。計算、臨時檔案和回環網路的成功/失敗計數及吞吐記錄在 -[v1 可開發性壓測](benchmarks/2026-08-29-cmp-v1-readiness.md)。目前 Windows LLVM 工具鏈 +[v1 可開發性壓測](benchmarks/2026-08-29-cmp-v1-readiness.md)。ThreadPool 的計數、並行和五輪 +Release 資料記錄在[執行緒池壓測](benchmarks/2026-08-29-cmp-thread-pool.md)。目前 Windows LLVM 工具鏈 不會產生 GNU depfile;如果模組介面包含的檔案發生變更,增量建置可能沿用舊的 BMI 或 目的檔。完整複驗時使用 `--cache=off`。 diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 6856091..86c57bb 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -5,21 +5,23 @@ ## 当前状态 CMP 是一个具备小型协程执行核心的 C++23 模块项目。根模块导出懒启动、单消费者的 -`mcpplibs::cmp::Task`、结构化变参/vector `when_all()`、eager `TaskGroup`、`RunLoop` 及其 -可复制的 `Scheduler` 句柄、无分配 `OneShotEvent`、可复用 `AsyncManualResetEvent` 和 RAII -`AsyncMutex`。汇合原语会持有每个子任务直到结束,事件负责发布外部信号。 +`mcpplibs::cmp::Task`、结构化变参/vector `when_all()`、eager `TaskGroup`、调用线程 +`RunLoop`、固定大小的 CPU `ThreadPool` 及其可复制 Scheduler 句柄、无分配 `OneShotEvent`、 +可复用 `AsyncManualResetEvent` 和 RAII `AsyncMutex`。汇合原语会持有每个子任务直到结束, +事件负责发布外部信号。 `RunLoop::run()` 是公共根任务执行边界,`Scheduler::schedule()` 用于显式地把挂起协程送回 对应运行循环。`schedule()`、`schedule_after()` 和 `schedule_at()` 都有接受 `std::stop_token` 的协作式取消重载;定时调度使用相对和绝对的 `steady_clock` 期限,且不创建 -定时线程。 +定时线程。`ThreadPool::Scheduler::schedule()` 会把 continuation 显式转移到任意固定 worker, +并采用相同的取消获胜规则。 仓库现有内容包括: - 一份 mcpp 包清单; -- 根模块 `mcpplibs.cmp` 及 Task、cancellation、RunLoop、join、event、mutex 模块分区; +- 根模块 `mcpplibs.cmp` 及 Task、cancellation、执行器、join、event、mutex 模块分区; - 覆盖契约、生命周期、异常、调度和线程行为的 gtest 测试; - 一个通过路径依赖使用根包的独立示例; -- 一个验证计算、文件与回环网络集成的本地 POSIX 压测 consumer; +- v1 可开发性压测和跨平台 ThreadPool 压测 consumer; - Linux、macOS 和 Windows 三套 CI 工作流。 ## 包和模块标识 @@ -66,6 +68,7 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp │ ├── task.cppm │ ├── cancellation.cppm │ ├── run_loop.cppm +│ ├── thread_pool.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -74,6 +77,7 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp ├── tests/ │ ├── cmp_test.cpp │ ├── run_loop_test.cpp +│ ├── thread_pool_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -82,13 +86,16 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp ├── benchmarks/v1-readiness/ │ ├── mcpp.toml │ └── src/main.cpp +├── benchmarks/thread-pool/ +│ ├── mcpp.toml +│ └── src/main.cpp └── mcpp.toml ``` ## 构建与测试 `.xlings.json` 固定项目使用的 mcpp 版本。`mcpp build` 构建自动推断的库目标。 -`mcpp test` 发现七个测试文件,并为每个文件链接 gtest 入口。90 项测试同时验证 Task 所有权和 +`mcpp test` 发现八个测试文件,并为每个文件链接 gtest 入口。105 项测试同时验证 Task 所有权和 对称转移、结构化汇合,以及根任务执行、普通与定时调度、异常传播、跨线程期限唤醒、无效 Scheduler、取消竞态、RunLoop 复用和不会增长调用栈的重复完成。 @@ -128,6 +135,7 @@ using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -160,6 +168,16 @@ Task print_concurrent_results(RunLoop::Scheduler scheduler) { co_return; } +Task print_worker_result( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const int result = 21 * 2; + co_await caller.schedule(); + std::println("Worker pool result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -228,9 +246,13 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke } int main() { + ThreadPool workers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); + loop.run(print_worker_result( + workers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -242,7 +264,8 @@ int main() { ``` 该示例在根测试目标之外,单独检查路径依赖解析、模块使用、外部协程编译和公共根任务驱动器。 -RunLoop 依次输出 `Coroutine result: 42`、`Concurrent result: 42` 和 `Task group result: 42`; +RunLoop 依次输出 `Coroutine result: 42` 和 `Concurrent result: 42`;worker pool 协程离开调用 +线程完成计算,显式回到 RunLoop 后输出 `Worker pool result: 42`,随后输出 `Task group result: 42`; 递归增长的 group 输出 `Recursive group result: 3`。其他协程演示一次性及两轮可复用事件并 输出 `Event signalled`、`Reusable event cycles: 2`,两个受保护 Task 输出 `Mutex result: 42`。最后一个结构化 group 使用 `cancel_and_join()`,从可取消就绪调度捕获 @@ -344,6 +367,20 @@ RunLoop 不拥有工作线程,也不提供自动线程亲和。外部 awaiter 显式等待原 Scheduler 才会把 continuation 送回对应 RunLoop。如果 Task 挂起后没有安排 其他线程或事件源恢复它,`run()` 可能无限等待。阻塞函数仍会阻塞协程当前所在的线程。 +`ThreadPool` 及其 Scheduler 遵循以下契约: + +- 构造时启动固定 worker 数;默认值在硬件并发数未知时归一为一,显式传零抛出 + `std::invalid_argument`; +- pool 不可移动;弱引用、可复制的 Scheduler 标识原 pool,但不延长其生命周期; +- `schedule()` 始终挂起并进入一个共享 FIFO,任意 worker 都可恢复,空闲 worker 会休眠; +- `schedule(stop_token)` 即使预先取消也会排队;worker claim 与取消原子地选择一个终态, + 用户代码只由 worker 恢复; +- continuation 在队列锁外恢复,每个已接纳条目都会唤醒一个 worker; +- 析构关闭接纳、排空已接纳条目并 join 所有 worker;之后调度抛出 `std::logic_error`; +- 在 pool 自己的 worker 中析构是无效生命周期安排,会立即终止; +- pool 只拥有线程、不拥有 Task,也不提供 timer、阻塞 I/O 适配、detached、resize、优先级或 + affinity API。 + 目前没有公共自由函数 `sync_wait`、detached 执行、独立 Timer 句柄、异步 I/O 后端、自定义 协程帧 allocator 或阻塞任务线程池。取消仍是显式的:Scheduler 等待和 `AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模块不提供隐式传播, @@ -361,14 +398,14 @@ CMP 名称中的 `C` 与 Go 运行时中的 `G` 相呼应,但这只说明命 1. TaskGroup 结果句柄及更多支持取消的原语; 2. channel 和更多结构化唤醒路径; -3. 多工作线程调度和工作窃取; +3. 代表性负载证明有必要时再加入 profiling 驱动的工作窃取; 4. 异步 I/O 集成; 5. 处理不可避免的阻塞工作的专用线程池; 6. 结果适配器和可选的协程帧分配策略。 -Task、cancellation、RunLoop、`when_all`、TaskGroup、OneShotEvent、AsyncManualResetEvent 与 -AsyncMutex 已经形成真实的公共边界,因此分别位于模块分区中。只有其他已实现 API 确实需要 -新边界时,才继续增加模块分区或实现单元。 +Task、cancellation、RunLoop、ThreadPool、`when_all`、TaskGroup、OneShotEvent、 +AsyncManualResetEvent 与 AsyncMutex 已经形成真实的公共边界,因此分别位于模块分区中。 +只有其他已实现 API 确实需要新边界时,才继续增加模块分区或实现单元。 ## 验证 @@ -383,13 +420,14 @@ cd examples/basic mcpp run ``` -预期结果是库构建成功、七个二进制中的 90 项测试全部通过,并且示例依次输出 -`Coroutine result: 42`、`Concurrent result: 42`、`Task group result: 42`、 +预期结果是库构建成功、八个二进制中的 105 项测试全部通过,并且示例依次输出 +`Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42`、 `Recursive group result: 3`、`Event signalled`、`Reusable event cycles: 2`、 `Mutex result: 42` 和 `Coroutine cancelled` 后以状态 0 退出。测试保留原有高容量栈安全检查, 并增加两万次 TaskGroup 递归接纳、十万次预取消就绪调度、五万个 manual event 等待者、两万次 嵌套可复用事件信号及 set/cancel 竞态;第四阶段重点竞态套件已连续执行多轮 Release 测试。 计算、临时文件和回环网络的成功/失败计数及吞吐记录在 -[v1 可开发性压测](benchmarks/2026-08-29-cmp-v1-readiness.md)。当前 Windows LLVM 工具链不会 +[v1 可开发性压测](benchmarks/2026-08-29-cmp-v1-readiness.md)。ThreadPool 的计数、并发和五轮 +Release 数据记录在[线程池压测](benchmarks/2026-08-29-cmp-thread-pool.md)。当前 Windows LLVM 工具链不会 生成 GNU depfile;如果模块接口包含的文件发生变化,增量构建可能复用旧的 BMI 或目标文件。 完整复验时使用 `--cache=off`。 diff --git a/docs/benchmarks/2026-08-29-cmp-thread-pool.md b/docs/benchmarks/2026-08-29-cmp-thread-pool.md new file mode 100644 index 0000000..04bb97a --- /dev/null +++ b/docs/benchmarks/2026-08-29-cmp-thread-pool.md @@ -0,0 +1,67 @@ +# CMP ThreadPool 本地压测 + +**日期:** 2026-08-29 +**结论:** 五轮共 8,410,240 次操作全部成功,非预期失败为 0。粗粒度 CPU 任务从 1 个 +worker 扩展到 8 个 worker 时取得约 7.56 倍加速;微小 continuation 在多个 worker 争用共享 +FIFO 时明显退化,因此当前数据支持保留简单的 v1 实现,不支持立即加入 work stealing。 + +## 环境与方法 + +- WSL2 Linux,AMD Ryzen 7 7735H,`hardware_concurrency() == 16`; +- mcpp 2026.8.11.2,LLVM 22.1.8,C++23 Modules; +- Release、strict、关闭构建缓存; +- 依次使用 1、2、4、8 个 worker,每种场景连续执行五轮; +- 计时使用 `std::chrono::steady_clock`,吞吐仅作为本机回归基线,不作为 CI 阈值。 + +构建和运行: + +```text +cd benchmarks/thread-pool +mcpp build --profile release --strict --cache=off +./target///bin/cmp-thread-pool-benchmark +``` + +四个场景分别是: + +1. `cpu_chunks`:512 个独立任务,每个执行 200,000 步整数计算; +2. `schedule_hops`:单个协程连续执行 200,000 次调度; +3. `nested_fanout`:worker 内创建并汇合 20,000 个一次性子任务; +4. `concurrent_reschedule`:8 个生产者各连续调度 25,000 次。 + +## 五轮结果 + +下表的时间列保留五轮原始耗时,吞吐为五轮中位数。`最大并发` 是五轮中观察到的最大值; +单协程 `schedule_hops` 的并发恒为 1,符合预期。 + +| 场景 | worker | 每轮操作数 | 五轮耗时(ms) | 中位耗时(ms) | 中位吞吐(op/s) | 最大并发 | +|---|---:|---:|---|---:|---:|---:| +| cpu_chunks | 1 | 512 | 149.914 / 151.185 / 147.880 / 149.350 / 165.922 | 149.914 | 3,415.3 | 1 | +| cpu_chunks | 2 | 512 | 75.478 / 75.621 / 74.722 / 76.830 / 77.553 | 75.621 | 6,770.6 | 2 | +| cpu_chunks | 4 | 512 | 37.737 / 38.632 / 38.151 / 39.178 / 40.809 | 38.632 | 13,253.3 | 4 | +| cpu_chunks | 8 | 512 | 21.243 / 19.946 / 19.750 / 19.821 / 19.664 | 19.821 | 25,831.2 | 8 | +| schedule_hops | 1 | 200,000 | 7.142 / 7.230 / 9.352 / 7.821 / 8.981 | 7.821 | 25,571,320.8 | 1 | +| schedule_hops | 2 | 200,000 | 20.719 / 17.935 / 22.690 / 17.047 / 17.673 | 17.935 | 11,151,572.1 | 1 | +| schedule_hops | 4 | 200,000 | 44.360 / 40.990 / 43.829 / 44.037 / 46.958 | 44.037 | 4,541,630.6 | 1 | +| schedule_hops | 8 | 200,000 | 49.282 / 46.146 / 48.056 / 48.592 / 51.078 | 48.592 | 4,115,872.8 | 1 | +| nested_fanout | 1 | 20,000 | 5.456 / 5.845 / 4.973 / 5.162 / 6.384 | 5.456 | 3,665,898.1 | 1 | +| nested_fanout | 2 | 20,000 | 8.273 / 5.745 / 7.644 / 7.699 / 7.580 | 7.644 | 2,616,342.5 | 2 | +| nested_fanout | 4 | 20,000 | 238.627 / 392.955 / 308.099 / 404.197 / 254.094 | 308.099 | 64,914.1 | 3 | +| nested_fanout | 8 | 20,000 | 499.838 / 490.593 / 501.042 / 510.774 / 491.988 | 499.838 | 40,013.0 | 2 | +| concurrent_reschedule | 1 | 200,000 | 7.567 / 7.365 / 7.409 / 7.389 / 9.068 | 7.409 | 26,995,398.6 | 1 | +| concurrent_reschedule | 2 | 200,000 | 29.460 / 26.653 / 30.734 / 30.354 / 31.903 | 30.354 | 6,588,986.0 | 2 | +| concurrent_reschedule | 4 | 200,000 | 34.130 / 34.793 / 33.723 / 34.705 / 35.741 | 34.705 | 5,762,814.4 | 4 | +| concurrent_reschedule | 8 | 200,000 | 46.512 / 47.858 / 46.298 / 42.684 / 48.672 | 46.512 | 4,299,942.7 | 5 | + +每一行每一轮的成功数都等于操作数,`unexpected_failures == 0`,状态均为 `PASS`。 + +## 判断 + +`cpu_chunks` 的 1→8 worker 中位耗时从 149.914 ms 降至 19.821 ms,约为 7.56 倍加速, +证明固定线程池能够承担预期的有限 CPU 并行工作。相反,三个微任务场景受共享 mutex、条件变量 +唤醒和队列争用支配;增加 worker 并不会让单个 continuation 更便宜,嵌套 fan-out 在 4/8 +worker 时尤其明显。 + +这些数据确定了 v1 的边界,但还不能证明 work stealing 是正确升级路径:微任务还同时包含 +awaiter、结构化汇合和通知成本。只有代表性应用的 profiler 将共享队列识别为主要瓶颈,且 +原型能稳定改善结果时,才单独设计 per-worker 队列、窃取、公平性和关闭协议。当前不增加该 +复杂度。 diff --git a/docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md b/docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md new file mode 100644 index 0000000..fac5de2 --- /dev/null +++ b/docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md @@ -0,0 +1,82 @@ +# CMP Phase 5 Thread Pool v1 Plan + +**Date:** 2026-08-29 +**Design:** `docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md` +**Status:** Complete; local verification passed + +## Completion record + +The shared-FIFO `ThreadPool`, cancellation and shutdown protocol, 15 deterministic tests, basic +example, and standard-library-only benchmark are implemented. Dev and Release strict builds and +the complete 105-test suite pass; the focused Release suite passed 30 consecutive runs. Five +benchmark rounds completed 8,410,240 operations with zero unexpected failures. Coarse CPU work +scaled by about 7.56x from one to eight workers, while tiny-task contention confirmed the documented +v1 ceiling. Work stealing was therefore not added without a separate representative profiling case. + +## 1. Preserve and re-check the phase-4 baseline + +Before implementation, work from an integrated phase-4 baseline and run the complete Dev suite +with cache disabled. Do not mix phase-4 delivery changes or phase-6 I/O work into the thread-pool +change. + +## 2. Lock the public contract with focused tests + +Add `tests/thread_pool_test.cpp` with the smallest deterministic tests for construction, +thread-count/Scheduler identity, always-suspending scheduling, worker identity, one-worker FIFO +order, explicit return to RunLoop, parallel occupancy, values, exceptions, and expired Scheduler +behavior. + +Use standard latches/barriers and counts for concurrency properties. Do not use elapsed-time +speedups as correctness checks. + +## 3. Implement the shared-FIFO worker core + +Add `src/thread_pool.cppm` and export it from `src/cmp.cppm`. Use a fixed set of `std::jthread` +workers, one mutex/condition-variable/deque state, weak Scheduler handles, `notify_one()` for every +accepted entry, and resume only after releasing the state lock. + +Implement exception-safe partial construction, explicit-zero rejection, default-count +normalization, queue-allocation failure propagation, graceful accepted-entry draining, close/enqueue +linearization, and self-destruction detection. Do not introduce a generic executor or queue layer. + +## 4. Add cancellation and race coverage + +Implement `schedule(std::stop_token)` with one atomic outcome in the awaiter. Cancellation and +worker claim compete with compare-exchange; the worker always consumes and resumes the entry. + +Cover pre-cancel, queued cancel, late cancel, repeated claim races, concurrent producers, nested +worker submissions, graceful close, rejection, large queues, and stack safety. Re-run focused +Release tests enough times to expose missed wake-ups or double resumes. + +## 5. Add one developer-facing example + +Extend `examples/basic` with a short CPU calculation that schedules onto `ThreadPool`, explicitly +returns to the original `RunLoop`, and prints a deterministic result. Keep the example structured; +do not add detached work or blocking-I/O claims. + +## 6. Measure before considering work stealing + +Add a standalone standard-library-only Release benchmark for coarse CPU chunks, tiny schedule hops, +nested fan-out, and concurrent rescheduling at one and multiple worker counts. Validate counts and +unexpected failures, record environment and median results, and keep timing thresholds out of CI. + +If profiling does not identify the shared queue as the dominant scaling bottleneck, stop here and +record that work stealing is not justified. If it does, write and review a separate internal +work-stealing design before implementation; preserve the public API and all wake-up, fairness, +cancellation, and shutdown contracts. + +## 7. Synchronize documentation and verify delivery + +Update all three README and architecture variants, the benchmark report, and `.agent/HANDOFF.md` +with implemented facts only. Run: + +```text +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off +cd examples/basic && mcpp run +``` + +Then run the standalone Release pressure consumer for multiple measured rounds. Record any check +that cannot run; do not describe an unexecuted check as passing. diff --git a/docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md b/docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md new file mode 100644 index 0000000..b096845 --- /dev/null +++ b/docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md @@ -0,0 +1,321 @@ +# CMP Phase 5 Thread Pool v1 Design + +**Date:** 2026-08-29 +**Status:** Implemented and locally verified +**Baseline:** Phase 4 is implemented and locally verified with 90/90 tests + +## Purpose + +Add one fixed-size CPU worker pool that lets a coroutine explicitly move execution away from a +caller-thread `RunLoop`. Keep the developer-facing operation to one familiar expression: + +```cpp +co_await workers.schedule(); +``` + +Phase 5 does not make `Task` carry an implicit executor. A coroutine runs on the thread that resumes +it until it suspends or completes. It explicitly awaits another scheduler when it needs different +thread affinity. + +The existing roadmap combines multi-worker scheduling and work stealing. This design separates +those decisions. A shared FIFO is the phase-5 v1 implementation because it is work-conserving, +auditable, and sufficient to measure the real contention of CMP workloads. Work stealing remains +an internal optimization gate, not a public promise: it receives a separate design only if the v1 +benchmark identifies the shared queue as the limiting bottleneck. + +## Public API + +Add a `:thread_pool` module partition and export it from `mcpplibs.cmp`. The public surface is below; +private state constructors and storage are omitted: + +```cpp +namespace mcpplibs::cmp { + +class ThreadPool final { +public: + class Scheduler final; + + ThreadPool(); + explicit ThreadPool(std::size_t workerCount); + + ThreadPool(const ThreadPool&) = delete; + ThreadPool& operator=(const ThreadPool&) = delete; + ThreadPool(ThreadPool&&) = delete; + ThreadPool& operator=(ThreadPool&&) = delete; + + ~ThreadPool(); + + [[nodiscard]] std::size_t thread_count() const noexcept; + [[nodiscard]] Scheduler get_scheduler() const noexcept; +}; + +class ThreadPool::Scheduler final { +public: + Scheduler() = delete; + Scheduler(const Scheduler&) = default; + Scheduler& operator=(const Scheduler&) = default; + Scheduler(Scheduler&&) noexcept = default; + Scheduler& operator=(Scheduler&&) noexcept = default; + ~Scheduler() = default; + + [[nodiscard]] auto schedule() const noexcept; + [[nodiscard]] auto schedule(std::stop_token token) const noexcept; + + friend bool operator==( + const Scheduler& left, + const Scheduler& right) noexcept; +}; + +} // namespace mcpplibs::cmp +``` + +The default constructor uses `std::thread::hardware_concurrency()`, normalizing an unknown result +of zero to one worker. An explicitly requested count of zero is a caller error and throws +`std::invalid_argument`. Other thread-creation failures cleanly stop and join already-created +workers, then propagate the original exception. + +`ThreadPool` is immovable because it is the visible lifetime anchor for its threads and Scheduler +identity. Scheduler is a cheap copyable handle backed by a weak reference to the pool state, like +`RunLoop::Scheduler`. Two handles compare equal exactly when they refer to the same pool state, +including after that state has expired. + +No public `submit`, `spawn`, `detach`, `resize`, `close`, `join`, priority, affinity, or metrics API is +added. `Task`, `when_all()`, and `TaskGroup` remain the structured ownership layer. + +## Intended Use + +```cpp +Task calculate( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const auto result = expensive_calculation(); + + // Only needed when the following code requires caller-thread affinity. + co_await caller.schedule(); + co_return result; +} + +int main() { + ThreadPool workers {}; + RunLoop loop {}; + return loop.run(calculate( + workers.get_scheduler(), + loop.get_scheduler())); +} +``` + +Waiting on the pool always suspends, even when called by one of the pool's own workers. The resumed +coroutine may run on any worker, including the worker that submitted it. No stable-worker or +completion-order guarantee is exposed. + +CPU work should be split into independently schedulable, finite pieces. A coroutine is +non-preemptive between suspension points: a long computation monopolizes one worker, and a +blocking system call blocks that worker. Native asynchronous I/O and the separate pool for +unavoidable blocking work remain phase 6. + +## Queue and Worker Protocol + +The private shared state contains only: + +- one `std::mutex`; +- one `std::condition_variable`; +- one unbounded `std::deque` of ready entries; +- one `accepting` flag. + +Each entry contains a coroutine handle and, for cancellable waits, a pointer to the outcome stored +inside the awaiting coroutine frame. The queue does not allocate a separate node owned by CMP; +normal `std::deque` storage growth can still allocate and can propagate `std::bad_alloc` before the +operation is published. + +Enqueue is linearized under the state mutex: + +1. reject if the state is no longer accepting work; +2. append the entry to the FIFO; +3. unlock; +4. call `notify_one()` for every accepted entry. + +Every worker repeats: + +1. wait with the predicate `!queue.empty() || !accepting`; +2. if the queue is empty and admission is closed, exit; +3. pop the oldest entry while holding the mutex; +4. select the entry's completion outcome, if any; +5. unlock and resume the coroutine. + +User code is never resumed while the queue mutex is held. Predicate waits handle spurious wake-ups, +idle workers sleep instead of spinning, and worker-origin submissions wake sleeping siblings in +the same way as external submissions. FIFO describes dequeue order after concurrent producers are +linearized by the mutex; multiple workers can begin or finish those continuations in a different +order. + +The queue is deliberately unbounded. A bounded queue can deadlock when all workers are occupied by +tasks that need to enqueue more work before they can finish. Backpressure requires a separate +asynchronous admission contract and is not smuggled into `schedule()`. + +## Cancellation + +`schedule(std::stop_token)` follows the existing Scheduler contract: + +- it always queues while the pool is active, including for a pre-cancelled token; +- cancellation that wins before a worker claims the entry causes `await_resume()` to throw + `OperationCancelled`; +- once the worker claims completion, a later stop request does not replace success; +- cancellation never resumes user code on the requesting thread. + +The awaiter owns an atomic `pending/completed/cancelled` outcome and its `std::stop_callback`. +The callback performs only a compare-exchange from `pending` to `cancelled`. A worker performs the +competing compare-exchange from `pending` to `completed`, then resumes the continuation regardless +of which outcome won. A cancelled entry stays in the FIFO until consumed, avoiding queue scans, +iterator invalidation, and a second removal protocol. + +`await_suspend()` first locks the weak pool state, then installs the callback, then attempts enqueue. +This ordering makes pre-cancellation observable without bypassing the queue and ensures an enqueue +allocation failure or concurrent close publishes no dangling entry. + +`await_resume()` first destroys the stop callback, then reads the selected outcome. This prevents +the callback from accessing a destroyed awaiter. Once enqueue succeeds, another worker may resume +and destroy the awaiter before `await_suspend()` returns, so `await_suspend()` must not access any +awaiter member after publication. + +An expired or closing Scheduler is a lifetime error, not a cancellation result. It throws +`std::logic_error` from the await expression even when the supplied token is already stopped, +because the pool never accepted that operation. + +## Lifetime and Shutdown + +Construction starts a fixed set of `std::jthread` workers. The pool owns execution threads but does +not own the Tasks whose continuations it resumes. The `jthread` stop token is not the shutdown +predicate: workers exit only after admission is closed and the accepted queue is empty, so automatic +stop requests cannot discard work. + +Destruction performs graceful executor shutdown: + +1. close admission under the state mutex; +2. wake every sleeping worker; +3. drain all entries accepted before the close linearization point; +4. let workers exit once the closed queue is empty; +5. join every worker before releasing the state. + +A concurrent enqueue therefore has exactly two outcomes: it is accepted and drained, or it observes +closure and throws `std::logic_error`. Entries are never silently discarded. A continuation that +was accepted may complete, suspend onto another executor, or attempt another pool schedule. The +last case is rejected after closure. Pool destruction consequently guarantees that no pool worker +is running when destruction returns; it does not mean every higher-level Task has completed. +Structured owners must still await their Tasks. + +Destroying a pool from one of its own workers cannot satisfy the join guarantee and would self- +deadlock. This is an invalid lifetime arrangement and terminates immediately. Destruction can also +wait indefinitely if user code running on a worker blocks indefinitely; the scheduler cannot +preempt arbitrary C++ code. + +Scheduler handles may safely outlive the pool, but scheduling through them fails with +`std::logic_error`. An awaiting coroutine frame must remain alive until its accepted queue entry is +consumed. Existing CMP structured ownership satisfies this rule; phase 5 does not introduce a +detached escape hatch. + +## Integration with Existing CMP Semantics + +- `Task` promises do not gain scheduler storage or implicit propagation. +- A child Task and its parent continue on whichever thread completes the awaited child unless they + explicitly schedule elsewhere. +- `RunLoop::run()` remains the caller-thread root boundary and can wait while work executes in the + pool. +- `when_all()` and `TaskGroup` retain their cross-thread, exactly-once completion protocols and are + the primary way to own parallel work. +- Timers stay on `RunLoop::Scheduler`; `ThreadPool::Scheduler` has no `schedule_after()` or + `schedule_at()`. +- Events and `AsyncMutex` keep their documented resumption-thread behavior. Code that needs a + particular thread awaits that thread's Scheduler explicitly. + +The implementation is isolated in `src/thread_pool.cppm`; no generic executor base class, factory, +queue abstraction, or third-party dependency is introduced. + +## Validation Contract + +Cross-platform tests must cover: + +- default construction, an explicit count, and rejection of zero; +- thread-count reporting and Scheduler equality for same versus different pools; +- `schedule()` always suspending and resuming on a pool worker; +- deterministic parallel occupancy of at least two workers using a test barrier rather than timing; +- one-worker FIFO dequeue order and multi-worker exactly-once execution; +- worker-origin rescheduling and nested fan-out without lost wake-ups or deadlock; +- values and exceptions through `Task`, `when_all()`, `TaskGroup`, and `RunLoop::run()`; +- explicit return from the pool to the original RunLoop thread; +- normal, pre-cancelled, queued-cancelled, and late-cancelled scheduling; +- cancellation versus worker claim selecting exactly one outcome under repeated races; +- concurrent producers, large queues, repeated scheduling, and native-stack safety; +- graceful draining of accepted entries, rejection during/after close, and expired Scheduler use. + +Timing ratios are not correctness assertions. CI tests use latches, barriers, counts, and thread IDs +to establish observable contracts without depending on machine speed. Platform-specific death +testing is not required; self-destruction remains a documented hard lifetime invariant and the +implementation performs the check before joining. + +## Performance Measurement and Work-Stealing Gate + +Add one standalone Release benchmark consumer with no dependency beyond CMP and the standard +library. Record successful operation counts, unexpected failures, elapsed time, throughput, and +observed maximum concurrency for: + +1. coarse independent CPU chunks at one and multiple worker counts; +2. repeated tiny `schedule()` hops, which stress the shared queue; +3. nested worker-origin fan-out, which exercises wake-up and utilization; +4. concurrent rescheduling producers, which stresses queue contention. + +The benchmark reports raw data and does not fail CI on speedup thresholds. Correct counts and the +explicit multi-worker occupancy test are hard requirements; elapsed-time results are local +regression evidence. + +Work stealing is considered only when profiling a representative workload shows that the shared +queue is the dominant scaling limit and a prototype gives a repeatable material improvement. Any +replacement must preserve the public API and additionally prove: + +- local and external submissions always wake idle peers when runnable work exists; +- no task is lost during steal, cancellation, or shutdown races; +- external work cannot starve behind an endless local stream; +- idle workers do not spin indefinitely; +- accepted work is still drained on close; +- the full correctness and pressure suites remain clean. + +This gate directly avoids a known failure mode in which a local-queue submission does not wake +sleeping workers, leaving a nominally multi-threaded pool non-work-conserving. + +## Deliberately Excluded + +- transparent migration, preemption, cooperative time budgets, and task priorities; +- per-worker public handles, pinning, NUMA policy, resizing, and custom allocators; +- detached task ownership or fire-and-forget submission; +- bounded admission and backpressure; +- timers, native asynchronous I/O, and blocking-call adaptation; +- an initial work-stealing implementation without benchmark evidence. + +## Reference Rationale + +- [cppcoro `static_thread_pool`](https://github.com/lewissbaker/cppcoro/blob/master/lib/static_thread_pool.cpp) + demonstrates the small + `co_await pool.schedule()` surface, but its local/global queues, stealing, spinning, sleeping, and + wake-up protocol are substantially more complex than CMP needs before measurement. +- [stdexec issue #1305](https://github.com/NVIDIA/stdexec/issues/1305) documents a real + non-work-conserving work-stealing failure and contrasts it with libunifex's simpler shared locked + queue. Correct wake-up is therefore part of the contract, not a later tuning detail. +- [Boost.Asio `thread_pool`](https://www.boost.org/doc/libs/latest/doc/html/boost_asio/reference/thread_pool.html) + validates the familiar fixed-thread RAII execution-context shape. +- [Folly `CPUThreadPoolExecutor`](https://github.com/facebook/folly/blob/main/folly/executors/CPUThreadPoolExecutor.h) + uses an unbounded concurrent queue by default and warns about deadlocks caused by blocking bounded + queues when pool work recursively submits work. +- [Tokio runtime scheduling](https://docs.rs/tokio/latest/tokio/runtime/) shows that production work + stealing also needs local/global fairness, bounded local queues, wake-up rules, and cooperative + budgeting; it is not merely replacing one deque with several. +- [oneTBB task scheduler](https://oneapi-spec.uxlfoundation.org/specifications/oneapi/v1.1-rev-1/elements/onetbb/source/task_scheduler.html) + reinforces the CPU-oriented, non-preemptive boundary and the danger of blocking dependencies + inside a finite worker set. + +## Acceptance Criteria + +Phase-5 v1 is complete locally: the module, tests, example, benchmark report, and +English/Simplified-Chinese/Traditional-Chinese public documentation agree; Dev and Release strict +builds and all 105 tests pass; the standalone example succeeds; and five local Release benchmark +rounds contain zero unexpected failures. Remote three-platform CI remains delivery verification, +not a locally claimed result. diff --git a/examples/basic/src/main.cpp b/examples/basic/src/main.cpp index 919dbcf..4d218ee 100644 --- a/examples/basic/src/main.cpp +++ b/examples/basic/src/main.cpp @@ -8,6 +8,7 @@ using mcpplibs::cmp::AsyncMutex; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -40,6 +41,27 @@ Task print_concurrent_results(RunLoop::Scheduler scheduler) { co_return; } +Task calculate_on_workers( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + + int result { 0 }; + for (int value { 1 }; value <= 6; ++value) { + result += value * 2; + } + + co_await caller.schedule(); + co_return result; +} + +Task print_worker_result( + ThreadPool::Scheduler workers, + RunLoop::Scheduler caller) { + const auto result = co_await calculate_on_workers(workers, caller); + std::println("Worker pool result: {}", result); +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -178,9 +200,13 @@ Task print_cancellation(RunLoop::Scheduler scheduler) { } int main() { + ThreadPool workers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); + loop.run(print_worker_result( + workers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_recursive_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); diff --git a/src/cmp.cppm b/src/cmp.cppm index 1bf4292..d6cf754 100644 --- a/src/cmp.cppm +++ b/src/cmp.cppm @@ -3,6 +3,7 @@ export module mcpplibs.cmp; export import :cancellation; export import :task; export import :run_loop; +export import :thread_pool; export import :when_all; export import :task_group; export import :one_shot_event; diff --git a/src/thread_pool.cppm b/src/thread_pool.cppm new file mode 100644 index 0000000..6fd929f --- /dev/null +++ b/src/thread_pool.cppm @@ -0,0 +1,301 @@ +export module mcpplibs.cmp:thread_pool; + +import std; +import :cancellation; + +namespace mcpplibs::cmp::detail { + +enum class ThreadPoolOutcome { + pending, + completed, + cancelled +}; + +struct ThreadPoolCancellation final { + std::atomic outcome_ { ThreadPoolOutcome::pending }; +}; + +struct ThreadPoolCancelCallback final { + ThreadPoolCancellation* cancellation_ {}; + + void operator()() const noexcept { + auto expected = ThreadPoolOutcome::pending; + cancellation_->outcome_.compare_exchange_strong( + expected, + ThreadPoolOutcome::cancelled, + std::memory_order_acq_rel, + std::memory_order_acquire); + } +}; + +struct ThreadPoolEntry final { + std::coroutine_handle<> continuation_ {}; + ThreadPoolCancellation* cancellation_ {}; +}; + +class ThreadPoolState final { +private: + std::mutex mutex_ {}; + std::condition_variable condition_ {}; + // ponytail: v1 使用共享 FIFO;基准证明锁竞争后再评审分片队列。 + std::deque ready_ {}; + bool accepting_ { true }; + +public: + void enqueue( + std::coroutine_handle<> continuation, + ThreadPoolCancellation* cancellation = nullptr) { + if (!continuation) { + throw std::invalid_argument { "cannot schedule an empty coroutine" }; + } + + { + const std::lock_guard lock { mutex_ }; + + if (!accepting_) { + throw std::logic_error { "thread pool is stopping" }; + } + + ready_.push_back(ThreadPoolEntry { + continuation, + cancellation + }); + } + + condition_.notify_one(); + } + + void run_worker() noexcept { + while (true) { + ThreadPoolEntry entry {}; + + { + std::unique_lock lock { mutex_ }; + condition_.wait(lock, [&] { + return !ready_.empty() || !accepting_; + }); + + if (ready_.empty()) { + return; + } + + entry = ready_.front(); + ready_.pop_front(); + + if (entry.cancellation_) { + auto expected = ThreadPoolOutcome::pending; + entry.cancellation_->outcome_.compare_exchange_strong( + expected, + ThreadPoolOutcome::completed, + std::memory_order_acq_rel, + std::memory_order_acquire); + } + } + + if (!entry.continuation_ || entry.continuation_.done()) { + std::terminate(); + } + + // 锁外恢复,协程再次调度到当前池时不会自锁。 + entry.continuation_.resume(); + } + } + + void close() noexcept { + { + const std::lock_guard lock { mutex_ }; + accepting_ = false; + } + + condition_.notify_all(); + } +}; + +} // namespace mcpplibs::cmp::detail + +export namespace mcpplibs::cmp { + +class ThreadPool final { +public: + class Scheduler final { + private: + [[nodiscard]] static std::shared_ptr lock_state_( + const std::weak_ptr& state) { + const auto lockedState = state.lock(); + + if (!lockedState) { + throw std::logic_error { + "scheduler's thread pool no longer exists" + }; + } + + return lockedState; + } + + class ScheduleAwaiter final { + private: + std::weak_ptr state_ {}; + + public: + explicit ScheduleAwaiter( + std::weak_ptr state) noexcept + : state_ { std::move(state) } {} + + [[nodiscard]] constexpr bool await_ready() const noexcept { + return false; + } + + void await_suspend(std::coroutine_handle<> continuation) { + const auto state = Scheduler::lock_state_(state_); + state->enqueue(continuation); + } + + constexpr void await_resume() const noexcept {} + }; + + class CancellableScheduleAwaiter final { + private: + using StopCallback = + std::stop_callback; + + std::weak_ptr state_ {}; + std::stop_token stopToken_ {}; + detail::ThreadPoolCancellation cancellation_ {}; + // 最先析构,阻止回调继续访问 awaiter 内状态。 + std::optional stopCallback_ {}; + + public: + CancellableScheduleAwaiter( + std::weak_ptr state, + std::stop_token stopToken) noexcept + : state_ { std::move(state) }, + stopToken_ { std::move(stopToken) } {} + + [[nodiscard]] constexpr bool await_ready() const noexcept { + return false; + } + + void await_suspend(std::coroutine_handle<> continuation) { + const auto state = Scheduler::lock_state_(state_); + + stopCallback_.emplace( + stopToken_, + detail::ThreadPoolCancelCallback { &cancellation_ }); + + // 发布后 awaiter 可能立即销毁,此后不得再读取成员。 + state->enqueue(continuation, &cancellation_); + } + + void await_resume() { + stopCallback_.reset(); + + if (cancellation_.outcome_.load( + std::memory_order_acquire) == + detail::ThreadPoolOutcome::cancelled) { + throw OperationCancelled {}; + } + } + }; + + std::weak_ptr state_ {}; + + explicit Scheduler( + const std::shared_ptr& state) noexcept + : state_ { state } {} + + friend class ThreadPool; + + public: + Scheduler() = delete; + Scheduler(const Scheduler&) = default; + Scheduler& operator=(const Scheduler&) = default; + Scheduler(Scheduler&&) noexcept = default; + Scheduler& operator=(Scheduler&&) noexcept = default; + ~Scheduler() = default; + + [[nodiscard]] auto schedule() const noexcept { + return ScheduleAwaiter { state_ }; + } + + [[nodiscard]] auto schedule( + std::stop_token stopToken) const noexcept { + return CancellableScheduleAwaiter { + state_, + std::move(stopToken) + }; + } + + friend bool operator==( + const Scheduler& left, + const Scheduler& right) noexcept { + return !left.state_.owner_before(right.state_) && + !right.state_.owner_before(left.state_); + } + }; + +private: + std::shared_ptr state_ { + std::make_shared() + }; + // state_ 必须晚于 worker 析构,确保线程退出前共享状态仍然存在。 + std::vector workers_ {}; + + [[nodiscard]] static std::size_t default_worker_count_() noexcept { + const auto count = std::thread::hardware_concurrency(); + return count == 0 ? 1 : static_cast(count); + } + +public: + ThreadPool() + : ThreadPool { default_worker_count_() } {} + + explicit ThreadPool(std::size_t workerCount) { + if (workerCount == 0) { + throw std::invalid_argument { + "thread pool requires at least one worker" + }; + } + + workers_.reserve(workerCount); + + try { + for (std::size_t index { 0 }; index < workerCount; ++index) { + workers_.emplace_back([state = state_] { + state->run_worker(); + }); + } + } catch (...) { + // 先关闭并唤醒,随后 jthread 的成员析构会安全 join。 + state_->close(); + throw; + } + } + + ThreadPool(const ThreadPool&) = delete; + ThreadPool& operator=(const ThreadPool&) = delete; + ThreadPool(ThreadPool&&) = delete; + ThreadPool& operator=(ThreadPool&&) = delete; + + ~ThreadPool() { + const auto currentThread = std::this_thread::get_id(); + + if (std::ranges::any_of(workers_, [&](const std::jthread& worker) { + return worker.get_id() == currentThread; + })) { + std::terminate(); + } + + // jthread 的 stop token 不参与关闭;worker 会先排空已接纳任务。 + state_->close(); + } + + [[nodiscard]] std::size_t thread_count() const noexcept { + return workers_.size(); + } + + [[nodiscard]] Scheduler get_scheduler() const noexcept { + return Scheduler { state_ }; + } +}; + +} // namespace mcpplibs::cmp diff --git a/tests/thread_pool_test.cpp b/tests/thread_pool_test.cpp new file mode 100644 index 0000000..ffceaba --- /dev/null +++ b/tests/thread_pool_test.cpp @@ -0,0 +1,572 @@ +#include + +import std; +import mcpplibs.cmp; + +namespace { + +using mcpplibs::cmp::OperationCancelled; +using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::Task; +using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::when_all; + +using Scheduler = ThreadPool::Scheduler; + +using namespace std::chrono_literals; + +static_assert(std::default_initializable); +static_assert(!std::copy_constructible); +static_assert(!std::move_constructible); +static_assert(!std::is_copy_assignable_v); +static_assert(!std::is_move_assignable_v); + +static_assert(!std::default_initializable); +static_assert(std::copy_constructible); +static_assert(std::move_constructible); +static_assert(std::equality_comparable); + +struct CancellationRecord final { + bool cancelled_ { false }; + std::thread::id thread_ {}; +}; + +Task schedule_once(Scheduler scheduler) { + co_await scheduler.schedule(); + co_return std::this_thread::get_id(); +} + +Task schedule_uncaught(Scheduler scheduler) { + co_await scheduler.schedule(); +} + +Task value_on_pool(Scheduler scheduler, int value) { + co_await scheduler.schedule(); + co_return value; +} + +Task fail_on_pool(Scheduler scheduler) { + co_await scheduler.schedule(); + throw std::runtime_error { "pool failure" }; + co_return 0; +} + +Task> return_to_run_loop( + Scheduler workers, + RunLoop::Scheduler caller) { + co_await workers.schedule(); + const auto workerThread = std::this_thread::get_id(); + co_await caller.schedule(); + co_return std::pair { workerThread, std::this_thread::get_id() }; +} + +Task block_worker( + Scheduler scheduler, + std::counting_semaphore<1>& entered, + std::latch& release) { + co_await scheduler.schedule(); + entered.release(); + release.wait(); +} + +Task record_on_pool( + Scheduler scheduler, + int value, + std::vector& order) { + co_await scheduler.schedule(); + order.push_back(value); +} + +Task occupy_worker( + Scheduler scheduler, + std::counting_semaphore<2>& entered, + std::latch& release, + std::mutex& threadsMutex, + std::set& threads) { + co_await scheduler.schedule(); + + { + const std::lock_guard lock { threadsMutex }; + threads.insert(std::this_thread::get_id()); + } + + entered.release(); + release.wait(); +} + +Task run_nested_fanout( + Scheduler scheduler, + std::counting_semaphore<2>& entered, + std::latch& release, + std::mutex& threadsMutex, + std::set& threads) { + co_await scheduler.schedule(); + + TaskGroup group {}; + group.spawn(occupy_worker( + scheduler, + entered, + release, + threadsMutex, + threads)); + group.spawn(occupy_worker( + scheduler, + entered, + release, + threadsMutex, + threads)); + co_await group.join(); +} + +Task schedule_once( + Scheduler scheduler, + std::stop_token stopToken) { + try { + co_await scheduler.schedule(stopToken); + co_return CancellationRecord { + false, + std::this_thread::get_id() + }; + } catch (const OperationCancelled&) { + co_return CancellationRecord { + true, + std::this_thread::get_id() + }; + } +} + +Task observe_cancellation( + Scheduler scheduler, + std::stop_token stopToken, + bool& cancelled, + std::thread::id& resumedThread) { + try { + co_await scheduler.schedule(stopToken); + } catch (const OperationCancelled&) { + cancelled = true; + } + + resumedThread = std::this_thread::get_id(); +} + +Task request_stop_after_claim( + Scheduler scheduler, + std::stop_source& stopSource) { + co_await scheduler.schedule(stopSource.get_token()); + co_return stopSource.request_stop(); +} + +Task race_cancellation( + Scheduler scheduler, + std::stop_token stopToken, + int& resumeCount) { + bool cancelled { false }; + + try { + co_await scheduler.schedule(stopToken); + } catch (const OperationCancelled&) { + cancelled = true; + } + + ++resumeCount; + co_return cancelled; +} + +Task schedule_pre_cancelled_many_times( + Scheduler scheduler, + std::stop_token stopToken, + int count) { + int cancellationCount { 0 }; + + for (int index { 0 }; index < count; ++index) { + try { + co_await scheduler.schedule(stopToken); + } catch (const OperationCancelled&) { + ++cancellationCount; + } + } + + co_return cancellationCount; +} + +Task reschedule_many_times( + Scheduler scheduler, + int count, + std::atomic& completed) { + for (int index { 0 }; index < count; ++index) { + co_await scheduler.schedule(); + completed.fetch_add(1, std::memory_order_relaxed); + } +} + +Task run_concurrent_producers( + Scheduler scheduler, + int producerCount, + int schedulesPerProducer) { + std::atomic completed {}; + TaskGroup group {}; + + for (int index { 0 }; index < producerCount; ++index) { + group.spawn(reschedule_many_times( + scheduler, + schedulesPerProducer, + completed)); + } + + co_await group.join(); + co_return completed.load(std::memory_order_relaxed); +} + +Task increment_once( + Scheduler scheduler, + std::atomic& completed) { + co_await scheduler.schedule(); + completed.fetch_add(1, std::memory_order_relaxed); +} + +Task try_schedule_during_close( + Scheduler scheduler, + std::atomic& resumeCount) { + bool accepted { false }; + + try { + co_await scheduler.schedule(); + accepted = true; + } catch (const std::logic_error&) { + } + + resumeCount.fetch_add(1, std::memory_order_relaxed); + co_return accepted; +} + +Scheduler make_expired_scheduler() { + ThreadPool pool { 1 }; + return pool.get_scheduler(); +} + +std::pair make_same_expired_schedulers() { + ThreadPool pool { 1 }; + return { + pool.get_scheduler(), + pool.get_scheduler() + }; +} + +TEST(CmpThreadPoolTest, ReportsConstructionAndSchedulerIdentity) { + const auto expectedDefaultThreadCount = std::max( + 1, + std::thread::hardware_concurrency()); + ThreadPool defaultPool {}; + ThreadPool first { 2 }; + ThreadPool second { 1 }; + + EXPECT_EQ(defaultPool.thread_count(), expectedDefaultThreadCount); + EXPECT_EQ(first.thread_count(), 2U); + EXPECT_EQ(second.thread_count(), 1U); + EXPECT_EQ(first.get_scheduler(), first.get_scheduler()); + EXPECT_NE(first.get_scheduler(), second.get_scheduler()); + EXPECT_THROW(ThreadPool { 0 }, std::invalid_argument); +} + +TEST(CmpThreadPoolTest, SchedulingAlwaysSuspendsAndUsesAWorker) { + ThreadPool pool { 1 }; + RunLoop loop {}; + const auto scheduler = pool.get_scheduler(); + const auto callerThread = std::this_thread::get_id(); + + EXPECT_FALSE(scheduler.schedule().await_ready()); + EXPECT_NE(loop.run(schedule_once(scheduler)), callerThread); +} + +TEST(CmpThreadPoolTest, ReturnsExplicitlyToTheRunLoopThread) { + ThreadPool pool { 2 }; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + const auto [workerThread, returnedThread] = loop.run(return_to_run_loop( + pool.get_scheduler(), + loop.get_scheduler())); + + EXPECT_NE(workerThread, callerThread); + EXPECT_EQ(returnedThread, callerThread); +} + +TEST(CmpThreadPoolTest, PreservesOneWorkerFifoDequeueOrder) { + constexpr int ENTRY_COUNT { 100 }; + ThreadPool pool { 1 }; + const auto scheduler = pool.get_scheduler(); + std::counting_semaphore<1> entered { 0 }; + std::latch release { 1 }; + std::vector order {}; + TaskGroup group {}; + order.reserve(ENTRY_COUNT); + + group.spawn(block_worker(scheduler, entered, release)); + const bool blockerEntered = entered.try_acquire_for(2s); + + for (int value { 0 }; value < ENTRY_COUNT; ++value) { + group.spawn(record_on_pool(scheduler, value, order)); + } + + release.count_down(); + RunLoop loop {}; + loop.run(group.join()); + + EXPECT_TRUE(blockerEntered); + EXPECT_EQ(order.size(), static_cast(ENTRY_COUNT)); + EXPECT_TRUE(std::ranges::equal( + order, + std::views::iota(0, ENTRY_COUNT))); +} + +TEST(CmpThreadPoolTest, WorkerOriginFanoutWakesASecondWorker) { + ThreadPool pool { 2 }; + RunLoop loop {}; + std::counting_semaphore<2> entered { 0 }; + std::latch release { 1 }; + std::mutex threadsMutex {}; + std::set threads {}; + std::exception_ptr driverException {}; + + std::jthread driver { [&] { + try { + loop.run(run_nested_fanout( + pool.get_scheduler(), + entered, + release, + threadsMutex, + threads)); + } catch (...) { + driverException = std::current_exception(); + } + } }; + + const bool firstEntered = entered.try_acquire_for(2s); + const bool secondEntered = entered.try_acquire_for(2s); + release.count_down(); + driver.join(); + + EXPECT_TRUE(firstEntered); + EXPECT_TRUE(secondEntered); + EXPECT_FALSE(driverException); + EXPECT_EQ(threads.size(), 2U); +} + +TEST(CmpThreadPoolTest, IntegratesValuesExceptionsWhenAllAndTaskGroup) { + ThreadPool pool { 4 }; + RunLoop loop {}; + const auto scheduler = pool.get_scheduler(); + + EXPECT_EQ(loop.run(value_on_pool(scheduler, 42)), 42); + EXPECT_THROW(loop.run(fail_on_pool(scheduler)), std::runtime_error); + + const auto [first, second] = loop.run(when_all( + value_on_pool(scheduler, 20), + value_on_pool(scheduler, 22))); + EXPECT_EQ(first + second, 42); + + EXPECT_EQ( + loop.run(run_concurrent_producers(scheduler, 8, 250)), + 2'000); +} + +TEST(CmpThreadPoolTest, SupportsNormalAndPreCancelledScheduling) { + ThreadPool pool { 1 }; + RunLoop loop {}; + std::stop_source stopped {}; + const auto callerThread = std::this_thread::get_id(); + stopped.request_stop(); + + const auto normal = loop.run(schedule_once( + pool.get_scheduler(), + std::stop_token {})); + const auto cancelled = loop.run(schedule_once( + pool.get_scheduler(), + stopped.get_token())); + + EXPECT_FALSE(normal.cancelled_); + EXPECT_TRUE(cancelled.cancelled_); + EXPECT_NE(normal.thread_, callerThread); + EXPECT_NE(cancelled.thread_, callerThread); +} + +TEST(CmpThreadPoolTest, CancelsAnEntryQueuedBehindAWorker) { + ThreadPool pool { 1 }; + const auto scheduler = pool.get_scheduler(); + std::counting_semaphore<1> entered { 0 }; + std::latch release { 1 }; + std::stop_source stopSource {}; + bool cancelled { false }; + std::thread::id resumedThread {}; + TaskGroup group {}; + + group.spawn(block_worker(scheduler, entered, release)); + const bool blockerEntered = entered.try_acquire_for(2s); + + // spawn 返回时,取消项已经确定排在被阻塞 worker 后面。 + group.spawn(observe_cancellation( + scheduler, + stopSource.get_token(), + cancelled, + resumedThread)); + const bool requestWon = stopSource.request_stop(); + release.count_down(); + + RunLoop loop {}; + loop.run(group.join()); + + EXPECT_TRUE(blockerEntered); + EXPECT_TRUE(requestWon); + EXPECT_TRUE(cancelled); + EXPECT_NE(resumedThread, std::thread::id {}); +} + +TEST(CmpThreadPoolTest, LateStopDoesNotReplaceAClaimedEntry) { + ThreadPool pool { 1 }; + RunLoop loop {}; + std::stop_source stopSource {}; + + EXPECT_TRUE(loop.run(request_stop_after_claim( + pool.get_scheduler(), + stopSource))); + EXPECT_TRUE(stopSource.stop_requested()); +} + +TEST(CmpThreadPoolTest, CancellationRaceResumesExactlyOnce) { + constexpr int RACE_COUNT { 500 }; + ThreadPool pool { 2 }; + RunLoop loop {}; + int completedCount { 0 }; + int cancelledCount { 0 }; + + for (int index { 0 }; index < RACE_COUNT; ++index) { + std::stop_source stopSource {}; + int resumeCount { 0 }; + std::jthread canceller { [&stopSource] { + stopSource.request_stop(); + } }; + + if (loop.run(race_cancellation( + pool.get_scheduler(), + stopSource.get_token(), + resumeCount))) { + ++cancelledCount; + } else { + ++completedCount; + } + + canceller.join(); + EXPECT_EQ(resumeCount, 1); + } + + EXPECT_EQ(completedCount + cancelledCount, RACE_COUNT); +} + +TEST(CmpThreadPoolTest, ConcurrentReschedulingIsExactlyOnceAndStackSafe) { + constexpr int PRODUCER_COUNT { 8 }; + constexpr int SCHEDULES_PER_PRODUCER { 2'000 }; + ThreadPool pool { 4 }; + RunLoop loop {}; + + EXPECT_EQ( + loop.run(run_concurrent_producers( + pool.get_scheduler(), + PRODUCER_COUNT, + SCHEDULES_PER_PRODUCER)), + PRODUCER_COUNT * SCHEDULES_PER_PRODUCER); +} + +TEST(CmpThreadPoolTest, RepeatedPreCancellationDoesNotGrowTheStack) { + constexpr int SCHEDULE_COUNT { 20'000 }; + ThreadPool pool { 2 }; + RunLoop loop {}; + std::stop_source stopSource {}; + stopSource.request_stop(); + + EXPECT_EQ( + loop.run(schedule_pre_cancelled_many_times( + pool.get_scheduler(), + stopSource.get_token(), + SCHEDULE_COUNT)), + SCHEDULE_COUNT); +} + +TEST(CmpThreadPoolTest, DestructionDrainsAcceptedEntries) { + constexpr int ENTRY_COUNT { 5'000 }; + auto pool = std::make_unique(2); + const auto scheduler = pool->get_scheduler(); + std::atomic completed {}; + TaskGroup group {}; + + for (int index { 0 }; index < ENTRY_COUNT; ++index) { + group.spawn(increment_once(scheduler, completed)); + } + + pool.reset(); + RunLoop loop {}; + loop.run(group.join()); + + EXPECT_EQ(completed.load(std::memory_order_relaxed), ENTRY_COUNT); + EXPECT_THROW(loop.run(schedule_uncaught(scheduler)), std::logic_error); +} + +TEST(CmpThreadPoolTest, ConcurrentCloseEitherAcceptsOrRejectsWithoutLoss) { + auto pool = std::make_unique(1); + const auto scheduler = pool->get_scheduler(); + std::counting_semaphore<1> entered { 0 }; + std::latch release { 1 }; + std::barrier start { 3 }; + std::atomic resumeCount {}; + std::optional accepted {}; + std::exception_ptr submitException {}; + TaskGroup blocker {}; + RunLoop submitLoop {}; + + blocker.spawn(block_worker(scheduler, entered, release)); + const bool blockerEntered = entered.try_acquire_for(2s); + + std::jthread destroyer { [&] { + start.arrive_and_wait(); + pool.reset(); + } }; + std::jthread submitter { [&] { + start.arrive_and_wait(); + try { + accepted.emplace(submitLoop.run(try_schedule_during_close( + scheduler, + resumeCount))); + } catch (...) { + submitException = std::current_exception(); + } + } }; + + start.arrive_and_wait(); + release.count_down(); + destroyer.join(); + submitter.join(); + + RunLoop blockerLoop {}; + blockerLoop.run(blocker.join()); + + EXPECT_TRUE(blockerEntered); + EXPECT_FALSE(submitException); + EXPECT_TRUE(accepted.has_value()); + EXPECT_EQ(resumeCount.load(std::memory_order_relaxed), 1); +} + +TEST(CmpThreadPoolTest, PreservesExpiredIdentityAndRejectsScheduling) { + RunLoop loop {}; + std::stop_source stopSource {}; + const auto [expired, sameExpired] = make_same_expired_schedulers(); + const auto otherExpired = make_expired_scheduler(); + stopSource.request_stop(); + + EXPECT_EQ(expired, sameExpired); + EXPECT_NE(expired, otherExpired); + EXPECT_THROW(loop.run(schedule_uncaught(expired)), std::logic_error); + EXPECT_THROW( + loop.run(schedule_once(expired, stopSource.get_token())), + std::logic_error); +} + +} // namespace From 701aa8b888102cd3a4931c5089e24ed118ee9b0f Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sat, 29 Aug 2026 23:25:58 +0800 Subject: [PATCH 03/17] =?UTF-8?q?=E5=AE=9E=E7=8E=B0=E7=AC=AC=E5=85=AD?= =?UTF-8?q?=E9=98=B6=E6=AE=B5=E9=98=BB=E5=A1=9E=E4=BB=BB=E5=8A=A1=E5=8D=B8?= =?UTF-8?q?=E8=BD=BD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 128 ++-- README.md | 56 +- README.zh.hant.md | 50 +- README.zh.md | 50 +- benchmarks/v1-readiness/src/main.cpp | 56 +- docs/architecture.md | 60 +- docs/architecture.zh.hant.md | 58 +- docs/architecture.zh.md | 57 +- .../benchmarks/2026-08-29-cmp-v1-readiness.md | 55 +- ...6-08-29-cmp-phase6a-blocking-offload-v1.md | 67 ++ ...-cmp-phase6a-blocking-offload-v1-design.md | 309 +++++++++ examples/basic/src/main.cpp | 18 + src/blocking.cppm | 72 ++ src/cmp.cppm | 1 + tests/blocking_test.cpp | 645 ++++++++++++++++++ 15 files changed, 1487 insertions(+), 195 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-29-cmp-phase6a-blocking-offload-v1.md create mode 100644 docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md create mode 100644 src/blocking.cppm create mode 100644 tests/blocking_test.cpp diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index ed3f70f..bbb493b 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -3,101 +3,93 @@ ## 项目概览 CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 `mcpplibs.cmp`。当前已实现 -懒启动唯一所有权 `Task`、变参/vector `when_all()`、静止点 `TaskGroup`、一次性与可复用 -事件、RAII `AsyncMutex`、带定时和取消的调用线程 `RunLoop`,以及固定大小的 CPU -`ThreadPool`。 +懒启动单消费者 `Task`、变参/vector `when_all()`、静止点 `TaskGroup`、一次性与可复用 +事件、RAII `AsyncMutex`、带定时和取消的调用线程 `RunLoop`、固定大小的 `ThreadPool`,以及 +用于隔离同步调用的 `run_blocking()`。 `.xlings.json` 固定 mcpp 2026.8.11.2;当前工具链为 LLVM 22.1.8,测试依赖为 `compat.gtest` 1.15.2。`examples/basic` 是独立 path-dependency consumer。 ## 当前目标与状态 -第五阶段 ThreadPool v1 已完成本地实现、测试、示例、压测、三语文档同步及复审修复,当前可 -进入用户审查及远程 CI 交付。本轮未执行任何 Git 或 GitHub 操作;提交、推送、PR 和 CI 均不 -应被视为已经完成。 +Phase 6A blocking offload v1 已完成本地实现、文档同步和验证;远程尚未更新。原生异步 I/O +不属于 6A,下一项设计工作是 Phase 6B Spec。 ## 已完成工作 -- 新增不可移动的固定大小 `ThreadPool` 和弱引用、可复制的 `ThreadPool::Scheduler`;根模块已 - 导出该分区。 -- `schedule()` 始终挂起并转移到任意 worker;`schedule(stop_token)` 支持 completion/cancel - 原子竞态获胜语义。 -- 实现共享 FIFO、休眠 worker、逐项唤醒、锁外恢复、构造失败清理、关闭/入队线性化、排空并 - join,以及 worker 内自析构终止保护。 -- 新增 15 项确定性 ThreadPool 测试,覆盖 API、FIFO、多 worker 唤醒、结构化组合、取消竞态、 - 高容量/栈安全、关闭与过期 Scheduler。 -- 复审后把默认 worker 数断言收紧为精确契约,并补齐同源与异源过期 Scheduler 的身份比较 - 测试。 -- `examples/basic` 增加协程内 worker 计算、显式回到 RunLoop 并打印 - `Worker pool result: 42` 的示例。 -- 新增标准库限定的 `benchmarks/thread-pool` consumer 和五轮数据报告;每轮覆盖粗粒度 CPU、 - 连续调度、worker 内 fan-out 和并发重调度。 -- README、架构文档、第五阶段设计/计划及压测报告已同步到实现事实。 -- `mcpp-style-ref` 不再重复维护易过期的分区清单,改以架构文档为当前事实来源;本轮涉及的 - 混合换行文件也已统一。 +- 新增 `mcpplibs.cmp:blocking` 分区并由根模块导出一个公共函数模板: + `run_blocking(blockingWorkers, returnTo, operation, stopToken)`。 +- helper 按值持有 callable,支持 `void`、可移动值、move-only callable/result 和异常传播; + callable 在指定 ThreadPool 上执行一次,结果只在显式返回 Scheduler 上发布。 +- 复用现有 `ThreadPool` 的调度、取消、队列和关闭契约;没有新增 `BlockingPool`、隐藏全局 + executor、future、类型擦除、依赖或 manifest 配置。 +- 新增 11 项确定性测试,覆盖懒启动、worker/返回线程亲和、返回另一 ThreadPool、值/void、 + move-only、异常、预取消/排队取消/晚取消、取消竞态、过期与 inactive Scheduler、RunLoop + 响应性,以及 5,000 个并发 offload 的 exactly-once 行为。 +- `examples/basic` 新增独立 blocking ThreadPool,并在协程中打印 `Blocking result: 42`。 +- `benchmarks/v1-readiness` 的文件与回环网络场景已从手写 `jthread` adapter 迁移到一个专用 + CMP ThreadPool 和 `run_blocking()`;负载及成功/失败硬检查保持不变。 +- 三份 README、三份架构文档、Phase 6A Design/Plan 和 v1 readiness 数据报告已同步到实现 + 事实。 ## 重要决策 -- v1 只公开 `ThreadPool`、`get_scheduler()`、`thread_count()` 和两个 `schedule()` 重载;不增加 - executor 基类、submit/detach、resize、优先级、亲和或公共 shutdown API。 -- 默认 worker 数使用 `hardware_concurrency()` 并把未知的零归一为一;显式传零抛出 - `std::invalid_argument`。 -- 队列是无界共享 `std::deque`。有界同步接纳可能让递归提交的全部 worker 互相等待,因此 - backpressure 必须另行设计。 -- 取消项留在 FIFO,由 worker 消费并恢复;取消回调不扫描队列,也不在请求取消的线程恢复用户 - 协程。 -- 析构关闭接纳并排空已经接纳的 continuation;ThreadPool 只拥有执行线程,Task 生命周期仍由 - `Task`、`when_all()` 和 `TaskGroup` 管理。 -- 本机数据证明粗粒度 CPU 并行有效,也证明微任务会争用共享队列;work stealing 只有在代表性 - workload 的 profiler 证明共享队列是主要瓶颈后,才进入单独设计。 +- 独立的普通 `ThreadPool` 实例就是 blocking pool;不维护行为相同的第二种线程池类型。 +- `run_blocking()` 是懒 `Task`,callable 和两个 Scheduler 都按值进入协程帧,避免临时对象 + 悬空。 +- 第一次调度接受可选 `std::stop_token`;取消只能跳过尚未被 worker claim 的 callable。 + 已经开始的同步调用不能被抢占。 +- 返回调度不接受取消 token,确保值、异常或取消结果不会滞留在 blocking worker。 +- 返回 Scheduler 失败时直接传播其异常;失效的完成上下文本身无法被 helper 修复。 +- Phase 6A 仍是每个运行中同步调用占用一个系统线程的隔离方案,不宣称原生非阻塞 I/O。 +- 队列沿用 ThreadPool 的无界共享 FIFO;背压、超时、强制中断和 worker replacement 均未在 + 没有实测需求前增加。 ## 修改 / 重要文件 -- 核心:`src/thread_pool.cppm`、`src/cmp.cppm` -- 测试:`tests/thread_pool_test.cpp` +- 核心:`src/blocking.cppm`、`src/cmp.cppm` +- 测试:`tests/blocking_test.cpp` - 示例:`examples/basic/src/main.cpp` -- 压测:`benchmarks/thread-pool/`、`docs/benchmarks/2026-08-29-cmp-thread-pool.md` -- 方案:`docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md`、 - `docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md` -- 公共文档:三份 README 与三份 `docs/architecture*` 文档 -- Agent 技能:`.agents/skills/mcpp-style-ref/SKILL.md` +- 压测:`benchmarks/v1-readiness/src/main.cpp`、 + `docs/benchmarks/2026-08-29-cmp-v1-readiness.md` +- 方案:`docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md`、 + `docs/superpowers/plans/2026-08-29-cmp-phase6a-blocking-offload-v1.md` +- 公共文档:`README.md`、`README.zh.md`、`README.zh.hant.md`、`docs/architecture.md`、 + `docs/architecture.zh.md`、`docs/architecture.zh.hant.md` ## 验证情况 - `mcpp build --profile dev --strict --cache=off`:通过。 -- `mcpp test --profile dev --strict --cache=off`:8 个二进制、105/105 通过。 +- `mcpp test --profile dev --strict --cache=off`:9 个二进制、116/116 通过。 - `mcpp build --profile release --strict --cache=off`:通过。 -- `mcpp test --profile release --strict --cache=off`:8 个二进制、105/105 通过。 -- Dev `thread_pool_test` 定向复验:15/15 通过。 -- Release 关键竞态测试子集连续执行 100 轮:100/100 通过。 -- `examples/basic` 的 `mcpp run`:通过,包含 `Worker pool result: 42`,退出码 0。 -- `mcpp-style-ref` 通过 `quick_validate.py` 校验;架构文档链接有效。 -- 项目文本文件换行复查未发现混合 CRLF/LF 文件。 -- ThreadPool Release benchmark:构建通过;五轮记录共 8,410,240 次操作全部成功、非预期失败 - 为 0;最终可执行性复验也全部 PASS。 -- 五轮中位数显示粗粒度 CPU 任务从 1 到 8 worker 约 7.56 倍加速;完整原始数据见压测报告。 -- 以上均为本机 Linux/WSL2 结果;本轮没有运行 GitHub 三平台 CI。 +- `mcpp test --profile release --strict --cache=off`:9 个二进制、116/116 通过。 +- Dev 定向 `blocking_test`:11/11 通过。 +- Release `CancellationRaceInvokesAtMostOnce` 连续执行 100 轮:100/100 通过。 +- `examples/basic` 的 `mcpp run`:通过,包含 `Blocking result: 42`,退出码 0。 +- `benchmarks/v1-readiness` Release strict 构建通过;迁移后执行 5 轮,compute、file_io 和 + network_loopback 每轮均 PASS,五轮非预期失败总数为 0。 +- 文件/网络每轮计数分别为 1,000/20,000 成功、100/100 预期失败、0 非预期失败;原始耗时和 + 吞吐已写入 benchmark 报告。 +- 当前 mcpp 仍输出 SubOS 缺少 `subos_info` 的既有环境提示,但所有构建和运行成功。 +- 以上均为本机 Linux/WSL2 结果;尚未执行 GitHub 三平台 CI 或其他远程操作。 ## 已知问题 / 风险 -- 共享 FIFO 在大量微小 continuation 与较多 worker 时存在显著 mutex/通知争用;这是已记录的 - v1 性能上限,不是丢任务或执行失败。 -- ThreadPool 不抢占;长计算或阻塞调用会占住一个 worker。异步 I/O 与专用 blocking pool 尚未 - 实现。 -- 析构会等待已接纳 continuation 返回;用户代码永久阻塞时析构也会永久等待。在自身 worker - 内析构属于硬生命周期错误并立即终止。 -- 三平台兼容性尚需远程 CI 确认。 -- 本机 mcpp 仍输出 SubOS 缺少 `subos_info` 的环境警告,但本轮所有构建和运行均成功。 +- 运行中的同步调用不可抢占;永久阻塞会永久占用 worker,并使等待它的 Task 和 pool 析构 + 无法完成。callable 如需协作取消,必须自行捕获并检查 token。 +- ThreadPool 使用无界共享 FIFO;持续生产快于消费时,应用需要限制自己的结构化 in-flight + 数量。 +- 返回 Scheduler 的 owner 必须持续存活;RunLoop Scheduler 还必须处于 active `run()` 中。 +- `run_blocking()` 是线程隔离,不是 epoll、io_uring、kqueue 或 IOCP 等原生异步 I/O。 +- 三平台兼容性仍需远程 CI 确认。 ## 剩余工作 -1. 用户审查第五阶段实现与压测结论。 -2. 获得明确授权后,才可执行对应范围的 commit、push、PR 或远程 CI 操作。 -3. 第五阶段合并并通过三平台 CI 后,再开始第六阶段异步 I/O / blocking pool 设计;不要把该 - 能力补进当前 ThreadPool。 +1. 用户审查 Phase 6A 本地改动;Commit、Push、PR 与 CI 均需分别获得明确授权。 +2. 用户确认进入下一设计阶段后,为一个窄资源族编写 Phase 6B 原生异步 I/O Spec;实现前仍需 + 单独批准。 ## 推荐下一步 -先审查 `src/thread_pool.cppm`、`tests/thread_pool_test.cpp` 和线程池压测报告。若认可当前共享 -FIFO v1 边界,再明确授权所需的 Git/GitHub 步骤完成交付;合并后从第六阶段的 I/O 契约与平台 -边界开始设计。 +先审查并提交 Phase 6A;随后只设计 Phase 6B 的最小原生异步 I/O 边界,优先选择单一资源族, +不把文件、网络和所有平台后端一次性绑定在同一阶段。 diff --git a/README.md b/README.md index 83ebd74..69fdc2b 100644 --- a/README.md +++ b/README.md @@ -21,15 +21,17 @@ > CMP provides a lazy, single-consumer `Task` / `Task`, structured variadic and vector > `when_all()`, an eager structured `TaskGroup`, one-shot and reusable events, an RAII `AsyncMutex`, > a caller-thread `RunLoop` with explicit and monotonic timed scheduling, and a fixed-size CPU -> `ThreadPool`. Ready scheduling on either executor, timed waits, reusable-event waits, and -> TaskGroup children can use explicit cooperative cancellation with `std::stop_token`; -> asynchronous I/O and detached execution are not implemented. +> `ThreadPool`. `run_blocking()` executes an owned synchronous callable on a dedicated pool instance +> and delivers its outcome through an explicit return Scheduler. Ready scheduling on either +> executor, timed waits, reusable-event waits, TaskGroup children, and queued blocking offloads can +> use explicit cooperative cancellation with `std::stop_token`; native asynchronous I/O and +> detached execution are not implemented. CMP is being built as a modern coroutine runtime and library on standard stackless C++ coroutines. Its explicit `co_await` model now covers fixed and incremental structured concurrency, one-time event notification, caller-thread and multi-worker scheduling, monotonic timers, and -cancellable waits and can grow, in small verified steps, toward asynchronous I/O and safe handling -of blocking work. +cancellable waits, plus structured isolation of blocking work. Native asynchronous I/O remains a +separate, incremental design step. ## Why CMP? @@ -56,8 +58,9 @@ promise that: - coroutine switching is safe directly inside a signal handler; - task migration is implicit, work stealing is already enabled, or arbitrary async I/O exists. -Those capabilities must be designed and verified individually. The expected direction is -explicit async I/O awaiters, a dedicated blocking pool, and cooperative safe points. +Those capabilities must be designed and verified individually. CMP now uses an explicitly +dedicated `ThreadPool` instance with `run_blocking()` for synchronous work; native async I/O awaiters +and cooperative safe points remain separate work. ## Quick Start @@ -79,9 +82,9 @@ mcpp run ``` The example prints `Coroutine result: 42`, `Concurrent result: 42`, `Worker pool result: 42`, -`Task group result: 42`, and `Recursive group result: 3`; it then demonstrates notifications with -`Event signalled` and `Reusable event cycles: 2`, prints `Mutex result: 42`, and finishes with -`Coroutine cancelled`. All messages come from `Task` coroutines. +`Blocking result: 42`, `Task group result: 42`, and `Recursive group result: 3`; it then demonstrates +notifications with `Event signalled` and `Reusable event cycles: 2`, prints `Mutex result: 42`, and +finishes with `Coroutine cancelled`. All messages come from `Task` coroutines. ## Current API @@ -96,6 +99,7 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -145,6 +149,20 @@ Task print_worker_result( co_return; } +Task print_blocking_result( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller) { + const auto result = co_await run_blocking( + blockingWorkers, + caller, + [] { + std::this_thread::sleep_for(1ms); + return 42; + }); + std::println("Blocking result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -214,12 +232,16 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke int main() { ThreadPool workers { 2 }; + ThreadPool blockingWorkers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_worker_result( workers.get_scheduler(), loop.get_scheduler())); + loop.run(print_blocking_result( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -292,6 +314,14 @@ RunLoop ready scheduling. The v1 implementation uses one work-conserving shared workers. Destruction closes admission, drains every accepted entry, and joins the workers. It does not own higher-level Tasks, and blocking calls still block their current worker. +`run_blocking(blockingWorkers, returnTo, operation, stopToken)` owns a synchronous callable in its +lazy Task frame, schedules it once to the selected ThreadPool, and publishes its value, `void`, +exception, or queued cancellation only after scheduling back to `returnTo`. Use a separate +ThreadPool instance for blocking work so it cannot occupy latency-sensitive CPU workers. A stop +request can skip work that has not been claimed; it cannot preempt a running synchronous call, and +the return schedule is intentionally not cancellable. This is thread-based isolation, not native +non-blocking I/O. + RunLoop is not a background thread and does not make blocking code asynchronous. A Task that suspends without arranging a future resume can leave `run()` waiting indefinitely. CMP does not provide automatic thread affinity: after an external awaiter resumes on another thread, explicitly @@ -308,6 +338,7 @@ await the desired Scheduler to return to its RunLoop. ├── src/cancellation.cppm # shared cooperative-cancellation exception ├── src/run_loop.cppm # RunLoop and Scheduler partition ├── src/thread_pool.cppm # fixed-size CPU worker scheduler +├── src/blocking.cppm # structured blocking-call offload ├── src/when_all.cppm # structured concurrent Task join ├── src/task_group.cppm # eager mutable structured Task scope ├── src/one_shot_event.cppm # allocation-free one-time notification @@ -316,6 +347,7 @@ await the desired Scheduler to return to its RunLoop. ├── tests/cmp_test.cpp # Task contract and lifetime tests ├── tests/run_loop_test.cpp # scheduler, boundary, and threading tests ├── tests/thread_pool_test.cpp # worker, cancellation, and shutdown tests +├── tests/blocking_test.cpp # blocking offload, affinity, and cancellation tests ├── tests/when_all_test.cpp # join ownership, result, and race tests ├── tests/task_group_test.cpp # mutable scope lifetime and race tests ├── tests/one_shot_event_test.cpp # event publication and race tests @@ -349,7 +381,7 @@ global mcpp installation. CMP does not track `mcpp.lock`; `.gitignore` enforces that repository policy. Runtime dependencies belong in `[dependencies]`; gtest is declared explicitly under `[dev-dependencies.compat]`. -The current local suite contains 105 tests across eight binaries. The POSIX-only Release pressure +The current local suite contains 116 tests across nine binaries. The POSIX-only Release pressure consumer and its recorded success/failure data are documented in the [v1 readiness benchmark](docs/benchmarks/2026-08-29-cmp-v1-readiness.md). Multi-worker correctness and performance data are documented in the @@ -366,7 +398,7 @@ Runtime work is split into independently reviewable phases: OneShotEvent, AsyncManualResetEvent, and AsyncMutex — implemented and pressure-tested; 5. fixed-size multi-worker scheduling — implemented and benchmarked; work stealing remains gated by profiling evidence; -6. asynchronous I/O integration and a blocking pool. +6. structured blocking offload — implemented; native asynchronous I/O remains separately gated. The remaining order is directional, not a promise that a listed feature is already implemented. diff --git a/README.zh.hant.md b/README.zh.hant.md index 9add5ae..68c317e 100644 --- a/README.zh.hant.md +++ b/README.zh.hant.md @@ -21,12 +21,14 @@ > CMP 已提供延遲啟動、單一消費者的 `Task` / `Task`、支援變參和 vector 的結構化 > `when_all()`、eager 結構化 `TaskGroup`、一次性與可複用事件、RAII `AsyncMutex`,以及在呼叫 > 執行緒運行、支援明確排程和單調時鐘定時排程的 `RunLoop`,以及固定大小的 CPU -> `ThreadPool`。兩種執行器的就緒排程、定時等待、可複用事件等待和 TaskGroup 子任務可明確 -> 使用基於 `std::stop_token` 的協作式取消;非同步 I/O 和 detached 執行尚未實作。 +> `ThreadPool`。`run_blocking()` 可在專用的 pool 實例上執行同步 callable,並透過明確的 +> 返回 Scheduler 交付結果。兩種執行器的就緒排程、定時等待、可複用事件等待、TaskGroup +> 子任務和排隊中的阻塞 offload 可明確使用基於 `std::stop_token` 的協作式取消;原生非同步 +> I/O 和 detached 執行尚未實作。 CMP 計畫以標準無堆疊 C++ 協程建構現代協程執行期與函式庫。明確的 `co_await` 模型現已 -涵蓋固定與增量結構化並行、一次性事件通知、呼叫執行緒與多 worker 排程、單調時鐘計時器和 -可取消等待,並將透過經過驗證的小步驟繼續探索非同步 I/O,以及阻塞工作的安全隔離。 +涵蓋固定與增量結構化並行、一次性事件通知、呼叫執行緒與多 worker 排程、單調時鐘計時器、 +可取消等待和阻塞工作的結構化隔離。原生非同步 I/O 仍是獨立的增量設計步驟。 ## 為什麼叫 CMP? @@ -51,8 +53,8 @@ C++ 標準協程是語言機制,不是完整執行期。因此 CMP 不會宣 - 可以直接在訊號處理器中安全切換協程; - task 會隱式遷移、work stealing 已啟用,或任意非同步 I/O 已經實作。 -這些能力必須分別設計和驗證。預期方向是明確的非同步 I/O awaiter、專用 blocking pool -以及協作式安全點。 +這些能力必須分別設計和驗證。CMP 目前透過明確的專用 `ThreadPool` 實例和 +`run_blocking()` 隔離同步工作;原生非同步 I/O awaiter 與協作式安全點仍需單獨設計。 ## 快速開始 @@ -73,8 +75,9 @@ cd examples/basic mcpp run ``` -範例會印出 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42` 和 -`Recursive group result: 3`,接著透過 `Event signalled` 與 `Reusable event cycles: 2` +範例會印出 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、 +`Blocking result: 42`、`Task group result: 42` 和 `Recursive group result: 3`,接著透過 +`Event signalled` 與 `Reusable event cycles: 2` 展示一次性及可複用通知,再印出 `Mutex result: 42` 和 `Coroutine cancelled`;所有輸出都在 `Task` 協程內部。 @@ -91,6 +94,7 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -140,6 +144,20 @@ Task print_worker_result( co_return; } +Task print_blocking_result( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller) { + const auto result = co_await run_blocking( + blockingWorkers, + caller, + [] { + std::this_thread::sleep_for(1ms); + return 42; + }); + std::println("Blocking result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -209,12 +227,16 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke int main() { ThreadPool workers { 2 }; + ThreadPool blockingWorkers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_worker_result( workers.get_scheduler(), loop.get_scheduler())); + loop.run(print_blocking_result( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -276,6 +298,12 @@ set 不做任何事。事件不可移動且必須比等待者活得更久;v1 work-conserving 共用 FIFO 和休眠 worker;解構關閉接納、排空全部已接納項並 join worker。 ThreadPool 不擁有上層 Task,阻塞呼叫仍會阻塞目前 worker。 +`run_blocking(blockingWorkers, returnTo, operation, stopToken)` 在其延遲 Task 框架中持有同步 +callable,將它排程到指定 ThreadPool 恰好執行一次,並在透過 `returnTo` 排程回來後交付值、 +`void`、例外或排隊取消。應使用獨立的 ThreadPool 實例隔離阻塞工作,避免佔用延遲敏感的 CPU +worker。停止要求可以跳過尚未被 worker 領取的工作,但不能搶佔已經執行的同步呼叫;返回排程 +刻意不可取消。這是基於執行緒的隔離,不是原生非阻塞 I/O。 + RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步程式碼。如果 Task 暫停後沒有 安排未來的恢復動作,`run()` 可能一直等待。CMP 不提供隱式執行緒親和:外部 awaiter 在其他 執行緒恢復協程後,需要明確等待目標 Scheduler 才會返回對應 RunLoop。 @@ -291,6 +319,7 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ├── src/cancellation.cppm # 共用的協作式取消例外 ├── src/run_loop.cppm # RunLoop 與 Scheduler 分割區 ├── src/thread_pool.cppm # 固定大小的 CPU worker 排程器 +├── src/blocking.cppm # 結構化阻塞呼叫 offload ├── src/when_all.cppm # 結構化並行 Task 匯合 ├── src/task_group.cppm # eager 可變結構化 Task 作用域 ├── src/one_shot_event.cppm # 無分配一次性通知 @@ -299,6 +328,7 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ├── tests/cmp_test.cpp # Task 契約和生命週期測試 ├── tests/run_loop_test.cpp # 排程、邊界和執行緒測試 ├── tests/thread_pool_test.cpp # worker、取消和關閉測試 +├── tests/blocking_test.cpp # 阻塞 offload、執行緒親和和取消測試 ├── tests/when_all_test.cpp # 匯合所有權、結果和競態測試 ├── tests/task_group_test.cpp # 可變作用域生命週期和競態測試 ├── tests/one_shot_event_test.cpp # 事件發布和競態測試 @@ -330,7 +360,7 @@ CI 在 Linux、macOS 和 Windows 上執行等價的建構、測試與獨立範 CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定。執行期相依放在 `[dependencies]`,gtest 明確宣告在 `[dev-dependencies.compat]` 中。 -目前本機套件包含 8 個測試二進位檔、105 項測試。僅用於 POSIX 的 Release 壓測 consumer 及其 +目前本機套件包含 9 個測試二進位檔、116 項測試。僅用於 POSIX 的 Release 壓測 consumer 及其 成功/失敗資料記錄在 [v1 可開發性壓測](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 多 worker 正確性和效能資料記錄在[執行緒池壓測](docs/benchmarks/2026-08-29-cmp-thread-pool.md)。 @@ -344,7 +374,7 @@ CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定 4. 單調時鐘 Timer v1、可取消就緒/定時等待、變參/vector 匯合、靜止點 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已實作並完成壓力驗證; 5. 固定大小的多 worker 排程——已實作並完成壓測;work stealing 仍需 profiling 證據; -6. 非同步 I/O 整合和 blocking pool。 +6. 結構化阻塞 offload——已實作;原生非同步 I/O 仍需單獨設計和驗證。 剩餘順序只是方向,不代表列出的能力已經實作。 diff --git a/README.zh.md b/README.zh.md index f368520..6e49277 100644 --- a/README.zh.md +++ b/README.zh.md @@ -21,12 +21,14 @@ > CMP 已提供懒启动、单消费者的 `Task` / `Task`、支持变参和 vector 的结构化 > `when_all()`、eager 结构化 `TaskGroup`、一次性与可复用事件、RAII `AsyncMutex`,以及在调用 > 线程运行、支持显式调度和单调时钟定时调度的 `RunLoop`,以及固定大小的 CPU `ThreadPool`。 -> 两种执行器的就绪调度、定时等待、可复用事件等待和 TaskGroup 子任务可显式使用基于 -> `std::stop_token` 的协作式取消;异步 I/O 和 detached 执行尚未实现。 +> `run_blocking()` 可在专用的 pool 实例上执行同步 callable,并通过显式的返回 Scheduler +> 交付结果。两种执行器的就绪调度、定时等待、可复用事件等待、TaskGroup 子任务和排队中的 +> 阻塞 offload 可显式使用基于 `std::stop_token` 的协作式取消;原生异步 I/O 和 detached +> 执行尚未实现。 CMP 计划基于标准无栈 C++ 协程构建现代协程运行时和库。显式 `co_await` 模型现已覆盖固定与 -增量结构化并发、一次性事件通知、调用线程与多 worker 调度、单调时钟定时器和可取消等待, -并将通过经过验证的小步骤继续探索异步 I/O 以及阻塞工作的安全隔离。 +增量结构化并发、一次性事件通知、调用线程与多 worker 调度、单调时钟定时器、可取消等待和 +阻塞工作的结构化隔离。原生异步 I/O 仍是独立的增量设计步骤。 ## 为什么叫 CMP? @@ -51,8 +53,8 @@ C++ 标准协程是语言机制,不是完整运行时。因此 CMP 不会宣 - 可以直接在信号处理器中安全切换协程; - task 会隐式迁移、work stealing 已启用,或任意异步 I/O 已经实现。 -这些能力必须分别设计和验证。预期方向是显式异步 I/O awaiter、专用 blocking pool -以及协作式安全点。 +这些能力必须分别设计和验证。CMP 目前通过显式的专用 `ThreadPool` 实例和 +`run_blocking()` 隔离同步工作;原生异步 I/O awaiter 与协作式安全点仍需单独设计。 ## 快速开始 @@ -73,8 +75,9 @@ cd examples/basic mcpp run ``` -示例会打印 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42` 和 -`Recursive group result: 3`,然后通过 `Event signalled` 与 `Reusable event cycles: 2` +示例会打印 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、 +`Blocking result: 42`、`Task group result: 42` 和 `Recursive group result: 3`,然后通过 +`Event signalled` 与 `Reusable event cycles: 2` 演示一次性及可复用通知,再打印 `Mutex result: 42` 和 `Coroutine cancelled`;所有输出都在 `Task` 协程内部。 @@ -91,6 +94,7 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -140,6 +144,20 @@ Task print_worker_result( co_return; } +Task print_blocking_result( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller) { + const auto result = co_await run_blocking( + blockingWorkers, + caller, + [] { + std::this_thread::sleep_for(1ms); + return 42; + }); + std::println("Blocking result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -209,12 +227,16 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke int main() { ThreadPool workers { 2 }; + ThreadPool blockingWorkers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_worker_result( workers.get_scheduler(), loop.get_scheduler())); + loop.run(print_blocking_result( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -276,6 +298,12 @@ set 不做任何事。事件不可移动且必须比等待者活得更久;v1 work-conserving 共享 FIFO 和休眠 worker;析构关闭接纳、排空全部已接纳项并 join worker。 ThreadPool 不拥有上层 Task,阻塞调用仍会阻塞当前 worker。 +`run_blocking(blockingWorkers, returnTo, operation, stopToken)` 在其懒 Task 帧中持有同步 +callable,将它调度到指定 ThreadPool 恰好执行一次,并在通过 `returnTo` 调度回来后交付值、 +`void`、异常或排队取消。应使用独立的 ThreadPool 实例隔离阻塞工作,避免占用延迟敏感的 CPU +worker。停止请求可以跳过尚未被 worker 领取的工作,但不能抢占已经运行的同步调用;返回调度 +刻意不可取消。这是基于线程的隔离,不是原生非阻塞 I/O。 + RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。如果 Task 挂起后没有安排未来的 恢复动作,`run()` 可能一直等待。CMP 不提供隐式线程亲和:外部 awaiter 在其他线程恢复协程 后,需要显式等待目标 Scheduler 才会返回对应 RunLoop。 @@ -291,6 +319,7 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ├── src/cancellation.cppm # 共享的协作式取消异常 ├── src/run_loop.cppm # RunLoop 与 Scheduler 分区 ├── src/thread_pool.cppm # 固定大小的 CPU worker 调度器 +├── src/blocking.cppm # 结构化阻塞调用 offload ├── src/when_all.cppm # 结构化并发 Task 汇合 ├── src/task_group.cppm # eager 可变结构化 Task 作用域 ├── src/one_shot_event.cppm # 无分配一次性通知 @@ -299,6 +328,7 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ├── tests/cmp_test.cpp # Task 契约和生命周期测试 ├── tests/run_loop_test.cpp # 调度、边界和线程测试 ├── tests/thread_pool_test.cpp # worker、取消和关闭测试 +├── tests/blocking_test.cpp # 阻塞 offload、线程亲和和取消测试 ├── tests/when_all_test.cpp # 汇合所有权、结果和竞态测试 ├── tests/task_group_test.cpp # 可变作用域生命周期和竞态测试 ├── tests/one_shot_event_test.cpp # 事件发布和竞态测试 @@ -330,7 +360,7 @@ CI 在 Linux、macOS 和 Windows 上执行等价的构建、测试和独立示 CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。运行时依赖放在 `[dependencies]`,gtest 明确声明在 `[dev-dependencies.compat]` 中。 -当前本地套件包含 8 个测试二进制、105 项测试。仅用于 POSIX 的 Release 压测 consumer 及其 +当前本地套件包含 9 个测试二进制、116 项测试。仅用于 POSIX 的 Release 压测 consumer 及其 成功/失败数据记录在 [v1 可开发性压测](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 多 worker 正确性和性能数据记录在[线程池压测](docs/benchmarks/2026-08-29-cmp-thread-pool.md)。 @@ -344,7 +374,7 @@ CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。 4. 单调时钟 Timer v1、可取消就绪/定时等待、变参/vector 汇合、静止点 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已实现并完成压力验证; 5. 固定大小的多 worker 调度——已实现并完成压测;work stealing 仍需 profiling 证据; -6. 异步 I/O 集成和 blocking pool。 +6. 结构化阻塞 offload——已实现;原生异步 I/O 仍需单独设计和验证。 剩余顺序只是方向,不代表列出的能力已经实现。 diff --git a/benchmarks/v1-readiness/src/main.cpp b/benchmarks/v1-readiness/src/main.cpp index a63a936..15ea300 100644 --- a/benchmarks/v1-readiness/src/main.cpp +++ b/benchmarks/v1-readiness/src/main.cpp @@ -13,12 +13,14 @@ import mcpplibs.cmp; namespace { -using mcpplibs::cmp::AsyncManualResetEvent; using mcpplibs::cmp::RunLoop; using mcpplibs::cmp::Task; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using Scheduler = RunLoop::Scheduler; +using BlockingScheduler = ThreadPool::Scheduler; using Clock = std::chrono::steady_clock; constexpr int COMPUTE_TASKS { 50'000 }; @@ -242,29 +244,6 @@ void send_all(int socket, std::string_view data) { return result != 0 && errno == ECONNREFUSED; } -template -Task run_external(Scheduler scheduler, Work work) { - AsyncManualResetEvent completed {}; - std::exception_ptr exception {}; - - std::jthread worker { [&] { - try { - work(); - } catch (...) { - exception = std::current_exception(); - } - completed.set(); - } }; - - co_await completed; - co_await scheduler.schedule(); - worker.join(); - - if (exception) { - std::rethrow_exception(exception); - } -} - Task failing_compute(Scheduler scheduler) { co_await scheduler.schedule(); throw std::invalid_argument { "expected compute failure" }; @@ -317,6 +296,7 @@ Task run_compute(Scheduler scheduler, Counters& counters) { } Task run_file_io( + BlockingScheduler blockingWorkers, Scheduler scheduler, const std::filesystem::path& directory, Counters& counters) { @@ -324,7 +304,7 @@ Task run_file_io( TaskGroup group {}; for (int workerIndex { 0 }; workerIndex < FILE_WORKERS; ++workerIndex) { - group.spawn(run_external(scheduler, [&, workerIndex] { + group.spawn(run_blocking(blockingWorkers, scheduler, [&, workerIndex] { const auto file = directory / std::format("worker-{}.bin", workerIndex); @@ -369,13 +349,16 @@ Task run_file_io( co_await group.join(); } -Task run_network_io(Scheduler scheduler, Counters& counters) { +Task run_network_io( + BlockingScheduler blockingWorkers, + Scheduler scheduler, + Counters& counters) { const std::string payload(NETWORK_PAYLOAD_SIZE, 'n'); TaskGroup group {}; for (int workerIndex { 0 }; workerIndex < NETWORK_WORKERS; ++workerIndex) { static_cast(workerIndex); - group.spawn(run_external(scheduler, [&] { + group.spawn(run_blocking(blockingWorkers, scheduler, [&] { int completed { 0 }; try { auto [client, server] = make_loopback_pair(); @@ -461,6 +444,9 @@ void print_metrics(const Metrics& metrics) { int main() { TemporaryDirectory directory {}; + ThreadPool blockingWorkers { + std::max(FILE_WORKERS, NETWORK_WORKERS) + }; std::vector results {}; results.reserve(3); @@ -475,19 +461,27 @@ int main() { "file_io", FILE_WORKERS * (FILE_ROUNDS + FILE_FAILURES), [&](Scheduler scheduler, Counters& counters) { - return run_file_io(scheduler, directory.path(), counters); + return run_file_io( + blockingWorkers.get_scheduler(), + scheduler, + directory.path(), + counters); })); results.emplace_back(measure( "network_loopback", NETWORK_WORKERS * (NETWORK_ROUNDS + NETWORK_FAILURES), - [](Scheduler scheduler, Counters& counters) { - return run_network_io(scheduler, counters); + [&](Scheduler scheduler, Counters& counters) { + return run_network_io( + blockingWorkers.get_scheduler(), + scheduler, + counters); })); std::println( - "environment,hardware_threads={} free_space_bytes={}", + "environment,hardware_threads={} blocking_workers={} free_space_bytes={}", std::thread::hardware_concurrency(), + blockingWorkers.thread_count(), std::filesystem::space(directory.path()).available); std::println( "scenario,operations,successes,expected_failures,unexpected_failures,elapsed_ms,ops_per_second,status"); diff --git a/docs/architecture.md b/docs/architecture.md index d456c99..b3749bb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -14,11 +14,13 @@ returns a suspended coroutine to that loop. `schedule()`, `schedule_after()`, an have `std::stop_token` overloads for cooperative cancellation; timed scheduling uses relative and absolute `steady_clock` deadlines without a timer thread. `ThreadPool::Scheduler::schedule()` explicitly transfers a continuation to any fixed worker and has the same cancellation-winner rule. +`run_blocking()` uses a caller-selected ThreadPool instance for synchronous work and publishes the +outcome only after an explicit return Scheduler is reached. The repository contains: - one mcpp package manifest; -- the root module `mcpplibs.cmp` with Task, cancellation, executor, join, event, and mutex partitions; +- the root module `mcpplibs.cmp` with Task, cancellation, executor, blocking, join, event, and mutex partitions; - gtest contract, lifetime, exception, scheduling, and threading tests; - one standalone path-dependency example; - local v1-readiness and cross-platform ThreadPool benchmark consumers; @@ -71,6 +73,7 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. │ ├── cancellation.cppm │ ├── run_loop.cppm │ ├── thread_pool.cppm +│ ├── blocking.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -80,6 +83,7 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. │ ├── cmp_test.cpp │ ├── run_loop_test.cpp │ ├── thread_pool_test.cpp +│ ├── blocking_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -97,7 +101,7 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. ## Build and tests `.xlings.json` pins the mcpp version used by the project. `mcpp build` builds the inferred library -target. `mcpp test` discovers eight test files and links a gtest entry point for each. The 105 tests +target. `mcpp test` discovers nine test files and links a gtest entry point for each. The 116 tests verify Task ownership and symmetric transfer together with structured joins, root execution, scheduling, exception propagation, timed and cross-thread wake-up, cancellation races, invalid scheduler use, loop reuse, and stack-safe repeated completion. @@ -141,6 +145,7 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -183,6 +188,20 @@ Task print_worker_result( co_return; } +Task print_blocking_result( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller) { + const auto result = co_await run_blocking( + blockingWorkers, + caller, + [] { + std::this_thread::sleep_for(1ms); + return 42; + }); + std::println("Blocking result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -252,12 +271,16 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke int main() { ThreadPool workers { 2 }; + ThreadPool blockingWorkers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_worker_result( workers.get_scheduler(), loop.get_scheduler())); + loop.run(print_blocking_result( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -273,8 +296,9 @@ compilation, and the public root runner independently of the root test targets. `print_answer()` on the main thread; a short monotonic timer expires before the coroutine prints `Coroutine result: 42`. The next root Task concurrently joins two timed values and prints `Concurrent result: 42`. A worker-pool coroutine computes away from the caller, explicitly returns -to the RunLoop, and prints `Worker pool result: 42`. The next coroutine eagerly spawns and joins two -void Tasks before printing +to the RunLoop, and prints `Worker pool result: 42`. A blocking callable runs on a separate pool, +returns to the RunLoop, and prints `Blocking result: 42`. The next coroutine eagerly spawns and joins +two void Tasks before printing `Task group result: 42`; a recursively growing group then prints `Recursive group result: 3`. Other coroutines print `Event signalled`, exercise two reusable-event cycles, and print `Reusable event cycles: 2`. Two guarded Tasks produce `Mutex result: 42`. A final structured group @@ -402,8 +426,17 @@ thread on which the coroutine currently executes. - the pool owns threads, not Tasks, and provides no timer, blocking-I/O adaptation, detached work, resize, priority, or affinity API. +`run_blocking(blockingWorkers, returnTo, operation, stopToken)` has the following contract: + +- the lazy Task owns a move-constructible callable and invokes it exactly once after a worker claim; +- a separate ThreadPool instance isolates blocking work from latency-sensitive CPU workers; +- value, `void`, exception, or queued cancellation is observed only after `returnTo.schedule()`; +- cancellation can skip queued work, but cannot preempt a synchronous call after it starts; +- the return schedule is intentionally uncancellable so every outcome reaches one executor; +- this is thread-based isolation and does not claim native non-blocking I/O. + There is no public free-standing `sync_wait`, detached execution, standalone Timer handle, -asynchronous I/O backend, custom frame allocator, or blocking-work pool. Cancellation remains +asynchronous I/O backend, custom frame allocator, or distinct blocking-pool type. Cancellation remains explicit: Scheduler waits and `AsyncManualResetEvent` accept tokens, and TaskGroup owns an optional shared stop channel, but the module provides no implicit propagation and no compatibility alias for the old scaffold module. @@ -425,10 +458,9 @@ package contract: 2. channels and additional structured wake-up paths; 3. profile-guided work stealing if representative workloads justify it; 4. asynchronous I/O integrations; -5. a dedicated pool for unavoidable blocking work; -6. result adapters and optional coroutine-frame allocation strategies. +5. result adapters and optional coroutine-frame allocation strategies. -Task, cancellation, RunLoop, ThreadPool, `when_all`, TaskGroup, OneShotEvent, +Task, cancellation, RunLoop, ThreadPool, blocking offload, `when_all`, TaskGroup, OneShotEvent, AsyncManualResetEvent, and AsyncMutex occupy separate module partitions because they are implemented public boundaries. Further partitions or implementation units are added only when another implemented API needs them. @@ -446,13 +478,15 @@ cd examples/basic mcpp run ``` -The expected result is a successful library build, 105 passing tests across eight binaries, and an -example that prints `Coroutine result: 42`, `Concurrent result: 42`, `Worker pool result: 42`, `Task group result: 42`, -`Recursive group result: 3`, `Event signalled`, `Reusable event cycles: 2`, `Mutex result: 42`, then +The expected result is a successful library build, 116 passing tests across nine binaries, and an +example that prints `Coroutine result: 42`, `Concurrent result: 42`, `Worker pool result: 42`, +`Blocking result: 42`, `Task group result: 42`, `Recursive group result: 3`, `Event signalled`, +`Reusable event cycles: 2`, `Mutex result: 42`, then `Coroutine cancelled` and exits with status 0. Tests retain the existing high-volume stack checks and add 20,000 recursive TaskGroup admissions, 100,000 pre-cancelled ready schedules, 50,000 manual -event waiters, 20,000 nested reusable-event signals, and set/cancel races. Focused phase-4 race -suites pass repeated Release runs. Compute, temporary-file, and loopback-network counts and +event waiters, 20,000 nested reusable-event signals, set/cancel races, queued blocking cancellation, +and 5,000 concurrent blocking offloads. Focused race suites pass repeated Release runs. Compute, +temporary-file, and loopback-network counts and throughput are recorded in the [v1 readiness benchmark](benchmarks/2026-08-29-cmp-v1-readiness.md). ThreadPool counts, concurrency, and five-round Release measurements are recorded in the diff --git a/docs/architecture.zh.hant.md b/docs/architecture.zh.hant.md index 5001f20..25fabfc 100644 --- a/docs/architecture.zh.hant.md +++ b/docs/architecture.zh.hant.md @@ -13,12 +13,13 @@ CMP 是一個具備小型協程執行核心的 C++23 模組專案。根模組匯 把暫停協程送回對應執行迴圈。`schedule()`、`schedule_after()` 和 `schedule_at()` 都有接受 `std::stop_token` 的協作式取消多載;定時排程使用相對和絕對的 `steady_clock` 期限,且不建立 計時執行緒。`ThreadPool::Scheduler::schedule()` 會把 continuation 明確轉移到任意固定 -worker,並採用相同的取消獲勝規則。 +worker,並採用相同的取消獲勝規則。`run_blocking()` 使用呼叫端選擇的 ThreadPool 實例執行 +同步工作,並只在到達明確返回 Scheduler 後發布結果。 儲存庫現有內容包括: - 一份 mcpp 套件清單; -- 根模組 `mcpplibs.cmp` 及 Task、cancellation、執行器、join、event、mutex 模組分割區; +- 根模組 `mcpplibs.cmp` 及 Task、cancellation、執行器、blocking、join、event、mutex 模組分割區; - 涵蓋契約、生命週期、例外、排程和執行緒行為的 gtest 測試; - 一個透過路徑相依使用根套件的獨立範例; - v1 可開發性壓測和跨平台 ThreadPool 壓測 consumer; @@ -69,6 +70,7 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m │ ├── cancellation.cppm │ ├── run_loop.cppm │ ├── thread_pool.cppm +│ ├── blocking.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -78,6 +80,7 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m │ ├── cmp_test.cpp │ ├── run_loop_test.cpp │ ├── thread_pool_test.cpp +│ ├── blocking_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -95,7 +98,7 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m ## 建置與測試 `.xlings.json` 固定專案使用的 mcpp 版本。`mcpp build` 建置自動推斷的函式庫目標。 -`mcpp test` 會找到八個測試檔案,並為每個檔案連結 gtest 進入點。105 項測試同時驗證 Task 所有權 +`mcpp test` 會找到九個測試檔案,並為每個檔案連結 gtest 進入點。116 項測試同時驗證 Task 所有權 和對稱轉移、結構化匯合,以及根任務執行、普通與定時排程、例外傳播、跨執行緒期限喚醒、 無效 Scheduler、取消競態、RunLoop 重複使用和不會增長呼叫堆疊的重複完成。 @@ -136,6 +139,7 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -178,6 +182,20 @@ Task print_worker_result( co_return; } +Task print_blocking_result( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller) { + const auto result = co_await run_blocking( + blockingWorkers, + caller, + [] { + std::this_thread::sleep_for(1ms); + return 42; + }); + std::println("Blocking result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -247,12 +265,16 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke int main() { ThreadPool workers { 2 }; + ThreadPool blockingWorkers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_worker_result( workers.get_scheduler(), loop.get_scheduler())); + loop.run(print_blocking_result( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -265,7 +287,8 @@ int main() { 這個範例在根測試目標之外,單獨檢查路徑相依解析、模組使用、外部協程編譯和公開根任務 驅動器。RunLoop 依序輸出 `Coroutine result: 42` 和 `Concurrent result: 42`;worker pool -協程離開呼叫執行緒完成計算,明確回到 RunLoop 後輸出 `Worker pool result: 42`,隨後輸出 +協程離開呼叫執行緒完成計算,明確回到 RunLoop 後輸出 `Worker pool result: 42`。阻塞 callable +在獨立 pool 上執行,返回 RunLoop 後輸出 `Blocking result: 42`,隨後輸出 `Task group result: 42`;遞迴增長的 group 輸出 `Recursive group result: 3`。其他協程展示 一次性及兩輪可複用事件並輸出 `Event signalled`、`Reusable event cycles: 2`,兩個受保護 Task 輸出 `Mutex result: 42`。最後一個結構化 group 使用 `cancel_and_join()`,從可取消就緒 @@ -382,8 +405,17 @@ Task;明確等待原 Scheduler 才會把 continuation 送回對應 RunLoop。 - pool 只擁有執行緒、不擁有 Task,也不提供 timer、阻塞 I/O 適配、detached、resize、優先級 或 affinity API。 +`run_blocking(blockingWorkers, returnTo, operation, stopToken)` 遵循以下契約: + +- 延遲 Task 持有可移動建構的 callable,並在 worker 領取後恰好呼叫一次; +- 使用獨立的 ThreadPool 實例,將阻塞工作與延遲敏感的 CPU worker 隔離; +- 值、`void`、例外或排隊取消只在 `returnTo.schedule()` 後可見; +- 取消可以跳過排隊工作,但不能搶佔已經開始的同步呼叫; +- 返回排程刻意不可取消,使每個結果都到達一個執行器; +- 這是基於執行緒的隔離,不表示原生非阻塞 I/O。 + 目前沒有公開自由函式 `sync_wait`、detached 執行、獨立 Timer 控制代碼、非同步 I/O 後端、 -自訂協程框架 allocator 或阻塞工作執行緒池。取消仍是明確的:Scheduler 等待和 +自訂協程框架 allocator 或獨立的阻塞執行緒池型別。取消仍是明確的:Scheduler 等待和 `AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模組不提供隱式 傳播,並且沒有保留舊骨架模組的相容別名。 @@ -402,10 +434,9 @@ CMP 名稱中的 `C` 與 Go 執行期中的 `G` 相呼應,但這只說明命 2. channel 和更多結構化喚醒路徑; 3. 代表性負載證明有必要時再加入 profiling 驅動的工作竊取; 4. 非同步 I/O 整合; -5. 處理無法避免之阻塞工作的專用執行緒池; -6. 結果適配器和可選的協程框架配置策略。 +5. 結果適配器和可選的協程框架配置策略。 -Task、cancellation、RunLoop、ThreadPool、`when_all`、TaskGroup、OneShotEvent、 +Task、cancellation、RunLoop、ThreadPool、blocking offload、`when_all`、TaskGroup、OneShotEvent、 AsyncManualResetEvent 與 AsyncMutex 已形成真實的公開邊界,因此分別位於模組分割區中。 只有其他已實作 API 確實需要新邊界時,才繼續增加模組分割區或實作單元。 @@ -422,13 +453,14 @@ cd examples/basic mcpp run ``` -預期結果是函式庫建置成功、八個二進位檔中的 105 項測試全部通過,而且範例依序輸出 -`Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42`、 -`Recursive group result: 3`、`Event signalled`、`Reusable event cycles: 2`、 +預期結果是函式庫建置成功、九個二進位檔中的 116 項測試全部通過,而且範例依序輸出 +`Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、 +`Blocking result: 42`、`Task group result: 42`、`Recursive group result: 3`、`Event signalled`、 +`Reusable event cycles: 2`、 `Mutex result: 42` 和 `Coroutine cancelled` 後以狀態 0 結束。測試保留原有高容量堆疊安全 檢查,並增加兩萬次 TaskGroup 遞迴接納、十萬次預取消就緒排程、五萬個 manual event 等待者、 -兩萬次巢狀可複用事件訊號及 set/cancel 競態;第四階段重點競態套件已連續執行多輪 Release -測試。計算、臨時檔案和回環網路的成功/失敗計數及吞吐記錄在 +兩萬次巢狀可複用事件訊號、set/cancel 競態、排隊阻塞取消和 5,000 個並行阻塞 offload;重點 +競態套件已連續執行多輪 Release 測試。計算、臨時檔案和回環網路的成功/失敗計數及吞吐記錄在 [v1 可開發性壓測](benchmarks/2026-08-29-cmp-v1-readiness.md)。ThreadPool 的計數、並行和五輪 Release 資料記錄在[執行緒池壓測](benchmarks/2026-08-29-cmp-thread-pool.md)。目前 Windows LLVM 工具鏈 不會產生 GNU depfile;如果模組介面包含的檔案發生變更,增量建置可能沿用舊的 BMI 或 diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 86c57bb..01be18c 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -13,12 +13,13 @@ CMP 是一个具备小型协程执行核心的 C++23 模块项目。根模块导 对应运行循环。`schedule()`、`schedule_after()` 和 `schedule_at()` 都有接受 `std::stop_token` 的协作式取消重载;定时调度使用相对和绝对的 `steady_clock` 期限,且不创建 定时线程。`ThreadPool::Scheduler::schedule()` 会把 continuation 显式转移到任意固定 worker, -并采用相同的取消获胜规则。 +并采用相同的取消获胜规则。`run_blocking()` 使用调用方选择的 ThreadPool 实例执行同步工作, +并只在到达显式返回 Scheduler 后发布结果。 仓库现有内容包括: - 一份 mcpp 包清单; -- 根模块 `mcpplibs.cmp` 及 Task、cancellation、执行器、join、event、mutex 模块分区; +- 根模块 `mcpplibs.cmp` 及 Task、cancellation、执行器、blocking、join、event、mutex 模块分区; - 覆盖契约、生命周期、异常、调度和线程行为的 gtest 测试; - 一个通过路径依赖使用根包的独立示例; - v1 可开发性压测和跨平台 ThreadPool 压测 consumer; @@ -69,6 +70,7 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp │ ├── cancellation.cppm │ ├── run_loop.cppm │ ├── thread_pool.cppm +│ ├── blocking.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -78,6 +80,7 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp │ ├── cmp_test.cpp │ ├── run_loop_test.cpp │ ├── thread_pool_test.cpp +│ ├── blocking_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -95,7 +98,7 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp ## 构建与测试 `.xlings.json` 固定项目使用的 mcpp 版本。`mcpp build` 构建自动推断的库目标。 -`mcpp test` 发现八个测试文件,并为每个文件链接 gtest 入口。105 项测试同时验证 Task 所有权和 +`mcpp test` 发现九个测试文件,并为每个文件链接 gtest 入口。116 项测试同时验证 Task 所有权和 对称转移、结构化汇合,以及根任务执行、普通与定时调度、异常传播、跨线程期限唤醒、无效 Scheduler、取消竞态、RunLoop 复用和不会增长调用栈的重复完成。 @@ -136,6 +139,7 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -178,6 +182,20 @@ Task print_worker_result( co_return; } +Task print_blocking_result( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller) { + const auto result = co_await run_blocking( + blockingWorkers, + caller, + [] { + std::this_thread::sleep_for(1ms); + return 42; + }); + std::println("Blocking result: {}", result); + co_return; +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -247,12 +265,16 @@ Task print_cancellation(RunLoop::Scheduler scheduler, std::stop_token toke int main() { ThreadPool workers { 2 }; + ThreadPool blockingWorkers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_worker_result( workers.get_scheduler(), loop.get_scheduler())); + loop.run(print_blocking_result( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); loop.run(print_mutex(loop.get_scheduler())); @@ -265,7 +287,8 @@ int main() { 该示例在根测试目标之外,单独检查路径依赖解析、模块使用、外部协程编译和公共根任务驱动器。 RunLoop 依次输出 `Coroutine result: 42` 和 `Concurrent result: 42`;worker pool 协程离开调用 -线程完成计算,显式回到 RunLoop 后输出 `Worker pool result: 42`,随后输出 `Task group result: 42`; +线程完成计算,显式回到 RunLoop 后输出 `Worker pool result: 42`。阻塞 callable 在独立 pool 上 +运行,返回 RunLoop 后输出 `Blocking result: 42`,随后输出 `Task group result: 42`; 递归增长的 group 输出 `Recursive group result: 3`。其他协程演示一次性及两轮可复用事件并 输出 `Event signalled`、`Reusable event cycles: 2`,两个受保护 Task 输出 `Mutex result: 42`。最后一个结构化 group 使用 `cancel_and_join()`,从可取消就绪调度捕获 @@ -381,8 +404,17 @@ RunLoop 不拥有工作线程,也不提供自动线程亲和。外部 awaiter - pool 只拥有线程、不拥有 Task,也不提供 timer、阻塞 I/O 适配、detached、resize、优先级或 affinity API。 +`run_blocking(blockingWorkers, returnTo, operation, stopToken)` 遵循以下契约: + +- 懒 Task 持有可移动构造的 callable,并在 worker 领取后恰好调用一次; +- 使用独立的 ThreadPool 实例,将阻塞工作与延迟敏感的 CPU worker 隔离; +- 值、`void`、异常或排队取消只在 `returnTo.schedule()` 后可见; +- 取消可以跳过排队工作,但不能抢占已经开始的同步调用; +- 返回调度刻意不可取消,使每个结果都到达一个执行器; +- 这是基于线程的隔离,不表示原生非阻塞 I/O。 + 目前没有公共自由函数 `sync_wait`、detached 执行、独立 Timer 句柄、异步 I/O 后端、自定义 -协程帧 allocator 或阻塞任务线程池。取消仍是显式的:Scheduler 等待和 +协程帧 allocator 或独立的阻塞线程池类型。取消仍是显式的:Scheduler 等待和 `AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模块不提供隐式传播, 并且没有保留旧脚手架模块的兼容别名。 @@ -400,10 +432,9 @@ CMP 名称中的 `C` 与 Go 运行时中的 `G` 相呼应,但这只说明命 2. channel 和更多结构化唤醒路径; 3. 代表性负载证明有必要时再加入 profiling 驱动的工作窃取; 4. 异步 I/O 集成; -5. 处理不可避免的阻塞工作的专用线程池; -6. 结果适配器和可选的协程帧分配策略。 +5. 结果适配器和可选的协程帧分配策略。 -Task、cancellation、RunLoop、ThreadPool、`when_all`、TaskGroup、OneShotEvent、 +Task、cancellation、RunLoop、ThreadPool、blocking offload、`when_all`、TaskGroup、OneShotEvent、 AsyncManualResetEvent 与 AsyncMutex 已经形成真实的公共边界,因此分别位于模块分区中。 只有其他已实现 API 确实需要新边界时,才继续增加模块分区或实现单元。 @@ -420,12 +451,14 @@ cd examples/basic mcpp run ``` -预期结果是库构建成功、八个二进制中的 105 项测试全部通过,并且示例依次输出 -`Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、`Task group result: 42`、 -`Recursive group result: 3`、`Event signalled`、`Reusable event cycles: 2`、 +预期结果是库构建成功、九个二进制中的 116 项测试全部通过,并且示例依次输出 +`Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、 +`Blocking result: 42`、`Task group result: 42`、`Recursive group result: 3`、`Event signalled`、 +`Reusable event cycles: 2`、 `Mutex result: 42` 和 `Coroutine cancelled` 后以状态 0 退出。测试保留原有高容量栈安全检查, 并增加两万次 TaskGroup 递归接纳、十万次预取消就绪调度、五万个 manual event 等待者、两万次 -嵌套可复用事件信号及 set/cancel 竞态;第四阶段重点竞态套件已连续执行多轮 Release 测试。 +嵌套可复用事件信号、set/cancel 竞态、排队阻塞取消和 5,000 个并发阻塞 offload;重点竞态 +套件已连续执行多轮 Release 测试。 计算、临时文件和回环网络的成功/失败计数及吞吐记录在 [v1 可开发性压测](benchmarks/2026-08-29-cmp-v1-readiness.md)。ThreadPool 的计数、并发和五轮 Release 数据记录在[线程池压测](benchmarks/2026-08-29-cmp-thread-pool.md)。当前 Windows LLVM 工具链不会 diff --git a/docs/benchmarks/2026-08-29-cmp-v1-readiness.md b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md index 453fd9c..aae2e4f 100644 --- a/docs/benchmarks/2026-08-29-cmp-v1-readiness.md +++ b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md @@ -7,8 +7,9 @@ ## 测试边界 本报告验证 CMP v1 的结构化生命周期、调度、跨线程唤醒和失败传播能够承载计算任务, -以及由外部工作线程适配的文件与网络 I/O。CMP v1 尚无原生异步 I/O 后端,因此 I/O 数据 -不是 epoll、io_uring、kqueue 或 IOCP 性能数据,也不代表多核协程调度吞吐。 +以及通过专用 CMP `ThreadPool` 与 `run_blocking()` 隔离的文件和网络 I/O。CMP v1 尚无原生 +异步 I/O 后端,因此 I/O 数据不是 epoll、io_uring、kqueue 或 IOCP 性能数据,也不代表 +内核异步 I/O 吞吐。 压测程序位于 `benchmarks/v1-readiness`,只使用标准库、CMP 和 POSIX 回环 socket;它不增加 项目依赖,也不进入跨平台 CI。任何计数不匹配或非预期失败都会让进程以非零状态退出。 @@ -21,6 +22,7 @@ - 临时目录文件系统:ext4; - mcpp:2026.8.11.2; - 编译器:LLVM 22.1.8; +- 阻塞 worker:4; - 配置:Release,关闭 mcpp 构建缓存。 ```bash @@ -35,10 +37,11 @@ mcpp build --profile release --strict --cache=off - `compute`:50,000 个经 `Scheduler` 调度的 Task,每个成功任务执行 128 轮整数运算;每 50 个 任务注入一个预期异常并在结构化子任务内捕获。 -- `file_io`:4 个适配线程,各执行 250 次 64 KiB 临时文件写入、读取和内容校验,再执行 - 25 次缺失文件读取以验证预期失败路径。 -- `network_loopback`:4 个适配线程,各执行 5,000 次 256 字节 TCP 回环请求/响应,再执行 - 25 次未监听端口连接,并且只把 `ECONNREFUSED` 计为预期失败。 +- `file_io`:4 个结构化 offload,各在专用 ThreadPool worker 上执行 250 次 64 KiB 临时文件 + 写入、读取和内容校验,再执行 25 次缺失文件读取以验证预期失败路径。 +- `network_loopback`:4 个结构化 offload,各在专用 ThreadPool worker 上执行 5,000 次 + 256 字节 TCP 回环请求/响应,再执行 25 次未监听端口连接,并且只把 `ECONNREFUSED` 计为 + 预期失败。 吞吐量按“成功操作 + 预期失败操作”的总操作数计算。 @@ -46,30 +49,30 @@ mcpp build --profile release --strict --cache=off | 轮次 | 场景 | 操作数 | 成功 | 预期失败 | 非预期失败 | 耗时 ms | ops/s | 状态 | | ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | -| 1 | compute | 50,000 | 49,000 | 1,000 | 0 | 30.127 | 1,659,624.3 | PASS | -| 1 | file_io | 1,100 | 1,000 | 100 | 0 | 185.222 | 5,938.8 | PASS | -| 1 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,453.289 | 8,193.1 | PASS | -| 2 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.295 | 1,831,847.8 | PASS | -| 2 | file_io | 1,100 | 1,000 | 100 | 0 | 183.017 | 6,010.4 | PASS | -| 2 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,455.883 | 8,184.4 | PASS | -| 3 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.182 | 1,839,425.5 | PASS | -| 3 | file_io | 1,100 | 1,000 | 100 | 0 | 219.769 | 5,005.2 | PASS | -| 3 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,424.811 | 8,289.3 | PASS | -| 4 | compute | 50,000 | 49,000 | 1,000 | 0 | 28.172 | 1,774,824.8 | PASS | -| 4 | file_io | 1,100 | 1,000 | 100 | 0 | 184.978 | 5,946.7 | PASS | -| 4 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,660.766 | 7,554.2 | PASS | -| 5 | compute | 50,000 | 49,000 | 1,000 | 0 | 30.833 | 1,621,635.6 | PASS | -| 5 | file_io | 1,100 | 1,000 | 100 | 0 | 185.683 | 5,924.1 | PASS | -| 5 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,560.138 | 7,851.1 | PASS | +| 1 | compute | 50,000 | 49,000 | 1,000 | 0 | 33.907 | 1,474,617.9 | PASS | +| 1 | file_io | 1,100 | 1,000 | 100 | 0 | 217.009 | 5,068.9 | PASS | +| 1 | network_loopback | 20,100 | 20,000 | 100 | 0 | 4,035.640 | 4,980.6 | PASS | +| 2 | compute | 50,000 | 49,000 | 1,000 | 0 | 64.119 | 779,795.3 | PASS | +| 2 | file_io | 1,100 | 1,000 | 100 | 0 | 421.216 | 2,611.5 | PASS | +| 2 | network_loopback | 20,100 | 20,000 | 100 | 0 | 3,210.152 | 6,261.4 | PASS | +| 3 | compute | 50,000 | 49,000 | 1,000 | 0 | 36.928 | 1,353,998.2 | PASS | +| 3 | file_io | 1,100 | 1,000 | 100 | 0 | 206.260 | 5,333.1 | PASS | +| 3 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,543.529 | 7,902.4 | PASS | +| 4 | compute | 50,000 | 49,000 | 1,000 | 0 | 28.541 | 1,751,852.5 | PASS | +| 4 | file_io | 1,100 | 1,000 | 100 | 0 | 423.989 | 2,594.4 | PASS | +| 4 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,627.611 | 7,649.5 | PASS | +| 5 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.133 | 1,842,742.1 | PASS | +| 5 | file_io | 1,100 | 1,000 | 100 | 0 | 168.893 | 6,513.0 | PASS | +| 5 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,492.924 | 8,062.8 | PASS | ## 汇总 | 场景 | 最短 / 中位 / 最长耗时 ms | 最高 / 中位 / 最低 ops/s | 五轮非预期失败 | | --- | ---: | ---: | ---: | -| compute | 27.182 / 28.172 / 30.833 | 1,839,425.5 / 1,774,824.8 / 1,621,635.6 | 0 | -| file_io | 183.017 / 185.222 / 219.769 | 6,010.4 / 5,938.8 / 5,005.2 | 0 | -| network_loopback | 2,424.811 / 2,455.883 / 2,660.766 | 8,289.3 / 8,184.4 / 7,554.2 | 0 | +| compute | 27.133 / 33.907 / 64.119 | 1,842,742.1 / 1,474,617.9 / 779,795.3 | 0 | +| file_io | 168.893 / 217.009 / 423.989 | 6,513.0 / 5,068.9 / 2,594.4 | 0 | +| network_loopback | 2,492.924 / 2,627.611 / 4,035.640 | 8,062.8 / 7,649.5 / 4,980.6 | 0 | 这些结果说明 v1 核心可以用于有明确所有权的协程编排、定时、显式取消、可复用通知,以及 -通过外部适配线程接入本地文件和网络工作。它们是当前机器上的回归基线,不是跨机器 SLA。 -原生异步 I/O、多 worker 调度、work stealing 和 blocking pool 仍属于后续阶段。 +通过独立 ThreadPool 实例和 `run_blocking()` 接入本地文件与网络工作。它们是当前机器上的 +回归基线,不是跨机器 SLA。原生异步 I/O 和 work stealing 仍属于后续阶段。 diff --git a/docs/superpowers/plans/2026-08-29-cmp-phase6a-blocking-offload-v1.md b/docs/superpowers/plans/2026-08-29-cmp-phase6a-blocking-offload-v1.md new file mode 100644 index 0000000..c0e7bfe --- /dev/null +++ b/docs/superpowers/plans/2026-08-29-cmp-phase6a-blocking-offload-v1.md @@ -0,0 +1,67 @@ +# CMP Phase 6A Blocking Offload v1 Plan + +**Date:** 2026-08-29 +**Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md` +**Status:** Complete + +## 1. Preserve the Phase 5 baseline + +Keep the implemented `ThreadPool` contract unchanged. Phase 6A adds no worker, queue, shutdown, +executor base, or dependency; it only composes existing scheduling and Task behavior. + +## 2. Add the single public helper + +Create `src/blocking.cppm`, export it from `src/cmp.cppm`, and implement +`run_blocking(blockingWorkers, returnTo, operation, stopToken)` as one lazy function template. + +Take both schedulers and the callable by value. Schedule to the blocking workers, invoke the owned +callable once, capture its value or exception, schedule back without cancellation, and only then +publish the outcome. Keep direct `void` and value branches; do not add futures or type erasure. + +## 3. Lock the contract with deterministic tests + +Add `tests/blocking_test.cpp` for laziness, worker/return affinity, value and void, move-only state, +exception transport, queued and late cancellation, scheduler lifetime failures, exactly-once race +behavior, structured concurrency, and stack safety. + +Use latches, semaphores, atomics, and thread IDs. Do not use timing ratios as correctness checks. + +## 4. Update the developer path + +Extend `examples/basic` with one deterministic `run_blocking()` call and print its result after +returning to RunLoop. Keep the existing explicit ThreadPool calculation example so the difference +between manual migration and structured blocking offload remains visible. + +Refactor only the file and loopback sections of `benchmarks/v1-readiness` from hand-written adapter +threads to one dedicated CMP ThreadPool plus `run_blocking()`. Preserve workloads and hard success / +failure counts so old and new reports remain comparable. + +## 5. Synchronize public documentation + +Update the three README and three architecture variants with the implemented API, explicit return +Scheduler, queued-only cancellation, dedicated ThreadPool-instance guidance, and the distinction +from kernel-native asynchronous I/O. Do not describe phase 6B as implemented. + +## 6. Verify and record + +Run focused tests first, then: + +```text +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off +cd examples/basic && mcpp run +cd benchmarks/v1-readiness && mcpp build --profile release --strict --cache=off +``` + +Run the focused cancellation race repeatedly and execute the readiness benchmark for measured +rounds. Update its report with new raw data rather than retaining adapter-thread measurements. +Finally update the Design status, this Plan, and `.agent/HANDOFF.md` with executed facts only. + +## Completion record + +Implemented the single `run_blocking()` API without a new pool type or dependency. Dev and Release +strict builds pass with 116/116 tests; the focused cancellation race passes 100 repeated Release +runs; the standalone example prints the blocking result; and five migrated readiness-benchmark +rounds pass with zero unexpected failures. diff --git a/docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md b/docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md new file mode 100644 index 0000000..0321fdd --- /dev/null +++ b/docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md @@ -0,0 +1,309 @@ +# CMP Phase 6A Blocking Offload v1 Design + +**Date:** 2026-08-29 +**Status:** Implemented and locally verified +**Baseline:** Phase 5 is locally verified with 105/105 tests +**Verification:** Phase 6A passes 116/116 tests in Dev and Release + +## Purpose + +Provide one structured helper for running an unavoidable synchronous operation away from its +current executor and delivering its value, exception, or queued-cancellation result back through an +explicit return Scheduler: + +```cpp +auto text = co_await run_blocking( + blockingWorkers.get_scheduler(), + caller, + [path = std::move(path)] { + return read_file(path); + }); +``` + +This is the first independently reviewable slice of roadmap phase 6. It makes blocking file, +database, resolver, and legacy-library calls safe to integrate without occupying a RunLoop or CPU +worker. It does not claim kernel-native asynchronous I/O. + +Phase 6 is split because portable blocking isolation and native I/O have different contracts. The +first can reuse the implemented ThreadPool exactly. The second requires platform handles, buffer +ownership, completion dispatch, concurrent close, and operating-system cancellation semantics, and +will receive a separate phase-6B design before code is added. + +## Decisions + +1. A dedicated `ThreadPool` instance is the blocking pool; CMP does not add a second pool type with + identical worker, queue, cancellation, and shutdown behavior. +2. `run_blocking()` owns the callable in its coroutine frame, invokes it once on the blocking pool, + and then explicitly schedules back to a caller-selected Scheduler. +3. Values, `void`, exceptions, and queued cancellation are delivered only after the return schedule + succeeds. +4. Cancellation may prevent a queued callable from starting, but cannot preempt a callable that a + worker has already claimed. +5. No native socket or file type, global executor, detached ownership, or third-party dependency is + introduced in phase 6A. + +## Public API + +Add one exported function in a `mcpplibs.cmp:blocking` module partition: + +```cpp +template +requires ( + std::move_constructible && + std::invocable && + ( + std::same_as, void> || + ( + std::is_object_v> && + !std::is_array_v> && + std::move_constructible> + ) + ) && + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } +) +[[nodiscard]] Task> run_blocking( + ThreadPool::Scheduler blockingWorkers, + ReturnScheduler returnTo, + Function operation, + std::stop_token stopToken = {}); +``` + +`Function` is taken by value deliberately. A coroutine initially suspends before its body runs, so +storing a forwarding reference could leave a temporary callable dangling. The by-value parameter is +copied or moved into the coroutine frame before the returned lazy Task is observed. + +The callable is invoked as `std::invoke(std::move(operation))`. It may therefore own move-only state +and is consumed exactly once. Its result must be `void` or a move-constructible, non-array object +accepted by `Task`; reference and array results are rejected rather than given surprising +lifetime semantics. Arguments belong in the callable capture, keeping the public overload set to +one function. + +`ReturnScheduler` is a constrained template rather than an executor base class. Both existing CMP +Scheduler types already provide `schedule()`, so callers may explicitly return to a `RunLoop` or a +different `ThreadPool` without adding type erasure or virtual dispatch. + +## Intended Use + +The pool is an ordinary `ThreadPool` with a dedicated role and an application-chosen size: + +```cpp +Task load_text( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller, + std::filesystem::path path) { + co_return co_await run_blocking( + blockingWorkers, + caller, + [path = std::move(path)] { + return read_file(path); + }); +} + +int main() { + ThreadPool blockingWorkers { 4 }; + RunLoop loop {}; + const auto text = loop.run(load_text( + blockingWorkers.get_scheduler(), + loop.get_scheduler(), + "config.json")); + std::println("Read {} bytes", text.size()); +} +``` + +Applications should not use the same pool for latency-sensitive CPU continuations and operations +that may block for an unbounded time. Separate instances provide isolation without a duplicate CMP +type. There is no universal blocking-worker count, so phase 6A does not guess one or create a hidden +process-wide pool. + +## Execution Protocol + +`run_blocking()` is a normal lazy `Task` and performs these steps only when awaited: + +1. await `blockingWorkers.schedule(stopToken)`; +2. if queued cancellation won, capture `OperationCancelled` without invoking the callable; +3. otherwise invoke the owned callable once on the selected worker; +4. store its result or capture its exception in the coroutine frame; +5. unconditionally await `returnTo.schedule()` without a cancellation token; +6. on the return Scheduler, rethrow the stored exception or cancellation, or return the stored + value. + +The first schedule always suspends while its ThreadPool is active. The operation therefore never +runs inline on the caller. A valid return schedule also suspends, so the awaiting parent observes +the outcome on the requested execution context. CMP still does not infer an ambient executor or +silently migrate arbitrary Tasks. + +Worker-scheduler rejection, including an expired or closing pool, is captured like any other +failure and delivered after a valid return schedule. If the return Scheduler itself is expired, +closed, or inactive, its scheduling error propagates immediately from the current execution +context: CMP cannot provide return affinity after that lifetime contract has already been broken. + +## Results and Exceptions + +- A non-void result is stored in the helper coroutine frame and moved through the existing Task + result path; no copy is required. +- Move-only callables and move-only results are supported. +- `void` uses a direct `if constexpr` path and does not require synthetic result storage. +- Any exception from scheduling onto the worker or invoking the callable is preserved with + `std::exception_ptr`, then rethrown after returning to the selected Scheduler. +- An exception from the return schedule takes precedence because the requested completion context + can no longer be reached. +- CMP does not wrap exceptions in a new error type or translate `std::system_error`; existing Task + propagation remains the public error channel. + +The implementation uses the two direct `if constexpr` branches needed for `void` and value results. +It does not add `std::future`, `std::packaged_task`, type-erased work items, or another result-state +abstraction. + +## Cancellation + +The optional token controls only admission to execution and reuses the ThreadPool's existing +pending/completed/cancelled race: + +- a pre-requested token still queues, then skips the callable when a worker consumes the entry; +- cancellation that wins while queued skips the callable and later throws `OperationCancelled` on + the return Scheduler; +- once a worker claims completion, the callable runs and its value or exception wins; +- requesting stop cannot terminate a synchronous system call or arbitrary C++ function; +- returning to `returnTo` is never cancellable, because cancellation must not strand the helper on + a blocking worker or publish its outcome on the wrong executor. + +A callable that supports cooperative cancellation may capture and inspect the same token itself. +`run_blocking()` does not inspect its signature, inject a token, or reinterpret the callable's own +result. Forceful thread cancellation is out of scope because it cannot preserve C++ object and lock +invariants. + +## Lifetime and Shutdown + +The blocking ThreadPool and the return Scheduler's owner must remain alive until the operation has +returned. The return `RunLoop::Scheduler` must additionally have an active `run()` while the helper +attempts to schedule back. + +The callable object, its returned value, and its captured exception live in the helper coroutine +frame. Captures held by value are therefore safe across both scheduler transfers. Explicit reference +or pointer captures remain the caller's responsibility; phase 6A does not turn borrowed data into +owned data. + +Destroying an unstarted `run_blocking()` Task destroys the owned callable without running it. Once +the worker entry is accepted, existing structured Task, `when_all()`, and `TaskGroup` ownership rules +must keep the awaiting frames alive. No detached path is added. + +ThreadPool shutdown keeps its existing contract: it closes admission, drains accepted entries, and +joins workers. A callable that never returns can therefore keep its Task and pool destruction +blocked indefinitely. This is an inherent limit of wrapping synchronous work, not a condition CMP +can safely hide. + +## Queueing and Backpressure + +Phase 6A inherits ThreadPool's unbounded FIFO. `run_blocking()` adds no second queue and no extra +allocation beyond its coroutine frame, callable/result storage, and the underlying deque growth. + +An application that can create blocking work faster than workers finish it must limit its own +in-flight structured tasks. A bounded blocking queue is deliberately excluded: synchronous +admission can deadlock recursive worker submissions, while asynchronous admission requires a new +backpressure and cancellation contract. That contract should be designed only for a measured use +case. + +## Module and Implementation Boundary + +Implementation changes are limited to: + +- `src/blocking.cppm`: the exported function template; +- `src/cmp.cppm`: re-export the partition; +- `tests/blocking_test.cpp`: focused contract and race tests; +- `examples/basic/src/main.cpp`: one short file-style blocking example; +- `benchmarks/v1-readiness/src/main.cpp`: replace its external adapter threads with a dedicated CMP + ThreadPool and `run_blocking()`; +- public documentation after behavior is implemented and verified. + +`src/blocking.cppm` imports `std`, `:task`, and `:thread_pool`. It owns no threads or shared state and +does not depend on `:run_loop`; the return Scheduler is compile-time constrained by the expression it +uses. No installed dependency or `mcpp.toml` change is required. + +## Validation Contract + +Cross-platform tests must cover: + +- laziness and exactly-once callable invocation; +- execution on a blocking worker and resumption on the requested RunLoop thread; +- returning to a different ThreadPool Scheduler; +- value, `void`, move-only callable, and move-only result paths; +- callable exceptions observed on the return Scheduler; +- pre-cancelled and queued-cancelled work never invoking the callable; +- late cancellation preserving the claimed callable's result; +- repeated cancellation-versus-claim races selecting one outcome; +- expired/closing blocking Scheduler and expired/inactive return Scheduler behavior; +- many concurrent offloads completing exactly once without growing the native call stack; +- one deliberately blocked callable while RunLoop continues processing another ready Task. + +Correctness tests use latches, barriers, atomics, and thread IDs rather than elapsed-time ratios. +Filesystem behavior in the standalone example or tests uses temporary files and deterministic +cleanup. Cross-platform root tests do not introduce platform socket headers. + +The existing POSIX-only `v1-readiness` benchmark is reused instead of adding another consumer. Its +file and loopback network workloads move from manually managed adapter `jthread`s to a dedicated +CMP ThreadPool plus `run_blocking()`. Success, expected-failure, and unexpected-failure counts remain +hard checks; timing remains local evidence and never a CI threshold. + +## Native I/O Phase 6B Gate + +`run_blocking()` is asynchronous relative to its caller but still consumes one operating-system +thread per running synchronous call. Documentation and examples must not describe it as non-blocking +or kernel-native asynchronous I/O. + +Before phase 6B implementation, a separate Spec must choose and prove: + +- the first resource family, preferably one narrow file or TCP surface rather than both at once; +- Linux, Windows, and macOS backend policy and any dependency policy; +- owned handle and buffer lifetimes across pending operations; +- completion-thread and explicit return-affinity behavior; +- immediate completion, partial transfer, EOF, retryable errors, and concurrent close semantics; +- cancellation versus completion with exactly-once resumption; +- event-loop shutdown with outstanding operations; +- deterministic platform tests and three-platform CI coverage. + +Linux `io_uring`, Windows IOCP, and readiness-based event loops expose materially different +submission and completion rules. Phase 6B must not hide those differences behind an API before its +smallest portable contract is known. + +## Deliberately Excluded + +- a duplicate `BlockingPool` wrapper around ThreadPool; +- a hidden singleton or automatically sized blocking pool; +- native asynchronous file, DNS, socket, pipe, console, or process I/O; +- operation timeouts, forceful interruption, or thread replacement for stuck calls; +- detached submission, fire-and-forget ownership, or futures; +- implicit capture of the current Scheduler; +- callable argument forwarding overloads, priorities, bounded queues, and backpressure; +- executor type erasure or a public generic Scheduler concept. + +## Reference Rationale + +- [Tokio `spawn_blocking`](https://docs.rs/tokio/latest/tokio/task/fn.spawn_blocking.html) separates + blocking work from async workers and documents the key limit that running synchronous work cannot + be aborted; CMP keeps that boundary but uses structured Tasks and an explicit pool. +- [Boost.Asio `thread_pool`](https://www.boost.org/doc/libs/latest/doc/html/boost_asio/reference/thread_pool.html) + demonstrates posting owned functions to a fixed worker set; CMP already has the required pool and + therefore does not clone it. +- [cppcoro `resume_on()`](https://github.com/lewissbaker/cppcoro#resume_on) makes completion affinity + explicit when an awaited operation finishes on another executor. `run_blocking()` adopts that + observable rule without adding a general awaitable transform. +- [libunifex `io_uring_context`](https://github.com/facebookexperimental/libunifex/blob/main/doc/api_reference.md#linuxio_uring_context) + combines a driven I/O context, scheduler, resource types, and asynchronous operations, showing why + native file I/O is a separate subsystem rather than a blocking-pool alias. +- [Microsoft IOCP documentation](https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports) + describes a completion queue tied to overlapped file and socket handles, reinforcing that a + Windows native backend needs explicit handle and completion lifetimes. +- [liburing](https://github.com/axboe/liburing) exposes setup, submission, completion, feature, and + kernel-version concerns for Linux `io_uring`; phase 6A deliberately takes on none of that platform + surface. + +## Acceptance Criteria + +Phase 6A is complete only when the single `run_blocking()` API, focused cross-platform tests, +standalone example, updated readiness benchmark, public documentation, and HANDOFF agree on the same +contracts; Dev and Release strict builds and all tests pass; repeated cancellation races have zero +lost or duplicate invocation; file and loopback benchmark counts have zero unexpected failures; and +the change contains no native-I/O claim, duplicate pool type, hidden executor, or new dependency. diff --git a/examples/basic/src/main.cpp b/examples/basic/src/main.cpp index 4d218ee..308d905 100644 --- a/examples/basic/src/main.cpp +++ b/examples/basic/src/main.cpp @@ -9,6 +9,7 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; using namespace std::chrono_literals; @@ -62,6 +63,19 @@ Task print_worker_result( std::println("Worker pool result: {}", result); } +Task print_blocking_result( + ThreadPool::Scheduler blockingWorkers, + RunLoop::Scheduler caller) { + const auto result = co_await run_blocking( + blockingWorkers, + caller, + [] { + std::this_thread::sleep_for(1ms); + return 42; + }); + std::println("Blocking result: {}", result); +} + Task add_delayed( RunLoop::Scheduler scheduler, std::chrono::milliseconds delay, @@ -201,12 +215,16 @@ Task print_cancellation(RunLoop::Scheduler scheduler) { int main() { ThreadPool workers { 2 }; + ThreadPool blockingWorkers { 2 }; RunLoop loop {}; loop.run(print_answer(loop.get_scheduler())); loop.run(print_concurrent_results(loop.get_scheduler())); loop.run(print_worker_result( workers.get_scheduler(), loop.get_scheduler())); + loop.run(print_blocking_result( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); loop.run(print_task_group(loop.get_scheduler())); loop.run(print_recursive_group(loop.get_scheduler())); loop.run(print_event(loop.get_scheduler())); diff --git a/src/blocking.cppm b/src/blocking.cppm new file mode 100644 index 0000000..015e5d1 --- /dev/null +++ b/src/blocking.cppm @@ -0,0 +1,72 @@ +export module mcpplibs.cmp:blocking; + +import std; +import :task; +import :thread_pool; + +export namespace mcpplibs::cmp { + +template +requires ( + std::move_constructible && + std::invocable && + ( + std::same_as, void> || + ( + std::is_object_v> && + !std::is_array_v> && + std::move_constructible> + ) + ) && + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } +) +[[nodiscard]] Task> run_blocking( + ThreadPool::Scheduler blockingWorkers, + ReturnScheduler returnTo, + Function operation, + std::stop_token stopToken = {}) { + using Result = std::invoke_result_t; + + std::exception_ptr exception {}; + + if constexpr (std::same_as) { + try { + co_await blockingWorkers.schedule(std::move(stopToken)); + std::invoke(std::move(operation)); + } catch (...) { + exception = std::current_exception(); + } + + // 返回路径不可取消,否则结果会滞留在 blocking worker。 + co_await returnTo.schedule(); + + if (exception) { + std::rethrow_exception(exception); + } + + co_return; + } else { + std::optional result {}; + + try { + co_await blockingWorkers.schedule(std::move(stopToken)); + result.emplace(std::invoke(std::move(operation))); + } catch (...) { + exception = std::current_exception(); + } + + // 返回后再发布值或异常,调用方始终在显式目标 Scheduler 上继续。 + co_await returnTo.schedule(); + + if (exception) { + std::rethrow_exception(exception); + } + + co_return std::move(*result); + } +} + +} // namespace mcpplibs::cmp diff --git a/src/cmp.cppm b/src/cmp.cppm index d6cf754..1d182d6 100644 --- a/src/cmp.cppm +++ b/src/cmp.cppm @@ -4,6 +4,7 @@ export import :cancellation; export import :task; export import :run_loop; export import :thread_pool; +export import :blocking; export import :when_all; export import :task_group; export import :one_shot_event; diff --git a/tests/blocking_test.cpp b/tests/blocking_test.cpp new file mode 100644 index 0000000..dfc6cfe --- /dev/null +++ b/tests/blocking_test.cpp @@ -0,0 +1,645 @@ +#include + +import std; +import mcpplibs.cmp; + +namespace { + +using mcpplibs::cmp::OperationCancelled; +using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::Task; +using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::run_blocking; + +using BlockingScheduler = ThreadPool::Scheduler; +using ReturnScheduler = RunLoop::Scheduler; + +using namespace std::chrono_literals; + +struct ThreadObservation final { + int value_ {}; + std::thread::id workerThread_ {}; + std::thread::id resumedThread_ {}; +}; + +struct CancellationObservation final { + bool prerequisiteReached_ { false }; + bool stopRequested_ { false }; + bool cancelled_ { false }; + int invocations_ {}; + std::optional result_ {}; + std::thread::id resumedThread_ {}; +}; + +struct RaceObservation final { + int completed_ {}; + int cancelled_ {}; + int invalid_ {}; +}; + +struct IsolationObservation final { + bool operationStarted_ { false }; + bool loopProgressedWhileBlocked_ { false }; + bool operationCompleted_ { false }; +}; + +class MoveOnlyOperation final { +private: + std::unique_ptr value_ {}; + std::atomic* invocations_ {}; + std::thread::id* workerThread_ {}; + +public: + MoveOnlyOperation( + std::unique_ptr value, + std::atomic& invocations, + std::thread::id& workerThread) noexcept + : value_ { std::move(value) }, + invocations_ { &invocations }, + workerThread_ { &workerThread } {} + + MoveOnlyOperation(const MoveOnlyOperation&) = delete; + MoveOnlyOperation& operator=(const MoveOnlyOperation&) = delete; + MoveOnlyOperation(MoveOnlyOperation&&) noexcept = default; + MoveOnlyOperation& operator=(MoveOnlyOperation&&) noexcept = default; + ~MoveOnlyOperation() = default; + + std::unique_ptr operator()() & = delete; + + std::unique_ptr operator()() && { + invocations_->fetch_add(1, std::memory_order_relaxed); + *workerThread_ = std::this_thread::get_id(); + ++*value_; + return std::move(value_); + } +}; + +static_assert(std::move_constructible); +static_assert(!std::copy_constructible); +static_assert(std::invocable); +static_assert(!std::invocable); + +using MoveOnlyOffload = decltype(run_blocking( + std::declval(), + std::declval(), + std::declval())); + +static_assert(std::same_as>>); + +Task observe_move_only( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + std::atomic& invocations) { + std::thread::id workerThread {}; + auto result = co_await run_blocking( + blockingWorkers, + returnTo, + MoveOnlyOperation { + std::make_unique(41), + invocations, + workerThread + }); + + co_return ThreadObservation { + *result, + workerThread, + std::this_thread::get_id() + }; +} + +Task observe_void( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + std::atomic& invocations) { + std::thread::id workerThread {}; + + co_await run_blocking(blockingWorkers, returnTo, [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + workerThread = std::this_thread::get_id(); + }); + + co_return ThreadObservation { + invocations.load(std::memory_order_relaxed), + workerThread, + std::this_thread::get_id() + }; +} + +Task observe_other_pool( + BlockingScheduler blockingWorkers, + BlockingScheduler returnTo) { + std::thread::id workerThread {}; + const auto value = co_await run_blocking( + blockingWorkers, + returnTo, + [&] { + workerThread = std::this_thread::get_id(); + return 42; + }); + + co_return ThreadObservation { + value, + workerThread, + std::this_thread::get_id() + }; +} + +Task observe_exception( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo) { + try { + static_cast(co_await run_blocking( + blockingWorkers, + returnTo, + []() -> int { + throw std::runtime_error { "blocking failure" }; + })); + } catch (const std::runtime_error& error) { + co_return ThreadObservation { + error.what() == std::string_view { "blocking failure" } ? 42 : 0, + {}, + std::this_thread::get_id() + }; + } + + co_return ThreadObservation {}; +} + +Task observe_cancellation( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + std::stop_token stopToken, + std::atomic& invocations) { + bool cancelled { false }; + + try { + co_await run_blocking( + blockingWorkers, + returnTo, + [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + }, + stopToken); + } catch (const OperationCancelled&) { + cancelled = true; + } + + co_return CancellationObservation { + true, + stopToken.stop_requested(), + cancelled, + invocations.load(std::memory_order_relaxed), + {}, + std::this_thread::get_id() + }; +} + +Task block_worker( + BlockingScheduler scheduler, + std::counting_semaphore<1>& entered, + std::latch& release) { + co_await scheduler.schedule(); + entered.release(); + release.wait(); +} + +Task capture_cancellation( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + std::stop_token stopToken, + std::atomic& invocations, + bool& cancelled, + std::thread::id& resumedThread) { + try { + co_await run_blocking( + blockingWorkers, + returnTo, + [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + }, + stopToken); + } catch (const OperationCancelled&) { + cancelled = true; + } + + resumedThread = std::this_thread::get_id(); +} + +Task run_queued_cancellation( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo) { + std::counting_semaphore<1> entered { 0 }; + std::latch release { 1 }; + std::stop_source stopSource {}; + std::atomic invocations {}; + bool cancelled { false }; + std::thread::id resumedThread {}; + TaskGroup group {}; + + group.spawn(block_worker(blockingWorkers, entered, release)); + const bool blockerEntered = entered.try_acquire_for(2s); + + // eager spawn 返回时,offload 已经排在被阻塞 worker 后面。 + group.spawn(capture_cancellation( + blockingWorkers, + returnTo, + stopSource.get_token(), + invocations, + cancelled, + resumedThread)); + const bool stopRequested = stopSource.request_stop(); + release.count_down(); + + co_await group.join(); + co_return CancellationObservation { + blockerEntered, + stopRequested, + cancelled, + invocations.load(std::memory_order_relaxed), + {}, + resumedThread + }; +} + +Task capture_late_result( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + std::stop_token stopToken, + std::counting_semaphore<1>& entered, + std::latch& release, + std::atomic& invocations, + std::optional& result, + bool& cancelled, + std::thread::id& resumedThread) { + try { + result.emplace(co_await run_blocking( + blockingWorkers, + returnTo, + [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + entered.release(); + release.wait(); + return 42; + }, + stopToken)); + } catch (const OperationCancelled&) { + cancelled = true; + } + + resumedThread = std::this_thread::get_id(); +} + +Task run_late_cancellation( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo) { + std::counting_semaphore<1> entered { 0 }; + std::latch release { 1 }; + std::stop_source stopSource {}; + std::atomic invocations {}; + std::optional result {}; + bool cancelled { false }; + std::thread::id resumedThread {}; + TaskGroup group {}; + + group.spawn(capture_late_result( + blockingWorkers, + returnTo, + stopSource.get_token(), + entered, + release, + invocations, + result, + cancelled, + resumedThread)); + + const bool operationEntered = entered.try_acquire_for(2s); + const bool stopRequested = stopSource.request_stop(); + release.count_down(); + + co_await group.join(); + co_return CancellationObservation { + operationEntered, + stopRequested, + cancelled, + invocations.load(std::memory_order_relaxed), + result, + resumedThread + }; +} + +Task run_cancellation_races( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + std::thread::id expectedThread, + int raceCount) { + RaceObservation observation {}; + + for (int index { 0 }; index < raceCount; ++index) { + std::stop_source stopSource {}; + std::atomic invocations {}; + std::jthread canceller { [&stopSource] { + stopSource.request_stop(); + } }; + + try { + const auto result = co_await run_blocking( + blockingWorkers, + returnTo, + [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + return 42; + }, + stopSource.get_token()); + ++observation.completed_; + + if (result != 42 || + invocations.load(std::memory_order_relaxed) != 1) { + ++observation.invalid_; + } + } catch (const OperationCancelled&) { + ++observation.cancelled_; + + if (invocations.load(std::memory_order_relaxed) != 0) { + ++observation.invalid_; + } + } catch (...) { + ++observation.invalid_; + } + + canceller.join(); + + if (std::this_thread::get_id() != expectedThread) { + ++observation.invalid_; + } + } + + co_return observation; +} + +BlockingScheduler make_expired_scheduler() { + ThreadPool pool { 1 }; + return pool.get_scheduler(); +} + +Task mark_run_loop_ready( + ReturnScheduler scheduler, + bool& progressed) { + co_await scheduler.schedule(); + progressed = true; +} + +Task observe_run_loop_progress( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo) { + std::counting_semaphore<1> entered { 0 }; + std::latch release { 1 }; + std::atomic completed { false }; + bool progressed { false }; + TaskGroup group {}; + + group.spawn(run_blocking(blockingWorkers, returnTo, [&] { + entered.release(); + release.wait(); + completed.store(true, std::memory_order_release); + })); + + const bool operationStarted = entered.try_acquire_for(2s); + group.spawn(mark_run_loop_ready(returnTo, progressed)); + + // marker 先于当前 continuation 入队,证明 RunLoop 在阻塞调用期间仍能推进。 + co_await returnTo.schedule(); + const bool progressedWhileBlocked = progressed && + !completed.load(std::memory_order_acquire); + release.count_down(); + + co_await group.join(); + co_return IsolationObservation { + operationStarted, + progressedWhileBlocked, + completed.load(std::memory_order_acquire) + }; +} + +Task increment_offloaded( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + std::atomic& invoked, + int& returned) { + co_await run_blocking(blockingWorkers, returnTo, [&] { + invoked.fetch_add(1, std::memory_order_relaxed); + }); + ++returned; +} + +Task> run_many_offloads( + BlockingScheduler blockingWorkers, + ReturnScheduler returnTo, + int operationCount) { + std::atomic invoked {}; + int returned { 0 }; + TaskGroup group {}; + + for (int index { 0 }; index < operationCount; ++index) { + group.spawn(increment_offloaded( + blockingWorkers, + returnTo, + invoked, + returned)); + } + + co_await group.join(); + co_return std::pair { + invoked.load(std::memory_order_relaxed), + returned + }; +} + +TEST(CmpBlockingTest, IsLazyAndReturnsMoveOnlyValueOnRequestedRunLoop) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + std::atomic invocations {}; + const auto callerThread = std::this_thread::get_id(); + auto task = observe_move_only( + blockingWorkers.get_scheduler(), + loop.get_scheduler(), + invocations); + + EXPECT_EQ(invocations.load(std::memory_order_relaxed), 0); + const auto observation = loop.run(std::move(task)); + + EXPECT_EQ(observation.value_, 42); + EXPECT_EQ(invocations.load(std::memory_order_relaxed), 1); + EXPECT_NE(observation.workerThread_, callerThread); + EXPECT_EQ(observation.resumedThread_, callerThread); +} + +TEST(CmpBlockingTest, SupportsVoidAndReturnsToRequestedRunLoop) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + std::atomic invocations {}; + const auto callerThread = std::this_thread::get_id(); + const auto observation = loop.run(observe_void( + blockingWorkers.get_scheduler(), + loop.get_scheduler(), + invocations)); + + EXPECT_EQ(observation.value_, 1); + EXPECT_NE(observation.workerThread_, callerThread); + EXPECT_EQ(observation.resumedThread_, callerThread); +} + +TEST(CmpBlockingTest, CanReturnToAnotherThreadPool) { + ThreadPool blockingWorkers { 1 }; + ThreadPool returnWorkers { 1 }; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + const auto observation = loop.run(observe_other_pool( + blockingWorkers.get_scheduler(), + returnWorkers.get_scheduler())); + + EXPECT_EQ(observation.value_, 42); + EXPECT_NE(observation.workerThread_, callerThread); + EXPECT_NE(observation.resumedThread_, callerThread); + EXPECT_NE(observation.workerThread_, observation.resumedThread_); +} + +TEST(CmpBlockingTest, TransportsCallableExceptionOnReturnScheduler) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + const auto observation = loop.run(observe_exception( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); + + EXPECT_EQ(observation.value_, 42); + EXPECT_EQ(observation.resumedThread_, callerThread); +} + +TEST(CmpBlockingTest, PreCancellationSkipsCallableAndReturnsBeforeThrowing) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + std::stop_source stopSource {}; + std::atomic invocations {}; + const auto callerThread = std::this_thread::get_id(); + stopSource.request_stop(); + + const auto observation = loop.run(observe_cancellation( + blockingWorkers.get_scheduler(), + loop.get_scheduler(), + stopSource.get_token(), + invocations)); + + EXPECT_TRUE(observation.cancelled_); + EXPECT_EQ(observation.invocations_, 0); + EXPECT_EQ(observation.resumedThread_, callerThread); +} + +TEST(CmpBlockingTest, QueuedCancellationSkipsCallable) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + const auto observation = loop.run(run_queued_cancellation( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); + + EXPECT_TRUE(observation.prerequisiteReached_); + EXPECT_TRUE(observation.stopRequested_); + EXPECT_TRUE(observation.cancelled_); + EXPECT_EQ(observation.invocations_, 0); + EXPECT_EQ(observation.resumedThread_, callerThread); +} + +TEST(CmpBlockingTest, LateCancellationPreservesClaimedResult) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + const auto observation = loop.run(run_late_cancellation( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); + + EXPECT_TRUE(observation.prerequisiteReached_); + EXPECT_TRUE(observation.stopRequested_); + EXPECT_FALSE(observation.cancelled_); + ASSERT_TRUE(observation.result_.has_value()); + EXPECT_EQ(*observation.result_, 42); + EXPECT_EQ(observation.invocations_, 1); + EXPECT_EQ(observation.resumedThread_, callerThread); +} + +TEST(CmpBlockingTest, CancellationRaceInvokesAtMostOnce) { + constexpr int RACE_COUNT { 500 }; + ThreadPool blockingWorkers { 2 }; + RunLoop loop {}; + const auto observation = loop.run(run_cancellation_races( + blockingWorkers.get_scheduler(), + loop.get_scheduler(), + std::this_thread::get_id(), + RACE_COUNT)); + + EXPECT_EQ(observation.completed_ + observation.cancelled_, RACE_COUNT); + EXPECT_EQ(observation.invalid_, 0); +} + +TEST(CmpBlockingTest, RejectsExpiredWorkerAndReturnSchedulers) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + RunLoop inactiveLoop {}; + std::atomic invocations {}; + const auto expiredWorker = make_expired_scheduler(); + const auto expiredReturn = make_expired_scheduler(); + + auto rejectedWorker = run_blocking( + expiredWorker, + loop.get_scheduler(), + [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + }); + EXPECT_THROW(loop.run(std::move(rejectedWorker)), std::logic_error); + EXPECT_EQ(invocations.load(std::memory_order_relaxed), 0); + + auto rejectedReturn = run_blocking( + blockingWorkers.get_scheduler(), + expiredReturn, + [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + }); + EXPECT_THROW(loop.run(std::move(rejectedReturn)), std::logic_error); + EXPECT_EQ(invocations.load(std::memory_order_relaxed), 1); + + auto inactiveReturn = run_blocking( + blockingWorkers.get_scheduler(), + inactiveLoop.get_scheduler(), + [&] { + invocations.fetch_add(1, std::memory_order_relaxed); + }); + EXPECT_THROW(loop.run(std::move(inactiveReturn)), std::logic_error); + EXPECT_EQ(invocations.load(std::memory_order_relaxed), 2); +} + +TEST(CmpBlockingTest, KeepsRunLoopResponsiveWhileCallableBlocks) { + ThreadPool blockingWorkers { 1 }; + RunLoop loop {}; + const auto observation = loop.run(observe_run_loop_progress( + blockingWorkers.get_scheduler(), + loop.get_scheduler())); + + EXPECT_TRUE(observation.operationStarted_); + EXPECT_TRUE(observation.loopProgressedWhileBlocked_); + EXPECT_TRUE(observation.operationCompleted_); +} + +TEST(CmpBlockingTest, ManyConcurrentOffloadsCompleteExactlyOnce) { + constexpr int OPERATION_COUNT { 5'000 }; + ThreadPool blockingWorkers { 4 }; + RunLoop loop {}; + const auto [invoked, returned] = loop.run(run_many_offloads( + blockingWorkers.get_scheduler(), + loop.get_scheduler(), + OPERATION_COUNT)); + + EXPECT_EQ(invoked, OPERATION_COUNT); + EXPECT_EQ(returned, OPERATION_COUNT); +} + +} // namespace From 816144f53e930c20e3c54032a32767576b836da3 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 00:20:27 +0800 Subject: [PATCH 04/17] =?UTF-8?q?=E5=AE=9E=E7=8E=B0=20TCP=20I/O=20?= =?UTF-8?q?=E4=B8=8A=E4=B8=8B=E6=96=87=E7=94=9F=E5=91=BD=E5=91=A8=E6=9C=9F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 97 +++- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 311 +++++++++++ ...-08-29-cmp-phase6b-tcp-client-v1-design.md | 501 ++++++++++++++++++ mcpp.toml | 3 + src/cmp.cppm | 1 + src/tcp.cppm | 130 +++++ tests/tcp_test.cpp | 23 + 7 files changed, 1050 insertions(+), 16 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md create mode 100644 docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md create mode 100644 src/tcp.cppm create mode 100644 tests/tcp_test.cpp diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index bbb493b..b5f0ac7 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -5,15 +5,18 @@ CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 `mcpplibs.cmp`。当前已实现 懒启动单消费者 `Task`、变参/vector `when_all()`、静止点 `TaskGroup`、一次性与可复用 事件、RAII `AsyncMutex`、带定时和取消的调用线程 `RunLoop`、固定大小的 `ThreadPool`,以及 -用于隔离同步调用的 `run_blocking()`。 +用于隔离同步调用的 `run_blocking()`。Phase 6B 已加入拥有单一私有 I/O driver 的 +`IoContext` 生命周期骨架。 -`.xlings.json` 固定 mcpp 2026.8.11.2;当前工具链为 LLVM 22.1.8,测试依赖为 -`compat.gtest` 1.15.2。`examples/basic` 是独立 path-dependency consumer。 +`.xlings.json` 固定 mcpp 2026.8.11.2;当前工具链为 LLVM 22.1.8,运行时依赖为 +`chriskohlhoff.asio` 1.38.1,测试依赖为 `compat.gtest` 1.15.2。`examples/basic` 是独立 +path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1 已完成本地实现、文档同步和验证;远程尚未更新。原生异步 I/O -不属于 6A,下一项设计工作是 Phase 6B Spec。 +Phase 6A blocking offload v1 已作为本地提交 `701aa8b` 完成,尚未推送。Phase 6B TCP client +v1 已进入实现:Plan 第 1、2 项完成,Asio backend gate 与 `IoContext` 生命周期通过完整 Dev +验证;尚未实现 native-completion bridge 或 `TcpStream` 公共 API。 ## 已完成工作 @@ -31,6 +34,27 @@ Phase 6A blocking offload v1 已完成本地实现、文档同步和验证;远 CMP ThreadPool 和 `run_blocking()`;负载及成功/失败硬检查保持不变。 - 三份 README、三份架构文档、Phase 6A Design/Plan 和 v1 readiness 数据报告已同步到实现 事实。 +- 完成 Phase 6B 依赖核查:mcpp-index 当前提供 `chriskohlhoff.asio@1.38.1`,其 C++23 模块 + `asio` 覆盖 Linux、macOS 和 Windows,能够复用 epoll、kqueue 与 IOCP 后端。 +- 新增并批准 Phase 6B TCP client v1 Design,确定首个原生异步 I/O 切片仅包含数值地址 TCP + 客户端、`IoContext`、move-only `TcpStream`、`read_some()`、`write_all()` 和显式返回 + Scheduler。 +- Spec 已明确 buffer/handle 生命周期、单读单写并发、EOF/partial transfer、错误映射、 + stop/close/completion 竞态、exactly-once、context drain/join 及三平台确定性测试门槛。 +- 新增 Phase 6B Implementation Plan,将实现拆为 dependency/backend gate、`IoContext` + 生命周期、单一 native-completion bridge、connect/read/write、取消/close/shutdown 竞态、 + 跨平台负载、readiness 客户端迁移、文档和完整验证十个顺序步骤。 +- 使用 mcpp 2026.8.11.2 精确加入 `[dependencies.chriskohlhoff] asio = "1.38.1"`。 +- 新增 `mcpplibs.cmp:tcp` 分区,私有导入 `std`、`asio`、`:cancellation` 和 `:task`;根模块 + 只重导出 `:tcp`,没有暴露 Asio 类型。 +- 实现不可复制、不可移动的公共 `IoContext`:共享私有状态拥有一个 `asio::io_context`、 + persistent work guard、admission mutex/flag 和线性弱 socket registry;公开对象拥有一个 + `std::jthread` driver。 +- `IoContext` shutdown 与普通 post 共用 admission 锁排序:先停止接纳,再在已接纳工作后关闭 + live socket、释放 guard、自然排空 handler 并 join;不调用 `io_context::stop()`,driver + 线程内自析构会终止而不是 self-join。 +- 新增 `tests/tcp_test.cpp`,以编译期断言锁定 `IoContext` 的默认构造及精确 copy/move traits, + 并以运行测试验证构造、driver 启动、析构关闭和 join 不挂起。 ## 重要决策 @@ -44,25 +68,50 @@ Phase 6A blocking offload v1 已完成本地实现、文档同步和验证;远 - Phase 6A 仍是每个运行中同步调用占用一个系统线程的隔离方案,不宣称原生非阻塞 I/O。 - 队列沿用 ThreadPool 的无界共享 FIFO;背压、超时、强制中断和 worker replacement 均未在 没有实测需求前增加。 +- Phase 6B 先做 TCP 而非文件:现有 Asio package 的 socket 能统一覆盖三平台,file backend + 不能提供同等平台面。 +- C++23 标准库没有网络 API,因此 6B 选择精确固定 `chriskohlhoff.asio@1.38.1`,私有导入, + 不向 CMP 公共 API 暴露 Asio 类型;不自研三套 OS backend。 +- `IoContext` v1 固定一个私有 driver thread,不暴露通用 post/run/线程数接口;应用结果始终 + 经调用者指定 Scheduler 返回。 +- socket state 只弱引用 context;共享 context state 由公开 `IoContext` 与 driver 持有, + socket 不能延长公开 driver 生命周期。v1 registry 使用 O(n) 弱引用扫描,只有实测成本显著 + 时才替换。 +- `TcpStream` 允许一项 pending read 与一项 pending write;同方向重叠直接拒绝,不增加隐式 + 排队。主动 close、context shutdown 与已发起 write 的取消均有明确关闭语义。 +- v1 不含 DNS、server、TLS、UDP、文件 I/O、timeout、socket option 或隐式 executor;这些都 + 没有在首个可验证切片前预建。 +- 新测试只在 `tests/tcp_test.cpp` 内用同步 Asio loopback fixture;不为测试增加公共 server。 +- 现有三平台 CI 会自动发现新测试,不预先修改 workflow;`examples/basic` 保持短小且不引入 + live network,仍作为外部 path consumer 回归运行。 +- readiness 只替换 TCP client;同步 loopback server 继续通过 Phase 6A `run_blocking()` + 运行,compute/file workload 与硬计数保持不变。 ## 修改 / 重要文件 -- 核心:`src/blocking.cppm`、`src/cmp.cppm` -- 测试:`tests/blocking_test.cpp` +- 当前实现:`mcpp.toml`、`src/tcp.cppm`、`src/cmp.cppm` +- Phase 6A 核心:`src/blocking.cppm` +- 测试:`tests/blocking_test.cpp`、`tests/tcp_test.cpp` - 示例:`examples/basic/src/main.cpp` - 压测:`benchmarks/v1-readiness/src/main.cpp`、 `docs/benchmarks/2026-08-29-cmp-v1-readiness.md` - 方案:`docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md`、 `docs/superpowers/plans/2026-08-29-cmp-phase6a-blocking-offload-v1.md` +- 已批准设计: + `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` +- 当前实施计划: + `docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md` - 公共文档:`README.md`、`README.zh.md`、`README.zh.hant.md`、`docs/architecture.md`、 `docs/architecture.zh.md`、`docs/architecture.zh.hant.md` ## 验证情况 -- `mcpp build --profile dev --strict --cache=off`:通过。 -- `mcpp test --profile dev --strict --cache=off`:9 个二进制、116/116 通过。 -- `mcpp build --profile release --strict --cache=off`:通过。 -- `mcpp test --profile release --strict --cache=off`:9 个二进制、116/116 通过。 +- Phase 6A baseline 的 `mcpp build --profile dev --strict --cache=off`:通过。 +- Phase 6A baseline 的 `mcpp test --profile dev --strict --cache=off`:9 个二进制、116/116 + 通过。 +- Phase 6A baseline 的 `mcpp build --profile release --strict --cache=off`:通过。 +- Phase 6A baseline 的 `mcpp test --profile release --strict --cache=off`:9 个二进制、 + 116/116 通过。 - Dev 定向 `blocking_test`:11/11 通过。 - Release `CancellationRaceInvokesAtMostOnce` 连续执行 100 轮:100/100 通过。 - `examples/basic` 的 `mcpp run`:通过,包含 `Blocking result: 42`,退出码 0。 @@ -72,6 +121,14 @@ Phase 6A blocking offload v1 已完成本地实现、文档同步和验证;远 吞吐已写入 benchmark 报告。 - 当前 mcpp 仍输出 SubOS 缺少 `subos_info` 的既有环境提示,但所有构建和运行成功。 - 以上均为本机 Linux/WSL2 结果;尚未执行 GitHub 三平台 CI 或其他远程操作。 +- Phase 6B backend gate:`mcpp build --profile dev --strict --cache=off` 通过;mcpp 下载并编译 + `chriskohlhoff.asio` 1.38.1,CMP 使用 LLVM 22.1.8 构建成功。 +- Phase 6B `IoContext` focused gate:`mcpp test tcp_test --profile dev --strict --cache=off` 的 + 1/1 测试通过。 +- 加入 `IoContext` 后,`mcpp build --profile dev --strict --cache=off` 再次通过; + `mcpp test --profile dev --strict --cache=off` 为 10 个二进制、117/117 通过。 +- 本 checkpoint 未运行当前 Phase 6B 的 Release、example 或 benchmark;它们仍属于后续 Plan + gate,既有 Phase 6A Release/example/benchmark 结果不代表 TCP API 已实现。 ## 已知问题 / 风险 @@ -82,14 +139,22 @@ Phase 6A blocking offload v1 已完成本地实现、文档同步和验证;远 - 返回 Scheduler 的 owner 必须持续存活;RunLoop Scheduler 还必须处于 active `run()` 中。 - `run_blocking()` 是线程隔离,不是 epoll、io_uring、kqueue 或 IOCP 等原生异步 I/O。 - 三平台兼容性仍需远程 CI 确认。 +- `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 + Windows 仍需后续远程 CI 验证。 +- Phase 6B 当前只有 backend 与 `IoContext` 生命周期;native-completion bridge、`TcpStream` + connect/read/write、取消竞态和 benchmark 迁移均尚未实现。 +- 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 用户审查 Phase 6A 本地改动;Commit、Push、PR 与 CI 均需分别获得明确授权。 -2. 用户确认进入下一设计阶段后,为一个窄资源族编写 Phase 6B 原生异步 I/O Spec;实现前仍需 - 单独批准。 +1. 按 Phase 6B Plan 第 3 项实现唯一的 native-completion bridge,锁定只有 Asio handler 能在 + native initiation 后恢复私有协程,并准备统一的 error/count/cancellation state。 +2. 完成本地 Dev/Release、race、example 与五轮 readiness 验证后,再等待远程三平台 CI 所需 + 的单独授权。 +3. 当前 Phase 6B 代码、manifest、Design、Plan 与 HANDOFF 均未提交;是否创建本地提交、 + Push、PR 或 CI 均需对应明确授权。 ## 推荐下一步 -先审查并提交 Phase 6A;随后只设计 Phase 6B 的最小原生异步 I/O 边界,优先选择单一资源族, -不把文件、网络和所有平台后端一次性绑定在同一阶段。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 3 项,只实现私有 native-completion +bridge 及其 exactly-once/cancellation 基础;connect 与 loopback fixture 留到第 4 项。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md new file mode 100644 index 0000000..3bb9705 --- /dev/null +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -0,0 +1,311 @@ +# CMP Phase 6B TCP Client v1 Implementation Plan + +**Date:** 2026-08-30 +**Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` +**Status:** In progress — `IoContext` lifecycle gate complete +**Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests + +## Execution Rule + +Implement the approved TCP-client slice in the order below. Keep the public API and exclusions in +the Design unchanged. At each gate, run the smallest focused check first and stop on a failure +instead of adding a fallback backend or broadening the API. + +The implementation remains one module partition and one test file. Reuse standalone Asio for the +OS backend, CMP `Task` for ownership, existing Scheduler types for return affinity, and +`OperationCancelled` for local cancellation. Do not add a generic I/O hierarchy, public server, +future, callback API, or hidden process-wide executor. + +## 1. Prove the packaged backend + +Add the exact dependency through mcpp: + +```text +mcpp add chriskohlhoff.asio@1.38.1 +``` + +Confirm the manifest records the explicit `chriskohlhoff` namespace and exact `1.38.1` version. +Do not add Asio separately to `examples/basic` or `benchmarks/v1-readiness`; both consume CMP as a +path dependency. + +Create `src/tcp.cppm` with: + +```cpp +export module mcpplibs.cmp:tcp; + +import std; +import asio; +import :cancellation; +import :task; +``` + +Re-export `:tcp` from `src/cmp.cppm`. Keep `asio` private: no public signature, alias, exported +declaration, or root-module re-export may mention an Asio type. + +Run a strict cache-off Dev build immediately. If the packaged module does not compile with the +pinned LLVM/mcpp environment, stop and diagnose that package boundary; do not replace it with +platform headers or three handwritten backends. + +**Completion — 2026-08-30:** mcpp 2026.8.11.2 added +`[dependencies.chriskohlhoff] asio = "1.38.1"`; `src/tcp.cppm` privately imports `asio` and the root +module re-exports only `:tcp`. `mcpp build --profile dev --strict --cache=off` passed with LLVM +22.1.8. No Phase 6B public type or platform fallback was added at this gate. + +## 2. Implement `IoContext` ownership and shutdown + +In `src/tcp.cppm`, add the public immovable `IoContext` and only the private state required by the +Design: + +- one `asio::io_context`; +- one persistent work guard; +- one admission mutex and accepting flag; +- one vector of weak socket states for shutdown; +- one `std::jthread` owned by `IoContext`. + +Keep the shared context state declared before the driver member, as the existing `ThreadPool` keeps +its shared state alive until worker join. Let the driver capture the shared state while +`IoContext` itself remains the object that closes admission and joins the thread. Socket handles +retain only a weak context reference and cannot extend the public driver lifetime. + +Linearize every accepted initiation/close post and the shutdown post through the same admission +mutex. Destruction must: + +1. reject new posts; +2. queue shutdown after all previously accepted posts; +3. close every live registered socket on the I/O thread; +4. reset the work guard; +5. drain native completion handlers far enough to enqueue their return-Scheduler hops; +6. join the driver. + +Do not call `io_context::stop()`. Detect destruction from the driver thread and terminate rather +than self-join. Propagate construction/thread-creation failures without leaving a thread alive. + +Use a linear weak-state registry for v1 and prune expired entries while registering or shutting +down. Add a short `ponytail:` comment that the O(n) scan should change only if measured shutdown or +registration cost becomes significant. + +Start `tests/tcp_test.cpp` with compile-time checks for the exact copy/move traits and runtime checks +for clean construction/destruction. The test imports `std`, `asio`, and `mcpplibs.cmp`; it must not +include platform socket headers. + +**Completion — 2026-08-30:** Added the public immovable `IoContext`. Its shared private state owns +one `asio::io_context`, persistent work guard, admission mutex/flag, and linear weak socket +registry; `IoContext` owns one `std::jthread`. Shutdown rejects new posts, queues socket closure and +guard release behind accepted posts, drains `io_context` without `stop()`, detects self-join, and +joins the driver. `tests/tcp_test.cpp` locks the public traits and clean lifecycle. The focused test +passed 1/1; strict cache-off Dev build and the full Dev suite passed 117/117 across 10 binaries. + +## 3. Add the single native-completion bridge + +Implement one private callback-to-CMP awaiter that normalizes the Asio completion forms to: + +```text +std::error_code + transferred byte count +``` + +Connect wraps its one-argument handler with a zero byte count; read and write use their native +counts. Keep the bridge private in `src/tcp.cppm` rather than adding a public completion-token or +executor abstraction. + +The per-operation state must keep alive: + +- the suspended private coroutine continuation; +- its `asio::cancellation_signal`; +- the error code and transferred count; +- cancellation provenance; +- a stop callback that holds weak operation/context references. + +Initiate the Asio operation on the sole I/O thread, then install the stop callback. The callback +sets cancellation provenance and asks the context to emit `cancellation_type::all` on that same I/O +thread. It must never resume the coroutine or touch the socket from the requesting thread. + +After native initiation, only the Asio handler may resume the private coroutine. Close and stop +only request that handler. This is the exactly-once rule; do not add a competing manual-resume +branch. + +Every public template is a non-coroutine wrapper that copies the context/socket state, concrete +return Scheduler, stop token, address string, and span descriptor before returning a private lazy +helper Task. This avoids storing a lazy `this`, `IoContext&`, or address `string_view` reference. + +Each helper captures value/error/cancellation, then awaits `returnTo.schedule()` without a stop +token before publishing the outcome. Use `std::scope_exit` for direction-admission cleanup rather +than creating another guard class. A return-Scheduler failure remains the observable exception. + +## 4. Implement connect and the loopback fixture + +Add `TcpStream::connect()` with the Design signature. On the I/O thread: + +1. accept the context post or retain its `std::logic_error`; +2. honor a pre-stopped token before parsing or opening a socket; +3. copy/parse only numeric IPv4 or IPv6 text; +4. construct and register one private socket state; +5. start `async_connect()` through the native-completion bridge; +6. return a move-only `TcpStream` only after a successful return-Scheduler hop. + +Use Asio's original `std::error_code` in `std::system_error`. Do not add a CMP network error enum, +reject port zero, or invoke a resolver. If the return hop fails after native connect succeeds, the +undelivered stream's RAII destruction must close the socket. + +In `tests/tcp_test.cpp`, add one small synchronous Asio loopback-server fixture on a `std::jthread`. +It binds port zero, exposes the selected port, uses latches/semaphores for protocol gates, owns all +server-side buffers, and joins deterministically. Keep protocol variants in this one fixture; do +not build a reusable production server layer for tests. + +Lock connect behavior with focused tests for: + +- Task laziness and copied temporary address storage; +- numeric IPv4 success and RunLoop return-thread affinity; +- IPv6 loopback when the host can bind it; +- invalid address and a test-owned non-listening endpoint as `std::system_error`; +- pre-cancelled connect with invalid address text yielding `OperationCancelled`, proving address + parsing and native initiation were skipped; +- returning through `ThreadPool::Scheduler`; +- inactive/expired return Scheduler behavior. + +Use timeouts only as deadlock guards. Do not use elapsed time or platform-specific error numbers as +pass conditions. + +## 5. Implement read, write, EOF, and overlap + +Add `read_some()` and `write_all()` against the same private socket state. + +All socket initiation occurs on the I/O thread. Maintain atomic logical-open/close state for +thread-safe `is_open()` and one pending flag per direction. One read and one write may coexist; +same-direction admission remains held until the public Task publishes its outcome. Reject overlap +with `std::logic_error` after the requested return hop rather than queueing hidden operations. + +For reads: + +- validate the resource before pre-cancellation and empty-buffer success; +- perform one `async_read_some()` for a non-empty span; +- return partial bytes immediately; +- retain EOF/error reported alongside bytes and deliver it on the next read; +- keep EOF sticky while preserving the write direction; +- mark the stream closed on a non-EOF socket error, while delivering one retained error before + later closed-state rejection. + +For writes: + +- validate the resource before pre-cancellation and empty-buffer success; +- use `asio::async_write()` for the all-or-error contract; +- close after any initiated write cancellation or socket error; +- return `void` on the exact full transfer; +- never retry a partial write internally. + +Add deterministic loopback tests for empty buffers, full write, partial read, server-send-then-close +with all bytes observed before repeated EOF, simultaneous read/write, a rejected second read, and a +rejected second write. To keep the second-write check deterministic, hold the first operation at a +gated return Scheduler until the overlap result is observed. + +## 6. Implement cancellation, close, and shutdown races + +Add thread-safe `close()` and `is_open()`. `close()` marks logical state closed synchronously, then +queues native close; it is idempotent, non-blocking, non-throwing, and a no-op on a moved-from +handle. `TcpStream` destruction performs the same request, while move construction transfers the +single logical ownership. + +Track enough private provenance to distinguish local token/close/context cancellation from an +unrelated socket error when Asio reports `operation_aborted`. Translate only the local cases to +`OperationCancelled`. + +Add deterministic tests for: + +- active pending-read cancellation with the stream remaining usable; +- close during a pending read and idempotent repeated close; +- moved-from `close()`/`is_open()` and rejected moved-from operations; +- close-versus-completion and stop-versus-completion repeated races, each selecting one outcome; +- active write cancellation closing the stream; +- `IoContext` destruction with a pending read, handler drain, return handoff, and driver join; +- a stream outliving its destroyed context and failing later operations safely. + +Prove a read is admitted before requesting active cancellation by using the deterministic +same-direction-overlap rejection while the server withholds data. For races, release server data +and request stop/close from a barrier; accept either documented winner but assert exactly one +completion and no duplicate buffer access. + +## 7. Add the load and cross-platform gate + +Extend the same test file with many concurrent IPv4 loopback clients. Give every client independent +stream/buffer ownership, count success/cancellation/error exactly once, and assert no lost or +duplicate completion. Keep the count high enough to exercise one I/O driver but low enough for all +three hosted CI systems. + +Run the full `tcp_test` under both Dev and Release, then repeat the two completion-race tests in the +Release binary for at least 100 rounds. A hang, duplicate completion, unexpected exception, or +buffer mismatch fails the gate. + +The existing Linux/macOS/Windows workflows already build the root package and auto-discover +`tests/tcp_test.cpp`; do not edit them merely to name the new test. Remote three-platform results +remain an acceptance gate and can be recorded only after the user separately authorizes the +required remote workflow. + +## 8. Migrate only the readiness TCP client + +Keep `benchmarks/v1-readiness` POSIX-only and retain its synchronous loopback server on the existing +blocking `ThreadPool`. Do not touch the compute or file scenarios. + +For each network worker: + +1. create a test-owned listening endpoint before starting its tasks; +2. run accept/read/echo on `run_blocking()`; +3. connect one `TcpStream` through a shared benchmark `IoContext`; +4. call `write_all()` and loop `read_some()` until the 256-byte echo is complete; +5. count the existing 5,000 successes; +6. connect to a held, non-listening loopback endpoint for the existing 25 expected failures. + +Preserve `NETWORK_WORKERS`, `NETWORK_ROUNDS`, `NETWORK_FAILURES`, payload size, CSV columns, and +hard accounting. Remove only the old blocking client socket operations that `TcpStream` replaces. +Timing remains local evidence, not a pass threshold. + +Build the benchmark in Release strict mode and run five measured rounds. Record the new raw data in +`docs/benchmarks/2026-08-29-cmp-v1-readiness.md`, explicitly labeling the server as blocking and the +client as Phase 6B native async TCP. + +## 9. Synchronize the developer documentation + +After behavior is verified, update the three README and three architecture variants with: + +- the `IoContext`/`TcpStream` public surface; +- numeric-address-only connect; +- borrowed buffer lifetime; +- explicit return Scheduler affinity; +- one-read/one-write concurrency and close/cancellation behavior; +- the distinction between Phase 6A thread isolation and Phase 6B native async TCP; +- the verified test and benchmark counts. + +Do not add a live-network call or a platform-specific loopback server to `examples/basic`; the +portable test fixture and readiness benchmark already exercise the API, while the basic example +should stay short and deterministic. Still run the unchanged example to verify that adding the +runtime dependency does not break an external path consumer. + +## 10. Full verification and state record + +Run focused checks first, then the complete local matrix: + +```text +mcpp build --profile dev --strict --cache=off +mcpp test --profile dev --strict --cache=off +mcpp build --profile release --strict --cache=off +mcpp test --profile release --strict --cache=off +cd examples/basic && mcpp run +cd benchmarks/v1-readiness && mcpp build --profile release --strict --cache=off +``` + +Run the readiness benchmark for five rounds and the focused Release cancellation/close races for +100 rounds. Record exact commands, test totals, race totals, benchmark counts, and failures; do not +describe an unexecuted platform or check as passing. + +Only after local verification: + +- mark the Design implemented locally; +- mark this Plan complete and add a concise completion record; +- update `.agent/HANDOFF.md` with implementation facts, remaining remote CI, and known limits. + +Phase 6B reaches the Design's final completion state only after Linux, macOS, and Windows CI all +pass. Until then, report it as locally implemented and verified with remote compatibility pending. + +## Scope Guard + +If implementation reveals a need for DNS, listener APIs, timeouts, generic endpoints, socket +options, multiple driver threads, an operation queue, or a second I/O resource family, stop and +write a separate design. None belongs in this plan. diff --git a/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md b/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md new file mode 100644 index 0000000..5f31d06 --- /dev/null +++ b/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md @@ -0,0 +1,501 @@ +# CMP Phase 6B TCP Client v1 Design + +**Date:** 2026-08-29 +**Status:** Approved for implementation +**Baseline:** Phase 6A is locally committed as `701aa8b` and passes 116/116 tests + +## Purpose + +Add the first kernel-backed asynchronous I/O surface to CMP: a small TCP client stream that works +on Linux, macOS, and Windows without occupying one CMP worker per pending socket operation. + +The developer-facing path stays explicit and short: + +```cpp +IoContext io {}; +RunLoop loop {}; + +auto request(RunLoop::Scheduler caller) -> Task { + auto stream = co_await TcpStream::connect( + io, + caller, + "127.0.0.1", + 8080); + + const std::array requestBytes { + std::byte { 'p' }, + std::byte { 'i' }, + std::byte { 'n' }, + std::byte { 'g' } + }; + co_await stream.write_all(caller, requestBytes); + + std::array reply {}; + const auto size = co_await stream.read_some(caller, reply); + co_return bytes_to_string(std::span { reply }.first(size)); +} +``` + +`IoContext` owns the operating-system completion driver. Every operation accepts an explicit +return Scheduler, so raw I/O completion never leaks the private I/O thread into application code. +`TcpStream` owns the socket while each pending operation borrows its supplied byte buffer. + +This phase is deliberately one narrow vertical slice. It proves the backend, lifetime, +cancellation, close, shutdown, and return-affinity contracts before CMP adds more resource +families. + +## Scope Decision + +Phase 6B v1 includes only: + +- outbound TCP connections to numeric IPv4 or IPv6 addresses; +- one move-only connected stream type; +- partial reads, write-all, explicit close, and open-state inspection; +- one private I/O driver thread per explicit `IoContext`; +- per-operation cancellation through `std::stop_token`; +- completion through a caller-selected CMP-compatible Scheduler. + +It excludes TCP listening, hostname resolution, TLS, UDP, Unix-domain sockets, file I/O, timeouts, +and socket options. None is needed to prove the first portable async-I/O contract. + +TCP is selected before file I/O because the available standalone Asio package exposes portable +socket support on all three CI systems. Its file support is not a matching three-platform surface: +the packaged configuration disables Linux `io_uring` support and macOS has no equivalent Asio file +backend. A single portable networking slice is smaller and more useful than three unrelated file +adapters. + +Numeric addresses avoid hiding blocking name resolution inside an API advertised as non-blocking. +DNS can be designed later with its own resolver lifetime, result list, cancellation, and timeout +rules. + +## Backend and Dependency + +Add one exact runtime dependency: + +```text +chriskohlhoff.asio@1.38.1 +``` + +The package is already present in `mcpp-index`, exposes the C++23 module `asio`, uses standalone +Asio with separate compilation, and supports Linux, macOS, and Windows. CMP privately imports it; +the public module never uses an Asio type and never `export import`s `asio`. + +This dependency is preferred because: + +1. C++23 has no standard networking API. +2. Asio already maps Linux to epoll, macOS to kqueue, and Windows to IOCP while presenting one + completion contract. +3. Hand-writing and maintaining those three backends would add substantially more race and + shutdown code without improving CMP's public API. +4. Standalone Asio is narrower than adding Boost and does not require a second general-purpose + event-loop framework. +5. The exact mcpp package version makes local and CI resolution reproducible. + +Updating the dependency is not part of Phase 6B implementation. A version change requires its own +compatibility check instead of silently following the newest upstream release. + +## Public API + +Add one `mcpplibs.cmp:tcp` partition and export these two types from `mcpplibs.cmp`: + +```cpp +namespace mcpplibs::cmp { + +class IoContext final { +public: + IoContext(); + + IoContext(const IoContext&) = delete; + IoContext& operator=(const IoContext&) = delete; + IoContext(IoContext&&) = delete; + IoContext& operator=(IoContext&&) = delete; + + ~IoContext(); +}; + +class TcpStream final { +public: + TcpStream() = delete; + TcpStream(const TcpStream&) = delete; + TcpStream& operator=(const TcpStream&) = delete; + TcpStream(TcpStream&&) noexcept; + TcpStream& operator=(TcpStream&&) = delete; + + ~TcpStream(); + + template + requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } + ) + [[nodiscard]] static Task connect( + IoContext& context, + ReturnScheduler returnTo, + std::string_view numericAddress, + std::uint16_t port, + std::stop_token stopToken = {}); + + template + requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } + ) + [[nodiscard]] Task read_some( + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken = {}); + + template + requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } + ) + [[nodiscard]] Task write_all( + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken = {}); + + void close() noexcept; + [[nodiscard]] bool is_open() const noexcept; +}; + +} // namespace mcpplibs::cmp +``` + +The API adds no executor base class. `ReturnScheduler` follows the existing `run_blocking()` +pattern and works with `RunLoop::Scheduler`, `ThreadPool::Scheduler`, or another scheduler with the +same `schedule()` awaitable expression. + +`IoContext` is immovable because it is the visible lifetime and shutdown anchor for its driver +thread. `TcpStream` is move-constructible so a connected stream can be returned from `Task`, but v1 +does not invent move-assignment semantics for replacing a live socket with pending operations. + +## Laziness and Argument Ownership + +All three asynchronous operations return ordinary lazy CMP Tasks. Calling a function only captures +its inputs; address parsing, context admission, socket initiation, and buffer access begin when the +Task is awaited. + +The public templates are non-coroutine wrappers that copy the necessary handles before returning a +private helper coroutine. This prevents lazy coroutine frames from retaining a dangling `this` or +`IoContext&` if the visible object is moved or destroyed before the Task starts. + +Argument lifetime rules are: + +- `numericAddress` is copied into owned storage before `connect()` returns; +- `ReturnScheduler` and `std::stop_token` are stored by value in the Task; +- the lazy Task holds the private socket state by shared ownership, and an accepted operation keeps + that state alive through its handler and return hop; +- read and write spans are borrowed, and their underlying storage must remain valid and unmodified + by the caller until the returned Task completes; +- destroying an unstarted Task performs no I/O and releases its captured values. + +The shared private socket state is an implementation lifetime device, not shared public socket +ownership. Only the move-only `TcpStream` is the logical owner. + +## I/O Context Model + +Each `IoContext` owns exactly one private `asio::io_context`, one work guard, and one driver thread. +Construction starts the driver; destruction performs graceful shutdown and joins it. Applications +normally share one `IoContext` across many streams rather than creating one context per connection. +If context allocation or driver-thread creation fails, construction releases partial state and +propagates the original exception; no background thread escapes a failed constructor. + +One driver thread is intentional for v1: + +- socket initiation, cancellation, close, and raw completion are serialized; +- Asio documents shared socket objects as unsafe for unsynchronized concurrent access; +- the public return Scheduler keeps application continuations off this thread; +- an adjustable worker count and strands add no value until a benchmark proves this driver is the + limiting resource. + +No user callback, Task parent, or ordinary CMP continuation runs directly from an Asio completion +handler. The handler resumes only the private operation helper. That helper stores the outcome and +then awaits `returnTo.schedule()` before publishing it. + +`IoContext` is not a general public executor: it has no `run()`, `post()`, worker-count, native +handle, or Asio access API. + +## Connect Contract + +`connect()` accepts a numeric address in the syntax understood by Asio's address parser and a +16-bit port. It copies and parses the address, creates the socket, registers its private state with +the context, and starts one asynchronous connect. + +On success it returns an open `TcpStream`. On failure the temporary socket is closed and no stream +escapes. Invalid text and operating-system connect errors are reported as `std::system_error` with +the original `std::error_code`; CMP does not create a duplicate networking error enum. + +Port zero is passed to the operating system rather than rejected by a CMP-specific rule. Hostnames +such as `localhost` are invalid in v1 because no resolver is invoked. + +Asio guarantees that an async-connect handler is not invoked inline, including an immediately +available result. CMP additionally performs the explicit return-Scheduler hop, so success and +failure are observed in the same execution context. + +## Read Contract + +`read_some()` performs at most one socket read and returns the number of bytes placed in the +borrowed span. A successful read may be shorter than the span; the caller loops when its protocol +requires an exact length. + +- An empty span completes successfully with zero without touching the socket. +- For a non-empty span, zero means orderly remote EOF. +- EOF is sticky: later non-empty reads also return zero until the stream is explicitly closed. +- If Asio reports bytes together with EOF or another terminal read error, CMP returns the bytes + first and retains the terminal condition for the next read. Received data is never discarded. +- A retained EOF produces zero; a retained non-EOF error throws `std::system_error`. +- A non-EOF socket error makes the stream logically closed. When it accompanied bytes, the retained + error is still delivered once before later operations reject the closed stream. +- Remote EOF closes only the receive direction. The caller may still write until it closes the + stream or another error makes the socket unusable. + +CMP does not retry a failed application-level read and does not add a read-exactly operation in +this phase. Repeated `read_some()` is the sufficient primitive. + +## Write Contract + +`write_all()` uses Asio's composed asynchronous write operation. It succeeds only after the entire +borrowed span has been consumed by the socket layer. An empty span succeeds without touching the +socket. + +After native write initiation, any cancellation or socket error closes the stream and throws. A +local cancellation throws `OperationCancelled`; any other socket error throws +`std::system_error`. A pre-cancelled operation initiates no write and leaves an otherwise open +stream usable. Because a failed initiated write may already have sent some prefix, callers must not +blindly retry it unless their protocol makes that safe. + +The method returns `void` because a successful call always transfers `buffer.size_bytes()` and a +failed all-or-error operation cannot make the partial count safe to reuse automatically. A separate +`write_some()` surface is unnecessary for v1. + +## Concurrent Operations + +A stream permits one pending read and one pending write at the same time, preserving normal TCP +full-duplex use. It rejects a second read or second write with `std::logic_error` rather than +silently interleaving buffers or inventing an ordering queue. + +Direction admission is selected exactly once when the lazy Task starts and remains held until that +Task publishes its outcome. A failed admission still attempts the requested return-Scheduler hop +before throwing. This makes overlap deterministic and prevents raw-completion timing from changing +whether a second operation is accepted. + +`connect()` cannot overlap with stream operations because no `TcpStream` exists until connection +success. `close()` may race any accepted operation and is governed by the close protocol below. + +## Cancellation + +Each operation accepts an optional `std::stop_token` and owns one Asio cancellation signal for that +operation. + +The protocol is: + +1. validate the resource and acquire context/direction admission; +2. if the token is already stopped, initiate no socket operation; +3. otherwise initiate the Asio operation on the I/O thread; +4. install a stop callback that queues cancellation onto that same I/O thread; +5. let the Asio completion handler select and publish the single outcome; +6. remove the callback, release direction admission, and return through `returnTo`. + +Initiation occurs before the callback can emit its Asio signal. A stop request racing callback +installation is still observed because `std::stop_callback` invokes a newly registered callback +for an already-stopped token. The callback never emits a signal or resumes a coroutine from the +requesting thread; it only asks the context to serialize cancellation. + +Cancellation is cooperative with the operating system. If completion has already won, success or +the socket error remains the result. If cancellation wins: + +- connect throws `OperationCancelled` and destroys its temporary socket; +- a read with no bytes throws `OperationCancelled` and leaves the stream usable; +- a read that delivered bytes returns those bytes and leaves the stream usable; +- an initiated `write_all()` throws `OperationCancelled` and closes the stream, even when Asio + reports that zero bytes were transferred; +- cancellation never resumes the awaiting Task more than once. + +The Asio handler is the only resume source after native initiation. Cancellation and close request +the handler; they do not separately resume the operation. This removes the usual duplicate-resume +race. + +A pre-stopped token still completes asynchronously through a valid return Scheduler. Resource +lifetime errors take precedence when the context or stream could not accept the operation at all. + +## Close Contract + +`close()` is thread-safe, idempotent, non-blocking, and non-throwing. It marks the logical stream +closed immediately, rejects later operations, and queues the native socket close onto its +`IoContext` thread. Calling it on a moved-from stream is a no-op. + +Destroying an owning `TcpStream` performs the same close request. Moving transfers that ownership, +so destruction of the moved-from handle does nothing and the destination becomes responsible for +closing the stream. + +The I/O thread linearizes native close against handlers already ready to run: + +- a completion selected before close retains its value or socket error; +- close selected first cancels pending operations, which complete with `OperationCancelled`; +- every accepted operation still completes exactly once through its requested return Scheduler. + +`is_open()` is a thread-safe snapshot of logical local state. It becomes false as soon as close is +requested or a fatal operation error closes the stream. Remote EOF alone does not make it false +because the write direction may remain usable. A moved-from stream reports false. + +No synchronous close-and-wait or half-close API is added. Structured code awaits its outstanding +operations before discarding their buffers and Scheduler owners. + +## Errors and Completion Affinity + +Public error mapping is intentionally small: + +- `OperationCancelled`: an accepted operation was cancelled by its token, explicit close, or + context shutdown before a result won; +- `std::system_error`: address parsing or an operating-system socket operation failed; +- `std::logic_error`: moved-from/closed stream use, an expired or closing context, same-direction + overlap, or another violated lifetime contract. + +Success, cancellation, system errors, and logic errors are all published only after +`returnTo.schedule()` succeeds. If native initiation cannot be accepted, the helper performs that +return hop from its current thread before reporting the failure. + +If the return Scheduler is expired, closed, or inactive, its scheduling exception propagates from +the current completion thread. CMP cannot provide affinity after the selected completion context +has ceased accepting work. The caller must keep the return Scheduler's owner active until the Task +finishes. + +CMP does not translate transient codes into automatic retries. TCP connect, read, and write retry +policy belongs to the application protocol; retrying a partially completed write can duplicate +data. + +## Context Shutdown + +`IoContext` destruction is graceful. It waits for accepted native handlers to be delivered, but +not for user Tasks after they have been handed to another Scheduler: + +1. close operation admission under the context's synchronization boundary; +2. ensure all initiation and close requests accepted before that point precede shutdown work; +3. close every registered live socket on the I/O thread; +4. let every accepted Asio operation run its completion handler exactly once; +5. release the persistent Asio work guard; +6. drain queued I/O handlers far enough to enqueue each return-Scheduler handoff; +7. join the driver thread before releasing the public context. + +Normal shutdown must not call `asio::io_context::stop()`: Asio documents that `stop()` abandons +unfinished operations and ready handlers. Closing sockets and draining the context preserves CMP's +exactly-once contract. + +An operation racing shutdown is either accepted and cancelled/drained or rejected with +`std::logic_error`; it is never silently lost. A `TcpStream` handle may outlive its `IoContext`, but +it is permanently closed and later operations fail safely. Its private shared state does not keep +the public context or driver thread alive. + +Destroying `IoContext` from its own driver thread cannot join safely and terminates, matching the +existing `ThreadPool` self-destruction boundary. Public continuations are not run there, so normal +CMP use does not encounter this case. + +The context cannot keep an application's return `RunLoop` or `ThreadPool` alive. Structured owners +must preserve the Task, its borrowed buffer, and its return executor until completion. Shutdown can +otherwise surface the return Scheduler's lifetime error as documented above. + +## Minimal Implementation Boundary + +Implementation should be limited to: + +- `mcpp.toml`: add the exact standalone Asio dependency; +- `src/tcp.cppm`: public types and the smallest private context/socket/operation state; +- `src/cmp.cppm`: re-export `:tcp`; +- `tests/tcp_test.cpp`: portable loopback contract and race tests; +- `benchmarks/v1-readiness/src/main.cpp`: replace only the TCP client adapter with `TcpStream`; +- existing README and architecture documents after behavior is verified. + +The loopback test server may import Asio directly and use synchronous operations on its own test +thread. CMP does not need a public server API merely to test a client. The existing readiness +benchmark may retain its blocking loopback server and Phase 6A file path; Phase 6B changes only the +client side that it claims to measure. + +No generic I/O base class, backend interface, socket factory, polymorphic executor, type-erased +handler, public endpoint type, or detached operation is added. The Asio module already supplies the +portable backend abstraction. + +## Validation Contract + +The implementation plan must leave deterministic tests for: + +- compile-time move/copy traits and operation laziness; +- numeric IPv4 connect, write-all, partial read, orderly EOF, and repeated EOF; +- an IPv6 loopback path when the host reports IPv6 loopback availability, without failing a host + that has IPv6 disabled; +- invalid numeric address and refused/failed connection as `std::system_error`; +- an empty read and empty write; +- every success and error resuming on the selected `RunLoop` thread; +- returning through a different `ThreadPool::Scheduler`; +- pre-cancellation without native initiation; +- cancellation of a pending connect or read and continued stream usability where specified; +- repeated cancellation-versus-completion races with exactly one outcome; +- one simultaneous read and write succeeding; +- deterministic rejection of a second pending read and second pending write; +- close before use, idempotent close, close during a pending read, and moved-from behavior; +- a server sending bytes and then closing, with all bytes observed before sticky EOF regardless of + how the platform groups its raw completions; +- context destruction with a pending operation, including exactly-once cancellation and driver + join; +- many concurrent loopback clients without lost or duplicate completions; +- invalid or inactive return Scheduler behavior. + +Tests use loopback addresses, ephemeral ports, latches/barriers, and explicit server protocol +messages. They do not access the public internet, use fixed sleeps as correctness conditions, or +assert platform-specific error numbers. IPv6 is the only capability-gated case; the required IPv4 +suite runs on Linux, macOS, and Windows CI. + +Dev and Release strict builds and the entire existing test suite must remain green. The readiness +benchmark retains hard success, expected-failure, and unexpected-failure counts. Timing is recorded +for comparison with the Phase 6A thread-blocking adapter but is not a CI threshold. + +## Acceptance Criteria + +Phase 6B v1 is complete only when: + +1. a consumer imports only `mcpplibs.cmp` and uses the API shown above; +2. pending sockets consume the one `IoContext` driver rather than one ThreadPool worker each; +3. no public Asio type or header leaks through the CMP module; +4. buffers and socket state remain valid through every accepted completion path; +5. cancellation, close, and shutdown resume each accepted operation exactly once; +6. user code observes all normal outcomes on its explicit return Scheduler; +7. loopback tests pass under Linux, macOS, and Windows CI; +8. the readiness benchmark reports zero unexpected failures; +9. docs distinguish native async TCP from Phase 6A blocking isolation; +10. excluded features remain excluded unless a failing acceptance case requires them. + +## Deliberately Excluded + +- async files, directories, pipes, terminals, and process I/O; +- TCP acceptors/servers and generic socket wrappers; +- DNS and service-name resolution; +- TLS, proxy negotiation, HTTP, framing, and serialization; +- UDP, multicast, Unix-domain sockets, and raw sockets; +- connect/read/write timeouts and deadline composition; +- `read_exactly()`, `read_until()`, `write_some()`, scatter/gather, and zero-copy APIs; +- half-close, keepalive, Nagle controls, bind, native handles, and arbitrary socket options; +- multiple I/O driver threads, strands, work stealing, metrics, and runtime tuning; +- implicit current-executor capture, callbacks, futures, or detached operations; +- automatic reconnect, retry, buffering, backpressure, and protocol queues. + +Each item can receive a separate design when a concrete consumer needs it. None is required to +validate this phase's portable ownership and completion model. + +## References + +- [mcpp-index standalone Asio package](https://github.com/mcpplibs/mcpp-index/blob/main/pkgs/c/chriskohlhoff.asio.lua) +- [Standalone Asio upstream](https://github.com/chriskohlhoff/asio) +- [Asio platform implementation notes](https://think-async.com/Asio/asio-1.38.2/doc/asio/overview/implementation.html) +- [`io_context` work and shutdown semantics](https://think-async.com/Asio/asio-1.38.2/doc/asio/reference/io_context.html) +- [Per-operation cancellation](https://think-async.com/Asio/asio-1.38.2/doc/asio/overview/core/cancellation.html) +- [`basic_stream_socket::async_read_some`](https://think-async.com/Asio/asio-1.38.2/doc/asio/reference/basic_stream_socket/async_read_some.html) +- [`async_write`](https://think-async.com/Asio/asio-1.38.2/doc/asio/reference/async_write/overload1.html) +- [`basic_socket::async_connect`](https://think-async.com/Asio/asio-1.38.2/doc/asio/reference/basic_socket/async_connect.html) +- [Asynchronous operation requirements](https://think-async.com/Asio/asio-1.38.2/doc/asio/reference/asynchronous_operations.html) +- [`basic_socket::close`](https://think-async.com/Asio/asio-1.38.2/doc/asio/reference/basic_socket/close.html) diff --git a/mcpp.toml b/mcpp.toml index 4474dd3..60353f9 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -13,3 +13,6 @@ default = "llvm@22.1.8" [dev-dependencies.compat] gtest = "1.15.2" + +[dependencies.chriskohlhoff] +asio = "1.38.1" diff --git a/src/cmp.cppm b/src/cmp.cppm index 1d182d6..607fa83 100644 --- a/src/cmp.cppm +++ b/src/cmp.cppm @@ -5,6 +5,7 @@ export import :task; export import :run_loop; export import :thread_pool; export import :blocking; +export import :tcp; export import :when_all; export import :task_group; export import :one_shot_event; diff --git a/src/tcp.cppm b/src/tcp.cppm new file mode 100644 index 0000000..5ad3f85 --- /dev/null +++ b/src/tcp.cppm @@ -0,0 +1,130 @@ +export module mcpplibs.cmp:tcp; + +import std; +import asio; +import :cancellation; +import :task; + +namespace mcpplibs::cmp::detail { + +class TcpSocketState; + +class IoContextState final { +private: + using WorkGuard = decltype(asio::make_work_guard( + std::declval())); + + asio::io_context ioContext_ {}; + WorkGuard workGuard_ { asio::make_work_guard(ioContext_) }; + std::mutex admissionMutex_ {}; + bool accepting_ { true }; + // ponytail: v1 线性清理弱引用;实测注册或关闭成本显著后再换索引结构。 + std::vector> sockets_ {}; + + void shutdown_on_driver_() noexcept; + + friend class TcpSocketState; + +public: + template + void post(Handler&& handler) { + const std::lock_guard lock { admissionMutex_ }; + + if (!accepting_) { + throw std::logic_error { "I/O context is stopping" }; + } + + asio::post(ioContext_, std::forward(handler)); + } + + void register_socket_on_driver( + const std::shared_ptr& socket) { + std::erase_if(sockets_, [](const auto& registered) { + return registered.expired(); + }); + sockets_.push_back(socket); + } + + void run_driver() noexcept { + try { + ioContext_.run(); + } catch (...) { + std::terminate(); + } + } + + void begin_shutdown() noexcept { + const std::lock_guard lock { admissionMutex_ }; + accepting_ = false; + asio::post(ioContext_, [this] { + shutdown_on_driver_(); + }); + } +}; + +class TcpSocketState final { +private: + std::weak_ptr context_ {}; + std::optional socket_ {}; + +public: + explicit TcpSocketState( + const std::shared_ptr& context) + : context_ { context }, + socket_ { std::in_place, context->ioContext_ } {} + + void close_on_driver() noexcept { + if (!socket_) { + return; + } + + std::error_code ignored {}; + socket_->close(ignored); + socket_.reset(); + } +}; + +void IoContextState::shutdown_on_driver_() noexcept { + for (auto& registered : sockets_) { + if (const auto socket = registered.lock()) { + socket->close_on_driver(); + } + } + + sockets_.clear(); + workGuard_.reset(); +} + +} // namespace mcpplibs::cmp::detail + +export namespace mcpplibs::cmp { + +class IoContext final { +private: + // 共享状态必须先构造、后析构,driver 才能安全排空并退出。 + std::shared_ptr state_ { + std::make_shared() + }; + std::jthread driver_ { [state = state_] { + state->run_driver(); + } }; + +public: + IoContext() = default; + + IoContext(const IoContext&) = delete; + IoContext& operator=(const IoContext&) = delete; + IoContext(IoContext&&) = delete; + IoContext& operator=(IoContext&&) = delete; + + ~IoContext() { + if (driver_.get_id() == std::this_thread::get_id()) { + std::terminate(); + } + + state_->begin_shutdown(); + driver_.join(); + } +}; + +} // namespace mcpplibs::cmp diff --git a/tests/tcp_test.cpp b/tests/tcp_test.cpp new file mode 100644 index 0000000..635da13 --- /dev/null +++ b/tests/tcp_test.cpp @@ -0,0 +1,23 @@ +#include + +import std; +import asio; +import mcpplibs.cmp; + +namespace { + +using mcpplibs::cmp::IoContext; + +static_assert(std::default_initializable); +static_assert(std::destructible); +static_assert(std::is_final_v); +static_assert(!std::copy_constructible); +static_assert(!std::move_constructible); +static_assert(!std::is_copy_assignable_v); +static_assert(!std::is_move_assignable_v); + +TEST(CmpTcpTest, IoContextStartsAndStopsCleanly) { + IoContext context {}; +} + +} // namespace From 76bb9bd6cee66950f9dbfcae9f74b639c2d02099 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 00:37:38 +0800 Subject: [PATCH 05/17] =?UTF-8?q?=E5=AE=9E=E7=8E=B0=20TCP=20=E5=8E=9F?= =?UTF-8?q?=E7=94=9F=E5=AE=8C=E6=88=90=E6=A1=A5=E6=8E=A5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 43 +++- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 14 +- src/tcp.cppm | 225 ++++++++++++++++++ 3 files changed, 268 insertions(+), 14 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index b5f0ac7..f56a076 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -14,9 +14,9 @@ path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1 已作为本地提交 `701aa8b` 完成,尚未推送。Phase 6B TCP client -v1 已进入实现:Plan 第 1、2 项完成,Asio backend gate 与 `IoContext` 生命周期通过完整 Dev -验证;尚未实现 native-completion bridge 或 `TcpStream` 公共 API。 +Phase 6A blocking offload v1 与 Phase 6B 第 1–2 项分别作为本地提交 `701aa8b`、`816144f` +完成,均尚未推送。Phase 6B Plan 第 3 项私有 native-completion bridge 已实现并通过编译门禁; +尚未实现 `TcpStream` 公共 API 或真实 socket operation。 ## 已完成工作 @@ -55,6 +55,16 @@ v1 已进入实现:Plan 第 1、2 项完成,Asio backend gate 与 `IoContext 线程内自析构会终止而不是 self-join。 - 新增 `tests/tcp_test.cpp`,以编译期断言锁定 `IoContext` 的默认构造及精确 copy/move traits, 并以运行测试验证构造、driver 启动、析构关闭和 join 不挂起。 +- 实现唯一的私有 `NativeOperationAwaiter`:一个共享 operation state 保存 continuation、Asio + cancellation signal、原始 error/count、取消来源、同步 initiation 异常和 stop callback。 +- completion handler 以一参数/两参数重载把 connect 与 read/write 统一为 + `std::error_code + transferred bytes`;native initiation 成功后,只有该 handler 可以恢复 + 私有协程,stop/close/shutdown 只能请求 native completion。 +- stop callback 只持有 operation/context 弱引用,并把 stop-token cancellation 排到 I/O 线程 + 后再记录来源、发出 `cancellation_type::all`;context 已停止接纳时由 shutdown close 路径 + 完成 operation。 +- 加入内部具体模板实例作为 Step 3 编译门禁,真实 connect initiation 在 Step 4 接入后删除; + 没有为私有 bridge 暴露测试 API 或提前加入公共 `TcpStream`。 ## 重要决策 @@ -77,6 +87,8 @@ v1 已进入实现:Plan 第 1、2 项完成,Asio backend gate 与 `IoContext - socket state 只弱引用 context;共享 context state 由公开 `IoContext` 与 driver 持有, socket 不能延长公开 driver 生命周期。v1 registry 使用 O(n) 弱引用扫描,只有实测成本显著 时才替换。 +- native initiation 成功前的同步异常可以直接恢复私有协程;成功后恢复权只属于 Asio handler。 + stop callback 不从请求线程 emit、访问 socket 或恢复 coroutine。 - `TcpStream` 允许一项 pending read 与一项 pending write;同方向重叠直接拒绝,不增加隐式 排队。主动 close、context shutdown 与已发起 write 的取消均有明确关闭语义。 - v1 不含 DNS、server、TLS、UDP、文件 I/O、timeout、socket option 或隐式 executor;这些都 @@ -127,8 +139,12 @@ v1 已进入实现:Plan 第 1、2 项完成,Asio backend gate 与 `IoContext 1/1 测试通过。 - 加入 `IoContext` 后,`mcpp build --profile dev --strict --cache=off` 再次通过; `mcpp test --profile dev --strict --cache=off` 为 10 个二进制、117/117 通过。 -- 本 checkpoint 未运行当前 Phase 6B 的 Release、example 或 benchmark;它们仍属于后续 Plan - gate,既有 Phase 6A Release/example/benchmark 结果不代表 TCP API 已实现。 +- 加入 native bridge 后,`mcpp build --profile dev --strict --cache=off` 与 + `mcpp build --profile release --strict --cache=off` 均通过;Dev 与 Release 全量测试均为 10 个 + 二进制、117/117 通过。 +- 当前 bridge 没有公共 socket operation,因而尚无真实 completion/cancellation runtime test; + 该门禁在 Step 4 的 loopback connect 测试完成。当前未重复执行 example 或 benchmark,既有 + Phase 6A 结果不代表 TCP API 已实现。 ## 已知问题 / 风险 @@ -141,20 +157,21 @@ v1 已进入实现:Plan 第 1、2 项完成,Asio backend gate 与 `IoContext - 三平台兼容性仍需远程 CI 确认。 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 -- Phase 6B 当前只有 backend 与 `IoContext` 生命周期;native-completion bridge、`TcpStream` - connect/read/write、取消竞态和 benchmark 迁移均尚未实现。 +- Phase 6B 当前已有 backend、`IoContext` 生命周期和私有 native-completion bridge;bridge 的 + runtime exactly-once/cancellation 尚待真实 socket 测试,`TcpStream` connect/read/write、 + close 竞态和 benchmark 迁移均尚未实现。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 按 Phase 6B Plan 第 3 项实现唯一的 native-completion bridge,锁定只有 Asio handler 能在 - native initiation 后恢复私有协程,并准备统一的 error/count/cancellation state。 +1. 按 Phase 6B Plan 第 4 项实现 `TcpStream::connect()` 和同步 Asio loopback fixture,删除内部 + bridge 编译 probe,并以真实 connect 锁定 laziness、地址复制、错误、取消和返回线程亲和。 2. 完成本地 Dev/Release、race、example 与五轮 readiness 验证后,再等待远程三平台 CI 所需 的单独授权。 -3. 当前 Phase 6B 代码、manifest、Design、Plan 与 HANDOFF 均未提交;是否创建本地提交、 - Push、PR 或 CI 均需对应明确授权。 +3. Phase 6B 第 1–2 项已本地提交为 `816144f`;当前 bridge、Plan 与 HANDOFF 尚未提交。Push、 + PR 或 CI 均需对应明确授权。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 3 项,只实现私有 native-completion -bridge 及其 exactly-once/cancellation 基础;connect 与 loopback fixture 留到第 4 项。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 4 项,实现 numeric-address +`TcpStream::connect()` 与单一同步 Asio loopback fixture,并用真实 operation 验证 bridge。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index 3bb9705..5d008f0 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — `IoContext` lifecycle gate complete +**Status:** In progress — native-completion bridge gate complete **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -131,6 +131,18 @@ Each helper captures value/error/cancellation, then awaits `returnTo.schedule()` token before publishing the outcome. Use `std::scope_exit` for direction-admission cleanup rather than creating another guard class. A return-Scheduler failure remains the observable exception. +**Completion — 2026-08-30:** Added one private `NativeOperationAwaiter` and shared operation state. +The completion handler overloads normalize connect and byte-transfer signatures to +`std::error_code` plus byte count. The state owns the continuation, cancellation signal, +error/count, cancellation origin, exception, and stop callback; the callback holds only weak +operation/context references and queues signal emission onto the I/O thread. After successful +native initiation, only the Asio handler resumes the private coroutine; a synchronous initiation +exception is the sole pre-initiation direct-resume path. An internal concrete instantiation checks +the otherwise-private template until Step 4 supplies real socket initiations. Strict cache-off Dev +and Release builds passed; both full suites passed 117/117 across 10 binaries. Runtime +completion/cancellation races remain intentionally gated on Step 4's real loopback connect rather +than a public test hook. + ## 4. Implement connect and the loopback fixture Add `TcpStream::connect()` with the Design signature. On the I/O thread: diff --git a/src/tcp.cppm b/src/tcp.cppm index 5ad3f85..7aa889a 100644 --- a/src/tcp.cppm +++ b/src/tcp.cppm @@ -95,6 +95,231 @@ void IoContextState::shutdown_on_driver_() noexcept { workGuard_.reset(); } +enum class NativeCancellationOrigin { + none, + stop_token, + stream_close, + context_shutdown +}; + +struct NativeOperationState final { + struct Result final { + std::error_code error_ {}; + std::size_t transferred_ {}; + NativeCancellationOrigin cancellation_ { + NativeCancellationOrigin::none + }; + }; + + struct StopRequest final { + std::weak_ptr operation_ {}; + std::weak_ptr context_ {}; + + void operator()() const noexcept; + }; + + struct CompletionHandler final { + std::shared_ptr operation_ {}; + + void operator()(std::error_code error) const noexcept; + + void operator()( + std::error_code error, + std::size_t transferred) const noexcept; + }; + + using StopCallback = std::stop_callback; + + std::coroutine_handle<> continuation_ {}; + asio::cancellation_signal cancellationSignal_ {}; + std::error_code error_ {}; + std::size_t transferred_ {}; + NativeCancellationOrigin cancellation_ { + NativeCancellationOrigin::none + }; + std::exception_ptr exception_ {}; + bool completed_ { false }; + // 最先释放,避免停止回调继续访问即将销毁的 operation state。 + std::optional stopCallback_ {}; + + void complete_on_driver( + std::error_code error, + std::size_t transferred) noexcept { + if (std::exchange(completed_, true)) { + std::terminate(); + } + + error_ = error; + transferred_ = transferred; + resume_on_driver_(); + } + + void fail_initiation_on_driver(std::exception_ptr exception) noexcept { + if (!exception || std::exchange(completed_, true)) { + std::terminate(); + } + + exception_ = std::move(exception); + resume_on_driver_(); + } + + void request_cancellation_on_driver( + NativeCancellationOrigin origin) noexcept { + if (completed_ || cancellation_ != NativeCancellationOrigin::none) { + return; + } + + cancellation_ = origin; + cancellationSignal_.emit(asio::cancellation_type::all); + } + + [[nodiscard]] Result take_result() { + if (!completed_) { + std::terminate(); + } + + stopCallback_.reset(); + + if (exception_) { + std::rethrow_exception(exception_); + } + + return Result { + error_, + transferred_, + cancellation_ + }; + } + +private: + void resume_on_driver_() noexcept { + const auto continuation = std::exchange(continuation_, {}); + + if (!continuation || continuation.done()) { + std::terminate(); + } + + continuation.resume(); + } +}; + +void NativeOperationState::StopRequest::operator()() const noexcept { + const auto operation = operation_.lock(); + const auto context = context_.lock(); + + if (!operation || !context) { + return; + } + + try { + context->post([operation] noexcept { + operation->request_cancellation_on_driver( + NativeCancellationOrigin::stop_token); + }); + } catch (const std::logic_error&) { + // Context shutdown 会关闭 socket,并由 native handler 完成该 operation。 + } catch (...) { + std::terminate(); + } +} + +void NativeOperationState::CompletionHandler::operator()( + std::error_code error) const noexcept { + operation_->complete_on_driver(error, 0); +} + +void NativeOperationState::CompletionHandler::operator()( + std::error_code error, + std::size_t transferred) const noexcept { + operation_->complete_on_driver(error, transferred); +} + +template +class NativeOperationAwaiter final { +private: + std::shared_ptr context_ {}; + std::stop_token stopToken_ {}; + Initiation initiation_; + std::shared_ptr operation_ { + std::make_shared() + }; + +public: + NativeOperationAwaiter( + std::shared_ptr context, + std::stop_token stopToken, + Initiation initiation) + : context_ { std::move(context) }, + stopToken_ { std::move(stopToken) }, + initiation_ { std::move(initiation) } {} + + [[nodiscard]] constexpr bool await_ready() const noexcept { + return false; + } + + void await_suspend(std::coroutine_handle<> continuation) { + auto context = context_; + auto operation = operation_; + auto stopToken = stopToken_; + auto initiation = std::move(initiation_); + + operation->continuation_ = continuation; + + // 发布后 continuation 可并发恢复,不再访问 awaiter 成员。 + context->post([ + context = std::move(context), + operation = std::move(operation), + stopToken = std::move(stopToken), + initiation = std::move(initiation) + ]() mutable noexcept { + try { + auto completion = asio::bind_cancellation_slot( + operation->cancellationSignal_.slot(), + NativeOperationState::CompletionHandler { operation }); + std::invoke( + std::move(initiation), + std::move(completion)); + } catch (...) { + operation->fail_initiation_on_driver( + std::current_exception()); + return; + } + + // initiation 成功后,只有 native handler 可以恢复 continuation。 + operation->stopCallback_.emplace( + std::move(stopToken), + NativeOperationState::StopRequest { + operation, + context + }); + }); + } + + [[nodiscard]] NativeOperationState::Result await_resume() { + return operation_->take_result(); + } +}; + +struct NativeBridgeCompileProbe final { + template + void operator()(Completion&&) const noexcept {} +}; + +static_assert(std::invocable< + NativeOperationState::CompletionHandler, + std::error_code>); +static_assert(std::invocable< + NativeOperationState::CompletionHandler, + std::error_code, + std::size_t>); +static_assert(std::is_nothrow_constructible_v< + NativeOperationState::StopCallback, + std::stop_token, + NativeOperationState::StopRequest>); + +// Step 4 接入真实 socket initiation 后删除此编译门禁。 +template class NativeOperationAwaiter; + } // namespace mcpplibs::cmp::detail export namespace mcpplibs::cmp { From 4f057676cc79496da6b8afd005081097c21af948 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 00:51:59 +0800 Subject: [PATCH 06/17] =?UTF-8?q?=E5=AE=9E=E7=8E=B0=20TCP=20=E5=AE=A2?= =?UTF-8?q?=E6=88=B7=E7=AB=AF=E8=BF=9E=E6=8E=A5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 43 +- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 21 +- src/tcp.cppm | 214 +++++++++- tests/tcp_test.cpp | 380 ++++++++++++++++++ 4 files changed, 633 insertions(+), 25 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index f56a076..7f7cd48 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -14,9 +14,9 @@ path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1 与 Phase 6B 第 1–2 项分别作为本地提交 `701aa8b`、`816144f` -完成,均尚未推送。Phase 6B Plan 第 3 项私有 native-completion bridge 已实现并通过编译门禁; -尚未实现 `TcpStream` 公共 API 或真实 socket operation。 +Phase 6A blocking offload v1、Phase 6B 第 1–2 项与第 3 项分别作为本地提交 `701aa8b`、 +`816144f`、`76bb9bd` 完成,均尚未推送。Phase 6B Plan 第 4 项 numeric-address +`TcpStream::connect()` 已实现并通过本地门禁;read/write 与公开 close API 尚未实现。 ## 已完成工作 @@ -63,8 +63,16 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–2 项分别作为本地提交 - stop callback 只持有 operation/context 弱引用,并把 stop-token cancellation 排到 I/O 线程 后再记录来源、发出 `cancellation_type::all`;context 已停止接纳时由 shutdown close 路径 完成 operation。 -- 加入内部具体模板实例作为 Step 3 编译门禁,真实 connect initiation 在 Step 4 接入后删除; - 没有为私有 bridge 暴露测试 API 或提前加入公共 `TcpStream`。 +- Step 3 曾加入内部具体模板实例作为 bridge 编译门禁,Step 4 接入真实 connect initiation 后已 + 删除;没有为私有 bridge 暴露测试 API。 +- 新增不可默认构造、不可复制、可 `noexcept` 移动构造且不可移动赋值的 `TcpStream`;公开 + `connect()` 是非协程包装器,在返回 lazy Task 前复制地址、Scheduler、token 与 context 弱句柄。 +- `connect()` 被 context 接纳后才在 I/O 线程检查预取消、解析 numeric IPv4/IPv6、创建并注册 + socket、发起 `async_connect()`;预取消不解析地址或打开 socket,系统错误保留原 error code。 +- native operation state 新增私有生命周期锚点,保证临时 connect socket 存活到 handler;成功 + 后先经显式 return Scheduler 再发布 move-only stream,return Scheduler 失败由内部 RAII 关闭。 +- `tests/tcp_test.cpp` 扩展为单一同步 Asio loopback fixture,覆盖 laziness/临时地址所有权、IPv4、 + 可用时 IPv6、拒绝连接、预取消、RunLoop/ThreadPool 亲和及 inactive/expired Scheduler。 ## 重要决策 @@ -142,9 +150,10 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–2 项分别作为本地提交 - 加入 native bridge 后,`mcpp build --profile dev --strict --cache=off` 与 `mcpp build --profile release --strict --cache=off` 均通过;Dev 与 Release 全量测试均为 10 个 二进制、117/117 通过。 -- 当前 bridge 没有公共 socket operation,因而尚无真实 completion/cancellation runtime test; - 该门禁在 Step 4 的 loopback connect 测试完成。当前未重复执行 example 或 benchmark,既有 - Phase 6A 结果不代表 TCP API 已实现。 +- connect 定向 `tcp_test` 在 Dev 与 Release 均为 8/8 通过;Release binary 重复执行 100 轮, + 800/800 用例通过。 +- 加入 connect 后,Dev 与 Release 的 strict cache-off 全量测试均为 10 个二进制、124/124 通过。 + 当前未重复执行 example 或 benchmark;它们保留到 Phase 6B 完整 API 与 readiness 迁移门禁。 ## 已知问题 / 风险 @@ -157,21 +166,21 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–2 项分别作为本地提交 - 三平台兼容性仍需远程 CI 确认。 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 -- Phase 6B 当前已有 backend、`IoContext` 生命周期和私有 native-completion bridge;bridge 的 - runtime exactly-once/cancellation 尚待真实 socket 测试,`TcpStream` connect/read/write、 - close 竞态和 benchmark 迁移均尚未实现。 +- Phase 6B 当前已有 backend、`IoContext` 生命周期、native-completion bridge 与 `connect()`; + pending connect 的主动取消竞态、read/write、公开 close、shutdown 竞态和 benchmark 迁移仍待 + 后续步骤完成。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 按 Phase 6B Plan 第 4 项实现 `TcpStream::connect()` 和同步 Asio loopback fixture,删除内部 - bridge 编译 probe,并以真实 connect 锁定 laziness、地址复制、错误、取消和返回线程亲和。 +1. 按 Phase 6B Plan 第 5 项实现 `read_some()`、`write_all()`、EOF/partial transfer 与单读单写 + admission,并扩展同一个 loopback fixture 的确定性协议门禁。 2. 完成本地 Dev/Release、race、example 与五轮 readiness 验证后,再等待远程三平台 CI 所需 的单独授权。 -3. Phase 6B 第 1–2 项已本地提交为 `816144f`;当前 bridge、Plan 与 HANDOFF 尚未提交。Push、 - PR 或 CI 均需对应明确授权。 +3. Phase 6B 第 3 项已本地提交为 `76bb9bd`;当前 connect、测试、Plan 与 HANDOFF 尚未提交。 + Push、PR 或 CI 均需对应明确授权。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 4 项,实现 numeric-address -`TcpStream::connect()` 与单一同步 Asio loopback fixture,并用真实 operation 验证 bridge。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 5 项,在现有 socket state 与 loopback +fixture 上实现 read/write、EOF、partial transfer 和同方向 overlap 门禁,不提前扩展范围。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index 5d008f0..bb0aaea 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — native-completion bridge gate complete +**Status:** In progress — connect gate complete **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -177,6 +177,25 @@ Lock connect behavior with focused tests for: Use timeouts only as deadlock guards. Do not use elapsed time or platform-specific error numbers as pass conditions. +**Self-review — 2026-08-30:** The public wrapper must copy the address and weak context handle +before returning the lazy Task. A pre-stopped token is checked only after the context accepts the +I/O-thread initiation, preserving resource-error precedence while still skipping parse/open. The +bridge operation state gains one private lifetime anchor so the temporary socket survives through +its native handler. Invalid return-Scheduler tests connect successfully first and exercise the +post-connect cleanup path. Step 4 adds only this internal destruction path; +public `close()` and `is_open()` remain Step 6 work. + +**Completion — 2026-08-30:** Added move-only `TcpStream` and its lazy numeric-address `connect()`. +The wrapper owns the address and a weak context handle before returning; accepted initiation checks +pre-cancellation, parses, creates/registers the socket, and calls `async_connect()` only on the I/O +thread. The bridge now retains an operation-specific lifetime anchor. Success, cancellation, parse +errors, connect errors, and context admission failures all attempt the explicit return-Scheduler +hop before publishing. Internal destruction closes an undelivered connected socket without +exposing Step 6 APIs early. The single synchronous loopback fixture covers IPv4, capability-gated +IPv6, laziness/address ownership, refusal, pre-cancellation, RunLoop/ThreadPool affinity, and +invalid return schedulers. Focused Dev and Release tests passed 8/8; the Release binary passed 100 +repetitions, and both full suites passed 124/124 across 10 binaries. + ## 5. Implement read, write, EOF, and overlap Add `read_some()` and `write_all()` against the same private socket state. diff --git a/src/tcp.cppm b/src/tcp.cppm index 7aa889a..50d6601 100644 --- a/src/tcp.cppm +++ b/src/tcp.cppm @@ -66,6 +66,7 @@ class TcpSocketState final { private: std::weak_ptr context_ {}; std::optional socket_ {}; + std::atomic logicallyOpen_ { false }; public: explicit TcpSocketState( @@ -73,7 +74,25 @@ public: : context_ { context }, socket_ { std::in_place, context->ioContext_ } {} + [[nodiscard]] asio::ip::tcp::socket& socket_on_driver() noexcept { + if (!socket_) { + std::terminate(); + } + + return *socket_; + } + + void mark_open_on_driver() noexcept { + if (!socket_ || logicallyOpen_.exchange( + true, + std::memory_order_release)) { + std::terminate(); + } + } + void close_on_driver() noexcept { + logicallyOpen_.store(false, std::memory_order_release); + if (!socket_) { return; } @@ -82,6 +101,29 @@ public: socket_->close(ignored); socket_.reset(); } + + void request_close( + const std::shared_ptr& self) noexcept { + if (!logicallyOpen_.exchange(false, std::memory_order_acq_rel)) { + return; + } + + const auto context = context_.lock(); + + if (!context) { + return; + } + + try { + context->post([self] noexcept { + self->close_on_driver(); + }); + } catch (const std::logic_error&) { + // Shutdown 已接管 registry 中的 socket。 + } catch (...) { + std::terminate(); + } + } }; void IoContextState::shutdown_on_driver_() noexcept { @@ -139,6 +181,8 @@ struct NativeOperationState final { }; std::exception_ptr exception_ {}; bool completed_ { false }; + // 保证 native handler 返回前,connect/read/write 使用的资源仍然有效。 + std::shared_ptr lifetime_ {}; // 最先释放,避免停止回调继续访问即将销毁的 operation state。 std::optional stopCallback_ {}; @@ -248,10 +292,13 @@ public: NativeOperationAwaiter( std::shared_ptr context, std::stop_token stopToken, + std::shared_ptr lifetime, Initiation initiation) : context_ { std::move(context) }, stopToken_ { std::move(stopToken) }, - initiation_ { std::move(initiation) } {} + initiation_ { std::move(initiation) } { + operation_->lifetime_ = std::move(lifetime); + } [[nodiscard]] constexpr bool await_ready() const noexcept { return false; @@ -300,9 +347,8 @@ public: } }; -struct NativeBridgeCompileProbe final { - template - void operator()(Completion&&) const noexcept {} +struct ConnectOperationState final { + std::shared_ptr socket_ {}; }; static_assert(std::invocable< @@ -317,13 +363,12 @@ static_assert(std::is_nothrow_constructible_v< std::stop_token, NativeOperationState::StopRequest>); -// Step 4 接入真实 socket initiation 后删除此编译门禁。 -template class NativeOperationAwaiter; - } // namespace mcpplibs::cmp::detail export namespace mcpplibs::cmp { +class TcpStream; + class IoContext final { private: // 共享状态必须先构造、后析构,driver 才能安全排空并退出。 @@ -334,6 +379,8 @@ private: state->run_driver(); } }; + friend class TcpStream; + public: IoContext() = default; @@ -352,4 +399,157 @@ public: } }; +class TcpStream final { +private: + std::shared_ptr state_ {}; + + explicit TcpStream( + std::shared_ptr state) noexcept + : state_ { std::move(state) } {} + + template + [[nodiscard]] static Task connect_impl_( + std::weak_ptr context, + ReturnScheduler returnTo, + std::string numericAddress, + std::uint16_t port, + std::stop_token stopToken); + +public: + TcpStream() = delete; + TcpStream(const TcpStream&) = delete; + TcpStream& operator=(const TcpStream&) = delete; + + TcpStream(TcpStream&& other) noexcept + : state_ { std::exchange(other.state_, {}) } {} + + TcpStream& operator=(TcpStream&&) = delete; + + ~TcpStream() { + if (state_) { + state_->request_close(state_); + } + } + + template + requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } + ) + [[nodiscard]] static Task connect( + IoContext& context, + ReturnScheduler returnTo, + std::string_view numericAddress, + std::uint16_t port, + std::stop_token stopToken = {}); +}; + +template +Task TcpStream::connect_impl_( + std::weak_ptr weakContext, + ReturnScheduler returnTo, + std::string numericAddress, + std::uint16_t port, + std::stop_token stopToken) { + auto connectState = std::make_shared(); + std::optional stream {}; + std::exception_ptr exception {}; + + try { + const auto context = weakContext.lock(); + + if (!context) { + throw std::logic_error { "I/O context no longer exists" }; + } + + const auto result = co_await detail::NativeOperationAwaiter { + context, + stopToken, + connectState, + [ + context, + connectState, + numericAddress = std::move(numericAddress), + port, + stopToken + ](auto completion) mutable { + if (stopToken.stop_requested()) { + throw OperationCancelled {}; + } + + std::error_code error {}; + const auto address = asio::ip::make_address( + numericAddress, + error); + + if (error) { + throw std::system_error { error }; + } + + auto socket = std::make_shared( + context); + connectState->socket_ = socket; + context->register_socket_on_driver(socket); + socket->socket_on_driver().async_connect( + asio::ip::tcp::endpoint { address, port }, + std::move(completion)); + } + }; + + if (result.error_) { + if (result.cancellation_ != + detail::NativeCancellationOrigin::none) { + throw OperationCancelled {}; + } + + throw std::system_error { result.error_ }; + } + + if (!connectState->socket_) { + std::terminate(); + } + + connectState->socket_->mark_open_on_driver(); + stream.emplace(TcpStream { connectState->socket_ }); + } catch (...) { + if (connectState->socket_) { + connectState->socket_->close_on_driver(); + } + + exception = std::current_exception(); + } + + // 返回路径不接受取消,保证成功与错误都回到调用者选择的 Scheduler。 + co_await returnTo.schedule(); + + if (exception) { + std::rethrow_exception(exception); + } + + co_return std::move(*stream); +} + +template +requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } +) +Task TcpStream::connect( + IoContext& context, + ReturnScheduler returnTo, + std::string_view numericAddress, + std::uint16_t port, + std::stop_token stopToken) { + return connect_impl_( + context.state_, + std::move(returnTo), + std::string { numericAddress }, + port, + std::move(stopToken)); +} + } // namespace mcpplibs::cmp diff --git a/tests/tcp_test.cpp b/tests/tcp_test.cpp index 635da13..ce214fc 100644 --- a/tests/tcp_test.cpp +++ b/tests/tcp_test.cpp @@ -7,6 +7,13 @@ import mcpplibs.cmp; namespace { using mcpplibs::cmp::IoContext; +using mcpplibs::cmp::OperationCancelled; +using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::Task; +using mcpplibs::cmp::TcpStream; +using mcpplibs::cmp::ThreadPool; + +using namespace std::chrono_literals; static_assert(std::default_initializable); static_assert(std::destructible); @@ -16,8 +23,381 @@ static_assert(!std::move_constructible); static_assert(!std::is_copy_assignable_v); static_assert(!std::is_move_assignable_v); +static_assert(!std::default_initializable); +static_assert(std::destructible); +static_assert(std::is_final_v); +static_assert(!std::copy_constructible); +static_assert(std::move_constructible); +static_assert(std::is_nothrow_move_constructible_v); +static_assert(!std::is_copy_assignable_v); +static_assert(!std::is_move_assignable_v); + +class LoopbackServer final { +private: + asio::io_context ioContext_ {}; + asio::ip::tcp::acceptor acceptor_ { ioContext_ }; + asio::ip::tcp::endpoint endpoint_ {}; + std::error_code startError_ {}; + std::binary_semaphore ready_ { 0 }; + std::binary_semaphore acceptedSignal_ { 0 }; + std::latch release_ { 1 }; + std::atomic accepted_ { false }; + std::jthread worker_ {}; + + void run_() noexcept { + ready_.release(); + + asio::ip::tcp::socket socket { ioContext_ }; + std::error_code error {}; + acceptor_.accept(socket, error); + + if (error) { + return; + } + + accepted_.store(true, std::memory_order_release); + acceptedSignal_.release(); + release_.wait(); + } + +public: + explicit LoopbackServer(const asio::ip::address& address) { + const asio::ip::tcp::endpoint requested { address, 0 }; + acceptor_.open(requested.protocol(), startError_); + + if (startError_) { + return; + } + + acceptor_.bind(requested, startError_); + + if (startError_) { + return; + } + + acceptor_.listen( + asio::socket_base::max_listen_connections, + startError_); + + if (startError_) { + return; + } + + endpoint_ = acceptor_.local_endpoint(startError_); + + if (startError_) { + return; + } + + worker_ = std::jthread { [this] { + run_(); + } }; + } + + LoopbackServer(const LoopbackServer&) = delete; + LoopbackServer& operator=(const LoopbackServer&) = delete; + LoopbackServer(LoopbackServer&&) = delete; + LoopbackServer& operator=(LoopbackServer&&) = delete; + + ~LoopbackServer() { + if (!worker_.joinable()) { + return; + } + + release_.count_down(); + + if (!accepted_.load(std::memory_order_acquire)) { + asio::io_context wakeContext {}; + asio::ip::tcp::socket wakeSocket { wakeContext }; + std::error_code ignored {}; + wakeSocket.connect(endpoint_, ignored); + } + + worker_.join(); + } + + [[nodiscard]] bool available() const noexcept { + return !startError_; + } + + [[nodiscard]] const std::error_code& start_error() const noexcept { + return startError_; + } + + [[nodiscard]] std::uint16_t port() const noexcept { + return endpoint_.port(); + } + + [[nodiscard]] bool wait_until_ready() { + return ready_.try_acquire_for(2s); + } + + [[nodiscard]] bool accepted() const noexcept { + return accepted_.load(std::memory_order_acquire); + } + + [[nodiscard]] bool wait_until_accepted() { + if (accepted()) { + return true; + } + + return acceptedSignal_.try_acquire_for(2s); + } +}; + +class NonListeningEndpoint final { +private: + asio::io_context ioContext_ {}; + asio::ip::tcp::acceptor acceptor_ { ioContext_ }; + std::uint16_t port_ {}; + +public: + NonListeningEndpoint() { + std::error_code error {}; + acceptor_.open(asio::ip::tcp::v4(), error); + + if (!error) { + acceptor_.bind( + asio::ip::tcp::endpoint { + asio::ip::address_v4::loopback(), + 0 + }, + error); + } + + if (!error) { + port_ = acceptor_.local_endpoint(error).port(); + } + + if (error) { + throw std::system_error { error }; + } + } + + [[nodiscard]] std::uint16_t port() const noexcept { + return port_; + } +}; + +enum class ConnectOutcome { + succeeded, + cancelled, + system_error +}; + +struct ConnectObservation final { + ConnectOutcome outcome_ {}; + std::thread::id resumedThread_ {}; +}; + +template +Task connect_and_observe( + IoContext& context, + ReturnScheduler returnTo, + std::string_view address, + std::uint16_t port) { + auto stream = co_await TcpStream::connect( + context, + std::move(returnTo), + address, + port); + co_return std::this_thread::get_id(); +} + +template +Task observe_connect_outcome( + IoContext& context, + ReturnScheduler returnTo, + std::string_view address, + std::uint16_t port, + std::stop_token stopToken = {}) { + try { + auto stream = co_await TcpStream::connect( + context, + std::move(returnTo), + address, + port, + std::move(stopToken)); + co_return ConnectObservation { + ConnectOutcome::succeeded, + std::this_thread::get_id() + }; + } catch (const OperationCancelled&) { + co_return ConnectObservation { + ConnectOutcome::cancelled, + std::this_thread::get_id() + }; + } catch (const std::system_error&) { + co_return ConnectObservation { + ConnectOutcome::system_error, + std::this_thread::get_id() + }; + } +} + +RunLoop::Scheduler make_expired_scheduler() { + RunLoop loop {}; + return loop.get_scheduler(); +} + TEST(CmpTcpTest, IoContextStartsAndStopsCleanly) { IoContext context {}; } +TEST(CmpTcpTest, ConnectIsLazyAndOwnsTemporaryAddress) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto task = TcpStream::connect( + context, + loop.get_scheduler(), + std::string { "127.0.0.1" }, + server.port()); + + EXPECT_FALSE(server.accepted()); + auto stream = loop.run(std::move(task)); + EXPECT_TRUE(server.wait_until_accepted()); +} + +TEST(CmpTcpTest, ConnectsIpv4AndReturnsToRunLoop) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + const auto resumedThread = loop.run(connect_and_observe( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + + EXPECT_EQ(resumedThread, callerThread); + EXPECT_TRUE(server.wait_until_accepted()); +} + +TEST(CmpTcpTest, ConnectsIpv6WhenLoopbackIsAvailable) { + LoopbackServer server { asio::ip::address_v6::loopback() }; + + if (!server.available()) { + GTEST_SKIP() << "IPv6 loopback unavailable: " + << server.start_error().message(); + } + + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + const auto resumedThread = loop.run(connect_and_observe( + context, + loop.get_scheduler(), + "::1", + server.port())); + + EXPECT_EQ(resumedThread, std::this_thread::get_id()); + EXPECT_TRUE(server.wait_until_accepted()); +} + +TEST(CmpTcpTest, ReportsAddressAndConnectErrorsOnReturnScheduler) { + NonListeningEndpoint endpoint {}; + IoContext context {}; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + + const auto invalidAddress = loop.run(observe_connect_outcome( + context, + loop.get_scheduler(), + "not-an-address", + endpoint.port())); + EXPECT_EQ(invalidAddress.outcome_, ConnectOutcome::system_error); + EXPECT_EQ(invalidAddress.resumedThread_, callerThread); + + const auto refused = loop.run(observe_connect_outcome( + context, + loop.get_scheduler(), + "127.0.0.1", + endpoint.port())); + EXPECT_EQ(refused.outcome_, ConnectOutcome::system_error); + EXPECT_EQ(refused.resumedThread_, callerThread); +} + +TEST(CmpTcpTest, PreCancellationSkipsInvalidAddressParsing) { + IoContext context {}; + RunLoop loop {}; + std::stop_source stopSource {}; + stopSource.request_stop(); + + const auto observation = loop.run(observe_connect_outcome( + context, + loop.get_scheduler(), + "not-an-address", + 0, + stopSource.get_token())); + + EXPECT_EQ(observation.outcome_, ConnectOutcome::cancelled); + EXPECT_EQ(observation.resumedThread_, std::this_thread::get_id()); +} + +TEST(CmpTcpTest, CanReturnThroughThreadPoolScheduler) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + ThreadPool returnWorkers { 1 }; + RunLoop loop {}; + const auto callerThread = std::this_thread::get_id(); + const auto resumedThread = loop.run(connect_and_observe( + context, + returnWorkers.get_scheduler(), + "127.0.0.1", + server.port())); + + EXPECT_NE(resumedThread, callerThread); + EXPECT_TRUE(server.wait_until_accepted()); +} + +TEST(CmpTcpTest, RejectsInactiveAndExpiredReturnSchedulers) { + { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop driver {}; + RunLoop inactive {}; + + EXPECT_THROW( + driver.run(TcpStream::connect( + context, + inactive.get_scheduler(), + "127.0.0.1", + server.port())), + std::logic_error); + EXPECT_TRUE(server.wait_until_accepted()); + } + + { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop driver {}; + + EXPECT_THROW( + driver.run(TcpStream::connect( + context, + make_expired_scheduler(), + "127.0.0.1", + server.port())), + std::logic_error); + EXPECT_TRUE(server.wait_until_accepted()); + } +} + } // namespace From a73c113a350628f929e2befd78a643155d11b2d2 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 01:09:50 +0800 Subject: [PATCH 07/17] =?UTF-8?q?=E5=AE=9E=E7=8E=B0=20TCP=20=E5=BC=82?= =?UTF-8?q?=E6=AD=A5=E8=AF=BB=E5=86=99?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 40 +- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 27 +- src/tcp.cppm | 366 +++++++++++- tests/tcp_test.cpp | 547 +++++++++++++++++- 4 files changed, 964 insertions(+), 16 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 7f7cd48..a940e74 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -14,9 +14,9 @@ path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1、Phase 6B 第 1–2 项与第 3 项分别作为本地提交 `701aa8b`、 -`816144f`、`76bb9bd` 完成,均尚未推送。Phase 6B Plan 第 4 项 numeric-address -`TcpStream::connect()` 已实现并通过本地门禁;read/write 与公开 close API 尚未实现。 +Phase 6A blocking offload v1、Phase 6B 第 1–3 项与第 4 项分别作为本地提交 `701aa8b`、 +`816144f`、`76bb9bd`、`4f05767` 完成,均尚未推送。Phase 6B Plan 第 5 项 read/write、EOF 与 +overlap 已实现并通过本地门禁;公开 close/cancellation/shutdown 竞态尚未实现。 ## 已完成工作 @@ -73,6 +73,18 @@ Phase 6A blocking offload v1、Phase 6B 第 1–2 项与第 3 项分别作为本 后先经显式 return Scheduler 再发布 move-only stream,return Scheduler 失败由内部 RAII 关闭。 - `tests/tcp_test.cpp` 扩展为单一同步 Asio loopback fixture,覆盖 laziness/临时地址所有权、IPv4、 可用时 IPv6、拒绝连接、预取消、RunLoop/ThreadPool 亲和及 inactive/expired Scheduler。 +- 新增 lazy `read_some()` 与 `write_all()` 非协程包装器;buffer span、Scheduler、token 与 socket + state 均在调用时按值捕获,实际 buffer 仍按 API 契约借用到 Task 完成。 +- socket state 使用每方向一个 atomic admission flag,允许一读一写并行并拒绝同方向重叠;flag + 保持到 return-Scheduler 跳转结束,跳转成功或失败均释放。 +- read 实现 empty、partial、bytes-before-error、一次 retained fatal error 与 sticky EOF;远端 EOF + 不关闭写方向,后续整条流关闭又不会被旧 sticky EOF 掩盖。 +- write 使用 Asio `async_write()` 完成全 buffer;已发起 write 的取消或错误关闭 stream,成功计数 + 不等于 buffer 大小时视为内部不变量破坏。 +- 当前 libc++ `std` 模块不提供 C++23 `std::scope_exit`;按实测改为在 return-Scheduler 成功与 + 异常两条路径显式释放 admission,没有新增 guard abstraction 或依赖。 +- 同一 loopback fixture 增加小型同步 protocol callback、partial/EOF/duplex gate 与 + `reuse_address`,没有增加 public server 或固定 sleep。 ## 重要决策 @@ -154,6 +166,13 @@ Phase 6A blocking offload v1、Phase 6B 第 1–2 项与第 3 项分别作为本 800/800 用例通过。 - 加入 connect 后,Dev 与 Release 的 strict cache-off 全量测试均为 10 个二进制、124/124 通过。 当前未重复执行 example 或 benchmark;它们保留到 Phase 6B 完整 API 与 readiness 迁移门禁。 +- read/write 定向 `tcp_test` 在 Dev 与 Release 均为 15/15 通过;Release 整套重复 10 轮为 + 150/150,通过两个 overlap test 重复 50 轮为 100/100。 +- 加入 read/write 后,Dev 与 Release 的 strict cache-off 全量测试均为 10 个二进制、131/131 + 通过。 +- 曾尝试 Release 整套重复 100 轮;前 24 轮 360/360 通过,第 25 轮因本机临时端口范围仅 + `60700–61000` 而出现 `Address already in use`。fixture 随后增加 `reuse_address`,上述 10 轮 + 整套与 50 轮 overlap 复验通过;该次资源耗尽不计为功能通过,也未隐藏。 ## 已知问题 / 风险 @@ -166,21 +185,20 @@ Phase 6A blocking offload v1、Phase 6B 第 1–2 项与第 3 项分别作为本 - 三平台兼容性仍需远程 CI 确认。 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 -- Phase 6B 当前已有 backend、`IoContext` 生命周期、native-completion bridge 与 `connect()`; - pending connect 的主动取消竞态、read/write、公开 close、shutdown 竞态和 benchmark 迁移仍待 - 后续步骤完成。 +- Phase 6B 当前已有 backend、`IoContext`、native bridge、connect/read/write 与 EOF/overlap; + pending operation 主动取消、公开 close/is_open、shutdown 竞态和 benchmark 迁移仍待后续步骤。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 按 Phase 6B Plan 第 5 项实现 `read_some()`、`write_all()`、EOF/partial transfer 与单读单写 - admission,并扩展同一个 loopback fixture 的确定性协议门禁。 +1. 按 Phase 6B Plan 第 6 项实现公开 `close()`/`is_open()`,补齐 stop/close/context shutdown + cancellation provenance 与 exactly-once race tests。 2. 完成本地 Dev/Release、race、example 与五轮 readiness 验证后,再等待远程三平台 CI 所需 的单独授权。 -3. Phase 6B 第 3 项已本地提交为 `76bb9bd`;当前 connect、测试、Plan 与 HANDOFF 尚未提交。 +3. Phase 6B 第 4 项已本地提交为 `4f05767`;当前 read/write、测试、Plan 与 HANDOFF 尚未提交。 Push、PR 或 CI 均需对应明确授权。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 5 项,在现有 socket state 与 loopback -fixture 上实现 read/write、EOF、partial transfer 和同方向 overlap 门禁,不提前扩展范围。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 6 项,在现有 state/bridge 上补齐公开 +close/is_open、取消来源与 stop/close/shutdown exactly-once 竞态,不引入 timeout 或队列。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index bb0aaea..83f0e60 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — connect gate complete +**Status:** In progress — read/write gate complete **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -228,6 +228,31 @@ with all bytes observed before repeated EOF, simultaneous read/write, a rejected rejected second write. To keep the second-write check deterministic, hold the first operation at a gated return Scheduler until the overlap result is observed. +**Self-review — 2026-08-30:** Read terminal state is synchronized separately from the atomic +logical-open flag: an error delivered with bytes is retained and observed once before later reads +reject the closed stream, while EOF remains sticky without closing the write direction. A local +cancellation that arrives with bytes returns those bytes and retains no cancellation error. +Read/write admission uses one atomic flag per direction and is held through the return-Scheduler +hop. The configured LLVM 22 libc++ `std` module does not provide C++23 `std::scope_exit`, so the +helper releases explicitly on both schedule success and exception without adding a guard type. +Context/resource validation precedes pre-cancellation and empty-buffer +success; those immediate paths touch no socket. The existing bridge lifetime anchor retains the +socket for native operations. Tests extend the one loopback fixture with small protocol callbacks; +the read gate has a timeout only to break deadlocks, and the second-write case gates the first +return hop behind an occupied one-worker `ThreadPool`. + +**Completion — 2026-08-30:** Added lazy non-coroutine `read_some()` and `write_all()` wrappers over +the existing socket state and native bridge. One atomic admission flag per direction permits a +read and write together while rejecting same-direction overlap through the requested return +Scheduler. Reads support empty buffers, partial delivery, retained terminal errors, sticky EOF, +and bytes-before-error; writes use Asio's composed `async_write()` and enforce all-or-error. The +configured libc++ lacks `std::scope_exit`, so admission release is explicit on both return-schedule +success and failure. The loopback fixture now accepts small synchronous protocols and enables +`reuse_address` for repeated tests. Focused Dev/Release passed 15/15; both full suites passed +131/131 across 10 binaries. Ten complete Release repetitions passed 150/150, and the two overlap +tests passed 100/100 across 50 repetitions. An earlier 100-repeat probe stopped in iteration 25 +when this host's 301-port ephemeral range was exhausted; no CMP assertion or operation failed. + ## 6. Implement cancellation, close, and shutdown races Add thread-safe `close()` and `is_open()`. `close()` marks logical state closed synchronously, then diff --git a/src/tcp.cppm b/src/tcp.cppm index 50d6601..ae3cd23 100644 --- a/src/tcp.cppm +++ b/src/tcp.cppm @@ -26,6 +26,14 @@ private: friend class TcpSocketState; public: + void ensure_accepting() { + const std::lock_guard lock { admissionMutex_ }; + + if (!accepting_) { + throw std::logic_error { "I/O context is stopping" }; + } + } + template void post(Handler&& handler) { const std::lock_guard lock { admissionMutex_ }; @@ -67,6 +75,10 @@ private: std::weak_ptr context_ {}; std::optional socket_ {}; std::atomic logicallyOpen_ { false }; + std::atomic readPending_ { false }; + std::atomic writePending_ { false }; + std::mutex readTerminalMutex_ {}; + std::optional readTerminal_ {}; public: explicit TcpSocketState( @@ -74,9 +86,9 @@ public: : context_ { context }, socket_ { std::in_place, context->ioContext_ } {} - [[nodiscard]] asio::ip::tcp::socket& socket_on_driver() noexcept { + [[nodiscard]] asio::ip::tcp::socket& socket_on_driver() { if (!socket_) { - std::terminate(); + throw std::logic_error { "TCP stream is closed" }; } return *socket_; @@ -90,6 +102,86 @@ public: } } + [[nodiscard]] std::shared_ptr lock_context() const { + const auto context = context_.lock(); + + if (!context) { + throw std::logic_error { "I/O context no longer exists" }; + } + + return context; + } + + void require_open() const { + if (!logicallyOpen_.load(std::memory_order_acquire)) { + throw std::logic_error { "TCP stream is closed" }; + } + } + + [[nodiscard]] bool read_available() { + const std::lock_guard lock { readTerminalMutex_ }; + + if (logicallyOpen_.load(std::memory_order_acquire)) { + return true; + } + + // fatal read 与字节同时完成时,关闭后仍需交付一次 retained error。 + return readTerminal_ && *readTerminal_ != asio::error::eof; + } + + [[nodiscard]] bool try_begin_read() noexcept { + bool expected { false }; + return readPending_.compare_exchange_strong( + expected, + true, + std::memory_order_acq_rel); + } + + void finish_read() noexcept { + if (!readPending_.exchange(false, std::memory_order_release)) { + std::terminate(); + } + } + + [[nodiscard]] bool try_begin_write() noexcept { + bool expected { false }; + return writePending_.compare_exchange_strong( + expected, + true, + std::memory_order_acq_rel); + } + + void finish_write() noexcept { + if (!writePending_.exchange(false, std::memory_order_release)) { + std::terminate(); + } + } + + [[nodiscard]] std::optional take_read_terminal() { + const std::lock_guard lock { readTerminalMutex_ }; + const auto terminal = readTerminal_; + + if (terminal && *terminal != asio::error::eof) { + readTerminal_.reset(); + } + + return terminal; + } + + void retain_read_terminal(std::error_code error) { + if (!error) { + std::terminate(); + } + + const std::lock_guard lock { readTerminalMutex_ }; + + if (readTerminal_) { + std::terminate(); + } + + readTerminal_ = error; + } + void close_on_driver() noexcept { logicallyOpen_.store(false, std::memory_order_release); @@ -415,6 +507,20 @@ private: std::uint16_t port, std::stop_token stopToken); + template + [[nodiscard]] static Task read_some_impl_( + std::shared_ptr state, + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken); + + template + [[nodiscard]] static Task write_all_impl_( + std::shared_ptr state, + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken); + public: TcpStream() = delete; TcpStream(const TcpStream&) = delete; @@ -444,6 +550,30 @@ public: std::string_view numericAddress, std::uint16_t port, std::stop_token stopToken = {}); + + template + requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } + ) + [[nodiscard]] Task read_some( + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken = {}); + + template + requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } + ) + [[nodiscard]] Task write_all( + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken = {}); }; template @@ -531,6 +661,202 @@ Task TcpStream::connect_impl_( co_return std::move(*stream); } +template +Task TcpStream::read_some_impl_( + std::shared_ptr state, + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken) { + std::optional transferred {}; + std::exception_ptr exception {}; + bool admitted { false }; + const auto releaseAdmission = [&] noexcept { + if (std::exchange(admitted, false)) { + state->finish_read(); + } + }; + + try { + if (!state) { + throw std::logic_error { "TCP stream was moved from" }; + } + + const auto context = state->lock_context(); + context->ensure_accepting(); + + if (!state->read_available()) { + throw std::logic_error { "TCP stream is closed" }; + } + + if (!state->try_begin_read()) { + throw std::logic_error { "TCP stream already has a pending read" }; + } + + admitted = true; + + if (const auto terminal = state->take_read_terminal()) { + if (*terminal == asio::error::eof) { + transferred = 0; + } else { + throw std::system_error { *terminal }; + } + } else { + state->require_open(); + + if (stopToken.stop_requested()) { + throw OperationCancelled {}; + } + + if (buffer.empty()) { + transferred = 0; + } else { + const auto native = co_await detail::NativeOperationAwaiter { + context, + stopToken, + state, + [state, buffer](auto completion) mutable { + state->socket_on_driver().async_read_some( + asio::buffer( + buffer.data(), + buffer.size_bytes()), + std::move(completion)); + } + }; + + if (native.transferred_ > buffer.size_bytes()) { + std::terminate(); + } + + if (!native.error_) { + transferred = native.transferred_; + + if (native.transferred_ == 0) { + state->retain_read_terminal(asio::error::eof); + } + } else if (native.transferred_ > 0) { + transferred = native.transferred_; + + if (native.error_ == asio::error::eof) { + state->retain_read_terminal(native.error_); + } else if (native.cancellation_ == + detail::NativeCancellationOrigin::none) { + state->retain_read_terminal(native.error_); + state->close_on_driver(); + } + } else if (native.cancellation_ != + detail::NativeCancellationOrigin::none) { + throw OperationCancelled {}; + } else if (native.error_ == asio::error::eof) { + state->retain_read_terminal(native.error_); + transferred = 0; + } else { + state->close_on_driver(); + throw std::system_error { native.error_ }; + } + } + } + } catch (...) { + exception = std::current_exception(); + } + + try { + co_await returnTo.schedule(); + } catch (...) { + releaseAdmission(); + throw; + } + + releaseAdmission(); + + if (exception) { + std::rethrow_exception(exception); + } + + co_return *transferred; +} + +template +Task TcpStream::write_all_impl_( + std::shared_ptr state, + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken) { + std::exception_ptr exception {}; + bool admitted { false }; + const auto releaseAdmission = [&] noexcept { + if (std::exchange(admitted, false)) { + state->finish_write(); + } + }; + + try { + if (!state) { + throw std::logic_error { "TCP stream was moved from" }; + } + + const auto context = state->lock_context(); + context->ensure_accepting(); + state->require_open(); + + if (!state->try_begin_write()) { + throw std::logic_error { "TCP stream already has a pending write" }; + } + + admitted = true; + + if (stopToken.stop_requested()) { + throw OperationCancelled {}; + } + + if (!buffer.empty()) { + const auto native = co_await detail::NativeOperationAwaiter { + context, + stopToken, + state, + [state, buffer](auto completion) mutable { + asio::async_write( + state->socket_on_driver(), + asio::buffer( + buffer.data(), + buffer.size_bytes()), + std::move(completion)); + } + }; + + if (native.error_) { + state->close_on_driver(); + + if (native.cancellation_ != + detail::NativeCancellationOrigin::none) { + throw OperationCancelled {}; + } + + throw std::system_error { native.error_ }; + } + + if (native.transferred_ != buffer.size_bytes()) { + state->close_on_driver(); + std::terminate(); + } + } + } catch (...) { + exception = std::current_exception(); + } + + try { + co_await returnTo.schedule(); + } catch (...) { + releaseAdmission(); + throw; + } + + releaseAdmission(); + + if (exception) { + std::rethrow_exception(exception); + } +} + template requires ( std::move_constructible && @@ -552,4 +878,40 @@ Task TcpStream::connect( std::move(stopToken)); } +template +requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } +) +Task TcpStream::read_some( + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken) { + return read_some_impl_( + state_, + std::move(returnTo), + buffer, + std::move(stopToken)); +} + +template +requires ( + std::move_constructible && + requires(const ReturnScheduler& scheduler) { + scheduler.schedule(); + } +) +Task TcpStream::write_all( + ReturnScheduler returnTo, + std::span buffer, + std::stop_token stopToken) { + return write_all_impl_( + state_, + std::move(returnTo), + buffer, + std::move(stopToken)); +} + } // namespace mcpplibs::cmp diff --git a/tests/tcp_test.cpp b/tests/tcp_test.cpp index ce214fc..0afc38a 100644 --- a/tests/tcp_test.cpp +++ b/tests/tcp_test.cpp @@ -10,6 +10,7 @@ using mcpplibs::cmp::IoContext; using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::RunLoop; using mcpplibs::cmp::Task; +using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::TcpStream; using mcpplibs::cmp::ThreadPool; @@ -33,6 +34,9 @@ static_assert(!std::is_copy_assignable_v); static_assert(!std::is_move_assignable_v); class LoopbackServer final { +public: + using Protocol = std::function; + private: asio::io_context ioContext_ {}; asio::ip::tcp::acceptor acceptor_ { ioContext_ }; @@ -42,6 +46,7 @@ class LoopbackServer final { std::binary_semaphore acceptedSignal_ { 0 }; std::latch release_ { 1 }; std::atomic accepted_ { false }; + Protocol protocol_ {}; std::jthread worker_ {}; void run_() noexcept { @@ -57,11 +62,19 @@ class LoopbackServer final { accepted_.store(true, std::memory_order_release); acceptedSignal_.release(); - release_.wait(); + + if (protocol_) { + std::invoke(protocol_, socket); + } else { + release_.wait(); + } } public: - explicit LoopbackServer(const asio::ip::address& address) { + explicit LoopbackServer( + const asio::ip::address& address, + Protocol protocol = {}) + : protocol_ { std::move(protocol) } { const asio::ip::tcp::endpoint requested { address, 0 }; acceptor_.open(requested.protocol(), startError_); @@ -69,6 +82,15 @@ class LoopbackServer final { return; } + // 重复测试会留下 TIME_WAIT;允许内核安全复用已关闭的 loopback 端口。 + acceptor_.set_option( + asio::socket_base::reuse_address { true }, + startError_); + + if (startError_) { + return; + } + acceptor_.bind(requested, startError_); if (startError_) { @@ -240,6 +262,183 @@ RunLoop::Scheduler make_expired_scheduler() { return loop.get_scheduler(); } +template +Task write_and_observe( + TcpStream& stream, + ReturnScheduler returnTo, + std::span buffer) { + co_await stream.write_all( + std::move(returnTo), + buffer); + co_return std::this_thread::get_id(); +} + +Task read_into( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::span buffer, + std::size_t& transferred) { + transferred = co_await stream.read_some( + std::move(returnTo), + buffer); +} + +template +Task write_bytes( + TcpStream& stream, + ReturnScheduler returnTo, + std::span buffer) { + co_await stream.write_all( + std::move(returnTo), + buffer); +} + +struct ReadAllObservation final { + std::vector bytes_ {}; + std::size_t repeatedEof_ {}; + std::thread::id resumedThread_ {}; +}; + +Task read_until_repeated_eof( + TcpStream& stream, + RunLoop::Scheduler returnTo) { + std::array buffer {}; + std::vector bytes {}; + + while (const auto size = co_await stream.read_some( + returnTo, + buffer)) { + bytes.insert( + bytes.end(), + buffer.begin(), + buffer.begin() + static_cast(size)); + } + + const auto repeatedEof = co_await stream.read_some( + returnTo, + buffer); + co_return ReadAllObservation { + std::move(bytes), + repeatedEof, + std::this_thread::get_id() + }; +} + +Task run_duplex( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::span readBuffer, + std::size_t& readSize, + std::span writeBuffer) { + TaskGroup group {}; + group.spawn(read_into( + stream, + returnTo, + readBuffer, + readSize)); + group.spawn(write_bytes( + stream, + returnTo, + writeBuffer)); + co_await group.join(); +} + +struct ReadOverlapObservation final { + bool rejected_ {}; + std::size_t firstRead_ {}; +}; + +Task reject_second_read( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::binary_semaphore& allowServerWrite) { + std::array firstBuffer {}; + std::array secondBuffer {}; + std::size_t firstRead {}; + bool rejected { false }; + TaskGroup group {}; + group.spawn(read_into( + stream, + returnTo, + firstBuffer, + firstRead)); + + try { + static_cast(co_await stream.read_some( + returnTo, + secondBuffer)); + } catch (const std::logic_error&) { + rejected = true; + } catch (...) { + } + + allowServerWrite.release(); + co_await group.join(); + co_return ReadOverlapObservation { rejected, firstRead }; +} + +Task occupy_worker( + ThreadPool::Scheduler scheduler, + std::binary_semaphore& entered, + std::atomic& release) { + co_await scheduler.schedule(); + entered.release(); + release.wait(false, std::memory_order_acquire); +} + +struct WriteOverlapObservation final { + bool workerEntered_ {}; + bool rejected_ {}; +}; + +Task reject_second_write( + TcpStream& stream, + RunLoop::Scheduler caller, + ThreadPool::Scheduler gatedReturn) { + std::binary_semaphore workerEntered { 0 }; + std::atomic releaseWorker { false }; + TaskGroup group {}; + group.spawn(occupy_worker( + gatedReturn, + workerEntered, + releaseWorker)); + + const bool entered = workerEntered.try_acquire_for(2s); + + if (!entered) { + releaseWorker.store(true, std::memory_order_release); + releaseWorker.notify_all(); + co_await group.join(); + co_return WriteOverlapObservation {}; + } + + const std::array first { + std::byte { 0x31 }, + std::byte { 0x32 }, + std::byte { 0x33 }, + std::byte { 0x34 } + }; + const std::array second { std::byte { 0x35 } }; + group.spawn(write_bytes( + stream, + gatedReturn, + std::span { first })); + + bool rejected { false }; + + try { + co_await stream.write_all(caller, std::span { second }); + } catch (const std::logic_error&) { + rejected = true; + } catch (...) { + } + + releaseWorker.store(true, std::memory_order_release); + releaseWorker.notify_all(); + co_await group.join(); + co_return WriteOverlapObservation { true, rejected }; +} + TEST(CmpTcpTest, IoContextStartsAndStopsCleanly) { IoContext context {}; } @@ -400,4 +599,348 @@ TEST(CmpTcpTest, RejectsInactiveAndExpiredReturnSchedulers) { } } +TEST(CmpTcpTest, EmptyReadAndWriteDoNotTouchTheSocket) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + + EXPECT_NO_THROW(loop.run(stream.write_all( + loop.get_scheduler(), + std::span {}))); + EXPECT_EQ( + loop.run(stream.read_some( + loop.get_scheduler(), + std::span {})), + 0U); +} + +TEST(CmpTcpTest, WritesTheEntireBorrowedBuffer) { + const std::array expected { + std::byte { 0x41 }, + std::byte { 0x42 }, + std::byte { 0x43 }, + std::byte { 0x44 } + }; + std::array received {}; + std::binary_semaphore serverDone { 0 }; + std::atomic serverSucceeded { false }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + std::error_code error {}; + const auto size = asio::read( + socket, + asio::buffer(received), + error); + serverSucceeded.store( + !error && size == received.size(), + std::memory_order_release); + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto resumedThread = loop.run(write_and_observe( + stream, + loop.get_scheduler(), + std::span { expected })); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_TRUE(serverSucceeded.load(std::memory_order_acquire)); + EXPECT_EQ(received, expected); + EXPECT_EQ(resumedThread, std::this_thread::get_id()); +} + +TEST(CmpTcpTest, ReadSomeReturnsAvailablePartialData) { + const std::array first { + std::byte { 0x51 }, + std::byte { 0x52 } + }; + const std::array second { + std::byte { 0x53 }, + std::byte { 0x54 } + }; + std::binary_semaphore firstSent { 0 }; + std::binary_semaphore allowSecond { 0 }; + std::binary_semaphore serverDone { 0 }; + std::atomic serverSucceeded { false }; + std::atomic gateTimedOut { false }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + std::error_code error {}; + const auto firstSize = asio::write( + socket, + asio::buffer(first), + error); + firstSent.release(); + + if (!allowSecond.try_acquire_for(2s)) { + gateTimedOut.store(true, std::memory_order_release); + } + + const auto secondSize = error + ? 0 + : asio::write( + socket, + asio::buffer(second), + error); + serverSucceeded.store( + !error && + firstSize == first.size() && + secondSize == second.size(), + std::memory_order_release); + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + ASSERT_TRUE(firstSent.try_acquire_for(2s)); + + std::array buffer {}; + const auto firstRead = loop.run(stream.read_some( + loop.get_scheduler(), + buffer)); + EXPECT_EQ(firstRead, first.size()); + EXPECT_TRUE(std::ranges::equal( + std::span { buffer }.first(firstRead), + first)); + + allowSecond.release(); + const auto secondRead = loop.run(stream.read_some( + loop.get_scheduler(), + buffer)); + EXPECT_EQ(secondRead, second.size()); + EXPECT_TRUE(std::ranges::equal( + std::span { buffer }.first(secondRead), + second)); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_FALSE(gateTimedOut.load(std::memory_order_acquire)); + EXPECT_TRUE(serverSucceeded.load(std::memory_order_acquire)); +} + +TEST(CmpTcpTest, DeliversAllBytesBeforeStickyEof) { + const std::vector payload { + std::byte { 0x61 }, + std::byte { 0x62 }, + std::byte { 0x63 }, + std::byte { 0x64 }, + std::byte { 0x65 } + }; + std::binary_semaphore serverDone { 0 }; + std::atomic serverSucceeded { false }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + std::error_code error {}; + const auto size = asio::write( + socket, + asio::buffer(payload), + error); + serverSucceeded.store( + !error && size == payload.size(), + std::memory_order_release); + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(read_until_repeated_eof( + stream, + loop.get_scheduler())); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_TRUE(serverSucceeded.load(std::memory_order_acquire)); + EXPECT_EQ(observation.bytes_, payload); + EXPECT_EQ(observation.repeatedEof_, 0U); + EXPECT_EQ(observation.resumedThread_, std::this_thread::get_id()); +} + +TEST(CmpTcpTest, SupportsOneSimultaneousReadAndWrite) { + const std::array request { + std::byte { 0x71 }, + std::byte { 0x72 }, + std::byte { 0x73 } + }; + const std::array response { + std::byte { 0x74 }, + std::byte { 0x75 } + }; + std::array received {}; + std::binary_semaphore serverDone { 0 }; + std::atomic serverSucceeded { false }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + std::error_code error {}; + const auto written = asio::write( + socket, + asio::buffer(response), + error); + const auto read = error + ? 0 + : asio::read( + socket, + asio::buffer(received), + error); + serverSucceeded.store( + !error && + written == response.size() && + read == request.size(), + std::memory_order_release); + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + std::array readBuffer {}; + std::size_t readSize {}; + loop.run(run_duplex( + stream, + loop.get_scheduler(), + readBuffer, + readSize, + request)); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_TRUE(serverSucceeded.load(std::memory_order_acquire)); + EXPECT_EQ(readSize, response.size()); + EXPECT_EQ(readBuffer, response); + EXPECT_EQ(received, request); +} + +TEST(CmpTcpTest, RejectsASecondPendingRead) { + std::binary_semaphore allowServerWrite { 0 }; + std::binary_semaphore serverDone { 0 }; + std::atomic gateTimedOut { false }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + const bool released = allowServerWrite.try_acquire_for(2s); + gateTimedOut.store(!released, std::memory_order_release); + const std::array oneByte { std::byte { 0x7a } }; + const std::array twoBytes { + std::byte { 0x7a }, + std::byte { 0x7b } + }; + std::error_code ignored {}; + + if (released) { + asio::write(socket, asio::buffer(oneByte), ignored); + } else { + asio::write(socket, asio::buffer(twoBytes), ignored); + } + + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(reject_second_read( + stream, + loop.get_scheduler(), + allowServerWrite)); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_FALSE(gateTimedOut.load(std::memory_order_acquire)); + EXPECT_TRUE(observation.rejected_); + EXPECT_EQ(observation.firstRead_, 1U); +} + +TEST(CmpTcpTest, RejectsASecondPendingWrite) { + const std::array expected { + std::byte { 0x31 }, + std::byte { 0x32 }, + std::byte { 0x33 }, + std::byte { 0x34 } + }; + std::array received {}; + std::binary_semaphore serverDone { 0 }; + std::atomic serverSucceeded { false }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + std::error_code error {}; + const auto size = asio::read( + socket, + asio::buffer(received), + error); + serverSucceeded.store( + !error && size == received.size(), + std::memory_order_release); + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + ThreadPool gatedReturn { 1 }; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(reject_second_write( + stream, + loop.get_scheduler(), + gatedReturn.get_scheduler())); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_TRUE(serverSucceeded.load(std::memory_order_acquire)); + EXPECT_TRUE(observation.workerEntered_); + EXPECT_TRUE(observation.rejected_); + EXPECT_EQ(received, expected); +} + } // namespace From 95ff43edfc9bd3f81019547e6d699abce03a1fd9 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 01:26:59 +0800 Subject: [PATCH 08/17] =?UTF-8?q?=E5=AE=9E=E7=8E=B0=20TCP=20=E5=8F=96?= =?UTF-8?q?=E6=B6=88=E4=B8=8E=E5=85=B3=E9=97=AD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 41 +- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 27 +- src/tcp.cppm | 134 +++- tests/tcp_test.cpp | 586 ++++++++++++++++++ 4 files changed, 759 insertions(+), 29 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index a940e74..164f285 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -6,7 +6,7 @@ CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 懒启动单消费者 `Task`、变参/vector `when_all()`、静止点 `TaskGroup`、一次性与可复用 事件、RAII `AsyncMutex`、带定时和取消的调用线程 `RunLoop`、固定大小的 `ThreadPool`,以及 用于隔离同步调用的 `run_blocking()`。Phase 6B 已加入拥有单一私有 I/O driver 的 -`IoContext` 生命周期骨架。 +`IoContext`,以及数值地址 TCP client 的 connect/read/write、取消和关闭生命周期。 `.xlings.json` 固定 mcpp 2026.8.11.2;当前工具链为 LLVM 22.1.8,运行时依赖为 `chriskohlhoff.asio` 1.38.1,测试依赖为 `compat.gtest` 1.15.2。`examples/basic` 是独立 @@ -14,9 +14,9 @@ path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1、Phase 6B 第 1–3 项与第 4 项分别作为本地提交 `701aa8b`、 -`816144f`、`76bb9bd`、`4f05767` 完成,均尚未推送。Phase 6B Plan 第 5 项 read/write、EOF 与 -overlap 已实现并通过本地门禁;公开 close/cancellation/shutdown 竞态尚未实现。 +Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完成,最新为 Step 5 +`a73c113`,均尚未推送。Phase 6B Plan 第 6 项 close/cancellation/shutdown 竞态已实现并通过 +本地门禁;下一步是第 7 项并发负载与跨平台门禁准备。 ## 已完成工作 @@ -85,6 +85,14 @@ overlap 已实现并通过本地门禁;公开 close/cancellation/shutdown 竞 异常两条路径显式释放 admission,没有新增 guard abstraction 或依赖。 - 同一 loopback fixture 增加小型同步 protocol callback、partial/EOF/duplex gate 与 `reuse_address`,没有增加 public server 或固定 sleep。 +- 新增线程安全、幂等、非阻塞且 `noexcept` 的 `TcpStream::close()` 与 `is_open()`;析构复用 + 同一关闭请求,move 后仅新 handle 保有关闭所有权。 +- socket 以弱引用记录 connect/read/write operation;stop、显式 close、write cancellation 与 + context shutdown 在 I/O 线程记录首个本地取消来源,再由唯一 native handler 恢复协程。 +- 显式关闭请求使用原子标记覆盖公开校验到 native initiation 的窄窗口;已接纳操作在 close + 获胜时稳定得到 `OperationCancelled`,而无关 socket error 仍保留 `system_error`。 +- TCP 测试新增 active/pre-cancellation、关闭幂等、move、write cancellation、两类完成竞态、 + context drain/join 和 surviving handle 边界,当前 `tcp_test` 为 23 项。 ## 重要决策 @@ -109,6 +117,8 @@ overlap 已实现并通过本地门禁;公开 close/cancellation/shutdown 竞 时才替换。 - native initiation 成功前的同步异常可以直接恢复私有协程;成功后恢复权只属于 Asio handler。 stop callback 不从请求线程 emit、访问 socket 或恢复 coroutine。 +- 显式 close 立即发布逻辑关闭状态,再把取消与 native close 排到 I/O 线程;资源失效检查先于 + pre-cancellation,远端 EOF 不关闭本地写方向。 - `TcpStream` 允许一项 pending read 与一项 pending write;同方向重叠直接拒绝,不增加隐式 排队。主动 close、context shutdown 与已发起 write 的取消均有明确关闭语义。 - v1 不含 DNS、server、TLS、UDP、文件 I/O、timeout、socket option 或隐式 executor;这些都 @@ -173,6 +183,10 @@ overlap 已实现并通过本地门禁;公开 close/cancellation/shutdown 竞 - 曾尝试 Release 整套重复 100 轮;前 24 轮 360/360 通过,第 25 轮因本机临时端口范围仅 `60700–61000` 而出现 `Address already in use`。fixture 随后增加 `reuse_address`,上述 10 轮 整套与 50 轮 overlap 复验通过;该次资源耗尽不计为功能通过,也未隐藏。 +- close/cancellation/shutdown 定向 `tcp_test` 在 Dev 与 Release 均为 23/23 通过;两套 strict + cache-off 全量测试均为 10 个二进制、139/139 通过。 +- Release 两项完成竞态连续执行 5 轮共 10/10 测试通过,覆盖 100 次 close/completion 与 + 500 次 stop/completion 竞态,没有丢失或重复完成。 ## 已知问题 / 风险 @@ -185,20 +199,19 @@ overlap 已实现并通过本地门禁;公开 close/cancellation/shutdown 竞 - 三平台兼容性仍需远程 CI 确认。 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 -- Phase 6B 当前已有 backend、`IoContext`、native bridge、connect/read/write 与 EOF/overlap; - pending operation 主动取消、公开 close/is_open、shutdown 竞态和 benchmark 迁移仍待后续步骤。 +- Phase 6B 本地已有 backend、`IoContext`、native bridge、connect/read/write、EOF/overlap、 + pending cancellation、close/is_open 与 shutdown 竞态;并发负载、readiness TCP client 迁移、 + 文档同步和最终本地矩阵仍待后续步骤。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 按 Phase 6B Plan 第 6 项实现公开 `close()`/`is_open()`,补齐 stop/close/context shutdown - cancellation provenance 与 exactly-once race tests。 -2. 完成本地 Dev/Release、race、example 与五轮 readiness 验证后,再等待远程三平台 CI 所需 - 的单独授权。 -3. Phase 6B 第 4 项已本地提交为 `4f05767`;当前 read/write、测试、Plan 与 HANDOFF 尚未提交。 - Push、PR 或 CI 均需对应明确授权。 +1. 按 Phase 6B Plan 第 7 项增加有界的多客户端并发负载测试,并完成两类竞态各至少 100 轮的 + Release 门禁。 +2. 继续第 8–10 项:迁移 readiness TCP client、同步六份开发文档并执行完整本地验证矩阵。 +3. 本地完成后仍需用户另行授权 push/PR,才能取得 Linux、macOS、Windows 远程 CI 结果。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 6 项,在现有 state/bridge 上补齐公开 -close/is_open、取消来源与 stop/close/shutdown exactly-once 竞态,不引入 timeout 或队列。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 7 项,在现有 loopback fixture 内加入 +跨平台有界并发客户端门禁,不增加 public server、通用 executor 或新依赖。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index 83f0e60..b8349e8 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — read/write gate complete +**Status:** In progress — cancellation/close/shutdown gate complete **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -279,6 +279,31 @@ same-direction-overlap rejection while the server withholds data. For races, rel and request stop/close from a barrier; accept either documented winner but assert exactly one completion and no duplicate buffer access. +**Self-review — 2026-08-30:** Each socket keeps weak connect/read/write operation slots populated +on the I/O thread before native initiation. Stop, explicit close, and context shutdown serialize on +that thread, record only the first local cancellation origin, and then ask the same Asio handler to +complete; none resumes a coroutine directly. Local cancel-and-close tags every live operation, +while an unrelated socket error performs an untagged native close and remains `system_error`. +Because an initiated write cancellation closes the whole stream, it also tags a simultaneous read +as locally cancelled. `close()` flips the atomic logical state before posting, so `is_open()` and +later admission observe closure immediately. Race tests accept exactly one documented outcome and +use barriers/semaphores only; the close race uses a bounded connection count to respect this host's +small ephemeral-port range. + +The review also found a validation-to-initiation close window. A socket-local atomic close-request +marker now makes an already-admitted read/write report `OperationCancelled` if explicit close wins +before native initiation; closed-resource validation still precedes a pre-stopped token. + +**Completion — 2026-08-30:** Added thread-safe `close()`/`is_open()` and made destruction use the +same idempotent close request. Weak per-direction operation slots let stop, close, write +cancellation, and context shutdown record local provenance on the I/O thread before native close; +only the Asio completion handler resumes an initiated operation. Tests cover active/pre- +cancellation, stream reuse, close idempotence and moved-from behavior, write cancellation, +close/completion and stop/completion races, context drain/join, surviving handles, return affinity, +and resource-error precedence. Focused Dev/Release passed 23/23; both full suites passed 139/139 +across 10 binaries. Five focused Release repetitions passed 10/10 tests, representing 100 close +races and 500 stop races with no lost or duplicate completion. + ## 7. Add the load and cross-platform gate Extend the same test file with many concurrent IPv4 loopback clients. Give every client independent diff --git a/src/tcp.cppm b/src/tcp.cppm index ae3cd23..aed71f1 100644 --- a/src/tcp.cppm +++ b/src/tcp.cppm @@ -8,6 +8,14 @@ import :task; namespace mcpplibs::cmp::detail { class TcpSocketState; +struct NativeOperationState; + +enum class NativeCancellationOrigin { + none, + stop_token, + stream_close, + context_shutdown +}; class IoContextState final { private: @@ -75,10 +83,14 @@ private: std::weak_ptr context_ {}; std::optional socket_ {}; std::atomic logicallyOpen_ { false }; + std::atomic closeRequested_ { false }; std::atomic readPending_ { false }; std::atomic writePending_ { false }; std::mutex readTerminalMutex_ {}; std::optional readTerminal_ {}; + std::weak_ptr connectOperation_ {}; + std::weak_ptr readOperation_ {}; + std::weak_ptr writeOperation_ {}; public: explicit TcpSocketState( @@ -118,9 +130,21 @@ public: } } + [[nodiscard]] bool is_open() const noexcept { + return logicallyOpen_.load(std::memory_order_acquire); + } + + [[nodiscard]] bool close_requested() const noexcept { + return closeRequested_.load(std::memory_order_acquire); + } + [[nodiscard]] bool read_available() { const std::lock_guard lock { readTerminalMutex_ }; + if (closeRequested_.load(std::memory_order_acquire)) { + return false; + } + if (logicallyOpen_.load(std::memory_order_acquire)) { return true; } @@ -182,6 +206,18 @@ public: readTerminal_ = error; } + void attach_connect_operation_on_driver( + const std::shared_ptr& operation) noexcept; + + void attach_read_operation_on_driver( + const std::shared_ptr& operation) noexcept; + + void attach_write_operation_on_driver( + const std::shared_ptr& operation) noexcept; + + void cancel_and_close_on_driver( + NativeCancellationOrigin origin) noexcept; + void close_on_driver() noexcept { logicallyOpen_.store(false, std::memory_order_release); @@ -196,6 +232,8 @@ public: void request_close( const std::shared_ptr& self) noexcept { + closeRequested_.store(true, std::memory_order_release); + if (!logicallyOpen_.exchange(false, std::memory_order_acq_rel)) { return; } @@ -208,7 +246,8 @@ public: try { context->post([self] noexcept { - self->close_on_driver(); + self->cancel_and_close_on_driver( + NativeCancellationOrigin::stream_close); }); } catch (const std::logic_error&) { // Shutdown 已接管 registry 中的 socket。 @@ -221,7 +260,8 @@ public: void IoContextState::shutdown_on_driver_() noexcept { for (auto& registered : sockets_) { if (const auto socket = registered.lock()) { - socket->close_on_driver(); + socket->cancel_and_close_on_driver( + NativeCancellationOrigin::context_shutdown); } } @@ -229,13 +269,6 @@ void IoContextState::shutdown_on_driver_() noexcept { workGuard_.reset(); } -enum class NativeCancellationOrigin { - none, - stop_token, - stream_close, - context_shutdown -}; - struct NativeOperationState final { struct Result final { std::error_code error_ {}; @@ -301,6 +334,10 @@ struct NativeOperationState final { void request_cancellation_on_driver( NativeCancellationOrigin origin) noexcept { + if (origin == NativeCancellationOrigin::none) { + std::terminate(); + } + if (completed_ || cancellation_ != NativeCancellationOrigin::none) { return; } @@ -339,6 +376,48 @@ private: } }; +void TcpSocketState::attach_connect_operation_on_driver( + const std::shared_ptr& operation) noexcept { + if (!operation || !connectOperation_.expired()) { + std::terminate(); + } + + connectOperation_ = operation; +} + +void TcpSocketState::attach_read_operation_on_driver( + const std::shared_ptr& operation) noexcept { + if (!operation || !readOperation_.expired()) { + std::terminate(); + } + + readOperation_ = operation; +} + +void TcpSocketState::attach_write_operation_on_driver( + const std::shared_ptr& operation) noexcept { + if (!operation || !writeOperation_.expired()) { + std::terminate(); + } + + writeOperation_ = operation; +} + +void TcpSocketState::cancel_and_close_on_driver( + NativeCancellationOrigin origin) noexcept { + const auto cancel = [origin]( + const std::weak_ptr& operation) noexcept { + if (const auto locked = operation.lock()) { + locked->request_cancellation_on_driver(origin); + } + }; + + cancel(connectOperation_); + cancel(readOperation_); + cancel(writeOperation_); + close_on_driver(); +} + void NativeOperationState::StopRequest::operator()() const noexcept { const auto operation = operation_.lock(); const auto context = context_.lock(); @@ -417,6 +496,7 @@ public: NativeOperationState::CompletionHandler { operation }); std::invoke( std::move(initiation), + operation, std::move(completion)); } catch (...) { operation->fail_initiation_on_driver( @@ -532,11 +612,19 @@ public: TcpStream& operator=(TcpStream&&) = delete; ~TcpStream() { + close(); + } + + void close() noexcept { if (state_) { state_->request_close(state_); } } + [[nodiscard]] bool is_open() const noexcept { + return state_ && state_->is_open(); + } + template requires ( std::move_constructible && @@ -604,7 +692,7 @@ Task TcpStream::connect_impl_( numericAddress = std::move(numericAddress), port, stopToken - ](auto completion) mutable { + ](const auto& operation, auto completion) mutable { if (stopToken.stop_requested()) { throw OperationCancelled {}; } @@ -622,6 +710,7 @@ Task TcpStream::connect_impl_( context); connectState->socket_ = socket; context->register_socket_on_driver(socket); + socket->attach_connect_operation_on_driver(operation); socket->socket_on_driver().async_connect( asio::ip::tcp::endpoint { address, port }, std::move(completion)); @@ -714,7 +803,15 @@ Task TcpStream::read_some_impl_( context, stopToken, state, - [state, buffer](auto completion) mutable { + [state, buffer]( + const auto& operation, + auto completion) mutable { + state->attach_read_operation_on_driver(operation); + + if (state->close_requested()) { + throw OperationCancelled {}; + } + state->socket_on_driver().async_read_some( asio::buffer( buffer.data(), @@ -813,7 +910,15 @@ Task TcpStream::write_all_impl_( context, stopToken, state, - [state, buffer](auto completion) mutable { + [state, buffer]( + const auto& operation, + auto completion) mutable { + state->attach_write_operation_on_driver(operation); + + if (state->close_requested()) { + throw OperationCancelled {}; + } + asio::async_write( state->socket_on_driver(), asio::buffer( @@ -824,13 +929,14 @@ Task TcpStream::write_all_impl_( }; if (native.error_) { - state->close_on_driver(); - if (native.cancellation_ != detail::NativeCancellationOrigin::none) { + state->cancel_and_close_on_driver( + native.cancellation_); throw OperationCancelled {}; } + state->close_on_driver(); throw std::system_error { native.error_ }; } diff --git a/tests/tcp_test.cpp b/tests/tcp_test.cpp index 0afc38a..6e33488 100644 --- a/tests/tcp_test.cpp +++ b/tests/tcp_test.cpp @@ -32,6 +32,8 @@ static_assert(std::move_constructible); static_assert(std::is_nothrow_move_constructible_v); static_assert(!std::is_copy_assignable_v); static_assert(!std::is_move_assignable_v); +static_assert(noexcept(std::declval().close())); +static_assert(noexcept(std::declval().is_open())); class LoopbackServer final { public: @@ -439,6 +441,264 @@ Task reject_second_write( co_return WriteOverlapObservation { true, rejected }; } +struct OperationObservation final { + int completions_ {}; + std::size_t transferred_ {}; + bool cancelled_ {}; + bool logicError_ {}; + bool systemError_ {}; + bool unexpected_ {}; + std::thread::id resumedThread_ {}; +}; + +Task observe_read( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::span buffer, + OperationObservation& observation, + std::stop_token stopToken = {}) { + try { + observation.transferred_ = co_await stream.read_some( + std::move(returnTo), + buffer, + std::move(stopToken)); + } catch (const OperationCancelled&) { + observation.cancelled_ = true; + } catch (const std::logic_error&) { + observation.logicError_ = true; + } catch (const std::system_error&) { + observation.systemError_ = true; + } catch (...) { + observation.unexpected_ = true; + } + + ++observation.completions_; + observation.resumedThread_ = std::this_thread::get_id(); +} + +Task observe_write( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::span buffer, + OperationObservation& observation, + std::stop_token stopToken = {}) { + try { + co_await stream.write_all( + std::move(returnTo), + buffer, + std::move(stopToken)); + } catch (const OperationCancelled&) { + observation.cancelled_ = true; + } catch (const std::logic_error&) { + observation.logicError_ = true; + } catch (const std::system_error&) { + observation.systemError_ = true; + } catch (...) { + observation.unexpected_ = true; + } + + ++observation.completions_; + observation.resumedThread_ = std::this_thread::get_id(); +} + +struct ActiveReadCancellationObservation final { + OperationObservation operation_ {}; + bool overlapRejected_ {}; + std::size_t reuseRead_ {}; +}; + +Task cancel_pending_read( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::stop_source& stopSource, + std::binary_semaphore& allowServerWrite) { + std::array pendingBuffer {}; + std::array overlapBuffer {}; + std::array reuseBuffer {}; + ActiveReadCancellationObservation result {}; + TaskGroup group {}; + group.spawn(observe_read( + stream, + returnTo, + pendingBuffer, + result.operation_, + stopSource.get_token())); + + try { + static_cast(co_await stream.read_some( + returnTo, + overlapBuffer)); + } catch (const std::logic_error&) { + result.overlapRejected_ = true; + } catch (...) { + } + + stopSource.request_stop(); + co_await group.join(); + allowServerWrite.release(); + result.reuseRead_ = co_await stream.read_some( + returnTo, + reuseBuffer); + co_return result; +} + +struct CloseReadObservation final { + OperationObservation operation_ {}; + bool overlapRejected_ {}; +}; + +Task close_pending_read( + TcpStream& stream, + RunLoop::Scheduler returnTo) { + std::array pendingBuffer {}; + std::array overlapBuffer {}; + CloseReadObservation result {}; + TaskGroup group {}; + group.spawn(observe_read( + stream, + returnTo, + pendingBuffer, + result.operation_)); + + try { + static_cast(co_await stream.read_some( + returnTo, + overlapBuffer)); + } catch (const std::logic_error&) { + result.overlapRejected_ = true; + } catch (...) { + } + + stream.close(); + stream.close(); + co_await group.join(); + co_return result; +} + +struct ActiveWriteCancellationObservation final { + OperationObservation operation_ {}; + bool overlapRejected_ {}; +}; + +Task cancel_pending_write( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::span buffer, + std::stop_source& stopSource) { + const std::array overlapByte { std::byte { 0x11 } }; + ActiveWriteCancellationObservation result {}; + TaskGroup group {}; + group.spawn(observe_write( + stream, + returnTo, + buffer, + result.operation_, + stopSource.get_token())); + + try { + co_await stream.write_all(returnTo, overlapByte); + } catch (const std::logic_error&) { + result.overlapRejected_ = true; + } catch (...) { + } + + stopSource.request_stop(); + co_await group.join(); + co_return result; +} + +Task destroy_context_with_pending_read( + std::unique_ptr& context, + TcpStream& stream, + RunLoop::Scheduler returnTo) { + std::array buffer {}; + OperationObservation result {}; + TaskGroup group {}; + group.spawn(observe_read( + stream, + returnTo, + buffer, + result)); + + std::jthread destroyer { [&context] { + context.reset(); + } }; + co_await group.join(); + destroyer.join(); + co_return result; +} + +Task run_close_completion_race( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::barrier<>& start) { + std::array buffer {}; + OperationObservation result {}; + TaskGroup group {}; + group.spawn(observe_read( + stream, + returnTo, + buffer, + result)); + + std::jthread closer { [&stream, &start] { + start.arrive_and_wait(); + stream.close(); + } }; + co_await group.join(); + closer.join(); + co_return result; +} + +struct RaceCounts final { + int completed_ {}; + int cancelled_ {}; + int invalid_ {}; +}; + +Task run_stop_completion_races( + TcpStream& stream, + RunLoop::Scheduler returnTo, + std::barrier<>& start, + int raceCount) { + RaceCounts counts {}; + + for (int index { 0 }; index < raceCount; ++index) { + std::array buffer {}; + std::stop_source stopSource {}; + OperationObservation result {}; + TaskGroup group {}; + group.spawn(observe_read( + stream, + returnTo, + buffer, + result, + stopSource.get_token())); + + std::jthread stopper { [&stopSource, &start] { + start.arrive_and_wait(); + stopSource.request_stop(); + } }; + co_await group.join(); + stopper.join(); + + if (result.completions_ != 1 || + result.logicError_ || + result.systemError_ || + result.unexpected_) { + ++counts.invalid_; + } else if (result.cancelled_) { + ++counts.cancelled_; + } else if (result.transferred_ == 1) { + ++counts.completed_; + } else { + ++counts.invalid_; + } + } + + co_return counts; +} + TEST(CmpTcpTest, IoContextStartsAndStopsCleanly) { IoContext context {}; } @@ -787,6 +1047,7 @@ TEST(CmpTcpTest, DeliversAllBytesBeforeStickyEof) { EXPECT_EQ(observation.bytes_, payload); EXPECT_EQ(observation.repeatedEof_, 0U); EXPECT_EQ(observation.resumedThread_, std::this_thread::get_id()); + EXPECT_TRUE(stream.is_open()); } TEST(CmpTcpTest, SupportsOneSimultaneousReadAndWrite) { @@ -943,4 +1204,329 @@ TEST(CmpTcpTest, RejectsASecondPendingWrite) { EXPECT_EQ(received, expected); } +TEST(CmpTcpTest, CancelsPendingReadAndKeepsStreamUsable) { + std::binary_semaphore allowServerWrite { 0 }; + std::binary_semaphore serverDone { 0 }; + std::atomic gateTimedOut { false }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + const bool released = allowServerWrite.try_acquire_for(2s); + gateTimedOut.store(!released, std::memory_order_release); + const std::array oneByte { std::byte { 0x21 } }; + const std::array twoBytes { + std::byte { 0x21 }, + std::byte { 0x22 } + }; + std::error_code ignored {}; + + if (released) { + asio::write(socket, asio::buffer(oneByte), ignored); + } else { + asio::write(socket, asio::buffer(twoBytes), ignored); + } + + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + std::stop_source stopSource {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(cancel_pending_read( + stream, + loop.get_scheduler(), + stopSource, + allowServerWrite)); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_FALSE(gateTimedOut.load(std::memory_order_acquire)); + EXPECT_TRUE(observation.overlapRejected_); + EXPECT_EQ(observation.operation_.completions_, 1); + EXPECT_TRUE(observation.operation_.cancelled_); + EXPECT_FALSE(observation.operation_.logicError_); + EXPECT_FALSE(observation.operation_.systemError_); + EXPECT_FALSE(observation.operation_.unexpected_); + EXPECT_EQ(observation.operation_.resumedThread_, + std::this_thread::get_id()); + EXPECT_EQ(observation.reuseRead_, 1U); + EXPECT_TRUE(stream.is_open()); +} + +TEST(CmpTcpTest, PreCancellationLeavesOpenStreamUsable) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + std::stop_source stopSource {}; + stopSource.request_stop(); + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + std::array buffer {}; + + EXPECT_THROW( + loop.run(stream.read_some( + loop.get_scheduler(), + buffer, + stopSource.get_token())), + OperationCancelled); + EXPECT_THROW( + loop.run(stream.write_all( + loop.get_scheduler(), + buffer, + stopSource.get_token())), + OperationCancelled); + EXPECT_TRUE(stream.is_open()); +} + +TEST(CmpTcpTest, CloseCancelsPendingReadAndIsIdempotent) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(close_pending_read( + stream, + loop.get_scheduler())); + + EXPECT_TRUE(observation.overlapRejected_); + EXPECT_EQ(observation.operation_.completions_, 1); + EXPECT_TRUE(observation.operation_.cancelled_); + EXPECT_FALSE(observation.operation_.logicError_); + EXPECT_FALSE(observation.operation_.systemError_); + EXPECT_FALSE(observation.operation_.unexpected_); + EXPECT_FALSE(stream.is_open()); + EXPECT_NO_THROW(stream.close()); + + std::stop_source stopSource {}; + stopSource.request_stop(); + std::array buffer {}; + EXPECT_THROW( + loop.run(stream.read_some( + loop.get_scheduler(), + buffer, + stopSource.get_token())), + std::logic_error); + EXPECT_THROW( + loop.run(stream.write_all( + loop.get_scheduler(), + buffer, + stopSource.get_token())), + std::logic_error); +} + +TEST(CmpTcpTest, MoveTransfersCloseOwnershipAndMovedFromUseFails) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + TcpStream moved { std::move(stream) }; + std::array buffer {}; + + EXPECT_FALSE(stream.is_open()); + EXPECT_TRUE(moved.is_open()); + EXPECT_NO_THROW(stream.close()); + EXPECT_THROW( + loop.run(stream.read_some(loop.get_scheduler(), buffer)), + std::logic_error); + EXPECT_THROW( + loop.run(stream.write_all(loop.get_scheduler(), buffer)), + std::logic_error); + + moved.close(); + moved.close(); + EXPECT_FALSE(moved.is_open()); +} + +TEST(CmpTcpTest, CancellingPendingWriteClosesStream) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + std::stop_source stopSource {}; + std::vector buffer( + 8U * 1024U * 1024U, + std::byte { 0x2a }); + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(cancel_pending_write( + stream, + loop.get_scheduler(), + buffer, + stopSource)); + + EXPECT_TRUE(observation.overlapRejected_); + EXPECT_EQ(observation.operation_.completions_, 1); + EXPECT_TRUE(observation.operation_.cancelled_); + EXPECT_FALSE(observation.operation_.logicError_); + EXPECT_FALSE(observation.operation_.systemError_); + EXPECT_FALSE(observation.operation_.unexpected_); + EXPECT_FALSE(stream.is_open()); +} + +TEST(CmpTcpTest, CloseCompletionRaceChoosesOneOutcome) { + constexpr int RACE_COUNT { 20 }; + IoContext context {}; + RunLoop loop {}; + int completed {}; + int cancelled {}; + int invalid {}; + + for (int iteration { 0 }; iteration < RACE_COUNT; ++iteration) { + std::barrier start { 2 }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + start.arrive_and_wait(); + const std::array byte { std::byte { 0x2b } }; + std::error_code ignored {}; + asio::write(socket, asio::buffer(byte), ignored); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(run_close_completion_race( + stream, + loop.get_scheduler(), + start)); + + if (observation.completions_ != 1 || + observation.logicError_ || + observation.systemError_ || + observation.unexpected_) { + ++invalid; + } else if (observation.cancelled_) { + ++cancelled; + } else if (observation.transferred_ == 1) { + ++completed; + } else { + ++invalid; + } + + EXPECT_FALSE(stream.is_open()); + } + + EXPECT_EQ(completed + cancelled, RACE_COUNT); + EXPECT_EQ(invalid, 0); +} + +TEST(CmpTcpTest, StopCompletionRaceChoosesOneOutcome) { + constexpr int RACE_COUNT { 100 }; + std::barrier start { 2 }; + std::binary_semaphore serverDone { 0 }; + std::atomic serverSucceeded { true }; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + for (int iteration { 0 }; iteration < RACE_COUNT; ++iteration) { + start.arrive_and_wait(); + + if (serverSucceeded.load(std::memory_order_acquire)) { + const std::array byte { std::byte { 0x2c } }; + std::error_code error {}; + asio::write(socket, asio::buffer(byte), error); + + if (error) { + serverSucceeded.store(false, std::memory_order_release); + } + } + } + + serverDone.release(); + } + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto counts = loop.run(run_stop_completion_races( + stream, + loop.get_scheduler(), + start, + RACE_COUNT)); + + ASSERT_TRUE(serverDone.try_acquire_for(2s)); + EXPECT_TRUE(serverSucceeded.load(std::memory_order_acquire)); + EXPECT_EQ(counts.completed_ + counts.cancelled_, RACE_COUNT); + EXPECT_EQ(counts.invalid_, 0); + EXPECT_TRUE(stream.is_open()); +} + +TEST(CmpTcpTest, ContextShutdownDrainsPendingReadAndClosesSurvivor) { + LoopbackServer server { asio::ip::address_v4::loopback() }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + auto context = std::make_unique(); + RunLoop loop {}; + auto stream = loop.run(TcpStream::connect( + *context, + loop.get_scheduler(), + "127.0.0.1", + server.port())); + const auto observation = loop.run(destroy_context_with_pending_read( + context, + stream, + loop.get_scheduler())); + std::array buffer {}; + + EXPECT_FALSE(context); + EXPECT_EQ(observation.completions_, 1); + EXPECT_TRUE(observation.cancelled_); + EXPECT_FALSE(observation.logicError_); + EXPECT_FALSE(observation.systemError_); + EXPECT_FALSE(observation.unexpected_); + EXPECT_EQ(observation.resumedThread_, std::this_thread::get_id()); + EXPECT_FALSE(stream.is_open()); + EXPECT_THROW( + loop.run(stream.read_some(loop.get_scheduler(), buffer)), + std::logic_error); + EXPECT_THROW( + loop.run(stream.write_all(loop.get_scheduler(), buffer)), + std::logic_error); + EXPECT_NO_THROW(stream.close()); +} + } // namespace From eeb0275ac87e145ac48716248497afdbf02325d8 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 01:33:31 +0800 Subject: [PATCH 09/17] =?UTF-8?q?=E5=A2=9E=E5=8A=A0=20TCP=20=E5=B9=B6?= =?UTF-8?q?=E5=8F=91=E8=B4=9F=E8=BD=BD=E9=97=A8=E7=A6=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 22 +- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 22 +- tests/tcp_test.cpp | 225 ++++++++++++++++-- 3 files changed, 244 insertions(+), 25 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 164f285..1c440bc 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -15,8 +15,8 @@ path-dependency consumer。 ## 当前目标与状态 Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完成,最新为 Step 5 -`a73c113`,均尚未推送。Phase 6B Plan 第 6 项 close/cancellation/shutdown 竞态已实现并通过 -本地门禁;下一步是第 7 项并发负载与跨平台门禁准备。 +`a73c113`,Step 6 已提交为 `95ff43e`,均尚未推送。Phase 6B Plan 第 7 项并发负载与本地 +跨平台门禁已实现并通过;下一步是第 8 项 readiness TCP client 迁移。 ## 已完成工作 @@ -93,6 +93,8 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 获胜时稳定得到 `OperationCancelled`,而无关 socket error 仍保留 `system_error`。 - TCP 测试新增 active/pre-cancellation、关闭幂等、move、write cancellation、两类完成竞态、 context drain/join 和 surviving handle 边界,当前 `tcp_test` 为 23 项。 +- 现有 loopback fixture 支持固定连接数;新增 32 客户端并发 echo 测试,每个 Task 独立持有 + stream/缓冲区并逐槽核对完成次数、结果、payload 与 RunLoop 返回线程。 ## 重要决策 @@ -187,6 +189,8 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 cache-off 全量测试均为 10 个二进制、139/139 通过。 - Release 两项完成竞态连续执行 5 轮共 10/10 测试通过,覆盖 100 次 close/completion 与 500 次 stop/completion 竞态,没有丢失或重复完成。 +- 加入并发负载后,`tcp_test` Dev/Release 均为 24/24;两套 strict cache-off 全量测试均为 + 10 个二进制、140/140。32/32 客户端成功,取消、错误、payload/线程不匹配和重复完成均为 0。 ## 已知问题 / 风险 @@ -200,18 +204,18 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 - Phase 6B 本地已有 backend、`IoContext`、native bridge、connect/read/write、EOF/overlap、 - pending cancellation、close/is_open 与 shutdown 竞态;并发负载、readiness TCP client 迁移、 - 文档同步和最终本地矩阵仍待后续步骤。 + pending cancellation、close/is_open、shutdown 竞态与 32 客户端负载;readiness TCP client + 迁移、文档同步和最终本地矩阵仍待后续步骤。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 按 Phase 6B Plan 第 7 项增加有界的多客户端并发负载测试,并完成两类竞态各至少 100 轮的 - Release 门禁。 -2. 继续第 8–10 项:迁移 readiness TCP client、同步六份开发文档并执行完整本地验证矩阵。 +1. 按 Phase 6B Plan 第 8 项只迁移 readiness 的 TCP client,保留同步 server、compute/file + 场景、常量、CSV 和硬计数。 +2. 继续第 9–10 项:同步六份开发文档并执行完整本地验证矩阵与五轮 benchmark。 3. 本地完成后仍需用户另行授权 push/PR,才能取得 Linux、macOS、Windows 远程 CI 结果。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 7 项,在现有 loopback fixture 内加入 -跨平台有界并发客户端门禁,不增加 public server、通用 executor 或新依赖。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 8 项,用共享 `IoContext`/`TcpStream` 替换 +readiness 的阻塞 TCP client;同步 loopback server 继续使用 `run_blocking()`。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index b8349e8..8655ed5 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — cancellation/close/shutdown gate complete +**Status:** In progress — load and local cross-platform gate complete **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -320,6 +320,26 @@ The existing Linux/macOS/Windows workflows already build the root package and au remain an acceptance gate and can be recorded only after the user separately authorizes the required remote workflow. +**Self-review — 2026-08-30:** Use 32 IPv4 clients: this admits many simultaneous connect/read/write +operations on the one driver while staying well below hosted-CI and local ephemeral-port limits. +Extend the existing synchronous fixture with an expected connection count and process tiny echo +sessions sequentially on its test thread; the clients remain concurrent because all Tasks start +before the group joins. Each client coroutine owns its stream and request/reply arrays, writes one +unique payload, reads until that payload is complete, and records exactly one success, +cancellation, or error in its own result slot. Aggregate completion, outcome, payload, and return- +thread counts only after structured join. Keep the existing internally repeated race tests; five +focused binary repetitions represent 100 close races and 500 stop races without creating 100 +copies of their already-repeated inner loops. No workflow or production API change belongs here. + +**Completion — 2026-08-30:** Extended the existing test fixture with a bounded connection count +and added one 32-client IPv4 echo load. All client Tasks start before structured join; every Task +owns its stream and buffers, verifies its unique echo, returns to the RunLoop thread, and records +one outcome. Focused Dev/Release passed 24/24 with 32/32 successes and zero cancellations, errors, +affinity mismatches, or duplicate completions. Both full strict cache-off suites passed 140/140 +across 10 binaries. Five focused Release repetitions passed 10/10 race tests, covering 100 close +races and 500 stop races. No production API or workflow changed; macOS/Windows confirmation remains +a separately authorized remote CI gate. + ## 8. Migrate only the readiness TCP client Keep `benchmarks/v1-readiness` POSIX-only and retain its synchronous loopback server on the existing diff --git a/tests/tcp_test.cpp b/tests/tcp_test.cpp index 6e33488..485897d 100644 --- a/tests/tcp_test.cpp +++ b/tests/tcp_test.cpp @@ -48,35 +48,54 @@ class LoopbackServer final { std::binary_semaphore acceptedSignal_ { 0 }; std::latch release_ { 1 }; std::atomic accepted_ { false }; + std::size_t connectionCount_ { 1 }; + std::atomic remainingConnections_ { 1 }; Protocol protocol_ {}; std::jthread worker_ {}; void run_() noexcept { ready_.release(); - asio::ip::tcp::socket socket { ioContext_ }; - std::error_code error {}; - acceptor_.accept(socket, error); + // 服务端串行处理会话;客户端仍在同一 IoContext 上并发等待。 + for (std::size_t index {}; index < connectionCount_; ++index) { + asio::ip::tcp::socket socket { ioContext_ }; + std::error_code error {}; + acceptor_.accept(socket, error); - if (error) { - return; - } + if (error) { + return; + } - accepted_.store(true, std::memory_order_release); - acceptedSignal_.release(); + const auto remaining = remainingConnections_.fetch_sub( + 1, + std::memory_order_acq_rel); - if (protocol_) { - std::invoke(protocol_, socket); - } else { - release_.wait(); + if (remaining == 0) { + std::terminate(); + } + + accepted_.store(true, std::memory_order_release); + + if (remaining == connectionCount_) { + acceptedSignal_.release(); + } + + if (protocol_) { + std::invoke(protocol_, socket); + } else { + release_.wait(); + } } } public: explicit LoopbackServer( const asio::ip::address& address, - Protocol protocol = {}) - : protocol_ { std::move(protocol) } { + Protocol protocol = {}, + std::size_t connectionCount = 1) + : connectionCount_ { connectionCount }, + remainingConnections_ { connectionCount }, + protocol_ { std::move(protocol) } { const asio::ip::tcp::endpoint requested { address, 0 }; acceptor_.open(requested.protocol(), startError_); @@ -130,7 +149,11 @@ class LoopbackServer final { release_.count_down(); - if (!accepted_.load(std::memory_order_acquire)) { + const auto remaining = remainingConnections_.load( + std::memory_order_acquire); + + // 失败路径唤醒未完成的 accept,避免 fixture 析构挂起。 + for (std::size_t index {}; index < remaining; ++index) { asio::io_context wakeContext {}; asio::ip::tcp::socket wakeSocket { wakeContext }; std::error_code ignored {}; @@ -699,6 +722,122 @@ Task run_stop_completion_races( co_return counts; } +enum class LoadOutcome { + succeeded, + cancelled, + error +}; + +struct LoadClientObservation final { + int completions_ {}; + LoadOutcome outcome_ { LoadOutcome::error }; + std::thread::id resumedThread_ {}; +}; + +Task run_echo_client( + IoContext& context, + RunLoop::Scheduler returnTo, + std::uint16_t port, + std::size_t clientIndex, + LoadClientObservation& observation) { + const std::array request { + std::byte { 0x43 }, + std::byte { 0x4d }, + std::byte { static_cast(clientIndex) }, + std::byte { + static_cast(clientIndex ^ 0x5aU) + } + }; + std::array reply {}; + + try { + auto stream = co_await TcpStream::connect( + context, + returnTo, + "127.0.0.1", + port); + co_await stream.write_all(returnTo, request); + + std::size_t received {}; + + while (received < reply.size()) { + const auto size = co_await stream.read_some( + returnTo, + std::span { reply }.subspan(received)); + + if (size == 0) { + break; + } + + received += size; + } + + observation.outcome_ = + received == reply.size() && reply == request + ? LoadOutcome::succeeded + : LoadOutcome::error; + } catch (const OperationCancelled&) { + observation.outcome_ = LoadOutcome::cancelled; + } catch (...) { + observation.outcome_ = LoadOutcome::error; + } + + ++observation.completions_; + observation.resumedThread_ = std::this_thread::get_id(); +} + +struct LoadCounts final { + int completions_ {}; + int succeeded_ {}; + int cancelled_ {}; + int errors_ {}; + int invalidCompletions_ {}; + int wrongThread_ {}; +}; + +Task run_concurrent_echo_clients( + IoContext& context, + RunLoop::Scheduler returnTo, + std::uint16_t port, + std::size_t clientCount) { + const auto expectedThread = std::this_thread::get_id(); + std::vector observations(clientCount); + TaskGroup group {}; + + for (std::size_t index {}; index < clientCount; ++index) { + group.spawn(run_echo_client( + context, + returnTo, + port, + index, + observations[index])); + } + + co_await group.join(); + + LoadCounts counts {}; + + for (const auto& observation : observations) { + counts.completions_ += observation.completions_; + counts.invalidCompletions_ += observation.completions_ != 1; + counts.wrongThread_ += observation.resumedThread_ != expectedThread; + + switch (observation.outcome_) { + case LoadOutcome::succeeded: + ++counts.succeeded_; + break; + case LoadOutcome::cancelled: + ++counts.cancelled_; + break; + case LoadOutcome::error: + ++counts.errors_; + break; + } + } + + co_return counts; +} + TEST(CmpTcpTest, IoContextStartsAndStopsCleanly) { IoContext context {}; } @@ -1529,4 +1668,60 @@ TEST(CmpTcpTest, ContextShutdownDrainsPendingReadAndClosesSurvivor) { EXPECT_NO_THROW(stream.close()); } +TEST(CmpTcpTest, ManyConcurrentClientsCompleteExactlyOnce) { + constexpr std::size_t CLIENT_COUNT { 32 }; + constexpr std::size_t PAYLOAD_SIZE { 4 }; + std::atomic serverCompletions {}; + std::atomic serverErrors {}; + LoopbackServer server { + asio::ip::address_v4::loopback(), + [&](asio::ip::tcp::socket& socket) { + std::array buffer {}; + std::error_code error {}; + const auto received = asio::read( + socket, + asio::buffer(buffer), + error); + std::size_t written {}; + + if (!error && received == buffer.size()) { + written = asio::write( + socket, + asio::buffer(buffer), + error); + } + + if (error || + received != buffer.size() || + written != buffer.size()) { + serverErrors.fetch_add(1, std::memory_order_relaxed); + } + + serverCompletions.fetch_add(1, std::memory_order_release); + }, + CLIENT_COUNT + }; + ASSERT_TRUE(server.available()) << server.start_error().message(); + ASSERT_TRUE(server.wait_until_ready()); + + IoContext context {}; + RunLoop loop {}; + const auto counts = loop.run(run_concurrent_echo_clients( + context, + loop.get_scheduler(), + server.port(), + CLIENT_COUNT)); + + EXPECT_EQ(counts.completions_, static_cast(CLIENT_COUNT)); + EXPECT_EQ(counts.succeeded_, static_cast(CLIENT_COUNT)); + EXPECT_EQ(counts.cancelled_, 0); + EXPECT_EQ(counts.errors_, 0); + EXPECT_EQ(counts.invalidCompletions_, 0); + EXPECT_EQ(counts.wrongThread_, 0); + EXPECT_EQ( + serverCompletions.load(std::memory_order_acquire), + static_cast(CLIENT_COUNT)); + EXPECT_EQ(serverErrors.load(std::memory_order_relaxed), 0); +} + } // namespace From bb18f52940b9c47d16af73f43fbfeccec1b47f3a Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 01:42:08 +0800 Subject: [PATCH 10/17] =?UTF-8?q?=E8=BF=81=E7=A7=BB=E5=8E=8B=E6=B5=8B=20TC?= =?UTF-8?q?P=20=E5=BC=82=E6=AD=A5=E5=AE=A2=E6=88=B7=E7=AB=AF?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 32 +-- benchmarks/v1-readiness/src/main.cpp | 216 +++++++++++------- .../benchmarks/2026-08-29-cmp-v1-readiness.md | 69 +++--- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 23 +- 4 files changed, 209 insertions(+), 131 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 1c440bc..db99252 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -15,8 +15,8 @@ path-dependency consumer。 ## 当前目标与状态 Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完成,最新为 Step 5 -`a73c113`,Step 6 已提交为 `95ff43e`,均尚未推送。Phase 6B Plan 第 7 项并发负载与本地 -跨平台门禁已实现并通过;下一步是第 8 项 readiness TCP client 迁移。 +`a73c113`,Step 6/7 已提交为 `95ff43e`/`eeb0275`,均尚未推送。Phase 6B Plan 第 8 项 +readiness TCP client 迁移已实现并通过五轮本地压测;下一步是第 9 项开发文档同步。 ## 已完成工作 @@ -30,8 +30,9 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 move-only、异常、预取消/排队取消/晚取消、取消竞态、过期与 inactive Scheduler、RunLoop 响应性,以及 5,000 个并发 offload 的 exactly-once 行为。 - `examples/basic` 新增独立 blocking ThreadPool,并在协程中打印 `Blocking result: 42`。 -- `benchmarks/v1-readiness` 的文件与回环网络场景已从手写 `jthread` adapter 迁移到一个专用 - CMP ThreadPool 和 `run_blocking()`;负载及成功/失败硬检查保持不变。 +- `benchmarks/v1-readiness` 的文件场景及同步回环 server 使用一个专用 CMP ThreadPool 和 + `run_blocking()`;网络 client 已迁移到共享 `IoContext` 与 `TcpStream`,负载及成功/失败 + 硬检查保持不变。 - 三份 README、三份架构文档、Phase 6A Design/Plan 和 v1 readiness 数据报告已同步到实现 事实。 - 完成 Phase 6B 依赖核查:mcpp-index 当前提供 `chriskohlhoff.asio@1.38.1`,其 C++23 模块 @@ -95,6 +96,9 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 context drain/join 和 surviving handle 边界,当前 `tcp_test` 为 23 项。 - 现有 loopback fixture 支持固定连接数;新增 32 客户端并发 echo 测试,每个 Task 独立持有 stream/缓冲区并逐槽核对完成次数、结果、payload 与 RunLoop 返回线程。 +- readiness 的四个 client 各保持一条长连接完成 5,000 次 256B echo,共享一个 I/O driver; + 四个同步 POSIX server 仍各占一个 blocking worker,100 次故障连接只接受 + `connection_refused`。 ## 重要决策 @@ -159,8 +163,8 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 - Dev 定向 `blocking_test`:11/11 通过。 - Release `CancellationRaceInvokesAtMostOnce` 连续执行 100 轮:100/100 通过。 - `examples/basic` 的 `mcpp run`:通过,包含 `Blocking result: 42`,退出码 0。 -- `benchmarks/v1-readiness` Release strict 构建通过;迁移后执行 5 轮,compute、file_io 和 - network_loopback 每轮均 PASS,五轮非预期失败总数为 0。 +- `benchmarks/v1-readiness` Phase 6B client 迁移后 Release strict cache-off 构建通过;最终 + 执行 5 轮,compute、file_io 和 network_loopback 每轮均 PASS,五轮非预期失败总数为 0。 - 文件/网络每轮计数分别为 1,000/20,000 成功、100/100 预期失败、0 非预期失败;原始耗时和 吞吐已写入 benchmark 报告。 - 当前 mcpp 仍输出 SubOS 缺少 `subos_info` 的既有环境提示,但所有构建和运行成功。 @@ -191,6 +195,8 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 500 次 stop/completion 竞态,没有丢失或重复完成。 - 加入并发负载后,`tcp_test` Dev/Release 均为 24/24;两套 strict cache-off 全量测试均为 10 个二进制、140/140。32/32 客户端成功,取消、错误、payload/线程不匹配和重复完成均为 0。 +- readiness 新网络路径每轮均为 20,000 成功、100 预期失败、0 非预期失败;五轮网络中位数 + 为 2,071.085 ms / 9,705.1 ops/s,原始结果已写入 benchmark 报告,耗时不作门槛。 ## 已知问题 / 风险 @@ -204,18 +210,18 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 - Phase 6B 本地已有 backend、`IoContext`、native bridge、connect/read/write、EOF/overlap、 - pending cancellation、close/is_open、shutdown 竞态与 32 客户端负载;readiness TCP client - 迁移、文档同步和最终本地矩阵仍待后续步骤。 + pending cancellation、close/is_open、shutdown 竞态、32 客户端负载及 readiness TCP client + 迁移;六份开发文档同步和最终本地矩阵仍待后续步骤。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 按 Phase 6B Plan 第 8 项只迁移 readiness 的 TCP client,保留同步 server、compute/file - 场景、常量、CSV 和硬计数。 -2. 继续第 9–10 项:同步六份开发文档并执行完整本地验证矩阵与五轮 benchmark。 +1. 按 Phase 6B Plan 第 9 项同步三份 README 与三份 architecture,准确说明 API、buffer + 生命周期、并发/取消/关闭语义和 Phase 6A/6B 边界。 +2. 继续第 10 项:执行完整本地验证矩阵、example、100 轮竞态门禁与五轮 benchmark。 3. 本地完成后仍需用户另行授权 push/PR,才能取得 Linux、macOS、Windows 远程 CI 结果。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 8 项,用共享 `IoContext`/`TcpStream` 替换 -readiness 的阻塞 TCP client;同步 loopback server 继续使用 `run_blocking()`。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 9 项,先从已验证的 public API/Design +提炼一份英文内容,再等义同步简中与繁中版本,不修改 example 的确定性行为。 diff --git a/benchmarks/v1-readiness/src/main.cpp b/benchmarks/v1-readiness/src/main.cpp index 15ea300..93ebb74 100644 --- a/benchmarks/v1-readiness/src/main.cpp +++ b/benchmarks/v1-readiness/src/main.cpp @@ -14,8 +14,10 @@ import mcpplibs.cmp; namespace { using mcpplibs::cmp::RunLoop; +using mcpplibs::cmp::IoContext; using mcpplibs::cmp::Task; using mcpplibs::cmp::TaskGroup; +using mcpplibs::cmp::TcpStream; using mcpplibs::cmp::ThreadPool; using mcpplibs::cmp::run_blocking; @@ -140,48 +142,31 @@ class TemporaryDirectory final { return Socket { handle }; } -[[nodiscard]] std::pair make_loopback_pair() { - auto listener = make_socket(); +[[nodiscard]] std::pair bind_loopback_socket() { + auto socket = make_socket(); sockaddr_in address {}; address.sin_family = AF_INET; address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); if (::bind( - listener.get(), + socket.get(), reinterpret_cast(&address), sizeof(address)) != 0) { throw_socket_error("bind"); } - if (::listen(listener.get(), 1) != 0) { - throw_socket_error("listen"); - } - socklen_t addressLength { sizeof(address) }; if (::getsockname( - listener.get(), + socket.get(), reinterpret_cast(&address), &addressLength) != 0) { throw_socket_error("getsockname"); } - auto client = make_socket(); - if (::connect( - client.get(), - reinterpret_cast(&address), - sizeof(address)) != 0) { - throw_socket_error("connect"); - } - - const int accepted = ::accept(listener.get(), nullptr, nullptr); - if (accepted < 0) { - throw_socket_error("accept"); - } - - return { std::move(client), Socket { accepted } }; + return { std::move(socket), ntohs(address.sin_port) }; } -void send_all(int socket, std::string_view data) { +void send_all(int socket, std::span data) { std::size_t offset { 0 }; while (offset < data.size()) { @@ -197,51 +182,43 @@ void send_all(int socket, std::string_view data) { } } -[[nodiscard]] std::string receive_exactly(int socket, std::size_t size) { - std::string data(size, '\0'); +void receive_exactly(int socket, std::span data) { std::size_t offset { 0 }; - while (offset < size) { + while (offset < data.size()) { const auto received = ::recv( socket, data.data() + offset, - size - offset, + data.size() - offset, 0); if (received <= 0) { throw_socket_error("recv"); } offset += static_cast(received); } - - return data; } -[[nodiscard]] bool non_listening_loopback_connect_is_refused() { - auto reservation = make_socket(); - sockaddr_in address {}; - address.sin_family = AF_INET; - address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); +void run_echo_server( + int listener, + const std::array& payload) { + const int acceptedHandle = ::accept(listener, nullptr, nullptr); - if (::bind( - reservation.get(), - reinterpret_cast(&address), - sizeof(address)) != 0) { - throw_socket_error("failure bind"); + if (acceptedHandle < 0) { + throw_socket_error("accept"); } - socklen_t addressLength { sizeof(address) }; - if (::getsockname( - reservation.get(), - reinterpret_cast(&address), - &addressLength) != 0) { - throw_socket_error("failure getsockname"); + Socket accepted { acceptedHandle }; + std::array request {}; + + for (int round { 0 }; round < NETWORK_ROUNDS; ++round) { + receive_exactly(accepted.get(), request); + + if (!std::ranges::equal(request, payload)) { + throw std::runtime_error { "server payload mismatch" }; + } + + send_all(accepted.get(), payload); } - auto probe = make_socket(); - const int result = ::connect( - probe.get(), - reinterpret_cast(&address), - sizeof(address)); - return result != 0 && errno == ECONNREFUSED; } Task failing_compute(Scheduler scheduler) { @@ -349,48 +326,117 @@ Task run_file_io( co_await group.join(); } -Task run_network_io( +Task run_network_worker( + IoContext& ioContext, BlockingScheduler blockingWorkers, Scheduler scheduler, + std::array payload, Counters& counters) { - const std::string payload(NETWORK_PAYLOAD_SIZE, 'n'); - TaskGroup group {}; + auto [listener, port] = bind_loopback_socket(); - for (int workerIndex { 0 }; workerIndex < NETWORK_WORKERS; ++workerIndex) { - static_cast(workerIndex); - group.spawn(run_blocking(blockingWorkers, scheduler, [&] { - int completed { 0 }; - try { - auto [client, server] = make_loopback_pair(); - for (; completed < NETWORK_ROUNDS; ++completed) { - send_all(client.get(), payload); - if (receive_exactly(server.get(), payload.size()) != payload) { - throw std::runtime_error { "server payload mismatch" }; - } + if (::listen(listener.get(), 1) != 0) { + throw_socket_error("listen"); + } - send_all(server.get(), payload); - if (receive_exactly(client.get(), payload.size()) != payload) { - throw std::runtime_error { "client payload mismatch" }; - } - counters.successes_.fetch_add(1); + // 服务端继续在线程池阻塞;客户端由 IoContext 原生异步驱动。 + TaskGroup serverGroup {}; + serverGroup.spawn(run_blocking( + blockingWorkers, + scheduler, + [listenerHandle = listener.get(), payload] { + run_echo_server(listenerHandle, payload); + })); + + int completed {}; + + try { + auto stream = co_await TcpStream::connect( + ioContext, + scheduler, + "127.0.0.1", + port); + std::array response {}; + + for (; completed < NETWORK_ROUNDS; ++completed) { + co_await stream.write_all(scheduler, payload); + std::size_t received {}; + + while (received < response.size()) { + const auto size = co_await stream.read_some( + scheduler, + std::span { response }.subspan(received)); + + if (size == 0) { + throw std::runtime_error { "unexpected server EOF" }; } - } catch (...) { - counters.unexpectedFailures_.fetch_add( - NETWORK_ROUNDS - completed); + + received += size; } - for (int failure { 0 }; failure < NETWORK_FAILURES; ++failure) { - try { - if (non_listening_loopback_connect_is_refused()) { - counters.expectedFailures_.fetch_add(1); - } else { - counters.unexpectedFailures_.fetch_add(1); - } - } catch (...) { - counters.unexpectedFailures_.fetch_add(1); - } + if (response != payload) { + throw std::runtime_error { "client payload mismatch" }; } - })); + + counters.successes_.fetch_add(1); + } + } catch (...) { + counters.unexpectedFailures_.fetch_add( + NETWORK_ROUNDS - completed); + static_cast(::shutdown(listener.get(), SHUT_RDWR)); + } + + bool serverFailed {}; + + try { + co_await serverGroup.join(); + } catch (...) { + serverFailed = true; + } + + if (serverFailed && completed == NETWORK_ROUNDS) { + counters.successes_.fetch_sub(1); + counters.unexpectedFailures_.fetch_add(1); + } + + auto [reservation, failurePort] = bind_loopback_socket(); + + for (int failure { 0 }; failure < NETWORK_FAILURES; ++failure) { + try { + static_cast(co_await TcpStream::connect( + ioContext, + scheduler, + "127.0.0.1", + failurePort)); + counters.unexpectedFailures_.fetch_add(1); + } catch (const std::system_error& error) { + if (error.code() == std::make_error_condition( + std::errc::connection_refused)) { + counters.expectedFailures_.fetch_add(1); + } else { + counters.unexpectedFailures_.fetch_add(1); + } + } catch (...) { + counters.unexpectedFailures_.fetch_add(1); + } + } +} + +Task run_network_io( + IoContext& ioContext, + BlockingScheduler blockingWorkers, + Scheduler scheduler, + Counters& counters) { + std::array payload {}; + payload.fill(std::byte { static_cast('n') }); + TaskGroup group {}; + + for (int remaining { NETWORK_WORKERS }; remaining > 0; --remaining) { + group.spawn(run_network_worker( + ioContext, + blockingWorkers, + scheduler, + payload, + counters)); } co_await group.join(); @@ -444,6 +490,7 @@ void print_metrics(const Metrics& metrics) { int main() { TemporaryDirectory directory {}; + IoContext ioContext {}; ThreadPool blockingWorkers { std::max(FILE_WORKERS, NETWORK_WORKERS) }; @@ -473,6 +520,7 @@ int main() { NETWORK_WORKERS * (NETWORK_ROUNDS + NETWORK_FAILURES), [&](Scheduler scheduler, Counters& counters) { return run_network_io( + ioContext, blockingWorkers.get_scheduler(), scheduler, counters); diff --git a/docs/benchmarks/2026-08-29-cmp-v1-readiness.md b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md index aae2e4f..a033724 100644 --- a/docs/benchmarks/2026-08-29-cmp-v1-readiness.md +++ b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md @@ -2,17 +2,18 @@ **日期:** 2026-08-29 -**结论:** 本机 5 轮 Release 压测全部通过;三类场景均无非预期失败。 +**结论:** Phase 6B TCP client 迁移后的本机 5 轮 Release 压测全部通过;三类场景均无 +非预期失败。 ## 测试边界 -本报告验证 CMP v1 的结构化生命周期、调度、跨线程唤醒和失败传播能够承载计算任务, -以及通过专用 CMP `ThreadPool` 与 `run_blocking()` 隔离的文件和网络 I/O。CMP v1 尚无原生 -异步 I/O 后端,因此 I/O 数据不是 epoll、io_uring、kqueue 或 IOCP 性能数据,也不代表 -内核异步 I/O 吞吐。 +本报告验证 CMP v1 的结构化生命周期、调度、跨线程唤醒和失败传播能够承载计算任务;文件 +和 TCP loopback server 继续通过专用 CMP `ThreadPool` 与 `run_blocking()` 隔离,TCP client +则使用 Phase 6B `IoContext`/`TcpStream` 原生异步路径。 -压测程序位于 `benchmarks/v1-readiness`,只使用标准库、CMP 和 POSIX 回环 socket;它不增加 -项目依赖,也不进入跨平台 CI。任何计数不匹配或非预期失败都会让进程以非零状态退出。 +压测程序位于 `benchmarks/v1-readiness`,只直接使用标准库、CMP 和服务端所需的 POSIX 回环 +socket;Asio 由 CMP 路径依赖传递,不在 benchmark 重复声明。该 benchmark 不进入跨平台 CI。 +任何计数不匹配或非预期失败都会让进程以非零状态退出。 ## 环境与复现 @@ -23,12 +24,13 @@ - mcpp:2026.8.11.2; - 编译器:LLVM 22.1.8; - 阻塞 worker:4; +- 原生异步 I/O driver:1; - 配置:Release,关闭 mcpp 构建缓存。 ```bash cd benchmarks/v1-readiness mcpp build --profile release --strict --cache=off -./target/x86_64-linux-gnu/f0cc001df3795148/bin/cmp-v1-readiness +./target/x86_64-linux-gnu/937ff8fc1b673bcb/bin/cmp-v1-readiness ``` 目标目录中的平台哈希可能随环境变化;如果路径不同,使用构建输出给出的可执行文件路径。 @@ -39,9 +41,9 @@ mcpp build --profile release --strict --cache=off 任务注入一个预期异常并在结构化子任务内捕获。 - `file_io`:4 个结构化 offload,各在专用 ThreadPool worker 上执行 250 次 64 KiB 临时文件 写入、读取和内容校验,再执行 25 次缺失文件读取以验证预期失败路径。 -- `network_loopback`:4 个结构化 offload,各在专用 ThreadPool worker 上执行 5,000 次 - 256 字节 TCP 回环请求/响应,再执行 25 次未监听端口连接,并且只把 `ECONNREFUSED` 计为 - 预期失败。 +- `network_loopback`:4 个同步 echo server 各占用一个专用 ThreadPool worker;4 个 + `TcpStream` 共用一个 `IoContext`,各执行 5,000 次 256 字节回环请求/响应,再对被占用但未 + 监听的端口执行 25 次连接,并且只把 `connection_refused` 计为预期失败。 吞吐量按“成功操作 + 预期失败操作”的总操作数计算。 @@ -49,30 +51,31 @@ mcpp build --profile release --strict --cache=off | 轮次 | 场景 | 操作数 | 成功 | 预期失败 | 非预期失败 | 耗时 ms | ops/s | 状态 | | ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | -| 1 | compute | 50,000 | 49,000 | 1,000 | 0 | 33.907 | 1,474,617.9 | PASS | -| 1 | file_io | 1,100 | 1,000 | 100 | 0 | 217.009 | 5,068.9 | PASS | -| 1 | network_loopback | 20,100 | 20,000 | 100 | 0 | 4,035.640 | 4,980.6 | PASS | -| 2 | compute | 50,000 | 49,000 | 1,000 | 0 | 64.119 | 779,795.3 | PASS | -| 2 | file_io | 1,100 | 1,000 | 100 | 0 | 421.216 | 2,611.5 | PASS | -| 2 | network_loopback | 20,100 | 20,000 | 100 | 0 | 3,210.152 | 6,261.4 | PASS | -| 3 | compute | 50,000 | 49,000 | 1,000 | 0 | 36.928 | 1,353,998.2 | PASS | -| 3 | file_io | 1,100 | 1,000 | 100 | 0 | 206.260 | 5,333.1 | PASS | -| 3 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,543.529 | 7,902.4 | PASS | -| 4 | compute | 50,000 | 49,000 | 1,000 | 0 | 28.541 | 1,751,852.5 | PASS | -| 4 | file_io | 1,100 | 1,000 | 100 | 0 | 423.989 | 2,594.4 | PASS | -| 4 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,627.611 | 7,649.5 | PASS | -| 5 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.133 | 1,842,742.1 | PASS | -| 5 | file_io | 1,100 | 1,000 | 100 | 0 | 168.893 | 6,513.0 | PASS | -| 5 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,492.924 | 8,062.8 | PASS | +| 1 | compute | 50,000 | 49,000 | 1,000 | 0 | 32.879 | 1,520,728.9 | PASS | +| 1 | file_io | 1,100 | 1,000 | 100 | 0 | 176.548 | 6,230.6 | PASS | +| 1 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,016.219 | 9,969.2 | PASS | +| 2 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.174 | 1,839,973.8 | PASS | +| 2 | file_io | 1,100 | 1,000 | 100 | 0 | 167.446 | 6,569.3 | PASS | +| 2 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,071.085 | 9,705.1 | PASS | +| 3 | compute | 50,000 | 49,000 | 1,000 | 0 | 29.339 | 1,704,210.2 | PASS | +| 3 | file_io | 1,100 | 1,000 | 100 | 0 | 189.751 | 5,797.1 | PASS | +| 3 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,026.998 | 9,916.1 | PASS | +| 4 | compute | 50,000 | 49,000 | 1,000 | 0 | 28.291 | 1,767,337.2 | PASS | +| 4 | file_io | 1,100 | 1,000 | 100 | 0 | 970.697 | 1,133.2 | PASS | +| 4 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,353.919 | 8,538.9 | PASS | +| 5 | compute | 50,000 | 49,000 | 1,000 | 0 | 30.551 | 1,636,609.6 | PASS | +| 5 | file_io | 1,100 | 1,000 | 100 | 0 | 457.299 | 2,405.4 | PASS | +| 5 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,503.267 | 8,029.5 | PASS | ## 汇总 | 场景 | 最短 / 中位 / 最长耗时 ms | 最高 / 中位 / 最低 ops/s | 五轮非预期失败 | | --- | ---: | ---: | ---: | -| compute | 27.133 / 33.907 / 64.119 | 1,842,742.1 / 1,474,617.9 / 779,795.3 | 0 | -| file_io | 168.893 / 217.009 / 423.989 | 6,513.0 / 5,068.9 / 2,594.4 | 0 | -| network_loopback | 2,492.924 / 2,627.611 / 4,035.640 | 8,062.8 / 7,649.5 / 4,980.6 | 0 | - -这些结果说明 v1 核心可以用于有明确所有权的协程编排、定时、显式取消、可复用通知,以及 -通过独立 ThreadPool 实例和 `run_blocking()` 接入本地文件与网络工作。它们是当前机器上的 -回归基线,不是跨机器 SLA。原生异步 I/O 和 work stealing 仍属于后续阶段。 +| compute | 27.174 / 29.339 / 32.879 | 1,839,973.8 / 1,704,210.2 / 1,520,728.9 | 0 | +| file_io | 167.446 / 189.751 / 970.697 | 6,569.3 / 5,797.1 / 1,133.2 | 0 | +| network_loopback | 2,016.219 / 2,071.085 / 2,503.267 | 9,969.2 / 9,705.1 / 8,029.5 | 0 | + +相同负载的 Phase 6A 阻塞 client 历史中位数为 2,627.611 ms / 7,649.5 ops/s;Phase 6B 原生 +异步 client 的本轮中位数为 2,071.085 ms / 9,705.1 ops/s。该差异只作为本机回归证据,不是 +跨机器 SLA 或 CI 阈值。结果说明 v1 核心可以同时承载结构化协程、blocking offload 和 +单 driver 原生异步 TCP;work stealing 仍需由独立的代表性负载证明必要性。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index 8655ed5..d54e0be 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — load and local cross-platform gate complete +**Status:** In progress — readiness TCP client migration complete **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -362,6 +362,27 @@ Build the benchmark in Release strict mode and run five measured rounds. Record `docs/benchmarks/2026-08-29-cmp-v1-readiness.md`, explicitly labeling the server as blocking and the client as Phase 6B native async TCP. +**Self-review — 2026-08-30:** Preserve the current workload shape: each of four workers creates one +ephemeral listener, runs one synchronous accept plus a 5,000-round echo loop through +`run_blocking()`, and uses one long-lived `TcpStream` for those rounds. All four streams share one +benchmark `IoContext`; each write and read explicitly returns to the measuring RunLoop. A worker +owns its listener, server Task, stream, and 256-byte buffers until structured join. For the 25 +failure rounds, keep one bound-but-not-listening socket alive and accept only +`std::errc::connection_refused` as expected. If a blocking server fails after the client has counted +every round, convert one success to unexpected failure so hard accounting remains exact. Reuse the +existing POSIX RAII socket and byte send/receive loops; remove only pair creation and blocking +client connect/send/receive code. Do not alter compute/file workloads, constants, CSV, or add a +benchmark dependency. + +**Completion — 2026-08-30:** Replaced only the readiness benchmark's blocking client with four +`TcpStream` clients sharing one `IoContext`. Each worker keeps one long-lived connection for 5,000 +256-byte echoes; its synchronous POSIX accept/read/write server remains isolated through +`run_blocking()`. One bound non-listening endpoint per worker drives 25 exact +`connection_refused` outcomes. Compute/file workloads, constants, CSV, and hard accounting are +unchanged. Release strict cache-off build passed, followed by five final runs in which every +scenario passed; network was 20,000 successes, 100 expected failures, and zero unexpected failures +per round. Raw data and the Phase 6A median comparison are recorded in the readiness report. + ## 9. Synchronize the developer documentation After behavior is verified, update the three README and three architecture variants with: From aff7ba254bb2631254db947e0106e0166372b748 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 01:52:15 +0800 Subject: [PATCH 11/17] =?UTF-8?q?=E5=90=8C=E6=AD=A5=20TCP=20=E5=BC=80?= =?UTF-8?q?=E5=8F=91=E6=96=87=E6=A1=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 28 ++++----- README.md | 58 +++++++++++++++---- README.zh.hant.md | 53 +++++++++++++---- README.zh.md | 49 +++++++++++++--- docs/architecture.md | 51 +++++++++++----- docs/architecture.zh.hant.md | 50 +++++++++++----- docs/architecture.zh.md | 49 +++++++++++----- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 19 +++++- 8 files changed, 269 insertions(+), 88 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index db99252..eda3953 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -14,9 +14,9 @@ path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1 与 Phase 6B 第 1–5 项已作为本地提交完成,最新为 Step 5 -`a73c113`,Step 6/7 已提交为 `95ff43e`/`eeb0275`,均尚未推送。Phase 6B Plan 第 8 项 -readiness TCP client 迁移已实现并通过五轮本地压测;下一步是第 9 项开发文档同步。 +Phase 6A blocking offload v1 与 Phase 6B 第 1–8 项均已作为本地提交完成且尚未推送。Phase 6B +Plan 第 9 项已同步六份开发文档,并通过未改动的独立示例验证;下一步是第 10 项完整本地门禁 +与状态收口。 ## 已完成工作 @@ -33,8 +33,8 @@ readiness TCP client 迁移已实现并通过五轮本地压测;下一步是 - `benchmarks/v1-readiness` 的文件场景及同步回环 server 使用一个专用 CMP ThreadPool 和 `run_blocking()`;网络 client 已迁移到共享 `IoContext` 与 `TcpStream`,负载及成功/失败 硬检查保持不变。 -- 三份 README、三份架构文档、Phase 6A Design/Plan 和 v1 readiness 数据报告已同步到实现 - 事实。 +- 三份 README 与三份架构文档已同步 `IoContext`/`TcpStream` API、生命周期、并发/取消边界、 + 10 个测试二进制/140 项测试及 readiness 结果;Phase 6A Design/Plan 和数据报告也与实现一致。 - 完成 Phase 6B 依赖核查:mcpp-index 当前提供 `chriskohlhoff.asio@1.38.1`,其 C++23 模块 `asio` 覆盖 Linux、macOS 和 Windows,能够复用 epoll、kqueue 与 IOCP 后端。 - 新增并批准 Phase 6B TCP client v1 Design,确定首个原生异步 I/O 切片仅包含数值地址 TCP @@ -93,7 +93,7 @@ readiness TCP client 迁移已实现并通过五轮本地压测;下一步是 - 显式关闭请求使用原子标记覆盖公开校验到 native initiation 的窄窗口;已接纳操作在 close 获胜时稳定得到 `OperationCancelled`,而无关 socket error 仍保留 `system_error`。 - TCP 测试新增 active/pre-cancellation、关闭幂等、move、write cancellation、两类完成竞态、 - context drain/join 和 surviving handle 边界,当前 `tcp_test` 为 23 项。 + context drain/join 和 surviving handle 边界,该阶段使 `tcp_test` 增至 23 项。 - 现有 loopback fixture 支持固定连接数;新增 32 客户端并发 echo 测试,每个 Task 独立持有 stream/缓冲区并逐槽核对完成次数、结果、payload 与 RunLoop 返回线程。 - readiness 的四个 client 各保持一条长连接完成 5,000 次 256B echo,共享一个 I/O driver; @@ -163,6 +163,8 @@ readiness TCP client 迁移已实现并通过五轮本地压测;下一步是 - Dev 定向 `blocking_test`:11/11 通过。 - Release `CancellationRaceInvokesAtMostOnce` 连续执行 100 轮:100/100 通过。 - `examples/basic` 的 `mcpp run`:通过,包含 `Blocking result: 42`,退出码 0。 +- Step 9 文档同步后再次运行未改动的 `examples/basic`:构建并运行通过,全部预期输出一致, + 退出码 0。 - `benchmarks/v1-readiness` Phase 6B client 迁移后 Release strict cache-off 构建通过;最终 执行 5 轮,compute、file_io 和 network_loopback 每轮均 PASS,五轮非预期失败总数为 0。 - 文件/网络每轮计数分别为 1,000/20,000 成功、100/100 预期失败、0 非预期失败;原始耗时和 @@ -210,18 +212,16 @@ readiness TCP client 迁移已实现并通过五轮本地压测;下一步是 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 - Phase 6B 本地已有 backend、`IoContext`、native bridge、connect/read/write、EOF/overlap、 - pending cancellation、close/is_open、shutdown 竞态、32 客户端负载及 readiness TCP client - 迁移;六份开发文档同步和最终本地矩阵仍待后续步骤。 + pending cancellation、close/is_open、shutdown 竞态、32 客户端负载、readiness TCP client + 迁移及六份开发文档;最终本地矩阵仍待后续步骤。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 -1. 按 Phase 6B Plan 第 9 项同步三份 README 与三份 architecture,准确说明 API、buffer - 生命周期、并发/取消/关闭语义和 Phase 6A/6B 边界。 -2. 继续第 10 项:执行完整本地验证矩阵、example、100 轮竞态门禁与五轮 benchmark。 -3. 本地完成后仍需用户另行授权 push/PR,才能取得 Linux、macOS、Windows 远程 CI 结果。 +1. 按 Phase 6B Plan 第 10 项执行完整本地验证矩阵、example、100 轮竞态门禁与五轮 benchmark。 +2. 本地完成后仍需用户另行授权 push/PR,才能取得 Linux、macOS、Windows 远程 CI 结果。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 9 项,先从已验证的 public API/Design -提炼一份英文内容,再等义同步简中与繁中版本,不修改 example 的确定性行为。 +按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 10 项,先执行 focused Release TCP 竞态 +门禁,再执行 Dev/Release 全量矩阵、独立示例与五轮 readiness benchmark,准确记录全部结果。 diff --git a/README.md b/README.md index 69fdc2b..7ad10cd 100644 --- a/README.md +++ b/README.md @@ -21,17 +21,19 @@ > CMP provides a lazy, single-consumer `Task` / `Task`, structured variadic and vector > `when_all()`, an eager structured `TaskGroup`, one-shot and reusable events, an RAII `AsyncMutex`, > a caller-thread `RunLoop` with explicit and monotonic timed scheduling, and a fixed-size CPU -> `ThreadPool`. `run_blocking()` executes an owned synchronous callable on a dedicated pool instance +> `ThreadPool`. Phase 6B adds an explicit `IoContext` and move-only `TcpStream` for native async +> numeric-address TCP clients. `run_blocking()` executes an owned synchronous callable on a +> dedicated pool instance > and delivers its outcome through an explicit return Scheduler. Ready scheduling on either > executor, timed waits, reusable-event waits, TaskGroup children, and queued blocking offloads can -> use explicit cooperative cancellation with `std::stop_token`; native asynchronous I/O and -> detached execution are not implemented. +> use explicit cooperative cancellation with `std::stop_token`; TCP operations use the same token +> model. Detached execution and other native I/O families are not implemented. CMP is being built as a modern coroutine runtime and library on standard stackless C++ coroutines. Its explicit `co_await` model now covers fixed and incremental structured concurrency, one-time event notification, caller-thread and multi-worker scheduling, monotonic timers, and -cancellable waits, plus structured isolation of blocking work. Native asynchronous I/O remains a -separate, incremental design step. +cancellable waits, structured isolation of blocking work, and one portable native async TCP client +slice. ## Why CMP? @@ -56,11 +58,12 @@ promise that: - a task is automatically equivalent to a Go goroutine; - an arbitrary blocking call becomes non-blocking; - coroutine switching is safe directly inside a signal handler; -- task migration is implicit, work stealing is already enabled, or arbitrary async I/O exists. +- task migration is implicit, work stealing is already enabled, or every I/O family is async. Those capabilities must be designed and verified individually. CMP now uses an explicitly -dedicated `ThreadPool` instance with `run_blocking()` for synchronous work; native async I/O awaiters -and cooperative safe points remain separate work. +dedicated `ThreadPool` instance with `run_blocking()` for synchronous work. Phase 6B provides native +async TCP clients; DNS, listening sockets, TLS, file I/O, and cooperative safe points remain +separate work. ## Quick Start @@ -99,6 +102,8 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::IoContext; +using mcpplibs::cmp::TcpStream; using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; @@ -322,6 +327,34 @@ request can skip work that has not been claimed; it cannot preempt a running syn the return schedule is intentionally not cancellable. This is thread-based isolation, not native non-blocking I/O. +The native TCP surface stays explicit and small: + +```cpp +Task exchange( + IoContext& io, + RunLoop::Scheduler caller, + std::string_view numericAddress, + std::uint16_t port, + std::span request, + std::span reply, + std::stop_token token = {}) { + auto stream = co_await TcpStream::connect( + io, caller, numericAddress, port, token); + co_await stream.write_all(caller, request, token); + co_return co_await stream.read_some(caller, reply, token); +} +``` + +Share one immovable `IoContext` across streams; it owns one private I/O driver. `TcpStream` is +move-only, and connect accepts numeric IPv4/IPv6 text only—no blocking DNS is hidden inside it. +Read/write spans borrow their storage until the returned Task completes. Every success or error +attempts the explicit return-Scheduler hop. One read and one write may coexist; another operation +in the same direction throws `std::logic_error`. Pre-cancellation leaves an open stream usable, +while cancellation after `write_all()` starts closes it. `close()` is thread-safe, idempotent, and +causes accepted operations to complete once with `OperationCancelled` when close wins. Phase 6A +isolates arbitrary synchronous calls on ThreadPool workers; Phase 6B is native async TCP, not a +generic async-I/O layer. + RunLoop is not a background thread and does not make blocking code asynchronous. A Task that suspends without arranging a future resume can leave `run()` waiting indefinitely. CMP does not provide automatic thread affinity: after an external awaiter resumes on another thread, explicitly @@ -332,13 +365,14 @@ await the desired Scheduler to return to its RunLoop. ```text . ├── .xlings.json # pinned project tool environment -├── mcpp.toml # package identity and test dependency +├── mcpp.toml # package identity, runtime, and test dependencies ├── src/cmp.cppm # root module interface ├── src/task.cppm # Task module partition ├── src/cancellation.cppm # shared cooperative-cancellation exception ├── src/run_loop.cppm # RunLoop and Scheduler partition ├── src/thread_pool.cppm # fixed-size CPU worker scheduler ├── src/blocking.cppm # structured blocking-call offload +├── src/tcp.cppm # native async TCP client and I/O context ├── src/when_all.cppm # structured concurrent Task join ├── src/task_group.cppm # eager mutable structured Task scope ├── src/one_shot_event.cppm # allocation-free one-time notification @@ -348,6 +382,7 @@ await the desired Scheduler to return to its RunLoop. ├── tests/run_loop_test.cpp # scheduler, boundary, and threading tests ├── tests/thread_pool_test.cpp # worker, cancellation, and shutdown tests ├── tests/blocking_test.cpp # blocking offload, affinity, and cancellation tests +├── tests/tcp_test.cpp # TCP lifecycle, cancellation, races, and load tests ├── tests/when_all_test.cpp # join ownership, result, and race tests ├── tests/task_group_test.cpp # mutable scope lifetime and race tests ├── tests/one_shot_event_test.cpp # event publication and race tests @@ -381,7 +416,7 @@ global mcpp installation. CMP does not track `mcpp.lock`; `.gitignore` enforces that repository policy. Runtime dependencies belong in `[dependencies]`; gtest is declared explicitly under `[dev-dependencies.compat]`. -The current local suite contains 116 tests across nine binaries. The POSIX-only Release pressure +The current local suite contains 140 tests across ten binaries. The POSIX-only Release pressure consumer and its recorded success/failure data are documented in the [v1 readiness benchmark](docs/benchmarks/2026-08-29-cmp-v1-readiness.md). Multi-worker correctness and performance data are documented in the @@ -398,7 +433,8 @@ Runtime work is split into independently reviewable phases: OneShotEvent, AsyncManualResetEvent, and AsyncMutex — implemented and pressure-tested; 5. fixed-size multi-worker scheduling — implemented and benchmarked; work stealing remains gated by profiling evidence; -6. structured blocking offload — implemented; native asynchronous I/O remains separately gated. +6. structured blocking offload — implemented; native async numeric-address TCP client — locally + implemented and verified, with remote three-platform CI still pending. The remaining order is directional, not a promise that a listed feature is already implemented. diff --git a/README.zh.hant.md b/README.zh.hant.md index 68c317e..452dbd2 100644 --- a/README.zh.hant.md +++ b/README.zh.hant.md @@ -21,14 +21,15 @@ > CMP 已提供延遲啟動、單一消費者的 `Task` / `Task`、支援變參和 vector 的結構化 > `when_all()`、eager 結構化 `TaskGroup`、一次性與可複用事件、RAII `AsyncMutex`,以及在呼叫 > 執行緒運行、支援明確排程和單調時鐘定時排程的 `RunLoop`,以及固定大小的 CPU -> `ThreadPool`。`run_blocking()` 可在專用的 pool 實例上執行同步 callable,並透過明確的 -> 返回 Scheduler 交付結果。兩種執行器的就緒排程、定時等待、可複用事件等待、TaskGroup -> 子任務和排隊中的阻塞 offload 可明確使用基於 `std::stop_token` 的協作式取消;原生非同步 -> I/O 和 detached 執行尚未實作。 +> `ThreadPool`。Phase 6B 新增明確的 `IoContext` 和 move-only `TcpStream`,用於原生非同步的 +> 數值位址 TCP client。`run_blocking()` 可在專用的 pool 實例上執行同步 callable,並透過 +> 明確的返回 Scheduler 交付結果。兩種執行器的就緒排程、定時等待、可複用事件等待、TaskGroup +> 子任務和排隊中的阻塞 offload 可明確使用基於 `std::stop_token` 的協作式取消;TCP 操作採用 +> 相同 token 模型。detached 執行和其他原生 I/O 類型尚未實作。 CMP 計畫以標準無堆疊 C++ 協程建構現代協程執行期與函式庫。明確的 `co_await` 模型現已 涵蓋固定與增量結構化並行、一次性事件通知、呼叫執行緒與多 worker 排程、單調時鐘計時器、 -可取消等待和阻塞工作的結構化隔離。原生非同步 I/O 仍是獨立的增量設計步驟。 +可取消等待和阻塞工作的結構化隔離,以及一個可攜式原生非同步 TCP client 切片。 ## 為什麼叫 CMP? @@ -51,10 +52,11 @@ C++ 標準協程是語言機制,不是完整執行期。因此 CMP 不會宣 - task 自動等同於 Go goroutine; - 任意阻塞呼叫會自動成為非阻塞呼叫; - 可以直接在訊號處理器中安全切換協程; -- task 會隱式遷移、work stealing 已啟用,或任意非同步 I/O 已經實作。 +- task 會隱式遷移、work stealing 已啟用,或所有 I/O 類型都已非同步化。 這些能力必須分別設計和驗證。CMP 目前透過明確的專用 `ThreadPool` 實例和 -`run_blocking()` 隔離同步工作;原生非同步 I/O awaiter 與協作式安全點仍需單獨設計。 +`run_blocking()` 隔離同步工作。Phase 6B 已提供原生非同步 TCP client;DNS、監聽 socket、 +TLS、檔案 I/O 和協作式安全點仍需分別設計。 ## 快速開始 @@ -94,6 +96,8 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::IoContext; +using mcpplibs::cmp::TcpStream; using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; @@ -304,6 +308,32 @@ callable,將它排程到指定 ThreadPool 恰好執行一次,並在透過 `r worker。停止要求可以跳過尚未被 worker 領取的工作,但不能搶佔已經執行的同步呼叫;返回排程 刻意不可取消。這是基於執行緒的隔離,不是原生非阻塞 I/O。 +原生 TCP 介面保持明確且精簡: + +```cpp +Task exchange( + IoContext& io, + RunLoop::Scheduler caller, + std::string_view numericAddress, + std::uint16_t port, + std::span request, + std::span reply, + std::stop_token token = {}) { + auto stream = co_await TcpStream::connect( + io, caller, numericAddress, port, token); + co_await stream.write_all(caller, request, token); + co_return co_await stream.read_some(caller, reply, token); +} +``` + +一個不可移動的 `IoContext` 可供多個 stream 共用,並擁有一個私有 I/O driver。`TcpStream` 只能 +移動;connect 只接受數值 IPv4/IPv6 文字,不會隱藏阻塞 DNS。read/write span 的底層儲存必須 +保持到返回的 Task 完成。每種成功或錯誤都會嘗試明確返回 Scheduler。一個 read 和一個 write +可以並存;同方向第二個操作拋出 `std::logic_error`。預取消不會關閉仍可用的 stream,而已發起 +的 `write_all()` 被取消後會關閉 stream。`close()` 執行緒安全、冪等;當 close 獲勝時,已接納 +操作恰好一次以 `OperationCancelled` 完成。Phase 6A 在執行緒池上隔離任意同步呼叫;Phase 6B +只提供原生非同步 TCP,不是通用非同步 I/O 層。 + RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步程式碼。如果 Task 暫停後沒有 安排未來的恢復動作,`run()` 可能一直等待。CMP 不提供隱式執行緒親和:外部 awaiter 在其他 執行緒恢復協程後,需要明確等待目標 Scheduler 才會返回對應 RunLoop。 @@ -313,13 +343,14 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ```text . ├── .xlings.json # 固定的專案工具環境 -├── mcpp.toml # 套件識別與測試相依 +├── mcpp.toml # 套件識別、執行期與測試相依 ├── src/cmp.cppm # 根模組介面 ├── src/task.cppm # Task 模組分割區 ├── src/cancellation.cppm # 共用的協作式取消例外 ├── src/run_loop.cppm # RunLoop 與 Scheduler 分割區 ├── src/thread_pool.cppm # 固定大小的 CPU worker 排程器 ├── src/blocking.cppm # 結構化阻塞呼叫 offload +├── src/tcp.cppm # 原生非同步 TCP client 與 I/O context ├── src/when_all.cppm # 結構化並行 Task 匯合 ├── src/task_group.cppm # eager 可變結構化 Task 作用域 ├── src/one_shot_event.cppm # 無分配一次性通知 @@ -329,6 +360,7 @@ RunLoop 不是背景執行緒,也不會把阻塞程式碼自動變成非同步 ├── tests/run_loop_test.cpp # 排程、邊界和執行緒測試 ├── tests/thread_pool_test.cpp # worker、取消和關閉測試 ├── tests/blocking_test.cpp # 阻塞 offload、執行緒親和和取消測試 +├── tests/tcp_test.cpp # TCP 生命週期、取消、競態和負載測試 ├── tests/when_all_test.cpp # 匯合所有權、結果和競態測試 ├── tests/task_group_test.cpp # 可變作用域生命週期和競態測試 ├── tests/one_shot_event_test.cpp # 事件發布和競態測試 @@ -360,7 +392,7 @@ CI 在 Linux、macOS 和 Windows 上執行等價的建構、測試與獨立範 CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定。執行期相依放在 `[dependencies]`,gtest 明確宣告在 `[dev-dependencies.compat]` 中。 -目前本機套件包含 9 個測試二進位檔、116 項測試。僅用於 POSIX 的 Release 壓測 consumer 及其 +目前本機套件包含 10 個測試二進位檔、140 項測試。僅用於 POSIX 的 Release 壓測 consumer 及其 成功/失敗資料記錄在 [v1 可開發性壓測](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 多 worker 正確性和效能資料記錄在[執行緒池壓測](docs/benchmarks/2026-08-29-cmp-thread-pool.md)。 @@ -374,7 +406,8 @@ CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定 4. 單調時鐘 Timer v1、可取消就緒/定時等待、變參/vector 匯合、靜止點 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已實作並完成壓力驗證; 5. 固定大小的多 worker 排程——已實作並完成壓測;work stealing 仍需 profiling 證據; -6. 結構化阻塞 offload——已實作;原生非同步 I/O 仍需單獨設計和驗證。 +6. 結構化阻塞 offload——已實作;數值位址原生非同步 TCP client——已完成本機實作與驗證, + 遠端三平台 CI 仍待確認。 剩餘順序只是方向,不代表列出的能力已經實作。 diff --git a/README.zh.md b/README.zh.md index 6e49277..66289b3 100644 --- a/README.zh.md +++ b/README.zh.md @@ -21,14 +21,15 @@ > CMP 已提供懒启动、单消费者的 `Task` / `Task`、支持变参和 vector 的结构化 > `when_all()`、eager 结构化 `TaskGroup`、一次性与可复用事件、RAII `AsyncMutex`,以及在调用 > 线程运行、支持显式调度和单调时钟定时调度的 `RunLoop`,以及固定大小的 CPU `ThreadPool`。 +> Phase 6B 新增显式 `IoContext` 和 move-only `TcpStream`,用于原生异步的数值地址 TCP client。 > `run_blocking()` 可在专用的 pool 实例上执行同步 callable,并通过显式的返回 Scheduler > 交付结果。两种执行器的就绪调度、定时等待、可复用事件等待、TaskGroup 子任务和排队中的 -> 阻塞 offload 可显式使用基于 `std::stop_token` 的协作式取消;原生异步 I/O 和 detached -> 执行尚未实现。 +> 阻塞 offload 可显式使用基于 `std::stop_token` 的协作式取消;TCP 操作采用同一 token 模型。 +> detached 执行和其他原生 I/O 类型尚未实现。 CMP 计划基于标准无栈 C++ 协程构建现代协程运行时和库。显式 `co_await` 模型现已覆盖固定与 增量结构化并发、一次性事件通知、调用线程与多 worker 调度、单调时钟定时器、可取消等待和 -阻塞工作的结构化隔离。原生异步 I/O 仍是独立的增量设计步骤。 +阻塞工作的结构化隔离,以及一个可移植的原生异步 TCP client 切片。 ## 为什么叫 CMP? @@ -51,10 +52,11 @@ C++ 标准协程是语言机制,不是完整运行时。因此 CMP 不会宣 - task 自动等同于 Go goroutine; - 任意阻塞调用会自动变成非阻塞调用; - 可以直接在信号处理器中安全切换协程; -- task 会隐式迁移、work stealing 已启用,或任意异步 I/O 已经实现。 +- task 会隐式迁移、work stealing 已启用,或所有 I/O 类型都已异步化。 这些能力必须分别设计和验证。CMP 目前通过显式的专用 `ThreadPool` 实例和 -`run_blocking()` 隔离同步工作;原生异步 I/O awaiter 与协作式安全点仍需单独设计。 +`run_blocking()` 隔离同步工作。Phase 6B 已提供原生异步 TCP client;DNS、监听 socket、TLS、 +文件 I/O 和协作式安全点仍需分别设计。 ## 快速开始 @@ -94,6 +96,8 @@ using mcpplibs::cmp::OperationCancelled; using mcpplibs::cmp::OneShotEvent; using mcpplibs::cmp::TaskGroup; using mcpplibs::cmp::ThreadPool; +using mcpplibs::cmp::IoContext; +using mcpplibs::cmp::TcpStream; using mcpplibs::cmp::run_blocking; using mcpplibs::cmp::when_all; @@ -304,6 +308,32 @@ callable,将它调度到指定 ThreadPool 恰好执行一次,并在通过 `r worker。停止请求可以跳过尚未被 worker 领取的工作,但不能抢占已经运行的同步调用;返回调度 刻意不可取消。这是基于线程的隔离,不是原生非阻塞 I/O。 +原生 TCP 接口保持显式且精简: + +```cpp +Task exchange( + IoContext& io, + RunLoop::Scheduler caller, + std::string_view numericAddress, + std::uint16_t port, + std::span request, + std::span reply, + std::stop_token token = {}) { + auto stream = co_await TcpStream::connect( + io, caller, numericAddress, port, token); + co_await stream.write_all(caller, request, token); + co_return co_await stream.read_some(caller, reply, token); +} +``` + +一个不可移动的 `IoContext` 可供多个 stream 共享,并拥有一个私有 I/O driver。`TcpStream` 只能 +移动;connect 只接受数值 IPv4/IPv6 文本,不会隐藏阻塞 DNS。read/write span 的底层存储必须 +保持到返回的 Task 完成。每种成功或错误都会尝试显式返回 Scheduler。一个 read 和一个 write +可以并存;同方向第二个操作抛出 `std::logic_error`。预取消不会关闭仍可用的 stream,而已发起 +的 `write_all()` 被取消后会关闭 stream。`close()` 线程安全、幂等;当 close 获胜时,已接纳 +操作恰好一次以 `OperationCancelled` 完成。Phase 6A 在线程池上隔离任意同步调用;Phase 6B +只提供原生异步 TCP,不是通用异步 I/O 层。 + RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。如果 Task 挂起后没有安排未来的 恢复动作,`run()` 可能一直等待。CMP 不提供隐式线程亲和:外部 awaiter 在其他线程恢复协程 后,需要显式等待目标 Scheduler 才会返回对应 RunLoop。 @@ -313,13 +343,14 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ```text . ├── .xlings.json # 固定的项目工具环境 -├── mcpp.toml # 包身份和测试依赖 +├── mcpp.toml # 包身份、运行时与测试依赖 ├── src/cmp.cppm # 根模块接口 ├── src/task.cppm # Task 模块分区 ├── src/cancellation.cppm # 共享的协作式取消异常 ├── src/run_loop.cppm # RunLoop 与 Scheduler 分区 ├── src/thread_pool.cppm # 固定大小的 CPU worker 调度器 ├── src/blocking.cppm # 结构化阻塞调用 offload +├── src/tcp.cppm # 原生异步 TCP client 与 I/O context ├── src/when_all.cppm # 结构化并发 Task 汇合 ├── src/task_group.cppm # eager 可变结构化 Task 作用域 ├── src/one_shot_event.cppm # 无分配一次性通知 @@ -329,6 +360,7 @@ RunLoop 不是后台线程,也不会把阻塞代码自动变成异步代码。 ├── tests/run_loop_test.cpp # 调度、边界和线程测试 ├── tests/thread_pool_test.cpp # worker、取消和关闭测试 ├── tests/blocking_test.cpp # 阻塞 offload、线程亲和和取消测试 +├── tests/tcp_test.cpp # TCP 生命周期、取消、竞态和负载测试 ├── tests/when_all_test.cpp # 汇合所有权、结果和竞态测试 ├── tests/task_group_test.cpp # 可变作用域生命周期和竞态测试 ├── tests/one_shot_event_test.cpp # 事件发布和竞态测试 @@ -360,7 +392,7 @@ CI 在 Linux、macOS 和 Windows 上执行等价的构建、测试和独立示 CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。运行时依赖放在 `[dependencies]`,gtest 明确声明在 `[dev-dependencies.compat]` 中。 -当前本地套件包含 9 个测试二进制、116 项测试。仅用于 POSIX 的 Release 压测 consumer 及其 +当前本地套件包含 10 个测试二进制、140 项测试。仅用于 POSIX 的 Release 压测 consumer 及其 成功/失败数据记录在 [v1 可开发性压测](docs/benchmarks/2026-08-29-cmp-v1-readiness.md)。 多 worker 正确性和性能数据记录在[线程池压测](docs/benchmarks/2026-08-29-cmp-thread-pool.md)。 @@ -374,7 +406,8 @@ CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。 4. 单调时钟 Timer v1、可取消就绪/定时等待、变参/vector 汇合、静止点 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已实现并完成压力验证; 5. 固定大小的多 worker 调度——已实现并完成压测;work stealing 仍需 profiling 证据; -6. 结构化阻塞 offload——已实现;原生异步 I/O 仍需单独设计和验证。 +6. 结构化阻塞 offload——已实现;数值地址原生异步 TCP client——已完成本地实现与验证, + 远程三平台 CI 仍待确认。 剩余顺序只是方向,不代表列出的能力已经实现。 diff --git a/docs/architecture.md b/docs/architecture.md index b3749bb..fdf8546 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -15,12 +15,15 @@ have `std::stop_token` overloads for cooperative cancellation; timed scheduling absolute `steady_clock` deadlines without a timer thread. `ThreadPool::Scheduler::schedule()` explicitly transfers a continuation to any fixed worker and has the same cancellation-winner rule. `run_blocking()` uses a caller-selected ThreadPool instance for synchronous work and publishes the -outcome only after an explicit return Scheduler is reached. +outcome only after an explicit return Scheduler is reached. Phase 6B adds an immovable `IoContext` +with one private driver and a move-only `TcpStream` for native async numeric-address TCP clients; +all public outcomes still pass through an explicit caller-selected Scheduler. The repository contains: - one mcpp package manifest; -- the root module `mcpplibs.cmp` with Task, cancellation, executor, blocking, join, event, and mutex partitions; +- the root module `mcpplibs.cmp` with Task, cancellation, executor, blocking, TCP, join, event, and + mutex partitions; - gtest contract, lifetime, exception, scheduling, and threading tests; - one standalone path-dependency example; - local v1-readiness and cross-platform ThreadPool benchmark consumers; @@ -48,8 +51,10 @@ There is no `src/main.cpp`, so mcpp infers a library target named `cmp`; the man need a `[lib]` or `[targets.cmp]` override. The C++23 baseline is written explicitly even though it is also mcpp's default. -`compat.gtest = "1.15.2"` is an explicitly namespaced development dependency used by the test -targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. +`chriskohlhoff.asio = "1.38.1"` is the exact runtime dependency privately used by the TCP partition; +no Asio type appears in CMP's public API. `compat.gtest = "1.15.2"` is an explicitly namespaced +development dependency used by the test targets. CMP does not track an `mcpp.lock` file; it is +excluded by `.gitignore`. ## Repository layout @@ -74,6 +79,7 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. │ ├── run_loop.cppm │ ├── thread_pool.cppm │ ├── blocking.cppm +│ ├── tcp.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -84,6 +90,7 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. │ ├── run_loop_test.cpp │ ├── thread_pool_test.cpp │ ├── blocking_test.cpp +│ ├── tcp_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -101,10 +108,10 @@ targets. CMP does not track an `mcpp.lock` file; it is excluded by `.gitignore`. ## Build and tests `.xlings.json` pins the mcpp version used by the project. `mcpp build` builds the inferred library -target. `mcpp test` discovers nine test files and links a gtest entry point for each. The 116 tests +target. `mcpp test` discovers ten test files and links a gtest entry point for each. The 140 tests verify Task ownership and symmetric transfer together with structured joins, root execution, scheduling, exception propagation, timed and cross-thread wake-up, cancellation races, invalid -scheduler use, loop reuse, and stack-safe repeated completion. +scheduler use, loop reuse, stack-safe repeated completion, and TCP lifecycle/load behavior. Each CI workflow installs the project tools, builds the library, runs the test suite, and runs `examples/basic`. The workflows are separate because tool installation and runner details differ @@ -435,11 +442,26 @@ thread on which the coroutine currently executes. - the return schedule is intentionally uncancellable so every outcome reaches one executor; - this is thread-based isolation and does not claim native non-blocking I/O. +`IoContext` and `TcpStream` have the following contract: + +- one immovable context owns one private I/O driver and may be shared by many streams; +- a move-only stream connects only to numeric IPv4/IPv6 text; DNS and listening APIs are absent; +- `connect()`, `read_some()`, and `write_all()` are lazy Tasks and publish success or failure only + after the explicit return Scheduler hop; +- read/write spans borrow their storage until the Task completes; a stream permits one pending read + and one pending write, while same-direction overlap throws `std::logic_error`; +- remote EOF is sticky for reads but leaves the write direction open; `write_all()` is all-or-error; +- pre-cancellation initiates no I/O and leaves an open stream usable; cancelling an initiated write + closes the stream; +- `close()` is thread-safe, idempotent, and non-blocking; close and context shutdown cancel accepted + operations exactly once when they win the completion race; +- Phase 6A isolates arbitrary synchronous work on threads, while Phase 6B is native async TCP only. + There is no public free-standing `sync_wait`, detached execution, standalone Timer handle, -asynchronous I/O backend, custom frame allocator, or distinct blocking-pool type. Cancellation remains -explicit: Scheduler waits and `AsyncManualResetEvent` accept tokens, and TaskGroup owns an optional -shared stop channel, but the module provides no implicit propagation and no compatibility alias for -the old scaffold module. +generic asynchronous-I/O hierarchy, custom frame allocator, or distinct blocking-pool type. +Cancellation remains explicit: Scheduler/event/TCP waits accept tokens, and TaskGroup owns an +optional shared stop channel, but the module provides no implicit propagation and no compatibility +alias for the old scaffold module. Capturing coroutine lambdas require particular care: invoking a temporary capturing lambda can leave the lazy coroutine referring to a destroyed closure. CMP does not yet provide a helper that @@ -457,10 +479,10 @@ package contract: 1. TaskGroup result handles and additional cancellation-aware primitives; 2. channels and additional structured wake-up paths; 3. profile-guided work stealing if representative workloads justify it; -4. asynchronous I/O integrations; +4. additional native I/O families such as DNS, listeners, TLS, or files; 5. result adapters and optional coroutine-frame allocation strategies. -Task, cancellation, RunLoop, ThreadPool, blocking offload, `when_all`, TaskGroup, OneShotEvent, +Task, cancellation, RunLoop, ThreadPool, blocking offload, TCP, `when_all`, TaskGroup, OneShotEvent, AsyncManualResetEvent, and AsyncMutex occupy separate module partitions because they are implemented public boundaries. Further partitions or implementation units are added only when another implemented API needs them. @@ -478,14 +500,15 @@ cd examples/basic mcpp run ``` -The expected result is a successful library build, 116 passing tests across nine binaries, and an +The expected result is a successful library build, 140 passing tests across ten binaries, and an example that prints `Coroutine result: 42`, `Concurrent result: 42`, `Worker pool result: 42`, `Blocking result: 42`, `Task group result: 42`, `Recursive group result: 3`, `Event signalled`, `Reusable event cycles: 2`, `Mutex result: 42`, then `Coroutine cancelled` and exits with status 0. Tests retain the existing high-volume stack checks and add 20,000 recursive TaskGroup admissions, 100,000 pre-cancelled ready schedules, 50,000 manual event waiters, 20,000 nested reusable-event signals, set/cancel races, queued blocking cancellation, -and 5,000 concurrent blocking offloads. Focused race suites pass repeated Release runs. Compute, +5,000 concurrent blocking offloads, a 32-client TCP load, and repeated TCP close/stop completion +races. Focused race suites pass repeated Release runs. Compute, temporary-file, and loopback-network counts and throughput are recorded in the [v1 readiness benchmark](benchmarks/2026-08-29-cmp-v1-readiness.md). ThreadPool counts, concurrency, diff --git a/docs/architecture.zh.hant.md b/docs/architecture.zh.hant.md index 25fabfc..93d0777 100644 --- a/docs/architecture.zh.hant.md +++ b/docs/architecture.zh.hant.md @@ -14,12 +14,14 @@ CMP 是一個具備小型協程執行核心的 C++23 模組專案。根模組匯 `std::stop_token` 的協作式取消多載;定時排程使用相對和絕對的 `steady_clock` 期限,且不建立 計時執行緒。`ThreadPool::Scheduler::schedule()` 會把 continuation 明確轉移到任意固定 worker,並採用相同的取消獲勝規則。`run_blocking()` 使用呼叫端選擇的 ThreadPool 實例執行 -同步工作,並只在到達明確返回 Scheduler 後發布結果。 +同步工作,並只在到達明確返回 Scheduler 後發布結果。Phase 6B 新增擁有一個私有 driver 的 +不可移動 `IoContext`,以及用於原生非同步數值位址 TCP client 的 move-only `TcpStream`;所有 +公開結果仍透過呼叫端明確選擇的 Scheduler 返回。 儲存庫現有內容包括: - 一份 mcpp 套件清單; -- 根模組 `mcpplibs.cmp` 及 Task、cancellation、執行器、blocking、join、event、mutex 模組分割區; +- 根模組 `mcpplibs.cmp` 及 Task、cancellation、執行器、blocking、TCP、join、event、mutex 模組分割區; - 涵蓋契約、生命週期、例外、排程和執行緒行為的 gtest 測試; - 一個透過路徑相依使用根套件的獨立範例; - v1 可開發性壓測和跨平台 ThreadPool 壓測 consumer; @@ -45,7 +47,8 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m `src/main.cpp`,因此 mcpp 會推斷出名為 `cmp` 的函式庫目標,不需要額外設定 `[lib]` 或 `[targets.cmp]`。雖然 C++23 也是 mcpp 的預設標準,清單中仍明確寫出這項基線。 -`compat.gtest = "1.15.2"` 是測試使用的明確命名空間開發相依。CMP 目前不追蹤 +`chriskohlhoff.asio = "1.38.1"` 是 TCP 分割區私下使用的精確執行期相依;CMP 公開 API 不暴露 +Asio 型別。`compat.gtest = "1.15.2"` 是測試使用的明確命名空間開發相依。CMP 目前不追蹤 `mcpp.lock`,該檔案由 `.gitignore` 排除。 ## 儲存庫結構 @@ -71,6 +74,7 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m │ ├── run_loop.cppm │ ├── thread_pool.cppm │ ├── blocking.cppm +│ ├── tcp.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -81,6 +85,7 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m │ ├── run_loop_test.cpp │ ├── thread_pool_test.cpp │ ├── blocking_test.cpp +│ ├── tcp_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -98,9 +103,9 @@ mcpp 套件由 `mcpplibs` 和 `cmp` 共同識別。使用端在 `[dependencies.m ## 建置與測試 `.xlings.json` 固定專案使用的 mcpp 版本。`mcpp build` 建置自動推斷的函式庫目標。 -`mcpp test` 會找到九個測試檔案,並為每個檔案連結 gtest 進入點。116 項測試同時驗證 Task 所有權 +`mcpp test` 會找到十個測試檔案,並為每個檔案連結 gtest 進入點。140 項測試同時驗證 Task 所有權 和對稱轉移、結構化匯合,以及根任務執行、普通與定時排程、例外傳播、跨執行緒期限喚醒、 -無效 Scheduler、取消競態、RunLoop 重複使用和不會增長呼叫堆疊的重複完成。 +無效 Scheduler、取消競態、RunLoop 重複使用、不會增長呼叫堆疊的重複完成,以及 TCP 生命週期和負載。 三套 CI 工作流程都會安裝專案工具、建置函式庫、執行測試並執行 `examples/basic`。不同 作業系統的工具安裝和執行環境不同,因此分別保留工作流程檔案。 @@ -414,10 +419,24 @@ Task;明確等待原 Scheduler 才會把 continuation 送回對應 RunLoop。 - 返回排程刻意不可取消,使每個結果都到達一個執行器; - 這是基於執行緒的隔離,不表示原生非阻塞 I/O。 -目前沒有公開自由函式 `sync_wait`、detached 執行、獨立 Timer 控制代碼、非同步 I/O 後端、 -自訂協程框架 allocator 或獨立的阻塞執行緒池型別。取消仍是明確的:Scheduler 等待和 -`AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模組不提供隱式 -傳播,並且沒有保留舊骨架模組的相容別名。 +`IoContext` 與 `TcpStream` 遵循以下契約: + +- 一個不可移動的 context 擁有一個私有 I/O driver,並可由多個 stream 共用; +- move-only stream 只連線數值 IPv4/IPv6 文字;不提供 DNS 或監聽 API; +- `connect()`、`read_some()` 和 `write_all()` 都是延遲 Task,成功或失敗只在明確返回 Scheduler + 跳轉後發布; +- read/write span 借用底層儲存直到 Task 完成;一個 stream 允許一個 pending read 和一個 + pending write,同方向重疊拋出 `std::logic_error`; +- 遠端 EOF 對 read 保持 sticky,但不關閉 write 方向;`write_all()` 採用全寫或錯誤契約; +- 預取消不發起 I/O,並保持 open stream 可用;已發起 write 被取消會關閉 stream; +- `close()` 執行緒安全、冪等且非阻塞;當 close 或 context shutdown 贏得競態時,已接納操作 + 恰好取消一次; +- Phase 6A 在執行緒上隔離任意同步工作,Phase 6B 只提供原生非同步 TCP。 + +目前沒有公開自由函式 `sync_wait`、detached 執行、獨立 Timer 控制代碼、通用非同步 I/O 層、 +自訂協程框架 allocator 或獨立的阻塞執行緒池型別。取消仍是明確的:Scheduler、event 和 TCP +等待接受 token,TaskGroup 可持有共享 stop 通道,但模組不提供隱式傳播,也沒有保留舊骨架 +模組的相容別名。 捕捉變數的協程 lambda 需要特別小心:立即呼叫一個暫時的捕捉 lambda,可能使延遲協程參考 已經銷毀的閉包。CMP 尚未提供延長該閉包生命週期的輔助函式。 @@ -433,11 +452,11 @@ CMP 名稱中的 `C` 與 Go 執行期中的 `G` 相呼應,但這只說明命 1. TaskGroup 結果控制代碼及更多支援取消的原語; 2. channel 和更多結構化喚醒路徑; 3. 代表性負載證明有必要時再加入 profiling 驅動的工作竊取; -4. 非同步 I/O 整合; +4. DNS、監聽、TLS 或檔案等其他原生 I/O 類型; 5. 結果適配器和可選的協程框架配置策略。 -Task、cancellation、RunLoop、ThreadPool、blocking offload、`when_all`、TaskGroup、OneShotEvent、 -AsyncManualResetEvent 與 AsyncMutex 已形成真實的公開邊界,因此分別位於模組分割區中。 +Task、cancellation、RunLoop、ThreadPool、blocking offload、TCP、`when_all`、TaskGroup、 +OneShotEvent、AsyncManualResetEvent 與 AsyncMutex 已形成真實的公開邊界,因此分別位於模組分割區中。 只有其他已實作 API 確實需要新邊界時,才繼續增加模組分割區或實作單元。 ## 驗證 @@ -453,14 +472,15 @@ cd examples/basic mcpp run ``` -預期結果是函式庫建置成功、九個二進位檔中的 116 項測試全部通過,而且範例依序輸出 +預期結果是函式庫建置成功、十個二進位檔中的 140 項測試全部通過,而且範例依序輸出 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、 `Blocking result: 42`、`Task group result: 42`、`Recursive group result: 3`、`Event signalled`、 `Reusable event cycles: 2`、 `Mutex result: 42` 和 `Coroutine cancelled` 後以狀態 0 結束。測試保留原有高容量堆疊安全 檢查,並增加兩萬次 TaskGroup 遞迴接納、十萬次預取消就緒排程、五萬個 manual event 等待者、 -兩萬次巢狀可複用事件訊號、set/cancel 競態、排隊阻塞取消和 5,000 個並行阻塞 offload;重點 -競態套件已連續執行多輪 Release 測試。計算、臨時檔案和回環網路的成功/失敗計數及吞吐記錄在 +兩萬次巢狀可複用事件訊號、set/cancel 競態、排隊阻塞取消、5,000 個並行阻塞 offload、 +32 客戶端 TCP 負載,以及重複的 TCP close/stop completion 競態;重點競態套件已連續執行多輪 +Release 測試。計算、臨時檔案和回環網路的成功/失敗計數及吞吐記錄在 [v1 可開發性壓測](benchmarks/2026-08-29-cmp-v1-readiness.md)。ThreadPool 的計數、並行和五輪 Release 資料記錄在[執行緒池壓測](benchmarks/2026-08-29-cmp-thread-pool.md)。目前 Windows LLVM 工具鏈 不會產生 GNU depfile;如果模組介面包含的檔案發生變更,增量建置可能沿用舊的 BMI 或 diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 01be18c..1378321 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -14,12 +14,14 @@ CMP 是一个具备小型协程执行核心的 C++23 模块项目。根模块导 `std::stop_token` 的协作式取消重载;定时调度使用相对和绝对的 `steady_clock` 期限,且不创建 定时线程。`ThreadPool::Scheduler::schedule()` 会把 continuation 显式转移到任意固定 worker, 并采用相同的取消获胜规则。`run_blocking()` 使用调用方选择的 ThreadPool 实例执行同步工作, -并只在到达显式返回 Scheduler 后发布结果。 +并只在到达显式返回 Scheduler 后发布结果。Phase 6B 新增拥有一个私有 driver 的不可移动 +`IoContext`,以及用于原生异步数值地址 TCP client 的 move-only `TcpStream`;所有公共结果仍 +通过调用方显式选择的 Scheduler 返回。 仓库现有内容包括: - 一份 mcpp 包清单; -- 根模块 `mcpplibs.cmp` 及 Task、cancellation、执行器、blocking、join、event、mutex 模块分区; +- 根模块 `mcpplibs.cmp` 及 Task、cancellation、执行器、blocking、TCP、join、event、mutex 模块分区; - 覆盖契约、生命周期、异常、调度和线程行为的 gtest 测试; - 一个通过路径依赖使用根包的独立示例; - v1 可开发性压测和跨平台 ThreadPool 压测 consumer; @@ -45,7 +47,8 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp `src/main.cpp`,因此 mcpp 会推断出名为 `cmp` 的库目标,不需要额外配置 `[lib]` 或 `[targets.cmp]`。虽然 C++23 也是 mcpp 的默认标准,清单中仍然明确写出这一基线。 -`compat.gtest = "1.15.2"` 是测试使用的显式命名空间开发依赖。CMP 当前不跟踪 +`chriskohlhoff.asio = "1.38.1"` 是 TCP 分区私有使用的精确运行时依赖;CMP 公共 API 不暴露 +Asio 类型。`compat.gtest = "1.15.2"` 是测试使用的显式命名空间开发依赖。CMP 当前不跟踪 `mcpp.lock`,该文件由 `.gitignore` 排除。 ## 仓库结构 @@ -71,6 +74,7 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp │ ├── run_loop.cppm │ ├── thread_pool.cppm │ ├── blocking.cppm +│ ├── tcp.cppm │ ├── when_all.cppm │ ├── task_group.cppm │ ├── one_shot_event.cppm @@ -81,6 +85,7 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp │ ├── run_loop_test.cpp │ ├── thread_pool_test.cpp │ ├── blocking_test.cpp +│ ├── tcp_test.cpp │ ├── when_all_test.cpp │ ├── task_group_test.cpp │ ├── one_shot_event_test.cpp @@ -98,9 +103,9 @@ mcpp 包由 `mcpplibs` 和 `cmp` 共同标识。使用方在 `[dependencies.mcpp ## 构建与测试 `.xlings.json` 固定项目使用的 mcpp 版本。`mcpp build` 构建自动推断的库目标。 -`mcpp test` 发现九个测试文件,并为每个文件链接 gtest 入口。116 项测试同时验证 Task 所有权和 +`mcpp test` 发现十个测试文件,并为每个文件链接 gtest 入口。140 项测试同时验证 Task 所有权和 对称转移、结构化汇合,以及根任务执行、普通与定时调度、异常传播、跨线程期限唤醒、无效 -Scheduler、取消竞态、RunLoop 复用和不会增长调用栈的重复完成。 +Scheduler、取消竞态、RunLoop 复用、不会增长调用栈的重复完成,以及 TCP 生命周期和负载。 三套 CI 工作流都会安装项目工具、构建库、运行测试并执行 `examples/basic`。不同操作系统 的工具安装和运行环境不同,因此分别保留工作流文件。 @@ -413,10 +418,24 @@ RunLoop 不拥有工作线程,也不提供自动线程亲和。外部 awaiter - 返回调度刻意不可取消,使每个结果都到达一个执行器; - 这是基于线程的隔离,不表示原生非阻塞 I/O。 -目前没有公共自由函数 `sync_wait`、detached 执行、独立 Timer 句柄、异步 I/O 后端、自定义 -协程帧 allocator 或独立的阻塞线程池类型。取消仍是显式的:Scheduler 等待和 -`AsyncManualResetEvent` 接受 token,TaskGroup 可持有共享 stop 通道,但模块不提供隐式传播, -并且没有保留旧脚手架模块的兼容别名。 +`IoContext` 与 `TcpStream` 遵循以下契约: + +- 一个不可移动的 context 拥有一个私有 I/O driver,并可由多个 stream 共享; +- move-only stream 只连接数值 IPv4/IPv6 文本;不提供 DNS 或监听 API; +- `connect()`、`read_some()` 和 `write_all()` 都是懒 Task,成功或失败只在显式返回 Scheduler + 跳转后发布; +- read/write span 借用底层存储直到 Task 完成;一个 stream 允许一个 pending read 和一个 + pending write,同方向重叠抛出 `std::logic_error`; +- 远端 EOF 对 read 保持 sticky,但不关闭 write 方向;`write_all()` 采用全写或错误契约; +- 预取消不发起 I/O,并保持 open stream 可用;已发起 write 被取消会关闭 stream; +- `close()` 线程安全、幂等且非阻塞;当 close 或 context shutdown 赢得竞态时,已接纳操作 + 恰好取消一次; +- Phase 6A 在线程上隔离任意同步工作,Phase 6B 只提供原生异步 TCP。 + +目前没有公共自由函数 `sync_wait`、detached 执行、独立 Timer 句柄、通用异步 I/O 层、自定义 +协程帧 allocator 或独立的阻塞线程池类型。取消仍是显式的:Scheduler、event 和 TCP 等待 +接受 token,TaskGroup 可持有共享 stop 通道,但模块不提供隐式传播,也没有保留旧脚手架 +模块的兼容别名。 捕获变量的协程 lambda 需要特别小心:立即调用一个临时的捕获 lambda,可能使懒协程引用 已经销毁的闭包。CMP 尚未提供延长该闭包生命周期的辅助函数。 @@ -431,11 +450,11 @@ CMP 名称中的 `C` 与 Go 运行时中的 `G` 相呼应,但这只说明命 1. TaskGroup 结果句柄及更多支持取消的原语; 2. channel 和更多结构化唤醒路径; 3. 代表性负载证明有必要时再加入 profiling 驱动的工作窃取; -4. 异步 I/O 集成; +4. DNS、监听、TLS 或文件等其他原生 I/O 类型; 5. 结果适配器和可选的协程帧分配策略。 -Task、cancellation、RunLoop、ThreadPool、blocking offload、`when_all`、TaskGroup、OneShotEvent、 -AsyncManualResetEvent 与 AsyncMutex 已经形成真实的公共边界,因此分别位于模块分区中。 +Task、cancellation、RunLoop、ThreadPool、blocking offload、TCP、`when_all`、TaskGroup、 +OneShotEvent、AsyncManualResetEvent 与 AsyncMutex 已经形成真实的公共边界,因此分别位于模块分区中。 只有其他已实现 API 确实需要新边界时,才继续增加模块分区或实现单元。 ## 验证 @@ -451,14 +470,14 @@ cd examples/basic mcpp run ``` -预期结果是库构建成功、九个二进制中的 116 项测试全部通过,并且示例依次输出 +预期结果是库构建成功、十个二进制中的 140 项测试全部通过,并且示例依次输出 `Coroutine result: 42`、`Concurrent result: 42`、`Worker pool result: 42`、 `Blocking result: 42`、`Task group result: 42`、`Recursive group result: 3`、`Event signalled`、 `Reusable event cycles: 2`、 `Mutex result: 42` 和 `Coroutine cancelled` 后以状态 0 退出。测试保留原有高容量栈安全检查, 并增加两万次 TaskGroup 递归接纳、十万次预取消就绪调度、五万个 manual event 等待者、两万次 -嵌套可复用事件信号、set/cancel 竞态、排队阻塞取消和 5,000 个并发阻塞 offload;重点竞态 -套件已连续执行多轮 Release 测试。 +嵌套可复用事件信号、set/cancel 竞态、排队阻塞取消、5,000 个并发阻塞 offload、32 客户端 +TCP 负载,以及重复的 TCP close/stop completion 竞态;重点竞态套件已连续执行多轮 Release 测试。 计算、临时文件和回环网络的成功/失败计数及吞吐记录在 [v1 可开发性压测](benchmarks/2026-08-29-cmp-v1-readiness.md)。ThreadPool 的计数、并发和五轮 Release 数据记录在[线程池压测](benchmarks/2026-08-29-cmp-thread-pool.md)。当前 Windows LLVM 工具链不会 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index d54e0be..4e04d03 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — readiness TCP client migration complete +**Status:** In progress — developer documentation synchronized **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -400,6 +400,23 @@ portable test fixture and readiness benchmark already exercise the API, while th should stay short and deterministic. Still run the unchanged example to verify that adding the runtime dependency does not break an external path consumer. +**Self-review — 2026-08-30:** Keep one semantic checklist across all six documents rather than +copying implementation internals: `IoContext` owns one driver and is shared; `TcpStream` is +move-only; connect accepts numeric IPv4/IPv6 only; spans borrow storage through Task completion; +every outcome attempts the explicit return Scheduler; one read and one write may coexist while +same-direction overlap fails; pre-cancellation leaves an open stream usable, initiated write +cancellation closes it, and close/context shutdown cancel accepted work exactly once. State that +Phase 6A isolates arbitrary synchronous work on threads while Phase 6B is native async TCP only, +not generic async I/O. Update module/test trees, the exact local 10-binary/140-test counts, and the +readiness result. Add a short documentation-only TCP snippet, but leave `examples/basic` unchanged. +English, Simplified Chinese, and Traditional Chinese must remain structurally equivalent. + +**Completion — 2026-08-30:** Synchronized all three README and architecture variants with the +verified `IoContext`/`TcpStream` surface, lifecycle and concurrency contracts, Phase 6A/6B +boundary, module/test layout, 140-test total, and readiness result. Removed stale statements that +native async I/O was wholly absent. Kept `examples/basic` unchanged; its path-consumer `mcpp run` +completed successfully with the expected output and exit status 0. + ## 10. Full verification and state record Run focused checks first, then the complete local matrix: From 7c059fa146e53fc99872afc82e1b554975b74026 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 01:57:52 +0800 Subject: [PATCH 12/17] =?UTF-8?q?=E5=AE=8C=E6=88=90=20TCP=20v1=20=E6=9C=AC?= =?UTF-8?q?=E5=9C=B0=E9=AA=8C=E8=AF=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 19 +++++---- .../benchmarks/2026-08-29-cmp-v1-readiness.md | 42 +++++++++---------- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 25 ++++++++++- ...-08-29-cmp-phase6b-tcp-client-v1-design.md | 2 +- 4 files changed, 56 insertions(+), 32 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index eda3953..0d03732 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -14,9 +14,8 @@ path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1 与 Phase 6B 第 1–8 项均已作为本地提交完成且尚未推送。Phase 6B -Plan 第 9 项已同步六份开发文档,并通过未改动的独立示例验证;下一步是第 10 项完整本地门禁 -与状态收口。 +Phase 6A blocking offload v1 与 Phase 6B 第 1–10 项均已在当前分支本地完成且尚未推送。Phase +6B 完整本地门禁已全部通过,Design/Plan 已标记为“本地实现并验证,远程 CI 待确认”。 ## 已完成工作 @@ -197,8 +196,11 @@ Plan 第 9 项已同步六份开发文档,并通过未改动的独立示例验 500 次 stop/completion 竞态,没有丢失或重复完成。 - 加入并发负载后,`tcp_test` Dev/Release 均为 24/24;两套 strict cache-off 全量测试均为 10 个二进制、140/140。32/32 客户端成功,取消、错误、payload/线程不匹配和重复完成均为 0。 -- readiness 新网络路径每轮均为 20,000 成功、100 预期失败、0 非预期失败;五轮网络中位数 - 为 2,071.085 ms / 9,705.1 ops/s,原始结果已写入 benchmark 报告,耗时不作门槛。 +- Phase 6B 最终门禁:focused Release 10/10 通过,覆盖 100 次 close 与 500 次 stop completion + 竞态;Dev/Release strict cache-off 构建均通过,两套全量测试均为 10 个二进制、140/140。 +- 最终门禁再次运行独立示例成功;readiness Release strict cache-off 构建成功,随后五轮共 + 15 个场景全部 PASS。每轮 compute/file/network 非预期失败均为 0,网络中位数为 + 2,078.127 ms / 9,672.2 ops/s,最新原始数据已写入 benchmark 报告。 ## 已知问题 / 风险 @@ -218,10 +220,9 @@ Plan 第 9 项已同步六份开发文档,并通过未改动的独立示例验 ## 剩余工作 -1. 按 Phase 6B Plan 第 10 项执行完整本地验证矩阵、example、100 轮竞态门禁与五轮 benchmark。 -2. 本地完成后仍需用户另行授权 push/PR,才能取得 Linux、macOS、Windows 远程 CI 结果。 +1. 本地开发与验证已完成;后续只有在用户另行授权 push/PR 后,才能取得 Linux、macOS、 + Windows 远程 CI 结果并达到 Design 的最终跨平台完成条件。 ## 推荐下一步 -按 `2026-08-30-cmp-phase6b-tcp-client-v1.md` 开始第 10 项,先执行 focused Release TCP 竞态 -门禁,再执行 Dev/Release 全量矩阵、独立示例与五轮 readiness benchmark,准确记录全部结果。 +等待用户另行授权远程操作;在此之前不 push、不创建 PR,也不宣称三平台 CI 已通过。 diff --git a/docs/benchmarks/2026-08-29-cmp-v1-readiness.md b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md index a033724..abb8d66 100644 --- a/docs/benchmarks/2026-08-29-cmp-v1-readiness.md +++ b/docs/benchmarks/2026-08-29-cmp-v1-readiness.md @@ -2,7 +2,7 @@ **日期:** 2026-08-29 -**结论:** Phase 6B TCP client 迁移后的本机 5 轮 Release 压测全部通过;三类场景均无 +**结论:** Phase 6B TCP client 最终本地门禁的 5 轮 Release 压测全部通过;三类场景均无 非预期失败。 ## 测试边界 @@ -47,35 +47,35 @@ mcpp build --profile release --strict --cache=off 吞吐量按“成功操作 + 预期失败操作”的总操作数计算。 -## 五轮原始结果 +## 最终门禁五轮原始结果 | 轮次 | 场景 | 操作数 | 成功 | 预期失败 | 非预期失败 | 耗时 ms | ops/s | 状态 | | ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | -| 1 | compute | 50,000 | 49,000 | 1,000 | 0 | 32.879 | 1,520,728.9 | PASS | -| 1 | file_io | 1,100 | 1,000 | 100 | 0 | 176.548 | 6,230.6 | PASS | -| 1 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,016.219 | 9,969.2 | PASS | -| 2 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.174 | 1,839,973.8 | PASS | -| 2 | file_io | 1,100 | 1,000 | 100 | 0 | 167.446 | 6,569.3 | PASS | -| 2 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,071.085 | 9,705.1 | PASS | -| 3 | compute | 50,000 | 49,000 | 1,000 | 0 | 29.339 | 1,704,210.2 | PASS | -| 3 | file_io | 1,100 | 1,000 | 100 | 0 | 189.751 | 5,797.1 | PASS | -| 3 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,026.998 | 9,916.1 | PASS | -| 4 | compute | 50,000 | 49,000 | 1,000 | 0 | 28.291 | 1,767,337.2 | PASS | -| 4 | file_io | 1,100 | 1,000 | 100 | 0 | 970.697 | 1,133.2 | PASS | -| 4 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,353.919 | 8,538.9 | PASS | -| 5 | compute | 50,000 | 49,000 | 1,000 | 0 | 30.551 | 1,636,609.6 | PASS | -| 5 | file_io | 1,100 | 1,000 | 100 | 0 | 457.299 | 2,405.4 | PASS | -| 5 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,503.267 | 8,029.5 | PASS | +| 1 | compute | 50,000 | 49,000 | 1,000 | 0 | 30.367 | 1,646,518.5 | PASS | +| 1 | file_io | 1,100 | 1,000 | 100 | 0 | 230.594 | 4,770.3 | PASS | +| 1 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,246.887 | 8,945.7 | PASS | +| 2 | compute | 50,000 | 49,000 | 1,000 | 0 | 31.310 | 1,596,926.1 | PASS | +| 2 | file_io | 1,100 | 1,000 | 100 | 0 | 179.101 | 6,141.8 | PASS | +| 2 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,165.041 | 9,283.9 | PASS | +| 3 | compute | 50,000 | 49,000 | 1,000 | 0 | 38.062 | 1,313,656.2 | PASS | +| 3 | file_io | 1,100 | 1,000 | 100 | 0 | 212.912 | 5,166.4 | PASS | +| 3 | network_loopback | 20,100 | 20,000 | 100 | 0 | 1,985.032 | 10,125.8 | PASS | +| 4 | compute | 50,000 | 49,000 | 1,000 | 0 | 29.358 | 1,703,112.6 | PASS | +| 4 | file_io | 1,100 | 1,000 | 100 | 0 | 181.380 | 6,064.6 | PASS | +| 4 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,078.127 | 9,672.2 | PASS | +| 5 | compute | 50,000 | 49,000 | 1,000 | 0 | 27.611 | 1,810,856.0 | PASS | +| 5 | file_io | 1,100 | 1,000 | 100 | 0 | 593.669 | 1,852.9 | PASS | +| 5 | network_loopback | 20,100 | 20,000 | 100 | 0 | 2,004.864 | 10,025.6 | PASS | ## 汇总 | 场景 | 最短 / 中位 / 最长耗时 ms | 最高 / 中位 / 最低 ops/s | 五轮非预期失败 | | --- | ---: | ---: | ---: | -| compute | 27.174 / 29.339 / 32.879 | 1,839,973.8 / 1,704,210.2 / 1,520,728.9 | 0 | -| file_io | 167.446 / 189.751 / 970.697 | 6,569.3 / 5,797.1 / 1,133.2 | 0 | -| network_loopback | 2,016.219 / 2,071.085 / 2,503.267 | 9,969.2 / 9,705.1 / 8,029.5 | 0 | +| compute | 27.611 / 30.367 / 38.062 | 1,810,856.0 / 1,646,518.5 / 1,313,656.2 | 0 | +| file_io | 179.101 / 212.912 / 593.669 | 6,141.8 / 5,166.4 / 1,852.9 | 0 | +| network_loopback | 1,985.032 / 2,078.127 / 2,246.887 | 10,125.8 / 9,672.2 / 8,945.7 | 0 | 相同负载的 Phase 6A 阻塞 client 历史中位数为 2,627.611 ms / 7,649.5 ops/s;Phase 6B 原生 -异步 client 的本轮中位数为 2,071.085 ms / 9,705.1 ops/s。该差异只作为本机回归证据,不是 +异步 client 的本轮中位数为 2,078.127 ms / 9,672.2 ops/s。该差异只作为本机回归证据,不是 跨机器 SLA 或 CI 阈值。结果说明 v1 核心可以同时承载结构化协程、blocking offload 和 单 driver 原生异步 TCP;work stealing 仍需由独立的代表性负载证明必要性。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index 4e04d03..aa5ad9a 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** In progress — developer documentation synchronized +**Status:** Locally complete — remote CI pending **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -443,6 +443,29 @@ Only after local verification: Phase 6B reaches the Design's final completion state only after Linux, macOS, and Windows CI all pass. Until then, report it as locally implemented and verified with remote compatibility pending. +**Self-review — 2026-08-30:** Run the two existing internally repeated Release race tests five +times, which exercises 100 close/completion and 500 stop/completion races without multiplying the +whole socket suite or exhausting this host's narrow ephemeral-port range. Any hang, duplicate, +invalid outcome, failed command, or hard-count mismatch stops the gate. Then run the exact +cache-off strict Dev/Release matrix, the unchanged path-consumer example, and a Release benchmark +build followed by five sequential runs of the produced binary. Benchmark timing is evidence only; +each run must retain exact compute/file/network success and expected-failure counts with zero +unexpected failures. Record only local Linux/WSL2 facts and leave remote CI pending. + +**Completion — 2026-08-30:** The focused Release command was: + +```text +mcpp test tcp_test --profile release --strict --cache=off -- --gtest_filter='CmpTcpTest.CloseCompletionRaceChoosesOneOutcome:CmpTcpTest.StopCompletionRaceChoosesOneOutcome' --gtest_repeat=5 +``` + +It passed 10/10 focused executions, covering 100 close and 500 stop completion races. Strict +cache-off Dev and Release builds passed; both full suites passed 140/140 tests across 10 binaries, +including 24/24 TCP tests. The unchanged standalone example printed every expected line and exited +0. The readiness Release build passed, followed by five sequential runs: each reported compute +49,000/1,000/0, file 1,000/100/0, and network 20,000/100/0 success/expected/unexpected counts. +All 15 scenario rows passed; the final raw timings are recorded in the readiness report. These are +local Linux/WSL2 + LLVM 22.1.8 results; Linux, macOS, and Windows remote CI remain pending. + ## Scope Guard If implementation reveals a need for DNS, listener APIs, timeouts, generic endpoints, socket diff --git a/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md b/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md index 5f31d06..ac8babe 100644 --- a/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md +++ b/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md @@ -1,7 +1,7 @@ # CMP Phase 6B TCP Client v1 Design **Date:** 2026-08-29 -**Status:** Approved for implementation +**Status:** Locally implemented and verified — remote CI pending **Baseline:** Phase 6A is locally committed as `701aa8b` and passes 116/116 tests ## Purpose From 8fbd46f1b09ceb7874be922f19a43594cb180d8e Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 02:00:51 +0800 Subject: [PATCH 13/17] =?UTF-8?q?=E7=BA=A0=E6=AD=A3=E6=97=A9=E6=9C=9F?= =?UTF-8?q?=E6=96=B9=E6=A1=88=E7=8A=B6=E6=80=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 3 +++ docs/superpowers/plans/2026-08-02-cmp-bootstrap.md | 2 ++ docs/superpowers/plans/2026-08-12-cmp-run-loop.md | 2 ++ docs/superpowers/plans/2026-08-12-cmp-timer-v1.md | 4 +++- docs/superpowers/plans/2026-08-23-cmp-cancellation-v1.md | 1 + docs/superpowers/specs/2026-08-02-cmp-bootstrap-design.md | 2 +- docs/superpowers/specs/2026-08-12-cmp-run-loop-design.md | 2 +- docs/superpowers/specs/2026-08-12-cmp-timer-v1-design.md | 2 +- .../specs/2026-08-23-cmp-cancellation-v1-design.md | 2 +- 9 files changed, 15 insertions(+), 5 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 0d03732..59e4373 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -16,6 +16,7 @@ path-dependency consumer。 Phase 6A blocking offload v1 与 Phase 6B 第 1–10 项均已在当前分支本地完成且尚未推送。Phase 6B 完整本地门禁已全部通过,Design/Plan 已标记为“本地实现并验证,远程 CI 待确认”。 +已审查全部现有 Spec/Plan;除 Phase 6B 远程门禁外,没有其他已批准但尚未执行的本地任务。 ## 已完成工作 @@ -34,6 +35,8 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–10 项均已在当前分支本 硬检查保持不变。 - 三份 README 与三份架构文档已同步 `IoContext`/`TcpStream` API、生命周期、并发/取消边界、 10 个测试二进制/140 项测试及 readiness 结果;Phase 6A Design/Plan 和数据报告也与实现一致。 +- 根据本地 `main` 的 PR #1–#4 合并提交,纠正 bootstrap、RunLoop、Timer、Cancellation 早期 + Design/Plan 的陈旧状态;未推断未留存的 Timer 远程检查明细。 - 完成 Phase 6B 依赖核查:mcpp-index 当前提供 `chriskohlhoff.asio@1.38.1`,其 C++23 模块 `asio` 覆盖 Linux、macOS 和 Windows,能够复用 epoll、kqueue 与 IOCP 后端。 - 新增并批准 Phase 6B TCP client v1 Design,确定首个原生异步 I/O 切片仅包含数值地址 TCP diff --git a/docs/superpowers/plans/2026-08-02-cmp-bootstrap.md b/docs/superpowers/plans/2026-08-02-cmp-bootstrap.md index e75d283..eee41cd 100644 --- a/docs/superpowers/plans/2026-08-02-cmp-bootstrap.md +++ b/docs/superpowers/plans/2026-08-02-cmp-bootstrap.md @@ -1,5 +1,7 @@ # CMP Bootstrap Implementation Plan +**Status:** Complete; merged via PR #1 + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking. **Goal:** Replace the generic `mcpplibs.mylib` scaffold with an import-only CMP C++23 module project, remove premature publishing templates, clean local clice artifacts, and leave a fully verified but uncommitted working tree. diff --git a/docs/superpowers/plans/2026-08-12-cmp-run-loop.md b/docs/superpowers/plans/2026-08-12-cmp-run-loop.md index 7be25bd..7a99ac5 100644 --- a/docs/superpowers/plans/2026-08-12-cmp-run-loop.md +++ b/docs/superpowers/plans/2026-08-12-cmp-run-loop.md @@ -1,5 +1,7 @@ # CMP RunLoop v1 Implementation Plan +**Status:** Complete; merged via PR #2 + **Goal:** Add the smallest scheduler-aware root execution API that makes Task convenient to use while keeping lifetime, synchronization, and failure handling inside CMP. diff --git a/docs/superpowers/plans/2026-08-12-cmp-timer-v1.md b/docs/superpowers/plans/2026-08-12-cmp-timer-v1.md index a24759d..6ef122f 100644 --- a/docs/superpowers/plans/2026-08-12-cmp-timer-v1.md +++ b/docs/superpowers/plans/2026-08-12-cmp-timer-v1.md @@ -1,6 +1,7 @@ # CMP Timer v1 Implementation Plan **Reviewed:** 2026-08-23 +**Status:** Complete; merged via PR #3 **Goal:** Add monotonic relative and absolute timer awaits to the existing Scheduler without a background thread, new dependency, cancellation model, or detached ownership. **Design:** `docs/superpowers/specs/2026-08-12-cmp-timer-v1-design.md` @@ -70,4 +71,5 @@ the user explicitly authorizes Git for that task. - Dev and Release test passed: 29 tests across two binaries; - Release RunLoop suite passed 10 additional consecutive runs; - standalone example printed `Coroutine result: 42` after awaiting its timer; -- Linux, macOS, and Windows CI remain pending until a later authorized push. +- implementation was merged via PR #3; this historical plan does not retain the individual remote + check results. diff --git a/docs/superpowers/plans/2026-08-23-cmp-cancellation-v1.md b/docs/superpowers/plans/2026-08-23-cmp-cancellation-v1.md index 011642e..7a62c2d 100644 --- a/docs/superpowers/plans/2026-08-23-cmp-cancellation-v1.md +++ b/docs/superpowers/plans/2026-08-23-cmp-cancellation-v1.md @@ -1,6 +1,7 @@ # CMP Cancellation v1 Implementation Plan **Date:** 2026-08-23 +**Status:** Complete; merged via PR #4 **Goal:** Add exactly-once cooperative cancellation to relative and absolute Scheduler waits with standard stop tokens, no inline coroutine resumption, and no allocation from stop callbacks. **Design:** `docs/superpowers/specs/2026-08-23-cmp-cancellation-v1-design.md` diff --git a/docs/superpowers/specs/2026-08-02-cmp-bootstrap-design.md b/docs/superpowers/specs/2026-08-02-cmp-bootstrap-design.md index 2e054c6..b99dfe8 100644 --- a/docs/superpowers/specs/2026-08-02-cmp-bootstrap-design.md +++ b/docs/superpowers/specs/2026-08-02-cmp-bootstrap-design.md @@ -1,7 +1,7 @@ # CMP Bootstrap Design **Date:** 2026-08-02 -**Status:** Approved for implementation +**Status:** Implemented and merged via PR #1 **Base:** `main` at `13b95c3` ## Purpose diff --git a/docs/superpowers/specs/2026-08-12-cmp-run-loop-design.md b/docs/superpowers/specs/2026-08-12-cmp-run-loop-design.md index 9dab970..d4d54e0 100644 --- a/docs/superpowers/specs/2026-08-12-cmp-run-loop-design.md +++ b/docs/superpowers/specs/2026-08-12-cmp-run-loop-design.md @@ -1,7 +1,7 @@ # CMP RunLoop v1 Design **Date:** 2026-08-12 -**Status:** Implemented and verified, pending user commit +**Status:** Implemented and merged via PR #2 **Base:** `feature/task-v1` at `7e24ee0` ## Purpose diff --git a/docs/superpowers/specs/2026-08-12-cmp-timer-v1-design.md b/docs/superpowers/specs/2026-08-12-cmp-timer-v1-design.md index 20ab00b..d9aa49b 100644 --- a/docs/superpowers/specs/2026-08-12-cmp-timer-v1-design.md +++ b/docs/superpowers/specs/2026-08-12-cmp-timer-v1-design.md @@ -2,7 +2,7 @@ **Date:** 2026-08-12 **Reviewed:** 2026-08-23 -**Status:** Implemented and locally verified; three-platform CI pending +**Status:** Implemented and merged via PR #3 **Base:** `main` at `e729782` **Branch:** `feature/timer-v1` diff --git a/docs/superpowers/specs/2026-08-23-cmp-cancellation-v1-design.md b/docs/superpowers/specs/2026-08-23-cmp-cancellation-v1-design.md index 62612c2..d64fef7 100644 --- a/docs/superpowers/specs/2026-08-23-cmp-cancellation-v1-design.md +++ b/docs/superpowers/specs/2026-08-23-cmp-cancellation-v1-design.md @@ -1,7 +1,7 @@ # CMP Cancellation v1 Design **Date:** 2026-08-23 -**Status:** Implemented; local and Linux/macOS/Windows CI verified +**Status:** Implemented, Linux/macOS/Windows CI verified, and merged via PR #4 **Base:** `main` at `c06705c` **Branch:** `feature/cancellation-v1` From 180404409900a23c531b7d1e0f5f7abab89f6391 Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 02:05:33 +0800 Subject: [PATCH 14/17] =?UTF-8?q?=E8=AE=B0=E5=BD=95=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=E5=AE=8C=E6=88=90=E6=80=A7=E5=AE=A1=E8=AE=A1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 59e4373..6e61235 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -14,9 +14,10 @@ path-dependency consumer。 ## 当前目标与状态 -Phase 6A blocking offload v1 与 Phase 6B 第 1–10 项均已在当前分支本地完成且尚未推送。Phase -6B 完整本地门禁已全部通过,Design/Plan 已标记为“本地实现并验证,远程 CI 待确认”。 -已审查全部现有 Spec/Plan;除 Phase 6B 远程门禁外,没有其他已批准但尚未执行的本地任务。 +Phase 4 completion、Phase 5 ThreadPool、Phase 6A blocking offload 与 Phase 6B 第 1–10 项均已 +本地完成;本阶段新增提交尚未推送。Phase 6B 完整本地门禁已全部通过,Design/Plan 已标记为 +“本地实现并验证,远程 CI 待确认”。已审查全部现有 Spec/Plan;没有尚未执行的本地任务, +Phase 5 与 Phase 6B 的远程三平台交付验证仍未由当前证据证明。 ## 已完成工作 @@ -37,6 +38,8 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–10 项均已在当前分支本 10 个测试二进制/140 项测试及 readiness 结果;Phase 6A Design/Plan 和数据报告也与实现一致。 - 根据本地 `main` 的 PR #1–#4 合并提交,纠正 bootstrap、RunLoop、Timer、Cancellation 早期 Design/Plan 的陈旧状态;未推断未留存的 Timer 远程检查明细。 +- 完成 13 份 Spec 与 13 份 Plan 的逐项审计:全部有明确状态、没有未勾选任务,本地 Markdown + 链接完整;11 个公开模块分区、10 个测试文件及 140 项测试与文档一致。 - 完成 Phase 6B 依赖核查:mcpp-index 当前提供 `chriskohlhoff.asio@1.38.1`,其 C++23 模块 `asio` 覆盖 Linux、macOS 和 Windows,能够复用 epoll、kqueue 与 IOCP 后端。 - 新增并批准 Phase 6B TCP client v1 Design,确定首个原生异步 I/O 切片仅包含数值地址 TCP @@ -213,18 +216,15 @@ Phase 6A blocking offload v1 与 Phase 6B 第 1–10 项均已在当前分支本 数量。 - 返回 Scheduler 的 owner 必须持续存活;RunLoop Scheduler 还必须处于 active `run()` 中。 - `run_blocking()` 是线程隔离,不是 epoll、io_uring、kqueue 或 IOCP 等原生异步 I/O。 -- 三平台兼容性仍需远程 CI 确认。 +- Phase 5 ThreadPool 与 Phase 6B TCP 的三平台交付验证仍需远程 CI 确认。 - `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 Windows 仍需后续远程 CI 验证。 -- Phase 6B 本地已有 backend、`IoContext`、native bridge、connect/read/write、EOF/overlap、 - pending cancellation、close/is_open、shutdown 竞态、32 客户端负载、readiness TCP client - 迁移及六份开发文档;最终本地矩阵仍待后续步骤。 - 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 ## 剩余工作 1. 本地开发与验证已完成;后续只有在用户另行授权 push/PR 后,才能取得 Linux、macOS、 - Windows 远程 CI 结果并达到 Design 的最终跨平台完成条件。 + Windows 结果,完成 Phase 5 与 Phase 6B 剩余的远程交付验证。 ## 推荐下一步 From 7c81a1002e85c5b2cb2cccb4d53bf8f322219c0d Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 02:25:40 +0800 Subject: [PATCH 15/17] =?UTF-8?q?=E4=BF=AE=E5=A4=8D=20CI=20=E5=86=B7?= =?UTF-8?q?=E7=BC=93=E5=AD=98=E5=B7=A5=E5=85=B7=E9=93=BE?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 280 +++++------------- .github/workflows/ci-linux.yml | 6 +- .github/workflows/ci-macos.yml | 3 +- .github/workflows/ci-windows.yml | 5 +- .xlings.json | 2 +- .../2026-08-30-cmp-ci-cold-cache-toolchain.md | 25 ++ ...8-30-cmp-ci-cold-cache-toolchain-design.md | 66 +++++ 7 files changed, 179 insertions(+), 208 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md create mode 100644 docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 6e61235..733f60f 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -2,230 +2,108 @@ ## 项目概览 -CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 `mcpplibs.cmp`。当前已实现 -懒启动单消费者 `Task`、变参/vector `when_all()`、静止点 `TaskGroup`、一次性与可复用 -事件、RAII `AsyncMutex`、带定时和取消的调用线程 `RunLoop`、固定大小的 `ThreadPool`,以及 -用于隔离同步调用的 `run_blocking()`。Phase 6B 已加入拥有单一私有 I/O driver 的 -`IoContext`,以及数值地址 TCP client 的 connect/read/write、取消和关闭生命周期。 +CMP 是使用 mcpp 构建的 C++23 Modules 协程运行时库,公开模块为 `mcpplibs.cmp`。 +v1 当前包含: -`.xlings.json` 固定 mcpp 2026.8.11.2;当前工具链为 LLVM 22.1.8,运行时依赖为 -`chriskohlhoff.asio` 1.38.1,测试依赖为 `compat.gtest` 1.15.2。`examples/basic` 是独立 -path-dependency consumer。 +- 懒启动、单消费者 `Task`; +- 变参/vector `when_all()` 与静止点 `TaskGroup`; +- `OneShotEvent`、`AsyncManualResetEvent`、RAII `AsyncMutex`; +- 带定时和 `std::stop_token` 取消的调用线程 `RunLoop`; +- 固定大小 `ThreadPool` 与结构化同步调用隔离 `run_blocking()`; +- 单私有 I/O driver 的 `IoContext`,以及数值地址 TCP client `TcpStream`。 + +项目固定 mcpp `2026.8.28.1`、LLVM `22.1.8`、Asio `1.38.1` 与 gtest `1.15.2`。 +三平台 CI 引导 xlings 固定为 `2026.8.27.5`。`examples/basic` 是独立 path-dependency +consumer。 ## 当前目标与状态 -Phase 4 completion、Phase 5 ThreadPool、Phase 6A blocking offload 与 Phase 6B 第 1–10 项均已 -本地完成;本阶段新增提交尚未推送。Phase 6B 完整本地门禁已全部通过,Design/Plan 已标记为 -“本地实现并验证,远程 CI 待确认”。已审查全部现有 Spec/Plan;没有尚未执行的本地任务, -Phase 5 与 Phase 6B 的远程三平台交付验证仍未由当前证据证明。 +Phase 4、Phase 5、Phase 6A 和 Phase 6B 第 1–10 项均已本地完成。当前分支为 +`feature/phase4-v1-readiness`,PR #10 已创建: + + + +PR 初次检查中,macOS arm64 与 Windows x86_64 已通过;Linux x86_64 在编译 CMP 前因 +mcpp/xlings 冷缓存运行时绑定失配失败。专用 Spec/Plan 已完成、自审并通过本地门禁;修复正 +交付到 PR #10,新的三平台结果待确认。用户已授权本次远程读取、当前分支推送和 PR 操作, +未授权合并。 ## 已完成工作 -- 新增 `mcpplibs.cmp:blocking` 分区并由根模块导出一个公共函数模板: - `run_blocking(blockingWorkers, returnTo, operation, stopToken)`。 -- helper 按值持有 callable,支持 `void`、可移动值、move-only callable/result 和异常传播; - callable 在指定 ThreadPool 上执行一次,结果只在显式返回 Scheduler 上发布。 -- 复用现有 `ThreadPool` 的调度、取消、队列和关闭契约;没有新增 `BlockingPool`、隐藏全局 - executor、future、类型擦除、依赖或 manifest 配置。 -- 新增 11 项确定性测试,覆盖懒启动、worker/返回线程亲和、返回另一 ThreadPool、值/void、 - move-only、异常、预取消/排队取消/晚取消、取消竞态、过期与 inactive Scheduler、RunLoop - 响应性,以及 5,000 个并发 offload 的 exactly-once 行为。 -- `examples/basic` 新增独立 blocking ThreadPool,并在协程中打印 `Blocking result: 42`。 -- `benchmarks/v1-readiness` 的文件场景及同步回环 server 使用一个专用 CMP ThreadPool 和 - `run_blocking()`;网络 client 已迁移到共享 `IoContext` 与 `TcpStream`,负载及成功/失败 - 硬检查保持不变。 -- 三份 README 与三份架构文档已同步 `IoContext`/`TcpStream` API、生命周期、并发/取消边界、 - 10 个测试二进制/140 项测试及 readiness 结果;Phase 6A Design/Plan 和数据报告也与实现一致。 -- 根据本地 `main` 的 PR #1–#4 合并提交,纠正 bootstrap、RunLoop、Timer、Cancellation 早期 - Design/Plan 的陈旧状态;未推断未留存的 Timer 远程检查明细。 -- 完成 13 份 Spec 与 13 份 Plan 的逐项审计:全部有明确状态、没有未勾选任务,本地 Markdown - 链接完整;11 个公开模块分区、10 个测试文件及 140 项测试与文档一致。 -- 完成 Phase 6B 依赖核查:mcpp-index 当前提供 `chriskohlhoff.asio@1.38.1`,其 C++23 模块 - `asio` 覆盖 Linux、macOS 和 Windows,能够复用 epoll、kqueue 与 IOCP 后端。 -- 新增并批准 Phase 6B TCP client v1 Design,确定首个原生异步 I/O 切片仅包含数值地址 TCP - 客户端、`IoContext`、move-only `TcpStream`、`read_some()`、`write_all()` 和显式返回 - Scheduler。 -- Spec 已明确 buffer/handle 生命周期、单读单写并发、EOF/partial transfer、错误映射、 - stop/close/completion 竞态、exactly-once、context drain/join 及三平台确定性测试门槛。 -- 新增 Phase 6B Implementation Plan,将实现拆为 dependency/backend gate、`IoContext` - 生命周期、单一 native-completion bridge、connect/read/write、取消/close/shutdown 竞态、 - 跨平台负载、readiness 客户端迁移、文档和完整验证十个顺序步骤。 -- 使用 mcpp 2026.8.11.2 精确加入 `[dependencies.chriskohlhoff] asio = "1.38.1"`。 -- 新增 `mcpplibs.cmp:tcp` 分区,私有导入 `std`、`asio`、`:cancellation` 和 `:task`;根模块 - 只重导出 `:tcp`,没有暴露 Asio 类型。 -- 实现不可复制、不可移动的公共 `IoContext`:共享私有状态拥有一个 `asio::io_context`、 - persistent work guard、admission mutex/flag 和线性弱 socket registry;公开对象拥有一个 - `std::jthread` driver。 -- `IoContext` shutdown 与普通 post 共用 admission 锁排序:先停止接纳,再在已接纳工作后关闭 - live socket、释放 guard、自然排空 handler 并 join;不调用 `io_context::stop()`,driver - 线程内自析构会终止而不是 self-join。 -- 新增 `tests/tcp_test.cpp`,以编译期断言锁定 `IoContext` 的默认构造及精确 copy/move traits, - 并以运行测试验证构造、driver 启动、析构关闭和 join 不挂起。 -- 实现唯一的私有 `NativeOperationAwaiter`:一个共享 operation state 保存 continuation、Asio - cancellation signal、原始 error/count、取消来源、同步 initiation 异常和 stop callback。 -- completion handler 以一参数/两参数重载把 connect 与 read/write 统一为 - `std::error_code + transferred bytes`;native initiation 成功后,只有该 handler 可以恢复 - 私有协程,stop/close/shutdown 只能请求 native completion。 -- stop callback 只持有 operation/context 弱引用,并把 stop-token cancellation 排到 I/O 线程 - 后再记录来源、发出 `cancellation_type::all`;context 已停止接纳时由 shutdown close 路径 - 完成 operation。 -- Step 3 曾加入内部具体模板实例作为 bridge 编译门禁,Step 4 接入真实 connect initiation 后已 - 删除;没有为私有 bridge 暴露测试 API。 -- 新增不可默认构造、不可复制、可 `noexcept` 移动构造且不可移动赋值的 `TcpStream`;公开 - `connect()` 是非协程包装器,在返回 lazy Task 前复制地址、Scheduler、token 与 context 弱句柄。 -- `connect()` 被 context 接纳后才在 I/O 线程检查预取消、解析 numeric IPv4/IPv6、创建并注册 - socket、发起 `async_connect()`;预取消不解析地址或打开 socket,系统错误保留原 error code。 -- native operation state 新增私有生命周期锚点,保证临时 connect socket 存活到 handler;成功 - 后先经显式 return Scheduler 再发布 move-only stream,return Scheduler 失败由内部 RAII 关闭。 -- `tests/tcp_test.cpp` 扩展为单一同步 Asio loopback fixture,覆盖 laziness/临时地址所有权、IPv4、 - 可用时 IPv6、拒绝连接、预取消、RunLoop/ThreadPool 亲和及 inactive/expired Scheduler。 -- 新增 lazy `read_some()` 与 `write_all()` 非协程包装器;buffer span、Scheduler、token 与 socket - state 均在调用时按值捕获,实际 buffer 仍按 API 契约借用到 Task 完成。 -- socket state 使用每方向一个 atomic admission flag,允许一读一写并行并拒绝同方向重叠;flag - 保持到 return-Scheduler 跳转结束,跳转成功或失败均释放。 -- read 实现 empty、partial、bytes-before-error、一次 retained fatal error 与 sticky EOF;远端 EOF - 不关闭写方向,后续整条流关闭又不会被旧 sticky EOF 掩盖。 -- write 使用 Asio `async_write()` 完成全 buffer;已发起 write 的取消或错误关闭 stream,成功计数 - 不等于 buffer 大小时视为内部不变量破坏。 -- 当前 libc++ `std` 模块不提供 C++23 `std::scope_exit`;按实测改为在 return-Scheduler 成功与 - 异常两条路径显式释放 admission,没有新增 guard abstraction 或依赖。 -- 同一 loopback fixture 增加小型同步 protocol callback、partial/EOF/duplex gate 与 - `reuse_address`,没有增加 public server 或固定 sleep。 -- 新增线程安全、幂等、非阻塞且 `noexcept` 的 `TcpStream::close()` 与 `is_open()`;析构复用 - 同一关闭请求,move 后仅新 handle 保有关闭所有权。 -- socket 以弱引用记录 connect/read/write operation;stop、显式 close、write cancellation 与 - context shutdown 在 I/O 线程记录首个本地取消来源,再由唯一 native handler 恢复协程。 -- 显式关闭请求使用原子标记覆盖公开校验到 native initiation 的窄窗口;已接纳操作在 close - 获胜时稳定得到 `OperationCancelled`,而无关 socket error 仍保留 `system_error`。 -- TCP 测试新增 active/pre-cancellation、关闭幂等、move、write cancellation、两类完成竞态、 - context drain/join 和 surviving handle 边界,该阶段使 `tcp_test` 增至 23 项。 -- 现有 loopback fixture 支持固定连接数;新增 32 客户端并发 echo 测试,每个 Task 独立持有 - stream/缓冲区并逐槽核对完成次数、结果、payload 与 RunLoop 返回线程。 -- readiness 的四个 client 各保持一条长连接完成 5,000 次 256B echo,共享一个 I/O driver; - 四个同步 POSIX server 仍各占一个 blocking worker,100 次故障连接只接受 - `connection_refused`。 +- Phase 4 补齐定时、取消、结构化并发、事件和互斥原语。 +- Phase 5 实现固定大小多 worker `ThreadPool` 并完成基准验证。 +- Phase 6A 实现显式 blocking worker 与返回 Scheduler 的 `run_blocking()`。 +- Phase 6B 实现一个跨平台原生异步 TCP client 切片:numeric connect、partial read、 + write-all、单读单写并发、取消、关闭、context drain/join、return affinity 和 exactly-once + 完成;未加入 DNS、server、TLS、UDP、文件 I/O、timeout 或隐式全局 executor。 +- 当前共有 11 个公开模块分区、10 个测试文件和 140 项测试。 +- readiness benchmark 的计算、文件 I/O 与网络回环场景均有成功/预期失败硬计数;网络 client + 使用 `IoContext`/`TcpStream`,同步 server 通过 `run_blocking()` 隔离。 +- 审查全部 13 份既有 Spec/Plan 后,已纠正早期状态;Phase 6B 本地任务全部完成。 +- 创建 PR #10 并取得首轮远程结果:macOS、Windows 成功;Linux 在 `Build library` 前置 + 工具链初始化失败,CMP 源码与测试未开始执行。 +- 根因是 mcpp `2026.8.11.2` 携带的旧 xlings 声明 `glibc@2.44`,而当前索引安装 + `2.44.2`。官方 mcpp `2026.8.28.1` 固定 xlings `2026.8.27.5` 并包含版本精化兼容路径。 +- 自审时纠正“只提升 mcpp”的初稿:Linux 日志证明 mcpp 冷启动会复制 workflow 的系统 + xlings,因此同时提升 `.xlings.json` 与三个 workflow pin,其他步骤不变。 ## 重要决策 -- 独立的普通 `ThreadPool` 实例就是 blocking pool;不维护行为相同的第二种线程池类型。 -- `run_blocking()` 是懒 `Task`,callable 和两个 Scheduler 都按值进入协程帧,避免临时对象 - 悬空。 -- 第一次调度接受可选 `std::stop_token`;取消只能跳过尚未被 worker claim 的 callable。 - 已经开始的同步调用不能被抢占。 -- 返回调度不接受取消 token,确保值、异常或取消结果不会滞留在 blocking worker。 -- 返回 Scheduler 失败时直接传播其异常;失效的完成上下文本身无法被 helper 修复。 -- Phase 6A 仍是每个运行中同步调用占用一个系统线程的隔离方案,不宣称原生非阻塞 I/O。 -- 队列沿用 ThreadPool 的无界共享 FIFO;背压、超时、强制中断和 worker replacement 均未在 - 没有实测需求前增加。 -- Phase 6B 先做 TCP 而非文件:现有 Asio package 的 socket 能统一覆盖三平台,file backend - 不能提供同等平台面。 -- C++23 标准库没有网络 API,因此 6B 选择精确固定 `chriskohlhoff.asio@1.38.1`,私有导入, - 不向 CMP 公共 API 暴露 Asio 类型;不自研三套 OS backend。 -- `IoContext` v1 固定一个私有 driver thread,不暴露通用 post/run/线程数接口;应用结果始终 - 经调用者指定 Scheduler 返回。 -- socket state 只弱引用 context;共享 context state 由公开 `IoContext` 与 driver 持有, - socket 不能延长公开 driver 生命周期。v1 registry 使用 O(n) 弱引用扫描,只有实测成本显著 - 时才替换。 -- native initiation 成功前的同步异常可以直接恢复私有协程;成功后恢复权只属于 Asio handler。 - stop callback 不从请求线程 emit、访问 socket 或恢复 coroutine。 -- 显式 close 立即发布逻辑关闭状态,再把取消与 native close 排到 I/O 线程;资源失效检查先于 - pre-cancellation,远端 EOF 不关闭本地写方向。 -- `TcpStream` 允许一项 pending read 与一项 pending write;同方向重叠直接拒绝,不增加隐式 - 排队。主动 close、context shutdown 与已发起 write 的取消均有明确关闭语义。 -- v1 不含 DNS、server、TLS、UDP、文件 I/O、timeout、socket option 或隐式 executor;这些都 - 没有在首个可验证切片前预建。 -- 新测试只在 `tests/tcp_test.cpp` 内用同步 Asio loopback fixture;不为测试增加公共 server。 -- 现有三平台 CI 会自动发现新测试,不预先修改 workflow;`examples/basic` 保持短小且不引入 - live network,仍作为外部 path consumer 回归运行。 -- readiness 只替换 TCP client;同步 loopback server 继续通过 Phase 6A `run_blocking()` - 运行,compute/file workload 与硬计数保持不变。 +- 只修改有证据的工具版本:mcpp `2026.8.28.1`、CI xlings `2026.8.27.5`;不升级到包含 + 无关构建规则变化的 mcpp `2026.8.29.1`。 +- 不用固定旧 glibc、增加缓存或手工安装载荷来隐藏冷缓存失配。 +- LLVM、Asio、gtest、CMP API/源码和 workflow 执行流程保持不变。 +- Asio 仅作私有三平台 socket backend,公共 API 不暴露 Asio 类型。 +- `IoContext` v1 保持单 driver;只有基准证明瓶颈后才考虑多 driver/strand。 +- `run_blocking()` 隔离同步调用,但不宣称可抢占,也不替代原生异步 I/O。 ## 修改 / 重要文件 -- 当前实现:`mcpp.toml`、`src/tcp.cppm`、`src/cmp.cppm` -- Phase 6A 核心:`src/blocking.cppm` -- 测试:`tests/blocking_test.cpp`、`tests/tcp_test.cpp` -- 示例:`examples/basic/src/main.cpp` -- 压测:`benchmarks/v1-readiness/src/main.cpp`、 - `docs/benchmarks/2026-08-29-cmp-v1-readiness.md` -- 方案:`docs/superpowers/specs/2026-08-29-cmp-phase6a-blocking-offload-v1-design.md`、 - `docs/superpowers/plans/2026-08-29-cmp-phase6a-blocking-offload-v1.md` -- 已批准设计: - `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -- 当前实施计划: +- 工具环境:`.xlings.json` +- CI:`.github/workflows/ci-linux.yml`、`ci-macos.yml`、`ci-windows.yml` +- 当前修复设计: + `docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md` +- 当前修复计划: + `docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md` +- Phase 6B:`src/tcp.cppm`、`tests/tcp_test.cpp`、 + `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md`、 `docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md` -- 公共文档:`README.md`、`README.zh.md`、`README.zh.hant.md`、`docs/architecture.md`、 - `docs/architecture.zh.md`、`docs/architecture.zh.hant.md` +- 当前状态:`.agent/HANDOFF.md` ## 验证情况 -- Phase 6A baseline 的 `mcpp build --profile dev --strict --cache=off`:通过。 -- Phase 6A baseline 的 `mcpp test --profile dev --strict --cache=off`:9 个二进制、116/116 - 通过。 -- Phase 6A baseline 的 `mcpp build --profile release --strict --cache=off`:通过。 -- Phase 6A baseline 的 `mcpp test --profile release --strict --cache=off`:9 个二进制、 - 116/116 通过。 -- Dev 定向 `blocking_test`:11/11 通过。 -- Release `CancellationRaceInvokesAtMostOnce` 连续执行 100 轮:100/100 通过。 -- `examples/basic` 的 `mcpp run`:通过,包含 `Blocking result: 42`,退出码 0。 -- Step 9 文档同步后再次运行未改动的 `examples/basic`:构建并运行通过,全部预期输出一致, - 退出码 0。 -- `benchmarks/v1-readiness` Phase 6B client 迁移后 Release strict cache-off 构建通过;最终 - 执行 5 轮,compute、file_io 和 network_loopback 每轮均 PASS,五轮非预期失败总数为 0。 -- 文件/网络每轮计数分别为 1,000/20,000 成功、100/100 预期失败、0 非预期失败;原始耗时和 - 吞吐已写入 benchmark 报告。 -- 当前 mcpp 仍输出 SubOS 缺少 `subos_info` 的既有环境提示,但所有构建和运行成功。 -- 以上均为本机 Linux/WSL2 结果;尚未执行 GitHub 三平台 CI 或其他远程操作。 -- Phase 6B backend gate:`mcpp build --profile dev --strict --cache=off` 通过;mcpp 下载并编译 - `chriskohlhoff.asio` 1.38.1,CMP 使用 LLVM 22.1.8 构建成功。 -- Phase 6B `IoContext` focused gate:`mcpp test tcp_test --profile dev --strict --cache=off` 的 - 1/1 测试通过。 -- 加入 `IoContext` 后,`mcpp build --profile dev --strict --cache=off` 再次通过; - `mcpp test --profile dev --strict --cache=off` 为 10 个二进制、117/117 通过。 -- 加入 native bridge 后,`mcpp build --profile dev --strict --cache=off` 与 - `mcpp build --profile release --strict --cache=off` 均通过;Dev 与 Release 全量测试均为 10 个 - 二进制、117/117 通过。 -- connect 定向 `tcp_test` 在 Dev 与 Release 均为 8/8 通过;Release binary 重复执行 100 轮, - 800/800 用例通过。 -- 加入 connect 后,Dev 与 Release 的 strict cache-off 全量测试均为 10 个二进制、124/124 通过。 - 当前未重复执行 example 或 benchmark;它们保留到 Phase 6B 完整 API 与 readiness 迁移门禁。 -- read/write 定向 `tcp_test` 在 Dev 与 Release 均为 15/15 通过;Release 整套重复 10 轮为 - 150/150,通过两个 overlap test 重复 50 轮为 100/100。 -- 加入 read/write 后,Dev 与 Release 的 strict cache-off 全量测试均为 10 个二进制、131/131 - 通过。 -- 曾尝试 Release 整套重复 100 轮;前 24 轮 360/360 通过,第 25 轮因本机临时端口范围仅 - `60700–61000` 而出现 `Address already in use`。fixture 随后增加 `reuse_address`,上述 10 轮 - 整套与 50 轮 overlap 复验通过;该次资源耗尽不计为功能通过,也未隐藏。 -- close/cancellation/shutdown 定向 `tcp_test` 在 Dev 与 Release 均为 23/23 通过;两套 strict - cache-off 全量测试均为 10 个二进制、139/139 通过。 -- Release 两项完成竞态连续执行 5 轮共 10/10 测试通过,覆盖 100 次 close/completion 与 - 500 次 stop/completion 竞态,没有丢失或重复完成。 -- 加入并发负载后,`tcp_test` Dev/Release 均为 24/24;两套 strict cache-off 全量测试均为 - 10 个二进制、140/140。32/32 客户端成功,取消、错误、payload/线程不匹配和重复完成均为 0。 -- Phase 6B 最终门禁:focused Release 10/10 通过,覆盖 100 次 close 与 500 次 stop completion - 竞态;Dev/Release strict cache-off 构建均通过,两套全量测试均为 10 个二进制、140/140。 -- 最终门禁再次运行独立示例成功;readiness Release strict cache-off 构建成功,随后五轮共 - 15 个场景全部 PASS。每轮 compute/file/network 非预期失败均为 0,网络中位数为 - 2,078.127 ms / 9,672.2 ops/s,最新原始数据已写入 benchmark 报告。 +- `xlings update && xlings install -y`:成功;`mcpp --version` 为 `2026.8.28.1`。 +- `mcpp build --profile dev --strict --cache=off`:通过,LLVM `22.1.8`。 +- `mcpp test --profile dev --strict --cache=off`:10 个二进制、140/140 通过。 +- `mcpp build --profile release --strict --cache=off`:通过。 +- `mcpp test --profile release --strict --cache=off`:10 个二进制、140/140 通过。 +- `examples/basic` 的 `mcpp run`:通过,全部十行既有示例输出正确,退出码 0。 +- Phase 6B 最终本地竞态门禁:100 次 close/completion 与 500 次 stop/completion 无丢失或 + 重复完成。 +- readiness Release 五轮共 15 行场景全部 PASS,非预期失败为 0;网络中位数 + 2,078.127 ms / 9,672.2 ops/s。 +- PR #10 首轮 CI:macOS、Windows 通过;Linux 因旧工具冷缓存失配失败。修复后的远程 CI + 尚未执行。 ## 已知问题 / 风险 -- 运行中的同步调用不可抢占;永久阻塞会永久占用 worker,并使等待它的 Task 和 pool 析构 - 无法完成。callable 如需协作取消,必须自行捕获并检查 token。 -- ThreadPool 使用无界共享 FIFO;持续生产快于消费时,应用需要限制自己的结构化 in-flight - 数量。 -- 返回 Scheduler 的 owner 必须持续存活;RunLoop Scheduler 还必须处于 active `run()` 中。 -- `run_blocking()` 是线程隔离,不是 epoll、io_uring、kqueue 或 IOCP 等原生异步 I/O。 -- Phase 5 ThreadPool 与 Phase 6B TCP 的三平台交付验证仍需远程 CI 确认。 -- `chriskohlhoff.asio@1.38.1` 只在本机 Linux/WSL2 + LLVM 22.1.8 完成模块编译;macOS 与 - Windows 仍需后续远程 CI 验证。 -- 单 I/O driver 是 v1 的刻意简化;只有 benchmark 证明它是瓶颈后才设计多 driver/strand。 +- 本机 `~/.mcpp` 仍保存旧 xlings `2026.8.11.2` 且缺少 `subos_info`;mcpp + `2026.8.28.1` 会提示 runtime facts inconclusive,但本地 LLVM 双配置构建、测试与示例 + 均成功。没有为本次验证修改用户的全局 xlings;远程 workflow 会在冷环境直接安装 + `2026.8.27.5`。 +- 工具版本修复只有在新一轮 Linux 冷缓存 CI 通过后才能视为完成。 +- 运行中的 blocking callable 不可抢占;永久阻塞会占用 worker。 +- ThreadPool 是无界共享 FIFO;应用需要限制持续生产时的结构化 in-flight 数量。 +- 单 I/O driver 是 v1 简化;尚无基准证据要求扩展。 ## 剩余工作 -1. 本地开发与验证已完成;后续只有在用户另行授权 push/PR 后,才能取得 Linux、macOS、 - Windows 结果,完成 Phase 5 与 Phase 6B 剩余的远程交付验证。 +1. 将本地修复交付到 PR #10,并等待 Linux、macOS、Windows 新一轮检查全部通过。 +2. 按实际远程结果更新 README、Phase 6B/工具修复文档与本 HANDOFF;再次提交、推送并确认 + 最终 CI。 +3. 不合并 PR #10,除非用户另行明确授权。 ## 推荐下一步 -等待用户另行授权远程操作;在此之前不 push、不创建 PR,也不宣称三平台 CI 已通过。 +提交并推送当前冷缓存工具链修复,然后监控 PR #10 三平台检查;若失败,只处理日志证实的 +兼容问题。 diff --git a/.github/workflows/ci-linux.yml b/.github/workflows/ci-linux.yml index 0f8fc66..bb0b3e5 100644 --- a/.github/workflows/ci-linux.yml +++ b/.github/workflows/ci-linux.yml @@ -23,9 +23,9 @@ concurrency: env: # Pinned rather than "newest": a bootstrap that floats turns an upstream - # release into a red build on an unrelated PR. Floor is xlings 0.4.69 - # (the index keys by (namespace, name) from there on) — never pin below it. - XLINGS_VERSION: v2026.8.11.2 + # release into a red build on an unrelated PR. 2026.8.27.5 is the cold-cache + # floor: older clients can declare glibc 2.44 while installing 2.44.2. + XLINGS_VERSION: v2026.8.27.5 XLINGS_NON_INTERACTIVE: '1' jobs: diff --git a/.github/workflows/ci-macos.yml b/.github/workflows/ci-macos.yml index 6575740..b512808 100644 --- a/.github/workflows/ci-macos.yml +++ b/.github/workflows/ci-macos.yml @@ -19,7 +19,8 @@ concurrency: cancel-in-progress: true env: - XLINGS_VERSION: v2026.8.11.2 + # Keep aligned with the Linux cold-cache floor. + XLINGS_VERSION: v2026.8.27.5 XLINGS_NON_INTERACTIVE: '1' jobs: diff --git a/.github/workflows/ci-windows.yml b/.github/workflows/ci-windows.yml index 8f5ca3a..7b76dfa 100644 --- a/.github/workflows/ci-windows.yml +++ b/.github/workflows/ci-windows.yml @@ -21,8 +21,9 @@ concurrency: env: # Both installers read this: the shell one takes it as an argument or an env - # var, the PowerShell one defaults its -Version parameter to it. - XLINGS_VERSION: v2026.8.11.2 + # var, the PowerShell one defaults its -Version parameter to it. Keep this + # aligned with the Linux cold-cache floor. + XLINGS_VERSION: v2026.8.27.5 XLINGS_NON_INTERACTIVE: '1' jobs: diff --git a/.xlings.json b/.xlings.json index 1fb0f9c..3fc6049 100644 --- a/.xlings.json +++ b/.xlings.json @@ -1,5 +1,5 @@ { "workspace": { - "mcpp": "2026.8.11.2" + "mcpp": "2026.8.28.1" } } diff --git a/docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md b/docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md new file mode 100644 index 0000000..f75e818 --- /dev/null +++ b/docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md @@ -0,0 +1,25 @@ +# CMP CI 冷缓存工具链修复计划 + +**日期:** 2026-08-30 +**设计:** `docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md` +**状态:** 执行中 + +## 执行规则 + +只处理 PR #10 已证实的冷缓存工具链失配。每一步先验证再继续;不修改 CMP 源码、依赖、 +编译器版本或未失败的 workflow 引导流程。 + +## 任务 + +- [x] 核对 PR #10 三平台结果与 Linux 失败步骤。 +- [x] 核对 mcpp 官方修复提交、版本边界与 xlings 版本下限。 +- [x] 自审设计:发现 mcpp 沙箱会复制 workflow 的 xlings,纠正为同步提升两个 pin。 +- [x] 将 `.xlings.json` 的 mcpp 固定为 `2026.8.28.1`。 +- [x] 将三个 workflow 的引导 xlings 固定为 `2026.8.27.5`,不改执行流程。 +- [x] 安装项目工具并确认实际 mcpp 版本。 +- [x] 运行 Dev/Release 严格无缓存构建和完整测试:两套均为 140/140。 +- [x] 运行 `examples/basic`,全部既有输出与退出状态正确。 +- [x] 自审最终差异并更新 HANDOFF;纠正 workflow 中已过时的版本下限注释。 +- [ ] 本地提交并推送当前分支。 +- [ ] 等待 PR #10 的 Linux、macOS、Windows 检查全部通过。 +- [ ] 根据实际远程结果同步项目当前状态文档。 diff --git a/docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md b/docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md new file mode 100644 index 0000000..3bed5e2 --- /dev/null +++ b/docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md @@ -0,0 +1,66 @@ +# CMP CI 冷缓存工具链修复设计 + +**日期:** 2026-08-30 +**状态:** 本地实现并验证,远程 CI 待确认 +**基线:** PR #10,提交 `1804044` + +## 问题 + +PR #10 的 macOS arm64 与 Windows x86_64 检查通过;Linux x86_64 在 `Build library` +阶段、尚未编译 CMP 代码前失败: + +```text +selected RuntimeBinding glibc@2.44 requires payload +'.../xim-x-glibc/2.44', but it is not installed +``` + +冷缓存中实际安装的是 `2.44.2`。项目固定的 mcpp `2026.8.11.2` 携带旧版 xlings, +运行时绑定使用编译期常量 `2.44`,而当前索引把该请求解析为 `2.44.2`,两种版本身份 +因此失配。本机已有缓存,所以此前本地验证无法复现。 + +## 选择 + +同时提升互相配套的两个精确版本: + +- `.xlings.json`:mcpp `2026.8.11.2` → `2026.8.28.1`; +- 三个平台 workflow:引导 xlings `2026.8.11.2` → `2026.8.27.5`。 + +mcpp 官方提交 `473ee0520b00` 表明 `2026.8.28.1` 是首个随包固定 xlings +`2026.8.27.5` 的 mcpp 版本;该 xlings 从索引读取运行时绑定,并保证声明版本优先于 +索引的 `latest`。这直接修复冷缓存中“声明 2.44、安装 2.44.2”的根因。 + +不选其他方案: + +- 不固定旧 glibc 载荷,也不增加 CI 缓存;两者只会隐藏索引与绑定失配。 +- 不使用 mcpp `2026.8.27.1`;它已能兼容唯一的版本精化,但尚未携带最终固定的 + xlings 版本。 +- 不升级到当前最新的 mcpp `2026.8.29.1`;它包含与本问题无关的构建规则改动。 +- 不只提升 mcpp。失败日志显示 mcpp 首次初始化会把 workflow 提供的系统 xlings 复制进 + 私有沙箱;保留旧引导版本会继续制造错误绑定,只能依赖新版 mcpp 的兼容回退。 + +## 修改边界 + +- 修改 `.xlings.json` 与三个 workflow 中的四个精确版本值及必要的下限注释;workflow + 步骤不变。 +- 保持 LLVM `22.1.8`、Asio `1.38.1`、gtest `1.15.2`、CMP API 与源码不变。 +- 记录本次诊断、验证和远程 CI 结果;只有取得结果后才更新“三平台待确认”的文档状态。 +- 不合并 PR #10。 + +## 验收 + +1. `xlings install -y` 成功,`mcpp --version` 报告 `2026.8.28.1`。 +2. Dev 与 Release 严格、关闭缓存的构建和 140 项测试全部通过。 +3. `examples/basic` 构建运行成功并输出既有示例结果。 +4. 推送后 PR #10 的 Linux、macOS、Windows 三项检查全部通过。 + +## 风险 + +mcpp 尚未到 1.0,工具版本提升可能改变模块扫描或清单校验。完整本地双配置门禁与远程 +三平台门禁共同约束该风险;若任一门失败,只修复有证据的兼容问题,不扩大 CMP API。 + +## 自审结论 + +最初方案只提升项目 mcpp。复查 Linux 日志与 mcpp `acquire_xlings_binary()` 后确认,冷启动 +时沙箱 xlings 来自 workflow 的系统版本;因此只提升 mcpp 虽可依赖版本精化回退,但没有 +消除错误绑定的来源。最终方案同时固定官方配套的 mcpp `2026.8.28.1` 与 xlings +`2026.8.27.5`,其余边界不变。 From 4a909ed5ecbd75719f59b9b760eaefb0484341ff Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 02:32:48 +0800 Subject: [PATCH 16/17] =?UTF-8?q?=E8=AE=B0=E5=BD=95=E4=B8=89=E5=B9=B3?= =?UTF-8?q?=E5=8F=B0=20CI=20=E9=AA=8C=E8=AF=81=E7=BB=93=E6=9E=9C?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 23 +++++++++---------- README.md | 4 ++-- README.zh.hant.md | 4 ++-- README.zh.md | 4 ++-- .../2026-08-29-cmp-phase5-thread-pool-v1.md | 4 +++- .../2026-08-30-cmp-ci-cold-cache-toolchain.md | 8 +++---- .../2026-08-30-cmp-phase6b-tcp-client-v1.md | 13 +++++++---- ...-08-29-cmp-phase5-thread-pool-v1-design.md | 8 +++---- ...-08-29-cmp-phase6b-tcp-client-v1-design.md | 2 +- ...8-30-cmp-ci-cold-cache-toolchain-design.md | 8 ++++++- 10 files changed, 45 insertions(+), 33 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 733f60f..e9f67d5 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -24,9 +24,9 @@ Phase 4、Phase 5、Phase 6A 和 Phase 6B 第 1–10 项均已本地完成。当 PR 初次检查中,macOS arm64 与 Windows x86_64 已通过;Linux x86_64 在编译 CMP 前因 -mcpp/xlings 冷缓存运行时绑定失配失败。专用 Spec/Plan 已完成、自审并通过本地门禁;修复正 -交付到 PR #10,新的三平台结果待确认。用户已授权本次远程读取、当前分支推送和 PR 操作, -未授权合并。 +mcpp/xlings 冷缓存运行时绑定失配失败。修复提交 `7c81a10` 已推送;随后 Linux、macOS、 +Windows 三项 CI 全部通过,Phase 5 与 Phase 6B 的远程交付门已满足。当前只剩同步最终状态 +文档、推送并确认文档提交后的 CI。用户未授权合并。 ## 已完成工作 @@ -46,6 +46,8 @@ mcpp/xlings 冷缓存运行时绑定失配失败。专用 Spec/Plan 已完成、 `2.44.2`。官方 mcpp `2026.8.28.1` 固定 xlings `2026.8.27.5` 并包含版本精化兼容路径。 - 自审时纠正“只提升 mcpp”的初稿:Linux 日志证明 mcpp 冷启动会复制 workflow 的系统 xlings,因此同时提升 `.xlings.json` 与三个 workflow pin,其他步骤不变。 +- 提交并推送 `7c81a10 修复 CI 冷缓存工具链`;PR #10 的三个冷环境均安装 xlings + `2026.8.27.5` 和 mcpp `2026.8.28.1`,完成构建、10 个测试二进制与独立示例。 ## 重要决策 @@ -82,8 +84,9 @@ mcpp/xlings 冷缓存运行时绑定失配失败。专用 Spec/Plan 已完成、 重复完成。 - readiness Release 五轮共 15 行场景全部 PASS,非预期失败为 0;网络中位数 2,078.127 ms / 9,672.2 ops/s。 -- PR #10 首轮 CI:macOS、Windows 通过;Linux 因旧工具冷缓存失配失败。修复后的远程 CI - 尚未执行。 +- PR #10 首轮 CI:macOS、Windows 通过;Linux 因旧工具冷缓存失配失败,CMP 未开始编译。 +- PR #10 修复后 CI:Linux x86_64、macOS arm64、Windows x86_64 全部通过,耗时分别为 + 2m32s、2m42s、2m10s;各平台均为 10 个测试二进制、0 个失败,示例输出成功。 ## 已知问题 / 风险 @@ -91,19 +94,15 @@ mcpp/xlings 冷缓存运行时绑定失配失败。专用 Spec/Plan 已完成、 `2026.8.28.1` 会提示 runtime facts inconclusive,但本地 LLVM 双配置构建、测试与示例 均成功。没有为本次验证修改用户的全局 xlings;远程 workflow 会在冷环境直接安装 `2026.8.27.5`。 -- 工具版本修复只有在新一轮 Linux 冷缓存 CI 通过后才能视为完成。 - 运行中的 blocking callable 不可抢占;永久阻塞会占用 worker。 - ThreadPool 是无界共享 FIFO;应用需要限制持续生产时的结构化 in-flight 数量。 - 单 I/O driver 是 v1 简化;尚无基准证据要求扩展。 ## 剩余工作 -1. 将本地修复交付到 PR #10,并等待 Linux、macOS、Windows 新一轮检查全部通过。 -2. 按实际远程结果更新 README、Phase 6B/工具修复文档与本 HANDOFF;再次提交、推送并确认 - 最终 CI。 -3. 不合并 PR #10,除非用户另行明确授权。 +1. 提交并推送最终状态文档,确认 PR #10 文档提交后的 Linux、macOS、Windows CI。 +2. 不合并 PR #10,除非用户另行明确授权。 ## 推荐下一步 -提交并推送当前冷缓存工具链修复,然后监控 PR #10 三平台检查;若失败,只处理日志证实的 -兼容问题。 +提交并推送最终状态文档,然后确认 PR #10 三平台 checks 保持全绿;未经授权不合并。 diff --git a/README.md b/README.md index 7ad10cd..0fcc5c4 100644 --- a/README.md +++ b/README.md @@ -433,8 +433,8 @@ Runtime work is split into independently reviewable phases: OneShotEvent, AsyncManualResetEvent, and AsyncMutex — implemented and pressure-tested; 5. fixed-size multi-worker scheduling — implemented and benchmarked; work stealing remains gated by profiling evidence; -6. structured blocking offload — implemented; native async numeric-address TCP client — locally - implemented and verified, with remote three-platform CI still pending. +6. structured blocking offload and native async numeric-address TCP client — implemented and + verified by Linux, macOS, and Windows CI in PR #10. The remaining order is directional, not a promise that a listed feature is already implemented. diff --git a/README.zh.hant.md b/README.zh.hant.md index 452dbd2..7e35a74 100644 --- a/README.zh.hant.md +++ b/README.zh.hant.md @@ -406,8 +406,8 @@ CMP 目前不追蹤 `mcpp.lock`,`.gitignore` 明確執行這項儲存庫約定 4. 單調時鐘 Timer v1、可取消就緒/定時等待、變參/vector 匯合、靜止點 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已實作並完成壓力驗證; 5. 固定大小的多 worker 排程——已實作並完成壓測;work stealing 仍需 profiling 證據; -6. 結構化阻塞 offload——已實作;數值位址原生非同步 TCP client——已完成本機實作與驗證, - 遠端三平台 CI 仍待確認。 +6. 結構化阻塞 offload 與數值位址原生非同步 TCP client——已實作,並由 PR #10 的 Linux、 + macOS 和 Windows CI 驗證。 剩餘順序只是方向,不代表列出的能力已經實作。 diff --git a/README.zh.md b/README.zh.md index 66289b3..c5bf337 100644 --- a/README.zh.md +++ b/README.zh.md @@ -406,8 +406,8 @@ CMP 当前不跟踪 `mcpp.lock`,`.gitignore` 明确执行这一仓库约定。 4. 单调时钟 Timer v1、可取消就绪/定时等待、变参/vector 汇合、静止点 TaskGroup、 OneShotEvent、AsyncManualResetEvent 和 AsyncMutex——已实现并完成压力验证; 5. 固定大小的多 worker 调度——已实现并完成压测;work stealing 仍需 profiling 证据; -6. 结构化阻塞 offload——已实现;数值地址原生异步 TCP client——已完成本地实现与验证, - 远程三平台 CI 仍待确认。 +6. 结构化阻塞 offload 与数值地址原生异步 TCP client——已实现,并由 PR #10 的 Linux、 + macOS 和 Windows CI 验证。 剩余顺序只是方向,不代表列出的能力已经实现。 diff --git a/docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md b/docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md index fac5de2..76ffe9c 100644 --- a/docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md +++ b/docs/superpowers/plans/2026-08-29-cmp-phase5-thread-pool-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-29 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md` -**Status:** Complete; local verification passed +**Status:** Complete; local and Linux/macOS/Windows CI verified ## Completion record @@ -12,6 +12,8 @@ the complete 105-test suite pass; the focused Release suite passed 30 consecutiv benchmark rounds completed 8,410,240 operations with zero unexpected failures. Coarse CPU work scaled by about 7.56x from one to eight workers, while tiny-task contention confirmed the documented v1 ceiling. Work stealing was therefore not added without a separate representative profiling case. +PR #10 later passed the current build, 140-test suite, and standalone example on all three CI +platforms. ## 1. Preserve and re-check the phase-4 baseline diff --git a/docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md b/docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md index f75e818..fb21e82 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md +++ b/docs/superpowers/plans/2026-08-30-cmp-ci-cold-cache-toolchain.md @@ -2,7 +2,7 @@ **日期:** 2026-08-30 **设计:** `docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md` -**状态:** 执行中 +**状态:** 完成 ## 执行规则 @@ -20,6 +20,6 @@ - [x] 运行 Dev/Release 严格无缓存构建和完整测试:两套均为 140/140。 - [x] 运行 `examples/basic`,全部既有输出与退出状态正确。 - [x] 自审最终差异并更新 HANDOFF;纠正 workflow 中已过时的版本下限注释。 -- [ ] 本地提交并推送当前分支。 -- [ ] 等待 PR #10 的 Linux、macOS、Windows 检查全部通过。 -- [ ] 根据实际远程结果同步项目当前状态文档。 +- [x] 本地提交并推送当前分支:`7c81a10 修复 CI 冷缓存工具链`。 +- [x] PR #10 的 Linux、macOS、Windows 检查全部通过。 +- [x] 根据实际远程结果同步项目当前状态文档。 diff --git a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md index aa5ad9a..d554be7 100644 --- a/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md +++ b/docs/superpowers/plans/2026-08-30-cmp-phase6b-tcp-client-v1.md @@ -2,7 +2,7 @@ **Date:** 2026-08-30 **Design:** `docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md` -**Status:** Locally complete — remote CI pending +**Status:** Complete; local and Linux/macOS/Windows CI verified in PR #10 **Baseline:** Local Phase 6A commit `701aa8b`, 116/116 Dev and Release tests ## Execution Rule @@ -441,7 +441,7 @@ Only after local verification: - update `.agent/HANDOFF.md` with implementation facts, remaining remote CI, and known limits. Phase 6B reaches the Design's final completion state only after Linux, macOS, and Windows CI all -pass. Until then, report it as locally implemented and verified with remote compatibility pending. +pass. PR #10 now satisfies that gate. **Self-review — 2026-08-30:** Run the two existing internally repeated Release race tests five times, which exercises 100 close/completion and 500 stop/completion races without multiplying the @@ -450,7 +450,7 @@ invalid outcome, failed command, or hard-count mismatch stops the gate. Then run cache-off strict Dev/Release matrix, the unchanged path-consumer example, and a Release benchmark build followed by five sequential runs of the produced binary. Benchmark timing is evidence only; each run must retain exact compute/file/network success and expected-failure counts with zero -unexpected failures. Record only local Linux/WSL2 facts and leave remote CI pending. +unexpected failures. Record the local Linux/WSL2 facts here; record the remote result separately. **Completion — 2026-08-30:** The focused Release command was: @@ -464,7 +464,12 @@ including 24/24 TCP tests. The unchanged standalone example printed every expect 0. The readiness Release build passed, followed by five sequential runs: each reported compute 49,000/1,000/0, file 1,000/100/0, and network 20,000/100/0 success/expected/unexpected counts. All 15 scenario rows passed; the final raw timings are recorded in the readiness report. These are -local Linux/WSL2 + LLVM 22.1.8 results; Linux, macOS, and Windows remote CI remain pending. +the local Linux/WSL2 + LLVM 22.1.8 results. + +**Remote completion — 2026-08-30:** PR #10 commit `7c81a10` installed xlings `2026.8.27.5` and +mcpp `2026.8.28.1`, then built CMP, passed all 10 test binaries, and ran `examples/basic` on Linux +x86_64, macOS arm64, and Windows x86_64. The three checks completed successfully in 2m32s, 2m42s, +and 2m10s respectively. ## Scope Guard diff --git a/docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md b/docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md index b096845..8fd5cbf 100644 --- a/docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md +++ b/docs/superpowers/specs/2026-08-29-cmp-phase5-thread-pool-v1-design.md @@ -1,7 +1,7 @@ # CMP Phase 5 Thread Pool v1 Design **Date:** 2026-08-29 -**Status:** Implemented and locally verified +**Status:** Implemented; local and Linux/macOS/Windows CI verified **Baseline:** Phase 4 is implemented and locally verified with 90/90 tests ## Purpose @@ -314,8 +314,8 @@ sleeping workers, leaving a nominally multi-threaded pool non-work-conserving. ## Acceptance Criteria -Phase-5 v1 is complete locally: the module, tests, example, benchmark report, and +Phase-5 v1 is complete: the module, tests, example, benchmark report, and English/Simplified-Chinese/Traditional-Chinese public documentation agree; Dev and Release strict builds and all 105 tests pass; the standalone example succeeds; and five local Release benchmark -rounds contain zero unexpected failures. Remote three-platform CI remains delivery verification, -not a locally claimed result. +rounds contain zero unexpected failures. PR #10 subsequently built the current library, ran the +current 140-test suite, and ran the standalone example successfully on Linux, macOS, and Windows. diff --git a/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md b/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md index ac8babe..f0dc4c5 100644 --- a/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md +++ b/docs/superpowers/specs/2026-08-29-cmp-phase6b-tcp-client-v1-design.md @@ -1,7 +1,7 @@ # CMP Phase 6B TCP Client v1 Design **Date:** 2026-08-29 -**Status:** Locally implemented and verified — remote CI pending +**Status:** Implemented; local and Linux/macOS/Windows CI verified in PR #10 **Baseline:** Phase 6A is locally committed as `701aa8b` and passes 116/116 tests ## Purpose diff --git a/docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md b/docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md index 3bed5e2..277fc0d 100644 --- a/docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md +++ b/docs/superpowers/specs/2026-08-30-cmp-ci-cold-cache-toolchain-design.md @@ -1,7 +1,7 @@ # CMP CI 冷缓存工具链修复设计 **日期:** 2026-08-30 -**状态:** 本地实现并验证,远程 CI 待确认 +**状态:** 完成;PR #10 Linux/macOS/Windows CI 已验证 **基线:** PR #10,提交 `1804044` ## 问题 @@ -64,3 +64,9 @@ mcpp 尚未到 1.0,工具版本提升可能改变模块扫描或清单校验 时沙箱 xlings 来自 workflow 的系统版本;因此只提升 mcpp 虽可依赖版本精化回退,但没有 消除错误绑定的来源。最终方案同时固定官方配套的 mcpp `2026.8.28.1` 与 xlings `2026.8.27.5`,其余边界不变。 + +## 远程验收结果 + +提交 `7c81a10` 推送后,PR #10 的三个冷环境均准确安装 xlings `2026.8.27.5` 与 mcpp +`2026.8.28.1`。Linux x86_64、macOS arm64、Windows x86_64 均完成库构建、10 个测试 +二进制和独立示例,耗时分别为 2m32s、2m42s、2m10s;三项检查全部成功。 From dab0531fa35502b7173a3b99c426fbded0dcb85a Mon Sep 17 00:00:00 2001 From: HikariTish Date: Sun, 30 Aug 2026 02:36:54 +0800 Subject: [PATCH 17/17] =?UTF-8?q?=E5=90=8C=E6=AD=A5=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=E4=BA=A4=E6=8E=A5=E7=8A=B6=E6=80=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agent/HANDOFF.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index e9f67d5..1517b33 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -25,8 +25,8 @@ Phase 4、Phase 5、Phase 6A 和 Phase 6B 第 1–10 项均已本地完成。当 PR 初次检查中,macOS arm64 与 Windows x86_64 已通过;Linux x86_64 在编译 CMP 前因 mcpp/xlings 冷缓存运行时绑定失配失败。修复提交 `7c81a10` 已推送;随后 Linux、macOS、 -Windows 三项 CI 全部通过,Phase 5 与 Phase 6B 的远程交付门已满足。当前只剩同步最终状态 -文档、推送并确认文档提交后的 CI。用户未授权合并。 +Windows 三项 CI 全部通过。状态文档提交 `4a909ed` 推送后的第二轮三平台 CI 也全部通过, +Phase 5 与 Phase 6B 的本地及远程交付门均已满足。PR #10 保持打开;用户未授权合并。 ## 已完成工作 @@ -87,6 +87,8 @@ Windows 三项 CI 全部通过,Phase 5 与 Phase 6B 的远程交付门已满 - PR #10 首轮 CI:macOS、Windows 通过;Linux 因旧工具冷缓存失配失败,CMP 未开始编译。 - PR #10 修复后 CI:Linux x86_64、macOS arm64、Windows x86_64 全部通过,耗时分别为 2m32s、2m42s、2m10s;各平台均为 10 个测试二进制、0 个失败,示例输出成功。 +- PR #10 状态文档提交 `4a909ed` 后再次全绿:Linux、macOS、Windows 分别为 2m34s、 + 2m49s、2m10s;代码与工具配置未再变化。 ## 已知问题 / 风险 @@ -100,9 +102,9 @@ Windows 三项 CI 全部通过,Phase 5 与 Phase 6B 的远程交付门已满 ## 剩余工作 -1. 提交并推送最终状态文档,确认 PR #10 文档提交后的 Linux、macOS、Windows CI。 -2. 不合并 PR #10,除非用户另行明确授权。 +1. 当前没有未完成的 v1 开发或验证任务。 +2. PR #10 尚未合并;合并需要用户另行明确授权。 ## 推荐下一步 -提交并推送最终状态文档,然后确认 PR #10 三平台 checks 保持全绿;未经授权不合并。 +等待用户审查 PR #10,并决定是否授权合并或开始下一阶段;未经授权不合并。