Skip to content

feat(ci): review issue content with bounded LLM follow-ups - #3060

Draft
PIKACHUIM wants to merge 3 commits into
mainfrom
ci/auto-issue-process
Draft

PIKACHUIM wants to merge 3 commits into
mainfrom
ci/auto-issue-process

Conversation

@PIKACHUIM

@PIKACHUIM PIKACHUIM commented Sep 8, 2026 •

Copy link
Copy Markdown
Member

Summary / 摘要

为通过现有提交规则检查的新建 Issue 增加 LLM 内容初审,判断描述是否足以让维护者开始处理,并提前提出必要的补充问题。

  • 在现有 Issue Auto Reply workflow 中增加后续 job,等待规则检查完成后读取 Issue 最新状态
  • 跳过机器人、公告、PR 及已关闭或锁定的 Issue
  • 模型根据标题、正文和现有标签识别内容问题,例如版本不具体、复现过程不清楚、日志不相关或功能场景不明确
  • 提示词要求配置、网络、授权和第三方排查方向有现有描述或日志支持,避免重复索取已提供或与问题无关的信息
  • 模型只输出 findings 数组,代码校验允许的枚举及最多 3 项的上限,拒绝多余字段
  • 代码将选中的问题映射为固定追问,合成一条注明自动初审性质的评论,并引导用户在当前 Issue 补充
  • 空结果静默结束,后续由维护者处理,第一版仅在新建时初审一次
  • 使用 Node.js 内置 fetch 调用模型和 GitHub API,无新增 npm 依赖
  • 模型输出非法或 API 请求失败时让 job 失败
  • 复用仓库现有固定提交的 checkout action,并关闭凭据持久化

需要配置仓库 Secrets:CUSTOM_API_BASE_URL、CUSTOM_API_KEY、CUSTOM_API_MODEL。其中基础地址应包含服务所需的版本路径,脚本追加 /chat/completions。

现有模板分类和提交规则检查保持原行为。新增内容初审仅发表评论,不修改标签、标题或 Issue 状态。评论内容来自代码中的固定文案,不拼接模型自由文本或 Issue 原文。

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。
  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了配置,新增上述仓库 Secrets。
  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Testing / 测试

  • node --check .github/issue-triage/triage.mjs
  • 通过 Node.js 测试运行器执行 9 组模拟 HTTP 测试,覆盖全部 10 种固定追问、空结果静默、三项上限与单条评论、未知枚举、恶意链接与提及、额外字段、旧协议拒绝、非法 JSON、忽略条件、API 失败和缺失配置
  • 模拟 GitHub API 仅接受读取当前 Issue 和发布当前 Issue 评论,确认没有标签或其他写操作
  • 此前已使用系统 Ruby YAML 标准库解析 workflow,并校验仅新建 Issue 触发、job 依赖及权限,本次未修改 workflow
  • git diff --cached --check
  • 真实 LLM API 连通性、内容判断质量与 GitHub Actions 端到端验证

模拟测试验证代码执行边界,未评估真实模型的提示注入抵抗能力或追问准确性。模型仍可能选择不必要或不准确的追问。模拟测试使用临时文件,完成后已清理,未连接真实 LLM 或 GitHub API。未运行 Go 测试,本次改动仅涉及 Issue 自动化流程。

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING。
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmt、go fmt 或 prettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • Codex

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Documentation / 文档

  • Tests / 测试

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

@jyxjjj
jyxjjj marked this pull request as draft September 8, 2026 08:03
@jyxjjj

jyxjjj commented Sep 8, 2026

Copy link
Copy Markdown
Member

标记一下暂不合并 等我空了看

@jyxjjj
jyxjjj force-pushed the ci/auto-issue-process branch from 9f630bd to 4a56691 Compare October 30, 2026 02:13
@jyxjjj jyxjjj changed the title feat(ci): auto process issues feat(ci): add lightweight LLM issue triage Oct 30, 2026
@jyxjjj jyxjjj changed the title feat(ci): add lightweight LLM issue triage feat(ci): review issue content with bounded LLM follow-ups Oct 30, 2026
jyxjjj and others added 3 commits November 2, 2026 11:57
- Run triage after the existing issue submission checks
- Skip bots, announcements, pull requests, and closed or locked issues
- Classify new issues and request missing information with a concise prompt
- Call the configured LLM and GitHub APIs with native Node.js fetch
- Validate model labels and comments before applying changes
- Restrict triage writes to adding labels and posting comments
- Fail the job on invalid model output or unsuccessful API requests
- Reuse the existing pinned checkout action without persisted credentials

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
- Replace free-form labels and comments with category and missing-information enums
- Limit automatic labels to bug and enhancement
- Generate follow-up comments exclusively from fixed application text
- Reject extra fields, unsupported values, and inconsistent category combinations before writes
- Bound missing-information arrays by the allowed fields for each category
- Clarify that issue content and forged role instructions are untrusted data

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
- Replace category and missing-field decisions with content-review findings
- Remove automatic classification labels already supplied by issue templates
- Provide fixed follow-up questions for ambiguous reports and feature requests
- Add evidence-based configuration, network, credential, and upstream checks to the prompt
- Limit each review to three findings and one comment
- Identify replies as automated preliminary reviews and direct users to the current issue
- Keep sufficient, unrelated, and uncertain reports silent through prompt guidance
- Reject extra fields, unsupported findings, and oversized findings arrays before writes

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
@jyxjjj
jyxjjj force-pushed the ci/auto-issue-process branch from de1f226 to e64c310 Compare November 2, 2026 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants