Repository navigation
feat(examples): add Pixel-Reasoner image and video rollouts - #1894
Open
Tournesoly wants to merge 1 commit into
Open
Tournesoly wants to merge 1 commit into
Tournesoly wants to merge 1 commit into
Conversation
Provide an inference-only visual-tool workflow that preserves reasoning across turns and exports complete attempts for offline evaluation. Pass optional native video inputs through SGLang without changing the existing positional request/response constructor arguments. Refs #1893 Signed-off-by: wens <wlithu@gmail.com>
Tournesoly
requested review from
CormickKneey,
HwVanICI,
Le8r0nJames,
PrometheusComing,
daihaowz,
fishcrap,
garrett4wade,
geshi001,
guozhihao-224,
sitabulaixizawaluduo and
yulangz
as code owners
November 6, 2026 09:40
5 of 15 tasks
Collaborator
There was a problem hiding this comment.
Can you provide a training script?
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add an inference-only Pixel-Reasoner image/video example under
examples/vlm, including a 27B configuration, multi-turn workflow, visual tools, basic verifier and setup instructions. Preserve reasoning across turns and export messages, token IDs, tool assets and terminal status for offline evaluation.Add optional video request/response fields and SGLang transport support. Existing positional dataclass arguments retain their order. The vLLM adapter explicitly rejects this new transport field until it is implemented.
Related Issue
Fixes #1893
Type of Change
Validation
pre-commit run --all-fileswas attempted: unrelated pyproject consistency, uv lock and CLI docs hooks could not run on the Python 3.10 development host because tomllib, uv and runtime dependencies were unavailable. Remaining all-file hooks passed.Checklist
pre-commit run --all-files)Additional Context
No training update, full Pixel reward or automatic resume/deduplication is included. The verifier is intentionally limited to normalized boxed-answer matching. Video currently targets SGLang's preprocessing API. Runtime compatibility is version-sensitive; README lists the tested stack and video dependencies.
Changes outside the example total 15 added lines in five files. Large-scale deployment and broader runtime portability were not re-tested. Public CI test coverage remains to be added; local test scripts and internal cluster/data paths are excluded from this PR.