Skip to content

Be more permissive of bad/missing UBI checksums - #974

Draft
BenSeverson wants to merge 1 commit into
ReFirmLabs:masterfrom
BenSeverson:ubi-permissive-checksums
Draft

BenSeverson wants to merge 1 commit into
ReFirmLabs:masterfrom
BenSeverson:ubi-permissive-checksums

Conversation

@BenSeverson

Copy link
Copy Markdown

UBI/UBIFS images recovered from flash dumps frequently have corrupt or zeroed
header checksums. Previously any such image was rejected outright by the header
parsers, so no signature was reported and no extraction was attempted.

Signature / structure parsing

The three UBI header parsers now treat the CRC as advisory and report whether it
validated, via a new crc_valid field. Because the checksum can no longer vouch
for the rest of the header, additional structural checks are applied when, and
only when, the CRC is bad or missing — images with valid checksums parse exactly
as they did before:

Header Checks added for bad/missing CRC
Superblock leb_size 1KB-16MB, leb_count 1-1M
Erase count all padding fields NULL, header offsets <= 1MB
Volume volume type dynamic/static, copy_flag <= 1, compat_type <= 5

Images that fail their checksums are reported at CONFIDENCE_MEDIUM with
", invalid header checksum" appended to the description. ubi_parser also tracks
whether any of the per-block volume headers failed its CRC, not just the erase
count header. Sizes are computed with saturating_mul, since unvalidated LEB
fields now feed the multiply.

A UBIFS image whose computed size runs past EOF is reported with an unknown size
and "(truncated)", rather than being discarded by the end-of-file sanity check in
Binwalk::scan, so a partial image can still be extracted. This part is
independent of the checksum work and can be dropped if unwanted.

Extraction

Both ubireader utilities are now invoked with -i (ignore block header errors)
and -w (warn-only on bad block reads). These gate exactly the CRC and
read-error paths in ubi/block/__init__.py and ubifs/walk.py in ubi_reader
0.8.15. -f (u-boot image_seq == 0 fix) was deliberately left out, since that
addresses a sequence number mismatch rather than a checksum.

Testing

Synthetic UBI and UBIFS images were built with correct CRCs and again with
corrupted ones:

  • Before this change, both corrupt images were completely undetected; after, both
    are detected and flagged. Output for the valid images is byte-identical to
    before.
  • False positive check: 200k records each of the UBIFS node magic, the UBI erase
    count magic and the UBI volume magic, followed by random bytes, produced zero
    detections.
  • cargo fmt clean; cargo clippy --all-targets reports only the two
    pre-existing romfs.rs warnings.

No test fixtures were added, since the repo has no UBI inputs and CI does not
install ubi_reader, so an extraction test would fail there. Extraction itself is
therefore unverified - ubi_reader was not installed in the dev environment, and
the flags' behavior was confirmed by reading its source rather than by running it.

🤖 Generated with Claude Code

UBI/UBIFS images recovered from flash dumps frequently have corrupt or
zeroed header checksums, and previously any such image was rejected
outright: no signature was reported, so no extraction was attempted.

The three UBI header parsers now treat the CRC as advisory and report
whether it validated. Because the checksum can no longer vouch for the
rest of the header, additional structural checks are applied when, and
only when, the CRC is bad or missing, so images with valid checksums
parse exactly as they did before:

  - superblock: sane leb_size and leb_count
  - EC header: NULL padding, bounded header offsets
  - volume header: known volume, copy and compatibility types

Images that fail their checksums are reported at medium confidence with
"invalid header checksum" in the description. A UBIFS image whose
reported size runs past EOF is now reported with an unknown size rather
than being discarded by the end-of-file sanity check, so a truncated
image can still be extracted.

Finally, both ubireader utilities are invoked with -i and -w, so that
blocks with bad header checksums are included in the extraction and bad
block reads are not fatal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant