rulezet-validation

Mirror rulezet.org's YARA rules and scan them against known-clean binaries. Any rule that fires on clean software is quarantined, with the evidence recorded.

Install

pip install git+https://github.com/rulezet/rulezet-validation

# or from a clone
git clone https://github.com/rulezet/rulezet-validation
cd rulezet-validation && pip install .      # uv pip install . works the same

# development
pip install -e ".[dev]" && pytest

One dependency: yara-python. No database, no services. State is files.

CLI

command	does
sync	shortcut for mirror sync
mirror sync	fetch rules, compile, run the gate
mirror gate	re-run the gate on the mirror on disk, no fetch
mirror check	re-scan the baseline, report only, move nothing
mirror recheck	put quarantined rules back on trial
mirror compile	recompile the mirror
mirror status	key, counts, last sync, stale verdicts
scan BINARY	what fires on one file
baseline list	files the gate will scan

flag	on	does
--limit N	sync	stop after N rules. Ignores the API key — the bulk endpoint cannot fetch a subset
--dump	sync	use the key anyway with --limit; downloads everything, then truncates
--full	sync	ignore the last-sync date
--meta-only	sync	refresh metadata only, leave rule files alone
--all	mirror recheck	re-evaluate every quarantined rule, not only stale ones
--mirror-dir	any	override the mirror location
--config	any	path to a config TOML

Configuration

Optional. First hit wins: $RULEZET_CONFIG, ./rulezet-validation.toml, ~/.config/rulezet-validation/config.toml.

Precedence, strongest first: environment, .env, config file, defaults.

mirror_dir = "data/rulezet"
released_file = "released.txt"
baseline_dirs = ["/usr/bin"]
baseline_max_files = 300
baseline_probes = true
baseline_exclude = []
baseline_exclude_defaults = true
allow_licenses = []          # e.g. ["cc0 1.0"]; empty keeps all. Needs an API key

Every setting has an environment equivalent:

setting	variable
mirror_dir	RULEZET_DIR
released_file	RULEZET_RELEASED_FILE
baseline_dirs	RULEZET_BASELINE_DIRS
baseline_max_files	RULEZET_BASELINE_MAX_FILES
baseline_probes	RULEZET_BASELINE_PROBES
baseline_exclude	RULEZET_BASELINE_EXCLUDE
baseline_exclude_defaults	RULEZET_BASELINE_EXCLUDE_DEFAULTS
allow_licenses	RULEZET_ALLOW_LICENSES
url	RULEZET_URL
api_key	RULEZET_API_KEY

List values are separated like PATH (: on Linux); booleans accept 1/true/yes/on. RULEZET_API_KEY is also read from a .env in the working directory; mirror status reports whether it reached the process.

The baseline

What counts as known-clean: the bundled uClibc ARM probes, plus baseline_dirs, minus the exclusions.

behaviour
sampling	even stride across the sorted directory, so a cap of 300 on /usr/bin is not just [ to cmp
symlinks	skipped, not followed
exclusions	do not consume a slot in baseline_max_files
probes	always included, never displaced by the cap

rulezet-validate baseline list      # exactly what will be scanned

Excluding files

layer	where	note
shipped	src/rulezet_validation/baseline/exclude.txt	capa, DIE, yara, ClamAV, radare2, metasploit, UPX, binwalk. One reason per entry, open to pull requests
local	baseline_exclude	additive; fnmatch against both the bare filename and the full path

baseline_dirs = ["/usr/bin", "/usr/sbin", "/opt/vendor-firmware/bin"]
baseline_max_files = 1500
baseline_exclude = ["/opt/ghidra/*", "our-internal-scanner"]
baseline_exclude_defaults = false    # ignore the shipped list entirely

the test
excluded	carries attack content or detection signatures as part of its job — capa embeds its own rules
not excluded	merely matches a rule — objdump lists every architecture it supports, and a rule reading that as a malware indicator is the rule's bug

Mirror layout

Default data/rulezet/, set by mirror_dir. Disposable: it writes its own .gitignore, and sync rebuilds it.

data/rulezet/
  rules/<uuid>.yara       one file per rule; the uuid is the yara namespace
  quarantine/<uuid>.yara  fired on the baseline
  rules.compiled          saved yara ruleset
  tags.json               {uuid: [tag, ...]}
  quarantine.json         the verdicts and their evidence   <-- what you want
  quarantine.txt          same, as a TSV index
  state.json              last sync + the baseline in use
  .gitignore              written on sync

file	gitignored
data/, and any other mirror_dir	yes	disposable, regenerated by sync
released.txt	no	hand-written, lives outside the mirror, commit it
.env, rulezet-validation.toml	yes

quarantine.json

A rule is quarantined if and only if it fired on the baseline and its uuid is not in released.txt. Nothing else moves a file.

{
  "baseline": {
    "count": 302,
    "signature": "742ff72ca0128ad4cfbe700e300b1cc55dd7bc17f82944d966ca41b37e34377b",
    "dirs": ["/usr/bin"],
    "exclude_patterns": ["capa*", "die"],
    "excluded_files": ["capa", "die", "diec"],
    "files": [
      {"name": "zsh", "path": "/usr/bin/zsh", "sha256": "…", "size": 968936}
    ]
  },
  "quarantined": {
    "adca4692-e22e-4952-a6b1-0baaa1e91ede": {
      "rule": "Linux_Generic_Threat_d94e1020",
      "status": "quarantined",
      "reason": "baseline_hit",
      "hits": 2,
      "first_seen": "2026-08-14T08:57:05Z",
      "last_checked": "2026-08-14T08:57:05Z",
      "rule_sha256": "a5b92da6a313f57d4e425da01f8f1d483f5d1181af7fbd06ebb1bff3b5c63c4f",
      "baseline_signature": "742ff72ca0128ad4…",
      "matched": [
        {
          "file": "clean_uclibc_fcntl.elf",
          "path": "<probes>/clean_uclibc_fcntl.elf",
          "sha256": "1794c4ad1d5872cfd2c1b3e81bb4d444df5b7c443f8310d6016fc2fe00e15d0c",
          "strings": ["$a1"],
          "offsets": ["0xd9d"],
          "offsets_total": 1,
          "offsets_truncated": false
        }
      ]
    }
  }
}

field	note
matched	every matching file, not a sample. Offsets capped at 64/file, flagged when truncated
path	publish-safe: $HOME becomes ~/…, bundled probes become <probes>/…, system paths kept verbatim
baseline.files	the whole corpus with hashes, so the verdict is reproducible
status	quarantined or cleared — history is kept, cleared entries stay

Releasing a rule

A clean-binary hit is not automatically a defect: network_smtp_raw firing on busybox is correct, since busybox has a sendmail applet. A uuid in released.txt is never quarantined again.

# reviewed: capability rule, MAIL FROM:/RCPT TO: on busybox is correct
ea0359a9-7798-410c-b9cf-3b9cfd72bbc0

Rechecking

A verdict is only true of the rule text and the corpus it was measured on.

stale reason	means
rule_changed	upstream edited the rule
baseline_changed	a probe, a corpus, or a binary changed
unverifiable	quarantined before hashes were recorded

rulezet-validate mirror status      # how many are stale, and why
rulezet-validate mirror recheck     # stale ones
rulezet-validate mirror recheck --all

recheck
does	returns rules to rules/, recompiles, re-gates. Still fires → back to quarantine
never	automatic. A sync reports staleness and stops
leaves alone	released.txt, first_seen

Embedding

from rulezet_validation import load_config, paths
from rulezet_validation.sync import sync

settings = load_config()
sync(settings, paths(settings), on_rules=my_indexer)

on_rules receives {uuid: metadata row} for everything fetched — the seam for indexing rule provenance into a host application's own store.

Status

Working: sync, tagging, compile, gate, quarantine records, recheck, CLI.

Each quarantine record carries a proposed_tag — false-positive:risk=high (20+ clean binaries hit), medium (5–19), low (1–4), or cannot-be-judged when there is no observation behind the record at all. It is a suggestion for the reviewer, not a verdict: nothing moves on it, and a risk tag already present upstream is recorded next to it as upstream_tag rather than overwriting or being overwritten.

Next:

1. rulezet-validate check RULE.yar — single-rule linter, no mirror needed.
2. baseline sync — fetch the corpora declared in baseline/manifest.toml.
3. Static checks feeding the risk proposal, so a rule can be judged on more than its hit count.

Licence

AGPL-3.0. See NOTICE for fetched and bundled artifacts, which are under their own terms.