Skip to content

feat: build named rulesets into query files and manifests - #717

Merged
mostafa merged 13 commits into
mainfrom
feat/convert-manifest
Nov 17, 2026
Merged

mostafa merged 13 commits into
mainfrom
feat/convert-manifest

Conversation

@mostafa

@mostafa mostafa commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Problem

A curated ruleset can feed several targets, but each target still needs a separate backend convert run, and nothing records what was generated. This adds a convert: config section and rsigma backend build, which converts every configured group into query files and manifests.

Configuration

rulesets:
  linux_edge:
    sources: [rules/linux]

convert:
  defaults:
    format: default
    skip_unsupported: true
    options:
      correlation_method: join
  groups:
    linux_edge_lynxdb:
      ruleset: linux_edge
      target: lynxdb
      pipelines: [sysmon]
      output: build/lynxdb/linux_edge.txt
      metadata:
        rule_group: linux
        time_window: 5m

Group fields override the defaults field by field and options merge per key. Groups replace wholesale across config layers, so a higher layer that redefines a group does not inherit the lower definition. Pipelines come only from the group or the defaults, never from the ruleset. Groups are file-only; RSIGMA_CONVERT__DEFAULTS__* works, but convert.groups.* is ignored.

Command

backend build runs every group, or the groups named with repeatable --group, in name order. For each group it writes the queries to output and a manifest to <output>.manifest.json. A failed group does not stop the others, including a group whose pipeline does not load. Groups whose output or manifest paths resolve to the same file are rejected up front, before anything is written. The command exits 2 when any rule failed to convert without skip_unsupported, 3 for configuration errors, and 1 for drift under --check, which writes nothing.

The manifest records manifest_version, group, ruleset, target, format, pipelines, output, generator, query count, one entry per detection or correlation document with its canonical hash and conversion status (converted, failed, skipped, or no_query for a rule that produced no query of its own, such as a rule absorbed into a correlation), and the group's metadata copied verbatim. Delegated targets materialize the resolved ruleset before sigma-cli runs, pass the group's pipelines to sigma-cli untouched, and omit per-rule status, because sigma-cli reports no per-rule outcome. Filter rules are applied during conversion and do not appear in rules.

Validation and config surface

config validate reports semantic_errors with {path, message} instead of ruleset_errors, where path is the dotted config path of the offending entry (for example rulesets.linux_edge or convert.groups.linux_edge_lynxdb). It covers the convert section: group names, ruleset references, the fields a group needs after defaults, and groups that write the same files. config show --for convert is supported. The rsigma.yaml template and the JSON schema include the section.

Docs

A new backend build CLI page, a "Convert named rulesets" guide section, ruleset guide links, the configuration catalogue and example, exit codes, the skill chooser, and the changelog entry.

Notes

convert_native and convert_delegated now back both backend convert and backend build, and the existing cli_convert tests pass unchanged. No parser or eval change, so no corpus regression or new fuzz target is needed. The only untrusted part of the section is the ruleset, which is already fuzzed.

Closes #689

… build

A pipeline that failed to load exited the process, so later groups never ran and no report was printed. backend build now loads pipelines with try_load_pipelines and fails just that group. Delegated groups pass their pipeline names to sigma-cli untouched, as backend convert does, instead of loading them as rsigma pipelines first.
A rule the backend returned nothing for was recorded as converted. The manifest now reports converted only when the backend returned a result for the rule's id, or its title when it has no id, so an id-less rule is no longer mistaken for a failed rule that shares its title. Drops the unused per-rule query copy.
Ruleset spec errors used the bare ruleset name as the path while convert errors used a dotted path. Every semantic error now names its config path (rulesets.<name>, eval, eval.ruleset, eval.rulesets, daemon, daemon.rulesets, convert.groups.<name>), and the ruleset entry messages name the entry.
Two groups with the same output, for example through convert.defaults.output, silently overwrote each other. config validate and backend build now reject a group whose output or manifest path another group already writes, and backend build exits 3 before writing anything.
@mostafa
mostafa merged commit 9b04d8f into main Nov 17, 2026
27 checks passed
@mostafa
mostafa deleted the feat/convert-manifest branch November 17, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ruleset manifests (DaC composition)

1 participant