Repository navigation
feat: build named rulesets into query files and manifests - #717
Merged
Merged
Conversation
… build A pipeline that failed to load exited the process, so later groups never ran and no report was printed. backend build now loads pipelines with try_load_pipelines and fails just that group. Delegated groups pass their pipeline names to sigma-cli untouched, as backend convert does, instead of loading them as rsigma pipelines first.
A rule the backend returned nothing for was recorded as converted. The manifest now reports converted only when the backend returned a result for the rule's id, or its title when it has no id, so an id-less rule is no longer mistaken for a failed rule that shares its title. Drops the unused per-rule query copy.
Ruleset spec errors used the bare ruleset name as the path while convert errors used a dotted path. Every semantic error now names its config path (rulesets.<name>, eval, eval.ruleset, eval.rulesets, daemon, daemon.rulesets, convert.groups.<name>), and the ruleset entry messages name the entry.
Two groups with the same output, for example through convert.defaults.output, silently overwrote each other. config validate and backend build now reject a group whose output or manifest path another group already writes, and backend build exits 3 before writing anything.
…ons in backend build
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A curated ruleset can feed several targets, but each target still needs a separate
backend convertrun, and nothing records what was generated. This adds aconvert:config section andrsigma backend build, which converts every configured group into query files and manifests.Configuration
Group fields override the defaults field by field and
optionsmerge per key. Groups replace wholesale across config layers, so a higher layer that redefines a group does not inherit the lower definition. Pipelines come only from the group or the defaults, never from the ruleset. Groups are file-only;RSIGMA_CONVERT__DEFAULTS__*works, butconvert.groups.*is ignored.Command
backend buildruns every group, or the groups named with repeatable--group, in name order. For each group it writes the queries tooutputand a manifest to<output>.manifest.json. A failed group does not stop the others, including a group whose pipeline does not load. Groups whose output or manifest paths resolve to the same file are rejected up front, before anything is written. The command exits 2 when any rule failed to convert withoutskip_unsupported, 3 for configuration errors, and 1 for drift under--check, which writes nothing.The manifest records
manifest_version, group, ruleset, target, format, pipelines, output, generator, query count, one entry per detection or correlation document with its canonical hash and conversion status (converted,failed,skipped, orno_queryfor a rule that produced no query of its own, such as a rule absorbed into a correlation), and the group'smetadatacopied verbatim. Delegated targets materialize the resolved ruleset before sigma-cli runs, pass the group's pipelines to sigma-cli untouched, and omit per-rule status, because sigma-cli reports no per-rule outcome. Filter rules are applied during conversion and do not appear inrules.Validation and config surface
config validatereportssemantic_errorswith{path, message}instead ofruleset_errors, wherepathis the dotted config path of the offending entry (for examplerulesets.linux_edgeorconvert.groups.linux_edge_lynxdb). It covers theconvertsection: group names, ruleset references, the fields a group needs after defaults, and groups that write the same files.config show --for convertis supported. Thersigma.yamltemplate and the JSON schema include the section.Docs
A new
backend buildCLI page, a "Convert named rulesets" guide section, ruleset guide links, the configuration catalogue and example, exit codes, the skill chooser, and the changelog entry.Notes
convert_nativeandconvert_delegatednow back bothbackend convertandbackend build, and the existingcli_converttests pass unchanged. No parser or eval change, so no corpus regression or new fuzz target is needed. The only untrusted part of the section is the ruleset, which is already fuzzed.Closes #689