Add new PURL type: 'nix' [old] - #314
Conversation
This adds a first version of the Nix package type. Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
| pkg:nix/glibc@2.39-52?drvpath=/nix/store/3nxf8kw3vgghz2y72b9qwi01sz62nhyk-glibc-2.39-52.drv&output=out&repository=github:user/nixpkgs | ||
| pkg:nix/systemd@255.6?drvpath=/nix/store/r34i4md1cmc19392zbbp9ya5nmd0av0k-systemd-255.6.drv&output=dev | ||
|
|
||
|
|
There was a problem hiding this comment.
You probably want to remove "nix" from the old list below.
| nix | ||
| --- | ||
|
|
||
| ``nix`` for Nix derivations: |
There was a problem hiding this comment.
| ``nix`` for Nix derivations: | |
| ``nix`` for Nix store paths: |
The PURL is for each store path, and we may have multiple PURL for different output paths in the same derivation.
| ``nix`` for Nix derivations: | ||
|
|
||
| - The default package repository is <https://github.com/NixOS/nixpkgs> but the ``repository`` qualifier can override the package repository. | ||
| - The ``name`` is the package ``pname`` field of the given derivation. It is case sensitive. |
There was a problem hiding this comment.
We don't always have the pname / version separate. Deep down pname and version are concatenated into a single string. Should something creating a PURL try to split this off? (Probably fine, but in that case, see the comment on the version field).
There was a problem hiding this comment.
the current state of name, pname, and versioning standards was part of why I didn't expect there to be a PURL standard yet for nix. (TBH I haven't checked on the current-state of discussions for a while so maybe it's settled lol)
Although it'd be great if this spec discussion helps push things forwards
|
|
||
| - The default package repository is <https://github.com/NixOS/nixpkgs> but the ``repository`` qualifier can override the package repository. | ||
| - The ``name`` is the package ``pname`` field of the given derivation. It is case sensitive. | ||
| - The ``version`` is the ``version`` field of the given derivation. |
There was a problem hiding this comment.
How do PURL look like for store paths where the version cannot be extracted? Is this field (and other fields) optional or required? The spec for other PURLs in this file do explicitly say what's required and what not.
| - The default package repository is <https://github.com/NixOS/nixpkgs> but the ``repository`` qualifier can override the package repository. | ||
| - The ``name`` is the package ``pname`` field of the given derivation. It is case sensitive. | ||
| - The ``version`` is the ``version`` field of the given derivation. | ||
| - The ``drvpath`` qualifier is the derivation path (``.drvPath``). |
There was a problem hiding this comment.
Do we want to have an outpath here too, or are PURL always for individual output paths, so this information would be redundant?
Is drvpath optional or required?
| - The ``name`` is the package ``pname`` field of the given derivation. It is case sensitive. | ||
| - The ``version`` is the ``version`` field of the given derivation. | ||
| - The ``drvpath`` qualifier is the derivation path (``.drvPath``). | ||
| - The ``output`` qualifier is the output field, by default: ``out`` |
There was a problem hiding this comment.
| - The ``output`` qualifier is the output field, by default: ``out`` | |
| - The ``output`` qualifier is the name of the output, which helps to distinguish in the case of multi-output derivations |
The "default" behavior is confusing. It doesn't describe if it should always be set, or if it is "out" in case the field is not present.
Or should it always be set to something, and the fact it's "out" in single-output derivations is an implementation detail of Nix that should be left out of the spec? I prefer this one, which is my the suggestion uses this variant.
I don't think it should be omitted in case of single-output derivations, it'd be odd if it suddenly would appear if an expression is refactored to multiple outputs.
There was a problem hiding this comment.
including the output field also accounts for if the default is changed in the future and shouldn't be very painful to just include
There was a problem hiding this comment.
we should just map this to outputName of the drv. Then this works with lib.getLib etc
|
|
||
| ``nix`` for Nix derivations: | ||
|
|
||
| - The default package repository is <https://github.com/NixOS/nixpkgs> but the ``repository`` qualifier can override the package repository. |
There was a problem hiding this comment.
Does this field even matter at all? With just the PURL we cannot reproduce the drv path anyways, there's a lot of other details (like rev, system, ...) we'd need. Also, the line gets blurry if you have a repo that imports nixpkgs, but overrides an expression, what should this be set to?
Inclined to remove this, this can only be a hint at most.
There was a problem hiding this comment.
With this field being mostly informative I feel we'd want a nar package type or at least a nar field to better identify where you got the package from if using a cache like cache.nixos.org. ommitted or something for if you built it locally?
There was a problem hiding this comment.
It is not identifying the package. If you have a nar sha256 hash, you can fetch it from any content-addressed store.
There was a problem hiding this comment.
"what" being covered by the hash, but "where" can also useful for some governance.
Also if you can find that hash in your content-addressed store then great, but if you can't you might at least want a possible alternative location to check to find the same artifact described by the PURL.
Also lightning quick response! ⚡
06kellyjac
left a comment
There was a problem hiding this comment.
General thanks for kicking this off.
Added some comments :)
| - The ``name`` is the package ``pname`` field of the given derivation. It is case sensitive. | ||
| - The ``version`` is the ``version`` field of the given derivation. | ||
| - The ``drvpath`` qualifier is the derivation path (``.drvPath``). | ||
| - The ``output`` qualifier is the output field, by default: ``out`` |
There was a problem hiding this comment.
including the output field also accounts for if the default is changed in the future and shouldn't be very painful to just include
|
|
||
| ``nix`` for Nix derivations: | ||
|
|
||
| - The default package repository is <https://github.com/NixOS/nixpkgs> but the ``repository`` qualifier can override the package repository. |
There was a problem hiding this comment.
With this field being mostly informative I feel we'd want a nar package type or at least a nar field to better identify where you got the package from if using a cache like cache.nixos.org. ommitted or something for if you built it locally?
| ``nix`` for Nix derivations: | ||
|
|
||
| - The default package repository is <https://github.com/NixOS/nixpkgs> but the ``repository`` qualifier can override the package repository. | ||
| - The ``name`` is the package ``pname`` field of the given derivation. It is case sensitive. |
There was a problem hiding this comment.
the current state of name, pname, and versioning standards was part of why I didn't expect there to be a PURL standard yet for nix. (TBH I haven't checked on the current-state of discussions for a while so maybe it's settled lol)
Although it'd be great if this spec discussion helps push things forwards
jkowalleck
left a comment
There was a problem hiding this comment.
- please also add examples to the test suite
- please also remove
nixfrom the list of "Other candidate types to define"
jkowalleck
left a comment
There was a problem hiding this comment.
please rebase onto latest master branch, and then remove nix from the Other candidate types to define
|
Thanks for the review, and sorry for the delay, this PR was opened to spark discussions and discussions are still continuing in the Nix ecosystem on what is the best format as we have some special challenges. In terms of expectations, I'd like to ensure that most stakeholders are willing to carry on this, if so, I will apply the changes and add some examples whenever I have time (the other stakeholders are of course welcome to send me patches :P). If we are not to continue this path, we will probably close this PR and document why we cannot continue on this path. Sorry again for the delay and thanks to flokli to pester me. |
|
So if you think this is the way to go, after the merge of PR #514, PURL tests and defs are now defined in new JSON schemas 👼 😇 😁 : ... therefore with the new approach... this PR would need to be updated. Sorry for the churn. ❤️ |
|
@RaitoBezarius any update on this? 😃 |
|
any update on this? |
|
No update - we do not have a candidate PR work because the minimum is a candidate PURL type definition in JSON Schema format. Plus tests for anything unique about the candidate PURL type |
|
Based on NixOS/nixpkgs#454333 , it seems the |
You mean because Nixpkgs will use non-nix PURLs to refer to non-nix packages that it downloads? I think it's an unrelated issue. If a package in Nixpkg contains a But I don't know if this On the other hand, if you just wanted to identify or locate the package instead of match vulnerabilities against it, I think it would be much more useful to describe the source of the derivation, eg I don't know if it's possible to work backwards from installed Nix derivations to this kind of information, which would be a significant barrier to adoption. I've never tried it. I don't know what the Nixpkgs team does for tracking vulnerabilities or if there is a Nixpkgs way to determine what vulnerabilities affect a system configuration, which would be good to align with here. It's interesting that the glibc package (and only the glibc package) has a patch suffix appended to the version number. |
Isn't that also the case if we use I am thinking something like this: where See https://www.nixhub.io/packages/systemd In this case, we have the version and the commit hash to pin down the exact commit that was used. Would this be a potential solution? |
|
I like the level of specificity from commit hashes, but I don't like that commit hashes aren't comparable without additional information. If you wanted to know if a vulnerability had been patched or not in a particular commit ID of the nixpkgs repository, you would need to have a copy of the commit graph of the nixpkgs repository containing both the fixed commit and the question commit, and I'd imagine for most tools that would need to do that kind of query that would involve adding special support for nixpkgs. Including the commit hash might still be the right thing to do anyway. |
|
@matt-phylum I think there is a way to avoid downloading or managing a full local commit graph.
Using "imlib2" as a sample:
Using the compare API:
The status field indicates This might resolve the comparison issue. In other words, once we identify the first commit that introduced the fix for a vulnerability, then every commit after that point can be considered free of that vulnerability, and every commit before it should be considered still affected. However, this commit_hash approach may not work well if in different branches? |
|
I don't think relying on internet access and GitHub's API for every comparison is really great either. It requires an active internet connection and managing third party API tokens, it's subject to GitHub outages and rate limiting, and it would require separate implementations and possibly separate API token management for other hosts like Codeberg. NUR has GitHub and git.catgirl.cloud and codeberg.org and git.kruemmelspalter.org etc. Another complicating factor is that packages are usually defined in Git repositories, but AFAIK this information is only available for Flakes. If somebody isn't using Flakes, they're probably just downloading a tarball. I'm not sure how that would work with timestamps either. |
|
RE: #314 (comment)
there is no GitHub needed here.
|
|
@matt-phylum Thanks. All your points are spot on and completely valid. @jkowalleck Thanks, but to use the native |
|
@chinyeungli Are you up for drafting a candidate PURL type definition JSON file following the PURL Type schema? |
Sure. Let me think about it and draft something. |
Nixpkgs channel tarballs embed the version information in the tarball as and are accessible through https://github.com/NixOS/nixpkgs/blob/2a5f1b8429b586a70033f92fc9045db1705c088c/lib/trivial.nix#L455 |
|
I created another PR with the new json schemas at #877 |
Removed draft from name in metadata Signed-off-by: Michael Herzog <mjherzog@aboutcode.org>
|
keeping open until we complete |
This adds a first version of the Nix package type.