nixpkgs-clarity: Add batch indexing for Nixpkgs in PurlDB

[pombredanne]

Working code for bulk indexing for all nixpkgs. Determine and index the corresponding source code repositories as needed using ecosystem conventions.

[chinyeungli]

We can use
https://github.com/pkgforge-dev/NixOS-Packages/
(https://raw.githubusercontent.com/pkgforge-dev/NixOS-Packages/main/nixpkgs.json)
to get all the nix package's name and the current version

Then we can use https://search.nixos.org/packages to get to the nix file and extract the package's metadata

Update:
search.nixos.org uses a protected Elasticsearch backend and we cannot use script to collect data from there.

[chinyeungli]

@pombredanne is the package type for nix is nix ?
However, I don't see it at https://github.com/package-url/purl-spec/tree/main/types
Perhaps we need to have nix support before having this issue to work?

For instance,
https://search.nixos.org/packages?channel=25.11&show=zola&query=zola
The PURL is pkg:nix/zola ?

[chinyeungli]

I’m thinking extracting metadata from https://github.com/NixOS/nixpkgs/blob/master/pkgs/. However, not every file contains useful information such as the homepage URL. For example, the package filelight is listed on search.nixos.org with homepage and source links, but its default.nix only provides the name and lacks those details. So, using only https://github.com/NixOS/nixpkgs/blob/master/pkgs/ may not be sufficient for gathering complete package metadata.

@pombredanne I am short on ideas about how to approach this issue.

[pombredanne]

@chinyeungli re: #757 (comment) and PURL type for nix... can you check the PURL spec or confer with @johnmhoran whether we have a mix or nixpkgs type?

[johnmhoran]

@chinyeungli

• For nix we have Add nix package type #314 -- Add new PURL type: 'nix' [old] package-url/purl-spec#314 -- opened 2024-07-22 by RaitoBezarius. 
• nix is also listed here: https://github.com/package-url/purl-spec/blob/main/docs/candidate-purl-types.md

[chinyeungli]

It seems the nix package type isn’t ready yet. I’ll hold it back until everything is finalized.

[chinyeungli]

See aboutcode-org/scancode.io#2155 for reference on approaches to get packages' metadata