feat(collections): auto-generate collection.md with maturity filtering#1316
feat(collections): auto-generate collection.md with maturity filtering#1316WilliamBerryiii wants to merge 3 commits intomainfrom
Conversation
…kers - thread channel and maturity parameters through extension readme generation - add template markers to collection.md files with backward-compatible processing - add marker validation rules to collection validation pipeline - add unit tests for maturity filtering, marker handling, and marker validation 🔧 - Generated by Copilot
- expand New-CollectionReadme help with in-place update and marker behavior (IV-001) - add ValidateNotNullOrEmpty to AllowedMaturities parameter (IV-002) - centralize marker strings as script-scope constants in both scripts (IV-003) - remove unused Existing key from Split-CollectionMdByMarkers return (IV-004) - add duplicate-BEGIN-marker edge case test (IV-005) - add empty-string-throws test for Split-CollectionMdByMarkers (IV-006) - standardize Set-ContentIfChanged encoding to utf8NoBOM (IV-007) 🔧 - Generated by Copilot
…togen-maturity-1256
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1316 +/- ##
==========================================
+ Coverage 87.63% 87.69% +0.05%
==========================================
Files 61 61
Lines 9328 9376 +48
==========================================
+ Hits 8175 8222 +47
- Misses 1153 1154 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Advisory review — this PR is from a maintainer. Findings are informational only.
Review Summary
The implementation is well-structured and the PR meets quality standards. The description is clear, linked issue is present, type-of-change selections are accurate, and the 19 new Pester tests provide solid coverage of the new behavior. The utf8 → utf8NoBOM encoding fix in Set-ContentIfChanged is a correct bug fix. The maturity-filtering design is clean and consistent with existing channel logic.
Two advisory observations are raised below.
Issue Alignment ✅
Closes #1256. The PR delivers auto-generation of collection.md artifact tables with channel-based maturity filtering and sentinel-marker support, which matches the stated intent. The write-back behavior, backward-compatible marker validation, and plugin regeneration are all coherent.
PR Template Compliance ✅
| Section | Status |
|---|---|
| Description | ✅ Complete and detailed |
| Related Issue | ✅ Closes #1256 |
| Type of Change | ✅ New feature + Script/automation — both correct |
| Testing | ✅ Comprehensive (2002 tests, 0 failures, 19 new tests, lint & plugin generation) |
| Checklist | ✅ All required items checked |
Coding Standards
💡 Split-CollectionMdByMarkers — incomplete function decoration (advisory)
Every other function in Prepare-Extension.ps1 uses [CmdletBinding()], [OutputType()], and a full comment block (.DESCRIPTION, .PARAMETER, .OUTPUTS, .EXAMPLE). The PSScriptAnalyzer configuration explicitly sets PSProvideCommentHelp.ExportedOnly = $false, so the convention applies to private functions too. The inline comment above includes a suggested complete signature.
💡 Duplicate marker constants — DRY concern (advisory)
$script:CollectionMdBeginMarker and $script:CollectionMdEndMarker are now defined identically in both Prepare-Extension.ps1 and Validate-Collections.ps1. Since CollectionHelpers.psm1 is imported by both, exporting the constants from there would eliminate the duplication and make any future string change a single-point edit.
Code Quality ✅
No bugs, security issues, or logic errors found. The Split-CollectionMdByMarkers parsing logic handles all the edge cases covered by tests (no markers, partial markers, reversed markers, footer content). The write-back side-effect in New-CollectionReadme is clearly documented in the function's comment block. Test coverage is thorough.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none| $script:CollectionMdBeginMarker = '<!-- BEGIN AUTO-GENERATED ARTIFACTS -->' | ||
| $script:CollectionMdEndMarker = '<!-- END AUTO-GENERATED ARTIFACTS -->' | ||
|
|
There was a problem hiding this comment.
Maintainability: Marker constants are duplicated between Validate-Collections.ps1 and Prepare-Extension.ps1
The same two string literals are now defined independently in two scripts (see Prepare-Extension.ps1 lines 68–69). If either marker string ever changes, both files must be updated in sync — a silent drift risk.
CollectionHelpers.psm1 is already imported by both scripts and is the natural home for shared collection constants. Consider exporting them from there, for example as module-level variables:
# In CollectionHelpers.psm1
$script:CollectionMdBeginMarker = '<!-- BEGIN AUTO-GENERATED ARTIFACTS -->'
$script:CollectionMdEndMarker = '<!-- END AUTO-GENERATED ARTIFACTS -->'
Export-ModuleMember -Variable CollectionMdBeginMarker, CollectionMdEndMarkerThen both calling scripts consume $CollectionMdBeginMarker directly after Import-Module, eliminating the duplication.
| function Split-CollectionMdByMarkers { | ||
| <# | ||
| .SYNOPSIS | ||
| Splits collection.md content at auto-generation markers. | ||
| #> | ||
| param( | ||
| [Parameter(Mandatory)] | ||
| [string]$Content | ||
| ) |
There was a problem hiding this comment.
Convention: Split-CollectionMdByMarkers is missing [CmdletBinding()], [OutputType()], and full comment-based help
Every other function in this file uses [CmdletBinding()] + [OutputType()] and provides a complete comment block (.SYNOPSIS, .DESCRIPTION, .PARAMETER, .OUTPUTS, .EXAMPLE). The PSScriptAnalyzer config sets PSProvideCommentHelp.ExportedOnly = $false, so the convention applies here too.
Suggested completion:
function Split-CollectionMdByMarkers {
<#
.SYNOPSIS
Splits collection.md content at auto-generation markers.
.DESCRIPTION
Parses the raw file content and returns a hashtable with Intro, Footer,
and HasMarkers fields. When the BEGIN/END markers are absent or appear in
the wrong order, HasMarkers is $false and Intro contains the original content.
.PARAMETER Content
Raw string content of the collection.md file. Must not be empty.
.OUTPUTS
[hashtable] With keys HasMarkers (bool), Intro (string), Footer (string).
.EXAMPLE
$parts = Split-CollectionMdByMarkers -Content (Get-Content 'ado.collection.md' -Raw)
if ($parts.HasMarkers) { Write-Host $parts.Intro }
#>
[CmdletBinding()]
[OutputType([hashtable])]
param(
[Parameter(Mandatory)]
[string]$Content
)
...Refs: PSScriptAnalyzer.psd1 lines 43–48, Set-ContentIfChanged as a comparable complete function.
katriendg
left a comment
katriendg
left a comment
There was a problem hiding this comment.
Left a few comments, I believe ensuring the plugin:generate takes care of generating the <collectionid>.collection.md file is a good approach, and we could have the Extension prepare regenerate what is needed based on maturity filtering.
Plugins always take everything without maturity filtering, so that in itself so the more I reflect on this, the more I think we need a generation for the plugins (the default), and regenerate one for stable/pre-release when packaging the extension.
Also this will impact the following documentation addition:
🟡 Comment 2 — Documentation gap: contributor guide has no mention of the two-zone collection.md convention
- File:
docs/contributing/ai-artifacts-common.mdand.github/copilot-instructions.md - Category: Documentation
- Severity: Should fix before merge
docs/contributing/ai-artifacts-common.md(the primary contributor guide for AI artifacts) extensivelydocuments*.collection.ymlstructure, maturity values, and plugin generation — but has no mention of:
- The
<!-- BEGIN AUTO-GENERATED ARTIFACTS -->/<!-- END AUTO-GENERATED ARTIFACTS -->marker format.- That
collection.mdhas two zones (hand-authored intro vs auto-generated table) and contributors must only edit the intro.- What command to run to refresh the artifact table after editing a YAML.
- How to seed a new collection with the correct marker structure.
.github/copilot-instructions.mdline 104 says "Only edit the intro zone; the artifact table is overwritten byplugin:generate" — this was added during the review session but the underlyingplugin:generatewiring gap (Comment 2 above) means this statement is currently inaccurate.A
### Collection Markdown Auto-Generationsubsection should be added todocs/contributing/ai-artifacts-common.mdexplaining the marker format, the two-zone rule, and the correct command to sync the artifact table. This section should be updated once Comment 2 is resolved so the documented command is accurate.
| $null = $artifactSections.AppendLine() | ||
| } | ||
|
|
||
| # Write back updated artifact section into collection.md when markers are present |
There was a problem hiding this comment.
🔴 Comment 1 — Critical Gap: collection.md writeback is only wired to extension:prepare, not plugin:generate
The sentinel-marker writeback that updates the artifact table in
collection.mdruns insideNew-CollectionReadme, which is called only fromInvoke-ExtensionCollectionsGeneration— and that function is only invoked bynpm run extension:prepare(and:prerelease).
npm run plugin:generatecallsGenerate-Plugins.ps1, which never callsPrepare-Extension.ps1and never writes back tocollection.md.This means:
- A contributor who adds an item to
ado.collection.ymland runsnpm run plugin:generate(as the docs instruct) will see theplugins/ado/README.mdupdate correctly, butcollections/ado.collection.mdwill remain stale — the artifact table between the markers will not reflect the new item.- The
collection.mdonly refreshes when someone runsextension:prepare, which contributors won't do unless they're on the extension release path..github/copilot-instructions.mdline 107 currently states: "runnpm run plugin:generateto regenerate plugin outputs underplugins/and update the artifact sections incollection.mdfiles." This is incorrect —plugin:generatedoes not updatecollection.md.Recommended fix: Move the
collection.mdwriteback intoGenerate-Plugins.ps1(or a shared helper), so thatplugin:generateis the single command that keeps bothplugins/*/README.mdandcollections/*.collection.mdin sync. Alternatively, add a dedicatednpm run collection:syncscript and update all contributor-facing docs to reference it as a required step after editing a collection YAML.
| Import-Module (Join-Path $PSScriptRoot "../collections/Modules/CollectionHelpers.psm1") -Force | ||
|
|
||
| # Auto-generation marker constants shared across Split-CollectionMdByMarkers and New-CollectionReadme | ||
| $script:CollectionMdBeginMarker = '<!-- BEGIN AUTO-GENERATED ARTIFACTS -->' |
There was a problem hiding this comment.
🟡 Comment 3 — Suggestion: marker string literals duplicated across two files
- File:
scripts/extension/Prepare-Extension.ps1(lines 68-69) andscripts/plugins/Modules/PluginHelpers.psm1 - Category: Maintainability
- Severity: Suggestion
Once Comment 1 is fixed, the marker strings
<!-- BEGIN AUTO-GENERATED ARTIFACTS -->and<!-- END AUTO-GENERATED ARTIFACTS -->will be defined in two separate files. If the strings ever change, both must be updated in sync — a silent breakage risk.Both files already import
CollectionHelpers.psm1. Exporting the constants from that module (or a newMarkerConstants.psm1) would give a single source of truth. At minimum, add a comment in each file cross-referencing the other.
3 participants
Description
Added auto-generation of
collection.mdartifact tables from YAML manifests with channel-based maturity filtering and sentinel marker support. The generation engine in Prepare-Extension.ps1 gained a-Channelparameter that controls which maturity levels appear in output, andNew-CollectionReadmewas rewritten to parse marker-delimited sections, filter items by maturity, and write back updated content preserving hand-written introductions and footers. Collection validation now detects mismatched or reversed markers as warnings for backward compatibility. The encoding forSet-ContentIfChangedwas corrected to UTF-8 without BOM. Nineteen new Pester tests cover maturity filtering, template marker handling, andSplit-CollectionMdByMarkersedge cases.Related Issue(s)
Closes #1256
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
prompt-builderagent and addressed all feedback.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md)Other:
.ps1,.sh,.py)Sample Prompts (for AI Artifact Contributions)
Testing
Automated testing:
Split-CollectionMdByMarkers: no markers, empty string throws, correct intro/footer parsing, partial markers, reversed markers, duplicate markersManual testing: Not performed.
Checklist
Required Checks
AI Artifact Contributions
Required Automated Checks
The following validation commands must pass before merging:
npm run lint:mdnpm run spell-checknpm run lint:frontmatternpm run validate:skillsnpm run lint:md-linksnpm run lint:ps