Skip to content

Fix release testing and status workflows - #4686

Open
mattleibow wants to merge 14 commits into
mainfrom
mattleibow-fix-release-skills
Open

Fix release testing and status workflows#4686
mattleibow wants to merge 14 commits into
mainfrom
mattleibow-fix-release-skills

Conversation

@mattleibow

@mattleibow mattleibow commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Description

Fix the release skills so prepublication testing uses the exact package pair produced by the selected managed build, Windows status tooling preserves native CLI output correctly, and final publication cannot select, duplicate, or approve the wrong build.

  • Resolve the platform's official Azure CLI launcher (az/az.cmd), decode UTF-8 or the platform encoding without replacement corruption, render executable pipeline-status output through one ASCII-only path with [OK], [WARN], [FAIL], [RUNNING], and [WAITING] markers plus deterministic backslash escaping for non-ASCII dynamic text, and propagate CLI failures. The skill Markdown retains presentation Unicode for user-facing guidance.
  • Derive the exact SkiaSharp internal test package from the managed build-number text before +, apply the same complete suffix to the HarfBuzzSharp base version, and verify both exact .version values on the internal test (EAP) feed. Preview/RC public versions equal the exact packages; stable public versions use the two base versions.
  • Pass both internal and public package-version pairs from release-testing to release-publish without independently selecting a latest package.
  • Inspect workloads, Appium, drivers, SDKs, and devices before setup, and require explicit approval before machine-level installation, update, or replacement.
  • Align the Android old-device matrix with the current UiAutomator2 API 26 minimum rather than pinning a legacy driver.
  • Include the PR [tests] Enable Windows MAUI Appium tests #3969-enabled MauiWindowsTests path in commands, execution order, monitoring, criteria, and final reports; keep the official Appium Windows driver and consent-gated install-wad setup actionable and focused.
  • Keep managed source verification inline in the publish workflow: use the numeric run ID to verify pipeline 10789, terminal success, the exact release branch/commit, and expected release label, then check for non-completed pipeline 25298 runs before asking for queue confirmation.
  • Use a focused queue-publish.py wrapper that accepts only the verified build-number string, rejects numeric or malformed values, infers stable versus preview/RC, performs one Azure POST with resources.pipelines.SkiaSharp.version set to that string, and prints the run ID/URL.
  • Keep Azure push approval human-only, verify the renamed run and Push Stable/Push Preview, and require completed/succeeded before separately polling the exact SkiaSharp and HarfBuzzSharp public versions. Poll exhaustion fails explicitly.
  • Refresh website notes by dispatching Sync - Release Notes & API Diffs once from main, reporting it started, and considering the step complete.
  • For every preview, RC, and stable release, dry-run/review/apply audit-milestones.ps1, then close only the exact release milestone when it exists; absent milestones succeed and open issues are surfaced before closure.

Upstream metadata checked on 2026-08-05: Appium 3.6.0, UiAutomator2 8.2.2, Windows driver 6.1.0, and the Appium system requirements. Azure queue behavior was checked against the official Run Pipeline REST API and live 4.151.1 source/publish runs.

Related issues

None.

Required skia PR

None.

Areas affected

  • Managed API (binding/)
  • Native / C API (externals/skia/src/c, include/c)
  • Generated P/Invoke bindings
  • Native dependency or Skia update (libpng, HarfBuzz, FreeType, zlib, milestone bump, …)
  • Views & integrations (MAUI, Uno, WPF, WinUI, Blazor, …)
  • Rendering output / visual behavior
  • Performance
  • Tests
  • Build, packaging, or CI
  • Documentation or samples

Changes

None — release tooling, tests, and documentation only; no public API or shipped runtime behavior change.

Testing

  • Ran 10 deterministic release-status tests, including direct official Windows .cmd launcher execution, checked subprocess errors, empty output, and a real byte-level cp1252 subprocess (Café) decoded without U+FFFD and rendered as Caf\xe9 through the single ASCII console path.
  • Ran 7 focused queue-wrapper tests covering stable and preview/RC request bodies, numeric and malformed input rejection, CLI failure propagation, and returned/missing run IDs.
  • Verified the queue wrapper contains one subprocess.run, uses text=True, and issues one az devops invoke POST with --query id -o tsv.
  • Verified exact build-number-derived package handoff for stable, preview, and RC examples; exact internal-feed matching for both packages; distinct public package URLs; the active publish-run guard; and explicit poll exhaustion failure.
  • Compiled all four affected Python script/test files with py_compile and confirmed the two executable Python scripts are ASCII source.
  • Validated release-status, release-testing, and release-publish with skill-creator/scripts/quick_validate.py under Python UTF-8 mode.
  • Checked changed Markdown links and command paths, XML parsing, Microsoft.Testing.Platform argument ordering, API 26 and Windows matrix consistency, consent gates, website-notes dispatch behavior, and every-release milestone flow.
  • Ran git diff --check.

Checklist

  • Tests added or updated (if omitted, explain why above)
  • Changes above lists all public API and behavioral changes (or "None.")
  • New/changed public API? Filed a docs issue in mono/SkiaSharp-API-docs so reference docs can be written later — N/A, no public API change
  • Native change? Companion mono/skia PR linked above and bindings regenerated — N/A, no native change

Make pipeline status execution portable across Windows and Unix while
surfacing Azure CLI failures instead of treating them as missing runs.

Test exact prepublication stable packages, use the supported Android API
26 floor, require approval for machine setup changes, and include the
Windows MAUI path throughout release testing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📦 Try the packages from this PR

Warning

Do not run these scripts without first reviewing the code in this PR.

Step 1 — Download the packages

bash / macOS / Linux:

curl -fsSL https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.sh | bash -s -- 4686

PowerShell / Windows:

iex "& { $(irm https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.ps1) } 4686"

Step 2 — Add the local NuGet source

dotnet nuget add source ~/.skiasharp/hives/pr-4686/packages --name skiasharp-pr-4686
More options
Option Description
--successful-only / -SuccessfulOnly Only use successful builds
--force / -Force Overwrite previously downloaded packages
--list / -List List available artifacts without downloading
--build-id ID / -BuildId ID Download from a specific build

Or download manually from Azure Pipelines — look for the nuget artifact on the build for this PR.

Remove the source when you're done:

dotnet nuget remove source skiasharp-pr-4686

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📊 SkiaSharp benchmarks — PR #4686

this PR (full source build) vs 🌙 nightly · Linux · Windows · macOS

Informational only — this never blocks the PR. 🟢 faster / less allocation · 🔴 slower / more allocation; moves under 5% are hidden as noise.

⏱️ Times are raw BenchmarkDotNet means, and the ⭐ PR and baseline legs run on separate CI runners, so microbenchmarks can swing run-to-run — treat small time deltas as noise. Allocations are deterministic and the reliable signal. The interactive perf-dashboard (linked below) applies smoothing for the trend view.

Highlights

⏱️ Time — 🔴 33 slower · 🟢 14 faster

  • 🔴 RasterImageLifecycleBenchmark.CreateDataWithReleaseProc(Count: 256) · Windows · 96.93 µs → 132.26 µs (+36%)
  • 🔴 PathBoundsBenchmark.TightBounds(Points: 1024) · Windows · 15.63 µs → 20.87 µs (+34%)
  • 🔴 MatrixMapPointsBenchmark.MapRadius(Points: 256) · Windows · 8.66 µs → 11.25 µs (+30%)
  • 🔴 PathBoundsBenchmark.TightBounds(Points: 64) · Windows · 1.52 µs → 1.90 µs (+25%)
  • 🟢 MatrixMapPointsBenchmark.MapRect(Points: 256) · Linux · 14.68 µs → 10.95 µs (-25%)
  • …and 42 more (see details below)
Full per-OS benchmark deltas

Linux

⏱️ Time (vs 🌙 nightly 4.152.0-nightly.4)

Benchmark baseline this PR Δ
MatrixMapPointsBenchmark.MapRect(Points: 256) 14.68 µs 10.95 µs 🟢 -25%
ColorMathBenchmark.PreMultiply(Colors: 4096) 12.86 µs 14.09 µs 🔴 +10%
MatrixMapPointsBenchmark.MapPoints(Points: 4096) 2.65 µs 2.48 µs 🟢 -6%

Windows

⏱️ Time (vs 🌙 nightly 4.152.0-nightly.4)

Benchmark baseline this PR Δ
RasterImageLifecycleBenchmark.CreateDataWithReleaseProc(Count: 256) 96.93 µs 132.26 µs 🔴 +36%
PathBoundsBenchmark.TightBounds(Points: 1024) 15.63 µs 20.87 µs 🔴 +34%
MatrixMapPointsBenchmark.MapRadius(Points: 256) 8.66 µs 11.25 µs 🔴 +30%
PathBoundsBenchmark.TightBounds(Points: 64) 1.52 µs 1.90 µs 🔴 +25%
RasterImageLifecycleBenchmark.CreateRasterImage(Count: 256) 246.13 µs 306.32 µs 🔴 +24%
MatrixMapPointsBenchmark.MapPoints(Points: 256) 178.0 ns 215.2 ns 🔴 +21%
MatrixMapPointsBenchmark.MapRect(Points: 256) 7.23 µs 8.73 µs 🔴 +21%
MatrixOpsBenchmark.Concat(Count: 4096) 230.17 µs 274.14 µs 🔴 +19%
MatrixMapPointsBenchmark.MapRadius(Points: 4096) 148.67 µs 174.60 µs 🔴 +17%
MatrixMapPointsBenchmark.MapRect(Points: 4096) 119.55 µs 140.28 µs 🔴 +17%
ColorParseBenchmark.Parse(Iterations: 1000) 74.31 µs 87.02 µs 🔴 +17%
MatrixOpsBenchmark.Invert(Count: 4096) 131.64 µs 153.91 µs 🔴 +17%
MatrixMapPointsBenchmark.MapPoint(Points: 256) 4.75 µs 5.53 µs 🔴 +16%
MatrixMapPointsBenchmark.MapVector(Points: 4096) 128.56 µs 148.02 µs 🔴 +15%
BitmapDrawBenchmark.DrawScaledTiles(Tiles: 256) 1.24 ms 1.42 ms 🔴 +14%
MatrixMapPointsBenchmark.MapVector(Points: 256) 8.12 µs 9.21 µs 🔴 +13%
MatrixMapPointsBenchmark.MapPoint(Points: 4096) 78.44 µs 88.49 µs 🔴 +13%
RuntimeEffectShaderBenchmark.DrawFrame 1.13 ms 1.27 ms 🔴 +13%
BitmapDrawBenchmark.DrawUnscaledTiles(Tiles: 256) 910.71 µs 1.02 ms 🔴 +12%
BitmapDrawBenchmark.DrawScaledTiles(Tiles: 64) 335.06 µs 375.82 µs 🔴 +12%
LargeImageScaleBenchmark.UpscaleCrossfade(Size: 1024) 2.56 ms 2.84 ms 🔴 +11%
CanvasDrawBenchmark.Draw(Shapes: 64) 2.66 ms 2.96 ms 🔴 +11%
LargeImageScaleBenchmark.UpscaleOpaque(Size: 1024) 736.58 µs 815.30 µs 🔴 +11%
BitmapDrawBenchmark.DrawUnscaledTiles(Tiles: 64) 242.13 µs 267.43 µs 🔴 +10%
LargeImageScaleBenchmark.UpscaleOpaque(Size: 2048) 2.92 ms 3.21 ms 🔴 +10%
…and 4 more

macOS

⏱️ Time (vs 🌙 nightly 4.152.0-nightly.4)

Benchmark baseline this PR Δ
MatrixMapPointsBenchmark.MapRect(Points: 4096) 56.76 µs 47.14 µs 🟢 -17%
RasterImageLifecycleBenchmark.CreateRasterImage(Count: 256) 180.99 µs 154.86 µs 🟢 -14%
SceneRenderBenchmark.RenderFrame(Complexity: 4) 6.25 ms 5.40 ms 🟢 -14%
RuntimeEffectShaderBenchmark.DrawFrame 593.01 µs 512.47 µs 🟢 -14%
PathBoundsBenchmark.TightBounds(Points: 1024) 9.35 µs 8.12 µs 🟢 -13%
PathBoundsBenchmark.TightBounds(Points: 64) 999.9 ns 871.8 ns 🟢 -13%
MatrixMapPointsBenchmark.MapVector(Points: 4096) 45.19 µs 39.59 µs 🟢 -12%
RasterImageLifecycleBenchmark.CreateDataWithReleaseProc(Count: 256) 79.71 µs 71.05 µs 🟢 -11%
SceneRenderBenchmark.RenderFrame(Complexity: 1) 1.32 ms 1.18 ms 🟢 -11%
MatrixMapPointsBenchmark.MapPoints(Points: 4096) 1.53 µs 1.68 µs 🔴 +10%
MatrixMapPointsBenchmark.MapPoints(Points: 256) 131.9 ns 144.7 ns 🔴 +10%
MatrixMapPointsBenchmark.MapRadius(Points: 4096) 108.13 µs 98.22 µs 🟢 -9%
MatrixMapPointsBenchmark.MapRect(Points: 256) 2.99 µs 3.23 µs 🔴 +8%
BitmapDrawBenchmark.DrawScaledTiles(Tiles: 64) 192.35 µs 178.02 µs 🟢 -7%
ColorParseBenchmark.Parse(Iterations: 1000) 65.85 µs 69.37 µs 🔴 +5%

📈 Full interactive perf-dashboard & run details →

mattleibow and others added 12 commits August 6, 2026 01:55
Validate publish resources before queueing, make Azure approval boundaries explicit, handle release-notes concurrency and optional milestones, and preserve native Azure CLI output bytes across Windows encodings.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Dispatch the release-notes workflow once from main and treat GitHub's acceptance as completion, leaving concurrency and supersession to the workflow.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Keep source-run verification and confirmation in the release workflow, then queue pipeline 25298 from the verified managed build number with focused format and duplicate guards.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Use one deterministic ASCII renderer for status output while preserving platform-aware decoding of Azure CLI bytes. Cover cp1252 subprocess output through the ASCII-safe console path and remove the obsolete Unicode fallback behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Use the exact ASCII-only status vocabulary, reduce website notes refresh to a single dispatch, and move publish source and duplicate checks into the pre-confirmation workflow. Replace the queue helper with a compact build-number-only script and focused tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Keep source and duplicate-run verification in the publish workflow, then use a sub-100-line helper for one Azure queue POST. Remove the duplicate confirmation and decoding framework and focus tests on request construction and CLI outcomes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Inline the concise publish queue procedure, remove the obsolete Azure reference, and restore milestone auditing and exact optional closure for preview, RC, and stable releases.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Restore the original two-phase every-release milestone rationale and audit procedure, while retaining exact paginated lookup and non-fatal missing-milestone handling.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Inline ASCII rendering in the status emitter, remove dead empty-output branches, and collapse one-use publish queue classification and URL return layers without changing behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Restore presentation Unicode in release-status guidance while keeping literal pipeline output examples and the executable ASCII-only.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Describe only the non-ASCII escaping the renderer performs and avoid redundantly escaping an already validated hexadecimal SHA.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
Derive the exact SkiaSharp and HarfBuzzSharp package pair from the selected managed build, guard publish queueing, and verify each public version independently. Keep Appium guidance focused on actionable setup and align coupled release documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ae9429ab-3288-4306-bef2-334a88751a86
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

📦 Artifact size report

Packages from this PR (build 1541942) vs the latest nightly baseline 4.152.0-nightly.4 (observed 2026-08-05).

Total .nupkg size: 569.8 MB → 569.2 MB (−624.9 KB, -0.1%)

Packages

⚠️ marks growth over 500.0 KB or 2%. Changes under 50.0 KB are treated as noise.

Package baseline this PR Δ Δ%
SkiaSharp.NativeAssets.WinUI 109.1 MB 108.4 MB 🟢 −746.8 KB -0.7%
SkiaSharp.NativeAssets.WebAssembly 69.7 MB 69.7 MB 🔴 +78.9 KB +0.1%

+38 package(s) unchanged (< 50.0 KB).

Per-file changes

SkiaSharp.NativeAssets.WinUI

File Size
runtimes/win-arm64/native/libGLESv2.pdb 64.5 MB → 64.4 MB (🟢 −144.0 KB)
runtimes/win-x86/native/libGLESv2.pdb 67.3 MB → 67.2 MB (🟢 −112.0 KB)
runtimes/win-x64/native/libGLESv2.pdb 66.6 MB → 66.5 MB (🟢 −96.0 KB)

Informational only — this never blocks the PR. Native binaries are labelled by os/arch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant