fix(editor): avoid enlarging captures past native pixel density

[itsmunzir]

Addresses the soft annotator preview reported in #170. This branch now covers two demonstrated mechanisms, one of which is new since the first draft.

Problem

1. A screen denser than the document. CaptureEditor::baseImageRect() fits the document frame to the available screen area with no upper bound:

const qreal scale =
    std::min<qreal>({1.0, available.width() / framedSize.width(),
                     available.height() / framedSize.height()});

When the screen's device ratio is higher than the pixels the document actually carries — a fractional-scale monitor whose surface ratio Qt still reports rounded up, or a capture taken on a coarser monitor and edited on a finer one — that fit enlarges the source. The editor resamples the capture into invented pixels and it reads soft, while Copy/Save keep every native pixel.

2. A fractional region at native density (this is the new part). Even at one device pixel per source pixel, a region drag normally ends between source pixels: Wayland reports pointer coordinates as fixed point, so a drag of 300.37 × 180.21 logical units covers 301 × 181 source pixels, which pixelSelection rounds outward and the export keeps. The editor drew the fractional selection instead, resampling the whole preview through a sub-pixel scale — soft on screen next to the sharp PNG at any density, including plain 1×.

Solution

// src/capture.cpp
QSizeF sourcePixelDensity(const CaptureData &capture);  // declared in capture.hpp

• Native density cap. baseImageRect() also bounds the fit by one device pixel per document pixel, so the annotator never enlarges a capture past the pixels it carries:

  const QSizeF density = sourcePixelDensity(capture_);
  const qreal deviceRatio = devicePixelRatioF();
  const qreal nativeLimit =
      deviceRatio > 0.0
          ? std::min(density.width(), density.height()) / deviceRatio
          : 1.0;
  const qreal scale =
      std::min<qreal>({1.0, available.width() / framedSize.width(),
                       available.height() / framedSize.height(), nativeLimit});

• Native-grid frame. When the frame is at that density (scale * devicePixelRatioF() equal to the source density on an axis), sourceFrameWidgetRect() rounds the selection outward to the whole source pixels pixelSelection crops to, and draws that rect. The document keeps its own coordinates; layers still map through editScale.

The export path is untouched: the on-screen frame is the only thing bounded/snapped. README, docs/editing-model.md, and CHANGELOG.md describe the rule.

Verification (local, offscreen)

• Direct C++ syntax checks on the touched sources passed.
• tests/editor-smoke.cpp runScaledSourcePaintingCheck (exit 226): a 300×300 native source declared as a 600×600 logical document — the denser-screen case — painted 169 resampled device pixels on the unpatched code (baseline failed) and reads back identity-wise with the patch; it also asserts the rendered export still equals the source.
• New tests/editor-smoke.cpp runFractionalRegionPaintingCheck (exit 227): a region dragged between pixels (500.35, 600.42 → 800.72, 780.63) exports 301 × 181, the on-screen frame covers exactly those 301 × 181 device pixels (the previous behavior showed a 300 × 180 fractional region), and every inspected device pixel inside the frame matches the exported image (tolerance ≤ 2). A whole-pixel control drag keeps its 301 × 181 frame.
• A throwaway 1× reproduction of a paragraph: the fractional frame measured ≈ 7.6 gray levels of blur and ≈ 0.5 px of edge spread before the change, 0 after. (A screen recording attached to the earlier comment measured roughly 10 gray levels / ≈ 0.6 px.)
• Pixel probes over the painted frame report 0 mismatching pixels and 0 residual gray-level blur.

Not run

• make check (the full smoke suite) — this environment is missing CMake and LayerShellQt, so the suite could not be built. CI has not run and this has not been validated by CI yet.

Risk / limitations

• The exact reporter hardware is untested. These are local probes of the mechanism, not proof the reporter's own machine is fixed.
• Density > device ratio still downscales. This patch never enlarges and does not sharpen anything below native density: a capture larger than the available area is still downscaled to fit, and no detail is invented.
• No new tests beyond the graded smoke checks: there is no in-tree coverage of the reporter's compositor/monitor configuration.

Context on the earlier draft

The first draft framed the issue as high-DPI rounding only. That was speculation about the reporter's setup: the reporter stated above that they run the default 1× scale, not fractional scaling, so mechanism 2 (a fractional region resampled at native density) is the better fit for their report. The fractional path is now fixed, but the reporter should confirm on their hardware before this is treated as resolved.

Please retry

@sophie-offshorly — thanks for the recording; it pointed at the fractional-region path. Could you pull the branch and retry the same capture/edit? Please re-test on the same monitor, and if the preview is still soft, share the output resolution / compositor scale and a sample capture so the mechanism can be confirmed rather than inferred. This stays a draft until you confirm.

Fixes no issue yet: #170 stays open until the reporter confirms.

cc @sophie-offshorly

[sophie-offshorly]

hi @itsmunzir thanks for creating a PR for this! i'll test your fix on my computer to see if it resolves the issue. as an answer to your question, I'm not using fractional scaling, I'm just using the default (1x)

[sophie-offshorly]

I tested your fix and the preview still looks slightly blurry. Here's a screen record of the issue

screenrecording-2026-09-29_17-06-21.mp4

[itsmunzir]

Thanks again for the recording — it pointed at the right path. I've pushed a follow-up to the same branch (24a3f57) that keeps the frame on the export's whole-pixel grid: a region dragged between pixels — the usual case on Wayland, where pointer coordinates are fixed point — is now rounded outward to the pixels Copy/Save keep instead of being drawn as a fractional rect and resampled. At 1× the exported crop of the drag above is 301×181, while the preview used to show 300×180.

Since you're on the default 1× scale (not fractional), this fractional-region path is the better fit for what you saw than the high-DPI rounding the first draft blamed. Could you pull the branch and retry the same capture/edit? If the preview still looks soft, sharing your output resolution, compositor scale, and a sample capture would let me confirm the remaining mechanism instead of guessing. I'll keep this a draft until you've had a look.