Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/repo-conventions/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ Subsystem conventions and invariants for KMReader. `AGENTS.md` holds repo-wide r
- Runtime GRDB migrations in `LocalDatabase` are immutable once committed: never mutate an already-registered migration (e.g. `create_runtime_schema_v1`, `00002_add_protected_server_flag`) or its helpers; they are the frozen baseline. Any table shape change is a new numbered migration after the latest one; fresh installs run baseline + all later migrations in order.
- Migration bodies and schema helpers belong only in `LocalDatabase.swift`; record models and feature stores must not define migration methods or table schemas. New migrations use migration-local literal table/column names rather than current model symbols so later model changes cannot rewrite migration history; do not rewrite frozen migrations solely to retrofit this rule.
- New persisted field: update the record model and `CodingKeys`, then add a migration backfilling a safe default. Validate both upgrade and fresh-install paths.
- Manga translations are isolated per Komga instance: `MangaTranslationPageKey` includes the instance id, source/geometry/provider/model signatures, and rows use `(instance_id, cache_key)` as the primary key. Each save prunes same-page rows whose model signature retired (legacy empty signature, or the same archive on an older inference version); other archives and same-signature language/provider/geometry variants survive, so stale entries converge lazily as pages are revisited. Each attempt has an owner; cancelled or superseded attempts must not update reader state or overwrite newer cached results. The format-version-2 model catalog and matching package manifests declare capabilities and per-component engine/version/files/labels; detector and recognizer are separate adapters, unknown engines fail before download, and manga translation consumes only supported `manga_ocr` archives. Two detector engines exist: `rt_detr_v2` (text-tight boxes, the label table drives recognize/orientation) and `yolo11n_seg` (single `bubble` label, whole-bubble boxes; the CoreML export drops the mask-proto branch because regions carry boxes only, confidence arrives sigmoid-squashed in-graph, input is the same stretched 640×640). Two recognizer engines share one CoreML I/O contract and the `MangaGreedyDecoder` loop, differing only in preprocessing selected per engine via `MangaOCRPreprocessing`: `manga_ocr_transformer` (stretch to 224×224, [-1,1]) and `manga_ocr_mobile` (letterbox to 224×224 on white, [0,1]); the letterbox path resamples with a hand-rolled bilinear filter matching the training-time coordinate mapping — CoreGraphics scaling shifts pixel centers and flips glyphs. Detection and recognition models are selected independently per role (`MangaModelRole`, storage keys `manga_ocr.detector`/`.recognizer`, the legacy combined key serves as fallback until the next select, and is dropped once both roles have explicit picks — downgrading past the role keys then finds no selection, though downloads survive and re-selecting fixes it): an archive stays the distribution unit, but the two roles may come from different archives, the one-tap select writes only the roles the archive supports, the management sheet offers each role only role-capable archives (`MangaModelSet.supports(_:for:)`, which deliberately tolerates unknown extra components so future components do not break this build's downloads), and the translation service keeps one prepared `MangaModelSet` per selected archive (two at most, shared when both roles pick the same archive). The model signature is `<detector sha256>:<recognizer sha256>:<inference cache version>`, so cache pruning keys on the detector prefix and legacy single-archive signatures retire through the same prefix. The catalog ships only role-split archives (capabilities `detection` or `ocr` with a single component); the legacy combined archives were withdrawn from the catalog, and already-downloaded ones keep working because the downloaded-model registry is independent of the catalog. Model rows show the archive's capability as chips (`MangaModelCapabilityChips`, derived from its components) alongside the catalog comment. Bundles are downloaded from R2 with byte-level progress feeding the management row's progress bar, SHA-256 verified, and stored under `Application Support/Models` in per-purpose, per-archive namespaces excluded from device backups; hashing and TAR extraction stream off MainActor, the downloaded-model registry is independent of the current catalog, and models never ship in the app bundle. Translation uses a selected OpenAI-compatible provider when configured, otherwise the on-device system provider on iOS 26+/macOS 26+; older versions report missing configuration before OCR instead of doing wasted recognition. Provider configurations, including API keys, live in GRDB. OpenAI-compatible translation and model-list requests share a 30-second hard timeout in `MangaProviderRequest`. Source language prefers series metadata before the default, target language defaults to the system language, comic metadata is optional in the LLM prompt, and overlay geometry is clamped to the displayed page bounds; bubbles keep an opaque center with a feathered edge and take the sepia tint used by the reader. Detector labels without orientation metadata fall back to physical box geometry (taller than wide, compared in display pixels — normalized heights on portrait pages hide near-square vertical bubbles), and each `MangaPageTranslation` stores its `targetLanguage` (legacy rows decode as empty). OCR crops are cut from the unrotated source image and brought into reading orientation before recognition (`MangaInferenceUtilities.readingOrientation` applies the page rotation only); the recognizer reads upright vertical text natively, so vertical columns must never be de-rotated — sideways glyphs are unreadable to it. Overlay text renders vertically when the region's resolved orientation says vertical — `unknown` from legacy caches falls back to a conservative normalized-box comparison — and the result's target language supports it (`MangaTranslationLanguage.supportsVerticalLayout`: Chinese/Japanese only, other targets stay horizontal); the `textDirection` preference (auto/horizontal/vertical) overrides the gate, with horizontal and vertical forcing one axis for every region. Vertical layout draws through CoreText vertical forms in `MangaVerticalTextCompositor`/`MangaVerticalTextLabel` with columns flowing right-to-left at a pinned 1.5em pitch, shrink-to-fit sizing, and a scale-to-fit fallback instead of clipping; the compositor also rewrites punctuation for vertical rendering (half-width to full-width, Unicode vertical presentation forms, one-cell compression of consecutive !?/?!/!!/?? with VS15 pinned on the emoji-capable marks), so the rendered text intentionally differs from the cached translation string. Loaded Core ML models are released when the reader closes and on memory warning without deleting their packages. Compute units are split per component: the detector runs on `.cpuAndGPU` (fixed 640×640 input, and `.all` distorts boxes on device) while the recognizer always runs on `.cpuOnly` — GPU/ANE specialize the decoder graph per sequence bucket and each specialization adds gigabytes of compile state, which memory-kills small devices when a decode crosses into long buckets; CPU speed is on par with flat memory. The mobile recognizer's encoder ships fp32 compute because fp16-compute convolutions diverge between ANE and CPU (CPU-run output flipped decoded glyphs), and the recognizer only ever runs on CPU. Live Text and local LLMs are not used.
- Manga translations are isolated per Komga instance: `MangaTranslationPageKey` includes the instance id, source/geometry/provider/model signatures, and rows use `(instance_id, cache_key)` as the primary key. Each save prunes same-page rows whose model signature retired (legacy empty signature, or the same archive on an older inference version); other archives and same-signature language/provider/geometry variants survive, so stale entries converge lazily as pages are revisited. Each attempt has an owner; cancelled or superseded attempts must not update reader state or overwrite newer cached results. The format-version-2 model catalog and matching package manifests declare capabilities and per-component engine/version/files/labels; detector and recognizer are separate adapters, unknown engines fail before download, and manga translation consumes only supported `manga_ocr` archives. Two detector engines exist: `rt_detr_v2` (text-tight boxes, the label table drives recognize/orientation) and `yolo11n_seg` (single `bubble` label, whole-bubble boxes; the CoreML export drops the mask-proto branch because regions carry boxes only, confidence arrives sigmoid-squashed in-graph, input is the same stretched 640×640). Two recognizer engines share one CoreML I/O contract and the `MangaGreedyDecoder` loop, differing only in preprocessing selected per engine via `MangaOCRPreprocessing`: `manga_ocr_transformer` (stretch to 224×224, [-1,1]) and `manga_ocr_mobile` (letterbox to 224×224 on white, [0,1]); the letterbox path resamples with a hand-rolled bilinear filter matching the training-time coordinate mapping — CoreGraphics scaling shifts pixel centers and flips glyphs. Detection and recognition models are selected independently per role (`MangaModelRole`, storage keys `manga_ocr.detector`/`.recognizer`, the legacy combined key serves as fallback until the next select, and is dropped once both roles have explicit picks — downgrading past the role keys then finds no selection, though downloads survive and re-selecting fixes it): an archive stays the distribution unit, but the two roles may come from different archives, the one-tap select writes only the roles the archive supports, the management sheet offers each role only role-capable archives (`MangaModelSet.supports(_:for:)`, which deliberately tolerates unknown extra components so future components do not break this build's downloads), and the translation service keeps one prepared `MangaModelSet` per selected archive (two at most, shared when both roles pick the same archive). The model signature is `<detector sha256>:<recognizer sha256>:<inference cache version>`, so cache pruning keys on the detector prefix and legacy single-archive signatures retire through the same prefix. The catalog ships only role-split archives (capabilities `detection` or `ocr` with a single component); the legacy combined archives were withdrawn from the catalog, and already-downloaded ones keep working because the downloaded-model registry is independent of the catalog. Model rows show the archive's capability as chips (`MangaModelCapabilityChips`, derived from its components) alongside the catalog comment. Bundles are downloaded from R2 with byte-level progress feeding the management row's progress bar, SHA-256 verified, and stored under `Application Support/Models` in per-purpose, per-archive namespaces excluded from device backups; hashing and TAR extraction stream off MainActor, the downloaded-model registry is independent of the current catalog, and models never ship in the app bundle. Translation uses a selected OpenAI-compatible provider when configured, otherwise the on-device system provider on iOS 26+/macOS 26+; older versions report missing configuration before OCR instead of doing wasted recognition. Provider configurations, including API keys, live in GRDB. OpenAI-compatible translation and model-list requests share a 30-second hard timeout in `MangaProviderRequest`. Source language prefers series metadata before the default, target language defaults to the system language, comic metadata is optional in the LLM prompt, and overlay geometry is clamped to the displayed page bounds; bubble backgrounds are feathered-edged blobs with an opaque center, tinted with the reader's sepia background; the shape is the user's `bubbleShape` preference (`ellipse` default, `roundedRect`, a rendering-only preference that takes effect through the settings-dismiss page refresh) and the sizing follows the detector's box semantics — each page stores the detector's `MangaTextBoxKind` (`text` for text-tight rt_detr boxes, `bubble` for whole-bubble yolo boxes; legacy cached rows decode as `text`, whose outgrown ellipse keeps corner text covered for either kind) and `MangaBubbleBackground` derives the background and text frames from the shape/kind pair: ellipses on text-tight boxes outgrow ×1.5 so the corners stay covered (√2 minimum), ellipses on whole-bubble boxes stay inscribed with the text area shrunk to 0.5 of the box so glyphs stay on the opaque core with a manga-like white margin, and rounded rectangles hug the box for either kind with the small-inset text area. Detector labels without orientation metadata fall back to physical box geometry (taller than wide, compared in display pixels — normalized heights on portrait pages hide near-square vertical bubbles), and each `MangaPageTranslation` stores its `targetLanguage` (legacy rows decode as empty). OCR crops are cut from the unrotated source image and brought into reading orientation before recognition (`MangaInferenceUtilities.readingOrientation` applies the page rotation only); the recognizer reads upright vertical text natively, so vertical columns must never be de-rotated — sideways glyphs are unreadable to it. Overlay text renders vertically when the region's resolved orientation says vertical — `unknown` from legacy caches falls back to a conservative normalized-box comparison — and the result's target language supports it (`MangaTranslationLanguage.supportsVerticalLayout`: Chinese/Japanese only, other targets stay horizontal); the `textDirection` preference (auto/horizontal/vertical) overrides the gate, with horizontal and vertical forcing one axis for every region. Vertical layout draws through CoreText vertical forms in `MangaVerticalTextCompositor`/`MangaVerticalTextLabel` with columns flowing right-to-left at a pinned 1.5em pitch, shrink-to-fit sizing, and a scale-to-fit fallback instead of clipping; the compositor also rewrites punctuation for vertical rendering (half-width to full-width, Unicode vertical presentation forms, one-cell compression of consecutive !?/?!/!!/?? with VS15 pinned on the emoji-capable marks), so the rendered text intentionally differs from the cached translation string. Loaded Core ML models are released when the reader closes and on memory warning without deleting their packages. Compute units are split per component: the detector runs on `.cpuAndGPU` (fixed 640×640 input, and `.all` distorts boxes on device) while the recognizer always runs on `.cpuOnly` — GPU/ANE specialize the decoder graph per sequence bucket and each specialization adds gigabytes of compile state, which memory-kills small devices when a decode crosses into long buckets; CPU speed is on par with flat memory. The mobile recognizer's encoder ships fp32 compute because fp16-compute convolutions diverge between ANE and CPU (CPU-run output flipped decoded glyphs), and the recognizer only ever runs on CPU. Live Text and local LLMs are not used.

### Read List Reading State

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import Foundation

/// User-selected bubble background shape; `MangaBubbleBackground` derives the
/// background and text geometry from it.
nonisolated enum MangaBubbleShape: String, Codable, Hashable, Sendable {
case ellipse
case roundedRect
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ nonisolated struct MangaPageTranslation: Codable, Hashable, Sendable {
let regions: [MangaTextRegion]
let status: MangaTranslationStatus
let targetLanguage: String
let boxKind: MangaTextBoxKind
let createdAt: Date

var isTranslated: Bool { status == .translated }
Expand All @@ -20,12 +21,14 @@ nonisolated struct MangaPageTranslation: Codable, Hashable, Sendable {
regions: [MangaTextRegion],
status: MangaTranslationStatus,
targetLanguage: String,
boxKind: MangaTextBoxKind,
createdAt: Date
) {
self.key = key
self.regions = regions
self.status = status
self.targetLanguage = targetLanguage
self.boxKind = boxKind
self.createdAt = createdAt
}

Expand All @@ -37,6 +40,9 @@ nonisolated struct MangaPageTranslation: Codable, Hashable, Sendable {
// Rows cached before the field existed fall back to an unknown
// language, which keeps their rendering horizontal.
targetLanguage = try container.decodeIfPresent(String.self, forKey: .targetLanguage) ?? ""
// Rows cached before the field existed fall back to text-tight geometry:
// its outgrown ellipse keeps corner text covered for either box kind.
boxKind = try container.decodeIfPresent(MangaTextBoxKind.self, forKey: .boxKind) ?? .text
createdAt = try container.decode(Date.self, forKey: .createdAt)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import Foundation

/// What a detector's box bounds: text-tight boxes (rt_detr) wrap the glyphs,
/// whole-bubble boxes (yolo) wrap the entire bubble. The overlay derives the
/// bubble background and text area geometry from this.
nonisolated enum MangaTextBoxKind: String, Codable, Hashable, Sendable {
case text
case bubble
}
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,24 @@ nonisolated struct MangaTranslationPreferences: Codable, Equatable, RawRepresent
var includeMetadata: Bool
var showTranslation: Bool
var textDirection: MangaTranslationTextDirection
var bubbleShape: MangaBubbleShape

init(
selectedProviderID: String = "",
sourceLanguage: String = MangaTranslationLanguage.defaultSource,
targetLanguage: String = "",
includeMetadata: Bool = false,
showTranslation: Bool = true,
textDirection: MangaTranslationTextDirection = .auto
textDirection: MangaTranslationTextDirection = .auto,
bubbleShape: MangaBubbleShape = .ellipse
) {
self.selectedProviderID = selectedProviderID
self.sourceLanguage = sourceLanguage
self.targetLanguage = targetLanguage
self.includeMetadata = includeMetadata
self.showTranslation = showTranslation
self.textDirection = textDirection
self.bubbleShape = bubbleShape
}

init?(rawValue: String) {
Expand All @@ -45,7 +48,8 @@ nonisolated struct MangaTranslationPreferences: Codable, Equatable, RawRepresent
includeMetadata: try container.decodeIfPresent(Bool.self, forKey: .includeMetadata) ?? false,
showTranslation: try container.decodeIfPresent(Bool.self, forKey: .showTranslation) ?? true,
textDirection: try container.decodeIfPresent(
MangaTranslationTextDirection.self, forKey: .textDirection) ?? .auto
MangaTranslationTextDirection.self, forKey: .textDirection) ?? .auto,
bubbleShape: try container.decodeIfPresent(MangaBubbleShape.self, forKey: .bubbleShape) ?? .ellipse
)
}

Expand All @@ -57,6 +61,7 @@ nonisolated struct MangaTranslationPreferences: Codable, Equatable, RawRepresent
try container.encode(includeMetadata, forKey: .includeMetadata)
try container.encode(showTranslation, forKey: .showTranslation)
try container.encode(textDirection, forKey: .textDirection)
try container.encode(bubbleShape, forKey: .bubbleShape)
}

nonisolated var rawValue: String {
Expand All @@ -73,5 +78,6 @@ nonisolated struct MangaTranslationPreferences: Codable, Equatable, RawRepresent
case includeMetadata
case showTranslation
case textDirection
case bubbleShape
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import CoreGraphics
import Foundation

nonisolated protocol MangaTextDetecting: Sendable {
var boxKind: MangaTextBoxKind { get }
func detect(
image: CGImage,
splitMode: PageSplitMode,
Expand Down
Loading