Mongosync Insights v0.8.1.14

[BigMarcio]

Refactor the libraries
Add menu, credits, config, and themes
Reorganize plots, create sections, and info
Add index progress plots to Live Monitoring
Add Estimated oplog time
Add Estimated CEA Catchup
Prepare MI to handle the new mongosync internal database name
Add save plots and parsed logs feature, for easy reload.
Add embedded verifier lag plots
Add copy log in plain text button
Add error patterns
Add log viewer
Fix bugs
Change layout for the new format
Update README

[Copilot]

Pull request overview

This PR updates Mongosync Insights to v0.8.1.14, focusing on a UI/layout refresh plus new analysis persistence and log exploration capabilities (including full-text search), while refactoring core code into a lib/ package and updating live monitoring/verifier endpoints.

Changes:

• Refactor app modules into migration/mongosync_insights/lib/ and update routes/template endpoints accordingly.
• Add snapshot persistence (save/load/delete) plus a SQLite-backed log store with FTS5 for Log Viewer search.
• Redesign templates: new sidebar (upload/settings/logout/credits), theme + color scheme support, refresh-time overrides, and expanded charts/log viewer UI.

Reviewed changes

Copilot reviewed 23 out of 29 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
migration/mongosync_insights/templates/verifier_metrics.html	Update verifier AJAX endpoint and refresh behavior.
migration/mongosync_insights/templates/upload_results.html	Add log viewer tab, sortable tables, copy actions, chart info overlays.
migration/mongosync_insights/templates/metrics.html	Update live monitoring endpoints, add endpoint warnings UI, refresh override.
migration/mongosync_insights/templates/home.html	Update form actions, add “Previous Analyses” list + duplicate upload dialog.
migration/mongosync_insights/templates/base.html	New sidebar navigation, settings/logout/credits overlays, upload dialog + duplicate checks.
migration/mongosync_insights/templates/base_home.html	Theme variables include + upload loading overlay styling.
migration/mongosync_insights/templates/_theme_vars.html	Theme/scheme variables + pre-paint theme application.
migration/mongosync_insights/requirements.txt	Add certifi dependency.
migration/mongosync_insights/README.md	Document new features (sidebar, snapshots, log viewer) and bump version text.
migration/mongosync_insights/mongosync_plot_utils.py	Remove legacy byte formatting helpers (moved to lib/utils.py).
migration/mongosync_insights/mongosync_insights.py	Refactor imports, rename routes, add logout/search/snapshot endpoints, cleanup hooks.
migration/mongosync_insights/lib/utils.py	Centralized byte size conversion helpers.
migration/mongosync_insights/lib/snapshot_store.py	New snapshot persistence layer (save/load/list/delete/cleanup).
migration/mongosync_insights/lib/otel_metrics.py	Add section annotations to metrics plots; remove unused helper.
migration/mongosync_insights/lib/mongosync_metrics.json	Add metrics catalog/metadata for plot organization.
migration/mongosync_insights/lib/migration_verifier.py	Update imports after refactor.
migration/mongosync_insights/lib/logs_metrics.py	Add log viewer tail buffer + SQLite log store build + snapshot save.
migration/mongosync_insights/lib/log_store.py	New SQLite+FTS5 log document store.
migration/mongosync_insights/lib/log_store_registry.py	New registry to map store_id to SQLite DB files (with TTL cleanup).
migration/mongosync_insights/lib/live_migration_metrics.py	Update internal DB name detection, endpoint plot rework, return warnings + plot payload.
migration/mongosync_insights/lib/file_decompressor.py	Update imports after refactor.
migration/mongosync_insights/lib/error_patterns.json	Add new error patterns for CEA failures/timeouts/fatal errors.
migration/mongosync_insights/lib/connection_validator.py	Move/introduce connection string sanitization helper.
migration/mongosync_insights/lib/app_config.py	Add version bump, credits, log viewer/store config, internal DB name auto-detection.
migration/mongosync_insights/lib/init.py	Package initializer.
migration/mongosync_insights/CONFIGURATION.md	Update prerequisites and document new log viewer/snapshot settings.

Comments suppressed due to low confidence (1)

migration/mongosync_insights/lib/logs_metrics.py:338

• The SQLite LogStore connection created for the upload (log_store = LogStore(db_path)) is never closed on the success path. This can leak file handles and keep SQLite/WAL files open longer than necessary. After flushing/building the FTS index (and before returning), explicitly close() the log_store instance (the registry can open its own connection later).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.