Skip to content

feat: implement Data Migration page - Excel upload, history, filters, send to Loan Queue - #28

Open
Abd1920 wants to merge 25 commits into
mainfrom
feature/data-migration
Open

Abd1920 wants to merge 25 commits into
mainfrom
feature/data-migration

Conversation

@Abd1920

@Abd1920 Abd1920 commented May 6, 2026

Copy link
Copy Markdown
Collaborator

Data Migration Feature Implementation

Overview

A new Data Migration page for data-entry role users to import historical
loan data from Excel files into the IDB system.

New Files

  • Client/src/pages/DataMigration.jsx - Full migration UI
  • server/src/models/migratedLoan.js - MigratedLoan MongoDB model
  • server/src/controllers/migrationController.js - Upload, list, send to queue, delete
  • server/src/routes/migrationRoutes.js - API routes

Features

Upload Tab

  • Drag & drop or click to browse Excel files (.xlsx, .xls)
  • Excel format requirements guide with required/optional fields
  • Step-by-step migration process guide
  • Download Excel template button
  • Import result summary (total, imported, skipped rows)
  • Skipped row error details shown after upload

Migration History Tab

  • All imported records with migration status badges
  • Checkbox selection (individual + select all)
  • Send selected records to Loan Queue
  • Delete individual records
  • Search by name or NIC
  • Filter by Region, Sector, Batch
  • Clear all filters option
  • Pagination

Sent to Queue Tab

  • Records already sent to Loan Queue
  • View in Loan Queue button per record
  • Same filter options

Tab Persistence

  • Active tab saved in sessionStorage
  • Refreshing browser stays on current tab

Backend

  • POST /api/migration/upload - Parse Excel, validate rows, save to DB
  • GET /api/migration - List with search, region, sector, batch filters
  • GET /api/migration/batches - Grouped batch summary
  • POST /api/migration/send-to-queue - Create real Loan records from selected
  • DELETE /api/migration/:id - Remove migration record

Packages Installed

  • xlsx - Excel file parsing
  • uuid - Unique batch ID generation
  • multer - File upload handling (memory storage)

Access Control

  • Only data-entry role can access /migration route
  • Protected at both route level (App.jsx) and sidebar level

fathimachamha2001-lang and others added 25 commits April 2, 2026 10:57
fix: remove duplicate root-level .gitignore and README files
fix: remove duplicate root-level README.md file
@Abd1920 Abd1920 self-assigned this May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants