🛡️ DataQualityAI (ByteFlow)

GenAI-Powered Stateless Data Governance & Risk Analysis Platform

---

📖 Overview

DataQualityAI (Project ByteFlow) is a sophisticated data analysis tool designed to assess, score, and remediate data quality issues in financial transaction datasets.

Unlike traditional tools that store sensitive data, DataQualityAI operates on a Zero-Persistence Architecture. It ingests data streams into volatile memory (RAM), calculates statistical metadata, and utilizes Google Gemini Pro to generate risk insights without ever committing Personal Identifiable Information (PII) to a persistent database.

Key Features

• Zero-Persistence Mode: Stateless processing using Python BytesIO streams ensures maximum privacy and GDPR compliance.
• GenAI Analyst: Automated risk assessment and remediation suggestions powered by Google Gemini.
• DQS Scoring Engine: Proprietary algorithm that calculates a Data Quality Score based on Validity, Completeness, Consistency, Timeliness, and Accuracy.
• Tactical Dashboard: A cyberpunk-inspired, dark-mode UI built with React for high-contrast visualization.
• Interactive Remediation: One-click generation of fix guides for detected data issues.

---

Architecture

The system follows a decoupled Client-Server architecture designed for security:

1. Frontend (React): Handles file uploads via a secure gateway and renders visualizations (Gauge charts, Dimension bars).
2. Ingestion Layer (Flask): Accepts CSV/Excel files and reads them directly into memory.
3. Processing Engine (Pandas): Extracts statistical metadata (null counts, type mismatches) and destroys the raw dataframe immediately after processing.
4. Reasoning Layer (Gemini): The metadata (not the raw data) is sent to the LLM to generate narrative insights.

---

Installation & Setup

Prerequisites

• Node.js & npm
• Python 3.9+
• Google Gemini API Key

1. Clone the Repository

git clone https://github.com/your-username/ByteFlow.git
cd ByteFlow

2. Backend Setup (Flask)

Navigate to the backend folder to set up the Python environment.

cd backend

Create virtual environment (Recommended):

python -m venv .venv

Activate it:

• Windows: .venv\Scripts\activate
• Mac/Linux: source .venv/bin/activate

Install dependencies:

pip install flask flask-cors pandas openpyxl google-generativeai python-dotenv

Configuration: Create a file named .env in the backend/ directory and add your API key:

GOOGLE_API_KEY=your_actual_api_key_here

Run Server:

python app.py

Output: Running on http://127.0.0.1:5000

3. Frontend Setup (React)

Open a new terminal window and navigate to the frontend folder.

cd frontend

Install Node modules:

npm install

Start the Application:

npm start

Opens http://localhost:3000 in your browser

---

Usage Guide

1. Select Profile: Choose a use case (e.g., KYC, Fraud, Analytics) on the dashboard top bar.
2. Ingest Data: Drag and drop a CSV or Excel file into the "Secure Ingestion Gateway" box.
3. Analyze: The system will process the file in milliseconds.
   • DQS Score: View the overall health of your data.
   • Dimensions: Check specific metrics like Completeness or Validity.
4. Remediate: Click "Apply Fix" on the suggested remediation cards to get step-by-step guides.
5. Chat: Use the floating AI Chatbot button to ask questions like "Why is the validity score so low?".

---