Hindsight

AI Tool Usage Analyzer & Workflow Optimizer

Hindsight analyzes your AI tool usage and shell history to identify patterns and suggest concrete optimizations. Learn from your actual workflows to improve productivity with MCP servers, custom skills, hooks, and automations.

Overview

Hindsight analyzes your development workflows to surface optimization opportunities you might otherwise miss. By examining shell history and AI tool interactions, it identifies patterns and generates actionable recommendations.

Supported Data Sources:

Shell history via McFly
Claude Code sessions (exported using claude-code-exporter)
Claude Desktop conversations (exported via Claude Desktop's export feature)
Extensible architecture for additional sources

Generated Recommendations:

🔌 MCP Server suggestions - Identify frequently-accessed APIs that could benefit from MCP integration
🎯 Custom Skills - Auto-generate reusable skills from repeated prompt patterns
⚡ Hooks & Automations - Detect command sequences suitable for automation
📝 Output Style Preferences - Codify formatting and style preferences
🔗 Integration Opportunities - Surface gaps where tools could be better connected

Features

🔍 Pattern Analysis

Command frequency analysis by directory and time
Sequence detection (common command chains)
Error pattern identification
AI interaction pattern analysis
Cross-tool workflow analysis

💡 Smart Recommendations

MCP Servers - Ranked by usage frequency
Custom Skills - Generated from repeated instructions
Hooks - Pre/post command automations
Aliases - For frequently used command sequences
Style Preferences - Automatic configuration suggestions

📊 Visual Dashboard

Web interface at http://localhost:8000
Pattern visualizations
Recommendation prioritization
Implementation templates
Impact tracking

⚙️ CLI Commands

# Interactive setup wizard (first-time setup)
uv run hindsight setup

# Run analysis (incremental by default)
uv run hindsight analyze

# Run full analysis (re-analyze all data)
uv run hindsight analyze --full

# Enable AI-powered template generation
uv run hindsight analyze --ai-templates

# Run with verbose error output
uv run hindsight analyze --verbose

# Start web interface
uv run hindsight serve

# Start on custom port
uv run hindsight serve --port 3000

# Monitor data sources for changes (auto-analyze)
uv run hindsight monitor

# Monitor with custom interval (seconds)
uv run hindsight monitor --interval 600

# Discover data sources (optional)
uv run hindsight init

Quick Start

Prerequisites

Python 3.11 or higher
uv package manager
At least one supported data source (see Data Export Guide below)

Installation

# Clone the repository
git clone https://github.com/lcatlett/hindsight.git
cd hindsight

# Run interactive setup wizard
uv run hindsight setup

The setup wizard will:

Detect available data sources on your system
Configure enabled sources
Generate an optimized configuration file
Validate paths and permissions
Optionally run a sample analysis

Data Export Guide

Before running Hindsight, you'll need to export data from your AI tools:

McFly (Shell History)

Install McFly: cantino/mcfly
Use it for a while to build up command history
Database is automatically created at ~/Library/Application Support/McFly/history.db (macOS)

Claude Code Sessions

Use a GitHub package to export your Claude Code sessions (search for claude-code-exporter)
Export sessions to a directory (e.g., ~/Desktop/sessions)

Configure the path in config/config.yaml:

claude_code:
  enabled: true
  path: ~/Desktop/sessions

Claude Desktop Conversations

Open Claude Desktop
Use the built-in export feature to export your conversations
Save to a directory (e.g., ~/Desktop/desktop)

Configure the path in config/config.yaml:

claude_desktop:
  enabled: true
  path: ~/Desktop/desktop

Note: Hindsight analyzes exported data locally. Your conversations and command history never leave your machine.

Usage

# Run analysis on your data
uv run hindsight analyze

# Start the web dashboard
uv run hindsight serve

# Open http://localhost:8000 to view recommendations

Configuration

Configuration is managed in config/config.yaml.

Quick Start:

# Copy the example configuration
cp config/config.example.yaml config/config.yaml

# Edit with your paths
nano config/config.yaml

Or use the interactive setup wizard:

uv run hindsight setup

Complete Configuration Reference

data_sources:
  mcfly:
    enabled: true
    path: ~/Library/Application Support/McFly/history.db
    # Path to McFly SQLite database

  claude_code:
    enabled: true
    path: ~/Desktop/sessions
    # Path to exported Claude Code sessions directory
    # Export using: https://github.com/search?q=claude-code-exporter
    # Set to null for auto-detection from standard locations

  claude_desktop:
    enabled: true
    path: ~/Desktop/desktop
    # Path to exported Claude Desktop conversations directory
    # Export via Claude Desktop's export feature
    # Set to null for auto-detection from standard locations

analysis:
  lookback_days: 30
  # Number of days of history to analyze (default: 30)

  min_sequence_frequency: 3
  # Minimum times a command sequence must appear to be considered (default: 3)

  min_command_frequency: 3
  # Minimum times a command must appear to be considered (default: 3)

  sequence_length: 3
  # Number of commands in a sequence to detect (default: 3)

output:
  database: ~/.hindsight/analytics.db
  # SQLite database path for storing analysis results

  cache_dir: ~/.hindsight/cache
  # Directory for caching intermediate analysis data

recommendations:
  ai_templates:
    enabled: true
    # Enable AI-powered custom template generation (requires ANTHROPIC_API_KEY)

    cache_dir: ~/.hindsight/template_cache
    # Directory for caching AI-generated templates

  min_pattern_confidence: 0.3
  # Minimum confidence score (0.0-1.0) for pattern-based recommendations

  effort_thresholds:
    quick_win: 2
    # Maximum hours for "quick win" effort level

    medium: 8
    # Maximum hours for "medium" effort level

    project: 40
    # Maximum hours for "project" effort level
    # Anything above is considered "epic"

Data Source Setup

McFly:

Install from cantino/mcfly
Database location is typically ~/Library/Application Support/McFly/history.db (macOS)

Claude Code:

Export sessions using a GitHub package like claude-code-exporter
Point path to the exported sessions directory
Set path: null for auto-detection from ~/Desktop/sessions, ~/Documents/claude-sessions, or ~/.claude/exports/sessions

Claude Desktop:

Export conversations via Claude Desktop's built-in export feature
Point path to the exported conversations directory
Set path: null for auto-detection from ~/Desktop/desktop, ~/Documents/claude-desktop, or ~/.claude/exports/desktop

📖 For complete configuration documentation, see CONFIGURATION.md

How It Works

Analysis Pipeline

# Run analysis on your data
uv run hindsight analyze

The analysis pipeline:

Loads command history and AI sessions from configured sources
Detects patterns, sequences, and error conditions
Generates prioritized recommendations
Stores results in a local SQLite database

Web Dashboard

# Start the web interface
uv run hindsight serve

The dashboard (http://localhost:8000) provides:

Prioritized recommendations with impact scores
Pattern visualizations and frequency charts
Implementation templates for each recommendation
Accept/dismiss workflow for tracking actions

Example Output

🔍 Running Hindsight Analysis...
✓ Config loaded
✓ Loaded 15,432 commands, 127 sessions

Analyzing patterns...
✓ Found 43 frequent commands
✓ Found 12 command sequences

Generating recommendations...
✓ Generated 8 recommendations

📊 Top 3 Recommendations:

1. Install GitHub MCP Server (priority: 47.2)
   47 commands could benefit from GitHub MCP integration
   Effort: quick_win, Impact: 8.5/10

2. Automate Git Commit Workflow (priority: 42.8)
   Sequence detected 23 times with 83% consistency
   Effort: quick_win, Impact: 7.0/10

3. Create 'gcp' alias (priority: 38.5)
   Command sequence used 31 times
   Effort: quick_win, Impact: 5.0/10

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Data Ingestion Layer                     │
├─────────────────────────────────────────────────────────────┤
│  McFly Reader  │  Claude Code  │  Claude Desktop │  Augment │
└────────────────┬────────────────────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────────────────────┐
│                   Pattern Analysis Engine                   │
├─────────────────────────────────────────────────────────────┤
│  Frequency  │  Sequences  │  Errors  │  AI Patterns  │ etc. │
└────────────────┬────────────────────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────────────────────┐
│                   Recommendation Engine                     │
├─────────────────────────────────────────────────────────────┤
│  MCP Servers  │  Skills  │  Hooks  │  Aliases  │  Styles   │
└────────────────┬────────────────────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────────────────────┐
│              Reporting & Interface (FastAPI + Web)          │
└─────────────────────────────────────────────────────────────┘

Project Structure

hindsight/
├── src/hindsight/
│   ├── readers/          # Data source readers (McFly, Claude Code, Claude Desktop)
│   ├── analyzers/        # Pattern detection and analysis
│   ├── recommendations/  # Recommendation generation engine
│   ├── db/              # Database schema and repositories
│   ├── api/             # FastAPI web backend
│   └── cli/             # Command-line interface
├── tests/               # Comprehensive test suite
├── config/              # Configuration files
├── QUICKSTART.md        # Quick start guide
└── README.md            # This file

Development

Running Tests

# Run all tests
uv run pytest tests/ -v

# Run with coverage report
uv run pytest tests/ --cov=src/hindsight --cov-report=html

# Run specific test file
uv run pytest tests/analyzers/test_sequence_detector.py -v

Code Quality

# Linting
uv run ruff check src/

# Type checking
uv run mypy src/

# Formatting
uv run ruff format src/

Extending Hindsight

To add a new data source reader:

Create a new reader class in src/hindsight/readers/
Implement the DataReader interface
Add configuration options to config/config.yaml
Register the reader in the analysis pipeline
Add tests in tests/readers/

See existing readers (McFly, Claude Code) for reference implementations.

FAQ

Q: How do I export my Claude Code sessions? A: Use a GitHub package like claude-code-exporter to export your sessions to a local directory. Point Hindsight to this directory in the configuration.

Q: How do I export my Claude Desktop conversations? A: Use Claude Desktop's built-in export feature to export conversations to a local directory. Configure the path in config/config.yaml.

Q: Do I need all data sources configured? A: No. Hindsight works with any combination of supported data sources. More sources provide richer insights, but the tool is useful with even a single source.

Q: What if I don't have McFly installed? A: Hindsight works without McFly, though it provides the most comprehensive command history data. You can use Claude Code and Claude Desktop sessions alone.

Q: Does this send my data anywhere? A: No. All analysis runs locally. Your command history and AI sessions never leave your machine. Optional AI-powered features (template generation) use the Claude API only when explicitly invoked with --ai-templates.

Q: What's the difference between incremental and full analysis? A: Incremental analysis (default) only processes new data since the last run, making it much faster. Full analysis (--full flag) re-analyzes all data from scratch.

Q: How much disk space does it use? A: Minimal. The SQLite database and cache typically use less than 100MB even with extensive history.

Q: Can I customize the recommendation logic? A: Yes. The recommendation engine is modular and extensible. You can adjust scoring weights, effort thresholds, confidence levels, and other parameters in config/config.yaml.

Q: What's the performance impact? A: Analysis typically completes in seconds to minutes depending on data volume. Incremental analysis is very fast. The web dashboard is lightweight and runs locally.

Q: What does the monitor command do? A: hindsight monitor watches your data sources for changes and automatically runs incremental analysis when new data is detected. Useful for continuous insights.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

Development Setup

git clone https://github.com/lcatlett/hindsight.git
cd hindsight
uv sync
uv run pytest tests/

License

MIT License - see LICENSE for details.

Acknowledgments

Built with FastAPI for the web framework
Powered by uv for dependency management
Shell history integration via McFly
Developed using Claude Code

Ready to optimize your workflow? Get started with the Quick Start guide above.

Navigation Menu

Repository files navigation

Hindsight

Overview

Features

🔍 Pattern Analysis

💡 Smart Recommendations

📊 Visual Dashboard

⚙️ CLI Commands

Quick Start

Prerequisites

Installation

Data Export Guide

McFly (Shell History)

Claude Code Sessions

Claude Desktop Conversations

Usage

Configuration

Complete Configuration Reference

Data Source Setup

How It Works

Analysis Pipeline

Web Dashboard

Example Output

Architecture

Project Structure

Development

Running Tests

Code Quality

Extending Hindsight

FAQ

Contributing

Development Setup

License

Acknowledgments

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
config		config
src/hindsight		src/hindsight
tests		tests
.gitignore		.gitignore
CONFIGURATION.md		CONFIGURATION.md
QUICKSTART.md		QUICKSTART.md
README.md		README.md
main.py		main.py
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Latest commit

History

Folders and files

Repository files navigation

Hindsight

Overview

Features

🔍 Pattern Analysis

💡 Smart Recommendations

📊 Visual Dashboard

⚙️ CLI Commands

Quick Start

Prerequisites

Installation

Data Export Guide

McFly (Shell History)

Claude Code Sessions

Claude Desktop Conversations

Usage

Configuration

Complete Configuration Reference

Data Source Setup

How It Works

Analysis Pipeline

Web Dashboard

Example Output

Architecture

Project Structure

Development

Running Tests

Code Quality

Extending Hindsight

FAQ

Contributing

Development Setup

License

Acknowledgments

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages