Liassistant

A Python application for converting documents (PDF, PPTX) to interactive LiaScript courses using AI-powered content enrichment.

Features

• Multi-format conversion: Converts PDF and PPTX documents to LiaScript format
• Asset extraction: Automatically extracts and references images, tables, and other assets
• AI enrichment (optional): Uses ML models for:
  • Image captioning (SmolVLM)
  • Code block detection and language identification
  • Mathematical formula extraction and LaTeX conversion
• LLM integration: Generates comprehensive course content using LiteLLM (supports Claude, GPT, Gemini, etc.)
• Flask web interface: Easy-to-use web UI for document upload and course generation

Installation

# Clone the repository
git clone <repo-url>
cd liassistant

# Create virtual environment with uv
uv venv .venv

# Activate it (optional - uv run works without activation)
source .venv/bin/activate  # On macOS/Linux
# or: .venv\Scripts\activate  # On Windows

# Install dependencies
uv sync

Quick Start

Running the Web Application

After installation, run the app:

# If you activated the venv
python liassistant/app.py
# or
python -m liassistant.app

# If you didn't activate, use uv run
uv run python liassistant/app.py

Open http://localhost:5000 in your browser.

Running Tests

Fast tests (28 tests, ~1 minute):

uv run pytest tests/ --ignore=tests/test_enriched_conversion.py -v
# or run all tests (enriched tests are skipped by default):
uv run pytest tests/ -v

Enrichment tests only (requires model setup, 2 tests):

# First, pre-download ML models (one-time setup)
uv run python scripts/download_docling_models.py

# Then run enrichment tests (will take 2-3 minutes per test)
uv run pytest tests/test_enriched_conversion.py -v

ML Model Setup for Document Enrichment

Document enrichment (image captions, code detection, formula extraction) uses large ML models that must be downloaded once and cached.

Pre-download Models

uv run python scripts/download_docling_models.py

Models are cached in ~/.cache/docling/models and reused across runs. First-time download takes 5-10 minutes and requires 2-3 GB of disk space.

Models Included

• SmolVLM-256M-Instruct: Image captioning
• Layout model: Document structure detection
• TableFormer: Table structure recognition
• Code/Formula model: Code and LaTeX extraction
• EasyOCR: Optical character recognition
• Picture Classifier: Image type classification

Configuration

Environment variables (create a .env file in the project root):

# LLM Configuration
# Option 1: Using Vertex AI (Google Cloud)
LITELLM_MODEL=vertex_ai/gemini-2.0-flash-001
VERTEX_PROJECT=your-gcp-project-id
VERTEX_LOCATION=us-central1
GOOGLE_APPLICATION_CREDENTIALS=path/to/your/service-key.json

# Option 2: Using OpenAI
LITELLM_MODEL=gpt-4
OPENAI_API_KEY=your-openai-api-key

# Option 3: Using Anthropic
LITELLM_MODEL=claude-3-5-sonnet
ANTHROPIC_API_KEY=your-anthropic-api-key

# File Storage (optional, defaults shown)
UPLOAD_FOLDER=files/uploads/
GENERATED_COURSES_FOLDER=files/generated_courses/
SESSION_FILE_DIR=files/flask_sessions/

Note: The app uses LiteLLM which supports many LLM providers. Set LITELLM_MODEL to your desired model and provide the corresponding API key.

Project Structure

.
├── liassistant/
│   ├── app.py                 # Flask web application
│   ├── static/                # CSS, JavaScript
│   ├── templates/             # HTML templates
│   └── utils/
│       ├── converter.py       # Document conversion using docling
│       ├── llm_handler.py     # LLM integration for course generation
│       ├── packager.py        # ZIP course packaging
│       └── prompts.py         # LLM prompts for course generation
├── tests/
│   ├── test_app.py            # Web app tests
│   ├── test_converter.py      # Document conversion tests
│   ├── test_enriched_conversion.py  # Enrichment tests (skipped by default)
│   ├── test_form_options.py   # Form parameter handling tests
│   ├── test_llm_handler.py    # LLM integration tests
│   └── test_packager.py       # ZIP packaging tests
├── scripts/
│   └── download_docling_models.py  # Pre-download ML models
└── pyproject.toml             # Project metadata and dependencies

Testing

Test Suites

• test_app.py: Flask routes and file upload handling (13 tests)
• test_converter.py: Document conversion without enrichment (4 tests, fast)
• test_enriched_conversion.py: Full enrichment tests (2 tests, skipped by default)
• test_form_options.py: Form parameter handling and prompt customization (2 tests)
• test_llm_handler.py: LLM integration and prompt generation (5 tests)
• test_packager.py: Course ZIP packaging (4 tests)

Running Specific Tests

# Run only converter tests
uv run pytest tests/test_converter.py -v

# Run only LLM tests
uv run pytest tests/test_llm_handler.py -v

# Run only app tests
uv run pytest tests/test_app.py -v

# Run with coverage
uv run pytest tests/ --cov=liassistant

Enrichment Mode

Default Behavior

In Production (Web App):

• ✅ Image captioning (SmolVLM)
• ✅ Code block language detection
• ✅ Mathematical formula extraction (LaTeX)
• ✅ Picture type classification
• ⚠️ Requires 2-3 GB disk space for cached ML models
• ⚠️ Adds 1-3 minutes per document (ML inference)

In Tests (Disabled by Default):

• ✅ Basic document conversion to Markdown
• ✅ Asset extraction and linking
• ✅ Fast execution (~2 seconds per test)
• ❌ No ML enrichment features (models disabled to keep tests fast)

To enable enrichment in tests:

# 1. Download ML models first
python scripts/download_docling_models.py

# 2. Edit tests/conftest.py or test file and set:
app_config = {
    "enable_enrichment": True,  # Set to True for enrichment
}

# 3. Run enriched tests
pytest tests/test_enriched_conversion.py -v

Disabling Enrichment in Production

To disable enrichment in the web application (for faster processing):

1. Edit liassistant/app.py and add before the convert_to_markdown call:

app.config['enable_enrichment'] = False

Or modify the conversion call to pass a config dict with enrichment disabled.

Performance Notes

• Without enrichment: Document conversion takes 5-30 seconds depending on size
• With enrichment: Add 1-3 minutes per document for ML inference
• First run after download: Models are auto-loaded from cache
• MPS/GPU: If running on Apple Silicon, MPS acceleration is available for ML models

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0). See the LICENSE.md file for details.

Support

For issues, questions, or suggestions, please open an issue on GitHub.