Skip to content

civil42-hackathon/parser

Repository files navigation

Parser API

Internal FastAPI service for ingesting unified event and unit data, storing current state in SQLite and exposing sync endpoints for local instances.

Phase 3 adds:

  • Scenario C demo orchestration
  • GDDKiA fallback-backed road ingestion
  • dispatch and coverage analysis
  • AI documentation explorer that turns docs into a declarative ingest spec
  • optional startup autoload so a fresh parser can expose a ready snapshot right after boot

Quick start

cp .env.example .env
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
uvicorn app.main:app --reload

Docker

cp .env.example .env
docker compose up --build

Key endpoints

  • POST /v1/ingest/events
  • POST /v1/ingest/units
  • POST /v1/ingest/mock-load
  • POST /v1/ingest/json
  • POST /v1/ingest/radio
  • POST /v1/ingest/crawl
  • POST /v1/ingest/gddkia
  • POST /v1/demo/bootstrap
  • POST /v1/demo/replay/next
  • POST /v1/analysis/dispatch
  • GET /v1/analysis/coverage
  • POST /v1/adapters/explore
  • GET /v1/adapters/explore/{exploration_id}
  • POST /v1/adapters/explore/{exploration_id}/dry-run
  • POST /v1/adapters/explore/{exploration_id}/ingest
  • GET /v1/events
  • GET /v1/units
  • GET /v1/snapshot
  • GET /v1/feed/changes
  • GET /v1/geo/events
  • GET /v1/geo/units
  • GET /v1/schema
  • GET /v1/health

Phase 3 env

  • ELEVENLABS_API_KEY
  • OPENAI_API_KEY
  • LLM_PROVIDER
  • RADIO_DEFAULT_SOURCE
  • CRAWLER_GIOS_ENABLED
  • OSRM_BASE_URL
  • EXPLORER_REQUEST_TIMEOUT_SECONDS
  • EXPLORER_CACHED_DOCS_ENABLED
  • DEMO_UNITS_PATH
  • DEMO_HOTSPOTS_PATH
  • DEMO_GDDKIA_CACHE_PATH
  • DEMO_DOCS_DIR
  • AUTO_BOOTSTRAP_ON_START
  • AUTO_REFRESH_GDDKIA_ON_START
  • AUTO_EXPLORE_GIOS_ON_START
  • STARTUP_SKIP_IF_DATA_EXISTS
  • GDDKIA_REFRESH_ENABLED
  • GDDKIA_REFRESH_INTERVAL_SECONDS

Startup autoload

On a fresh SQLite database, the parser can now preload demo-ready data during startup:

  • demo bootstrap seeds PSP/OSP units and hotspot data
  • GDDKiA cache/live ingest runs once on startup
  • optional recurring GDDKiA refresh can be enabled
  • optional auto-explore of GIOŚ can be enabled, but is off by default

Recommended defaults for local demo are already included in .env.example.

Important:

  • if the database already contains data and STARTUP_SKIP_IF_DATA_EXISTS=true, startup autoload will not re-seed
  • if you want a fully fresh auto-seeded instance, point DATABASE_PATH at a new empty file

Demo rehearsal

Bootstrap + replay:

python scripts/rehearse_scenario_c.py --base-url http://127.0.0.1:8100

Explorer-only flow:

python scripts/demo_explorer.py --base-url http://127.0.0.1:8100

Force real LLM analysis through HTTP instead of demo heuristics:

curl -X POST http://127.0.0.1:8100/v1/adapters/explore \
  -H 'Content-Type: application/json' \
  -d '{
    "source_name":"live ai gios",
    "source_id":"live_ai_gios",
    "docs_type":"url",
    "docs_url":"https://powietrze.gios.gov.pl/pjp/content/api",
    "api_base_url":"https://api.gios.gov.pl/pjp-api/v1/rest",
    "prefer_cached_docs": false,
    "force_llm": true
  }'

Tests

pytest -q

About

pa-pa-pa-paaarseeer!!!! :D

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages