DataHarbor is a high-performance, full-stack web application that provides researchers and developers with a secure, intuitive interface to access and manage data on XRootD-backed storage. Built with a Go backend and Vue.js frontend, it delivers seamless file browsing, directory navigation, and secure file operations through direct XROOTD integration.
DataHarbor empowers users who need to:
- Browse & Navigate: Explore your storage with an intuitive web interface, view metadata, and perform secure file operations
- High-Performance Streaming: Download files directly from remote storage with zero temporary storage overhead
- Multi-File Upload: Drag-and-drop or select multiple files; chunked, resumable uploads with SHA-256 integrity verification and per-user concurrency control
- Secure Access: Enterprise-grade OIDC authentication with session management
- Cross-Platform: Access HPC storage from any device through a modern web browser
- Observability: Comprehensive logging and performance metrics for file operations
- Backend: Go REST API server with XROOTD client integration
- Frontend: Vue 3 SPA with Element Plus UI components
- Authentication: OpenID Connect (OIDC) with Backend-For-Frontend (BFF) pattern
- Security: HTTP-only cookies, server-side session management
- Storage: XROOTD protocol for high-performance data access
Zero-configuration development environment with all tools pre-installed:
- Install Docker Desktop + Dev Containers extension
- Clone and open:
git clone https://github.com/AnarManafov/dataharbor.git && code dataharbor - Click "Reopen in Container" when prompted
- Run
make dev— access athttps://localhost:5173
🛠️ Dev Container Guide — Full setup details, WSL2 instructions, troubleshooting.
Run the full stack (frontend + backend + XRootD) in containers:
cd dataharbor/docker
cp .env.example .env && nano .env
docker compose up -d
# Access at https://localhost📦 Docker Deployment Guide — Development & production setup, certificates, troubleshooting.
For running services directly on your machine without containers:
- Go 1.26+
- Node.js 20+
git clone https://github.com/AnarManafov/dataharbor.git && cd dataharbor
# Install dependencies
make deps
# Start development servers
make dev # Or: make dev-frontend / make dev-backendAccess at https://localhost:5173 (accept the self-signed certificate warning).
💡 Complete documentation is available in the
docs/folder
| Document | Description |
|---|---|
| Dev Container Guide | Zero-config development environment (recommended) |
| Setup Guide | Manual environment setup, dependencies, and first steps |
| Development Guide | Development workflow, Git conventions, and testing |
| Backend Configuration | Go backend config, environment variables, and YAML settings |
| Frontend Configuration | Vue.js frontend config, SSL setup, and deployment settings |
| Document | Description |
|---|---|
| System Architecture | Overall architecture, design patterns, and component interactions |
| Authentication System | OIDC integration, BFF pattern, and security model |
| Document | Description |
|---|---|
| Backend Development | Go API development, middleware, and XROOTD integration |
| Frontend Development | Vue.js development, components, and state management |
| Document | Description |
|---|---|
| REST API Reference | Complete API endpoints, request/response examples |
| XROOTD Integration | XROOTD client configuration and file operations |
| Document | Description |
|---|---|
| Docker Deployment | Docker Compose setup, certificates, troubleshooting |
| Deployment Guide | Production deployment and environment setup |
| Testing Guide | Testing strategies, coverage requirements, and test execution |
| Troubleshooting Guide | Comprehensive issue resolution and debugging |
- Development: Dev Container (recommended) or Manual Setup
- Docker Deployment: Docker Guide — run full stack in containers
- Architecture: System Architecture → Authentication
- API Development: Backend Guide → API Reference
- UI Development: Frontend Guide
- Production: Deployment Guide
make buildmake updateSee DEVELOPMENT.md for detailed dependency management guidelines.
# Backend tests with coverage
make test
# Verbose output
make test-verbose
# Race detection
make test-race# Sync versions across components
make sync-versions
# Prepare release
make prepare-releasedataharbor/
├── .devcontainer/ # Dev Container configuration (VS Code)
├── app/ # Go backend application
│ ├── common/ # Shared utilities (logger, XRootD client)
│ ├── config/ # Configuration management
│ ├── controller/ # HTTP request handlers
│ ├── middleware/ # Authentication, CORS, logging middleware
│ ├── response/ # API response helpers
│ └── route/ # API route definitions
├── web/ # Vue.js frontend application
│ └── src/
│ ├── api/ # API client and HTTP services
│ ├── components/ # Reusable Vue components
│ ├── router/ # Vue Router configuration
│ ├── store/ # Vuex state management
│ └── views/ # Page-level components
├── docker/ # Docker Compose deployments
│ ├── backend/ # Backend Dockerfile
│ ├── frontend/ # Frontend Dockerfile
│ ├── nginx/ # Gateway configuration
│ └── xrootd/ # XRootD server container
├── docs/ # Developer documentation
└── packaging/ # RPM packaging and build scripts
This is an internal project for GSI developers. To contribute:
- Create a feature branch from
master - Follow the coding standards outlined in development docs
- Add appropriate tests for new functionality
- Update documentation as needed
- Submit a pull request with detailed description
See LICENSE file for details.
- XROOTD - High-performance data access system
- Vue.js - Progressive JavaScript framework
- Gin - Go web framework
- Element Plus - Vue 3 component library