Helios (Hierarchical Experimental Learning and Intelligent Optimization System) is an orchestrator-agnostic adaptive campaign decision layer for closed-loop experimentation.
Helios decides which campaign-level action should happen next, including optimization strategy selection, validation, failure-aware recovery, context acquisition, human/LLM query, dynamic objective/constraint handling, and future scale/fidelity-aware decisions.
HELIOS treats optimizers, simulators, automation services, and external systems as downstream tools, backends, or evidence sources. This README focuses on the campaign decision layer: the typed inputs it consumes, the actions it can recommend, the evidence it records, and the replay/validation path that keeps those decisions auditable.
For each campaign round, HELIOS turns scientific context into a bounded campaign-level decision:
- Optimize — choose the optimization strategy, mode, backend, and candidate-generation path.
- Validate — decide when proxy progress needs mechanism, repeatability, or higher-fidelity validation.
- Recover — route failure-aware recovery when results, constraints, measurements, or backends look unreliable.
- Acquire context — ask for literature, prior-campaign evidence, diagnostics, or missing experimental context.
- Ask a human or LLM — request human observation or LLM-supported context only at the language/knowledge boundary.
- Revise objectives or constraints — handle dynamic objective hierarchy, proxy gaps, constraints, and safety envelopes.
- Escalate scale or fidelity — provide the decision surface for future scale/fidelity-aware campaign moves.
The output is an auditable campaign decision envelope with evidence, rationale, selected policy/action, optional candidate portfolio, expected value, and replayable outcome accounting.
TaskContract / campaign context
|
v
RoundContext + objective state + failure history + backend memory
|
v
Adaptive campaign policy
|
+--> CampaignIntent / OptimizationMode / backend recommendation
+--> validation, recovery, context, human/LLM, objective/constraint action
|
v
Decision trace + evidence + reward/outcome + replay record
The live path is conservative by design: rule-based, auditable, and bounded by explicit safety gates. Learning-based policies do not replace it by default. They move through replay evaluation, shadow records, canary runs, promotion gates, and approval workflows before they can influence live decisions.
- Orchestrator-agnostic campaign decision layer — keeps campaign-level decision authority separate from any downstream backend.
- Context-aware policy — uses objective hierarchy, proxy-gap state, failure attribution, backend memory, Nexus diagnostics, candidate/failure-zone memory, and bandit/learned-policy signals.
- Dynamic action vocabulary — represents optimization, validation, calibration, failure diagnosis, context seeking, human observation, safety-constraint tightening, stopping, and future scale/fidelity choices.
- Candidate and backend arbitration — combines local baselines, Nexus signals, candidate pools, safety gates, and provenance into a traceable portfolio.
- Failure-aware recovery — separates scientific negative evidence from measurement, backend, constraint, and downstream tool failures.
- Trace, reward, and replay — records
StrategyTrace,StrategyEvidence,StrategyOutcome,StrategyReward, typedFailureEvent, and replay summaries. - LLM boundary discipline — LLMs can help translate intent, gather context, or generate review notes; they do not steer the live optimization loop.
| Surface | Responsibility | Representative modules |
|---|---|---|
| Contract and context | Typed campaign goal, objectives, constraints, budget, safety, and round context | app/contracts/, app/services/round_context.py, app/services/objective_state.py |
| Campaign policy | Decide next campaign-level action and strategy mode | app/services/strategy_selector.py, app/services/strategy_actions.py, app/services/decision_layer.py |
| Evidence and memory | Attach diagnostics, prior-campaign evidence, failure history, and backend memory | app/services/decision_trace.py, app/services/backend_memory.py, app/optimization/candidate_memory.py, app/optimization/failure_zone_memory.py |
| Candidate/backend arbitration | Build, gate, score, and explain candidate/backend choices | app/optimization/service.py, app/optimization/pool_service.py, app/optimization/decision_policy.py, app/optimization/provenance.py |
| Adaptive substrate | Shadow-only scientific activity mode, dynamic action space, and value-of-information assessment | app/services/adaptive_campaign_substrate.py, app/services/campaign_mode.py, app/services/dynamic_action_space.py, app/services/value_of_information.py |
| Outcome and replay | Evaluate decision quality, reward components, and replay summaries | app/services/decision_outcome.py, app/services/verifiable_reward.py, app/services/decision_replay.py, app/services/policy_evaluation.py |
HELIOS uses scientific context, objective hierarchy, typed failure attribution, backend performance memory, candidate/failure-zone memory, Nexus diagnostics and bandit/learned-policy signals to decide which campaign-level action should happen next. Today that includes CampaignIntent, OptimizationMode, and candidate-generation backend selection; the same layer owns validation, failure-aware recovery, context acquisition, human/LLM query, dynamic objective/constraint handling, and future scale/fidelity-aware decisions.
The optimization stack is split by authority boundary:
app/services/optimization_intelligence.pyenriches strategy selection with optional Nexus diagnostics, similar-campaign evidence, and backend recommendations. It emits structured evidence; it does not choose a live candidate.app/optimization/nexus_provider.pyandapp/optimization/nexus_backend.pyadapt Nexus profiling andnexus_*algorithm plugins behind HELIOS provider/backend interfaces. Nexus remains an advisor/backend, not campaign authority.app/optimization/service.py,app/optimization/pool_service.py, andapp/optimization/candidate_pool.pybuild the multi-source candidate portfolio.app/optimization/decision_policy.pyis the hard gate and arbitration authority for concrete candidates. It enforces bounds, deduplication, safety hook results, and ranks survivors with the strategy decision's utility model.app/optimization/loop_integration.pyis the campaign-loop seam. Deep candidate-pool arbitration is controlled byENABLE_CANDIDATE_ARBITRATIONand defaults off.app/optimization/provenance.pyrecords selected portfolios, rejected candidates, scored pools, and strategy decisions so "why this candidate, not that one?" can be audited.
The shadow-only adaptive substrate proposes a scientific-activity CampaignMode, assesses the action space, and scores candidate value-of-information as an advisory artifact. It changes no routing by default and is gated by ADAPTIVE_SUBSTRATE_SHADOW_ENABLED.
See docs/adaptive_campaign_substrate.md.
The loop-engineering layer records each observe-decide-act-evaluate unit as a replayable episode: loop spec, signals, decision, outcome, reward, and replay summary. This makes workflow data usable for offline evaluation, shadow/canary promotion, failure attribution, and future policy improvement without changing the live path.
The first pure service layer is app/services/loop_engineering.py. It is dependency-light and side-effect-free by design: it does not call downstream services, mutate campaign state, write to the database, or promote learned policies.
git clone https://github.com/SissiFeng/HELIOS.git
cd HELIOS
pip install -e ".[dev]"Run the focused validation for the current positioning/reporting boundary:
pytest tests/test_system_validation_report.pyRun the broader decision-layer tests as needed:
pytest \
tests/test_decision_layer.py \
tests/test_decision_trace.py \
tests/test_decision_outcome.py \
tests/test_decision_replay.py \
tests/test_verifiable_reward.py \
tests/test_policy_evaluation.py \
tests/test_adaptive_campaign_substrate.py \
tests/test_shadow_trace_comparison.py \
tests/test_backend_memory.py \
tests/test_candidate_pool.py| Variable | Default | Description |
|---|---|---|
LLM_PROVIDER |
mock |
LLM provider for language/knowledge-boundary tasks only |
LLM_MODEL |
provider default | Model ID passed to the configured provider |
CONTEXTUAL_DECISION_SHADOW_ENABLED |
false |
Record the legacy contextual decision shadow trace per round |
ADAPTIVE_SUBSTRATE_SHADOW_ENABLED |
false |
Record the adaptive campaign substrate shadow snapshot per round |
ENABLE_CANDIDATE_ARBITRATION |
false |
Enable deep candidate-pool arbitration instead of legacy generation fallback |
HELIOS is framed as an orchestrator-agnostic adaptive campaign decision layer. The product boundary is campaign-level decision authority rather than ownership of downstream automation or presentation surfaces. The live campaign policy remains rule-based and auditable by default; Nexus is optimization advisor/backend/tool paths, not campaign decision authorities. Learned policy and self-evolution paths are offline, shadow, canary, and approval-gated.
The architecture validation report is version-controlled at docs/HELIOS_ARCHITECTURE_VALIDATION.md. It is a static evidence pack for the current validation boundary.
Run the validation suite:
bash scripts/run_validation_suite.shEquivalent targeted test command:
pytest \
tests/test_candidate_memory.py \
tests/test_failure_zone_memory.py \
tests/test_offline_closed_loop_sdl.py \
tests/test_offline_scenario_benchmarks.py \
tests/test_policy_evolution.py \
tests/test_policy_evolution_workflow_e2e.py \
tests/test_learned_policy.py \
tests/test_system_validation_report.py \
tests/test_backend_selection.pyHELIOS/
├── app/
│ ├── contracts/ # Typed campaign contracts and query/task models
│ ├── optimization/ # Candidate pools, backend facades, arbitration, provenance
│ ├── services/ # Campaign policy, evidence, reward, replay, objective/failure logic
│ ├── api/v1/endpoints/ # Service API surfaces
│ └── core/ # Config, DB, startup lifecycle
├── docs/
│ ├── HELIOS_ARCHITECTURE_VALIDATION.md
│ ├── adaptive_campaign_substrate.md
│ └── development_progress.md
├── tests/ # Pytest coverage for policy, replay, validation, and evidence layers
├── benchmarks/ # Offline method and policy evaluation harnesses
├── models/ # Learned-policy checkpoints and replay artifacts
├── pyproject.toml # Dependencies and tool config
└── README.md
MIT — see LICENSE for details.