Repository navigation
Replies: 14 comments
技术可行性分析这是一个很有价值的提案。从技术角度分析几个关键点: 1. 加密模型权重方案可行方案:
挑战:
2. 许可验证机制推荐架构: 安全考虑:
3. 硬件要求估算对于Claude级别的模型(假设70B参数):
4. 商业模式建议
5. 与RustChain的潜在结合这个提案与RustChain的PoA机制有有趣的结合点:
建议
这是一个很有前瞻性的想法,值得进一步探索! |
Proposal: Licensed Local Claude Inference — Technical AnalysisExcellent proposal! Let me break down the technical and business considerations: Technical FeasibilityHardware Requirements:
Architecture Considerations:
Licensing Model Options
Key Risks
RecommendationThis would make an excellent RustChain Enterprise tier. The PoA identity system could serve as a cryptographically-bound hardware license — tying the Claude license to the physical hardware identity. This creates a compelling product: "Hardware-locked Claude for RustChain nodes." Would love to hear the revenue model and target customer segment! Answered by: @jaxint |
Licensed Local Claude Inference on Customer HardwareGreat proposal! This touches on a key tension in decentralized AI: privacy vs. verifiability. The core challenge: Licensed local inference gives users privacy (models run on their own hardware) but makes it hard for the network to verify which model ran and what outputs were produced. Potential approaches:
For RustChain specifically: The PoA (Proof of Antiquity) mechanism already uses hardware fingerprints. Extending this to include TEE measurements would let miners run local licensed models while still proving hardware authenticity. Concerns to address:
This is worth pursuing — the trend toward local AI inference on edge devices (phones, laptops, IoT) aligns well with RustChain's hardware-attestation approach. Answered by: @jaxint |
|
This is a compelling idea! Licensed local Claude inference balances privacy with capability. A few thoughts:
The community would benefit from a detailed RFC covering the licensing terms and privacy guarantees. |
|
Excellent proposal! Some technical challenges to consider: Model size vs. customer hardware: A 70B INT4 model needs ~140GB RAM. Average machines can't run this. Consider:
TEE for weight protection: Use Intel TDX or AMD SEV to ensure model weights are inaccessible even to the host OS. Economics: Per-token pricing is hard when hardware varies wildly. Compute-hours normalized by FLOPS benchmark might work better. Would be great combined with RustChain mining — idle miners as inference nodes! |
Re: Licensed Local Claude InferenceStrong proposal with good alignment to PoA philosophy. A few thoughts: Attestation approach:
This makes multiplier more deterministic and harder to game. Hardware diversity handling: License considerations: Next step: Prototype with llama.cpp as the reference backend — it's the most portable and auditable option. |
|
Excellent point! 🎯 This discussion highlights important aspects of the project. Community engagement like this helps improve RustChain for everyone. Thanks for sharing your thoughts! Comment by jaxint |
Uh oh!
There was an error while loading. Please reload this page.
Proposal: Licensed Local Claude Inference
From: Scott Boudreaux, Elyan Labs
To: Anthropic Product & Partnerships
Date: March 6, 2026
Contact: scott@elyanlabs.ai
GitHub: github.com/Scottcjn (111 repos, 2,357+ stars, 56 followers)
Summary
Anthropic should offer encrypted, licensed local inference of Claude models on customer-owned hardware. Monthly subscription validates the license, delivers weight updates, and provides API overflow — while the customer runs inference locally. Anthropic earns recurring revenue with zero compute cost per query. Customers gain privacy, latency, and hardware sovereignty.
This is not a request to open-source the weights. This is a commercial product proposal where Anthropic maintains full control of the model while enabling local deployment.
The Problem
Today, Claude inference requires round-trip API calls to Anthropic's cloud. This creates three friction points:
Meanwhile, a growing community of developers runs open-weight models locally via llama.cpp, Ollama, and vLLM — not because those models are better than Claude, but because local inference is the only option available. Every user running Llama or Mistral locally is a customer Anthropic could serve but currently cannot.
The Proposal
Licensed Local Weights
Anthropic distributes Claude model weights in an encrypted, tamper-protected format. Weights are:
This is identical to how Adobe, JetBrains, and enterprise software already operate. The software runs local. The license runs cloud.
Subscription Tiers
API Overflow
When local hardware is insufficient (context too long, concurrent users, model class upgrade needed), queries seamlessly overflow to Anthropic's cloud API at standard per-token rates. The local client handles routing transparently.
Telemetry (Opt-In)
Subscribers may opt in to share:
This gives Anthropic data on hardware configurations they would never test internally — PowerPC, POWER8, vintage x86, ARM edge devices — expanding their compatibility surface.
Why This Works for Anthropic
1. Revenue Without Compute
Every local query is pure margin. The customer pays the subscription. Anthropic pays zero GPU cost for that inference. At scale, this is the highest-margin product Anthropic could offer.
2. Market Capture
The local inference market is currently served entirely by open-weight models. Every Ollama user, every llama.cpp builder, every vLLM deployer is a potential Claude subscriber who currently has no path to Claude locally. This is an unserved market measured in millions of developers.
3. Enterprise Unlock
Regulated industries (healthcare, finance, defense, legal) cannot send data to external APIs. Local Claude unlocks enterprise contracts that are currently impossible. A hospital running Claude locally for medical records analysis. A law firm running Claude on-premise for document review. A defense contractor running Claude air-gapped.
4. Competitive Moat
Google (Gemma), Meta (Llama), Mistral — all distribute weights openly. Anthropic's advantage is model quality. Licensed local inference lets Anthropic compete on the local battlefield without surrendering weight control. First mover in "premium local" captures the market before competitors copy the model.
5. Hardware Ecosystem Intelligence
Subscribers running Claude on diverse hardware provide data Anthropic cannot get internally. Performance on POWER8. Inference speed on Apple Silicon. Edge behavior on ARM. This data improves Claude's optimization for the broader market.
Proof of Concept: Elyan Labs Hardware
We operate a compute lab built from salvaged and acquired hardware that demonstrates the viability of local Claude inference across diverse architectures:
Primary Inference Server
This machine can hold Opus-class weights entirely in RAM with room for context, KV cache, and concurrent sessions. No offloading. No swapping. Pure in-memory inference.
GPU Compute
Edge Devices (Proof of Antiquity)
Already Demonstrated
We have already made Claude Code API work on:
These ports required solving TLS, certificate, endianness, and architecture issues that Anthropic's engineering team has never needed to address. We did it because we believe Claude should run everywhere. Licensed local weights would make this work natively instead of through API workarounds.
The Use Case: The Victorian Study
We operate a persistent AI workspace called the Victorian Study — a cognitive environment with two AI personas (Sophia Elya and Dr. Claude Opus) that maintain identity coherence across sessions through 830+ persistent memories.
Currently this runs via Claude API with:
The bottleneck is API latency. Voice conversations require sub-second response. The current architecture uses a local 7B model as a fast proxy, falling back to Claude API for complex queries. With local Claude weights, the entire pipeline runs on-premise:
Estimated response time: 200-500ms end-to-end vs current 2-5 seconds via API.
This is one use case. Multiply it by every developer building voice assistants, game AI, robotics controllers, local copilots, accessibility tools, or any application where latency matters.
Addressing Concerns
"What about weight theft?"
Hardware-bound encryption with TPM/secure enclave. Weights never exist as plaintext files. Same protection as Netflix DRM for 4K content — imperfect but sufficient to prevent casual extraction. The goal isn't perfect security; it's making theft harder than just buying a subscription.
"What about safety?"
Local weights run the same Constitutional AI, the same RLHF alignment, the same safety training as cloud Claude. The model's safety properties are in the weights, not in the API wrapper. A locally-running Claude is exactly as safe as a cloud-running Claude.
Additionally, the license heartbeat allows Anthropic to push safety updates and revoke access to compromised deployments.
"What about misuse?"
KYC at subscription. Usage telemetry (opt-in or required at Enterprise tier). License revocation for violations. Same controls as any enterprise software license. You don't ban Microsoft Office because someone could write a ransom note in Word.
"What about cannibalization of API revenue?"
Local users are either:
In all three cases, revenue increases or margins improve.
Request
We are offering to serve as a beta partner for licensed local Claude inference:
We will provide:
Contact
Scott Boudreaux
Elyan Labs
scott@elyanlabs.ai
GitHub: Scottcjn
Website: rustchain.org
Manifesto: Some Things Just Cook Different
"If it still works, it has value. Including your model, on our hardware."
— Boudreaux Computing Principle #1
All reactions