[chlins]

This pull request introduces a comprehensive proposal for a new Model Sync Adapter Framework in Harbor, aimed at enabling seamless synchronization of AI models from upstream model hubs (starting with Hugging Face) into Harbor as OCI artifacts. The framework is designed to be extensible, policy-driven, and deterministic, while reusing Harbor's proven architectural patterns without modifying the existing OCI replication system.

The most important changes are:

Framework Design and Architecture

• Proposes a dedicated Model Sync Adapter Framework that runs parallel to Harbor's replication system, introducing a new adapter contract tailored for model hubs with a fetch → convert → push pipeline.
• Details a modular architecture: adapters per hub, system-level sync policies, deterministic packaging, and reuse of Harbor's registry management and execution/task management.

Adapter Contract and Extensibility

• Defines a narrow, hub-agnostic Model Adapter contract (resolve + stream) to make adding new model sources straightforward and maintainable, with Hugging Face as the reference implementation.

Deterministic Packaging and Provenance

• Describes a deterministic packaging process that produces bit-identical OCI artifacts for the same upstream revision and file filters, ensuring idempotence, deduplication, and reliable provenance tracking via tags and annotations.

API, Data Model, and UI Integration

[elliott-davis]

Something I don't see called out in this proposal is secrets management. Some models require the acceptance of a EULA and today their access is gated behind a users token that has accepted the EULA. Is there any effort to support these models in this proposal?

[elliott-davis]

Will this proposal make any attempt to differentiate model artifacts from container artifacts in a repository? A confused developer may try and pull one of these converted models expecting an engine to be packaged with it like a NVIDIA NIM and will experience a failure.

[elliott-davis]

Overall this proposal aligns with what I would like to accomplish for model syncing in Harbor.

[chlins]

Something I don't see called out in this proposal is secrets management. Some models require the acceptance of a EULA and today their access is gated behind a users token that has accepted the EULA. Is there any effort to support these models in this proposal?

@elliott-davis Good question. Gated models are covered by the existing token mechanism rather than a new secrets subsystem: the user completes the EULA or license acceptance on the hub side with their own account beforehand, and the access token configured in Harbor's registry management (encrypted at rest) carries that entitlement. From Harbor's perspective this is plain token authentication when fetching; it implements no hub-specific consent flow. I've updated the Security Considerations section to make this explicit.

[chlins]

Will this proposal make any attempt to differentiate model artifacts from container artifacts in a repository? A confused developer may try and pull one of these converted models expecting an engine to be packaged with it like a NVIDIA NIM and will experience a failure.

@elliott-davis Yes. Converted models are model-spec OCI artifacts with their own artifactType and media types, so Harbor's UI can distinguish them from container images and will render them with a distinct icon, the same way charts, SBOMs, and signatures are differentiated today. On the pull path, however, both go through the standard Distribution v2 API, and the server should not intercept or reject pulls based on artifact type. So the safeguard is discoverability (distinct type and icon in the UI, plus the model-spec media types for tooling to inspect), not server-side gating; whether an artifact is runnable is up to the consuming tooling.

[Vad1mo]

I don't think the claim that registry adapter won't work holds true, given the determinism the proposal already guarantees (Goal 4).

If the packer turns a given revision plus file filters into a byte-identical manifest, the adapter isn't faking registry semantics. It derives them. The same packer could sit behind PullManifest, and PullBlob(repository, digest) would map the digest back to a file path:

[Vad1mo]

I think we still should strive for making it work with adapters, I think it will not only be easier to implement and maintain, but also for users to adapt and understand. The whole ecosystem in build around that too. cli, pulumi, tf proviers and all the scriptiin people has been made over the years. IMO it is worth reusing that.

The adapter has policies, cron, executions, bandwidth limits, chunked resume and quota from replication, and pull-through from proxy cache via PERMITTED_REGISTRY_TYPES_FOR_PROXY_CACHE (your Phase 2). No new table, API or job kind is needed. The core changes left are a file filter type, a source-only flag in RegistryInfo, and a Redis digest → (repo, commit, path) map because proxy cache creates a new adapter per request.

Now it is only Hugging Face what if other model source become availible and needed replication too and the same capabilities?
we would reimplement the same thing over again.

It is not worth it, there no additional benifits in doing so.

[Vad1mo]

One good example where we failed to do it right in the first place, are the P2P preheat functionality. Harbor has 100lk deployments, theon only reported problem and users of this feature is the company who have made the PR. We never reecieve feature reqeuests, issues regarding that feature excapt from the original authors. and if it breaks it goes unnotices, until the origial team is migrating harbot to new version and notice that error. I don't want to repeat that again.

[Vad1mo]

To compare the two designs on running code, I opened a draft that takes the replication/proxy route: goharbor/harbor#24008. It adds a huggingface registry adapter. Existing pull replication and proxy cache then bring Hub models into Harbor as model-spec artifacts, with no new table, API, job kind or Administration section.

The adapter builds the manifest from the Hub API for a pinned commit, and the output is deterministic. One raw layer per file; LFS sha256 is the layer digest; non-LFS files are hashed and checked against their git blob id. The same commit always gives the same digest, so replication's digest comparison handles idempotence and no revision cursor is needed.

Results on a dev Harbor with HuggingFaceTB/SmolLM2-135M:

• Pull replication worked, and Harbor typed the artifact CNAI. A second run logged "already exists, skip".
• A proxy-cache project served the same model, and modctl pull + modctl extract worked.
• Both paths produced the same digest.
• All extracted files are byte-identical to the Hub at that commit.

Some points in this proposal hold up, and the PR says where:

• preview before import;
• per-sync file filters;
• a source-only capability;
• immutable revision tags;
• an ArtifactRegistry split into read and write parts.

Each can be added as a generic replication/registry extension that other adapters also benefit from, so none of them needs a parallel framework. The PR description lists the compromises I made and the Hub quirks that affect both designs, for example the anonymous 401 for missing models, case-sensitive ?author=, and the tree API's 1000-entry pages.

Happy to fold anything from this proposal into that PR, or to work on it together with @chlins and @elliott-davis.

[Vad1mo]

The main reasons the ArtifactHub adapter was removed were its low-quality, buggy implementation and the upcoming OCI support.

[chlins]

Fair point, I checked and it went out together with the chartmuseum backend in goharbor/harbor#18191, so it doesn't really say much about non-OCI sources. I'll drop that bullet.

[Vad1mo]

https://github.com/llmmanorg/llmman is a good reference. llmman transfer hf.co/... builds a model-spec manifest straight from HF and streams it into the push without touching local disk.

[chlins]

Thanks, didn't know about llmman. Streaming straight from HF is what we had in mind too, so this makes the PoC argument kind of outdated. I'll update that part.

[chlins]

Thanks for putting goharbor/harbor#24008 together, having running code to compare against is really helpful. Let me sync with @wy65701436 on the direction first, will come back here by next week.

[wy65701436]

Thanks @Vad1mo and @chlins for putting together — having running code on both sides makes this a much better discussion. I’ve gone through both，

Where I agree with:

• The claim that an adapter “must fabricate registry semantics” doesn’t hold once the packer is deterministic.
• Re-implementing transfer infrastructure is not worth it. Chunked copy, bandwidth limits, retry, quota accounting, cancellation, execution/task tracking — that is ten years of hardening in replication, and I don’t want a second copy of it.
• Proxy-cache for models falling out of the same adapter is a real advantage.
• The P2P preheat point is fair.

Where I still think “separate” is right — but narrower than the proposal says

• The part of the proposal I want to keep is the product surface, not the execution engine.
• Replication is a copy of the same artifact; the user model is “mirror this registry”. Model import is a conversion; the user model is “bring this model in, show me what’s in it first, let me pick files, pin the revision, tell me the license”. Those are different enough that I don’t want them to share a policy form, a filter model and an API. Concretely, the “generic extensions” in your table — browse API, file filter, revision tags, source-only capability — are in practice all for one adapter. Putting them into the replication policy model makes replication heavier for every other adapter and still gives model users a worse UX than a dedicated page would. That’s the “don’t touch replication” concern on my side: not the transfer code, the policy model.

Proposed middle ground:

• Hugging Face or other AI model registry adapter with deterministic packer
• Keep a separate Model Sync policy, API and UI: preview, file filters, revision tags, license. This is where we iterate independently and where nothing can regress replication.
• Don’t add a second execution engine. A model sync run becomes a execution with a different vendor type; same execution/task tables, same job, same logs.

Does this split work for you? If there’s a reason the policy layer also has to be replication, I’d like to hear it, because that’s the one place I’m still choosing “separate”.

[wy65701436]

One more structural point.

Replication is symmetric by design: a policy has a src_registry and a dest_registry, either side can be the local Harbor, and the same adapter serves as source or target — Harbor-to-Harbor DR and multi-site is the whole reason it exists.

Model Sync is strictly one-way: the source is always an external hub, the destination is always this Harbor, and there is no push direction — you can’t sync an OCI artifact back into a Hugging Face repo. Modeling that as a replication rule means a source-only capability flag, a UI that hides the type in the destination picker, a form full of greyed-out push/pull and namespace options, and an adapter where half the interface is MethodNotAllowed stubs. A dedicated Model Sync policy simply doesn’t have those fields: source → model → revision → destination repo. And the two features chain rather than overlap: Model Sync gets the model in, replication moves it between Harbors afterwards — exactly as it does for any other artifact.

[Vad1mo]

• Hugging Face or other AI model registry adapter with deterministic packer
• Keep a separate Model Sync policy, API and UI: preview, file filters, revision tags, license. This is where we iterate independently and where nothing can regress replication.
• Don’t add a second execution engine. A model sync run becomes a execution with a different vendor type; same execution/task tables, same job, same logs.

Do I understand your idea correctly here.

you want to reuse the existing Harbor replication/proxy logic but provide new user facing interfaces/apis for model?

[chlins]

Yes, that's it. Reuse what's common underneath: the adapter and deterministic packer, the transfer code (chunked copy, bandwidth limit, retry, quota), and the execution/task tables and job. Those which could be shared libraries.

What we want to keep separate is the user facing part: models as their own interface, with their own policy, API and UI. Pulling a model in is a different task for the user than mirroring a registry, and keeping it separate also gives Harbor a clear entry point for moving further towards an AI artifact registry later, without piling model specific things onto the replication policy.

[Vad1mo]

What we want to keep separate is the user facing part: models as their own interface, with their own policy, API and UI. Pulling a model in is a different task for the user than mirroring a registry, and keeping it separate also gives Harbor a clear entry point for moving further towards an AI artifact registry later, without piling model specific things onto the replication policy.

Ok, can you outline what would be the additional functionality on UI, API and what are the specific use cases/scenarios you think the user wants to perform.

Based on that I can see and run a multiple experiments, so we can together see and compare the various options, that we think would fit best.

[chlins]

Sure, will put together the use cases and the UI/API delta with @wy65701436.

In the meantime there's a PoC environment running the proposal's Model Sync at http://172.237.79.67:8088/harbor/model-sync, probably the quickest way to get a feel for the user side before we compare on paper.

[Vad1mo]

In the meantime there's a PoC environment running the proposal's Model Sync at

Is the code to that also in public?

[chlins]

In the meantime there's a PoC environment running the proposal's Model Sync at

Is the code to that also in public?

@Vad1mo No, the code is currently on my local branch. For production, I recommend splitting it into multiple small PRs for review.

[Vad1mo]

Following up on #297 (comment). Instead of comparing on paper, I built both UI directions on the same backend and ran the same scenario through each.

Shared backend, identical for both options: the huggingface adapter from goharbor/harbor#24008 plus four generic extensions in pkg/reg:

• ArtifactResolver.Resolve(repo, ref, filter): preview of a source before copying (resolved commit, tags the copy will get, license, task, file list with class and size, predicted digest)
• file filter with include/exclude patterns; the packer stays deterministic per (commit, filter)
• optional immutable revision tag sha-<12> next to the moving tag
• source_only in RegistryInfo, replacing the type-specific validation

In both options a run is executed by the replication copy flow, transfer, job and execution/task tables. There is no second packer or transfer.

Scenario: HuggingFaceTB/SmolLM2-135M @ main, include *.json. Both options produce the same artifact sha256:2dc8fb68cff37ff1… with tags main and sha-93efa2f097d5, and a rerun skips with the same digest. Without a filter the preview predicts sha256:2b6f786b…, the digest from #24008.

Option A: separate Model Sync, modelled on this proposal and the PoC

Administration > Model Sync with its own model_sync_policy table and /model-sync/* API (same paths and shapes as the PoC, plus exclude filters). The portal module is the PoC's UI, ported. Runs use the vendor type MODEL_SYNC but execute through the replication copy flow.

Area	Lines added
Backend (controller, policy DAO/manager, handler, RBAC)	~1,880
Swagger	702
Portal	~3,600
Migration	23
Tests and generated mocks	~1,720

Found while building it: one policy is one model (no org/**), a pinned commit can't be used as revision (same as replication), there is no proxy cache path, and replication needed a non-persisted DestRepository override because its namespace rewrite always keeps the source's last path component.

A1: Model Sync entry in Administration

A2: Policy list

A3: Policy form with resolved preview (include *.json, exclude onnx/, .bin)*

A4: Executions

A5: Task list with log link

A6: Resulting model artifact. The raw descriptor JSON is today's artifact view; the Model panel in Option B is independent of the import path and would work here too

A7: Rerun skipped with the same digest

Option B follows in the next comment.

[Vad1mo]

Option B: extend Registries, Replication and the artifact view

Same backend and scenario as Option A above. No new policy, table or API besides POST /registries/{id}/preview. The replication rule form is driven by the source registry's capabilities; there is no huggingface check in the portal:

• source supports file: Files section with Include / Exclude
• source supports preview: Preview panel in the rule form, with the same data as Option A
• source supports revision tag: checkbox "Also tag each copy with its source revision"
• adapter-supplied filter labels: Name becomes "Model", Tag becomes "Revision"
• source_only registries are not offered as push destinations
• a Docker Hub or Harbor source shows the form exactly as before

The artifact view gets a Model panel for any CNAI artifact (licenses, source, source revision) instead of the raw descriptor JSON. The endpoint form uses the adapter's credential pattern, so Hugging Face only asks for a token.

Area	Lines added
Backend	130
Swagger	151
Portal	1,121
Tests	551

Bandwidth limit, chunked copy, override, org/** patterns and proxy cache stay available because it is still replication.

B1: Hugging Face endpoint, token only

B2: Rule form with Hugging Face source: Model, Revision, Files

B3: Preview inside the rule form

B4: Rule form with Docker Hub source, unchanged

B5: Executions succeeded

B6: Model artifact with the Model panel

Where this leaves the comparison: both options deliver the same preview, file filters, revision tags and license display, because those are backend capabilities. The difference is a dedicated entry point and wording (A) against roughly a quarter of the code and no second policy model (B).

@chlins @wy65701436 which of your use cases can't be done with Option B? That is what I'd like to understand before we decide. Both branches are ready to push if you want to run them.

[bupd]

Option B seems more simple would be easier to maintain.