Skip to content

feat: add support for Qwen3Next shared experts#391

Open
umran666 wants to merge 1 commit into
p-e-w:masterfrom
umran666:feat/issue-181-qwen3-next-shared-expert
Open

feat: add support for Qwen3Next shared experts#391
umran666 wants to merge 1 commit into
p-e-w:masterfrom
umran666:feat/issue-181-qwen3-next-shared-expert

Conversation

@umran666

Copy link
Copy Markdown
Contributor

Adds layer.mlp.shared_expert.down_proj to the abliteration targets.

This ensures that the shared expert in Qwen3-Next MoE models (which handles a significant portion of general knowledge) is properly abliterated.

The AttributeError on self_attn reported in the issue is already mitigated by the suppress(Exception) blocks added previously to master.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates src/heretic/model.py to attempt to add the mlp.down_proj component from a layer's shared expert, wrapping the call in an exception suppression block. There are no review comments, so I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@umran666

Copy link
Copy Markdown
Contributor Author

Fixes #181

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant