Issues

• DBRX: attentions_input is the pre-norm residual (input of norm_attn_norm), unlike other architectures

  ndif-team/nnterp#56 · Filter by author Butanium opened on Aug 28, 2026

• Sublayer output multipliers (Granite residual_multiplier, Falcon-H1 attention_out_multiplier) make attentions_output / mlps_output differ from the residual contribution by a constant factor

  ndif-team/nnterp#54 · Filter by author Butanium opened on Aug 28, 2026

• attentions_output / mlps_output are not the residual contribution on post-sublayer-norm ("sandwich") architectures (Gemma-2/3, GLM-4, OLMo-2)

  ndif-team/nnterp#53 · Filter by author Butanium opened on Aug 28, 2026
  5

• generate is now on nnsight too

  ndif-team/nnterp#46 · Filter by author Butanium opened on Apr 30, 2026

• [dev] current vllm .layers_output is very likely wrong

  ndif-team/nnterp#42 · Filter by author Butanium opened on Apr 14, 2026
  4

• Gemma 3n not supported: AltUp 4D hidden states break nnterp assumptions

  ndif-team/nnterp#35 · Filter by author Butanium opened on Mar 25, 2026

• Should we add support for hybrid mamba models?

  ndif-team/nnterp#18 · Filter by author Butanium opened on Oct 6, 2025

• generate docs from notebooks

  ndif-team/nnterp#13 · Filter by author Butanium opened on Aug 25, 2025

• add test for caching

  ndif-team/nnterp#12 · Filter by author Butanium opened on Aug 25, 2025
  1

• Add down_proj renaming

  good first issue
  ndif-team/nnterp#11 · Filter by author Butanium opened on Aug 23, 2025

• vLLM support

  ndif-team/nnterp#6 · Filter by author Butanium opened on Feb 15, 2025
  2