Repository navigation
Charge a copy held in its source's memory only what it adds, in the model and its capacity screen - #157
Charge a copy held in its source's memory only what it adds, in the model and its capacity screen#157asyms wants to merge 1 commit into
Conversation
…odel and its capacity screen
Stream AIE Metrics Regression Guard⚠ 5 cell(s) flagged (total_latency > 0.1% tol): hardware_two_conv[eyeriss_like_dual_core], hardware_two_conv[eyeriss_like_quad_core], hardware_two_conv[meta_prototype], hardware_two_conv[simba_small], hardware_two_conv[tpu_like_quad_core] 16 of 16 cells captured Provenance: baseline hardware_swiglu — 8 hardware
hardware_two_conv — 8 hardware (⚠ 5 flagged)
To regenerate baseline: |
The memory model counted a copy only beyond its source when the whole transfer stayed in one memory. A multicast that also reaches other memories paid in full for the copy it lands next to its source, though that reader reads it in place: in the attention head on the TPU-like quad core, the exp output and its copy for the div were both counted on the 128 KB vector core, which made the head infeasible. The capacity screen also counted such copies in full while the model it guards did not.
_memory_loadsnow charges any copy held in its source's memory only what it holds beyond the source, andcapacity_screenapplies the same rule to pinned tensors. A transfer that stays in one memory is charged as before.Test: the attention head solves on the TPU-like quad core. It fails without the change.
Stacked on #156.