You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
in the current code the distance function is dispatched via the server abi, that is costly and not necessary. It should be possible to keep a distance function pointer in the index graph object for direct call to calculate the distance
Neither. The pointer-vs-pointer-table distinction isn't the axis that matters — both [1] and [2] pay the same expensive per-call work. What won (~1.3×) is a third thing that removes that work entirely: it doesn't go A→B through the server at all.
Per distance call, grounded in the code:
[1] Baseline (server VDF dispatch): IndexGraph::distance → m_index.helper() → ctx_.helper_fn() crosses the .so boundary into the server's dispatch_profile_call, which does profile_for_key + find_fn_binding (linear scan) + type-ctx chase → call_vdf: marshal both operands into vef_invalue_t[], build vef_context_t/vef_vdf_args_t → indirect binding.vdf() back across the boundary → extension svector_distance_impl: alloc two per-call buffers, from_encoded both operands → native kernel. That's A → (*fn) → B plus a boundary crossing + marshalling + per-call decode.
[2] The server-side ResolvedFn cache we just tried (= A → A1(*fn[idx]) → B): memoizes {binding, tc} per index so it skips profile_for_key + find_fn_binding + type-ctx fetch. But it still does call_vdf (full marshalling) + the indirect cross-.so binding.vdf call + per-call decode of both operands. Only the lookup is removed. → ~0% on D=32 Mac. That's the proof the table lookup was never the cost.
[3] Extension native pointer (opt1-native-distance, the ~1.3× on GCP D=32): at index-open, resolve the metric to a native fn pointer (m_dist_fn) once. Per call, decode into reused per-traversal buffers and call m_dist_fn(a,b) directly — no server round-trip, no vef_invalue_t marshalling, no boundary crossing, no per-call alloc.
So, directly: [1] vs [2] is ~a wash — that indirection is cheap either way (removing the table gave 0%). The win is [3], on a different axis entirely: not "direct pointer vs table pointer," but "does the per-call path cross the module boundary + marshal args + re-decode operands, or not." The dispatch form was never the bottleneck; the cross-.so VDF round-trip + marshalling + per-call decode was.
(Two ground-truth notes: the L2 helper is l2_squared_distance / dist_squared_l2; and the checked-out [3] already folds in a decode-once reuse for operand a, so its 1.3× is "native-pointer dispatch + reused decode buffers" — but the dispatch removal is the structural change, orthogonal to table-vs-pointer.)
Activity
tomas-villagesql commentedon Aug 26, 2026
A(fn)→B vs A→A1(fn[idx])→B — which is winning?
Neither. The pointer-vs-pointer-table distinction isn't the axis that matters — both [1] and [2] pay the same expensive per-call work. What won (~1.3×) is a third thing that removes that work entirely: it doesn't go A→B through the server at all.
Per distance call, grounded in the code:
[1] Baseline (server VDF dispatch): IndexGraph::distance → m_index.helper() → ctx_.helper_fn() crosses the .so boundary into the server's dispatch_profile_call, which does profile_for_key + find_fn_binding (linear scan) + type-ctx chase → call_vdf: marshal both operands into vef_invalue_t[], build vef_context_t/vef_vdf_args_t → indirect binding.vdf() back across the boundary → extension svector_distance_impl: alloc two per-call buffers, from_encoded both operands → native kernel. That's A → (*fn) → B plus a boundary crossing + marshalling + per-call decode.
[2] The server-side ResolvedFn cache we just tried (= A → A1(*fn[idx]) → B): memoizes {binding, tc} per index so it skips profile_for_key + find_fn_binding + type-ctx fetch. But it still does call_vdf (full marshalling) + the indirect cross-.so binding.vdf call + per-call decode of both operands. Only the lookup is removed. → ~0% on D=32 Mac. That's the proof the table lookup was never the cost.
[3] Extension native pointer (opt1-native-distance, the ~1.3× on GCP D=32): at index-open, resolve the metric to a native fn pointer (m_dist_fn) once. Per call, decode into reused per-traversal buffers and call m_dist_fn(a,b) directly — no server round-trip, no vef_invalue_t marshalling, no boundary crossing, no per-call alloc.
So, directly: [1] vs [2] is ~a wash — that indirection is cheap either way (removing the table gave 0%). The win is [3], on a different axis entirely: not "direct pointer vs table pointer," but "does the per-call path cross the module boundary + marshal args + re-decode operands, or not." The dispatch form was never the bottleneck; the cross-.so VDF round-trip + marshalling + per-call decode was.
(Two ground-truth notes: the L2 helper is l2_squared_distance / dist_squared_l2; and the checked-out [3] already folds in a decode-once reuse for operand a, so its 1.3× is "native-pointer dispatch + reused decode buffers" — but the dispatch removal is the structural change, orthogonal to table-vs-pointer.)