File guru.hpp

↰ Parent directory (include/treeweave)

The guru interface: the batch pipeline as public, caller-driven stages — caller-owned scratch, caller-driven fusion, no per-call allocation. Named after FFTW’s guru interface, the established precedent for an expert API exposing the planner/executor internals.

Definition (include/treeweave/guru.hpp)

Detailed Description

The public Function batch evaluators fuse the stages (classify -> histogram -> scan -> scatter -> per-leaf SIMD eval -> gather back) into one call. That composition is right for most uses, but a caller that fits several Functions over sub-ranges of one domain (e.g. a hard function partitioned into regimes — the standard treeweave methodology: split at the singularities/asymptotes, subtract the singular part so each piece fits a polynomial, evaluate the piecewise part per regime, then add the analytically-known part back) re-pays the crossing costs: a range-partition sweep plus per-range scatter, a separate post-processing sweep, fresh allocations every call. The pieces here re-expose the stages so such callers can:

  • classify each point once into one combined key (key = range_base + f.leaf_id(x) for the regime’s fit — the classification machinery is the public Function interface: leaf_id/leaf_ids, has_fast_quantize(), num_leaves(), out_of_domain_id(); LaneQuantizer is the SIMD lane-level twin of leaf_id, so the combined key folds branchlessly in registers),

  • run ONE counting sort over those keys, on caller-owned buffers that persist across calls (exclusive_scan + scatter when the histogram is fused into the classify sweep, otherwise the one-shot counting_sort),

  • evaluate each packed run straight through polyfit’s SIMD kernel and fuse their own elementwise post-processing while the run’s data is still in registers (eval_leaf_aos / eval_leaf_soa on the runs of for_each_run, fill_out_of_domain on the out-of-domain bucket), then gather back through rank,

  • do the same run-level fusion on sorted input with zero scratch (for_each_sorted_run: sorted input needs no counting sort at all).

Preconditions mirror the library’s: ids and ranks are std::uint32_t, so a tile stays below 2^32 points. Classification semantics are identical to the public paths point-for-point. A point is out-of-domain (OOD) when the fit cannot evaluate it — below lower, above upper, or NaN/±Inf; classification maps it to the sentinel id f.out_of_domain_id() (== num_leaves()), via the positive-logic gate that keeps the closed upper endpoint in-domain.

Thread safety: everything here is reentrant and stateless; the buffers are caller arguments. The usual rule applies — disjoint output slices and disjoint scratch per thread.

Combined-key rules, in one place:

  • Keys are fit-local ids with the fit’s OWN sentinel FIRST folded, then offset per regime: key = (id == f.out_of_domain_id()) ? my_ood_bucket / : (range_base + id). The fold compares against the fit’s own sentinel — never against an offset number. Lay the per-regime spaces back-to-back without holes; the aggregated OOD bucket comes last.

  • After the sort, rank[i] holds the packed slot of point i — the INVERSE permutation, which the writeback consumes directly: out[i] = packed_out[rank[i]] per output component (gather), unconditionally — every slot belongs to exactly one bin, so OOD buckets too. (eval_scatter_sorted in <treeweave/eval_scatter.hpp> records the opposite, FORWARD permutation: its perm[k] is the original index of packed slot k.)

  • Tile at what keeps the per-point scratch (keys 4 B + rank 4 B + packed in and out streams) L2-resident; on a 2 MiB L2 host the valley is 2^14 points for a 4-output fit (sweep {16384, 32768, 65536} when the layout differs).

  • Contract violations (key >= counts.size() into scatter, the OOD sentinel into eval_leaf_*) are assert-guarded: they fail cleanly in Debug and undefined in NDEBUG — build Debug/ASan when wiring a new caller.

  • Classification is SIMD when f.has_fast_quantize() (leaf_id / leaf_ids, and any tables fused from the fit’s quantize view) and per-point tree descent otherwise; the ids are identical either way, only the cost moves. ARM/NEON, wasm, and x86 below AVX2 run the same semantics entirely through the public scalar machinery.

Includes

  • algorithm

  • array

  • cassert

  • concepts

  • cstddef

  • cstdint

  • ranges

  • span

  • treeweave/detail/quantize.hpp

  • treeweave/treeweave.hpp (File treeweave.hpp)

  • type_traits

  • xsimd/xsimd.hpp

Namespaces

Classes

Functions