Files
fidc-backtest-engine/docs/selection-risk-plan-performance-20260914.md
T

7.1 KiB

Selection Risk Plan Performance

Status

Candidate tested, not deployed. The change removes selection calls that have no possible effect under the current frozen policy. It does not disable any configured rule, execution-day check or strategy expression. Engine time falls slightly in the measured cases; this is not the solution to the main remaining data construction cost and is not a general whole-backtest speedup claim.

The independent intraday-clock counterexample remains open. This work does not remove that test or its evidence, change the execution clock, or turn day-level parity into full framework acceptance. The published service stays at e81bf47/c98bcc3. Source d5b682c6 remains frozen; research and signal work stay paused. No trading operation was submitted.

Evidence Leading to the Change

The official HTTP diagnostic replay btr_1789322878865_2871869_0 preserved the original canonical and result-store SHA. Ten bounded Boris-only GDB snapshots showed source inventory, PreparedDayBuilder, factor normalization and price series construction, followed by repeated selection risk calls. GDB pauses are not normal performance measurements and snapshot counts are not flamegraph percentages. Source/target PID, binary SHA and CPU/thread resources stayed fixed.

The diagnostic helper now shares the existing canonical executable policy with the saved-run profiler: it accepts both audited build roots and immutable API release directories, but not arbitrary paths. Seven related tests passed.

Implementation and Correctness

StaticRiskRuleConfig reports whether selection has an enabled state rule or an enabled nonempty blacklist. The strategy computes this once before iterating candidate symbols. If no such rule exists, the old selection function would always return None, so that no-op call is omitted. Explicit universe conditions, market/factor checks and all buy/sell execution paths are unchanged.

The ten state flags are also shared with the existing missing-risk-state checks to avoid maintaining three separate flag lists. Blacklist presence is kept separate: a blacklist is not missing market-risk data. No cross-strategy cache, strategy identifier, fixed date, trading time or account state is introduced.

Tests enumerate all 4,096 combinations of ten selection flags and blacklist enabled/populated states. The flag list is checked against the serialized configuration, so adding a selection field requires updating the activation test. Further tests retain missing-state behavior and show that paused buys and sells remain rejected when selection checks are inactive.

On 177: 805 core unit/integration tests passed (9 ignored), 448 runner tests passed (9 ignored), 119 API tests passed (5 ignored), and 28 benchmark/profiler tests passed. These counts do not resolve the independently recorded clock failure, which is not part of this frozen committed test tree.

Reproducible Shared-Input Method

Each of the twelve replays has a new process and a new private result root. The official runner benchmark gained --shared-runtime-cache. It resolves the explicit cache root from the declared Boris service, requires canonical private storage, hashes existing inputs before and after, and refuses any changed or removed original. This mode cannot invoke copied-input disposal.

All twelve runs used the same 9,257 files / 12,596,608,049 bytes. Their complete input inventories, file identities and byte SHA values are equal. No new Arrow or binary cache input appeared. No backtest result was reused. Hash preparation and verification are outside the measured runner interval; this is a shared warm-input test, not raw-disk cold IO. Unlike the earlier copied-cache method, it does not allocate another approximately 2 GB per replay on the nearly full SSD. Original inputs and every result remain intact.

The common execution interval is 2021-08-23 through 2025-11-17 with 10,000,000 initial cash and each case's unchanged frozen strategy/bundle. This is not five complete execution years. CPU affinity and 8 Rayon / 16 Tokio threads match the declared reference service; no global resource limit was increased.

Measurements

Case Wall seconds Source validation Data preparation Engine
Rotation control 1 30.986 12.244 8.445 8.790
Rotation candidate 1 12.977 0.004 5.132 6.682
Rotation control 2 13.275 0.209 5.190 6.739
Rotation candidate 2 12.927 0.005 5.128 6.640
Rotation candidate 3 12.926 0.006 5.113 6.664
Rotation control 3 13.044 0.005 5.159 6.732
Trend 40 control 15.628 0.694 5.234 8.199
Trend 40 candidate 14.826 0.004 5.276 8.087
Pullback 40 control 13.776 0.005 4.893 7.374
Pullback 40 candidate 13.928 0.003 5.137 7.239
Volume momentum 80 control 18.176 0.004 4.585 11.154
Volume momentum 80 candidate 18.627 0.005 5.200 11.013

Rotation engine medians are 6.739 versus 6.664 seconds, approximately 1.1%. The other paired engine reductions are approximately 1.4%, 1.8% and 1.3%. These are small CPU-path improvements. Pullback and volume total latency did not improve because their preparation times were higher. The first control's Source wait and unexplained slower construction are recorded, not attributed to this code or discarded to manufacture a large speedup. Peak RSS stays about 6.76-6.83 GiB; there is no significant memory reduction claim.

Each case matches its independent prior baseline for all six canonical sections and store bytes: 21,393 / 29,776 / 31,862 / 51,300 fills. Result receipts, runtime/strategy identities, physical manifests and 3,506 fact blocks were verified. The shared input inventory SHA is in the acceptance record. Full unaltered receipts remain on 177; the repository stores the compact verified summary rather than repeating the 9,257-file inventory in every document.

Remaining Work

Prioritize direct typed-column reuse during daily snapshot and DataSet construction; approximately five seconds of preparation remain in these warm cases. Do not skip normalization, NULL, adjustment or date/uniqueness checks. Source cold-query and contract-validation latency remain separate tasks under the Source freeze. The earlier cache-boundary candidate still needs its missing cold/same-window acceptance, and this combined candidate has no HTTP publication gate yet. Financial PIT, minute-clock behavior, signal lifecycle and UI factor condition acceptance are not claimed complete.

  • Candidate engine: d2aa16a2f0.
  • Candidate service source: 4e23c7558d8301ba697543c39d5604289bb82c53.
  • Control runner SHA: b90886b80634c7565ca215fbe1f9ed0cbb5a6bd967373a9b1f6753be5164737d.
  • Candidate runner SHA: 1bda2d3acc016ca5addbb12e33cfcc31a23ece562f1d7d1ff8a825fbc83873fb.
  • Candidate API SHA: 30ac3b50996e1769c1d93bd5d302a23c4af7ebe773d3e8110ee278c44aeb9501.
  • Evidence root: /srv/fidc/canonical/run/research/selection-risk-plan-20260914.
  • HTTP diagnostic: /srv/fidc/canonical/run/research/http-phase-profile-20260914.

Verified acceptance summary.