add youdens j

[lshpaner]

Youden and F1 aren’t the same criterion. F1 ignores true negatives entirely, Youden weights sensitivity and specificity equally, and Youden is the conventional operating-point choice in clinical prediction papers.

• New function in threshold_optimization.py, exported from __init__.py. Returns (threshold, None) so the tuple shape matches find_optimal_threshold_beta and existing call sites don't need special-casing. Beta doesn't exist for Youden, hence the None.

• Vectorized over the threshold grid, so it's a single boolean broadcast instead of a loop. Uses strict > to match threshold_tune.

• Raises if either input is empty, and raises if y has only one class, since J is undefined there. Returning NaN instead would let a broken threshold leak downstream into whatever's consuming it.

Nothing existing changes. It's additive.

Tests

19 in test_threshold_optimization.py. The three that matter:

• test_youden_maximizes_j reimplements J independently and checks nothing in the grid beats what got returned

• test_youden_imbalanced_gives_usable_operating_point runs a 10% prevalence fixture and asserts the thing actually flags positives, which is the regression guard for the case above

• test_youden_single_class checks it raises instead of quietly handing back NaN

• Add threshold_testing_rf_youden.py testing script:

Threshold after grid search (uncalibrated): 0.260
Fitting model with best params and tuning for best threshold ...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  9.85it/s]
roc_auc after calibration: 0.9509882478632479
Threshold after calibration (F-beta sweep) : 0.110
Found optimal threshold via Youden's J: 0.22
Sensitivity: 0.904, Specificity: 0.886
Threshold from Youden's J                  : 0.220

==============================================================================
Threshold selected on validation, evaluated on test
==============================================================================
validation prevalence : 0.2430
test AUC              : 0.9346
test average precision: 0.8103

              threshold    TP    FN    FP     TN  sensitivity  specificity  precision     F1  youden_J
criterion                                                                                             
F-beta sweep       0.11  93.0  11.0  52.0  272.0        0.894        0.840      0.641  0.747     0.734
Youden's J         0.22  86.0  18.0  37.0  287.0        0.827        0.886      0.699  0.758     0.713

------------------------------------------------------------------------------
F1  = 2TP / (2TP + FP + FN)          <- TN absent
J   = TP/(TP+FN) + TN/(TN+FP) - 1    <- TN present
------------------------------------------------------------------------------