Within-Dataset Loss/Complexity Tradeoff
Generated by scripts/run_within_dataset_tradeoff_analysis.py.
This analysis avoids pooling unrelated reconstruction targets into one regression. It fits the loss/complexity pattern separately within each recovered Donker section, then checks whether the pattern repeats across sections.
Decision trees and random forests use adjusted tree complexity max(1, leaves * log2(leaves)); a 16-leaf tree therefore counts as 64 parameters. Random forests in this run use 50 estimators.
The line diagnostics use an ordinary least-squares fit over the five family-best points in each dataset. linear_p_value is the two-sided p-value for a nonzero slope.
- Negative within-dataset log-log slopes:
9/9 - Exact p-adic loss between dummy and the best high-capacity family:
9/9 - Exact p-adic adjusted parameters between dummy and the largest high-capacity family:
9/9 - Best decision tree lower loss than exact p-adic:
9/9 - Best decision tree fewer adjusted parameters than exact p-adic:
7/9
Family Medians
| model_group | datasets | median_loss | median_adjusted_parameters | median_log10_loss | median_log10_adjusted_parameters |
|---|---|---|---|---|---|
| Dummy | 9 | 0.2241 | 1.0000 | -0.6495 | 0.0000 |
| Decision tree | 9 | 0.0801 | 43.3038 | -1.0963 | 1.6365 |
| Exact p-adic | 9 | 0.1643 | 60.0000 | -0.7843 | 1.7782 |
| Logistic | 9 | 0.0705 | 443.3333 | -1.1521 | 2.6467 |
| Random forest | 9 | 0.0493 | 6096.6947 | -1.3072 | 3.7851 |
Dataset Diagnostics
| dataset | family_points | slope_log_loss_vs_log_params | r_squared | linear_p_value | spearman_rho | spearman_p | padic_loss | padic_parameters | decision_tree_loss | decision_tree_parameters | best_high_capacity_loss |
|---|---|---|---|---|---|---|---|---|---|---|---|
| donker_ath_1cor | 5 | -0.2487 | 0.8742 | 0.0197 | -0.9000 | 0.0374 | 0.1602 | 60.0000 | 0.1171 | 37.3248 | 0.0319 |
| donker_ath_2cor_titus | 5 | -0.2198 | 0.7213 | 0.0686 | -0.7000 | 0.1881 | 0.1643 | 60.0000 | 0.0585 | 43.3038 | 0.0219 |
| donker_ath_acts | 5 | -0.0898 | 0.8655 | 0.0218 | -1.0000 | 0.0000 | 0.1111 | 60.0000 | 0.0960 | 62.2011 | 0.0544 |
| donker_ath_acts_13_28 | 5 | -0.1642 | 0.8406 | 0.0284 | -1.0000 | 0.0000 | 0.3575 | 40.0000 | 0.1808 | 73.6824 | 0.1133 |
| donker_ath_acts_1_12 | 5 | -0.0744 | 0.8642 | 0.0222 | -0.8000 | 0.1041 | 0.0669 | 60.0000 | 0.0601 | 48.8419 | 0.0482 |
| donker_ath_catholicepistles | 5 | -0.2602 | 0.6203 | 0.1137 | -0.6000 | 0.2848 | 0.0527 | 48.0000 | 0.0492 | 25.8009 | 0.0349 |
| donker_ath_heb | 5 | -0.2571 | 0.7600 | 0.0540 | -0.9000 | 0.0374 | 0.3540 | 59.8333 | 0.1145 | 38.1310 | 0.0438 |
| donker_ath_paulineepistles | 5 | -0.2013 | 0.7657 | 0.0520 | -0.9000 | 0.0374 | 0.1856 | 60.0000 | 0.0766 | 36.9921 | 0.0547 |
| donker_ath_rom | 5 | -0.1192 | 0.5604 | 0.1455 | -0.9000 | 0.0374 | 0.1728 | 60.0000 | 0.0801 | 44.3052 | 0.0640 |
Figures
- Faceted within-dataset plot:
outputs/figures/within_dataset_tradeoff_facets.png - Family-best pooled reference plot:
outputs/figures/within_dataset_tradeoff_family_best.png
Interpretation note: the faceted figure is the main evidence. The pooled family-best plot is only a compact overview after the within-section summaries have been computed.