Reports / dimensionality_and_variant_similarity.md
Dimensionality and Derived Variant-State Similarity
This report is exploratory. The witness-row embeddings use the processed Donker-derived feature matrices. The feature-state analysis treats each observed reading state as a support pattern over witnesses, so two states are similar when they occur in similar sets of witnesses.
This does not yet compare original manuscript readings directly. The local files contain derived encodings and a sample apparatus page, not the full source-level collation for all variation units.
Witness Row Embeddings
Figures are written to outputs/figures/*_witness_*_embedding.png. UMAP and MDS use the default row distance for each encoding: Jaccard for binary flags and simple matching for multistate states. PCA uses one-hot encoded feature states.
Nearest Witness Pairs
Binary
| dataset |
dataset_label |
witness_a |
witness_b |
distance |
| athanasius_epistles_data_bin_csv |
Binary |
F |
G |
0.0555556 |
| athanasius_epistles_data_bin_csv |
Binary |
M223 |
M2423 |
0.210526 |
| athanasius_epistles_data_bin_csv |
Binary |
U1 |
M1739 |
0.411765 |
| athanasius_epistles_data_bin_csv |
Binary |
B |
C |
0.444444 |
| athanasius_epistles_data_bin_csv |
Binary |
D |
G |
0.458333 |
| athanasius_epistles_data_bin_csv |
Binary |
C |
D |
0.47619 |
| athanasius_epistles_data_bin_csv |
Binary |
D |
F |
0.5 |
| athanasius_epistles_data_bin_csv |
Binary |
P46 |
U1 |
0.526316 |
| athanasius_epistles_data_bin_csv |
Binary |
P46 |
M1739 |
0.55 |
| athanasius_epistles_data_bin_csv |
Binary |
A |
C |
0.555556 |
Multistate
| dataset |
dataset_label |
witness_a |
witness_b |
distance |
| athanasius_epistles_data_multistate_csv |
Multistate |
F |
G |
0.0333333 |
| athanasius_epistles_data_multistate_csv |
Multistate |
M223 |
M2423 |
0.0666667 |
| athanasius_epistles_data_multistate_csv |
Multistate |
U1 |
M1739 |
0.233333 |
| athanasius_epistles_data_multistate_csv |
Multistate |
A |
M1739 |
0.266667 |
| athanasius_epistles_data_multistate_csv |
Multistate |
B |
C |
0.266667 |
| athanasius_epistles_data_multistate_csv |
Multistate |
A |
C |
0.3 |
| athanasius_epistles_data_multistate_csv |
Multistate |
U1 |
A |
0.3 |
| athanasius_epistles_data_multistate_csv |
Multistate |
A |
M2423 |
0.333333 |
| athanasius_epistles_data_multistate_csv |
Multistate |
C |
M1739 |
0.333333 |
| athanasius_epistles_data_multistate_csv |
Multistate |
D |
F |
0.333333 |
Feature-State Objects
| dataset_label |
feature_states |
mean_support |
| Binary |
36 |
4.75 |
| Multistate |
83 |
4.33735 |
Figures are written to outputs/figures/*_feature_state_*_embedding.png and outputs/figures/*_feature_state_support_similarity_heatmap.png.
Most Similar Feature-State Pairs
Binary
| feature_state_a |
feature_state_b |
jaccard_similarity |
support_overlap |
support_union |
support_a |
support_b |
| Phil.2.5.2=1 |
1Cor.15.55.1.b1=1 |
0.888889 |
8 |
9 |
P46;U1;A;B;C;D;F;G;M1739 |
P46;U1;B;C;D;F;G;M1739 |
| 1Cor.15.55.1.b2=1 |
1Cor.11.2.1=1 |
0.833333 |
5 |
6 |
A;D;F;G;M223;M2423 |
D;F;G;M223;M2423 |
| 1Cor.9.22.1.b2=1 |
1Cor.11.2.1=1 |
0.833333 |
5 |
6 |
C;D;F;G;M223;M2423 |
D;F;G;M223;M2423 |
| 1Cor.1.24.1=1 |
1Cor.15.53.1.b1=1 |
0.8 |
8 |
10 |
U1;A;B;C;D;F;G;M223;M1739;M2423 |
A;B;C;D;G;M223;M1739;M2423 |
| 1Cor.9.16.2=1 |
1Cor.11.2.2.b2=1 |
0.8 |
4 |
5 |
B;C;D;F;G |
C;D;F;G |
| 1Cor.11.2.1=1 |
1Cor.4.6.3.b3=1 |
0.8 |
4 |
5 |
D;F;G;M223;M2423 |
F;G;M223;M2423 |
| 2Cor.1.10.1.b2=1 |
1Cor.4.6.3.b3=1 |
0.8 |
4 |
5 |
F;G;M223;M1739;M2423 |
F;G;M223;M2423 |
| 1Cor.4.6.3.b1=1 |
Rom.15.19.1.b1=1 |
0.777778 |
7 |
9 |
P46;U1;A;B;C;D;M223;M1739 |
P46;U1;A;B;C;M223;M1739;M2423 |
| 1Cor.15.53.1.b1=1 |
1Cor15.54.1.b2=1 |
0.75 |
6 |
8 |
A;B;C;D;G;M223;M1739;M2423 |
A;B;C;D;M223;M2423 |
| 1Cor.15.55.1.b1=1 |
1Cor.2.4.1.b1=1 |
0.75 |
6 |
8 |
P46;U1;B;C;D;F;G;M1739 |
P46;U1;B;D;F;G |
| 1Cor.15.55.1.b1=1 |
Col.1.16.1.b2=1 |
0.75 |
6 |
8 |
P46;U1;B;C;D;F;G;M1739 |
P46;U1;D;F;G;M1739 |
| 1Cor.11.2.2.b2=1 |
Rom.15.19.1.b2=1 |
0.75 |
3 |
4 |
C;D;F;G |
D;F;G |
| Rom.8.22.1=1 |
Phil.2.6.1=1 |
0.75 |
3 |
4 |
P46;B;F;G |
P46;F;G |
| Col.1.16.1.b1=1 |
1Cor.15.47.1.b2=1 |
0.75 |
3 |
4 |
A;B;M223;M2423 |
A;M223;M2423 |
| 1Cor.1.24.1=1 |
Phil.2.5.2=1 |
0.727273 |
8 |
11 |
U1;A;B;C;D;F;G;M223;M1739;M2423 |
P46;U1;A;B;C;D;F;G;M1739 |
Multistate
| feature_state_a |
feature_state_b |
jaccard_similarity |
support_overlap |
support_union |
support_a |
support_b |
| Phil.2.6.1=1 |
Rom.9.5.1=1 |
1 |
9 |
9 |
Ath;U1;A;B;C;D;M223;M1739;M2423 |
Ath;U1;A;B;C;D;M223;M1739;M2423 |
| Rom.15.19.1=2 |
Rom.15.19.2=2 |
1 |
8 |
8 |
P46;U1;A;B;C;M223;M1739;M2423 |
P46;U1;A;B;C;M223;M1739;M2423 |
| 1Cor.10.13.1=2 |
1Cor.11.2.3=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.10.13.1=2 |
1Cor.15.55.1=4 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.10.13.1=2 |
1Cor.9.22.1=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.10.13.1=2 |
Rom.15.19.1=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.11.2.3=3 |
1Cor.15.55.1=4 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.11.2.3=3 |
1Cor.9.22.1=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.11.2.3=3 |
Rom.15.19.1=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.15.55.1=4 |
1Cor.9.22.1=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.15.55.1=4 |
Rom.15.19.1=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.2.4.1=2 |
Phil.2.6.1=2 |
1 |
3 |
3 |
P46;F;G |
P46;F;G |
| 1Cor.9.22.1=3 |
Rom.15.19.1=3 |
1 |
3 |
3 |
D;F;G |
D;F;G |
| 1Cor.15.47.1=3 |
1Cor.15.55.1=3 |
1 |
3 |
3 |
A;M223;M2423 |
A;M223;M2423 |
| 1Cor.4.6.3=5 |
1Cor15.54.1=5 |
1 |
2 |
2 |
F;G |
F;G |
Output Tables
outputs/tables/witness_dimensionality_embeddings.csvoutputs/tables/witness_nearest_pairs.csvoutputs/tables/variant_feature_state_support.csvoutputs/tables/variant_feature_state_similarity_pairs.csvoutputs/tables/variant_feature_state_embeddings.csv