TabularBench

TabularBench: Adversarial robustness benchmark for tabular data.

Documentation.

Leaderboard

You are currently viewing results of the leaderboard. View MOEVA attack results here.

Among models that demonstrate strong robustness to constrained adversarial attacks (high ADV+CTR), we observe that some achieve this robustness solely by consistently predicting the “1” class. This behavior is evident in their poor accuracy and precision. Therefore, we rank models based on their average performance across clean accuracy (Accuracy) and constrained adversarial accuracy (ADV+CTR).

Jump to dataset:

CTU

^ back to top

Rank Architecture Training Augmentation ID ADV+CTR ADV AUC Accuracy Precision Recall MCC
1 RLN Adv TableGAN 97.54% 97.49% 97.54% 0.9903 99.90% 89.62% 97.54% 0.9345
2 RLN Adv WGAN 97.54% 97.44% 97.54% 0.9901 99.92% 92.33% 97.54% 0.9486
3 RLN Adv None 97.30% 97.05% 97.30% 0.9898 99.90% 90.41% 97.30% 0.9374
4 RLN Adv TVAE 97.54% 96.76% 97.54% 0.9875 98.57% 33.82% 97.54% 0.5700
5 RLN Adv CutMix 95.33% 95.33% 95.33% 0.9869 99.97% 100.00% 95.33% 0.9762
6 RLN Std CutMix 95.33% 95.33% 95.33% 0.9885 99.96% 98.73% 95.33% 0.9699
7 TabTransformer Std None 95.33% 95.33% 95.33% 0.9788 99.95% 98.23% 95.33% 0.9675
8 TabTransformer Adv CutMix 95.33% 95.33% 95.33% 0.9796 99.95% 98.23% 95.33% 0.9675
9 TabTransformer Adv WGAN 95.33% 95.33% 95.33% 0.9836 99.95% 98.23% 95.33% 0.9675
10 TabTransformer Std WGAN 95.33% 95.33% 95.33% 0.9816 99.95% 98.23% 95.33% 0.9675
11 STG Std WGAN 95.33% 95.33% 95.33% 0.9864 99.95% 98.23% 95.33% 0.9675
12 STG Std TableGAN 95.33% 95.33% 95.33% 0.9861 99.95% 98.23% 95.33% 0.9675
13 TabNet Std TableGAN 95.33% 95.33% 95.33% 0.9801 99.95% 98.23% 95.33% 0.9675
14 STG Std CutMix 95.33% 95.33% 95.33% 0.9861 99.95% 98.23% 95.33% 0.9675
15 STG Std None 95.33% 95.33% 95.33% 0.9884 99.95% 98.23% 95.33% 0.9675
16 STG Adv WGAN 95.33% 95.33% 95.33% 0.9847 99.95% 98.23% 95.33% 0.9675
17 TabTransformer Adv None 95.33% 95.33% 95.33% 0.9846 99.95% 98.23% 95.33% 0.9675
18 TabTransformer Adv TableGAN 95.33% 95.28% 95.33% 0.9791 99.96% 98.73% 95.33% 0.9699
19 STG Std CT-GAN 95.58% 95.33% 95.58% 0.9904 99.88% 89.02% 95.58% 0.9218
20 RLN Adv CT-GAN 97.54% 96.66% 97.54% 0.9923 98.50% 32.73% 97.54% 0.5605
21 VIME Adv WGAN 95.33% 95.18% 95.33% 0.9785 99.96% 99.74% 95.33% 0.9749
22 STG Adv TableGAN 95.09% 95.09% 95.09% 0.9845 99.96% 100.00% 95.09% 0.9749
23 STG Adv None 95.09% 95.09% 95.09% 0.9865 99.96% 99.23% 95.09% 0.9712
24 TabNet Std WGAN 95.09% 95.09% 95.09% 0.9832 99.95% 98.47% 95.09% 0.9674
25 RLN Std WGAN 96.56% 95.04% 96.56% 0.9906 99.88% 88.71% 96.56% 0.9250
26 TabTransformer Std CutMix 95.33% 94.94% 95.33% 0.9767 99.95% 98.23% 95.33% 0.9675
27 STG Adv CutMix 94.59% 94.55% 94.59% 0.9845 99.96% 100.00% 94.59% 0.9724
28 VIME Adv CutMix 95.09% 94.25% 94.74% 0.9759 99.96% 99.74% 95.09% 0.9737
29 RLN Std CT-GAN 97.54% 95.63% 97.54% 0.9936 98.57% 33.84% 97.54% 0.5702
30 VIME Adv None 95.09% 94.00% 94.20% 0.9825 99.96% 99.74% 95.09% 0.9737
31 TabTransformer Std TableGAN 95.09% 93.91% 95.09% 0.9776 99.95% 98.72% 95.09% 0.9687
32 RLN Std None 97.79% 94.00% 97.79% 0.9906 99.82% 81.89% 97.79% 0.8941
33 RLN Std TVAE 97.30% 93.22% 97.30% 0.9883 98.71% 36.16% 97.30% 0.5891
34 TabTransformer Std TVAE 96.31% 96.07% 96.31% 0.9768 94.27% 11.10% 96.31% 0.3166
35 STG Adv CT-GAN 96.07% 96.02% 95.92% 0.9863 92.96% 9.19% 96.07% 0.2855
36 VIME Adv TableGAN 95.09% 85.50% 89.39% 0.9788 99.96% 99.74% 95.09% 0.9737
37 STG Std TVAE 96.31% 96.07% 96.31% 0.9841 89.01% 6.10% 96.31% 0.2274
38 RLN Std TableGAN 97.54% 81.43% 97.54% 0.9924 99.88% 88.03% 97.54% 0.9261
39 VIME Std TableGAN 95.09% 66.98% 95.09% 0.9840 99.96% 99.74% 95.09% 0.9737
40 TabTransformer Adv TVAE 98.28% 98.28% 98.28% 0.9741 60.87% 1.82% 98.28% 0.1030
41 STG Adv TVAE 98.28% 98.28% 98.23% 0.9811 43.56% 1.27% 98.28% 0.0717
42 VIME Std None 95.09% 40.84% 95.09% 0.9873 99.96% 99.74% 95.09% 0.9737
43 VIME Std CutMix 95.09% 35.04% 95.09% 0.9907 99.96% 99.74% 95.09% 0.9737
44 VIME Std WGAN 95.33% 22.85% 95.33% 0.9771 99.97% 100.00% 95.33% 0.9762
45 TabNet Adv CT-GAN 100.00% 100.00% 100.00% 0.9768 1.57% 0.75% 100.00% 0.0079
46 VIME Std TVAE 100.00% 100.00% 100.00% 0.9496 0.75% 0.74% 100.00% 0.0010
47 VIME Adv CT-GAN 100.00% 100.00% 100.00% 0.7411 0.74% 0.74% 100.00% 0.0000
48 VIME Std CT-GAN 100.00% 100.00% 100.00% 0.9724 0.74% 0.74% 100.00% 0.0000
49 VIME Adv TVAE 100.00% 100.00% 100.00% 0.7273 0.74% 0.74% 100.00% 0.0000
50 TabNet Adv TVAE 100.00% 100.00% 100.00% 0.9764 0.74% 0.74% 100.00% 0.0000
51 TabNet Adv TableGAN 1.47% 1.43% 1.43% 0.9930 99.27% 100.00% 1.47% 0.1210
52 TabNet Std None 96.07% 0.00% 96.07% 0.9963 99.94% 95.83% 96.07% 0.9592
53 TabNet Adv None 0.25% 0.20% 0.20% 0.9781 99.26% 50.00% 0.25% 0.0347
54 TabNet Std CutMix 0.00% 0.00% 0.00% 0.9823 99.26% 0.00% 0.00% 0.0000
55 TabNet Std CT-GAN 0.00% 0.00% 0.00% 0.9861 99.26% 0.00% 0.00% 0.0000
56 TabNet Std TVAE 0.00% 0.00% 0.00% 0.9865 99.26% 0.00% 0.00% 0.0000
57 TabNet Adv CutMix 0.00% 0.00% 0.00% 0.9823 99.26% 0.00% 0.00% 0.0000
58 TabNet Adv WGAN 0.00% 0.00% 0.00% 0.9868 99.26% 0.00% 0.00% 0.0000
59 TabTransformer Adv CT-GAN 100.00% 94.40% 100.00% 0.6272 4.49% 0.77% 100.00% 0.0170
60 TabTransformer Std CT-GAN 100.00% 94.40% 100.00% 0.6304 4.38% 0.77% 100.00% 0.0168
61 RLN Adv GOGGLE 96.31% 96.07% 96.31% - - - - -
62 STG Std GOGGLE 95.33% 95.33% 95.33% - - - - -
63 TabNet Adv GOGGLE 95.33% 95.33% 95.33% - - - - -
64 RLN Std GOGGLE 95.33% 95.33% 95.33% - - - - -
65 TabTransformer Adv GOGGLE 95.33% 95.23% 95.33% - - - - -
66 TabTransformer Std GOGGLE 95.33% 95.18% 95.33% - - - - -
67 STG Adv GOGGLE 95.09% 95.09% 95.09% - - - - -
68 VIME Adv GOGGLE 95.09% 94.99% 94.89% - - - - -
69 VIME Std GOGGLE 95.09% 76.90% 95.09% - - - - -
70 TabNet Std GOGGLE 95.33% 72.33% 76.56% - - - - -

LCLD

^ back to top

Rank Architecture Training Augmentation ID ADV+CTR ADV AUC Accuracy Precision Recall MCC
1 STG Adv CT-GAN 82.00% 81.16% 82.00% 0.7002 50.50% 26.61% 83.28% 0.2118
2 TabTransformer Adv CT-GAN 79.50% 78.50% 79.50% 0.6945 52.63% 27.09% 80.27% 0.2119
3 TabTransformer Adv CutMix 72.50% 70.96% 72.50% 0.7016 59.56% 29.41% 72.33% 0.2299
4 VIME Adv CT-GAN 78.90% 76.84% 78.90% 0.7009 53.50% 27.49% 80.30% 0.2203
5 STG Adv TVAE 68.90% 67.80% 68.90% 0.7056 62.53% 30.68% 68.69% 0.239
6 STG Adv GOGGLE 69.40% 68.20% 69.40% 0.6979 61.58% 30.04% 68.68% 0.229
7 TabTransformer Adv None 73.90% 70.28% 73.90% 0.7111 59.01% 29.33% 73.83% 0.2333
8 VIME Adv TVAE 67.80% 66.12% 67.80% 0.7054 62.79% 30.81% 68.41% 0.2401
9 TabTransformer Adv TVAE 68.70% 66.50% 68.70% 0.7024 61.96% 30.36% 69.09% 0.2353
10 RLN Adv TVAE 67.00% 64.34% 67.00% 0.7079 63.55% 31.20% 67.61% 0.2435
11 RLN Adv None 69.50% 63.04% 69.48% 0.7158 62.75% 30.93% 69.28% 0.2448
12 STG Adv TableGAN 62.70% 60.08% 62.70% 0.6995 65.68% 31.83% 62.07% 0.2352
13 STG Adv WGAN 61.30% 59.68% 61.30% 0.6991 65.73% 31.80% 61.69% 0.2336
14 TabTransformer Adv GOGGLE 63.60% 60.54% 63.60% 0.6988 64.49% 31.18% 63.62% 0.2307
15 TabTransformer Adv WGAN 66.50% 62.84% 66.50% 0.6879 61.49% 29.57% 66.36% 0.2144
16 VIME Adv GOGGLE 56.80% 53.22% 56.80% 0.7032 68.53% 33.37% 56.88% 0.2393
17 TabTransformer Adv TableGAN 60.80% 56.44% 60.80% 0.694 65.11% 31.25% 61.43% 0.2251
18 VIME Adv TableGAN 56.30% 53.68% 56.30% 0.6955 67.66% 32.66% 57.40% 0.2318
19 VIME Adv CutMix 57.00% 52.86% 57.00% 0.7006 68.21% 33.19% 57.51% 0.239
20 TabNet Adv GOGGLE 100.00% 100.00% 100.00% 0.6829 20.09% 20.09% 100.00% 0
21 VIME Adv WGAN 61.70% 53.04% 61.70% 0.6994 65.96% 32.00% 61.77% 0.2367
22 STG Std None 66.40% 53.60% 66.40% 0.7087 64.56% 31.67% 65.99% 0.2452
23 STG Std TVAE 64.90% 50.50% 64.90% 0.7063 65.17% 31.87% 64.53% 0.2436
24 RLN Adv CutMix 58.10% 47.02% 58.10% 0.7058 68.34% 33.40% 57.99% 0.2432
25 STG Std TableGAN 51.00% 44.22% 51.00% 0.7016 71.04% 34.88% 50.91% 0.2378
26 RLN Adv CT-GAN 73.70% 54.34% 73.70% 0.7036 58.20% 29.04% 74.90% 0.2317
27 STG Adv CutMix 37.60% 36.20% 37.60% 0.7033 75.83% 39.26% 37.10% 0.2316
28 STG Std GOGGLE 57.70% 43.28% 57.70% 0.7037 67.74% 33.05% 59.08% 0.2422
29 RLN Adv TableGAN 68.80% 47.88% 68.80% 0.7045 62.94% 30.83% 67.95% 0.239
30 STG Std CT-GAN 83.30% 59.50% 83.30% 0.7053 50.26% 26.63% 84.07% 0.2149
31 RLN Adv WGAN 66.10% 40.18% 66.10% 0.7054 64.58% 31.56% 65.29% 0.2414
32 STG Std WGAN 61.40% 37.74% 61.40% 0.7047 66.93% 32.61% 60.60% 0.2412
33 STG Std CutMix 35.20% 22.16% 35.20% 0.7068 76.60% 40.38% 34.66% 0.2312
34 RLN Adv GOGGLE 67.80% 31.96% 67.80% 0.71 64.41% 31.66% 66.63% 0.2472
35 STG Adv None 15.60% 12.08% 15.60% 0.6788 78.82% 43.19% 17.18% 0.1699
36 TabNet Adv TableGAN 11.60% 11.36% 11.70% 0.6522 73.18% 22.51% 13.72% 0.0226
37 TabNet Std CT-GAN 2.90% 2.12% 3.00% 0.6874 78.45% 26.96% 4.25% 0.031
38 TabNet Std TableGAN 1.30% 1.04% 1.40% 0.6887 79.30% 25.50% 1.59% 0.0152
39 TabNet Std CutMix 0.00% 0.00% 0.10% 0.6998 79.91% 100.00% 0.00% 0.003
40 TabNet Std WGAN 0.00% 0.00% 0.10% 0.6649 79.91% 0.00% 0.00% 0
41 TabNet Std GOGGLE 0.00% 0.00% 0.10% 0.6727 79.91% 0.00% 0.00% 0
42 TabNet Adv WGAN 0.00% 0.00% 0.10% 0.6879 79.91% 0.00% 0.00% 0
43 TabNet Adv TVAE 0.00% 0.00% 0.10% 0.6959 79.91% 0.00% 0.00% 0
44 TabNet Adv CutMix 0.00% 0.00% 0.10% 0.6375 79.91% 0.00% 0.00% 0
45 TabNet Adv CT-GAN 0.00% 0.00% 0.10% 0.6947 79.91% 0.00% 0.00% 0
46 TabNet Adv None 0.00% 0.00% 0.10% 0.656 79.91% 0.00% 0.00% 0
47 TabNet Std TVAE 0.00% 0.00% 0.10% 0.6667 79.89% 24.84% 0.04% 0.0022
48 VIME Adv None 65.50% 10.36% 65.50% 0.7127 65.13% 32.05% 65.71% 0.2499
49 TabTransformer Std TableGAN 69.30% 10.14% 69.30% 0.7096 63.62% 31.28% 67.77% 0.2452
50 VIME Std CutMix 52.30% 2.00% 52.30% 0.7101 71.04% 35.26% 52.83% 0.2485
51 TabTransformer Std GOGGLE 68.90% 7.88% 68.90% 0.7122 63.82% 31.48% 68.10% 0.2493
52 TabTransformer Std None 69.50% 7.88% 69.50% 0.7172 63.32% 31.42% 69.87% 0.2542
53 TabTransformer Std CutMix 67.70% 7.30% 67.70% 0.7115 63.80% 31.41% 67.73% 0.247
54 TabTransformer Std CT-GAN 72.40% 8.14% 72.40% 0.7111 61.44% 30.42% 71.46% 0.2438
55 TabTransformer Std WGAN 70.10% 5.54% 70.10% 0.7108 63.44% 31.27% 68.43% 0.2472
56 TabTransformer Std TVAE 70.30% 4.82% 70.30% 0.7155 63.41% 31.39% 69.27% 0.2517
57 VIME Std TableGAN 60.70% 0.46% 60.70% 0.708 67.62% 33.23% 60.63% 0.2495
58 VIME Std GOGGLE 64.40% 0.50% 64.40% 0.7139 66.62% 32.83% 63.25% 0.2527
59 VIME Std None 67.00% 2.38% 67.00% 0.7142 64.46% 31.79% 67.13% 0.2506
60 VIME Std TVAE 66.80% 0.74% 66.80% 0.7138 65.36% 32.23% 65.69% 0.2524
61 TabNet Std None 67.40% 0.42% 67.40% 0.7224 65.60% 32.62% 66.83% 0.2615
62 VIME Std WGAN 65.90% 0.66% 65.90% 0.7081 64.83% 31.83% 65.76% 0.2468
63 RLN Std WGAN 67.90% 0.52% 67.90% 0.7125 64.40% 31.69% 66.82% 0.2483
64 RLN Std GOGGLE 67.30% 0.02% 67.30% 0.7169 64.84% 32.09% 67.19% 0.2552
65 RLN Std TableGAN 69.30% 0.14% 69.30% 0.7118 64.25% 31.64% 67.21% 0.2488
66 RLN Std None 68.30% 0.02% 68.30% 0.7187 64.13% 31.78% 68.51% 0.255
67 RLN Std TVAE 70.00% 0.00% 70.00% 0.7173 63.33% 31.40% 69.66% 0.2531
68 RLN Std CutMix 68.90% 0.00% 68.90% 0.7152 63.26% 31.29% 69.31% 0.2504
69 RLN Std CT-GAN 70.50% 0.12% 70.50% 0.7094 61.96% 30.57% 70.27% 0.2422
70 VIME Std CT-GAN 77.30% 1.76% 77.30% 0.7061 57.09% 28.71% 76.61% 0.2313

Malware

^ back to top

Rank Architecture Training Augmentation ID ADV+CTR ADV AUC Accuracy Precision Recall MCC
1 VIME Std None 95.00% 95.00% 95.00% 0.9893 95.49% 96.09% 95.01% 0.9098
2 STG Std None 93.00% 92.80% 93.00% 0.9907 94.88% 96.90% 92.93% 0.8985
3 RLN Std None 95.00% 90.20% 95.00% 0.9926 97.08% 97.81% 96.43% 0.9417
4 TabTransformer Std None 95.00% 89.80% 95.00% 0.9941 97.46% 98.33% 96.65% 0.9493
5 TabNet Std None 99.00% 80.20% 99.00% 0.9944 96.78% 96.32% 97.40% 0.9356
6 TabNet Adv None 99.00% 99.00% 99.00% - - - - -
7 RLN Adv None 96.00% 95.20% 96.00% - - - - -
8 TabTransformer Adv None 95.00% 94.20% 95.00% - - - - -
9 STG Adv None 93.00% 92.80% 93.00% - - - - -
10 VIME Adv None 92.00% 92.00% 92.00% - - - - -

URL

^ back to top

Rank Architecture Training Augmentation ID ADV+CTR ADV AUC Accuracy Precision Recall MCC
1 STG Adv None 94.30% 89.96% 90.32% 0.9485 86.22% 81.17% 94.31% 0.7341
2 TabNet Adv CutMix 93.00% 89.70% 89.64% 0.9348 86.05% 81.45% 93.35% 0.7287
3 TabNet Std CutMix 95.40% 89.34% 89.38% 0.9469 86.05% 80.16% 95.80% 0.735
4 TabNet Std TVAE 93.80% 89.08% 89.16% 0.9495 86.13% 81.29% 93.88% 0.7315
5 TabNet Adv CT-GAN 90.10% 89.90% 89.90% 0.9428 85.26% 81.93% 90.46% 0.709
6 TabNet Adv WGAN 89.80% 89.60% 89.60% 0.9562 85.26% 82.14% 90.11% 0.7085
7 TabNet Std GOGGLE 93.20% 89.56% 89.60% 0.934 85.13% 80.26% 93.18% 0.7118
8 STG Adv GOGGLE 93.90% 85.62% 86.04% 0.9542 88.23% 84.25% 94.05% 0.7699
9 TabNet Adv TVAE 94.00% 87.22% 86.98% 0.9418 86.44% 81.67% 93.96% 0.7372
10 TabNet Std TableGAN 89.60% 87.76% 87.50% 0.9382 85.78% 83.04% 89.94% 0.7181
11 STG Adv WGAN 94.20% 81.16% 81.28% 0.9628 89.63% 86.24% 94.31% 0.7961
12 STG Adv TableGAN 92.10% 80.86% 81.62% 0.9558 88.76% 86.19% 92.30% 0.7771
13 STG Adv CT-GAN 93.90% 79.78% 80.28% 0.9589 89.55% 86.33% 93.96% 0.794
14 STG Adv TVAE 95.70% 79.52% 80.40% 0.9608 88.93% 84.34% 95.63% 0.7857
15 STG Std GOGGLE 93.90% 74.52% 75.92% 0.9623 90.33% 87.60% 93.96% 0.8088
16 TabNet Std CT-GAN 99.40% 94.82% 94.78% 0.951 69.90% 62.52% 99.39% 0.4929
17 VIME Adv GOGGLE 91.90% 73.68% 74.92% 0.9551 90.42% 89.15% 92.04% 0.8088
18 VIME Adv None 93.40% 69.80% 72.68% 0.973 92.48% 91.75% 93.35% 0.8496
19 TabNet Adv None 99.50% 91.82% 91.88% 0.9472 70.03% 62.62% 99.39% 0.495
20 VIME Adv CutMix 92.00% 68.56% 70.68% 0.9673 91.78% 91.49% 92.13% 0.8355
21 STG Std CT-GAN 92.20% 69.26% 77.04% 0.9673 90.99% 89.80% 92.48% 0.8201
22 STG Std TVAE 94.10% 68.78% 73.28% 0.9692 91.29% 89.20% 93.96% 0.8271
23 RLN Adv GOGGLE 93.90% 66.10% 66.54% 0.9688 92.65% 91.63% 93.88% 0.8533
24 VIME Adv CT-GAN 91.00% 66.92% 69.00% 0.9651 91.21% 91.32% 91.08% 0.8242
25 TabTransformer Adv CT-GAN 93.00% 66.00% 66.42% 0.9625 91.60% 90.33% 93.18% 0.8324
26 STG Std WGAN 92.50% 65.54% 75.20% 0.97 91.29% 90.27% 92.56% 0.8262
27 TabNet Std WGAN 99.80% 95.18% 95.34% 0.9462 61.24% 56.35% 99.74% 0.3523
28 VIME Adv WGAN 89.70% 64.98% 70.52% 0.966 90.99% 91.87% 89.94% 0.82
29 VIME Adv TableGAN 88.70% 64.54% 65.16% 0.96 90.64% 92.27% 88.71% 0.8134
30 RLN Adv CT-GAN 93.80% 62.46% 62.78% 0.9727 92.52% 91.40% 93.88% 0.8507
31 VIME Adv TVAE 89.90% 63.60% 71.08% 0.9643 90.81% 91.54% 89.94% 0.8164
32 RLN Adv CutMix 94.30% 60.78% 60.86% 0.9769 93.31% 92.38% 94.40% 0.8663
33 TabTransformer Adv TVAE 93.40% 61.20% 61.54% 0.9692 92.48% 91.67% 93.44% 0.8497
34 TabNet Adv GOGGLE 84.80% 66.54% 66.64% 0.9387 86.79% 87.99% 85.21% 0.7361
35 RLN Adv TVAE 94.10% 59.80% 60.22% 0.9762 92.74% 91.65% 94.05% 0.8551
36 STG Std None 93.30% 57.98% 59.60% 0.9729 91.95% 90.81% 93.35% 0.8393
37 TabTransformer Adv None 93.90% 56.72% 57.80% 0.9743 93.09% 92.27% 94.05% 0.8619
38 RLN Adv None 95.20% 56.20% 56.62% 0.9768 93.31% 91.67% 95.28% 0.8668
39 TabTransformer Adv WGAN 92.70% 56.94% 58.02% 0.97 92.13% 91.62% 92.74% 0.8426
40 RLN Adv TableGAN 91.30% 55.46% 55.68% 0.9714 92.48% 93.31% 91.51% 0.8497
41 RLN Adv WGAN 93.30% 54.70% 55.24% 0.9764 92.74% 92.29% 93.26% 0.8548
42 VIME Std CT-GAN 92.70% 54.76% 91.00% 0.9678 91.56% 90.60% 92.74% 0.8314
43 TabTransformer Adv GOGGLE 91.70% 54.14% 55.44% 0.9645 91.51% 91.30% 91.78% 0.8303
44 VIME Std WGAN 92.20% 51.86% 90.54% 0.9685 91.73% 91.26% 92.30% 0.8347
45 VIME Std None 92.50% 49.54% 53.26% 0.9736 92.78% 92.89% 92.65% 0.8556
46 VIME Std CutMix 92.50% 46.66% 91.34% 0.9706 92.21% 92.07% 92.39% 0.8443
47 STG Std TableGAN 87.60% 46.86% 57.52% 0.9677 90.77% 93.31% 87.84% 0.8168
48 VIME Std TVAE 90.90% 44.36% 88.58% 0.9685 91.38% 91.93% 90.73% 0.8277
49 VIME Std GOGGLE 89.30% 44.50% 85.70% 0.9598 90.03% 90.81% 89.06% 0.8007
50 VIME Std TableGAN 87.50% 42.98% 75.04% 0.9626 90.46% 93.02% 87.49% 0.8107
51 TabTransformer Adv TableGAN 89.80% 40.90% 42.08% 0.9669 91.91% 93.55% 90.03% 0.8387
52 TabTransformer Adv CutMix 85.00% 40.30% 40.36% 0.9563 89.98% 93.69% 85.74% 0.8025
53 STG Adv CutMix 75.50% 42.66% 42.24% 0.9543 84.21% 90.90% 76.03% 0.6935
54 STG Std CutMix 79.40% 39.74% 44.36% 0.9596 86.70% 92.42% 79.97% 0.7408
55 TabTransformer Std CT-GAN 94.20% 25.28% 87.98% 0.9761 93.31% 92.67% 94.05% 0.8662
56 RLN Std CT-GAN 94.20% 21.94% 85.50% 0.9803 93.92% 93.80% 94.05% 0.8784
57 RLN Std WGAN 93.50% 21.44% 91.12% 0.9824 94.01% 94.51% 93.44% 0.8802
58 TabTransformer Std WGAN 93.60% 20.02% 88.68% 0.9804 93.44% 93.44% 93.44% 0.8688
59 TabTransformer Std TVAE 95.20% 16.84% 90.10% 0.9783 93.66% 92.50% 95.01% 0.8735
60 TabNet Std None 93.40% 11.04% 29.86% 0.9862 94.58% 95.37% 93.70% 0.8917
61 RLN Std None 94.40% 10.84% 90.14% 0.9843 94.53% 94.49% 94.58% 0.8906
62 RLN Std CutMix 94.10% 8.62% 92.56% 0.9834 94.36% 94.47% 94.23% 0.8871
63 TabTransformer Std None 93.60% 8.92% 82.50% 0.9809 94.01% 94.28% 93.70% 0.8802
64 RLN Std TVAE 94.20% 8.14% 91.24% 0.9825 94.14% 93.91% 94.40% 0.8828
65 TabTransformer Std GOGGLE 93.00% 4.90% 5.10% 0.9737 93.09% 93.16% 93.00% 0.8618
66 RLN Std GOGGLE 93.60% 3.88% 3.88% 0.9776 93.83% 93.72% 93.96% 0.8766
67 RLN Std TableGAN 91.00% 3.90% 3.90% 0.9802 93.35% 95.25% 91.25% 0.8678
68 TabTransformer Std CutMix 90.40% 1.82% 68.70% 0.9675 93.04% 95.39% 90.46% 0.862
69 TabTransformer Std TableGAN 89.90% 2.00% 2.00% 0.9754 92.83% 95.45% 89.94% 0.8579
70 TabNet Adv TableGAN 0.80% 0.00% 0.00% 0.9291 50.39% 100.00% 0.79% 0.0629

WIDS

^ back to top

Rank Architecture Training Augmentation ID ADV+CTR ADV AUC Accuracy Precision Recall MCC
1 RLN Adv GOGGLE 78.28% 69.14% 77.44% 0.8575 77.70% 25.64% 78.21% 0.3581
2 RLN Adv TVAE 77.47% 71.05% 77.21% 0.8543 75.65% 23.72% 77.40% 0.3321
3 TabTransformer Adv GOGGLE 76.50% 67.52% 75.49% 0.8534 78.42% 26.05% 76.44% 0.3573
4 TabTransformer Adv TVAE 74.72% 66.71% 74.30% 0.8578 79.04% 26.39% 74.68% 0.3555
5 RLN Adv TableGAN 76.01% 66.06% 75.43% 0.8605 79.69% 27.29% 75.96% 0.3698
6 RLN Adv None 77.96% 66.61% 77.31% 0.8667 78.90% 26.76% 77.88% 0.3701
7 TabTransformer Adv CT-GAN 78.12% 68.07% 77.63% 0.8592 76.90% 24.91% 78.21% 0.3493
8 RLN Adv WGAN 77.63% 67.62% 77.60% 0.8583 77.00% 24.90% 77.56% 0.3471
9 TabTransformer Adv None 77.31% 65.06% 76.66% 0.8694 79.44% 27.23% 77.24% 0.3733
10 TabTransformer Adv WGAN 76.50% 65.19% 75.88% 0.8638 78.75% 26.38% 76.44% 0.3611
11 RLN Adv CutMix 68.07% 59.87% 67.52% 0.8586 83.37% 30.73% 68.11% 0.3793
12 TabTransformer Adv TableGAN 72.61% 62.20% 72.41% 0.8576 80.58% 27.71% 72.44% 0.363
13 STG Std None 77.63% 63.79% 77.31% 0.8658 78.23% 26.05% 77.56% 0.3608
14 STG Adv CT-GAN 85.25% 73.78% 85.25% 0.8412 66.79% 19.32% 85.10% 0.2928
15 STG Adv TVAE 67.75% 59.81% 67.68% 0.8451 80.74% 27.08% 67.79% 0.341
16 VIME Adv TVAE 72.12% 61.78% 72.12% 0.8461 78.74% 25.64% 72.12% 0.3389
17 RLN Std None 77.47% 60.94% 77.15% 0.8695 79.56% 27.38% 77.40% 0.3755
18 STG Std TVAE 75.85% 62.11% 75.53% 0.8573 77.60% 25.16% 75.80% 0.3448
19 RLN Std TVAE 80.23% 62.11% 79.58% 0.8676 77.59% 25.86% 80.29% 0.3673
20 VIME Adv WGAN 71.47% 60.62% 71.47% 0.8451 79.07% 25.87% 71.47% 0.3395
21 STG Adv TableGAN 68.88% 56.60% 68.85% 0.8531 82.94% 30.19% 68.75% 0.3762
22 VIME Adv GOGGLE 70.18% 59.19% 69.89% 0.8403 80.18% 26.83% 70.03% 0.3455
23 RLN Std CutMix 76.99% 58.70% 76.66% 0.8698 80.16% 27.97% 76.92% 0.3805
24 STG Adv GOGGLE 78.77% 66.00% 78.77% 0.8368 72.70% 21.75% 78.69% 0.31
25 STG Std WGAN 74.55% 58.35% 74.39% 0.8634 80.00% 27.37% 74.36% 0.3656
26 TabTransformer Adv CutMix 59.97% 50.76% 59.94% 0.8508 86.75% 35.78% 60.10% 0.3952
27 RLN Std TableGAN 78.77% 58.93% 78.61% 0.8683 77.34% 25.36% 78.53% 0.3557
28 RLN Std WGAN 77.47% 57.37% 77.47% 0.8656 78.16% 25.95% 77.40% 0.3592
29 STG Std GOGGLE 74.23% 57.24% 73.91% 0.8512 78.05% 25.31% 74.20% 0.3416
30 STG Std CT-GAN 87.84% 71.22% 87.68% 0.852 63.83% 18.34% 87.82% 0.2845
31 STG Std TableGAN 67.75% 49.82% 67.10% 0.861 84.56% 32.61% 67.63% 0.3955
32 VIME Adv None 72.12% 52.09% 72.12% 0.8581 81.74% 29.08% 71.96% 0.3759
33 VIME Std TableGAN 71.80% 50.15% 71.80% 0.8641 82.79% 30.47% 71.63% 0.389
34 STG Adv None 62.56% 45.19% 62.56% 0.8645 87.54% 38.15% 62.66% 0.4244
35 VIME Std None 72.29% 50.34% 71.31% 0.8652 82.29% 29.84% 72.12% 0.3842
36 RLN Std GOGGLE 77.31% 52.54% 75.04% 0.8638 79.74% 27.58% 77.40% 0.3778
37 STG Adv WGAN 62.56% 46.39% 62.33% 0.8548 85.48% 33.45% 62.50% 0.3844
38 VIME Std TVAE 72.61% 50.57% 72.61% 0.8582 80.84% 28.05% 72.60% 0.3671
39 VIME Adv CutMix 54.30% 43.50% 53.52% 0.8486 87.75% 37.42% 54.33% 0.3852
40 VIME Std CutMix 69.85% 47.65% 69.37% 0.8572 83.30% 30.89% 69.71% 0.3865
41 VIME Std WGAN 75.53% 51.22% 75.36% 0.861 79.56% 27.04% 75.32% 0.365
42 VIME Adv TableGAN 55.27% 42.27% 55.27% 0.8528 88.15% 38.76% 55.29% 0.3994
43 VIME Std GOGGLE 70.18% 49.14% 69.69% 0.8492 81.15% 27.99% 70.03% 0.3581
44 TabTransformer Std TVAE 77.63% 49.30% 76.27% 0.871 80.12% 28.03% 77.56% 0.3832
45 STG Adv CutMix 53.16% 41.17% 52.32% 0.8512 88.04% 38.05% 53.04% 0.3847
46 STG Std CutMix 56.73% 38.54% 55.92% 0.8626 88.47% 39.95% 56.73% 0.4142
47 TabTransformer Std CutMix 70.99% 43.44% 70.50% 0.8662 83.51% 31.41% 70.83% 0.3955
48 TabTransformer Std None 75.53% 45.87% 74.55% 0.8738 80.97% 28.68% 75.48% 0.3835
49 RLN Std CT-GAN 76.17% 47.16% 75.85% 0.8623 78.80% 26.37% 76.12% 0.36
50 TabTransformer Std CT-GAN 77.96% 44.05% 77.63% 0.8682 79.90% 27.86% 78.04% 0.383
51 TabTransformer Std GOGGLE 78.61% 38.35% 73.26% 0.8726 80.46% 28.55% 78.37% 0.3916
52 TabTransformer Std TableGAN 75.04% 37.60% 75.04% 0.8688 80.78% 28.35% 74.84% 0.3778
53 TabTransformer Std WGAN 76.34% 37.63% 76.34% 0.8664 79.67% 27.32% 76.28% 0.3713
54 TabNet Adv GOGGLE 100.00% 100.00% 100.00% 0.8618 8.97% 8.97% 100.00% 0
55 VIME Std CT-GAN 100.00% 100.00% 100.00% 0.4821 8.97% 8.97% 100.00% 0
56 TabNet Std GOGGLE 100.00% 100.00% 100.00% 0.8562 8.97% 8.97% 100.00% 0
57 VIME Adv CT-GAN 100.00% 100.00% 100.00% 0.4816 8.97% 8.97% 100.00% 0
58 TabNet Adv TVAE 100.00% 100.00% 100.00% 0.8636 8.97% 8.97% 100.00% 0
59 TabNet Adv TableGAN 100.00% 100.00% 100.00% 0.8599 8.97% 8.97% 100.00% 0
60 TabNet Std TableGAN 100.00% 100.00% 100.00% 0.8643 8.97% 8.97% 100.00% 0
61 TabNet Std CT-GAN 100.00% 100.00% 100.00% 0.8525 8.97% 8.97% 100.00% 0
62 RLN Adv CT-GAN 100.00% 100.00% 100.00% 0.4246 8.97% 8.97% 100.00% 0
63 TabNet Adv CT-GAN 100.00% 100.00% 100.00% 0.8627 8.97% 8.97% 100.00% 0
64 TabNet Adv WGAN 100.00% 99.16% 99.64% 0.8554 8.97% 8.97% 100.00% 0
65 TabNet Std CutMix 0.00% 0.00% 0.00% 0.8658 91.03% 0.00% 0.00% 0
66 TabNet Std None 79.74% 5.28% 73.06% 0.8704 77.69% 25.86% 79.65% 0.3652
67 TabNet Std WGAN 78.61% 0.00% 45.58% 0.865 79.47% 27.49% 78.69% 0.3809
68 TabNet Adv None 98.38% 58.38% 82.50% 0.8348 10.41% 8.98% 98.40% 0.003
69 TabNet Std TVAE 98.38% 40.62% 47.49% 0.8569 10.41% 8.98% 98.40% 0.003
70 TabNet Adv CutMix 100.00% 37.44% 67.07% 0.8587 8.97% 8.97% 100.00% 0
This site is open source. Improve this page.