TabularBench: Adversarial robustness benchmark for tabular data.
Documentation .
MOEVA attack
You are currently viewing results for MOEVA attack. View leaderboard .
Among models that demonstrate strong robustness to constrained adversarial attacks (high ADV+CTR), we observe that some achieve this robustness solely by consistently predicting the “1” class.
This behavior is evident in their poor accuracy and precision.
Therefore, we rank models based on their average performance across clean accuracy (Accuracy) and constrained adversarial accuracy (ADV+CTR).
Jump to dataset:
CTU
^ back to top
Rank
Architecture
Training
Augmentation
ID
ADV+CTR
AUC
Accuracy
Precision
Recall
MCC
1
RLN
Adv
None
97.30%
97.05%
0.9898
99.90%
90.41%
97.30%
0.9374
2
STG
Std
None
95.33%
95.33%
0.9884
99.95%
98.23%
95.33%
0.9675
3
TabTransformer
Adv
None
95.33%
95.33%
0.9846
99.95%
98.23%
95.33%
0.9675
4
TabTransformer
Std
None
95.33%
95.33%
0.9788
99.95%
98.23%
95.33%
0.9675
5
STG
Adv
None
95.09%
95.09%
0.9865
99.96%
99.23%
95.09%
0.9712
6
VIME
Adv
None
95.09%
94.00%
0.9825
99.96%
99.74%
95.09%
0.9737
7
RLN
Std
None
97.79%
94.00%
0.9906
99.82%
81.89%
97.79%
0.8941
8
VIME
Std
None
95.09%
40.84%
0.9873
99.96%
99.74%
95.09%
0.9737
9
TabNet
Std
None
96.07%
0.00%
0.9963
99.94%
95.83%
96.07%
0.9592
10
TabNet
Adv
None
0.25%
0.20%
0.9781
99.26%
50.00%
0.25%
0.0347
LCLD
^ back to top
Rank
Architecture
Training
Augmentation
ID
ADV+CTR
AUC
Accuracy
Precision
Recall
MCC
1
TabTransformer
Adv
None
73.90%
71.40%
0.7111
59.01%
29.33%
73.83%
0.2333
2
RLN
Adv
None
69.50%
64.86%
0.7158
62.75%
30.93%
69.28%
0.2448
3
STG
Std
None
66.40%
55.42%
0.7087
64.56%
31.67%
65.99%
0.2452
4
STG
Adv
None
15.60%
15.52%
0.6788
78.82%
43.19%
17.18%
0.1699
5
VIME
Std
None
67.00%
24.06%
0.7142
64.46%
31.79%
67.13%
0.2506
6
VIME
Adv
None
65.50%
18.14%
0.7127
65.13%
32.05%
65.71%
0.2499
7
TabNet
Adv
None
0.00%
0.10%
0.656
79.91%
0.00%
0.00%
0
8
TabTransformer
Std
None
69.50%
10.72%
0.7172
63.32%
31.42%
69.87%
0.2542
9
TabNet
Std
None
67.40%
0.82%
0.7224
65.60%
32.62%
66.83%
0.2615
10
RLN
Std
None
68.30%
0.80%
0.7187
64.13%
31.78%
68.51%
0.255
URL
^ back to top
Rank
Architecture
Training
Augmentation
ID
ADV+CTR
AUC
Accuracy
Precision
Recall
MCC
1
STG
Adv
None
94.30%
89.96%
0.9485
86.22%
81.17%
94.31%
0.7341
2
VIME
Adv
None
93.40%
70.04%
0.973
92.48%
91.75%
93.35%
0.8496
3
TabNet
Adv
None
99.50%
91.92%
0.9472
70.03%
62.62%
99.39%
0.495
4
STG
Std
None
93.30%
58.16%
0.9729
91.95%
90.81%
93.35%
0.8393
5
TabTransformer
Adv
None
93.90%
56.78%
0.9743
93.09%
92.27%
94.05%
0.8619
6
RLN
Adv
None
95.20%
56.26%
0.9768
93.31%
91.67%
95.28%
0.8668
7
VIME
Std
None
92.50%
56.50%
0.9736
92.78%
92.89%
92.65%
0.8556
8
RLN
Std
None
94.40%
23.60%
0.9843
94.53%
94.49%
94.58%
0.8906
9
TabTransformer
Std
None
93.60%
18.18%
0.9809
94.01%
94.28%
93.70%
0.8802
10
TabNet
Std
None
93.40%
17.54%
0.9862
94.58%
95.37%
93.70%
0.8917
WIDS
^ back to top
Rank
Architecture
Training
Augmentation
ID
ADV+CTR
AUC
Accuracy
Precision
Recall
MCC
1
TabTransformer
Adv
None
77.31%
69.63%
0.8694
79.44%
27.23%
77.24%
0.3733
2
RLN
Adv
None
77.96%
68.78%
0.8667
78.90%
26.76%
77.88%
0.3701
3
RLN
Std
None
77.47%
67.68%
0.8695
79.56%
27.38%
77.40%
0.3755
4
STG
Std
None
77.63%
68.82%
0.8658
78.23%
26.05%
77.56%
0.3608
5
VIME
Std
None
72.29%
59.42%
0.8652
82.29%
29.84%
72.12%
0.3842
6
VIME
Adv
None
72.12%
59.25%
0.8581
81.74%
29.08%
71.96%
0.3759
7
TabTransformer
Std
None
75.53%
59.16%
0.8738
80.97%
28.68%
75.48%
0.3835
8
STG
Adv
None
62.56%
49.27%
0.8645
87.54%
38.15%
62.66%
0.4244
9
TabNet
Adv
None
98.38%
87.49%
0.8348
10.41%
8.98%
98.40%
0.003
10
TabNet
Std
None
79.74%
13.87%
0.8704
77.69%
25.86%
79.65%
0.3652