ASU |
Event Calculus Answer Set Programming |
Benchmark Problems We have compared the performance of different systems on 14 benchmark commonsense reasoning problems. |
*Commuter includes events with duration and EC_dur.lp has to be used instead of EC.lp [DEC.lp]. **ThielscherCircuit includes causal constraints and ECCausal.lp has to be used with EC.lp [DEC.lp]. We notice that adding 'option renaming off' to the event calculus description reduces the numbers of variables and clauses, and Click other problems to see the results. |
<A: number of atoms, C: number of clauses, R: number of ground rules> |
||||
Problem (max time) |
DEC Reasoner |
ECASP with |
ECASP with |
ECASP with |
LPARSE + CMODELS |
GRINGO + CLASP(D) |
CLINGO |
||
BusRide (2) |
Can't handle |
0.04 |
0.02 |
Can't handle |
Disjunctive |
(0.03+0.01) |
(0.02+0.00) |
Disjunctive |
|
Events Axioms |
A:29/R:216/C:31 |
A:83/R:115 |
Rules |
|
BusRide (40) |
Can't handle |
7.84 |
0.48 |
Can't handle |
Disjunctive |
(7.12+0.72) |
(0.39+0.09) |
Disjunctive |
|
Events Axioms |
A:709/R:130973/C:714 |
A:3408/R:51705 |
Rules |
|
ChessBoard (2) |
0.00 |
0.03 |
0.02 |
Can't handle Disjunctive Rules |
(0.00+0.00) |
(0.02+0.01) |
(0.01+0.01) |
||
A:27/C:52 |
A:38/R:151/C:55 |
A:65/R:84 |
||
ChessBoard (40) |
0.10 |
0.16 |
0.05 |
Can't handle Disjunctive Rules |
(0.10+0.00) |
(0.12+0.04) |
(0.03+0.02) |
||
A:369/C:812 |
A:736/R:2595/C:1391 |
A:1483/R:1730 |
||
CoinToss (4) |
0.00 |
0.03 |
0.02 |
0.01 |
(0.00+0.00) |
(0.02+0.01) |
(0.01+0.01) |
||
A:25/C:66 |
A:58/R:195/C:109 |
A:82/R:119 |
||
CoinToss (40) |
0.00 |
0.39 |
0.06 |
0.04 |
(0.00+0.00) |
(0.22+0.17) |
(0.05+0.01) |
||
A:205/C:606 |
A:1462/R:5019/C:3646 |
A:2782/R:4476 |
||
*Commuter (15) |
Can't handle |
493.55 |
44.67 |
28.91 |
Compound |
(446.34+47.21) |
(37.92+6.75) |
||
Events |
A:4913/R:7383943/C:4952 |
A:24698/R:5381620 |
||
DeadOrAlive (3) |
0.00 |
0.03 |
0.02 |
0.02 |
(0.00+0.00) |
(0.02+0.01) |
(0.01+0.01) |
||
A:36/C:76 |
A:125/R:188/C:0 |
A:70/R:98 |
||
DeadOrAlive (40) |
0.30 |
0.22 |
0.06 |
0.05 |
(0.20+0.10) |
(0.19+0.03) |
(0.04+0.02) |
||
A:369/C:890 |
A:3066/R:4470/C:0 |
A:2533/R:3652 |
||
Happy (2) |
0.00 |
0.09 |
0.02 |
0.02 |
(0.00+0.00) |
(0.03+0.06) |
(0.02+0.00) |
||
A:24/C:46 |
A:910/R:1557/C:0 |
A:38/R:63 |
||
Happy (40) |
0.10 |
0.10 |
0.04 |
0.04 |
(0.10+0.00) |
(0.09+0.01) |
(0.03+0.01) |
||
A:328/C:730 |
A:1246/R:1893/C:0 |
A:754/R:1197 |
||
KitchenSink (5) |
0.20 |
0.33 |
0.05 |
0.04 |
(0.20+0.00) |
(0.30+0.03) |
(0.04+0.01) |
||
A:102/C:593 |
A:1512/R:3816/C:0 |
A:1158/R:1828 |
||
KitchenSink (25) |
71.10 |
43.40 |
2.47 |
2.04 |
(70.70+0.40) |
(37.30+6.1) |
(1.73+0.74) |
||
A:1014/C:12109 |
A:123452/R:482018/C:0 |
A:114968/R:179195 |
||
KitchenSink (40) |
Ran over 1 hour |
268.53 |
15.04 |
12.81 |
and |
(224.62+43.91) |
(10.28+4.76) |
||
didn't terminate |
A:705747/R:2912813/C:0 |
A:676343/R:1042040 |
||
RussianTurkey (4) |
0.00 |
0.03 |
0.02 |
0.01 |
(0.00+0.00) |
(0.02+0.01) |
(0.01+0.01) |
||
A:35/C:87 |
A:55/R:172/C:64 |
A:82/R:110 |
||
RussianTurkey (40) |
0.10 |
0.20 |
0.04 |
0.04 |
(0.10+0.00) |
(0.14+0.06) |
(0.04+0.00) |
||
A:287/C:807 |
A:1171/R:3349/C:1513 |
A:1963/R:2800 |
||
StolenCar (2) |
0.00 |
0.03 |
0.02 |
0.01 |
(0.00+0.00) |
(0.02+0.01) |
(0.01+0.01) |
||
A:10/C:20 |
A:55/R:172/C:64 |
A:82/R:110 |
||
StolenCar (40) |
0.20 |
1.63 |
0.15 |
0.1 |
(0.20+0.00) |
(0.91+0.72) |
(0.12+0.03) |
||
A:162/C:1144 |
A:1651/R:23200/C:9854 |
A:1930/R:19027 |
||
StuffyRoom (2) |
0.00 |
0.03 |
0.02 |
0.02 |
(0.00+0.00) |
(0.02+0.01) |
(0.02+0.00) |
||
A:27/C:57 |
A:83/R:122/C:0 |
A:47/R:68 |
||
StuffyRoom (40) |
0.20 |
0.11 |
0.04 |
0.04 |
(0.10+0.10) |
(0.10+0.01) |
(0.04+0.00) |
||
A:369/C:931 |
A:1956/R:2603/C:0 |
A:1464/R:1902 |
||
Supermarket (12) |
0.00 |
0.05 |
0.03 |
0.03 |
(0.00+0.00) |
(0.05+0.00) |
(0.03+0.00) |
||
A:104/C:752 |
A:471/R:620/C:0 |
A:315/R:458 |
||
Supermarket (40) |
0.10 |
0.14 |
0.05 |
0.05 |
(0.10+0.00) |
(0.12+0.02) |
(0.04+0.01) |
||
A:328/C:2488 |
A:2461/R:2946/C:0 |
A:1969/R:2448 |
||
**ThielscherCircuit (1) |
0.20 |
0.05 |
0.03 |
0.03 |
(0.20+0.00) |
(0.05+0.00) |
(0.03+0.00) |
||
A:68/C:1114 |
A:346/R:569/C:0 |
A:108/R:140 |
||
**ThielscherCircuit (20) |
5.30 |
0.55 |
0.09 |
0.07 |
(5.00+0.30) |
(0.49+0.06) |
(0.07+0.02) |
||
A:714/C:21254 |
A:4336/R:11399/C:0 |
A:1743/R:5669 |
||
**ThielscherCircuit (40) |
11.10 |
2.58 |
0.33 |
0.22 |
(10.50+0.60) |
(2.28+0.30) |
(0.24+0.09) |
||
A:1394/C:42454 |
A:10096/R:54519/C:0 |
A:4812/R:35458 |
||
WalkingTurkey (3) |
Can't handle |
0.03 |
0.02 |
0.02 |
Effect |
(0.02+0.01) |
(0.01+0.01) |
||
Constraints |
A:112/R:150/C:0 |
A:64/R:96 |
||
WalkingTurkey (40) |
Can't handle |
0.13 |
0.04 |
0.03 |
Effect |
(0.12+0.01) |
(0.04+0.00) |
||
Constraints |
A:2156/R:2527/C:0 |
A:1664/R:2029 |
||
Yale (3) |
0.00 |
0.02 |
0.01 |
0.01 |
(0.00+0.00) |
(0.02+0.00) |
(0.01+0.00) |
||
A:28/C:64 |
A:91/R:127/C:0 |
A:56/R:78 |
||
Yale (40) |
0.00 |
0.15 |
0.04 |
0.03 |
(0.00+0.00) |
(0.13+0.02) |
(0.03+0.01) |
||
A:287/C:767 |
A:2201/R:3057/C:0 |
A:1872/R:2613 |
||