ASU

Event Calculus Answer Set Programming

Benchmark Problems

 

We have compared the performance of different systems on 14 benchmark commonsense reasoning problems.

 

*Commuter includes events with duration and EC_dur.lp has to be used instead of EC.lp [DEC.lp].

 

**ThielscherCircuit includes causal constraints and ECCausal.lp has to be used with EC.lp [DEC.lp].

   We notice that adding 'option renaming off'  to the event calculus description reduces the numbers of variables and clauses, and
   time spent when executed using the DEC reasoner.

 

 

Click other problems to see the results.

 

<A: number of atoms, C: number of clauses, R: number of ground rules>

 

Problem (max time)

DEC Reasoner

ECASP with

ECASP with

ECASP with

LPARSE + CMODELS

GRINGO + CLASP(D)

CLINGO

BusRide (2)

Can't handle

0.04

0.02

Can't handle

BusRide2-ea.e

Disjunctive

(0.03+0.01)

(0.02+0.00)

Disjunctive

BusRide2-ea.lp

Events Axioms

A:29/R:216/C:31

A:83/R:115

Rules

BusRide (40)

Can't handle

7.84

0.48

Can't handle

BusRide40-ea.e

Disjunctive

(7.12+0.72)

(0.39+0.09)

Disjunctive

BusRide40-ea.lp

Events Axioms

A:709/R:130973/C:714

A:3408/R:51705

Rules

ChessBoard (2)

0.00

0.03

0.02

Can't handle

 

Disjunctive

 

Rules

ChessBoard2.e

(0.00+0.00)

(0.02+0.01)

(0.01+0.01)

ChessBoard2-ea.e

A:27/C:52

A:38/R:151/C:55

A:65/R:84

ChessBoard2-ea.lp

ChessBoard (40)

0.10

0.16

0.05

Can't handle

 

Disjunctive

 

Rules

ChessBoard40.e

(0.10+0.00)

(0.12+0.04)

(0.03+0.02)

ChessBoard40-ea.e

A:369/C:812

A:736/R:2595/C:1391

A:1483/R:1730

ChessBoard40-ea.lp

CoinToss (4)

0.00

0.03

0.02

0.01

CoinToss4.e

(0.00+0.00)

(0.02+0.01)

(0.01+0.01)

CoinToss4-ea.e

A:25/C:66

A:58/R:195/C:109

A:82/R:119

CoinToss4-ea.lp

CoinToss (40)

0.00

0.39

0.06

0.04

CoinToss40.e

(0.00+0.00)

(0.22+0.17)

(0.05+0.01)

CoinToss40-ea.e

A:205/C:606

A:1462/R:5019/C:3646

A:2782/R:4476

CoinToss40-ea.lp

*Commuter (15)

Can't handle

493.55

44.67

28.91

Commuter15-ea.e

Compound

(446.34+47.21)

(37.92+6.75)

Commuter15-ea.lp

Events

A:4913/R:7383943/C:4952

A:24698/R:5381620

DeadOrAlive (3)

0.00

0.03

0.02

0.02

DeadOrAlive3.e

(0.00+0.00)

(0.02+0.01)

(0.01+0.01)

DeadOrAlive3-ea.e

A:36/C:76

A:125/R:188/C:0

A:70/R:98

DeadOrAlive3-ea.lp

DeadOrAlive (40)

0.30

0.22

0.06

0.05

DeadOrAlive40.e

(0.20+0.10)

(0.19+0.03)

(0.04+0.02)

DeadOrAlive40-ea.e

A:369/C:890

A:3066/R:4470/C:0

A:2533/R:3652

DeadOrAlive40-ea.lp

Happy (2)

0.00

0.09

0.02

0.02

Happy2.e

(0.00+0.00)

(0.03+0.06)

(0.02+0.00)

Happy2-ea.e

A:24/C:46

A:910/R:1557/C:0

A:38/R:63

Happy2-ea.lp

Happy (40)

0.10

0.10

0.04

0.04

Happy40.e

(0.10+0.00)

(0.09+0.01)

(0.03+0.01)

Happy40-ea.e

A:328/C:730

A:1246/R:1893/C:0

A:754/R:1197

Happy40-ea.lp

KitchenSink (5)

0.20

0.33

0.05

0.04

KitchenSink_M5.e

(0.20+0.00)

(0.30+0.03)

(0.04+0.01)

KitchenSink_M5-ea.e

A:102/C:593

A:1512/R:3816/C:0

A:1158/R:1828

KitchenSink_M5-ea.lp

KitchenSink (25)

71.10

43.40

2.47

2.04

KitchenSink_M25.e

(70.70+0.40)

(37.30+6.1)

(1.73+0.74)

KitchenSink_M25-ea.e

A:1014/C:12109

A:123452/R:482018/C:0

A:114968/R:179195

KitchenSink_M25-ea.lp

KitchenSink (40)

Ran over 1 hour

268.53

15.04

12.81

KitchenSink_M40-ea.e

and

(224.62+43.91)

(10.28+4.76)

KitchenSink_M40-ea.lp

didn't terminate

A:705747/R:2912813/C:0

A:676343/R:1042040

RussianTurkey (4)

0.00

0.03

0.02

0.01

RussianTurkey4.e

(0.00+0.00)

(0.02+0.01)

(0.01+0.01)

RussianTurkey4-ea.e

A:35/C:87

A:55/R:172/C:64

A:82/R:110

RussianTurkey4-ea.lp

RussianTurkey (40)

0.10

0.20

0.04

0.04

RussianTurkey40.e

(0.10+0.00)

(0.14+0.06)

(0.04+0.00)

RussianTurkey40-ea.e

A:287/C:807

A:1171/R:3349/C:1513

A:1963/R:2800

RussianTurkey40-ea.lp

StolenCar (2)

0.00

0.03

0.02

0.01

StolenCar2.e

(0.00+0.00)

(0.02+0.01)

(0.01+0.01)

StolenCar2-ea.e

A:10/C:20

A:55/R:172/C:64

A:82/R:110

StolenCar2-ea.lp

StolenCar (40)

0.20

1.63

0.15

0.1

StolenCar40.e

(0.20+0.00)

(0.91+0.72)

(0.12+0.03)

StolenCar40-ea.e

A:162/C:1144

A:1651/R:23200/C:9854

A:1930/R:19027

StolenCar40-ea.lp

StuffyRoom (2)

0.00

0.03

0.02

0.02

StuffyRoom2.e

(0.00+0.00)

(0.02+0.01)

(0.02+0.00)

StuffyRoom2-ea.e

A:27/C:57

A:83/R:122/C:0

A:47/R:68

StuffyRoom2-ea.lp

StuffyRoom (40)

0.20

0.11

0.04

0.04

StuffyRoom40.e

(0.10+0.10)

(0.10+0.01)

(0.04+0.00)

StuffyRoom40-ea.e

A:369/C:931

A:1956/R:2603/C:0

A:1464/R:1902

StuffyRoom40-ea.lp

Supermarket (12)

0.00

0.05

0.03

0.03

Supermarket12.e

(0.00+0.00)

(0.05+0.00)

(0.03+0.00)

Supermarket12-ea.e

A:104/C:752

A:471/R:620/C:0

A:315/R:458

Supermarket12-ea.lp

Supermarket (40)

0.10

0.14

0.05

0.05

Supermarket40.e

(0.10+0.00)

(0.12+0.02)

(0.04+0.01)

Supermarket40-ea.e

A:328/C:2488

A:2461/R:2946/C:0

A:1969/R:2448

Supermarket40-ea.lp

**ThielscherCircuit (1)

0.20

0.05

0.03

0.03

ThielscherCircuit1.e

(0.20+0.00)

(0.05+0.00)

(0.03+0.00)

ThielscherCircuit1-ea.e

A:68/C:1114

A:346/R:569/C:0

A:108/R:140

ThielscherCircuit1-ea.lp

**ThielscherCircuit (20)

5.30

0.55

0.09

0.07

ThielscherCircuit20.e

(5.00+0.30)

(0.49+0.06)

(0.07+0.02)

ThielscherCircuit20-ea.e

A:714/C:21254

A:4336/R:11399/C:0

A:1743/R:5669

ThielscherCircuit20-ea.lp

**ThielscherCircuit (40)

11.10

2.58

0.33

0.22

ThielscherCircuit40.e

(10.50+0.60)

(2.28+0.30)

(0.24+0.09)

ThielscherCircuit40-ea.e

A:1394/C:42454

A:10096/R:54519/C:0

A:4812/R:35458

ThielscherCircuit40-ea.lp

WalkingTurkey (3)

Can't handle

0.03

0.02

0.02

WalkingTurkey3-ea.e

Effect

(0.02+0.01)

(0.01+0.01)

WalkingTurkey3-ea.lp

Constraints

A:112/R:150/C:0

A:64/R:96

WalkingTurkey (40)

Can't handle

0.13

0.04

0.03

WalkingTurkey40-ea.e

Effect

(0.12+0.01)

(0.04+0.00)

WalkingTurkey40-ea.lp

Constraints

A:2156/R:2527/C:0

A:1664/R:2029

Yale (3)

0.00

0.02

0.01

0.01

Yale3.e

(0.00+0.00)

(0.02+0.00)

(0.01+0.00)

Yale3-ea.e

A:28/C:64

A:91/R:127/C:0

A:56/R:78

Yale3-ea.lp

Yale (40)

0.00

0.15

0.04

0.03

Yale40.e

(0.00+0.00)

(0.13+0.02)

(0.03+0.01)

Yale40-ea.e

A:287/C:767

A:2201/R:3057/C:0

A:1872/R:2613

Yale40-ea.lp