Did you know ... Search Documentation:
Packs (add-ons) for SWI-Prolog

Package "sparqlprog"

Title:Logic programming with SPARQL
Rating:Not rated. Create the first rating!
Latest version:0.2.9
SHA1 sum:f514581d938fd1d4e3a44608f6739284214a04d8
Author:Chris Mungall <cjmungall@lbl.gov>
Samer Abdallah <s.abdallah@ucl.ac.uk>
Maintainer:Chris Mungall <cjmungall@lbl.gov>
Packager:Chris Mungall <cjmungall@lbl.gov>
Download URL:https://github.com/cmungall/sparqlprog/releases/*.zip
Requires:dcgutils
regex
typedef

Reviews

No reviews. Create the first review!.

Details by download location

VersionSHA1#DownloadsURL
0.0.21c49dfeb059f83ed07b9c8945ed2cdc4e5cc9e4b1https://github.com/cmungall/sparqlprog/archive/v0.0.2.zip
0.0.391bbd7b9c31c3ebe13f9fdb1da191f53e0fe0a1b1https://github.com/cmungall/sparqlprog/archive/v0.0.3.zip
0.0.6c80373e23db1ccb75248276a17dd4b71a52e3a0d1https://github.com/cmungall/sparqlprog/archive/v0.0.6.zip
0.0.77990e13230ff7d3489a2df8ac4b36c1ce6a9ddaa1https://github.com/cmungall/sparqlprog/archive/v0.0.7.zip
0.0.8adf96236fc76e434b82fa8b02af994d051b675581https://github.com/cmungall/sparqlprog/archive/v0.0.8.zip
0.0.9bb1fd0c143173ee186b4054c7a05a8f10ef644f32https://github.com/cmungall/sparqlprog/archive/v0.0.9.zip
0.1.0d9c1df98279f37d31a72cba8a9f7aaeb71e9afef3https://github.com/cmungall/sparqlprog/archive/v0.1.0.zip
0.1.1a6e2df8b99b5fb440435b124460203aba65046461https://github.com/cmungall/sparqlprog/archive/v0.1.1.zip
0.1.2f5e920eda2db78861cbbd508c6ca601c04ec8f3f4https://github.com/cmungall/sparqlprog/archive/v0.1.2.zip
0.1.34c044583dbec3665c0cf9261f3470e8281389b3717https://github.com/cmungall/sparqlprog/archive/v0.1.3.zip
0.2.0189828a96597a560d40cec856d0901608c6512532https://github.com/cmungall/sparqlprog/archive/v0.2.0.zip
0.2.12a09021ab9de25733e33a24cc42cd5383f15f3f23https://github.com/cmungall/sparqlprog/archive/v0.2.1.zip
0.2.280b8bea8378d5be6be28664e84d378729734ab013https://github.com/cmungall/sparqlprog/archive/v0.2.2.zip
0.2.39efc96d497da156ea292827dc3df2b7e3675e73f10https://github.com/cmungall/sparqlprog/archive/v0.2.3.zip
0.2.4daa67b3b4b7b3fda95c2f48fe4ee5d6d5711efaf4https://github.com/cmungall/sparqlprog/archive/v0.2.4.zip
0.2.577db7c6fc5180132609d5fcd8983e92fdd69b76610https://github.com/cmungall/sparqlprog/archive/v0.2.5.zip
0.2.8d892005c6cb22eec94b1fff3fbf1402d6ff4ab112https://github.com/cmungall/sparqlprog/archive/v0.2.8.zip
0.2.92976ed06fdc0b8e806db3c3907fa81a3e695d07f205https://github.com/cmungall/sparqlprog.git
558d058065d82be4dc8a5f3bc08f40c84565d98a9https://github.com/cmungall/sparqlprog.git
ccd647204e4fd55fe5c9707a6c2905739bb98f481529https://github.com/cmungall/sparqlprog.git
ed9cf5645e5efabe311984de318b204d753392a563https://github.com/cmungall/sparqlprog/archive/v0.2.9.zip
f514581d938fd1d4e3a44608f6739284214a04d8318https://github.com/cmungall/sparqlprog.git

sparqlprog - programming with SPARQL

![Build Status](https://travis-ci.org/cmungall/sparqlprog) ![Join the chat at https://gitter.im/sparqlprog/Lobby](https://gitter.im/sparqlprog/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) pack

sparqlprog is a programming language and environment that can be used to write composable modular building blocks that can be executed as federated SPARQL queries.

Example of use (command line):

pl2sparql  -u sparqlprog/ontologies/ebi -u sparqlprog/ontologies/faldo  -s ebi "\
  protein_coding_gene(G), \
  location(G,L,B,E,grcm38:'11'), \
  B >= 101100523,E =< 101190725, \
  orthologous_to(G,H),in_taxon(H,taxon:'9606')" \
  "h(G,H)"

The command passes a logic program query to sparqlprog. In this case, the query is a conjunction of conditions involving different variables (each indicated with a leading upper-case letter):

  1. G is a protein coding gene
  2. G is located on mouse chromosome 11, with an interval bounded by B (begin) and E (end)
  3. The interval is within a certain range
  4. G is homologus to H
  5. H is a human gene (indicated by taxon ID 9606)
  6. The results are bound to a tuples h(G,H) (i.e. two column table)

This logic query compiles down to a SPARQL query for fetching G and H. The query is then executed on the EBI RDF Platform, giving:

Mouse GeneHuman Gene
ensembl:ENSMUSG00000035198ensembl:ENSG00000131462
ensembl:ENSMUSG00000017167ensembl:ENSG00000108797
ensembl:ENSMUSG00000044052ensembl:ENSG00000184451
ensembl:ENSMUSG00000017802ensembl:ENSG00000141699
ensembl:ENSMUSG00000045007ensembl:ENSG00000037042
ensembl:ENSMUSG00000035172ensembl:ENSG00000068137

How does this work? The query compilation makes use of pre-defined n-ary predicates, such as this one defined in the faldo module:

location(F,L,B,E,R) :-
  rdf(F,faldo:location,L),
  begin(L,PB),position(PB,B),reference(PB,R),
  end(L,PE),position(PE,E),reference(PE,R).

The :- connects a rule head to a rule body. In this case the body is a conjuncation of goals. Each of these may be defined in their own rules. Typically everything bottoms out at a call over a 3-ary predicate rdf(S,P,O) which maps to a single triple. In this case the vocabulary used for genomic locations is faldo.

This approach allows for composability of queries. Rather that repeating the same verbose SPARQL each time in different queries, reusable modules can be defined.

In addition to providing a composable language that compiles to SPARQL, this package provides a complete turing-complete environment for mixing code and queries in a relational/logic programming paradigm. See below for examples.

Quick Start (for prolog hackers)

See the sparqlprog module docs

See also the specification

Quick Start (for Python hackers)

See the sparqlprog-python package

This provides a Python interface to a sparqlprog service

You can also see demonstration notebooks:

Quick Start (for everyone else)

There are a variety of ways to use this framework:

  • Executing queries on remote services via command line
  • Compiling logic queries to SPARQL queries, for use in another framework
  • Programmatically within a logic program (interleaving remote and local operations)
  • Programmatically from a language like python/javascript, using a sparqlprog service

Consult the appropriate section below:

Running queries from the command line

See the [examples](./examples/) directory for all command line examples

First install, making sure the [bin](bin) directory is in your path. This will give you access to the the pl2sparql script.

For full options, run:

pl2sparql --help

Note you should also have a number of convenience scripts in your path. For example the pq-wd script is simply a shortcut for

pl2sparql -s wikidata -u sparqlprog/ontologies/wikidata  ARGS

This will give you access to a number of convenience predicates such as positive_therapeutic_predictor/2 (for drug queries). The -u option uses the wikidata module, and the -s option sets the service to the one with handle dbpedia (the mapping from a handle to the full service URL is defined in the wikidata module).

The best way to learn is to look at the [examples/](examples), together with the corresponding set of rules in [prolog/sparqlprog/ontologies](prolog/sparqlprog/ontologies).

For example [examples/monarch-examples.sh](examples/monarch-examples.sh) has:

pq-mi  'label(D,DN),literal_exact_match(DN,"peroxisome biogenesis disorder"),\
   rdfs_subclass_of(D,C),owl_equivalent_class(C,E),has_phenotype(E,Z)'\
   'x(C,CN,E,Z)'

This finds a disease with a given name, finds equivalent classes of transitive reflexive subclasses, and then finds phenotypes for each

Compiling logic programs to SPARQL

You can use pl2sparql (see above for installation) to compile a program with bindings to a SPARQL query by using the -C option. The SPARQL query can then be used without any dependence on sparqlprog. E.g.

pq-ebi -C "\
  protein_coding_gene(G), \
  location(G,L,B,E,grcm38:'11'), \
  B >= 101100523,E =< 101190725, \
  homologous_to(G,H),in_taxon(H,taxon:'9606')" \
  "h(G,H)"

will generate the following SPARQL:

SELECT ?g ?h WHERE {?g <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.obolibrary.org/obo/SO_0001217> . ?g <http://biohackathon.org/resource/faldo#location> ?l . ?l <http://biohackathon.org/resource/faldo#begin> ?v0 . ?v0 <http://biohackathon.org/resource/faldo#position> ?b . ?v0 <http://biohackathon.org/resource/faldo#reference> <http://rdf.ebi.ac.uk/resource/ensembl/90/mus_musculus/GRCm38/11> . ?l <http://biohackathon.org/resource/faldo#end> ?v1 . ?v1 <http://biohackathon.org/resource/faldo#position> ?e . ?v1 <http://biohackathon.org/resource/faldo#reference> <http://rdf.ebi.ac.uk/resource/ensembl/90/mus_musculus/GRCm38/11> . FILTER (?b >= 101100523) . FILTER (?e <= 101190725) . ?g <http://semanticscience.org/resource/SIO_000558> ?h . ?h <http://purl.obolibrary.org/obo/RO_0002162> <http://identifiers.org/taxonomy/9606>}

withOUT executing it remotely

note: indentation and URI shortening are on the cards for future releases.

Using a public sparqlprog service

Public pengines service: https://evening-falls-87315.herokuapp.com/pengine

Pengines is a framework for running logic program environments as a web service. They can be used by clients in any language (client libraries in python, javascript seem to be mature; as well as separate prolog clients as well).

See the docs on the pengines framework.

There is an example of how to contact this service in javascript in [bin/sprog-client.js](bin/sprog-client.js). You will need to do a `npm install pengines`, and change the server URL.

Pengines allows the client to send logic programs to the server, and then to invoke them. For example:

pengines = require('pengines');

peng = pengines({
    server: "https://evening-falls-87315.herokuapp.com/pengine",
    ask: "q(X)",
    chunk: 100,
    sourceText: "q(X):- (wd ?? continent(X)).\n"
}
).on('success', handleSuccess).on('error', handleError);
function handleSuccess(result) {
    console.log('# Results: '+ result.data.length);
    for (var i = 0; i < result.data.length; i++) {
        console.log(result.data[i])
    }
    if (result.data.length == 0) {
        console.log("No results!")
    }
}
function handleError(result) {
    console.error(result)
}

Note that any safe subset of prolog can be passed as a program. In this case we are passing a small program:

`q(X):- (wd ?? continent(X))`

This trivially defines a unary predicate q/1. The argument is bound to any continent. The ?? is a special infix binary predicate, the left side is the service name and the right side is the query to be compiled.

The ask portion of the javascript will simply pass the query to the server.

Using a local sparqlprog service

You can start a sparqlprog service running locally:

docker run -p 9083:9083 cmungall/sparqlprog

(requires docker)

This creates a pengines service at http://localhost:9083/pengine

There is an example of how to contact this service in javascript in [sprog-client.js](bin/sprog-client.js). You will need to do:

npm install pengines

SWISH

TODO

Use within logic programs

For this example, consider writing a music band recommender, based on similarity of genres. dbpedia has triples linking bands to genres, so we will use that.

We will write a program dbpedia_rules.pl that contains definitions of predicates we will use.

First we define a binary predicate that counts the number of bands per genre:

genre_num_bands(G,Count) :-
        aggregate_group(count(distinct(B)),[G],(rdf(B,dbont:genre,G),band(B)),Count).

you can try this with:

`pq-dbpedia -c examples/dbpedia/dbpedia_rules.pl "genre_num_bands(G,Count)"`

this will give results like:

http://dbpedia.org/resource/Independent_music,184
http://dbpedia.org/resource/Funky_Club_Music,1
http://dbpedia.org/resource/Ghettotech,2
http://dbpedia.org/resource/Indian_folk_music,1
http://dbpedia.org/resource/Bakersfield_Sound,1
http://dbpedia.org/resource/Punk_Rawk,1
http://dbpedia.org/resource/Go-go,6
http://dbpedia.org/resource/Jazz_pop,3
http://dbpedia.org/resource/Dubstep,74
http://dbpedia.org/resource/Alt.folk,1
http://dbpedia.org/resource/AfroHouse,1
http://dbpedia.org/resource/Electro-disco,1
http://dbpedia.org/resource/Math_Rock,15

we are doing this because we want to weight band similarity according to how rare a genre is. If two bands share the genre of 'independent music' it is not remarkable, but if two bands share a rarer genre like 'Ghettotech' then we will weight that higher.

we can explicitly bind this to dbpedia using ??/2:

get_genre_num_bands(G,Count) :-
        ??(dbpedia,genre_num_bands(G,Count)).

we can define the Information Content (IC) of a genre G as `-log2(Pr(G))`:

genre_ic(G,IC) :-
        get_genre_num_bands(G,Count),
        get_num_bands(Total),
        seval(-log(Count/Total)/log(2), IC).

This makes use of:

:- table get_num_bands/1.
get_num_bands(Count) :-
        ??(dbpedia,num_bands(Count)).
num_bands(Count) :-
        aggregate(count(distinct(B)),band(B),Count).

Note we are tabling (memoizing) the call to fetch the total number of bands. This means it will only be called once per sparqlprog session.

Finally we can define a 3-ary predicate that compares any two bands and bindings the 3rd arg to a similarity score that is the sum of the ICs of all genres held in common. (for simplicity, we do not penalize unmatched genres, or try to use sub/super genre categories yet):

pair_genre_ic(A,B,SumIC) :-
        get_all_genres(A,SA),
        get_all_genres(B,SB),
        ord_intersection(SA,SB,I),
        aggregate(sum(IC),G^(member(G,I),genre_ic(G,IC)),SumIC).

This is a normal prolog goal and can be executed in a normal prolog context, or from the command line:

`pq-dbpedia -c examples/dbpedia/dbpedia_rules.pl -e "pair_genre_ic(dbr:'Metallica',dbr:'Megadeth',IC)"`

The -e option tells the script to execute the query directly rather than try and compile everything to a single SPARQL query (this may be possible, but could be highly inefficient). It is only when the prolog engine executes the ?? goals that a remote SPARQL will be executed.

If we want to adapt this program to search rather than compare two given bands, we can modify it slightly so that it does not waste cycles querying on bands that have no genres in common:

pair_genre_ic(A,B,SumIC) :-
        get_all_genres(A,SA),
        ??(dbpedia,has_shared_genre(A,B,_)),
        get_all_genres(B,SB),
        ord_intersection(SA,SB,I),
        aggregate(sum(IC),G^(member(G,I),genre_ic(G,IC)),SumIC).

Example of running this:

`pq-dbpedia -c examples/dbpedia/dbpedia_rules.pl -e "pair_genre_ic(dbr:'Voivod_(band)',B,IC),IC>=10"`

Note this is slow, as it will iterate across each band performing queries to gather stats. There are various approaches to optimizing this, but the core idea here is that the logic can be shuffled back and forth between the portion that is compiled to SPARQL and executed remotely, and the portion that is executed locally by a logic engine.

Using a local triplestore

You can use sparqlprog with any local or remote triplestore that supports the SPARQL protocol. If you have RDF files and want to get started, here is one quick route (assuming you have docker):

  1. Place your files in [data](examples/data)
  2. Run `make bg-run`

This will run blazegraph within a docker container

Discussion

SPARQL provides a declarative way of querying a triplestore. One of its limitations is the lack of ability to compose queries and reuse repeated patterns across multiple queries. Sparqlprog is an extension of SPARQL and a subset of Prolog for relational rule-oriented programming using SPARQL endpoints.

Prolog programmers guide

This package provides a more natural (from a Prolog point of view) interface to SPARQL endpoints. There are two layers. The first, lower layer, defines a DCG for generating SPARQL queries from a structured term. The second provides a translation from representation that looks more or less like a Prolog goal built from rdf/3 goals (with conjunction, disjunction etc) to a term in the term language understood by the SPARQL DCG.

In addition, the library provides a mechanism to register known SPARQL endpoints so that they can be referred to by a short name, or to enable a query to be run against all registered endpoints.

The library is based on the idea implemented in Yves Raimond's swic package, but the code has been completely re-implemented.

You just need SWI Prolog with its Semantic Web libraries.

Simple usage

The (??)/2 and (??)/1 operators have a high precedence so that conjuction and disjunctive queries can be written to the right of it without parentheses:

?- rdf_register_prefix(foaf,'http://xmlns.com/foaf/0.1/')
?- rdf_register_prefix(dbont,'http://dbpedia.org/ontology/')
?- sparql_endpoint( dbp, 'http://dbpedia.org/sparql/').
?- debug(sparkle).  % to show queries

?-	dbp ?? rdf(Class,rdf:type,owl:'Class'), rdf(Instance,rdf:type,Class).
?- dbp ?? rdf(Person,rdf:type,foaf:'Person'),
          rdf(Person,foaf:Name,Name),
          filter(regex('Colt.*',Name)).
?- dbp ?? rdf(A,rdf:type,dbont:'Photographer'); rdf(A, rdf:type, dbont:'MusicalArtist').

Clause expansion

If the following clause is defined:

cls(Class) :-
        rdf(Class,rdf:type,owl:'Class').

Then cls/1 can be used in queries, e.g.

?-  dbp ?? cls(X).

The cls/1 goal will be expanded.

More complex goals can be defined; for example, this queries for existential restrictions:

subclass_of(C,D) :- rdf(C,rdfs:subClassOf,Restr).
svf_edge(C,P,D) :-
        subclass_of(C,Restr),
        rdf(Restr,owl:onProperty,P),
        rdf(Restr,owl:someValuesFrom,D).

Only a subset of prolog can be expanded in this way. Conjunction, disjunction (or multiple clauses), negation are supported. Terminals rdf/3, rdf/4, and some predicates from the rdfs library are supported. In future a wider set of constructs may be supported, e.g. setof/3.

It is also possible to use create_sparql_construct/3 and create_sparl_construct/4 to generate SPARQL queries for a limited subset of pure prolog that can be executed outside the prolog environment - effectively a limited prolog to SPARQL compiler.

Comparison with SPIN

TODO https://spinrdf.org/

Credits

The majority of code in this repo was developed by Samer Abdallah, as part of the sparkle package. Some of this code came from Yves Raimond's swic package.

Extensions were implemented by Chris Mungall. In particular

  • goal rewriting
  • DCG extensions: aggregates, filter operators
  • predicate definitions for vocabularies used by various triplestores (faldo, ebi, wikidata, dbpedia, go, monarch)

Contents of pack "sparqlprog"

Pack contains 152 files holding a total of 1.1M bytes.