lib.pl -- Predicate based code development.

This pack implements methods for loading code into SWI Prolog programs.

Main innovations

Lazy loading

One of the major innovations the library introduces, is that of progressive, lazy loading of packs. That is, if only a specific predicate is (lazily) required from a pack(lib)-aware pack, only that and its dependent code will be loaded.

That is, your code can load things like

?- lib( stoics_lib:kv_compose/3 ).
?- lib( stoics_lib:kv_decompose/3 ).

and only the relevant parts of the pack(stoics_lib) will be loaded.

If later on your code decides to do a

?- lib(stoics_lib).

The remainder of the library loads up quietly and politely.

Please note that this is, at top level at least, orthogonal to any other loading.

You can still do

?- use_module( library(stoics_lib) ).

and get the whole thing into memory.

A good example of how to create a lazy pack is pack(stoics_lib), http://stoics.org.uk/~nicos/sware/stoics_lib v0.3. An example of how to lazy load things from stoics_lib is the latest pack(debug_call) http://stoics.org.uk/~nicos/sware/debug_call v0.4.

Cells

As of version 2.0 the pack supports hierarchical module de-composition.

A cell compose pack, is build by a skeleton module that all cells depend on and then a number of independent cells that can be loaded independently as well as in combination.

There are at least 2 reasons why one would like decomposable modules: (a) resources, and (b) clarity of interface. Only loading parts of a module can result in smaller memory consumption as irrelevant bits are not loaded. Also, if modules have long lists of defined predicates, like bio_db v2.0, then loading only conceptually clear sub-set of a module allows programmer to focus on the predicates that are relevant to a specific task.

pack(bio_db) was the driving force for developing cell based packs and it provides natural cell units. At the top level there are two cells, hs for human biological data and mouse for mouse data. Each cell is further broken to a number of cells each corresponding to the source database where data is converted from. For instance hs contains sub-cells: ense, gont, hgnc, ncbi, pros, strg and unip.

See pack(bio_db/cell/hs.pl) and pack(bio_db/cell/mouse.pl).

Cell based pack can still be viewed and loaded as normal module files. For instance,

?- use_module(library(bio_db)).

Loads the whole interface (all cells), without the user needing to be aware of anything. The only difference is that the user will not be able to see all the module predicates at the first line of file pack(bio_db/prolog/bio_db.pl)).

?- lib(bio_db).

Also loads everything.

?- lib(& bio_db).

Loads the skeleton of the module (cells usually load the module dependencies like this). That is, file pack(prolog/bio_db.pl), but not the cell files in pack(cell/ * ).

?- lib(& bio_db(hs)).

Loads hs cell, which in this case comprises of number of sub-cells.

?- lib(& bio_db(hs)).

Loads hs cell (and skeleton). hs comprises of a number of sub-cells.

?- lib(& bio_db(hs(hgnc))).

Loads the hs/hgnc primary cell (and the skeleton).

?- use_module( pack('bio_db/cell/hs/hgnc') ).
?- lib(@ bio_db)

Loads all sub-cells of a library.

?- load_files( library(bio_db) ).

Will load everything even if cell based loading ahs taken place. (use_module(library(bio_db)) would work.)

Suggested code

The library supports suggested loading and code execution. These operations are meant for fringe features that are not, by default reported if missing. Reporting in form of warnings can be turned on by either setting flag lib_suggests_warns to true (globally controlled), or passing option (local, controlled by developer).

Prolog lag lib_suggests_warns can take values:

auto
(default flag value), silent by default unless loading code presents option suggests_warns(true)
false
never warn when suggested features are missing
true
always warn when features are missing.
:- lib(suggests(wgraph),[]).
:- lib(real).
:- lib(suggests(call(lib_r("GGally"),[]).

Other features

General points

Pack(lib) plays reasonably well with the documentation server. Bar, the normal limitations of the server. By convention and to help locating the module docs, lazy packs should define (Pack)/0 predicate in same file as the mods docs. Searching for that on doc server, should make it easy enough to get to it.

Although this library, pack(lib), contains a number of involved features it can also be used as a straight forward shorthand, replacement for use_module(library(Lib)).

 ?- lib(Atomic).

is equivelant to use_module(library(Atomic)) if Atomic is a system library or an installed Pack, while it will interogate the SWI pack server for matching packs if Atomic is atomic and not an installed pack.

In addition the library allows for loading with initializations turned off.

Repositories

Code is managed in repositories (also repo) that can be either packs or libs (ie local directories).

A pack is a unit of programs as managed by built-in SWI package manager (library(prolog_pack)). A lib (library) is a directory containing a number of program files.

pack(lib) supports a number of ways to organise your code and load it, but it comes to its own when code s organised as predicate-in-a-file fashion. In this mode of development a predicate such as kv_decompose/3 would be defined on file kv_decompose.pl which will only containing code for defining this predicate, with the possible exception of helper predicates that are too specific to be of outside interest.

kv_decompose( [], [], [] ).
kv_decompose( [K-V|T], [K|Tk], [V|Tv] ) :-
    kv_decompose( T, Tk, Tv ).

Lib code is considered as coming from the special pack user.

Code-tables

Associated with each repository are 2 types of code-tables: (code-)_indices_ and (file-)_locators_.

A code-index maps a predicate identifier along with its source repo to an absolute file name of the source that defines it. Indices are of the form:

lib_tables:lib_index( Pname, Parity, Repo, AbsFile ).

File locators store all filenames in a repository. These can be named matched predicate names that need to be loaded. pack(lib) can be directed to assume that files from a specific repository exhibit this homonyms property. Locators are of the form:

lib_tables:lib_homonym( Stem, Repo, AbsFile ).

For each index file loaded for a repository, the following is asserted:

lib_tables:lib_loaded_indices(Repo,File)

and for each locator

lib_tables:lib_loaded_homonyms(Repo,Stem,File)

When loading a repository the user can choose whether to load indices and locators independently.

Loading source code

During the process of loading code into memory, lib/1 and /2 directives are used to locate code to which the specific code depends.

There are three main categories of operations:

These operations are all specific to the loading context. This is achieved by creating meta-predicates that identify which part of the repository base each context has access to.

Attachment of repository is registered via lib_tables:lib_attached_indices(To,PackIG) and lib_tables:lib_attached_homonyms(To,PackFG).

lib_tables:lib_attached_indices(bims,options).
lib_tables:lib_attached_homonyms(bims,false).

attaches the indices but not the file locators.

Since all code from directory-libs load to a single module (user), loading code has either access to all such code, or to none.

Conventions

Packs are expected to have matching top directories and main files. The main file of a pack should be within top directory prolog/. (The directory convention is set by library(prolog_pack)). For example for pack bims the following file should exist in packs directory:

bims/prolog/bims.pl

For packs the main code directory is src/. Additionally src/lib and src/auxil are treated as code directories.

Internals

Variables

Repo
repository
Pack
prolog pack (installed or locally addressed)
Lib
directory containing code
Root
absolute reference to the root directory of a Repo
Pn
predicate name
Idx
library index term
Hmn
library file homonym term
Pa
predicate arity
Cxt
context module

Predicate names

Pack info

This is a complete re-write of pack(requires) v1.1.

Listens to debug(lib).

author
- nicos angelopoulos
version
- 1.0 2017/3/6
- 1.1 2017/3/9, lazy loading
- 1.2 2017/3/11, fixed missing cut, added lib(version(V,D))
- 1.3+4 2017/8/8, fixed multi-source for user, improved contact to server, install while lazy loading
- 1.5 2017/8/15
- 1.6 2018/3/18, lib/2 suggests(), lib/2, promise() via hot-swapping, private packs
- 1.7 2018/4/5, auto-install missing was broken
- 2.2 2018/11/26, cell based module compositionality, & operator (by default load everything)
- 2.3 2019/4/18, lib_code_loader/3 hook & lib_r/2, suggests failure messages via lib_suggests_warns flag & options
- 2.4 2019/4/22, small fix release
- 2.5 2019/5/8, bioc (for bioconductor) load term
- 2.6 2020/3/8, fixed cell-loading warnings
- 2.7 2020/3/8, compatibility with pack changes in SWI-8.2, fixed layout breaking tags
- 2.8 2020/9/18, minor changes, library(lists) explicit loading + info messages
- 2.9 2021/1/23, honour developer suggests_warns(false), logic needs further work
- 2.10 2022/12/29 bring up to date for bio_db 4:0
See also
- http://stoics.org.uk/~nicos/sware/lib
 lib(+Operand)
 lib(+Operand, +Opts)
Loads code or/and indices of Repo into the current context.

When Repo homonym(Repository) then only the homonims of local dir (adjusted for pack dir structure) are added to as coming from Repository.

Operand
One of
homonyms(From)
attach homonyms From pack
init(Lib)
init(Lib, Cxt)
declare initilization call (library can be loaded without this firing, if so needed, as is the case for lib_mkindex/1)
suggests(Lib)
suggests(Lib, SugOpts)
it is likely you need Lib for full functionalilty. If Lib is a known library it is loaded other wise nothing is loaded.
This is useful for fringe functionalities that depend on external libraries, where we do not want the average user to do anything if library (Lib) is not there. See lib_suggests/2 for details of how to enable warning messages.
promise(Pred, Load)
Pred is needed for functionality and it can be found by loading Load, but it will only happen at Pred's first call.
expects(Pid, Mess)
expects(Pid, Mess, Call)
complains if Pid is not defined at loading time. Mess should be a debug style message with one ~w which will be called with Pid as its printing argument. If call is present, is called after the printing of the message.
version(Vers, Date)
return version and publication date

Opts

index(Idx)
whether to load indices
homonym(Hnym)
whether to load homonym file-locators
load(Load)
whether to load the main entry point of Repo
mode(Mode=self)
makes missing message more acurate (other value: suggests)
suggest(Dn=true)
suggest the library is downloaded if it is not locally installed ?
type(Type)
enforce a particular type of repository (pack or lib)

The defaults depend on whether Repo is a pack or a lib.

opts(Opts=PackDefs)
PackDefs= [load(true),index(true),homonym(false),type(pack)]
opts(Opts=LibDefs)
LibDefs = [load(false),index(true),homonym(true),type(lib)]

When invoked with code attaching operands (SysLibrary, Pack or Lib) the predicate will first load anything that needs to be loaded in their native module and then import predicates from that module. Attaching a lib or pack means that the predicates pointed to by indices and by file name from the target pack/lib become available to the importee. Option index(Idx) controls whether LibIndex.pl based indices are attahced whereas homonym(Hmns) control the attachment of the file names from within the filesystem of the target.

For example to only import the interface predicates of pack ex1 use

?- lib(ex1, [type(pack),load(true),index(false),homonym(false)]).

Assume that ex1 is a pack that is not installed on your Prolog installation, but you have its sources unpacked on local dir /tmp/ex1/ you can load it interface predicates with:

For example to only import the interface predicates of pack ex1 use

?- lib('/tmp/ex1', [type(pack),load(true),index(false),homonym(false)]).

Assume there is a file src/lib/foo.pl in ex1 defining predicate foo/1, then you can load its code with

?- lib('/tmp/ex1', [type(pack),load(true),index(false),homonym(true)]).
?- lib(foo/1).

The above will first load foo.pl (by means of matching its filename to the predicate name) into ex1: and then assuming that this loaded foo/1 it will import it into current context (here this is =user+).

Assuming foo.pl also defines predicate bar/2 and there is a file src/LibIndex.pl within ex1 containing the line

lib_index( bar, 2, swipl(_), user, 'lib/foo.pl' ).

Then the code for foo_bar/2 can be loaded with

?- lib('/tmp/ex1', [type(pack),load(true),index(true),homonym(false)]).
?- lib(bar/2).

Pack lib can be used to create and access skeleton packs. These packs, may load very little interface code but their code base can be loaded on demand and piece-meal. That is if a specific non-interface predicate is required, it will be located and loaded along with all its dependencies.

An example of such a pack is stoics_lib. The following commands: 1. load the minimal interface,, 2, load the code for a specific non-interface predicate.

?- lib(stoics_lib).
?- lib(kv_decompose/3).

The above two directives can be shortened to:

?- lib(stoics_lib:kv_decompose/3).

Current version can be found by:

?-
    lib( version(Vers,Date) ).
Vers = 2:10:0,
Date = date(2022, 12, 29).
author
- nicos angelopoulos
version
- 2:10 2022/12/29
To be done
- when predicate is missing from stoics_lib while loading from b_real, we get clash between main and lazy, error should be clearer (the pred select_all/3 was actually not defined in file either)