Term and goal expansion

Logtalk supports a term and goal expansion mechanism that can be used to define source-to-source transformations. Two common uses are the definition of language extensions and domain-specific languages.

Logtalk improves upon the term-expansion mechanism found on some Prolog systems by providing the user with fine-grained control on if, when, and how expansions are applied. It allows declaring in a source file itself which expansions, if any, will be used when compiling it. It allows declaring which expansions will be used when compiling a file using compile and loading predicate options. It also allows defining a default expansion for all source files. It defines a concept of hook objects that can be used as building blocks to create custom and reusable expansion workflows with explicit and well-defined semantics. It prevents the simple act of loading expansion rules affecting subsequent compilation of files. It prevents conflicts between groups of expansion rules of different origins. It avoids a set of buggy expansion rules from breaking other sets of expansion rules.

Defining expansions

Term and goal expansions are defined using, respectively, the predicates term_expansion/2 and goal_expansion/2, which are declared in the expanding built-in protocol. Note that, unlike Prolog systems also providing these two predicates, they are not declared as multifile predicates in the protocol. This design decision is key for giving the programmer full control of the expansion process and preventing the problems inflicting most of the Prolog systems that provide a term-expansion mechanism.

An example of an object defining expansion rules:

:- object(an_object,
    implements(expanding)).

    term_expansion(ping, pong).
    term_expansion(
        colors,
        [white, yellow, blue, green, read, black]
    ).

    goal_expansion(a, b).
    goal_expansion(b, c).
    goal_expansion(X is Expression, true) :-
        catch(X is Expression, _, fail).

:- end_object.

These predicates can be explicitly called using the expand_term/2 and expand_goal/2 built-in methods or called automatically by the compiler when compiling a source file (see the section below on hook objects).

In the case of source files referenced in include/1 directives, expansions are only applied automatically when the directives are found in source files, not when used as arguments in the create_object/4, create_protocol/3, and create_category/4, predicates. This restriction prevents inconsistent results when, for example, an expansion is defined for a predicate with clauses found in both an included file and as argument in a call to the create_object/4 predicate.

Clauses for the term_expansion/2 predicate are called until one of them succeeds. The returned expansion can be a single term or a list of terms (including the empty list). For example:

| ?- an_object::expand_term(ping, Term).

Term = pong
yes

| ?- an_object::expand_term(colors, Colors).

Colors = [white, yellow, blue, green, read, black]
yes

When no term_expansion/2 clause applies, the same term that we are trying to expand is returned:

| ?- an_object::expand_term(sounds, Sounds).

Sounds = sounds
yes

Clauses for the goal_expansion/2 predicate are recursively called on the expanded goal until a fixed point is reached. For example:

| ?- an_object::expand_goal(a, Goal).

Goal = c
yes

| ?- an_object::expand_goal(X is 3+2*5, Goal).

X = 13,
Goal = true
yes

When no goal_expansion/2 clause applies, the same goal that we are trying to expand is returned:

| ?- an_object::expand_goal(3 =:= 5, Goal).

Goal = (3=:=5)
yes

The goal-expansion mechanism prevents an infinite loop when expanding a goal by checking that a goal to be expanded was not the result from a previous expansion of the same goal. For example, consider the following object:

:- object(fixed_point,
    implements(expanding)).

    goal_expansion(a, b).
    goal_expansion(b, c).
    goal_expansion(c, (a -> b; c)).

:- end_object.

The expansion of the goal a results in the goal (a -> b; c) with no attempt to further expand the a, b, and c goals as they have already been expanded.

Goal-expansion applies to goal arguments of control constructs, meta-arguments in built-in or user defined meta-predicates, meta-arguments in local user-defined meta-predicates, meta-arguments in meta-predicate messages when static binding is possible, and initialization/1, if/1, and elif/1 directives.

Expanding grammar rules

A common term expansion is the translation of grammar rules into predicate clauses. This transformation is performed automatically by the compiler when a source file entity defines grammar rules. It can also be done explicitly by calling the expand_term/2 built-in method. For example:

| ?- logtalk::expand_term((a --> b, c), Clause).

Clause = (a(A,B) :- b(A,C), c(C,B))
yes

Note that the default translation of grammar rules can be overridden by defining clauses for the term_expansion/2 predicate.

Bypassing expansions

Terms and goals wrapped by the {}/1 control construct are not expanded. For example:

| ?- an_object::expand_term({ping}, Term).

Term = {ping}
yes

| ?- an_object::expand_goal({a}, Goal).

Goal = {a}
yes

This also applies to source file terms and source file goals when using hook objects (discussed next).

Hook objects

Term and goal expansion of a source file during its compilation is performed by using hook objects. A hook object is simply an object implementing the expanding built-in protocol and defining clauses for the term and goal expansion hook predicates. Hook objects must be compiled and loaded prior to being used to expand a source file.

To compile a source file using a hook object, we can use the hook compiler flag in the second argument of the logtalk_compile/2 and logtalk_load/2 built-in predicates. For example:

| ?- logtalk_load(source_file, [hook(hook_object)]).
...

In alternative, we can use a set_logtalk_flag/2 directive in the source file itself. For example:

:- set_logtalk_flag(hook, hook_object).

To use multiple hook objects in the same source file, simply write each directive before the block of code that it should handle. For example:

:- object(h1,
    implements(expanding)).

    term_expansion((:- public(a/0)), (:- public(b/0))).
    term_expansion(a, b).

:- end_object.

:- object(h2,
    implements(expanding)).

    term_expansion((:- public(a/0)), (:- public(c/0))).
    term_expansion(a, c).

:- end_object.

:- set_logtalk_flag(hook, h1).

:- object(s1).

    :- public(a/0).
    a.

:- end_object.


:- set_logtalk_flag(hook, h2).

:- object(s2).

    :- public(a/0).
    a.

:- end_object.

| ?- {h1, h2, s}.
...

| ?- s1::b.
yes

| ?- s2::c.
yes

It is also possible to define a default hook object by defining a global value for the hook flag by calling the set_logtalk_flag/2 predicate. For example:

| ?- set_logtalk_flag(hook, hook_object).

yes

Note that, due to the set_logtalk_flag/2 directive being local to a source file, using it to specify a hook object will override any defined default hook object or any hook object specified as a logtalk_compile/2 or logtalk_load/2 predicate compiler option for compiling or loading the source file.

Note

Clauses for the term_expansion/2 and goal_expansion/2 predicates defined within an object or a category are never used in the compilation of the object or the category itself.

Virtual source file terms and loading context

When using a hook object to expand the terms of a source file, two virtual file terms are generated: begin_of_file and end_of_file. These terms allow the user to define term-expansions before and after the actual source file terms.

Logtalk also provides a logtalk_load_context/2 built-in predicate that can be used to access the compilation/loading context when performing expansions. The logtalk built-in object also provides a set of predicates that can be useful, notably when adding Logtalk support for language extensions originally developed for Prolog.

As an example of using the virtual terms and the logtalk_load_context/2 predicate, assume that you want to convert plain Prolog files to Logtalk by wrapping the Prolog code in each file using an object (named after the file) that implements a given protocol. This could be accomplished by defining the following hook object:

:- object(wrapper(_Protocol_),
    implements(expanding)).

    term_expansion(begin_of_file, (:- object(Name,implements(_Protocol_)))) :-
        logtalk_load_context(file, File),
        os::decompose_file_name(File,_ , Name, _).

    term_expansion(end_of_file, (:- end_object)).

:- end_object.

Assuming, e.g., my_car.pl and lease_car.pl files to be wrapped and a car_protocol protocol, we could then load them using:

| ?- logtalk_load(
         ['my_car.pl', 'lease_car.pl'],
         [hook(wrapper(car_protocol))]
     ).

yes

Note

When a source file also contains plain Prolog directives and predicates, these are term-expanded but not goal-expanded (with the exception of the initialization/1, if/1, and elif/1 directives, where the goal argument is expanded to improve code portability across backends).

Default compiler expansion workflow

When compiling a source file, the compiler will first try, by default, the source file-specific hook object specified using a local set_logtalk_flag/2 directive, if defined. If that expansion fails, it tries the hook object specified using the hook/1 compiler option in the logtalk_compile/2 or logtalk_load/2 goal that compiles or loads the file, if defined. If that expansion fails, it tries the default hook object, if defined. If that expansion also fails, the compiler tries the Prolog dialect-specific expansion rules found in the adapter file (which are used to support non-standard Prolog features).

User defined expansion workflows

Sometimes we have multiple hook objects that we need to combine and use in the compilation of a source file. Logtalk includes a hook_flows library that supports two basic expansion workflows: a pipeline of hook objects, where the expansion results from a hook object are fed to the next hook object in the pipeline, and a set of hook objects, where expansions are tried until one of them succeeds. These workflows are implemented as parametric objects, allowing combining them to implement more sophisticated expansion workflows. There is also a hook_objects library that provides convenient hook objects for defining custom expansion workflows. This library includes a hook object that can be used to restore the default expansion workflow used by the compiler.

For example, assuming that you want to apply the Prolog backend-specific expansion rules defined in its adapter file, using the backend_adapter_hook library object, passing the resulting terms to your own expansion when compiling a source file, we could use the goal:

| ?- logtalk_load(
         source,
         [hook(hook_pipeline([backend_adapter_hook, my_expansion]))]
     ).

As a second example, we can prevent expansion of a source file using the library object identity_hook by adding as the first term in a source file the directive:

:- set_logtalk_flag(hook, identity_hook).

The file will be compiled as-is as any hook object (specified as a compiler option or as a default hook object) and any backend adapter expansion rules are overridden by the directive.

Using Prolog defined expansions

In order to use clauses for the term_expansion/2 and goal_expansion/2 predicates defined in plain Prolog, simply specify the pseudo-object user as the hook object when compiling source files. When using backend Prolog compilers that support a module system, it can also be specified a module containing clauses for the expanding predicates as long as the module name doesn’t coincide with an object name. When defining a custom workflow, the library object prolog_module_hook/1 can be used as a workflow step. For example, assuming a module functions defining expansion rules that we want to use:

| ?- logtalk_load(
         source,
         [hook(hook_set([prolog_module_hook(functions), my_expansion]))]
     ).

But note that Prolog module libraries may provide definitions of the expansion predicates that are not compatible with the Logtalk compiler. In particular, when setting the hook object to user, be aware of any Prolog library that is loaded, possibly by default or implicitly by the Prolog system, that may be contributing definitions of the expansion predicates. It is usually safer to define a specific hook object for combining multiple expansions in a fully controlled way.

Note

The user object declares term_expansion/2 and goal_expansion/2 as multifile and dynamic predicates. This helps in avoiding predicate existence errors when compiling source files with the hook flag set to user as these predicates are only natively declared by some of the supported backend Prolog compilers.

Debugging expansions

The term_expansion/2 and goal_expansion/2 predicates can be debugged like any other object predicates. Note that expansions can often be manually tested by sending expand_term/2 and expand_goal/2 messages to a hook object with the term or goal whose expansion you want to check as argument. An alternative to the debugging tools is to use a monitor for the runtime messages that call the predicates. For example, assume a expansions_debug.lgt file with the contents:

:- initialization(
    define_events(after, edcg, _, _, expansions_debug)
).


:- object(expansions_debug,
    implements(monitoring)).

    after(edcg, term_expansion(T,E), _) :-
        writeq(term_expansion(T,E)), nl.

:- end_object.

We can use this monitor to help debug the expansion rules of the edcg library when applied to the edcgs example using the queries:

| ?- {expansions_debug}.
...

| ?- set_logtalk_flag(events, allow).
yes

| ?- {edcgs(loader)}.
...
term_expansion(begin_of_file,begin_of_file)
term_expansion((:-object(gemini)),[(:-object(gemini)),(:-op(1200,xfx,-->>))])
term_expansion(acc_info(castor,A,B,C,true),[])
term_expansion(pass_info(pollux),[])
term_expansion(pred_info(p,1,[castor,pollux]),[])
term_expansion(pred_info(q,1,[castor,pollux]),[])
term_expansion(pred_info(r,1,[castor,pollux]),[])
term_expansion((p(A)-->>B is A+1,q(B),r(B)),(p(A,C,D,E):-B is A+1,q(B,C,F,E),r(B,F,D,E)))
term_expansion((q(A)-->>[]),(q(A,B,B,C):-true))
term_expansion((r(A)-->>[]),(r(A,B,B,C):-true))
term_expansion(end_of_file,end_of_file)
...

This solution does not require compiling the edcg hook object in debug mode or access to its source code (e.g., to modify its expansion rules to emit debug messages). We could also simply use the user pseudo-object as the monitor object:

| ?- assertz((
         after(_, term_expansion(T,E), _) :-
            writeq(term_expansion(T,E)), nl
     )).
yes

| ?- define_events(after, edcg, _, Sender, user).
yes

Another alternative is to use a pipeline of hook objects with the library hook_pipeline/1 and write_to_stream_hook objects to write the expansion results to a file. For example, using the unique.lgt test file from the edcgs library directory:

| ?- {hook_flows(loader), hook_objects(loader)}.
...

| ?- open('unique_expanded.lgt', write, Stream),
     logtalk_compile(
         unique,
         [hook(hook_pipeline([edcg,write_to_stream_hook(Stream,[quoted(true)])]))]
     ),
     close(Stream).
...

The generated unique_expanded.lgt file will contain the clauses resulting from the expansion of the EDCG rules found in the unique.lgt file by the edcg hook object expansion.