This page contains benchmark results for some Prolog compilers. The main goal of this page it to give you some data for comparing predicate performance in plain Prolog and using Logtalk objects. Benchmark results are provided for both static code and dynamic code.

All the tests have been performed using the benchmarks example distributed with Logtalk version 2.42.4 (r6128), using either static binding or dynamic binding but with optional features (including events support) disabled. This provides two relevant scenarios for comparing Logtalk performance with plain Prolog performance. The worst case scenario corresponds to only using dynamic binding with event support enabled. The benchmarks example contains loader files for easily setting up different test scenarios.

The benchmarks example provides list length and naive list reverse predicates defined in plain Prolog, in a Prolog module, and in a Logtalk object (predicate definitions are the same in all cases). The following goals are used for the first two benchmark tests:

s11: generate_list(30, List), my_length(List, _)

s12: generate_list(30, List), module:mod_length(List, _)

s13: generate_list(30, List), object::length(List, _)

s21: generate_list(30, List), my_nrev(List, _)

s22: generate_list(30, List), module:mod_nrev(List, _)

s23: generate_list(30, List), object::nrev(List, _)

These benchmark tests use a list of 30 elements as an argument to the list predicates. Increasing the list length may lead to decreasing performance differences between plain Prolog and Logtalk as the list length computation time starts to outweigh the overhead of the message sending mechanism. Likewise, decreasing the list length may lead to increasing performance differences between plain Prolog and Logtalk (up to the point you will be closing on the Logtalk message sending mechanism overhead when compared to plain Prolog predicate calls). However, these tests make use of common library predicates where static binding is easily enabled, eliminating the message sending mechanism overheads. The next two examples deal with graph search:

s31: maze_solve(1, 7, _)

s32: module:mod_maze_solve(1, 7, _)

s33: maze::solve(1, 7, _)

s41: graph_path(0, 4, _)

s42: module:mod_graph_path(0, 4, _)

s43: graph::path(0, 4, _)

When static binding is used, the performance of each set of goals is expected to be similar. The performance of Logtalk can be worse due to the overhead of the extra argument added to each compiled object predicate for carrying execution context information. This overhead depends on the Prolog abstract machine and on the optimizations used to pass unchanged arguments between predicate calls.

Category predicates can be called using either the ::/1 or the :/1 control constructs. When using the :/1 control construct, the lookup for both the predicate declaration and the predicate definition begins in this and is restricted to the imported categories. Depending on how the category is compiled, Logtalk may use static binding for :/1 calls, providing the same performance level as calls to local object predicates. The following goals are used for the benchmark tests:

c1: leaf::obj_local

c2: leaf::ctg_direct

c3: leaf::ctg_self

The obj_local method calls a local object predicate; the performance of such calls is equal or close to plain Prolog. The ctg_direct method uses the :/1 control construct to call an imported category predicate. The ctg_self method uses the ::/1 message sending control construct to call an imported category predicate. While the :/1 calls may use static binding, the ::/1 calls always use dynamic binding and a lookup caching mechanism. Note that the choice between either control construct is not simply a question of performance as the control constructs provide different semantics for calling imported category predicates. All three predicates perform the same computation (generating a list of twenty elements and calculating its length) using local predicates.

Dynamic code tests include both object database updates and creating and abolishing dynamic objects. The benchmarks example provides an object named database, which defines a set of predicates for testing the Logtalk built-in database methods as described below. The following goals are used for the benchmark tests:

d1: create_object(xpto, [], [], []), abolish_object(xpto)

d2: plain_dyndb(_)

d3: database::this_dyndb(_)

d4: database::self_dyndb(_)

d5: database::obj_dyndb(_)

The first test simply creates and abolishes a (dynamic) object. The remaining tests are used for benchmarking object database updates, comparing with plain Prolog database updates. The *_dyndb tests simply assert (using assertz/1) and retract a clause (using retract/1) of a dynamic predicate with arity one. The plain_dyndb(_) test uses the Prolog built-in database predicates. The other three tests use the Logtalk built-in database methods, using a direct method call (this_dyndb(_)), a call using ::/1 (self_dyndb(_)), and a call using ::/2 (obj_dyndb(_)).

Apple MacBook Pro 15.4" Core 2 Duo 2.8GHz, 4GB RAM, MacOS X Intel 10.6.6. All results are given in number of calls per second. By default, the benchmark code repeats each goal up to 100000 times in order to get more accurate results. The last columns show the trade-off between plain Prolog and Logtalk. Dynamic binding is never used in the Prolog module tests.

All results are given in number of calls per second. By default, the benchmark code repeats each goal up to 100000 times in order to get more accurate results. The last column shows the trade-off between static binding (c2) and dynamic binding (c3) when calling category predicates.

All results are given in number of calls per second. By default, the benchmark code repeats each goal up to 100000 times in order to get more accurate results. The last column shows the trade-off between plain Prolog (d2) and Logtalk using static binding (d3).

• It's surprisingly difficult to get stable results, specially with some Prolog compilers. One the reasons seems to be the operating-system constant shuffling of processes between the cores.
• Some results are odd, either above the expected maximum (100% of plain Prolog performance) or much lower than what's reasonable to expect. This happens mostly on the most simple benchmark goals. Benchmarks where a more significant amount of work is performed seem to be more (but not complete) immune to these issues.
• All benchmark tests use the default memory allocation for the different program areas. Changing the size of these program areas can have a big impact on the benchmark results (e.g. increasing stack size to avoid wasting time expanding the stack or doing garbage collection).
• Logtalk usually performs better with Prolog compilers with mature virtual machines when compared with Prolog compilers with younger and less optimized virtual machines. The presence, in Logtalk compiled code, of a hidden execution context predicate argument is a particular sensitive point in virtual machines optimization as this extra argument is usually passed unchanged between local predicate calls.
• These are too few and too limited benchmark tests to effectively compare Prolog compiler performance.
• Processor caches sometimes result in tests one order of magnitude better than the results posted above.
• Some of the Prolog built-in predicates used for measuring CPU time are not as accurate as we would like. Despite each benchmark goal being proved by default 100000 times, repeating the tests always show some variation on the final results. Increasing the number of repetions often helps in getting more stable results.
• The more real work your object and category predicates do, the less significant the Logtalk message sending overheads become.
• In real-life applications, only testing can give you a balanced view on the trade-offs between plain Prolog performance and Logtalk programming features. Nevertheless, and as a general and rough estimate, you can expect a performance penalty between 5% and 10% when using static binding and a performance penalty between 10% and 20% when using dynamic binding.