This is a complementary tool for the
lgtunit tool for automatically
creating bug report issues for failed tests in GitHub or GitLab servers.
This tool requires that the GitHub and GitLab CLIs be installed. For the installation instructions see:
This tool can be loaded using the query:
| ?- logtalk_load(issue_creator(loader)).
But in the most common usage scenario, this tool is automatically loaded
logtalk_tester automation script.
logtalk_tester automation script accepts a
-b option for
automatically using this tool (see the script man page for details). In
the most simple case, this option possible values are
gitlab. For example:
$ logtalk_tester \ -p gnu \ -b github \ -s "/home/jdoe/foo/" \ -u https://github.com/jdoe/foo/tree/55aa900775befa135e0d5b48ea63098df8b97f5c/
logtalk_tester script must be called from a git repo
directory or one of its sub-directories, which is a common setup in
CI/CD pipelines. Moreover, prior to running the tests, the CLI must be
used, if required, to authenticate and login to the server where the bug
report issues will be created:
gh auth login --hostname <string> --with-token < token.txt
glab auth login --hostname <string> --token <string>
The access token must ensure the necessary scopes that allow bug reports
to be created. See the CLIs documentation for details. Typically, the
auth command is called from the CI/CD pipeline definition scripts.
However, depending on the CI/CD workflow, the authentication may be done
The bug reports are created using by default the label
assigned to the author of the latest commit of the git repo. The
option can also be used to override the label with a comma separated set
of labels. For example, to use both
$ logtalk_tester \ -p gnu \ -b github:bug,auto \ -s "/home/jdoe/foo/" \ -u https://github.com/jdoe/foo/tree/55aa900775befa135e0d5b48ea63098df8b97f5c/
Note that the labels must be predefined in the issue tracker server for the bug report to be successfully created.
The bug reports use Markdown formatting, which is the default in GitHub and GitLab issue trackers.
But reports are only created for non-flaky tests. The bug report title and labels are used to prevent creating duplicated bug reports. Therefore, the same labels should be used for multiple runs of the same tests and preserved when editing the bug reports.
There are cases where we may want to postpone or temporarily disable the
automatic creation of bug reports. E.g. a WIP branch that’s known to
break multiple tests. A solution is to define a pull/merge request
NO_AUTO_BUG_REPORTS, that can then be checked by the
CI/CD workflow. For example, we can test the presence of that label to
AUTO_BUG_REPORTS environment variable to either an empty
string or a
-b option and use:
logtalk_tester.sh $AUTO_BUG_REPORTS -p ...
GitLab creates CI/CD pipelines in a detached HEAD state. As a consequence, the bug reports always show “Git branch: HEAD”.
This tool is in a beta stage of development. Your feedback is most appreciated.