Subcommands

Jug is organised as a series of subcommands. They are called by jug subcommand jugfile.py [OPTIONS]. This is similar to applications such as version control systems.

Major Subcommands

execute

The main subcommand of jug is execute. Execute executes all your tasks. If multiple jug processes are running at the same time, they will synchronise so that each will run different tasks and combine the results.

It works in the following loop:

while not all_done():
    t = next_task()
    if t.lock():
        t.run()
        t.unlock()

The actual code is much more complex, of course.

status

You can check the status of your computation at any time with status.

shell

Shell drops you into an ipython shell where your jugfile has been loaded. You can look at the results of any Tasks that have already run. It works even if other tasks are running in the background.

IPython needs to be installed for shell to work.

More information about jug shell

Minor Subcommands

check

Check is simple: it exits with status 0 if all tasks have run, 1 otherwise. Useful for shell scripting.

sleep-until

This subcommand will simply wait until all tasks are finished before exiting. It is useful for monitoring a computation (especially if your terminal has an option to display a pop-up or bell when it detects activity). It does not monitor whether errors occur!

invalidate

You can invalidate a group of tasks (by name). It deletes all results from those tasks and from any tasks that (directly or indirectly) depend on them. You need to give the subcommand the name with the --invalid option.

cleanup

Removes all elements in the store that are not used by your jugfile.

Extending Subcommands

Note

This feature is still experimental

Subcommands are now implemented using a modular design that allows extending jug with additional commands. This API is currently in experimental stage and may change in the future.

The following serves as an example of how to extend jug’s commands.

Lets assume you wanted to create a custom report and have it available as:

$ jug my-fancy-report

One way to achieve this is to add the following code to ~/.config/jug/jug_user_commands.py:

from jug.subcommands import SubCommand
class FancyReport(SubCommand):
    "Produces a fancy report of my results"
    name = "my-fancy-report"
    def run(self, *args, **kwargs):
        ...
fancy_report = FancyReport()

The first line of the class docstring is important as it will be shown in jug’s usage help page. The name attribute is also required and should be the name of your subcommand on the command-line.

The body of the method run() defines what should happen when you call the subcommand jug my-fancy-report.

The run function will receive the following objects:

* ``options``  - object representing command-line and user options
* ``store``    - backend object reponsible for handling jobs
* ``jugspace`` - a namespace of jug internal variables (better not touch)

additional objects may be introduced in the future so make sure your function uses *args, **kwargs to maintain compatibility.

Finally, in order to register the subcommand, you must instanciate the subcommand.

If your subcommand needs configurable options you can expose them via command-line by defining two additional methods:

class FancyReport(SubCommand):
    ...
    def parse(self, parser):
        parser.add_argument('--tofile', action='store',
                            dest='report_tofile',
                            help='Name of file to use for report')
    def parse_defaults(self):
        return {
            "report_tofile": "report.txt",
        }
fancy_report = FancyReport()

The first method configures argparse arguments that will be available as jug my-fancy-report --tofile myreport.txt. These will also be avaiable to the run() method as part of the options object.

The second defines default values in case of omission. The key should match the dest= attribute of add_argument() and the value should be any object to be used by your run() method. Note that the value received in the command-line will be automatically converted to the same type as this default (i.e. if your default is True any --tofile john would result in bool("john") -> True).

For more information on parser configuration refer to argparse’s documentation.

NOTE: A few words of caution, we cannot rely on argparse’s default= option since it doesn’t allow distinguishing between user supplied and built-in (default) values. For the same reason, don’t use action= with store_true or store_false instead use store_const and const= with True or False. Failing to do so will cause any matching setting on jugrc to not have any effect.