Worked-Out Example 0

Decrypting a Password

Problem: crack an encrypted file by brute force. Assume that the password is a five-letter lower-case word and that you know that the plain text contains my name.

(The complete code for this example and a secret message comes with the jug source)

This is the ultimate parallel problem: try very many keys (26**5 ~ 11M), but there is no interaction between the different tasks.

The brute force version is very simple:

  1. for p in product(letters, repeat=5):
  2.     text = decode(ciphertext, p)
  3.     if isgood(text):
  4.         passwd = "".join(map(chr, p))
  5.         print('%s:%s' % (passwd, text))

However, if we have more than one processor, we’d like to be able to tell jug to use multiple processors.

We cannot simply have each password be its own task: 11M tasks would be too much!

So, we are going to iterate over the first letter and a task will consist of trying every possibility starting with that letter:

  1. @TaskGenerator
  2. def decrypt(prefix, suffix_size):
  3.     res = []
  4.     for p in product(letters, repeat=suffix_size):
  5.         text = decode(ciphertext, np.concatenate([prefix, p]))
  6.         if isgood(text):
  7.             passwd = "".join(map(chr, p))
  8.             res.append((passwd, text))
  9.     return res
  11. @TaskGenerator
  12. def join(partials):
  13.     return list(chain(*partials))
  15. fullresults = join([decrypt([let], 4) for let in letters])

Here, the decrypt function returns a list of all the good passwords it found. To simplify things, we call the join function which concatenates all the partial results into a single list for convenience.

Now, run jug:

  1. $ jug execute jugfile.py &
  2. $ jug execute jugfile.py &
  3. $ jug execute jugfile.py &
  4. $ jug execute jugfile.py &

You can run as many simultaneous processes as you have processors. To see what is happening, type:

  1. $ jug status jugfile.py

And you will get an output such as:

  1. Task name                                    Waiting       Ready    Finished     Running
  2. ----------------------------------------------------------------------------------------
  3. jugfile.join                                       1           0           0           0
  4. jugfile.decrypt                                    0          14           8           4
  5. ........................................................................................
  6. Total:                                             1          14           8           4

There are two task functions: decrypt, of which 8 are finished, 14 are ready to run, and 4 are currently running; and join which has a single instance, which is waiting: it cannot run until all the decrypt tasks have finished.

Eventually, everything will be finished and your results will be saved in directory jugdata in files with names such as jugdata/5/4/a1266debc307df7c741cb7b997004f The name is simply a hash of the task description (function and its arguments).

In order to make sense of all of this, we write a final script, which loads the results and prints them on stdout:

  1. import jug
  2. jug.init('jugfile.py', 'jugdata')
  3. import jugfile
  5. results = jug.task.value(jugfile.fullresults)
  6. for p, t in results:
  7.     print("%s\n\n    Password was '%s'" % (t, p))

jug.init takes the jugfile name (which happens to be jugfile.py) and the data directory name.

jug.task.value takes a jug.Task and loads its result. It handles more complex cases too, such as a list of tasks (and returns a list of their results).