jina.serve.runtimes.base module

class jina.serve.runtimes.base.BaseRuntime(args, \*kwargs*)[source]

Bases: object

A Jina Runtime is a procedure that blocks the main process once running (i.e. run_forever()), therefore should be put into a separated thread/process, or inside the main process of a docker container.

Any program/library/package/module that blocks the main process, can be formulated into a BaseRuntime class and then be started from a Pod.

In the sequel, we call the main process/thread as M, the process/thread blocked Runtime as S.

In Jina, a Pod object is used to manage a Runtime object’s lifecycle. A Pod acts as a multiprocessing.Process or threading.Thread, it starts from M and once the S is spawned, it uses Runtime as a context manager:

1. __init__()

1. meth

   __enter__

2. run_forever(). Note that this will block S, step 3 won’t be reached until it is unblocked by cancel().

3. When an error occurs during run_forever or cancel signal is reached by the runtime. The run_forever method is cancelled and the managed context is closed. The __exit__ of Runtime guarantees that the Runtime is properly shut by calling teardown.

The __init__() and teardown() pair together, which defines instructions that will be executed before and after. In subclasses, teardown is optional.

In order to cancel the run_forever method of a Runtime, you can use their static cancel method that will make sure that the runtime is properly cancelled.

• Use threading.Event or multiprocessing.Event, while run_forever() polls for this event

• Use GrpcConnectionPool to send a TERMINATE message, while run_forever() polls for this message

Note, another way to jump out from run_forever() is raise exceptions from it. This will immediately move to teardown().

Note

Rule of thumb on exception handling: if you are not sure if you should handle exception inside run_forever(), cancel(), teardown(), then DO NOT catch exception in them. Exception is MUCH better handled by Pod.

See also

Pod for managing a Runtime object’s lifecycle.

• run_forever()[source]

  Running the blocking procedure inside S. Note, once this method is called, S is blocked.

  Note

  If this method raises any exception, teardown() will be called.

  See also

  cancel() for cancelling the forever loop.

• teardown()[source]

  Method called immediately after run_forever() is unblocked. You can tidy up things here. Optional in subclasses. The default implementation does nothing.