Configure process runtime

You can use the default configurations of process runtime in the conf/functions_worker.yml file.

If you want to customize more parameters, refer to the following example.

  1. functionRuntimeFactoryClassName: org.apache.pulsar.functions.runtime.process.ProcessRuntimeFactory
  2. functionRuntimeFactoryConfigs:
  3.   # the directory for storing the function logs
  4.   logDirectory:
  5.   # change the jar location only when you put the java instance jar in a different location
  6.   javaInstanceJarLocation:
  7.   # change the python instance location only when you put the python instance jar in a different location
  8.   pythonInstanceLocation:
  9.   # change the extra dependencies location:
  10.   extraFunctionDependenciesDir:

For more details, see code.

Set runtime parameter with configuration file

Pulsar Functions now supports setting runtime parameters using a configuration file in Python.

Example

You can start a Python runtime using the configuration file config.ini with the following command.

  1. pulsar-admin functions localrun \
  2.   --py /path/to/python_instance.py \
  3.   --config-file /path/to/config.ini \
  4.   --classname MyFunction \
  5.   --logging_level debug \
  6.   --inputs persistent://public/default/my-input-topic \
  7.   --output persistent://public/default/my-output-topic \
  8.   --log-topic persistent://public/default/functions-logs

--config-file is the path to the configuration file. Note that:

  • The --config-file should be written in .ini format, with each parameter being configured as key = value.

    Example

    1. [DEFAULT]
    2. logging_level = info
    3. max_pending_async_requests = 1000
    4. max_concurrent_requests = 50

  • When you set a parameter through both the configuration file and the command line, like logging_level in the example above, the value set through the command line will take precedence over the one set through the configuration file. As a result, the value of logging_level is debug.