PyPy’s Configuration Handling

Due to more and more available configuration options it became quite annoying tohand the necessary options to where they are actually used and even moreannoying to add new options. To circumvent these problems configurationmanagement was introduced. There all the necessary options are stored in aconfiguration object, which is available nearly everywhere in the RPythontoolchain and in the standard interpreter so that adding new options becomestrivial. Options are organized into a tree. Configuration objects can becreated in different ways, there is support for creating an optparse commandline parser automatically.

Main Assumption

Configuration objects are produced at the entry points and handed down towhere they are actually used. This keeps configuration local but availableeverywhere and consistent. The configuration values are created using thecommand line.

API Details

The handling of options is split into two parts: the description of whichoptions are available, what their possible values and defaults are and how theyare organized into a tree. A specific choice of options is bundled into aconfiguration object which has a reference to its option description (andtherefore makes sure that the configuration values adhere to the optiondescription).This splitting is remotely similar to the distinction between types andinstances in the type systems of the rtyper: the types describe what sort offields the instances have.

The Options are organized in a tree. Every option has a name, as does everyoption group. The parts of the full name of the option are separated by dots:e.g. config.translation.thread.

Description of Options

All the constructors take a name and a doc argument as first argumentsto give the option or option group a name and to document it. Most constructorstake a default argument that specifies the default value of the option. Ifthis argument is not supplied the default value is assumed to be None.Most constructorsalso take a cmdline argument where you can specify what the command lineoption should look like (for example cmdline=”-v –version”). If cmdline isnot specified a default cmdline option is created that uses the name of theoption together with its full path. If None is passed in as cmdline thenno command line option is created at all.

Some options types can specify requirements to specify that a particular choicefor one option works only if a certain choice for another option is used. Arequirement is specified using a list of pairs. The first element of the pairgives the path of the option that is required to be set and the second elementgives the required value.

OptionDescription

This class is used to group suboptions.

init(self, name, doc, children)

children is a list of option descriptions (including OptionDescription instances for nested namespaces).

ChoiceOption

Represents a choice out of several objects. The option can also have the valueNone.

init(self, name, doc, values, default=None, requires=None, cmdline=DEFAULT)

values is a list of values the option can possibly take, requires is a dictionary mapping values to lists of of two-element tuples.

BoolOption

Represents a choice between True and False.

init(self, name, doc, default=None, requires=None, suggests=None, cmdline=DEFAULT, negation=True)

default specifies the default value of the option. requires is a list of two-element tuples describing the requirements when the option is set to true, suggests is a list of the same structure but the options in there are only suggested, not absolutely necessary. The difference is small: if the current option is set to True, both the required and the suggested options are set. The required options cannot be changed later, though. negation specifies whether the negative commandline option should be generated.

IntOption

Represents a choice of an integer.

init(self, name, doc, default=None, cmdline=DEFAULT)

FloatOption

Represents a choice of a floating point number.

init(self, name, doc, default=None, cmdline=DEFAULT)

StrOption

Represents the choice of a string.

init(self, name, doc, default=None, cmdline=DEFAULT)

Configuration Objects

Config objects hold the chosen values for the options (of the default,if no choice was made). A Config object is described by anOptionDescription instance. The attributes of the Config objects are thenames of the children of the OptionDescription. Example:

>>> from rpython.config.config import OptionDescription, Config, BoolOption
>>> descr = OptionDescription("options", "", [
...     BoolOption("bool", "", default=False)])
>>>
>>> config = Config(descr)
>>> config.bool
False
>>> config.bool = True
>>> config.bool
True

Description of the (useful) methods on Config:

init(self, descr, overrides):

descr is an instance of OptionDescription that describes the configuration object. overrides can be used to set different default values (see method override).

override(self, overrides):

override default values. This marks the overridden values as defaults, which makes it possible to change them (you can usually change values only once). overrides is a dictionary of path strings to values.

set(self, kwargs):

“do what I mean”-interface to option setting. Searches all paths starting from that config for matches of the optional arguments and sets the found option if the match is not ambiguous.

Production of optparse Parsers

To produce an optparse parser use the function to_optparse. It will createan option parser using callbacks in such a way that the config object used forcreating the parser is updated automatically.

to_optparse(config, useoptions=None, parser=None):

Returns an optparse parser. config is the configuration object for which to create the parser. useoptions is a list of options for which to create command line options. It can contain full paths to options or also paths to an option description plus an additional “.*” to produce command line options for all sub-options of that description. If useoptions is None, then all sub-options are turned into cmdline options. parser can be an existing parser object, if None is passed in, then a new one is created.

The usage of config objects in PyPy

The two large parts of PyPy, the Python interpreter and the RPythontoolchain, have two separate sets of options. The translation toolchain optionscan be found on the config attribute of all TranslationContextinstances and are described in rpython/config/translationoption.py. The interpreter optionsare attached to the object space, also under the name config and aredescribed in pypy/config/pypyoption.py. Both set of options aredocumented in the Configuration Options for PyPy section.