experimentor.models.experiments package

Submodules

experimentor.models.experiments.base_experiment module

Base Experiment Model

Base class for the experiments. BaseExperiment defines the common patterns that every experiment should have. Importantly, it starts an independent process called publisher, that will be responsible for broadcasting messages that are appended to a queue. The messages rely on the pyZMQ library and should be tested further in order to assess their limitations. The general pattern is that of the PUB/SUB, with one publisher and several subscribers.

The messages should include a topic and data. For this, the elements in the queue should be dictionaries with two keywords: data and topic. data['data'] will be serialized through the use of cPickle, and is handled automatically by pyZQM through the use of send_pyobj. The subscribers should be aware of this and use either unpickle or recv_pyobj.

In order to stop the publisher process, the string 'stop' should be placed in data['data']. The message will be broadcast and can be used to stop other processes, such as subscribers.

Todo

Check whether the serialization of objects with cPickle may be a bottleneck for performance.

license:MIT, see LICENSE for more details copyright:2020 Aquiles Carattino

class experimentor.models.experiments.base_experiment.BaseExperiment

Bases: experimentor.models.models.BaseModel

class experimentor.models.experiments.base_experiment.Experiment(filename=None)

Bases: experimentor.models.experiments.base_experiment.BaseExperiment

Base class to define experiments. Should keep track of the basic methods needed regardless of the experiment to be performed. For instance, a way to start and a way to finalize a measurement. This class is not meant to be instantiated directly, but should be subclassed in each project.

Parameters:filename (str or None) – Path to the config file that will be loaded. Ideally it should be an absolute path, to prevent problems. If you submit a relative path, it will depend on how you are running the program if the file will be found or not.

config

Properties object to store the values of the parameters of the experiments. See experimentor.models.properties to understand the options and how it works

Type:Properties

logger

The logger of the experiment, this is for internal use only

Type:logger

alive_threads

connect(method, topic, *args, **kwargs)

Async method that connects the running publisher to the given method on a specific topic.

Parameters:

• method – method that will be connected on a given topic
• topic (str) – the topic that will be used by the subscriber to discriminate what information to collect.
• args – extra arguments will be passed to the subscriber, which in turn will pass them to the function
• kwargs – extra keyword arguments will be passed to the subscriber, which in turn will pass them to the function

connections

finalize()

Needs to be overridden by child classes.

list_alive_threads

load_configuration(filename, loader=<class 'yaml.loader.SafeLoader'>)

Loads the configuration file in YAML format.

Parameters:filename (str) – full path to where the configuration file is located. Raises:FileNotFoundError – if the file does not exist.

static make_filename(folder: Union[str, tuple], filename: str)

This routine will check if the folder to store data exists, and create it if not. It will also check if the file exists, if it does, it will increase by 1 a counter until an available name appears, and return both the directory and the filename.

Parameters:

• filename – if it contains a ‘{i}’ or similar in its specification, it will use it as a counter, if not, the number will be prepended to the filename
• folder – either a string with the full path to the folder (bear in mind differences of folder separators) or a tuple that will be joined using os.path.join

num_threads

set_up()

Needs to be overridden by child classes.

start

Base signal which implements the common pattern for defining, emitting and connecting a signal

stop_subscribers()

Puts the proper data into every alive subscriber in order to stop it.

update_config(**kwargs)

class experimentor.models.experiments.base_experiment.FormatDict

Bases: dict

Simple solution to do partial formatting of strings. For example:

>>> a = 'fiber_end_{cartridge}_{i:04}.npy'
>>> cartridge = 123
>>> a.format_map(FormatDict(cartridge=cartridge))
'fiber_end_123_{i:04}.npy'

class experimentor.models.experiments.base_experiment.FormatPlaceholder(key)

Bases: object

class experimentor.models.experiments.base_experiment.MetaExperiment(name, bases, attrs)

Bases: experimentor.models.meta.MetaModel

Meta Model type which will be responsible for keeping track of all the created experiments. It will also be responsible for registering the publisher, in order to have only one throughout the program and accessible from other parts of the program. This meta class may be overkill, since in principle every program will be only one experiment, but this is left as an effort to be future-proof.

Note

Defining meta classes may generate a feeling of obscurantism in the code. It may be wise to remove it and find a simpler/straightforward approach.

Module contents