experimentor.core package

Submodules

experimentor.core.app module

experimentor.core.data_source module

Data Source

These objects are defined in models and are meant to be used to broadcast information across different objects, either on different threads, processes, or computers. In their core, they are ZMQ Publishers and hold the necessary information in order to create a subscriber based on them.

class experimentor.core.data_source.DataSource

Bases: object

connect()

finalize()

initialize()

experimentor.core.exceptions module

exception experimentor.core.exceptions.DuplicatedParameter

Bases: experimentor.core.exceptions.ExperimentorException

exception experimentor.core.exceptions.ExperimentDefinitionException

Bases: experimentor.core.exceptions.ExperimentorException

exception experimentor.core.exceptions.ExperimentorException

Bases: Exception

exception experimentor.core.exceptions.ModelDefinitionException

Bases: experimentor.core.exceptions.ExperimentorException

experimentor.core.measurement_parameters module

Measurement Classes

Collection of classes useful for developing the logic of a measurement

Section author: Aquiles Carattino

class experimentor.core.measurement_parameters.Parameter(units=None, ui_class=None)

Bases: object

Parameters that belong to a measurement. They allow to define units, limits and ui_classes.

name = ''

experimentor.core.measurement_procedure module

class experimentor.core.measurement_procedure.Procedure(procedure)

Bases: object

Decorator to check the validity of a procedure before performing a measurement

check_parameters(cls, *args, **kwargs)

experimentor.core.meta module

class experimentor.core.meta.ExperimentorProcess(*args, **kwargs)

Bases: multiprocessing.context.Process

class experimentor.core.meta.ExperimentorThread(*args, **kwargs)

Bases: threading.Thread

class experimentor.core.meta.MetaProcess(name, bases, attrs)

Bases: type

Meta Class that should be shared by all processes in order to be sure they all switch off nicely when done.

get_instances(recursive=False)

Get all instances of this class in the registry. If recursive=True search subclasses recursively

experimentor.core.publisher module

Publisher

Publishers are responsible for broadcasting the message over the ZMQ PUB/SUB architecture. The publisher runs continuously on a separated process and grabs elements from a queue, which in turn are sent through a socket to any other processes listening.

Todo

In the current implementation, data is serialized for being added to a Queue, then deserialized by the publisher and serialized again to be sent. These three steps could be simplify into one if, for example, one assumes that objects where pickled. There is also a possibility of assuming numpy arrays and using a zero-copy strategy.

copyright:Aquiles Carattino license:MIT, see LICENSE for more details

class experimentor.core.publisher.Publisher(event, name=None)

Bases: experimentor.core.meta.ExperimentorProcess

Publisher class in which the queue for publishing messages is defined and also a separated process is started. It is important to have a new process, since the serialization/deserialization of messages from the QUEUE may be a bottleneck for performance.

run()

Start a new process that will be responsible for broadcasting the messages.

Todo

Find a way to start the publisher on a different port if the one specified is in use.

stop()

experimentor.core.publisher.start_publisher()

Wrapper function to start the publisher. It takes care of checking that there is only one publisher running by storing it in the settings.

Todo

Find a good way of starting a publisher once per measurement cycle.

experimentor.core.pusher module

Pusher

New in version 0.2.0.

Half the ZMQ implementation is abut broadcasting information from a publisher to different subscribers. However, the other half is giving information to the publisher to broadcast. We are doing this with a PUSH/PULL pattern. The pusher is therefore able to send information to the Publisher to then broadcast. There can be many instances of pushers, but only one publisher. In other words, this is a fan-in type of architecture.

class experimentor.core.pusher.Pusher(port=None)

Bases: object

The Pusher is class that wraps some common methods of the ZMQ PUSH/PULL architecture.

Warning

The main problem with this pattern is that if there is not PULL on the other side, a queue will build up on the PUSH side. This happens if, for example, we close the publisher but we keep generating data. Eventually the queue will outgrow the memory and the computer will crash.

Parameters:port (int) – The port on which to connect the PUSH end. If not specified, it will grab the default value from settings

pusher

The socket where the communication happens

Type:socket

i

The number of messages that were pushed from a given

Type:int

topic_i

Number of data frames sent on each topic. For example: topic_i[‘topic’]

Type:dict

lock

In case the same pusher is shared between different threads, this ensures the messages are sent in the proper block

Type:RLOCK

finish()

publish(data, topic='')

Publish data on a given topic. This is the core of the Pusher object.

Parameters:

• data – Data can be any Python object, provided that it is serializable
• topic (str) – The topic on which the data is being transmitted. If nothing is specified, it will be a broad transmission, meaning that every subscriber will receive it.

experimentor.core.signal module

class experimentor.core.signal.Signal

Bases: object

Base signal which implements the common pattern for defining, emitting and connecting a signal

emit(payload=None, **kwargs)

Emitting a signal relies on the owner’s publisher or whatever method the owner specifies for broadcasting. In principle this allows for some flexibility in case owners use different ways of broadcasting information. For example, the owner could be a QObject and it could use the internals of Qt to emitting signals.

url

experimentor.core.subscriber module

Subscriber

Example script on how to run separate processes to process the data coming from a publisher like the one on publisher.py. The first process just grabs the frame and puts it in a Queue. The Queue is then used by another process in order to analyse, process, save, etc. It has to be noted that on UNIX systems, getting from a queue with Queue.get() is particularly slow, much slower than serializing a numpy array with cPickle.

class experimentor.core.subscriber.Subscriber(func, url, topic)

Bases: threading.Thread

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

stop()

experimentor.core.subscriber_process module

Subscriber

Example script on how to run separate processes to process the data coming from a publisher like the one on publisher.py. The first process just grabs the frame and puts it in a Queue. The Queue is then used by another process in order to analyse, process, save, etc. It has to be noted that on UNIX systems, getting from a queue with Queue.get() is particularly slow, much slower than serializing a numpy array with cPickle.

Warning

This is work in process. On Windows, since processes are spawned, the subscriber would not work as expected. That is why we work with Threads instead.

class experimentor.core.subscriber_process.Subscriber(func, topic, publish_topic=None, args=None, kwargs=None)

Bases: experimentor.core.meta.ExperimentorProcess

run()

Method to be run in sub-process; can be overridden in sub-class

stop()

Module contents