experimentor.models package

Submodules

experimentor.models.action module

Action

An action is an event that gets triggered on a device. For example, a camera can have an action acquire or read. They should normally be associated with the pressing of a button. Action is a handy decorator to register methods on a model and have quick access to them when building a user interface. They are multi-threaded by default, however, they share the same executor, defined at the model-level. Therefore, if a device is able to run several actions simultaneously, different executors can be defined at the moment of Action instantiation.

To extend Actions, the best is to sub class it and re implement the get_executor method, or any other method relevant to change the expected behavior.

Examples

A general purpose model can implement two methods: initialize and auto_calibrate, we can use the Actions to increment their usability:

class TestModel:
    @Action
    def initialize(self):
        print('Initializing')

    @Action
    def auto_calibrate(self):
        print('Auto Calibrating')

tm = TestModel()
tm.initialize()
tm.auto_calibrate()
print(tm.get_actions())
license:MIT, see LICENSE for more details
copyright:2020 Aquiles Carattino
class experimentor.models.action.Action(method=None, **kwargs)[source]

Bases: object

Decorator for methods in models. Actions are useful when working with methods that run once, and are normally associated with pressing of a button. Actions are multi-threaded by default, using a single executor that returns a future.

Even though Actions (intended as the method in a model) can take arguments, it may be a better approach to store the parameters as attributes before triggering an action. In this way, triggering an action would be equivalent to pressing a button. In the same way, actions can store return values as attribute in the model itself, avoiding the need to keep track of the future returned by the action. Be aware of potential racing conditions that may arise when using shared memory to exchange information.

Todo

Define a clear protocol for exchanging information with models. Should it be state-based (i.e. storing parameters as attributes in the class) or statement based (i.e. passing parameters as arguments of methods).

get_executor()[source]

Gets the executor either explicitly defined as an argument when instantiating the Action, or grabs it from the parent instance, and thus is shared between all action in a model.

To change the behavior, subclass Action and overwrite this method.

get_lock()[source]

Gets the lock specified in the keyword arguments while creating the Action, or defaults to the lock stored in the instance and thus shared between all actions in the model.

Deprecated since version 0.3.0: Since v0.3.0 we are favoring concurrent.futures instead of lower-level threading for Actions.

get_run()[source]

Generates the run function that will be applied to the method. It looks a big convoluted, but it is one of the best approaches to make it easy to extend the Actions in the longer run. The return callable grabs the executor from the method self.get_executor().

Returns:A function that takes two arguments: method and instance and that submits them to an executor
Return type:callable
set_action(method)[source]

Wrapper that returns this own class but initializes it with a method and a previously stored dict of kwargs. This method is what happens when the Action itself is defined with arguments.

Parameters:method (callable) – The method that is decorated by the Action
Returns:Returns an instance of the Action using the previously stored kwargs but adding the method
Return type:Action

experimentor.models.decorators module

Decorators

Useful decorators for models.

license:MIT, see LICENSE for more details
copyright:2020 Aquiles Carattino
experimentor.models.decorators.avoid_repeat(func)[source]
experimentor.models.decorators.make_async_thread(func)[source]

Simple decorator to make a method run on a separated thread. This decorator will not work on simple functions, since it requires the first argument to be an instantiated class (self). It will store the method in an attribute of the class, called _threads`, or it will create it if it does not exist yet.

TODO: Check what happens with the _thread list and inherited classes. Is there a risk that the list will be
shared? If the list is defined as a class attribute instead of an object attribute, most likely it will. If it is defined outside of the scope and then linked to the class, also.

Warning

In complex scenarios, this simple decorator can give raise to mistakes, i.e. objects having access to other objects threads.

experimentor.models.decorators.not_implemented(func)[source]

Raises a warning in the logger in case the method was not implemented by child classes, but it does not prevent the program from running.

experimentor.models.exceptions module

Model Exceptions

license:MIT, see LICENSE for more details
copyright:2020 Aquiles Carattino
exception experimentor.models.exceptions.ExperimentorException[source]

Bases: Exception

Base exception for all experimentor modules

exception experimentor.models.exceptions.LinkException[source]

Bases: experimentor.models.exceptions.PropertyException

exception experimentor.models.exceptions.ModelException[source]

Bases: experimentor.models.exceptions.ExperimentorException

exception experimentor.models.exceptions.PropertyException[source]

Bases: experimentor.models.exceptions.ModelException

exception experimentor.models.exceptions.SignalException[source]

Bases: experimentor.models.exceptions.ExperimentorException

experimentor.models.feature module

Features

Features in a model are those parameters that can be read, set, or both. They were modeled after Lantz Feat objects, and the idea is that they can encapsulate common patterns in device control. They are similar to Settings in behavior, except for the absence of a cache. Features do communicate with the device when reading a value.

For example, a feature could be the value of an analog input on a DAQ, or the temperature of a camera. They are meant to be part of a measurement, their values can change in loops in order to make a scan. Features can be used as decorators in pretty much the same way @propery can be used. The only difference is that they register themselves in the models properties object, so it is possible to update values either by submitting a value directly to the Feature or by sending a dictionary to the properties and updating all at once.

It is possible to mark a feature as a setting. In this case, the value will not be read from the device, but it will be cached. In case it is needed to refresh a value from the device, it is possible to use a specific argument, such as None. For example:

@Feature(setting=True, force_update_arg=0)
def exposure(self):
    self.driver.get_exposure()

@exposure.setter
def exposure(self, exposure_time):
    self.driver.set_exposure(exposure_time)

Todo

It is possible to define complex behavior such as unit conversion, limit checking, etc. We should narrow down what is appropriate for a model and what should go into the Controller.

Todo

A useful pattern is to catch the exception raised by the controllers if a value is out of range, or with the wrong units.

license:MIT, see LICENSE for more details
copyright:2020 Aquiles Carattino
class experimentor.models.feature.Feature(fget=None, fset=None, fdel=None, doc=None, **kwargs)[source]

Bases: object

Properties that belong to models. It makes easier the setting and getting of attributes, while at the same time it keeps track of the properties of each model. A Feature is, fundamentally, a descriptor, that extends some functionality by accepting keyword arguments when defining.

Todo

There is a lot of functionality that can be implemented, but that hasn’t yet, such as checking limits, unit conversion, etc.

name

The name of the feature, it must be unique since it will be used as a key in a dictionary.

Type:str
kwargs

If the feature is initialized with arguments, they will be stored here. Only keyword arguments are allowed.

Type:dict
deleter(fdel)[source]
getter(fget)[source]
kwargs = None
name = ''
setter(fset)[source]

experimentor.models.meta module

Meta Models

license:MIT, see LICENSE for more details
copyright:2020 Aquiles Carattino
class experimentor.models.meta.MetaModel(name, bases, attrs)[source]

Bases: type

Meta Model type which will be responsible for keeping track of all the created models in the program. This is very useful for things like automatically building a GUI, initializing/finishing all the devices, etc. and also to perform checks at the beginning of the runtime, by doing introspection on all the defined models, regardless of whether they are instantiated later on or no.

One of the tasks is to generate a list of signals available in each model. Signals are specified as class attributes and therefore they can be accounted for before instantiating the class. Once the class is being instantiated, each object will re-instantiate the signals in order to keep its own copy, and establishing the proper owner of the signal.

get_instances(recursive=False)[source]

Get all instances of this class in the registry.

Parameters:recursive (bool) – Search for instances recursively through inherited objects
get_models(recursive=False)[source]

Gets all the models which share the MetaModel origin.

Parameters:recursive (bool) – Search recurisvely in sub classes of the model

experimentor.models.models module

Models

Models are a buffer between user interactions and real devices. Models should define at least some basic common properties, for example how to read a value from a sensor and how to apply a value to an actuator. Models can also take care of manipulating data, for example calculating an FFT and returning it to the user.

license:MIT, see LICENSE for more details
copyright:2020 Aquiles Carattino
class experimentor.models.models.BaseModel[source]

Bases: object

All models should inherit from this base model. It defines some basic methods and checks that prevent errors later at runtime.

_features

Dictionary-like object to store the properties of the model

Type:ExpDict
_actions

List-like object to store the available actions. It also stores a lock to prevent multiple actions to be triggered at the same time

Type:ExpList
_settings

Dictionary-like object where the settings are stored. This dictionary is also used to retrieve the latest known value of the setting.

Type:ExpDict
_signals

Dictionary-like object to store the signals of the model

Type:ExpDict
_subscribers

Dictionary-like object storing the subscribers to different signals arising from this model

Type:ExpDict
classmethod as_process(*args, **kwargs)[source]

Instantiate the model as a ProxyObject that will run on a separate process.

Warning

This is WORK IN PROGRESS and will remain so for the foreseeable future.

clean_up_threads()[source]

Keep only the threads that are alive.

create_context()[source]

Creates the ZMQ context. In case of wanting to use a specific context (perhaps globally defined), overwrite this method in the child classes. This method is called during the model instantiation.

create_publisher()[source]

Creates a ZMQ publisher. It will be used by signals to broadcast their information. There is a delay before returning the publisher to guarantee that it was properly initialized before actually using it.

Returns:
  • zmq.Publisher – Returns the initialized publisher
  • .. todo:: This method has a high chance of being converted to an Action in order to let it run in parallel
emit(signal_name, payload, **kwargs)[source]

Emits a signal using the publisher bound to the model. It uses the method BaseModel.get_publisher() to get the publisher to use. You can override that method in order to use a different publisher (for example, an experiment-based publisher instead of a model-based one.

Notes

If subscribers are too slow, a queue will build up on the publisher, which may lead to the model itself crashing. It is important to be sure subscribers can keep up.

Parameters:
  • signal_name (str) – The name of the signal is used as a topic for the publisher. Remember that in PyZMQ, topics are filtered on the subscriber side, therefore everything is always broadcasted broadly, which can be a bottleneck for performance in case there are many subscribers.
  • payload – It will be sent by the publisher. In case it is a numpy array, it will use a zero-copy strategy. For the rest, it will send using send_pyobj, which serializes the payload using pickle. This can be a slow process for complex objects.
  • kwargs – Optional keyword arguments to make the method future-proof. Rigth now, the only supported keyword argument is meta, which will append to the current meta_data being broadcast. For numpy arrays, metadata is a dictionary with the following keys: numpy, dtype, shape. For non-numpy objects, the only key is numpy. The submitted metadata is appended to the internal metadata, therefore be careful not to overwrite its keys unless you know what you are doing.
finalize()[source]

Finalizes the model. It only takes care of closing the publisher. Child classes should implement their own finalize methods (they get called automatically), and either close the publisher explicitly or use this method.

classmethod get_actions()[source]

Returns the list of actions stored in the model. In case this behavior needs to be extended, the method can be overwritten in any child class.

get_context()[source]

Gets the context. By default it is stored as a ‘private’ attribute of the model. Overwrite this method in child classes if there is need to extend functionality.

Returns:The context created with self.create_context()
Return type:zmq.Context
classmethod get_features()[source]

Returns the dict-like features of the model. If this behavior needs to be extended, the method can be overwritten by any child class.

get_publisher()[source]

Returns the publisher stored as a private attribute, and initialized during instantiation of the model. Consider overwriting it in order to extend functionality.

get_publisher_port()[source]

ZMQ allows to create publishers that bind to an available port without specifying which one. This flexibility means that we should check to which port the publisher was bound if we want to use it. See self.create_publisher() for more details.

Returns:The port to which the publisher is bound. A string of integers
Return type:str
get_publisher_url()[source]

Each publisher can run on a different computer. This method should return the URL in which to connect to the publisher.

Todo

Right now it only returns localhost, this MUST be improved

initialize()[source]
classmethod set_actions(actions)[source]

Method to store actions in the model. It is a convenience method that can be overwritten by child classes.

subscribers
class experimentor.models.models.ExpDict[source]

Bases: dict

class experimentor.models.models.ExpList[source]

Bases: list

lock = <Lock(owner=None)>
class experimentor.models.models.ProxyObject(cls, *args, **kwargs)[source]

Bases: object

Creates an object that can run on a separate process. It uses pipes to exchange information in and out. This is experimental and not meant to be used in a real application. It is here as a way of documenting one of the possible directions.

Note

Right now we are using the multiprocessing pipes to exchange information, it would be useful to use the zmq options in order to have a consistent interface through the project.

experimentor.models.properties module

Properties

Every model in Experimentor has a set of properties that define their state. A camera has, for example, an exposure time, a DAQ card has a delay between data points, and an Experiment holds global parameters, such as the number of repetitions a measurement should take.

In many situations, the parameters are stored as a dictionary, mainly because they are easy to retrieve from a file on the hard drive and to access from within the class. We want to keep that same approach, but adding extra features.

Features of Properties

Each parameter stored on a property will have three values: new_value, value, old_value, which represent the value which will be set, the value that is currently set and the value that was there before. In this way it is possible to just update on the device those values that need updating, it is also possible to revert back to the previously known value.

Each value will also be marked with a flag to_update in case the value was changed, but not yet transmitted to the device. This allows us to collect all the values we need, for example looping through a user interface, reading a config file, and applying only those needed whenever desired.

The Properties have also another smart feature, achieved through linking. Linking means building a relationship between the parameters stored within the class and the methods that need to be executed in order to get or set those values. In the linking procedure, we can set only getter methods for read-only properties, or both methods. A general apply function then allows to use the known methods to set the values that need to be updated to the device.

Future Roadmap

We can consider forcing methods to always act on properties defined as new/known/old in order to use that information as a form of cache and validation strategy.

license:MIT, see LICENSE for more details
copyright:2021 Aquiles Carattino
class experimentor.models.properties.Properties(parent: experimentor.models.models.BaseModel, **kwargs)[source]

Bases: object

Class to store the properties of models. It keeps track of changes in order to monitor whether a specific value needs to be updated. It also allows to keep track of what method should be triggered for each update.

all()[source]

Returns a dictionary with all the known values.

Returns:properties – All the known values
Return type:dict
apply(property, force=False)[source]

Applies the new value to the property. This is provided that the property is marked as to_update, or forced to be updated.

Parameters:
  • property (str) – The string identifying the property
  • force (bool (default: False)) – If set to true it will update the propery on the device, regardless of whether it is marked as to_update or not.
apply_all()[source]

Applies all changes marked as ‘to_update’, using the links to methods generated with :meth:~link

Links the properties defined as ModelProp in the models using their setters and getters.

fetch(prop)[source]

Fetches the desired property from the device, provided that a link is available.

fetch_all()[source]

Fetches all the properties for which a link has been established and updates the value. This method does not alter the to_update flag, new_value, nor old_value.

classmethod from_dict(parent, data)[source]

Create a Properties object from a dictionary, including the linking information for methods. The data has to be passed in the following form: {property: [value, getter, setter]}, where getter and setter are the methods used by :meth:~link.

Parameters:
  • parent – class to which the properties are attached
  • data (dict) – Information on the values, getter and setter for each property
get_property(prop)[source]

Get the information of a given property, including the new value, value, old value and if it is marked as to be updated.

Returns:prop – The requested property as a dictionary
Return type:dict

Link properties to methods for update and retrieve them.

Parameters:linking (dict) –

Dictionary in where information is stored as parameter=>[getter, setter], for example:

linking = {'exposure_time': [self.get_exposure, self.set_exposure]}

In this case, exposure_time is the property stored, while get_exposure is the method that will be called for getting the latest value, and set_exposure will be called to set the value. In case set_exposure returns something different from None, no extra call to get_exposure will be made.
to_update()[source]

Returns a dictionary containing all the properties marked to be updated.

Returns:props – all the properties that still need to be updated
Return type:dict

Unlinks the properties and the methods. This is just to prevent overwriting linkings under the hood and forcing the user to actively unlink before linking again.

Parameters:unlink_list (list) – List containing the names of the properties to be unlinked.
update(values: dict)[source]

Updates the values in the same way the update method of a dictionary works. It, however, stores the values as a new value, it does not alter the values stored. For updating the proper values use self.upgrade().

After updating the values, use self.apply_all() to send the new values to the device.

upgrade(values, force=False)[source]

This method actually overwrites the values stored in the properties. This method should be used only when the real values generated by a device are known. It will change the new values to None, it will set the value to value, and it will set the to_update flag to false.

Parameters:
  • values (dict) – Dictionary in the form {property: new_value}
  • force (bool) – If force is set to True, it will create the missing properties instead of raising an exception.

Module contents