Concepts
The main objects manipulated by the platform are:
- component: a bundled piece of software that provides a set of services/functionalities. The provided software can do anything, ranging from input devices, signal-processing, networking, graphical interfaces, etc. A public repository of components has been set up to centralize all existing software and enforce easy reuse of components written by other people.
- pipeline of components: it is the interconnection schema of a set of components. It describes and sets up communication channels between components. An application is then described by a pipeline interconnecting a set of components.
Components
Components are the base objects manipulated by the OpenInterface platform. Each one represents a bundled piece of software that provides a set of services/functionalities. The provided software can do anything, ranging from input devices driver, signal-processing, networking, graphical interfaces, etc.
Components are characterized by their interfaces, that is, the data that can come inside or outside these black boxes. The access points where data can come inside or outside the components are called pins.
There are three types of pins:
- sink pins: for receiving data.
- source pins: for receiving and sending data.
- callback pins: for sending (a)synchronous events.
To be able to manipulate a component, the OpenInterface platform needs to know the description of the interface of the component. This description is made by using the CIDL description language (Component Interface Description Language - see the OpenInterface Description Languages Specification for more details). Once the CIDL description of the component has been generated, a component can be reused easily in any OpenInterface application. A component can also be reused outside the platform with no furhter modification.
A public repository of reusable OpenInterface components has been set up on the forge.openinterface.org web site, so that anyone can easily reuse the various components, implementing some modalities and other useful tools, that are being contributed by the multimodal community and other users of the OpenInterface Platform.
Connectors
Connectors are used to link components together, they encapsulate generic features such as data transformation, filtering, communication protocols, etc. The name connector does not restrict to communication oriented (OSC, TCP, UDP, Multicast, etc) functionnalities but also includes generic features such as data fusion, data dispatching, etc. Because of their generic goal, connectors cannot have fixed CIDL description, their description is built on the fly whenever the inputs and outputs are connected. This is why a connector must be implemented within kernel to benefit from the genericity of the platform architecture.
Pipelines
OpenInterface components can be composed together to create a network of components managing some advanced task. Such an inter-connection of components is called a pipeline.
Components can be composed in pipelines by connecting together sinks and sources of various components. In this way, data produced by callback pins of some component will be sent to sink pins of another component, letting the components exchanging data and events. By using this mechanism you can create complex application pipelines performing advanced interaction tasks or techniques.
Heterogeneous Component Composition
A great feature of the OpenInterface platform is that the components used in a same application can be heterogeneous, in the sense that they can be written in different programming languages. The OpenInterface platform will manage the translation and the communication of the data among the different programming languages using existing tools. The OpenInterface platform currently supports C/C++, Java and Matlab, but support of other languages can be added rather easily.
Pipeline Description and Configuration Language
In order to be manipulated by the OpenInterface platform, a pipeline must be described in the PDCL description language (Pipeline Description and Configuration Language - see the OpenInterface Description Languages Specification for more details). A PDCL description defines the components that are used in the pipeline and how they are connected together. A pipeline can then be executed by using the OpenInterface platform, that will read the PDCL description of the pipeline, initialize its various components, and establish the communication between them.
Pipeline Manipulation
Pipelines can be created and manipulated easily by using the OpenInterface Design&Development Environment: SKEMMI. Once a pipeline has been created, it is then easy to modify it and exchange some components by other ones, allowing for instance to rapidly explore and evaluate designed multimodal interactions.