Overview

The OpenInterface platform is a set of tools aimed at the rapid development and prototyping of multimodal applications. By using it, you can easily and rapidly extend your application with new input and output modalities, and explore multimodal input configurations.

OpenInterface Platform Overview

OpenInterface embeds a set of pure and combined modalities as reusable components and generic mechanisms for combining modalities.

  • At design-time, for specifying the multimodal interaction of a particular interactive system under design, the designer can reuse and assemble OI components. From this pipeline, the code of the interaction part of the system is executed. The platform therefore allows rapid development of multimodal interaction by assembling components. It is a central tool for supporting a truly iterative User-Centered Design (UCD) process by enabling the rapid development of early high fidelity prototypes for exploration, prototypes of different design options, and testbeds for experimental evaluation.

  • At run-time, the platform includes mechanisms for dynamic reconfiguration of the components in order to support dynamic adaptation of the interaction through rewiring and reconfiguration, etc.

 

OpenInterface Workbench Overview

OpenInterface Runtime Platform: OIKernel

The OpenInterface Runtime allows integrating heterogeneous software code (C/C++, Java, Matlab, C#) in a single application with minimal programing effort. The Kernel is implemented in C++ to optimize performance, as well as to benefit from existing C++ bindings available for other languages. This allows  to provide a portable platform capable of integrating heterogeneous components in a single application. An overview of how each tools of the workbench interact with others is illustrated in the figure above, where each component is registered into the OpenInterface platform using the CIDL (Component Interface Description Language). The platform then automatically generates proxies to perform the actual integration. Using SKEMMI or the (multi-language) OIKernel API, users can edit the components properties and compose an execution pipeline of a multimodal application. This execution pipeline is either interpreted at once by the kernel, or dynamically built by the application.

The use of the OpenInterface Platform requires the following two steps:

  1. Integrate new modalities, devices, and functional cores (i.e. components) into the platform. The software can be provided in any supported programming languages (C/C++, Java, Matlab and .NET; extensions can be easily added), and semi-automatic tools are provided to ease that process.
  2. Combine, evaluate, modify assembly of components as a fully functional multimodal application. To ease that design process, SKEMMI the dedicated front-end, can be used to dynamically or statically combine components, generate a running application, implement new components, etc.. External applications (i.e. not integrated within OpenInterface) can also build a running multimodal pipeline by using the provided multi-language API.

OpenInterface Design Platform: SKEMMI

SKEMMI is an extensible multimodal interaction design tool (based on Eclipse) built on top of the runtime platform kernel, which provide features such as cooperative multi-level application design and component design, implementation and tailoring. Direct manipulation and design-by-demonstration techniques are respectively natively supported by the graphical editor and by dedicated components integrated in the database.

Using this tool, technical designers and programmers can effectively collaborate on the common realization of a multimodal application using a user-centered design approach.