Observer

The Observer is a versatile library implemented in C++ to add obsersability and steering capabilities to any soft-real time system generated from a UML-RT model created in the Papyrus-RT tool. It supports the following features:

• Observation of the execution of soft real-time systems modelled in UML-RT through Model-to-Model (M2M) transformations (using Epsilon)
• Steering of the execution
• Trace generation with customizable format of traces
• Possibility to connect multiple monitors seamlessly
• Traces can be transmitted to monitors instantaneously or periodically
• Various formats of communication supported (TCP/IP, Shared Memory, MQTT, ...)
• Various formats of serialization supported (Text, XML, ...)
• Easily extensible to add new formats of communication/serialization
• Instrumentation of a system can be done manually or automatically through a Graphical User interface (GUI) in Papyrus-RT
• Adaptable to UML models with minor changes to the Observer library
• Adaptable to non-modelled systems (C++ library used in standalone)

Table of Contents

1. Basic Set-up
2. Installation
3. Getting Started
4. Advanced Configuration
5. List of Available Add-ons
6. Creating New Add-ons
7. Extending the Observer
8. Using the Observer library for (Non-Modelling) Standalone Projects (C++)
9. Updating the Observer

Basic Set-up

Prior any use, a working version of Papyrus-RT 1.0 user version along with Epsilon 1.6 (Interim) must be installed.

**Note: **Papyrus-RT 1.0 user version only works with Java 8 64-bit. If you are running Ubuntu and have a more recent version of Java installed, you can install Java 8 along with your current Java version using the following commands: $ sudo apt-get install openjdk-8-jre -y{:.bash} followed by $ sudo update-alternatives --config java{:.bash}

To install Epsilon, proceed the following steps:

• Open Papyrus-RT
• Add the following update site: http://download.eclipse.org/epsilon/interim/
• Select the Epsilon Interim update site and install all features of the three categories: Epsilon Core, EMF integration, and UML integration
• Complete the installation and restart Papyrus-RT

Installation

Installing the Observer library can be done using two methods:

• Automatically by using the Observer Eclipse update site
• Manually by importing the Observer plugins onto your Eclipse Workspace

Method 1: Using The Observer Update Site

• If not already done, open Papyrus-RT
• Add the following update site: http://mase.cs.queensu.ca/observer
• Select the Observer update site using the Work with field
• Click on Select All to select all features of the three categories
• Complete the installation and restart Papyrus-RT

Note: It is not mandatory to install all communication and serialization add-ons, but at least one of each category must be installed in order to be able to generate code. For example, for using only TCP/IP communication and text serialization, you have to select Observer feature, TCP/IP Socket Support Add-on, and Text Support Add-on Feature.

Method 2: Manually Importing the Plug-ins into the Workspace

The Git repository is structured into the following directories:

• Observer/ Contains the core plug-ins used by the Observer along with its add-ons
• Models/ Contains models to which the Observer can be applied
• Monitors/ Contains code examples for monitoring the observed models
• Development/ Contains development code. This is only useful if you plan to create your own custom version of the Observer
• experiments/ Some experiments we made
• Features/ Contains the different Eclipse features and update site

If not already done, open Papyrus-RT. You can now manually import the Observer plug-ins in your Workspace. At least the following plug-ins must be retrieved from the git repository:

• ca.queensu.cs.observer:Contains the Observer UML library
• ca.queensu.cs.observer.method.socket: TCP/IP communication support
• ca.queensu.cs.observer.serializer.text: Add-on to serialize events generated by the Observer in pure text format.
• ca.queensu.cs.observer.ui: Contains the Epsilon EOL transformation rules used to instrument a UML-RT model

Note: As for the previous method, feel free to import the other plug-ins to add alternative communication and serialization capabilities.

Getting Started

The following section will guide you through the process of generating observable code and executing your system along with monitors to observe its execution.

It is decomposed into the following steps:

1. Generate observable code
2. (optional) Filter the capsules to observe
3. Compile and execute the system
4. Observer the system execution
5. Steer the system execution

In the following, we assume that you installed the TCP/IP add-on along with the text serialization format. For this tutorial, we will use the model located under Models/ParcelRouter of the Git repository.

Step 1: Generate Observable Code

As for the installation, two methods are possible to generate observable code:

• The quickest method is through the Graphical User Interface (GUI)
• The other method is through an Epsilon EOL Program Configuration

Method a) Using the Graphical User Interface (GUI)

The quickest method to use the Observer to generate Observable code is using the Graphical User Interface (GUI) provided with the Observer library.

• Import the model located under Models/ParcelRouter in your workspace
• Open the model (in the Project Explorer view, open ParcelRouter/model). A Papyrus graphical editor will be opened
• (optional) In the Model Explorer view, right-click on RootElement and under Observer, select the communication method and serialization format you want to use
• In the Model Explorer view, right-click on RootElement and select Generate Observable Code

Note: A common mistake is to confuse the Model Explorer view with the Project Explorer view. Make sure your perform the two last steps in the Model Explorer view and not in the Project Explorer view. The Model Explorer view should be located below the Project Explorer view. If you do not see it, make you you have the Papryus perspective opened. If you still do not see it, restart the Papyrus perspective by right-clicking on the Papyrus icon on the top right corner, then choose Reset.

A modal window will open when the instrumented code of the system is generated. The source code is located under a newly created project named ParcelRouter_CDTProject under a src folder.

Method b) Using an Epsilon EOL Program Configuration (Manual Configuration)

• Import the model located under Models/ParcelRouter in your workspace
• Create an Epsilon EOL Target Configuration :
• Select Run/Run Configurations
• Double-click on EOL Program to create a new configuration
• In the Source tab, browse to select /ca.queensu.cs.observer.ui/EOLScripts/UMLRTObserverInstrumentation.eol
• In the Model tab, add a new Epsilon Model by clicking on Add, then:
  • Select UML Model. If you do not see this option, it is likely that you have not installed the Epsilon UML Integration feature ;
  • Set the values of both Name and Aliases to UMLRTModel ;
  • For the Model file field, browse to select /ParcelRouter/model.uml ;
  • Leave all other fields as set by default ;
• In the Parameters tab, create the following variables:

Name	Description	Format	Examples
observerPath	Set the path to the Observer UML library. It must be prefixed by any valid Eclipse URI scheme (file:, platform:/resource, and platform:plugin)	String	file:<ObserverPath>/libraries/observer.uml for absolute URI platform:resource/ca.queensu.cs.observer/libraries/observer.uml for Eclipse-based URI
method_src	Absolute path to the source file of the method to be used	String	[absolutePathTo_ca.queensu.cs.observer.methods.socket]/cpplib/src/MethodImpl.cc to use the TCP/IP Socket communication add-on
method_include	Absolute path to the include file of the method to be used	String	[absolutePathTo_ca.queensu.cs.observer.methods.socket]/cpplib/include/MethodImpl.hh to use the TCP/IP Socket communication add-on
serializer_src	Absolute path to the source file of the serializer to be used	String	[absolutePathTo_ca.queensu.cs.observer.serializers.text]/cpplib/src/SerializerImpl.cc to use the text serialization add-on
serializer_include	Absolute path to the include file of the serializer to be used	String	[absolutePathTo_ca.queensu.cs.observer.serializers.text]/cpplib/include/SerializerImpl.hh to use the text serialization add-on
unobserved_capsules	Coma-separated list of capsule names that will be ignored during the instrumentation process	String	Gen,Sensor

• Hit Run

The Eclipse console should output the following content:

Number of capsules: 8
Number of states: 14

--
Start (re-)building
-> Load observer
-> Add the Observer capsule part
Done.

-> Add the serializer artifact
Done.

-> Add the method artifact
Done.

-> Instrumentation of capsule: Switcher
-> Instrumentation of capsule: Bin
-> Instrumentation of capsule: Chute
-> Instrumentation of the Observer capsule
Done.
--

• Open the model (in the Project Explorer view, open ParcelRouter/model). A Papyrus graphical editor will be opened
• In the Model Explorer view, right-click on RootElement and select (Re)Generate Code

Notes:

• If during set-up you selected other serialization and communication methods, you should reflect it when setting the Epsilon variables in the Parameters tab
• The observerPath variable is an Eclipse-compliant URI that can be defined following an available URI scheme:
  • The file:/ URI scheme is used to refer to absolute files on your Hard drive
  • The platform:/resources/plugin-name URI scheme is used to refer to a plug-in imported in you Eclipse workspace
  • <PATH_TO_GIT_REPOSITORY> should be replaced with the absolute path containing your Git repository
• This method directly affects the model and therefore, the diagram can be messed up.

Step 2: (Optional) Filter the Capsules to Observe

Optionally, you can reduce the set of trace events you want to observer by reducing the set of observable capsules. This can be done through the Model Explorer.

• Unfold the RootElement item in the Model Explorer to reveal all capsules of the system
• Right-click on the capsule you do not want to observe and select Observer/Un-observe

A similar configuration is possible when instrumenting the model manually using Epsilon EOL (see step 1).

Note: there is currently no visual feedback on the diagrams nor the Model Explorer to see which capsules are observed and which are not. To see whether a capsule is observed or not, redo the two steps above and see whether Un-observe or Observer is disabled. Visual feedback should be implemented in a future release.

Step 3: Compile and Execute the System

You should now have a project called ParcelRouter_CDTProject created in you workspace.

• Open a terminal and navigate into ParcelRouter_CDTProject/src
• Compile the program: $ make
• Execute the code: $ ./Parcel_RouterMain

At this point, the Parcel Router executes in a loop and routes parcels randomly generated to some bins based on the parcels' tags. What actually the instrumented program is doing, it:

1. Creates a TCP/IP server with a default port set to 8080 (if you selected the socket communication method) ;
2. Registers all capsules of the system. This action is used for steering the model (see below) ;
3. For each event generated by the model, an Observer capsule added to the model will forward it to any external monitoring tools supporting TCP/IP and listening to the port8080

Step 4: Observe the System Execution

To retrieve the events generated by the Observer capsule (and to test that everything works by the same occasion), you can create a TCP/IP client connecting to port 8080. Alongwith the implementation of the Observer, we provide two monitors:

• ParcelRouterAnimation: a GTK animation animating the execution of the Parcel Router system through the Observer capsule
• TCPClient: a simple TCP/IP client to read from and write into TCP/IP socket communication.

Both monitors are located under the Monitors directory. For Steps 4 and 5, we only focus on the TCPClient monitor

• If not already done, import the TCPClient plug-in into your workspace.
• From the Project Explorer, right-click on the TCPClient plug-in and select Run As/Java Application
• If you have stopped the execution of the Parcel Router system from Step 2, relaunch it: $ ./Parcel_RouterMain
• On the TCPClient, click on Connect
• Check that traces are generated.

Step 5: Steer the System Execution

To see how to steer the execution, first make a subtle change to the generated code source from Step 3:

• Open a terminal and navigate into ParcelRouter_CDTProject/src
• Open Gen.cc with your favorite editor
• Comment line 174: timer.informEvery(UMLRTTimespec(4,0));

Note: The above steps are not necessary, but it simplifies the visualization of the effect of steering the system execution. Indeed, we saw on Step 4 that the system generates a lot of traces and it will be difficult to distinguish traces generated by the steering from traces generated by the normal execution of the system. Concretly, the above modification just prevents the parcel router from generating a parcel every four seconds.

Note 2: Off course, the normal way would have been to make the change in the model itself and (re-)generate the code ;) However, since this tutorial is not an introduction to UML-RT and Papyrus-RT, we did not want to overload the already dense content.

• Compile the code again and execute the system again
• Open again TCPClient and connect again to port 8080
• In the field below the list of events that are generated, enter a command and press Enter

To guarantee that events generated by the Observer and commands received by the external monitoring tools are formatted in the same serialization format, commands sent to the Observer are seen by the Observer as event coming from the external world (i.e., the monitors) rather than from the internal components of the system. The counterpart is that commands are not simply strings and must exactly comply to the format of the events generated by the Observer. We assume in the following that events are generated using the text formatting from the Text serialization format plugin (which should be the case if you followed the previous steps). That means that a command should be of the following format:

 commandId|sourceName|capsuleInstance|eventSource|eventKind|seconds|nanoseconds|cpuTick|params|

In practice, only sourceName, capsuleInstance, eventSource, eventKind, and params are read by the Observer for parsing a command from external monitoring tools. You can use any value for the other fields but you must comply with the format above unless you use another serialization format or you customized the formatting of the string).

Three commands are available:

• List: command the Observer to send the list of capsule parts currently running. This is done by the following command: 1|||9|31|0|0|0||. The output is displayed in the TCPClient monitor
• Show: give some information about a specific capsule, in particular the transitions that can be triggered from outside. The name of the capsule is passed through the sourceName field. An example of the use of this command is: 1|Gen|TcpClient|9|32|0|0|0||. The output is displaed in the TCPClient monitor. It shows that two transitions can be triggered for the Gen capsule: enter_setFree and timer_timeout. We will trigger the second one to manually generate a parcel.
• Trigger: manually trigger a transition of a capsule. An example of use of this command is: 1|Gen|Parcer_Router.Gen:0|9|33|0|0|0|operation:timer_timeout|

If every goes well, the system will generate a parcel that will go down the multiple parcel router stages.

Advanced Configuration

Advanced configuration can be passed to the Observer through the creation of a config file located at the root of the binary obtained from Step 2 above. A sample of the file is given below:

# Main configuration

# Remove the two lines below if you want to receive traces instantaneously
periodicity=1000 # 1000 microseconds
mode=periodic

# Configuration per Add-ons
[text]
  keyValueSeparator=: # a semi-colon is used as separator between the key name and the value for each field
  format=eventId|sourceName|eventSource|eventKind|seconds|nanoseconds|params # filter the fields to generate
  paramSeparator=; # separators between parameters. It has no effect when the "params" field is not included in the format option
  separator=| # separator between fields

[socket]
  Port=8080 # listen on port 8080

# ...

Configuring the Add-ons using the GUI (beta)

Configuration pages are generated for each add-on to be able to configure the add-on in the GUI and to generate the corresponding config file.

• If not already done, open Papyrus-RT
• In the top menubar, open Window/Preferences
• Under Papyrus-RT/Observer, open the subpreference page corresponding to the Add-on you want to configure
• Configure the different options of the add-on. A preview of the config file is displayed on-the-fly

Known limitations

• The subpreference pages may not appear. In that case, you have to explicitely open the Papyrus-RT/Observer preference page, even though this one is empty
• The corresponding config is not generated for now (TODO). It must be manually created.
• It is not possible to configure the main options of the Observer (mode and periodicity) for now (TODO)

List of Available Add-ons

This section lists the different add-ons that adds serialization and communication capabilities to the Observer

Serializers

Text

Basic serialization in text. Support custom formats and custom separators.

Name	Description	Format	Default value
separator	character separator between fields	Character
format	format of the trace events	String	eventId
keyValueSeparator	character separator between the key and value of each field	Character	:
parameterSeparator	Character separator between parameters (has no effect if format does not contain "params")	Character	;

XML

(To document)

JSON

(To document)

CTF

(To document)

Communication Method

TCP/IP

TCP/IP communication support. The modelled system plays the role of master and can accept multiple clients to connect. Its only configuration is the port to establish the TCP/IP communication.

Name	Description	Format	Default value
port	Port of the TCP/IP communication	Integer	8080

MQTT

MQTT communication support.

Name	Description	Format	Default value
port	Port of the MQTT communication	Integer	1883
address	Adress of the MQTT broker	String	mqtt.jahed.ca
username	Username for normal autentication	String
password	Password for normal autentication	String
subTopic	Subscribe topic	String	observer_out
pubTopic	Publish topic	String	observer_in

Shared Memory

Shared memory communication support.

Name	Description	Format	Default value
name	Area name	String	EventArea
qName	Queue name	String	EventQ
withLock	Whether a lock is used	Boolean	true
size	Size of the queue	Integer	9999999
mode	Subscribe topic	Enumeration	Server

StandardIO

Basic IO communication supported via std::cout and std::cin. Useful for debugging. No configuration is supported.

Creating New Add-ons

Creating new add-ons for supporting other methods of serialization/communication can be easily done in Papyrus-RT. Examples of serializers and communication methods are available in the Git repository. To create a new add-on, proceed to the following steps:

• Create a new Eclipse plug-in: File/New/Project... then select Plug-in project. Complete the information of your new plug-in
• Create a cpplib directory (or whathever name you want to use) at the root of your new plug-in. This directory contains all C++ code of your new add-on. We suggest you to take examples on existing add-ons to create yours. Specially, this directory must contain:
  • MethodImpl.hh(for a new communication method) or SerializerImpl.hh (for a new serializer): this file must extend Method.hh or Serializer.hh located under the Observer C++ library.
  • MethodImpl.cc or SerializerImpl.cc for the implementation of your add-on
  • a C++ test file to test your method
• To let users of the GUI select your add-on, open the Manifest file (META-INF/MANIFEST.MG) of your Eclipse plug-in, open the Depedencies tab and add the following dependency: ca.queensu.cs.observer.ui
• Open the Extensions tab and create a new extension by selecting either the ca.queensu.cs.observer.communications or ca.queensu.cs.observer.serializations extension point.

An example of extension for the TCP/IP Socket communication is the following:

 <extension
         point="ca.queensu.cs.observer.communications">
      <communication
            cpp_include_file="cpplib/include/MethodImpl.hh"
            cpp_source_file="cpplib/src/MethodImpl.cc"
            description="This is the configuration page for the Socket serialization format"
            label="Socket"
            name="socket"
            preference_page="ca.queensu.cs.observer.ui.preferences.ObserverPreferencePage">
         <attribute
               default_value="8080"
               is_required="true"
               label="Port of the TCP connection"
               name="Port"
               type="Integer">
         </attribute>
      </communication>
   </extension>

It must include the relative path to your MethodImpl.hh (resp. SerializerImpl.hh) and MethodImpl.cc (resp. SerializerImpl.cc) files, a description, label, and name of the add-on. A default preference page ca.queensu.cs.observer.ui.preferences.ObserverPreferencePage can be used. If your plug-in is configurable, attributes can be defined and will automatically populate your preference page. If you prefer to use a custom preference page, change the value of the preference_page field.

Current limitation: The schema associated with the extension points are not provided with the build when installing from the update site. This will be fixed soon.

Extending the Observer

The Observer library has been developped from the ground up to be easily extended, whether you want to to support specific events or to support other UML-based languages. This section gives some advices for extending the Observer library.

Structure of the Observer C++ library

The Observer library is located under the ca.queensu.cs.observer.cpp C++ project. The src directory contains the following source files:

• Config.cc: used to parse config files
• Event.cc: contains the "meta-model" of an event
• Method.cc: interface used to implement communication method add-ons
• Serializer.cc: interface used to implement serializer add-ons.
• Observer.cc: a main class to use the Observer in a standalone C++ program
• ObserverTest.cc: a class test to test the Observer

Testing the Observer

A Makefile is provided for testing the Observer library. Simply open a terminal and locate the ca.queensu.cs.observer.cpp directory, then type: $ make

By default, the Observer will use the text serializer and TCP/IP (socket) communication method. Those can be overriden in the config.mk file.

Modifying the Structure of the Event Class

Some of the fields of the Event class are specific to UML-RT while others are more generic and can be used for other UML-based systems. You may want to change the structure of the Event class if:

• You want to use the Observer library for UML-RT models, but you need to add new fields to address specific concerns (e.g., power consumption)
• You want to use the Observer library outside of UML-RT and you want to get rid of some fields that are specific to UML-RT (e.g., capsule, capsule instance)

The majority of the changes you would have to do are located in the Event.hh and Event.cc files. Modifying the files may lead to incompatibility with existing add-ons.

Upon the addition or the removal of some fields, the setField() and getField() functions must be updated. They are responsible for setting/getting the fields in a text format, so that serializers can used the fields agnostically of their original types.

Important Note: Change performed to the Observer C++ library are not automatically reflected to the UML-RT library used by Papyrus-RT. TO reflect those change, you must update the UML-RT model of the Observer located under ca.queensu.cs.observer/libraries. This can be done automatically thanks to an updated that is available under the Development directory of the repository. See this section.

Modifying the Generation of Events

Epsilon EOL is responsible for transforming a UML-RT model of a system into an observer version of the same model so as to support the observation and the steering of the system execution.

If you want to change the supported taxonomy of events generated by the Observer capsule, you need to update the EOL transformation rules. Those rules are located under ca.queensu.cs.observer.ui/EOLScripts. The entry point of the rules is located in the file named UMLRTObserverInstrumentation.eol.

Using the Observer library for (Non-Modelling) Standalone Projects (C++)

The Observer library being written in C++, you can choose to use it outside of Papyrus-RT. It can be used for any C++ projects requiring observation capabilities. You can also use it with any modelling languages that outputs C++ code for a modelled system.

Updating the Observer

If you plan to make some changes to the Observer C++ library, those changes are not reflected on the Observer library. To do it, yo have to proceed the following steps:

[TODO]