Skip to content

Home

Jump to bottom
J.S.I edited this page Apr 9, 2024 · 16 revisions

Introdution

Over time, the official documentation for this framework will be available here.

The Digital Twinnings Stacks are a cluster of scripts that are designed to enable fast and easy data communication integration into Unity3D. While there are some packages available online, I found them unnecessarily complicated or costly. Luckily, many different communication integrations can be found online, like for example gpvigano published his own MQTT implementation (M2MqttUnity), which is partially used within DTStacks. However, there are many methods of communication which are integrated (UDP, Serial) or are scheduled to be integrated in the future (ROS#, TCP, OPC-UA).

However, by having a collection of ready-to-use elements, anyone will be able to integrate any IoT device with their Unity project!

Table of Content

  1. Home
  2. Development Environment Setup
  3. QuickStart Guide
  4. Technical Concept
  5. Reference Frames (ROS, DTStacks, Unity)
  6. Robot Operating System

Background

The origin of this framework is the hope to create an easy-to-use tool for home automation/monitoring applications. While this was developed for personal use, it was tested on an industrial setup used as an alternative to ROS# to establish a communication pipeline to unity-based apps. Mainly for ARVR-research purposes. The project was first initiated as a trial, within the EIT-MirrorLabs Project which resulted in an official dataset published by the Technical University of Delft. However, in here still ROS# was the name of game and the MQTT integration was not reliable at this stage.

By now, the framework has matured to a level where it serves its purpose neatly and can be used to by developers with minimal level of experience. In practice it was found to be more difficult to set up the external devices for the communication protocol rather than the DTStacks within Unity3D

Below is an example of the achieved synchronization between a UR-10 Manipulator and a hologram projected using a Microsoft HoloLens.

Due to these industrial efforts, an elaborate stack of the most common message types of ROS is included in the base framework. Furthermore, the 3-stage data flow was implemented, to simplify the switching between different communication modules without having to adapt the entire scene in Unity.

DTStacks, the Digital-Twinning-Stacks, published and maintained by fate4gle.

Please find the description of the framework within this wiki. Over time, this wiki will be further expanded and more conclusive. If anything crucial is missing, please raise an issue and it will be covered as soon as possible.

Content

  1. Home
    1. Introduction
    2. Background
  2. Development Environment Setup
    1. Unity
      1. Add Unity Install
    2. Visual Studio
  3. Quickstart Guide
    1. Prerequisits
    2. Create A Project
    3. Including DTStacks
    4. Create a Communicaiton Pipeline
    5. Setup of Publisher and Subscriber
  4. Technical Concept
    1. 3-Stage-Pipeline
    2. Data-Entry and Message Overwrites
    3. Clients (e.g. Subscriber)
    4. Message
    5. JSON Overwrites
  5. References-Frames
    1. Vectors and Angles in Messges
    2. Converters
  6. Robot Operating System (ROS)
    1. Set up ROS for MQTT
      1. mqtt_bridge installation (Depreciated)
      2. mqtt_client installation
    2. ROS Messages

Clone this wiki locally