compilation under windows or cross-compilation

[femust]

Hey,

so I am running the script build_mingw.sh and I got the following error:

-- Could NOT find ZIP (missing: ZIP_LIBRARY ZIP_INCLUDE_DIR)

since on ubuntu I have this library, I'm not sure why it can't find it, I mean, I am assuming it's due to compilation under windows, but then the question arises how should I install this zip library under ubuntu for cross compilation?

[clemensschiffer]

Hi,

the dependencies have to be available in the same cross-compile configuration.
Have you seen this old issue, I had: #5
I had xerces and libzip build from source with the same cross-compile script.

Does that help?

[femust]

With xerces I solved the problem but libzip.... it's like building dependency to dependency to dependency and there is no end ...

[clemensschiffer]

For zlib there is a package available (at least in Ubunutu 18.04) then libzib is built on top of that:
apt install libz-mingw-w64-dev

I do have a docker file with all that somewhere, actually I meant to include that in the CI-process here....

[femust]

I guess for docker file example I will appreciate :) (but IMO after installing libz that you recommended I did step forward, but now linking problem with xerces), it's like never-ending story.

[clemensschiffer]

Found it.
Note: I had to redact some urls since I had those code repos hosted internally, so replace [[ url ]] parts with
a respective code repo.
I think one of the dependencies may not be on github but I downloaded it from sourceforge and put it in a internal git-repo, could also use a wget there.

After all that I still had to tell cmake where to find ASIO, I put:

    if(NOT ASIO_ROOT)
      set(ASIO_ROOT /usr/x86_64-w64-mingw32)
      if(ASIO_FIND_REQUIRED)
        message(FATAL_ERROR "ASIO headers could not be found.")
      else()

into cmake/FindASIO.cmake, you can also set the cmake-variable to that effect.
I copied all the dependencies into /var/deps to they could be copied with the build dcp executable to windows.
You can find all dependencies of your binary by running e.g. ldd dcpmaster.

Building can be quite frustrating, that's why docker is great: once you got it you got it.
Share here what did work for you.
Hopefully I'll get around to integrating this docker into CI-process so the lib is tested on win on every PR.

FROM ubuntu:18.04 

MAINTAINER Clemens Schiffer <clemens.schiffer@v2c2.at>

USER root 
RUN apt-get update; apt-get upgrade -y; 
RUN apt-get -y install  git unzip wget build-essential g++-7 cmake gcc-7 libicu-dev build-essential libzip-dev libxerces-c-dev libssl-dev
RUN apt-get -y install mingw-w64 binutils-mingw-w64 libz-mingw-w64-dev mingw-w64-tools
RUN apt-get -y install net-tools network-manager vim iputils-ping 

RUN update-alternatives --set x86_64-w64-mingw32-gcc /usr/bin/x86_64-w64-mingw32-gcc-posix
RUN update-alternatives --set x86_64-w64-mingw32-g++ /usr/bin/x86_64-w64-mingw32-g++-posix

# Directory to be mounted to the outside:
RUN mkdir -p /var/mybuild 
WORKDIR /var/mybuild

# build dir
RUN mkdir -p /var/build 

# DCPlib: Pull only so other can use by Toolchain, build later 
RUN git clone https://github.com/modelica/DCPLib.git /var/build/DCPLib
RUN cp /var/build/DCPLib/Toolchain-mingw-w64-ubuntu.cmake /var/build/

# Xerces: Pull and build
RUN git clone [[ Xerces-C URL ]]  /var/build/xerces
RUN sed -ri "s|Windows.h|windows.h|" /var/build/xerces/tests/src/ThreadTest/ThreadTest.cpp
RUN cd /var/build/xerces/ && \
	mkdir -p build && cd build && \
	cmake -DCMAKE_TOOLCHAIN_FILE=/var/build/Toolchain-mingw-w64-ubuntu.cmake ..  && \
	cmake --build . && \
	make install

# asio: Pull and build
RUN git clone [[ asio URL ]] /var/asio
RUN cd /var/asio/asio  && \
	chmod +x configure && \
	./configure --without-boost --prefix=/usr/x86_64-w64-mingw32 && \
	cd include && \
	make install

# Libzip: Pull and build
RUN git clone [[ libzip_1.5.2.git URL ]] /var/build/libzip
RUN sed -ri "s|Windows.h|windows.h|" /var/build/libzip/lib/zip_crypto_win.c 
RUN cd /var/build/libzip/ && \
	mkdir -p build && cd build && \
	cmake -DCMAKE_TOOLCHAIN_FILE=/var/build/Toolchain-mingw-w64-ubuntu.cmake ..  && \
	cmake --build . && \
	make install

# Copy dependencies to final: to be cp to /var/mybuild by docker-compose up
RUN mkdir -p /var/deps
RUN cp /usr/lib/gcc/x86_64-w64-mingw32/7.3-posix/libgcc_s_seh-1.dll /var/deps
RUN cp /usr/lib/gcc/x86_64-w64-mingw32/7.3-posix/libstdc++-6.dll /var/deps
RUN cp /usr/x86_64-w64-mingw32/lib/libwinpthread-1.dll /var/deps
RUN cp /usr/x86_64-w64-mingw32/bin/libxerces-c.dll /var/deps

# build my copy of the dcplib:
RUN cd /var/build/DCPLib && git pull
RUN cd /var/build/DCPLib && mkdir -p build && cd build && \
	cmake -DCMAKE_TOOLCHAIN_FILE=../Toolchain-mingw-w64-ubuntu.cmake .. && \
	grep -rl 'isystem' | xargs sed -i 's/isystem/I/g' && \
	cmake --build . --target install

# DCP Examples cross-compile build can start: ....

[masoud-najafi]

Hi,
I would like to build DCPLIB on Win64, but it is not clear in the Readme, if I can install DCPLIB directly in Windows, or I need to do cross-compilation with MinGW. Do I need to install Ubuntu? I wonder if there is any step-by-step help for Windows?
Thanks
Masoud
Altair

[clemensschiffer]

Cross-Compiling is easiest because you can rely on the system packages for dependencies, all the steps should be in the docker file excerpt above.

Building natively on Windows:
Basically you need cmake and a C++11 compiler such as visual compiler.
I had to build zlib, libz and Xerces-C from source and put the asio headers 2 folders up from the project.
Make sure to always select the same architecture such as -A x64.
(Note: Additionally, I had two versions of Xerces installed which confused cmake, so watch out for that.)

[femust]

@clemensschiffer
So I have a co-simulation with a few FMUs and for some reason, the results are strongly dependent on the OS that I am running, their configuration
Have a look, here is a plot of one variable from the model (orange - ground truth, blue - dcplib)

• so when I run everything (slaves and master) on Linux here is the plot
  

• when I run all slaves on Linux and master on Windows, the plot is the same
  

BUT

• when I run slaves on Windows I got the following results (don't mind the length of the plot because sim time is different)
  
• and finally, when master is on Linux and all slaves are on Windows I got the following errors:
  

do you know what may be the reason?

[clemensschiffer]

I think we should keep this issue strictly to compilation.
If you think this may be an issue with DCPLib it would be most helpful to have a self-contained minimal example that reproduces this behavior, best in a new issue.

For your example:
There should be no difference at all. Is data transferred via the master? If the master is only configuring that should make no difference where it is running (even so it should only be latency ).

Are you sure this is the exact same code including configuration, time_resolution, input/output connections and the model itself?
Are you running SRT or NRT? In NRT there should be deterministically zero difference.
If SRT do you use STC_run with time = 0, i.e. now, or STC_run with starting time, if the later is the systemtime on all systems the same?

Do the supplied examples differ per os? I can't believe that they do....

[svanimisetti]

@clemensschiffer - Thanks for posting the Dockerfile. Is it possible to build the project with cmake+MSVC generators? Based on what I read from this thread, there are other dependencies for this project (ASIO, zip, etc) - and it would be impractical (and perhaps impossible) to build this on Window using cmake+MSVC. Is the option possible option is to cross-compile on Linux using mingw-win32 or mingw-win64?

[femust]

@svanimisetti from my experience use pacman.
In powershell

iex ((New-Object System.Net.WebClient).DownloadString('[https://community.chocolatey.org/install.ps1')](https://community.chocolatey.org/install.ps1%27))

choco install mingw -y

choco install cmake --installargs '"ADD_CMAKE_TO_PATH=System"' -y

install msys2 https://www.msys2.org/

and in msys

pacman -Syu
pacman -Su
pacman -S --needed base-devel mingw-w64-x86_64-toolchain
pacman -S mingw-w64-x86_64-zlib
pacman -S mingw-w64-x86_64-libzip
pacman -S mingw-w64-x86_64-boost
pacman -S mingw-w64-x86_64-xerces-c
pacman -S make
pacman -S mingw-w64-x86_64-asio
pacman -S mingw-w64-x86_64-cmake

then in dcplib create a build directory and

cmake -G "MinGW Makefiles" ..
cmake --build . --target install

[clemensschiffer]

Hello @svanimisetti,
sorry for the late reply.
Basically you can do what @femust suggested, I think this still qualifies as cross-compiling.

Building natively with MSVC is possible, as I wrote above:

Building natively on Windows: Basically you need cmake and a C++11 compiler such as visual compiler. I had to build zlib, libz and Xerces-C from source and put the asio headers 2 folders up from the project. Make sure to always select the same architecture such as -A x64. (Note: Additionally, I had two versions of Xerces installed which confused cmake, so watch out for that.)

You definitely need Xerces-c follow the instructions here:
https://xerces.apache.org/xerces-c/build-3.html
It is also a cmake project.
ASIO(the non-boost variant) is header only, so no compiling needed.
Libz and zlib can be omitted, if you do not need the functionality.
Have you tried? Did you run into specific problems?

I hope I find the time soon to put this native build into the CI/CD process here, such that it later on it will only be copy/paste. ;)

Kind regards,
Clemens

[svanimisetti]

@clemensschiffer - Sorry for such a delayed response, but I was able to push through during the summer vacation and managed to compile DCPLib on Windows. As @masoud-najafi had also requested, I wanted to build directly on Windows with MinGW GCC, without the need to cross-compile via Linux or WSL. I was able to run all the cases successfully. If it is of any interest, I can write-up a small documentation and send PR your way. Let me know if that will be helpful. Otherwise, I can always document the learnings in this issue thread. Kindly advise.

[svanimisetti]

@femust - you have shown an example of FMU using DCP on win64 and linux64.

So I have a co-simulation with a few FMUs and for some reason, the results are strongly dependent on the OS that I am running, their configuration

How are you able to combine the FMU with DCP? I am trying to achieve a similar objective. I am now able to compile the DCPLib on win64 and linux64. My next step is to embed the DCPLib inside FMU and connect a co-simulation master FMU on win64 to slave on linux64. I am running into issues. Do you have any documentation demonstrating how you achieved this?

I don't mean to hijack this issue thread, but any information you can share will be greatly appreciated.

[svanimisetti]

If it is of any interest, I can write-up a small documentation and send PR your way.

@clemensschiffer - I just noticed that the documentation for installation is in the wiki (https://github.com/modelica/DCPLib.wiki.git). Unfortunately, there is no way to modify the wiki attached to main repo and send a pull request. Looks like the only way is to document the steps here.