Compilation takes a long time

[nandlab]

Hello,

building a program with MQTT_CPP takes very long. I am adding mqtt_cpp as a git submodule.
The following example program takes 35 (!) seconds to compile on my linux machine:

#include <iostream>
#include "mqtt_client_cpp.hpp"

using namespace std;

int main()
{
    cout << "Hello World!" << endl;
    return 0;
}

This is my CMake file:

cmake_minimum_required(VERSION 3.5)

project(MqttCppTestInclusion LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

add_executable(MqttCppTestInclusion main.cpp)
set(MQTT_CPP_INCLUDE "${CMAKE_CURRENT_SOURCE_DIR}/mqtt_cpp/include")
include_directories(${CMAKE_SOURCE_DIR} ${MQTT_CPP_INCLUDE})
set(THREADS_PREFER_PTHREAD_FLAG ON)
find_package(Threads REQUIRED)
find_package(Boost REQUIRED system)
target_link_libraries(MqttCppTestInclusion PRIVATE Threads::Threads Boost::system)
target_link_libraries(MqttCppTestInclusion LINK_PUBLIC)

Even if I make a small change somewhere which is not related to MQTT, it always takes the same time to recompile.

Is there a way to compile programs with mqtt_cpp more efficiently?

Could maybe separating the function declarations and definitions in mqtt_cpp into separate files increase the incremental build speed?

[redboltz]

I understand the issue.
I just investigated the file that takes longer compile times.
I think that property_variant.hpp is one of them.

#include <iostream>
#include "mqtt/property_variant.hpp"

using namespace std;

int main()
{
    cout << "Hello World!" << endl;
    return 0;
}

How long does it takes on your environment?

[AndreaRicchi]

Hi,
I have the same issue on a bigger project. Every class that include the MQTT client takes a very long time to build. I've traced the build steps for a file and the result is that 90% of the time is used for the mqtt/client.hpp class.

Here the CLang trace JSON. You can open it in chrome://tracing: pushactionrunner.cpp.json.gz

I tried bumping to the v12 version, using PCH, using STD types instead of BOOST; but nothing improved the build times.

[redboltz]

@AndreaRicchi

I opend the trace using chrome. I'm not sure how to use it.

I have the same issue on a bigger project. Every class that include the MQTT client takes a very long time to build. I've traced the build steps for a file and the result is that 90% of the time is used for the mqtt/client.hpp class.

#include <iostream>
// Switch one of them
#include "mqtt/client.hpp"
#include "mqtt/endpoint.hpp"
#include "mqtt/property_variant.hpp"

using namespace std;

int main()
{
    cout << "Hello World!" << endl;
    return 0;
}

Do you mean only mqtt/client.hpp case takes a long time and mqtt/endpoint.hpp and mqtt/property_variant.hpp take a short time?

[AndreaRicchi]

@redboltz

Based on the flame graph there are 2 issues. The first one is the source parsing. In particular the mqtt/client.hpp took 6 seconds. Off these 6 seconds, 4 are used for the mqtt/endpoint.hpp.

The second issue is related to the template function istantiation. You can follow this link on how to read that file.

[redboltz]

Thank you for the information!
I will take a look it.

Currently, my priority is runtime performance.
Compile time is actually long but I personally can accept it. But if there is a good way to increase the speed without runtime performance penalty and flexibility penalty, I will apply it. I beleive that there is some good way but I'm not an expert of compiling time performance tuning. So I think that I might need long time to analyse it and find a solution. Pull requests are welcome :)

[nandlab]

@redboltz

I understand the issue. I just investigated the file that takes longer compile times. I think that property_variant.hpp is one of them.

If I write the following include statement in the hello world program, the compilation takes 7 seconds in my environment:

#include "mqtt/property_variant.hpp"

[AndreaRicchi]

@redboltz

I know that compile time is not as important as runtime performance. But when the library is used in a more complex application where, as in my example, the MqttClient is store as a class member. This means that all the classes that need to send a MQTT messages will include that header and the build time increase tremendously.
At the moment I solved this issue moving all the mqtt_cpp references in the source file and using the MqttClient as a static variable instead of class member, but is not C++ style. Maybe we can add a more complex example using classes specifying that using any library include in the class header could heavily impact on the compile time.
If you agree on that a I can create a PR with the example. 😄

[nandlab]

@redboltz

Currently, my priority is runtime performance.
Compile time is actually long but I personally can accept it. But if there is a good way to increase the speed without runtime performance penalty and flexibility penalty, I will apply it.

How is compile time related to runtime performance?

Normally, in the most libraries, headers, which are included in the user program, are lightweight and only contain the necessary function declarations.
The actual function definitions, on the other hand, are in separate library source files (*.c and *.cpp). They have to be build once, so that a static library archive (*.a) or a shared library (*.so) is created.
Then the user program does not have to compile the library code every time, instead it just links to it (statically or dynamically).

If you work on a larger project, where every file includes (mostly indirectly) a header, which takes half a minute to compile, the total compile time becomes huge. Here's a project I work on: RoomControl. On my machine it takes 2 minutes to compile, where most of the time is caused by the mqtt_client_cpp.hpp header.

I also tried out the paho.mqtt.cpp library. This demo program with paho.mqtt.cpp takes 2 seconds to compile.

The mqtt_cpp library can be very useful, because it integrates with Boost ASIO, which can also be used for other asynchronous functionality. But unfortunately, the compile time right now makes the development in my application very hard.

Unfortunately I can not provide a pull request, because I do not know the library well enough to optimize it.

It would be very nice, if you could find the time to split this useful library into separate *.hpp headers (only with declarations) and *.cpp source files.

[redboltz]

How is compile time related to runtime performance?

For example, function calls via virtual function table could affect runtime performance.
So I use template based approach. It tends to increase compile time but good runtime performance.
However, I understand a combination of MQTT_ALWAYS_INLINE and virtual function in the same compilation unit (implementation is in the same cpp file) can be inlined.
It is just an example. There are lot of similar tricks.

I have no plan to provide a library files on mqtt_cpp. Header only is important for me, so far.
If separated library files are provided, many trouble would happen.
Different compile option make SEGV easily on the bad combination of so and application program.
It's easy to predict users report the library doesn't work on their environment. I want to avoid it.

In addtion, I think that the Boost Libraries tend to be header only as long as it can.

I'm looking for decreasing compile time way on header only library. The most of boost libraries achieve that, so I guess there is a way. I will analyze using the tool introduced by @AndreaRicchi and will try to find a solution.
I need time to do that.

[AndreaRicchi]

@nandlab

I checked your code, that was the same situation as me. As I previously said I temporarily resolved the issue by moving the

#include "mqtt_client_cpp.hpp"

    typedef std::shared_ptr<
        mqtt::callable_overlay<
            mqtt::async_client<
                mqtt::tcp_endpoint<
                    mqtt::as::ip::tcp::socket,
                    mqtt::strand
                >
            >
        >
    >
    MQTTAsyncClient_t;

in the CPP file as a static variable. This improved enormously the compile time.

[BrettHemes-3M]

This got me bad too... under the pattern of having an MQTT client as a class member I very quickly had my compile times exceed 5-10 minutes for a relatively simple project. By making the client a static variable as suggested, my compile time is back to around 15 seconds.

I have enjoyed developing with this library but do not see this workaround as an acceptable long term solution.

Any updates on future fixes?

[redboltz]

I am working on fixing delivery guarantee issue and reviewing authentication and authorization.
Unfortunately, no significant updates for this issue so far.

NOTE:
mqtt_cpp design principles are header-only and value semantics (as long as it can be used)
See https://wandbox.org/permlink/Rh1X8yI8DMAEOqzU

[redboltz]

I analized compile process using -ftime-trace.
It seems that non-template inline functions are always processed.
So I templatized them to avoid instanciate.
Off course actually used functions are instantiate but unused functions (especially many client factory functions) are not instantiated.
Try #931

[ineffective]

I solved my problems with long compilation times by creating thin, non-templated wrapper for mqtt-client that I use throughout the code. mqtt-client itself is created in separate cpp file that provides factory method. Works like a charm.

[redboltz]

@ineffective could you show me your code? I'm interested in which part do you wrap. Only factory functions, or class template client, async_client, sync_client, and endpoint.