When building RTSP server libssl was missing. Before building server just install libssl sources: `sudo apt-get install libssl-dev`