Skip to content

Modify source file to load curl library at runtime #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kanjoe24 wants to merge 6 commits into from
+94 −31
Closed

Modify source file to load curl library at runtime #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6c36615
Modify gh #79 : Modify source file to load curl library at runtime
kanjoe24 May 28, 2025
2d509c6
Fix gh #79 : Fix review comments
kanjoe24 May 29, 2025
f5c9b0e
Address gh#79 : Addressing review comments
kanjoe24 Jun 5, 2025
70f3db9
Fix gh #79 : Fixing further review comments
kanjoe24 Jun 11, 2025
0035a79
Add gh #79 : Adding few more improvements
kanjoe24 Jun 12, 2025
0f86b9a
Add gh #79 : Adding few more improvements
kanjoe24 Jun 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 2 additions & 14 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,8 @@ XLDFLAGS += $(LIBWEBSOCKETS_DIR)/lib/libwebsockets.a
CURL_DIR = $(FRAMEWORK_BUILD_DIR)/curl
ifneq ($(wildcard $(CURL_DIR)),)
INC_DIRS += $(CURL_DIR)/include
XLDFLAGS += $(CURL_DIR)/lib/libcurl.a
else
# Commands to run if the directory does not exist
XLDFLAGS += -lcurl
endif
XLDFLAGS += -ldl

# UT Control library Requirements
SRC_DIRS += ${TOP_DIR}/src
Expand All @@ -93,16 +90,6 @@ ifeq ($(TARGET),arm)
#CC := arm-rdk-linux-gnueabi-gcc -mthumb -mfpu=vfp -mcpu=cortex-a9 -mfloat-abi=soft -mabi=aapcs-linux -mno-thumb-interwork -ffixed-r8 -fomit-frame-pointer
# CFLAGS will be overriden by Caller as required
INC_DIRS += $(UT_DIR)/sysroot/usr/include
XLDFLAGS += $(OPENSSL_LIB_DIR)/libssl.a $(OPENSSL_LIB_DIR)/libcrypto.a -ldl
else
#linux case
# Check if the directory exists
ifneq ($(wildcard $(OPENSSL_LIB_DIR)),)
XLDFLAGS += $(OPENSSL_LIB_DIR)/libssl.a $(OPENSSL_LIB_DIR)/libcrypto.a -ldl
else
# Commands to run if the directory does not exist
XLDFLAGS += -lssl -lcrypto
endif
endif

# Defaults for target linux
Expand Down Expand Up @@ -150,6 +137,7 @@ list:
@$(ECHOE) ${YELLOW}SRC_DIRS:${NC} ${SRC_DIRS}
@$(ECHOE) ${YELLOW}CFLAGS:${NC} ${CFLAGS}
@$(ECHOE) ${YELLOW}XLDFLAGS:${NC} ${XLDFLAGS}
@$(ECHOE) ${YELLOW}XCFLAGS:${NC} ${XCFLAGS}
@$(ECHOE) ${YELLOW}TARGET_LIB:${NC} ${TARGET_LIB}

clean:
Expand Down
6 changes: 3 additions & 3 deletions configure.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,13 +288,13 @@ build_curl()
mkdir -p ${CURL_BUILD_DIR}
if [ "$TARGET" = "arm" ]; then
# For arm
./configure CPPFLAGS="-I${OPENSSL_BUILD_DIR}/include" LDFLAGS="-L${OPENSSL_BUILD_DIR}/lib" --prefix=${CURL_BUILD_DIR} --host=arm --with-ssl=${OPENSSL_BUILD_DIR} --with-pic --without-libpsl --without-libidn2 --disable-docs --disable-libcurl-option --disable-alt-svc --disable-headers-api --disable-hsts --without-libgsasl --without-zlib
./configure CPPFLAGS="-I${OPENSSL_BUILD_DIR}/include" LDFLAGS="-L${OPENSSL_BUILD_DIR}/lib" --prefix=${CURL_BUILD_DIR} --host=arm-none-linux-gnu --with-ssl=${OPENSSL_BUILD_DIR} --enable-shared --enable-static --with-pic --without-libpsl --without-libidn2 --disable-docs --disable-libcurl-option --disable-alt-svc --disable-headers-api --disable-hsts --without-libgsasl --without-zlib
else
# For linux
if [ "$OPENSSL_IS_SYSTEM_INSTALLED" -eq 1 ]; then
./configure --prefix=${CURL_BUILD_DIR} --with-ssl --without-zlib --without-libpsl --without-libidn2 --disable-docs --disable-libcurl-option --disable-alt-svc --disable-headers-api --disable-hsts --without-libgsasl
./configure --prefix=${CURL_BUILD_DIR} --with-ssl --without-zlib --without-libpsl --without-libidn2 --disable-docs --disable-libcurl-option --disable-alt-svc --disable-headers-api --disable-hsts --without-libgsasl --enable-shared --enable-static
Copy link
Contributor

@Ulrond Ulrond Jun 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand this correctly

shared - because you load the .so in the dlopen right?
static - because for external consumption you take ut-control.so & include the .a into that .so?

you may need to also consider stripped the symbols for the external libraries from ut-control.so, because you may not get what you expect..

say ut-control.so is loaded by you, which your expecting the libcurl.a also to be loaded.

But in the same process, someone else already loaded their libcurl.a, the one that's already present will then be used not your one.

that's the disadvantages of not having static libraries which you control embedded into your .so's

else
./configure CPPFLAGS="-I${OPENSSL_BUILD_DIR}/include" LDFLAGS="-L${OPENSSL_BUILD_DIR}/lib" --prefix=${CURL_BUILD_DIR} --with-ssl=${OPENSSL_BUILD_DIR} --with-pic --without-zlib --without-libpsl --without-libidn2 --disable-docs --disable-libcurl-option --disable-alt-svc --disable-headers-api --disable-hsts --without-libgsasl
./configure CPPFLAGS="-I${OPENSSL_BUILD_DIR}/include" LDFLAGS="-L${OPENSSL_BUILD_DIR}/lib" --prefix=${CURL_BUILD_DIR} --with-ssl=${OPENSSL_BUILD_DIR} --with-pic --without-zlib --without-libpsl --without-libidn2 --disable-docs --disable-libcurl-option --disable-alt-svc --disable-headers-api --disable-hsts --without-libgsasl --enable-shared --enable-static
fi
fi
make $@; make $@ install
Expand Down
95 changes: 82 additions & 13 deletions src/ut_kvp.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
#include <unistd.h>
#include <assert.h>
#include <curl/curl.h>
#include <dlfcn.h>

/* Application Includes */
#include <ut_kvp.h>
Expand Down Expand Up @@ -62,6 +63,60 @@ static struct fy_node* process_include(const char *filename, int depth, struct f
static void merge_nodes(struct fy_node *mainNode, struct fy_node *includeNode);
static void remove_include_keys(struct fy_node *node);

// Typedefs for curl function pointers
typedef CURL* (*curl_easy_init_t)(void);
typedef CURLcode (*curl_easy_setopt_t)(CURL *, CURLoption, ...);
typedef CURLcode (*curl_easy_perform_t)(CURL *);
typedef CURLcode (*curl_easy_getinfo_t)(CURL *, CURLINFO, ...);
typedef void (*curl_easy_cleanup_t)(CURL *);
typedef const char* (*curl_easy_strerror_t)(CURLcode);

// Function pointers for curl functions
static curl_easy_init_t kvp_curl_easy_init;
static curl_easy_setopt_t kvp_curl_easy_setopt;
static curl_easy_perform_t kvp_curl_easy_perform;
static curl_easy_getinfo_t kvp_curl_easy_getinfo;
static curl_easy_cleanup_t kvp_curl_easy_cleanup;
static curl_easy_strerror_t kvp_curl_easy_strerror;
static void *curl_lib = NULL;

int init_curl_symbols()
{
UT_LOG_DEBUG("Initializing curl symbols from %s", "libcurl.so.4");
curl_lib = dlopen("libcurl.so.4", RTLD_LAZY);
assert(curl_lib != NULL);
if (curl_lib == NULL)
{
UT_LOG_ERROR("dlopen failed for curl library %s: %s with major version 4\n", "libcurl.so.4", dlerror());
return -1;
}

// Load the curl function symbols
kvp_curl_easy_init = (curl_easy_init_t)dlsym(curl_lib, "curl_easy_init");
assert(kvp_curl_easy_init != NULL);
kvp_curl_easy_setopt = (curl_easy_setopt_t)dlsym(curl_lib, "curl_easy_setopt");
assert(kvp_curl_easy_setopt != NULL);
kvp_curl_easy_perform = (curl_easy_perform_t)dlsym(curl_lib, "curl_easy_perform");
assert(kvp_curl_easy_perform != NULL);
kvp_curl_easy_getinfo = (curl_easy_getinfo_t)dlsym(curl_lib, "curl_easy_getinfo");
assert(kvp_curl_easy_getinfo != NULL);
kvp_curl_easy_cleanup = (curl_easy_cleanup_t)dlsym(curl_lib, "curl_easy_cleanup");
assert(kvp_curl_easy_cleanup != NULL);
kvp_curl_easy_strerror = (curl_easy_strerror_t)dlsym(curl_lib, "curl_easy_strerror");
assert(kvp_curl_easy_strerror != NULL);

if ((kvp_curl_easy_init == NULL) || (kvp_curl_easy_setopt == NULL) || (kvp_curl_easy_perform == NULL) ||
(kvp_curl_easy_getinfo == NULL) || (kvp_curl_easy_cleanup ==NULL) || (kvp_curl_easy_strerror == NULL))
{
UT_LOG_ERROR("dlsym failed for one or more symbols\n");
dlclose(curl_lib);
curl_lib = NULL;
return -1;
}

return 0;
}

ut_kvp_instance_t *ut_kvp_createInstance(void)
{
ut_kvp_instance_internal_t *pInstance = malloc(sizeof(ut_kvp_instance_internal_t));
Expand Down Expand Up @@ -136,6 +191,13 @@ ut_kvp_status_t ut_kvp_open(ut_kvp_instance_t *pInstance, char *fileName)
node = process_node(fy_document_root(pInternal->fy_handle), 0);
remove_include_keys(node);

if (curl_lib)
{
// Close the curl library if it was opened
dlclose(curl_lib);
curl_lib = NULL;
}

if (node == NULL)
{
UT_LOG_ERROR("Unable to process node");
Expand Down Expand Up @@ -951,17 +1013,24 @@ static struct fy_node* process_include(const char *filename, int depth, struct f

if (strncmp(filename, "http:", 5) == 0 || strncmp(filename, "https:", 6) == 0)
{
// URL include
// init curl symbols if not already initialized
if (curl_lib == NULL && init_curl_symbols() != 0)
{
UT_LOG_ERROR("Error: Could not initialize curl symbols\n");
return NULL;
}

// Initialize the memory chunk to store the downloaded data
mChunk.memory = malloc(1);
mChunk.size = 0;

if (!mChunk.memory)
if (mChunk.memory == NULL)
{
UT_LOG_ERROR( "Error: Not enough memory to store curl response\n");
return NULL;
}

CURL *curl = curl_easy_init();
CURL *curl = kvp_curl_easy_init();
if (!curl)
{
UT_LOG_ERROR( "Error: Could not initialize curl\n");
Expand All @@ -970,25 +1039,25 @@ static struct fy_node* process_include(const char *filename, int depth, struct f
}

CURLcode res;
curl_easy_setopt(curl, CURLOPT_URL, filename);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_memory_callback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&mChunk);
res = curl_easy_perform(curl);
kvp_curl_easy_setopt(curl, CURLOPT_URL, filename);
kvp_curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_memory_callback);
kvp_curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&mChunk);
res = kvp_curl_easy_perform(curl);
if (res != CURLE_OK)
{
UT_LOG_ERROR( "Error: curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
UT_LOG_ERROR( "Error: curl_easy_perform() failed: %s\n", kvp_curl_easy_strerror(res));
free(mChunk.memory);
curl_easy_cleanup(curl);
kvp_curl_easy_cleanup(curl);
return NULL;
}

long response_code;
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code);
kvp_curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code);
if (response_code != 200)
{
UT_LOG_ERROR( "Error: HTTP request failed with code %ld\n", response_code);
free(mChunk.memory);
curl_easy_cleanup(curl);
kvp_curl_easy_cleanup(curl);
return NULL;
}

Expand All @@ -997,15 +1066,15 @@ static struct fy_node* process_include(const char *filename, int depth, struct f
{
UT_LOG_ERROR("Error: Cannot parse included content\n");
free(mChunk.memory);
curl_easy_cleanup(curl);
kvp_curl_easy_cleanup(curl);
return NULL;
}

process_node(fy_document_root(doc), depth + 1);
// UT_LOG_DEBUG("%s memory chunk = \n%s\n", __FUNCTION__, mChunk.memory);

// free(mChunk.memory); // fy_document_build_from_malloc_string(): The string is expected to have been allocated by malloc(3) and when the document is destroyed it will be automatically freed.
curl_easy_cleanup(curl);
kvp_curl_easy_cleanup(curl);
return fy_document_root(doc);
}
else
Expand Down
6 changes: 6 additions & 0 deletions tests/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ build:
$(ROOT_DIR)/build.sh
make -C ./ut-core test
@cp $(LIB_DIR)/lib* ${BIN_DIR}
@if [ -d $(LIB_DIR)/../curl ]; then \
Copy link
Contributor

@Ulrond Ulrond Jun 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes no sense.

the test builds test, it has no-idea what ut-control builds, wants to build or what it has internally to it.

the test calls the main ut-control makefile, which builds whatever libraries it requires ( which are copied to the output directory ), because that's what the ut-control makefile knows about.

cp $(LIB_DIR)/../curl/lib/lib*.so* ${BIN_DIR}; \
fi
@if [ -d $(LIB_DIR)/../openssl ]; then \
cp $(LIB_DIR)/../openssl/lib/lib*.so* ${BIN_DIR}; \
fi
@cp ${ROOT_DIR}/src/*.sh ${BIN_DIR}
@cp ${ROOT_DIR}/websocket-clients/* ${BIN_DIR}
@$(ECHOE) ${GREEN}Copy Assets to [${BIN_DIR}/assets] ${NC}
Expand Down
2 changes: 1 addition & 1 deletion tests/src/ut_control_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,6 @@ MY_DIR="$(dirname $SCRIPT_EXEC)"

cd "$(dirname "$0")"

export LD_LIBRARY_PATH=/usr/lib:/lib:/home/root:${MY_DIR}
export LD_LIBRARY_PATH=/usr/lib:/lib:/home/root:${MY_DIR}:${MY_DIR}/lib

./ut_control_test $@