libc: suppress ASan warnings for overflow test cases by lukkrusz · Pull Request #468 · phoenix-rtos/phoenix-rtos-tests

lukkrusz · 2026-03-23T12:27:12Z

JIRA: CI-657

Description

Fixes: phoenix-rtos/phoenix-rtos-project#1547

Motivation and Context

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Chore (refactoring, style fixes, git/CI config, submodule management, no code logic changes)

How Has This Been Tested?

Already covered by automatic testing.
New test added: (add PR link here).
Tested by hand on: (list targets here).

Checklist:

My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing linter checks and tests passed.
My changes generate no new compilation warnings for any of the targets.

Special treatment

This PR needs additional PRs to work (list the PRs, preferably in merge-order).
I will merge this PR by myself when appropriate.

gemini-code-assist · 2026-03-23T12:27:26Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where AddressSanitizer (ASan) would report errors during tests specifically designed to check for memory allocation overflow conditions. By conditionally redirecting these overflow tests to use the underlying __libc_ prefixed allocation functions when ASan is active on Linux, the tests can now correctly verify error handling without triggering ASan warnings, ensuring the test suite remains robust and accurate under ASan.

Highlights

ASan Warning Suppression: Modified stdlib_alloc.c to conditionally use __libc_malloc, __libc_calloc, and __libc_realloc in overflow test cases when AddressSanitizer (ASan) is enabled on Linux, preventing false-positive warnings from intentional overflow tests.
Conditional Compilation: Introduced the __linux_with_ASan__ macro and conditional compilation to selectively declare and use __libc_ prefixed allocation functions only when both __linux__ and __SANITIZE_ADDRESS__ are defined.
Test Case Adaptation: Updated the malloc_overflow, calloc_overflow, and realloc_overflow test cases to utilize the newly defined TEST_CALLOC and TEST_REALLOC macros, or directly __libc_malloc, ensuring these tests correctly verify error handling without triggering ASan runtime errors.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces conditional compilation to suppress AddressSanitizer warnings for memory allocation overflow tests, which is a reasonable approach. My review includes a suggestion to refactor the preprocessor macros for better consistency and maintainability. I've also identified and suggested a fix for a logical bug in one of the overflow tests where an incorrect variable was being used, causing the test to not perform its intended check.

gemini-code-assist · 2026-03-23T12:29:22Z

libc/stdlib/stdlib_alloc.c

 	TEST_ASSERT_NOT_NULL(ptr);
 	TEST_ASSERT_EACH_EQUAL_INT(0, ptr1, BLOCK_SIZE / sizeof(int));
-	TEST_ASSERT_NULL(realloc(ptr, SIZE_MAX));
+	TEST_ASSERT_NULL(TEST_REALLOC(ptr, SIZE_MAX));


The intent of this test block seems to be to check for overflow in realloc when using a pointer originally allocated by calloc (ptr1). However, realloc is called with ptr, which was allocated by malloc and already tested in the previous block. You should use ptr1 here to test the calloc path.

TEST_ASSERT_NULL(TEST_REALLOC(ptr1, SIZE_MAX));

Huh, that does seem like a bug in the original test case

libc/stdlib/stdlib_alloc.c

github-actions · 2026-03-23T12:45:03Z

Unit Test Results

9 553 tests ±0 8 961 ✅ ±0 53m 3s ⏱️ +36s
591 suites ±0 592 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit f4ff319. ± Comparison against base commit 6ed92c4.

♻️ This comment has been updated with latest results.

JIRA: CI-657

damianloew · 2026-03-24T09:36:39Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces macros to bypass ASan instrumentation for memory allocation functions in overflow tests, which is a valid approach to handle the intentional overflow checks. The changes correctly apply these macros in the test cases. Additionally, a bug in the realloc_overflow test where an incorrect pointer was used has been fixed. My only suggestion is to refactor the preprocessor logic for defining the new macros to improve its readability and maintainability.

libc/stdlib/stdlib_alloc.c

damianloew · 2026-03-24T09:33:24Z

libc/stdlib/stdlib_alloc.c

 	return "allocator_may_return_null=1";
 }

+#if defined(__linux__) && defined(__SANITIZE_ADDRESS__)


For sure we should leave a comment why we are doing that - it's not sth typical in this code.

Also, mention that it's valid only for glibc

damianloew · 2026-03-24T09:39:52Z

libc/stdlib/stdlib_alloc.c

 }

+#if defined(__linux__) && defined(__SANITIZE_ADDRESS__)
+#define __linux_with_ASan__


Identifiers beginning with __ (double underscore) are reserved for the implementation (compiler and standard library) by the C standard. Defining such a macro in user code is undefined behaviour and risks silently conflicting with compiler/system headers in future toolchain versions.

Suggested change

#define __linux_with_ASan__

#define ALLOC_TEST_LINUX_ASAN

as gemini pointed out - no need for extra define.

is this needed to be included in #pragma diagnostic?

as you-re embeddid multiple ifdefs (you're inside #ifndef __phoenix__ here - please add comments to #endif with the scope name you're leaving)

I decided to pursue the approach without an extra define. All the rest is done/ fixed

lukkrusz · 2026-03-24T10:54:31Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a mechanism to bypass AddressSanitizer interception for memory allocation functions in overflow tests. This is achieved by using macros that call internal glibc functions directly when ASan is active on Linux. The changes correctly apply these macros to the overflow tests for malloc, calloc, and realloc. The pull request also includes fixes for existing bugs within the realloc_overflow test case. I have one suggestion to make the conditional compilation directive more precise to avoid potential issues on non-glibc Linux distributions.

libc/stdlib/stdlib_alloc.c

Fixes: https://github.com/issues/assigned?issue=phoenix-rtos%7Cphoenix-rtos-project%7C1547 JIRA: CI-657

lukkrusz · 2026-03-24T11:39:31Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces macros to bypass AddressSanitizer's interception of memory allocation functions, allowing overflow test cases to execute as intended. The changes are confined to the allocation tests and correctly use glibc-specific internal functions (__libc_malloc, etc.) under the appropriate preprocessor conditions. Additionally, a pre-existing bug in the realloc_overflow test case, where a wrong pointer was being used, has been corrected. The implementation is sound and achieves its goal.

gemini-code-assist bot reviewed Mar 23, 2026

View reviewed changes

lukkrusz force-pushed the lukkrusz/stdlib_alloc_suppress_overflow_warnings branch from 6df98c1 to 3e406da Compare March 23, 2026 15:05

libc: fix incorrect ptr use in realloc_overflow

c01974b

JIRA: CI-657

lukkrusz force-pushed the lukkrusz/stdlib_alloc_suppress_overflow_warnings branch 2 times, most recently from 532c73c to 7d38dd8 Compare March 24, 2026 08:10

lukkrusz requested a review from damianloew March 24, 2026 08:46

lukkrusz marked this pull request as ready for review March 24, 2026 08:46

gemini-code-assist bot reviewed Mar 24, 2026

View reviewed changes

libc/stdlib/stdlib_alloc.c Outdated Show resolved Hide resolved

damianloew requested changes Mar 24, 2026

View reviewed changes

lukkrusz force-pushed the lukkrusz/stdlib_alloc_suppress_overflow_warnings branch from 7d38dd8 to 29baeea Compare March 24, 2026 10:51

gemini-code-assist bot reviewed Mar 24, 2026

View reviewed changes

libc/stdlib/stdlib_alloc.c Outdated Show resolved Hide resolved

libc: suppress ASan warnings for overflow test cases

f4ff319

Fixes: https://github.com/issues/assigned?issue=phoenix-rtos%7Cphoenix-rtos-project%7C1547 JIRA: CI-657

lukkrusz force-pushed the lukkrusz/stdlib_alloc_suppress_overflow_warnings branch from 29baeea to f4ff319 Compare March 24, 2026 11:31

gemini-code-assist bot reviewed Mar 24, 2026

View reviewed changes

lukkrusz requested a review from damianloew March 24, 2026 15:17

Conversation

lukkrusz commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Types of changes

How Has This Been Tested?

Checklist:

Special treatment

Uh oh!

gemini-code-assist bot commented Mar 23, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

lukkrusz Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Test Results

Uh oh!

damianloew commented Mar 24, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

damianloew Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

damianloew Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

lukkrusz Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

damianloew Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

nalajcie Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

lukkrusz Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

lukkrusz commented Mar 24, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

lukkrusz commented Mar 24, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lukkrusz commented Mar 23, 2026 •

edited

Loading

github-actions bot commented Mar 23, 2026 •

edited

Loading