Graphviper changes needed for astroviper demonstrator#45
Conversation
|
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
This PR adds a new toolviper.utils.memory_management module intended to support the “Graphviper / astroviper demonstrator” by providing small helpers for RSS reporting and returning memory to the OS via allocator-specific calls.
Changes:
- Added
get_rss_gb()to report current process RSS viapsutil. - Added
memory_setup()to tune glibcmallopt(M_MMAP_THRESHOLD, ...)on Linux. - Added
free_memory()to trigger GC and attempt allocator memory release (glibcmalloc_trimon Linux;malloc_zone_pressure_reliefon macOS).
Comments suppressed due to low confidence (2)
src/toolviper/utils/memory_management.py:40
- The macOS path calls
malloc_zone_pressure_relief(None, 0)without definingargtypes/restype. With ctypes defaults, passingNoneis likely to raise a conversionTypeError(and the current broadexcept Exception: passwill silently swallow it), meaning this branch may never actually run. Define the function signature (void*/size_t) and pass actypes.c_void_p(0)NULL zone; also consider narrowing the exception handling and/or logging failures so unexpected issues aren't hidden.
lib = ctypes.CDLL("libSystem.B.dylib")
# malloc_zone_pressure_relief(zone=NULL, goal=0) asks all zones to
# release as much memory as possible back to the OS.
lib.malloc_zone_pressure_relief(None, 0)
except Exception:
pass
src/toolviper/utils/memory_management.py:39
- New memory-management utilities are introduced without tests. Since this repo has unit tests for other
toolviper.utilsmodules, please add tests that validate (via monkeypatchingsys.platform/ctypes) thatmemory_setup()/free_memory()are no-ops or safely handled on unsupported platforms, and thatget_rss_gb()returns the expected unit conversion.
On Linux this calls glibc's mallopt(M_MMAP_THRESHOLD, threshold).
On macOS the system allocator handles this automatically; the call is skipped.
"""
import sys, ctypes
if sys.platform == "linux":
ctypes.CDLL("libc.so.6").mallopt(-3, threshold)
def free_memory():
"""Return free memory pages to the OS.
On Linux this calls glibc's malloc_trim(0).
On macOS, malloc_zone_pressure_relief is used as the closest equivalent.
"""
import sys, ctypes
import gc
gc.collect()
if sys.platform == "linux":
ctypes.CDLL("libc.so.6").malloc_trim(0)
elif sys.platform == "darwin":
try:
lib = ctypes.CDLL("libSystem.B.dylib")
# malloc_zone_pressure_relief(zone=NULL, goal=0) asks all zones to
# release as much memory as possible back to the OS.
lib.malloc_zone_pressure_relief(None, 0)
except Exception:
pass
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| def memory_setup(threshold: int = 131072): |
| def get_rss_gb(): | ||
| import psutil, os | ||
|
|
||
| return psutil.Process(os.getpid()).memory_info().rss / 1e9 |
| ctypes.CDLL("libc.so.6").mallopt(-3, threshold) | ||
|
|
||
|
|
||
| def free_memory(): |
| elif sys.platform == "darwin": | ||
| try: | ||
| lib = ctypes.CDLL("libSystem.B.dylib") |
| def get_rss_gb(): | ||
| import psutil, os | ||
|
|
||
| return psutil.Process(os.getpid()).memory_info().rss / 1e9 |
No description provided.