Skip to content
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
## Changelog

### v0.4.3

#### Improved
Added memory uses tracker

### v0.4.2

#### Improved
Expand Down
58 changes: 58 additions & 0 deletions doc/compare_memory_tracking_methords.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Memory Usage Measurement: `valgrind` vs `/usr/bin/time`

## Overview

This document compares two common methods: **Valgrind (Massif)** and **`/usr/bin/time`**, highlighting their strengths and limitations.

## Comparison Table

| Feature | Valgrind (Massif) | `/usr/bin/time` |
| --------------------- | -------------------- | --------------------- |
| Heap Memory Tracking | ✅ Yes | ✅ Yes |
| Stack Memory Tracking | ⚠️ Incomplete | ✅ Yes |
| Standard Tool | ❌ No (needs install) | ✅ Yes (pre-installed) |
| Ease of Use | Moderate | Easy |
| Output Integration | Complex | Simple (shell output) |

---

## Use Case Analysis

### Using Valgrind:

**Variables Used / Memory Used:**

* Method: `valgrind`
* Stack: 5 MB
* Heap: 30 MB

**Estimated Usage:** \~31 MB
**Expected Usage:** \~35 MB

> Observation: Underestimates stack usage due to limited stack tracking capabilities.

---

### Using `/usr/bin/time`:

**Variables Used / Memory Used:**

* Method: `/usr/bin/time`
* Stack: 5 MB
* Heap: 30 MB

**Estimated Usage:** \~36.5 MB
**Expected Usage:** \~35 MB

> Observation: Much closer to actual usage. Accurately reflects both stack and heap usage.

---

## Conclusion

* Use **`/usr/bin/time`** for **accurate total memory measurement**, especially when stack usage is involved.
* Use **Valgrind (Massif)** when you need **detailed heap allocation breakdowns**, such as per function or call stack.

> **Recommendation**: For a code judging system where total memory consumption matters more than internal profiling, prefer `/usr/bin/time`.

---
32 changes: 25 additions & 7 deletions services/code.service.js
Original file line number Diff line number Diff line change
Expand Up @@ -230,9 +230,10 @@ const _executeCode = async (req, res, response) => {
let command
if (language === 'java') {
// Remove ulimit as a temp fix
command = `cd /tmp/ && timeout ${langConfig.timeout}s ${langConfig.run}`
command = `cd /tmp/ && /usr/bin/time -f "%M" -o /tmp/memory_report.txt timeout ${langConfig.timeout}s ${langConfig.run}`
} else {
command = `cd /tmp/ && ulimit -v ${langConfig.memory} && ulimit -m ${langConfig.memory} && timeout ${langConfig.timeout}s ${langConfig.run}`
// Execute command with memory limits and resource monitoring for non-Java languages
command = `cd /tmp/ && ulimit -v ${langConfig.memory} && ulimit -m ${langConfig.memory} && /usr/bin/time -f "%M" -o /tmp/memory_report.txt timeout ${langConfig.timeout}s ${langConfig.run}`
}

// Check if there is any input that is to be provided to code execution
Expand All @@ -244,14 +245,31 @@ const _executeCode = async (req, res, response) => {
}

const outputLog = await _runScript(command, res, true)

let memoryKB = null
try {
const path = '/tmp/memory_report.txt'
await fs.promises.access(path, fs.constants.F_OK)
const memoryReport = await fs.promises.readFile(path, 'utf8')
memoryKB = parseInt(memoryReport.trim(), 10)
} catch (err) {
console.warn(`Memory report not found or failed to read: ${err.message}`)
}

// Adjust if you're on Alpine to divide by 4
response.memory = memoryKB ? memoryKB / 4 : null
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The comment suggests dividing the memory usage by 4 if running on Alpine. However, /usr/bin/time -f "%M" is documented to report the maximum resident set size in Kilobytes on Linux systems. Dividing by 4 implies the output is in 4KB pages, which is not the standard behavior of %M. Relying on this division is fragile and could lead to incorrect memory reporting if the execution environment changes or if /usr/bin/time behaves as documented (reporting in KB). It's best to remove this division unless there's a specific, confirmed reason for it on the target platform that overrides the standard documentation.

Suggested change
response.memory = memoryKB ? memoryKB / 4 : null
response.memory = memoryKB ? memoryKB : null

console.log('memory used', response.memory)

response.output =
outputLog.error !== undefined
? _prepareErrorMessage(outputLog, language, command)
: outputLog.result.stdout

if (outputLog.error) {
response.error = 1
}
} else {
}
else {
response.error = 1
}
} catch (e) {
Expand Down Expand Up @@ -399,7 +417,7 @@ const _getAiScore = async (langConfig, question, response, points, userAnswer, r

const _executeStatement = (db, sql) => {
return new Promise((resolve, reject) => {
db.all(sql, function(err, rows) {
db.all(sql, function (err, rows) {
if (err) {
reject(err);
} else {
Expand Down Expand Up @@ -887,14 +905,14 @@ const _postCleanUp = async (type, staticServerInstance = undefined, jasmineServe
await _cleanUpDir(appConfig.multifile.workingDir, appConfig.multifile.submissionFileDownloadPath)
switch (type) {
case FRONTEND_STATIC_JASMINE:
if(staticServerInstance) {
if (staticServerInstance) {
staticServerInstance.close(() => {
logger.info('Exiting static server in post cleanup')
})
}
break
case FRONTEND_REACT_JASMINE:
if(jasmineServer) {
if (jasmineServer) {
logger.info('Exiting react setup server in post cleanup')
process.kill(-jasmineServer.pid)
}
Expand All @@ -917,7 +935,7 @@ const _executeMultiFile = async (req, res, response) => {
let result
if (req?.non_editable_files) {
const isValidSubmission = await _checkIntegrity(req.non_editable_files)
if(!isValidSubmission) throw new Error(`A non editable file has been modified, exiting...`)
if (!isValidSubmission) throw new Error(`A non editable file has been modified, exiting...`)
}
switch (req.type) {
case FRONTEND_STATIC_JASMINE:
Expand Down
Loading