Node JS Challenge

We have a daily log file that is X in size. We need to convert this CSV data into JSON format. The problem? Our cloud instance only has 512MB of RAM. If you try to fs.readFileSync the whole file, the server crashes with "Heap Out of Memory."

Task

Write a script using Node Streams (fs.createReadStream) to transform the file line-by-line without loading the whole thing into memory.

Rules

• You can’t use fs.readFile or fs.readFileSync.

• You must use the stream API (or a library like csv-parser that uses streams internally).

• You must include e.g. process.memoryUsage() logs at the end to prove you didn't spike the RAM

Setup

Run

node generate_big_file.js

Then run

node main.js