Skip to content

Reduce memory usage and reduce hanging#59

Open
Aaaapril4 wants to merge 8 commits intoAI4EPS:mainfrom
Aaaapril4:memory
Open

Reduce memory usage and reduce hanging#59
Aaaapril4 wants to merge 8 commits intoAI4EPS:mainfrom
Aaaapril4:memory

Conversation

@Aaaapril4
Copy link
Contributor

Met an extreme case that needs more than 40 GB of memory for 40 CPUs, and one CPU hangs for more than 1 day, leaving other CPUs idle.

  • Release memory manually after each processing
  • Add a timeout to lock
  • Make each chunk smaller to reduce memory usage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant