Skip to content

romansloutsky/CLICEO

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

63 Commits
 
 
 
 
 
 
 
 

Repository files navigation

CLICEO provides robust handling of worker pools from the standart library's multiprocessing module. It was written with two use cases in mind:

  • Having workers launch potentially long-running command line processes using the standard library's subprocess module, and
  • Running workers in an interactive shell (e.g. ipython)

This project was borne out of frustration with constant manual cleanup of zombie processes and the clutter of files they created when they terminated incorrectly -- particularly after ctrl-C / KeyboardInterrupt. As such, the main components are:

  • The CLIcontextManager class, which can create temporary files and directories which are cleaned up automatically
  • The CommandLineCaller class, a generic CLI controller which runs a command in the CLI context and handles output capture
  • The Worker and PoolManager classes which combine to track and, when necessary, kill PIDs, and return tracebacks from workers to the parent process to simplify debugging

CLICEO hasn't been updated from Python 2 because it's been a very long time since I've needed to run hundreds sequence alignments and phylogenetic reconstructions in parallel. It was a life saver while I was developing ASPEN.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages