This study presents an open framework, CellMigrationGym, to standardize the DRL approach to study cell migration and infer the underlying biology. Built upon common packages (OpenAI Gym, PyBullet, and DRL libraries), the CellMigrationGym provides powerful and flexible functions to investigate cell migration behavior. This study also presents a demonstration of CellMigrationGym to investigate a representative cell migration behavior, namely, Cpaaa intercalation. The demonstration reveals technical details of the framework setup, and most importantly, it also displays valuable functions of CellMigrationGym for cell migration study, such as 1) multiple observational data preparation and standardization, 2) different migration mechanisms exploration (such as gradient-driven and collective cell movements associated), and 3) evaluation of neighboring cell’s influence on the cell migration.
gym 0.18.0
pybullet 3.0.8
pytorch 0.4.1
numpy 1.19.4
matplotlib 3.3.3
scikit-learn 0.22
./DRL: Folder for DRL related model and data
-- ./Embryo: Environment for DRL.
-- ./saved_data/: Folder that used for saving the output data when exploring the successful scenarios for DRL. Currently stored npy file are used for figure 7.
-- ./trained_models/: Folder with all the pre-trained DRL models.
---- drl_model.pkl: DRL model of the trained DQN
---- neighbor_model.pkl: Trained Neighbor Relationship Model
./HDRL: Folder for DRL related model and data.
--./Embryo: Environment for HDRL.
--./saved_data/: Folder that used for saving the output data when exploring the successful scenarios for HDRL. Currently stored npy file are used for figure 8.
--./trained_models/:Folder with all the pre-trained HDRL models.
---- hdrl_llmodel.pkl: Trained lower-level HDQN model.
---- hdrl_hlmodel.pkl: Trained Higher-level HDQN model.
---- neighbor_model.pkl: Trained Neighbor Relationship Model.
./data/: Folder with textual data of nuclei.
-- ./Cpaaa_[0-2]: Textual embryonic data of nuclei for Cpaaa migration training and evaluation.
-- data_description.txt: A brief description of the input textual embryonic data.
./observation: Data with textual data of nuclei for observational purpose.
-- ./embryo_data/[01-06]: Textual embryonic data of nuclei for observational purpose.
-- ./saved_data: Processed observational data for analysis.
-- observation_analysis.ipynb: Observational data analysis for figure 6.
simulation_data_visualization.ipynb: Simulation data analysis for figure 7, 8 and 10.
Explore the successful scenarios with DRL: Command: python3 ./DRL/simulation.py --em [0-2]
Explore the successful scenarios with HDRL: Command: python3 ./HDRL/simulation.py --em [0-2]
Two Files are generated in the 'saved_data' folder after the evaluation:
cpaaa_locations.pkl: Location of Cpaaa at each time step.
target_locations.pkl: Location of the target cell (ABarpaapp) at each time step.
Two pickle files generated for each run above can be used for data analysis and visualization for figure 7,8 and 10.
Q: No module name 'sklearn.forest.ensemble' after installing scikit-learn.
A: sklearn.ensemble.forest was renamed to sklearn.ensemble._forest, Please install an older sklearn version: pip install scikit-learn==0.22
Q: ImportError: numpy.core.multiarray failed to import.
A: Please try pip install numpy --upgrade
Q: Unable to install pytorch 0.4.1.
A: Please try pip install torch==0.4.1
Q: Unable to install gym with version 0.18.0.
A: Please try a newer verison: pip install gym==0.21.0
Will update after paper submission.