Set up volumes and bind mounts for Hadoop cluster

[huy-dataguy]

In the current setup, whenever a container is stopped or removed, all Hadoop cluster data, including distributed file system data and metadata, is lost. This makes it difficult to maintain persistent storage across container restarts.

To solve this issue, I introduce:

1. Docker Volumes:

• Store HDFS data and metadata persistently.
• Ensure data is retained even when containers are stopped or rebuilt.

2. Bind Mounts:

• Mount a data/ folder to facilitate easy file sharing between the host and container.
• Mount a scripts/ folder containing shell scripts for:
• Quickly starting HDFS and YARN.
• Running MapReduce WordCount tests with a single command.
• ...

Benefits:

• Data persistence: Prevents loss of Hadoop data across container restarts.
• Ease of use: Simplifies execution of common commands via pre-written scripts.
• Improved workflow: Enables seamless data transfer from host to container for practice and experimentation.

This enhancement improves the usability and persistence of the Hadoop cluster in a Docker environment