JVM Client Code Debugging

Setup Remote Debugging

Clone HDFS locally (not in vagrant)

Using SSL: git clone git@github.com:apache/hadoop.git

Using HDFS: git clone https://github.com/apache/hadoop.git

Checkout version 3.0a git checkout 1002c58

Open project in IntelliJ CE Be sure to import the .pom file

Setup remote debug run configuration Debugger Mode: Attach Host: 33.33.33.33 Port: 8000

Setup Vagrant

Run this command in vagrant to make the HDFS library wait for a debugger on startup: export HADOOP_OPTS="$HADOOP_OPTS -agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=y"

Then, run the HDFS command you are interested in i.e. hdfs dfs -copyFromLocal foo /bar

Finally, press the debug button in IntelliJ, and the debugger should attach and start the client in Vagrant. The execution of the client should stop at any breakpoints that you have added in IntelliJ.

HDFS Tests

To run local HDFS tests To run the HDFS tests locally, you must build Hadoop on your machine. Follow the Hadoop developer guide here, and build Hadoop using Maven: https://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment