ClassFormatError and NoSuchMethodError when using GZoltar to execute tests for JxPath #67
Description
Context
When using GZoltar to execute tests for JxPath-1, most of the tests failed with unexpected errors, i.e., 201 ClassFormatError and 83 NoSuchMethodError.
Steps to Reproduce
- Create a script
localize.shwith the following content. Please change the value of "work_dir" and "D4J_HOME" correspondingly.#!/bin/bash work_dir="/tmp/test" export D4J_HOME="/Users/xxx/defects4j/" rm -rf "$work_dir"; mkdir "$work_dir" export _JAVA_OPTIONS="-Xmx6144M -XX:MaxHeapSize=4096M" export MAVEN_OPTS="-Xmx1024M" export ANT_OPTS="-Xmx6144M -XX:MaxHeapSize=4096M" # # Get GZoltar # cd "$work_dir" if [ ! -d "$work_dir/gzoltar" ]; then git clone https://github.com/GZoltar/gzoltar.git cd "$work_dir/gzoltar" mvn clean package fi export GZOLTAR_AGENT_JAR="$work_dir/gzoltar/com.gzoltar.agent.rt/target/com.gzoltar.agent.rt-1.7.4-SNAPSHOT-all.jar" export GZOLTAR_CLI_JAR="$work_dir/gzoltar/com.gzoltar.cli/target/com.gzoltar.cli-1.7.4-SNAPSHOT-jar-with-dependencies.jar" # # # # Get D4J # # # cd "$work_dir" # git clone https://github.com/rjust/defects4j.git # cd "$work_dir/defects4j" # ./init.sh export TZ='America/Los_Angeles' # some D4J's requires this specific TimeZone export LC_ALL=en_US.UTF-8 export LANG=en_US.UTF-8 export LANGUAGE=en_US.UTF-8 # # Checkout JxPath-1b, compile it, and get its metadata # PID="JxPath" BID="1" # Checkout cd "$work_dir" rm -rf "$PID-${BID}b"; "$D4J_HOME/framework/bin/defects4j" checkout -p "$PID" -v "${BID}b" -w "$PID-${BID}b" # Compile cd "$work_dir/$PID-${BID}b" "$D4J_HOME/framework/bin/defects4j" compile # Collect metadata cd "$work_dir/$PID-${BID}b" test_classpath=$($D4J_HOME/framework/bin/defects4j export -p cp.test) src_classes_dir=$($D4J_HOME/framework/bin/defects4j export -p dir.bin.classes) src_classes_dir="$work_dir/$PID-${BID}b/$src_classes_dir" test_classes_dir=$($D4J_HOME/framework/bin/defects4j export -p dir.bin.tests) test_classes_dir="$work_dir/$PID-${BID}b/$test_classes_dir" echo "$PID-${BID}b's classpath: $test_classpath" >&2 echo "$PID-${BID}b's bin dir: $src_classes_dir" >&2 echo "$PID-${BID}b's test bin dir: $test_classes_dir" >&2 # # Collect unit tests to run GZoltar with # cd "$work_dir/$PID-${BID}b" unit_tests_file="$work_dir/$PID-${BID}b/unit_tests.txt" relevant_tests="*" # Note, you might want to consider the set of relevant tests provided by D4J, i.e., $D4J_HOME/framework/projects/$PID/relevant_tests/$BID java -cp "$D4J_HOME/framework/projects/lib/junit-4.11.jar:$test_classpath:$test_classes_dir:$GZOLTAR_CLI_JAR" \ com.gzoltar.cli.Main listTestMethods \ "$test_classes_dir" \ --outputFile "$unit_tests_file" \ --includes "$relevant_tests" head "$unit_tests_file" # # Collect classes to perform fault localization on # Note: the `sed` commands below might not work on BSD-based distributions such as MacOS. # cd "$work_dir/$PID-${BID}b" loaded_classes_file="$D4J_HOME/framework/projects/$PID/loaded_classes/$BID.src" if [[ "$OSTYPE" == "darwin"* ]]; then normal_classes=$(cat "$loaded_classes_file" | gsed 's/$/:/' | gsed ':a;N;$!ba;s/\n//g') inner_classes=$(cat "$loaded_classes_file" | gsed 's/$/$*:/' | gsed ':a;N;$!ba;s/\n//g') else normal_classes=$(cat "$loaded_classes_file" | sed 's/$/:/' | sed ':a;N;$!ba;s/\n//g') inner_classes=$(cat "$loaded_classes_file" | sed 's/$/$*:/' | sed ':a;N;$!ba;s/\n//g') fi classes_to_debug="$normal_classes$inner_classes" echo "Likely faulty classes: $classes_to_debug" >&2 # # Run GZoltar # cd "$work_dir/$PID-${BID}b" ser_file="$work_dir/$PID-${BID}b/gzoltar.ser" java -XX:MaxPermSize=4096M -javaagent:$GZOLTAR_AGENT_JAR=destfile=$ser_file,buildlocation=$src_classes_dir,includes=$classes_to_debug,excludes="",inclnolocationclasses=false,output="FILE" \ -cp "$D4J_HOME/framework/projects/lib/junit-4.11.jar:$src_classes_dir:$test_classpath:$test_classes_dir:$GZOLTAR_CLI_JAR" \ com.gzoltar.cli.Main runTestMethods \ --testMethods "$unit_tests_file" \ --collectCoverage # # Generate fault localization report # cd "$work_dir/$PID-${BID}b" java -cp "$D4J_HOME/framework/projects/lib/junit-4.11.jar:$src_classes_dir:$test_classpath:$GZOLTAR_CLI_JAR" \ com.gzoltar.cli.Main faultLocalizationReport \ --buildLocation "$src_classes_dir" \ --granularity "line" \ --inclPublicMethods \ --inclStaticConstructors \ --inclDeprecatedMethods \ --dataFile "$ser_file" \ --outputDirectory "$work_dir/$PID-${BID}b" \ --family "sfl" \ --formula "ochiai" \ --metric "entropy" \ --formatter "txt"
- Execute the script and redirect the output to a log file:
bash localize.sh |& tee localize.log - In the
localize.logfile, we can observe that there are 83NoSuchMethodErrorand 201ClassFormatError. However, JxPath-1b should only fail 2 tests withNullPointerException.
Expected behaviour
The test execution should only fail 2 tests with NullPointerException.
Environment (please complete the following information, if relevant):
I have reproduced the problem on both of my MacOS and Linux machines.
- MacOS 12.0.1 (21A559)
- Ubuntu 18.04.6 LTS