Error running basic_summary with docker container #15
Description
I just ran through the installation instructions with the docker container. While the example in the instructions for basic_http worked, basic_summary fails as follows:
root@2390b772f892:/src/packetpig# ./pigrun.py -f pig/examples/basic_summary.pig
['pig -v',
'-x local',
'-f pig/examples/basic_summary.pig',
'-param pcap=data/web.pcap',
'-param output=output',
'-param n=1',
'-param snortconfig=lib/snort/etc/snort.conf',
'-param tcppath=lib/scripts/tcp.py',
'-param dnspath=lib/scripts/dns_parser.py']
Executing pig -v -x local -f pig/examples/basic_summary.pig -param pcap=data/web.pcap -param output=output -param n=1 -param snortconfig=lib/snort/etc/snort.conf -param tcppath=lib/scripts/tcp.py -param dnspath=lib/scripts/dns_parser.py
2014-04-14 00:31:03,151 [main] INFO org.apache.pig.Main - Logging error messages to: /src/packetpig/pig_1397435463143.log
2014-04-14 00:31:03,245 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
2014-04-14 00:31:03,472 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2014-04-14 00:31:03,472 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - pig.usenewlogicalplan is set to true. New logical plan will be used.
2014-04-14 00:31:03,521 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for packets: $0, $1, $2, $3, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32
2014-04-14 00:31:03,567 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: summary: Store(file:/tmp/temp-1405130183/tmp-2050501636:org.apache.pig.impl.io.InterStorage) - scope-6 Operator Key: scope-6)
2014-04-14 00:31:03,573 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-04-14 00:31:03,587 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-04-14 00:31:03,587 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2014-04-14 00:31:03,632 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=
2014-04-14 00:31:03,639 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2014-04-14 00:31:03,656 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2014-04-14 00:31:05,902 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2014-04-14 00:31:05,925 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-14 00:31:05,925 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2014-04-14 00:31:05,934 [Thread-1] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-04-14 00:31:06,052 [Thread-1] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2014-04-14 00:31:06,054 [Thread-1] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2014-04-14 00:31:06,251 [Thread-12] INFO org.apache.hadoop.mapred.LocalJobRunner - Waiting for map tasks
2014-04-14 00:31:06,253 [pool-1-thread-1] INFO org.apache.hadoop.mapred.LocalJobRunner - Starting task: attempt_local_0001_m_000000_0
2014-04-14 00:31:06,296 [pool-1-thread-1] INFO org.apache.hadoop.util.ProcessTree - setsid exited with exit code 0
2014-04-14 00:31:06,302 [pool-1-thread-1] INFO org.apache.hadoop.mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@64498713
2014-04-14 00:31:06,306 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 358584
Input split[0]:
Length = 358584
Locations:
2014-04-14 00:31:06,364 [Thread-12] INFO org.apache.hadoop.mapred.LocalJobRunner - Map task executor complete.
2014-04-14 00:31:06,365 [Thread-12] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.Exception: org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while computing count in COUNT
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:349)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while computing count in COUNT
at org.apache.pig.builtin.COUNT.exec(COUNT.java:71)
at org.apache.pig.builtin.COUNT.exec(COUNT.java:51)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:245)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:322)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:360)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:237)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:648)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:218)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.pig.data.DataBag
at org.apache.pig.builtin.COUNT.exec(COUNT.java:57)
... 18 more
2014-04-14 00:31:06,426 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local_0001
2014-04-14 00:31:06,426 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2014-04-14 00:31:11,427 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_local_0001 has failed! Stop running all dependent jobs
2014-04-14 00:31:11,431 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2014-04-14 00:31:11,432 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2014-04-14 00:31:11,432 [main] INFO org.apache.pig.tools.pigstats.PigStats - Detected Local mode. Stats reported below may be incomplete
2014-04-14 00:31:11,433 [main] INFO org.apache.pig.tools.pigstats.PigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
0.20.2-cdh3u6 0.8.1-cdh3u6 root 2014-04-14 00:31:03 2014-04-14 00:31:11 UNKNOWN
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_local_0001 packets,summary MAP_ONLY Message: Job failed! Error - NA file:/tmp/temp-1405130183/tmp-2050501636,
Input(s):
Failed to read data from "file:///src/packetpig/data/web.pcap"
Output(s):
Failed to produce result in "file:/tmp/temp-1405130183/tmp-2050501636"
Job DAG:
job_local_0001
2014-04-14 00:31:11,433 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2014-04-14 00:31:11,436 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias summary
2014-04-14 00:31:11,436 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias summary
at org.apache.pig.PigServer.openIterator(PigServer.java:765)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:644)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:171)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:147)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:396)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
at org.apache.pig.PigServer.openIterator(PigServer.java:755)
... 7 more
Details also at logfile: /src/packetpig/pig_1397435463143.log
root@2390b772f892:/src/packetpig# cat pig_1397435463143.log
Pig Stack Trace
ERROR 1066: Unable to open iterator for alias summary
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias summary
at org.apache.pig.PigServer.openIterator(PigServer.java:765)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:644)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:171)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:147)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:396)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
at org.apache.pig.PigServer.openIterator(PigServer.java:755)