-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathresults2Nodes
More file actions
98 lines (98 loc) · 3.81 KB
/
results2Nodes
File metadata and controls
98 lines (98 loc) · 3.81 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
14/12/01 22:14:19 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/12/01 22:14:20 INFO client.RMProxy: Connecting to ResourceManager at adelie-05.cs.wisc.edu/128.105.112.48:8032
14/12/01 22:14:20 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
14/12/01 22:14:20 INFO input.FileInputFormat: Total input paths to process : 7
14/12/01 22:14:20 INFO mapreduce.JobSubmitter: number of splits:7
14/12/01 22:14:20 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1417493626080_0002
14/12/01 22:14:20 INFO impl.YarnClientImpl: Submitted application application_1417493626080_0002
14/12/01 22:14:20 INFO mapreduce.Job: The url to track the job: http://adelie-05.cs.wisc.edu:8088/proxy/application_1417493626080_0002/
14/12/01 22:14:20 INFO mapreduce.Job: Running job: job_1417493626080_0002
14/12/01 22:14:24 INFO mapreduce.Job: Job job_1417493626080_0002 running in uber mode : false
14/12/01 22:14:24 INFO mapreduce.Job: map 0% reduce 0%
14/12/01 22:14:30 INFO mapreduce.Job: map 14% reduce 0%
14/12/01 22:14:31 INFO mapreduce.Job: map 29% reduce 0%
14/12/01 22:14:34 INFO mapreduce.Job: map 71% reduce 0%
14/12/01 22:14:35 INFO mapreduce.Job: map 100% reduce 0%
14/12/01 22:14:36 INFO mapreduce.Job: map 100% reduce 100%
14/12/01 22:14:37 INFO mapreduce.Job: Job job_1417493626080_0002 completed successfully
14/12/01 22:14:37 INFO mapreduce.Job: Counters: 67
File System Counters
FILE: Number of bytes read=3782
FILE: Number of bytes written=781561
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=505252477
HDFS: Number of bytes written=1856
HDFS: Number of read operations=24
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=7
Launched reduce tasks=1
Data-local map tasks=7
Total time spent by all maps in occupied slots (ms)=44874
Total time spent by all reduces in occupied slots (ms)=2274
Total time spent by all map tasks (ms)=44874
Total time spent by all reduce tasks (ms)=2274
Total vcore-seconds taken by all map tasks=44874
Total vcore-seconds taken by all reduce tasks=2274
Total megabyte-seconds taken by all map tasks=45950976
Total megabyte-seconds taken by all reduce tasks=2328576
Map-Reduce Framework
Map input records=4102511
Map output records=27669
Map output bytes=603327
Map output materialized bytes=3818
Input split bytes=978
Combine input records=27669
Combine output records=154
Reduce input groups=86
Reduce shuffle bytes=3818
Reduce input records=154
Reduce output records=86
Spilled Records=308
Shuffled Maps =7
Failed Shuffles=0
Merged Map outputs=7
GC time elapsed (ms)=983
CPU time spent (ms)=23980
Physical memory (bytes) snapshot=2033004544
Virtual memory (bytes) snapshot=7842185216
Total committed heap usage (bytes)=1601175552
MapCount
galapagos-39.cs.wisc.edu=4891662
galapagos-40.cs.wisc.edu=3313350
MapDuration
galapagos-39.cs.wisc.edu=8381
galapagos-40.cs.wisc.edu=6101
MapLineCount
galapagos-39.cs.wisc.edu=2445830
galapagos-40.cs.wisc.edu=1656671
ReduceCount
galapagos-39.cs.wisc.edu=93
galapagos-40.cs.wisc.edu=147
ReduceDuration
galapagos-39.cs.wisc.edu=124
galapagos-40.cs.wisc.edu=107
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TotalTechWordCount
Fri Nov 14=2136
Fri Nov 21=5431
Sat Nov 29=211
Thu Nov 13=19891
TotalWordCount
Fri Nov 14=5596862
Fri Nov 21=16104623
Sat Nov 29=854368
Thu Nov 13=59430973
File Input Format Counters
Bytes Read=505251499
File Output Format Counters
Bytes Written=1856