2015-03-23 05:39:48,540 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-03-23 05:39:49,154 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-03-23 05:39:49,154 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system started
2015-03-23 05:39:49,198 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2015-03-23 05:39:49,198 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1427088391284_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@7abbee99)
2015-03-23 05:39:49,638 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2015-03-23 05:39:51,083 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /tmp/hadoop-ubuntu/nm-local-dir/usercache/ubuntu/appcache/application_1427088391284_0001
2015-03-23 05:39:55,492 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2015-03-23 05:39:57,417 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2015-03-23 05:39:57,559 INFO [main] org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5acb5e96
2015-03-23 05:39:57,624 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=1503238528, maxSingleShuffleLimit=375809632, mergeThreshold=992137472, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-03-23 05:39:57,675 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000094_0 Thread started: EventFetcher for fetching Map Completion Events
2015-03-23 05:39:57,750 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-daca09d5:13562 with 1 to fetcher#5
2015-03-23 05:39:57,750 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-daca09d5:13562 to fetcher#5
2015-03-23 05:39:57,751 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9cca0993:13562 with 1 to fetcher#1
2015-03-23 05:39:57,752 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9cca0993:13562 to fetcher#1
2015-03-23 05:39:57,752 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6fca0960:13562 with 1 to fetcher#4
2015-03-23 05:39:57,753 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6fca0960:13562 to fetcher#4
2015-03-23 05:39:57,761 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8fca0980:13562 with 1 to fetcher#2
2015-03-23 05:39:57,761 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8fca0980:13562 to fetcher#2
2015-03-23 05:39:57,796 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-33ca093c:13562 with 1 to fetcher#3
2015-03-23 05:39:57,796 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-33ca093c:13562 to fetcher#3
2015-03-23 05:39:57,853 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000094_0: Got 48 new map-outputs
2015-03-23 05:39:58,446 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000011_0 sent hash and received reply
2015-03-23 05:39:58,448 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000013_0 sent hash and received reply
2015-03-23 05:39:58,450 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000021_0 sent hash and received reply
2015-03-23 05:39:58,452 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000038_0 sent hash and received reply
2015-03-23 05:39:58,483 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000013_0 decomp: 908728 len: 908732 to MEMORY
2015-03-23 05:39:58,485 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000011_0 decomp: 913095 len: 913099 to MEMORY
2015-03-23 05:39:58,485 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000021_0 decomp: 910240 len: 910244 to MEMORY
2015-03-23 05:39:58,492 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000038_0 decomp: 910252 len: 910256 to MEMORY
2015-03-23 05:39:58,529 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000017_0 sent hash and received reply
2015-03-23 05:39:58,532 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000017_0 decomp: 913668 len: 913672 to MEMORY
2015-03-23 05:39:58,544 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913095 bytes from map-output for attempt_1427088391284_0001_m_000011_0
2015-03-23 05:39:58,599 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910240 bytes from map-output for attempt_1427088391284_0001_m_000021_0
2015-03-23 05:39:58,771 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910240, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->4555983
2015-03-23 05:39:58,774 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8fca0980:13562 freed by fetcher#2 in 1013ms
2015-03-23 05:39:58,774 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6bca0964:13562 with 1 to fetcher#2
2015-03-23 05:39:58,774 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6bca0964:13562 to fetcher#2
2015-03-23 05:39:58,775 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913095, inMemoryMapOutputs.size() -> 2, commitMemory -> 910240, usedMemory ->4555983
2015-03-23 05:39:58,775 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-33ca093c:13562 freed by fetcher#3 in 979ms
2015-03-23 05:39:58,776 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-04c90a0b:13562 with 1 to fetcher#3
2015-03-23 05:39:58,776 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-04c90a0b:13562 to fetcher#3
2015-03-23 05:39:58,783 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000009_0 sent hash and received reply
2015-03-23 05:39:58,784 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000009_0 decomp: 910533 len: 910537 to MEMORY
2015-03-23 05:39:58,784 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000046_0 sent hash and received reply
2015-03-23 05:39:58,878 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910252 bytes from map-output for attempt_1427088391284_0001_m_000038_0
2015-03-23 05:39:58,880 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908728 bytes from map-output for attempt_1427088391284_0001_m_000013_0
2015-03-23 05:39:58,882 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000046_0 decomp: 904273 len: 904277 to MEMORY
2015-03-23 05:39:58,884 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910252, inMemoryMapOutputs.size() -> 3, commitMemory -> 1823335, usedMemory ->6370789
2015-03-23 05:39:58,884 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-daca09d5:13562 freed by fetcher#5 in 1134ms
2015-03-23 05:39:58,884 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6cca0963:13562 with 1 to fetcher#5
2015-03-23 05:39:58,885 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6cca0963:13562 to fetcher#5
2015-03-23 05:39:58,885 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908728, inMemoryMapOutputs.size() -> 4, commitMemory -> 2733587, usedMemory ->6370789
2015-03-23 05:39:58,886 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9cca0993:13562 freed by fetcher#1 in 1135ms
2015-03-23 05:39:58,886 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-92ca099d:13562 with 1 to fetcher#1
2015-03-23 05:39:58,886 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-92ca099d:13562 to fetcher#1
2015-03-23 05:39:59,015 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000023_0 sent hash and received reply
2015-03-23 05:39:59,016 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000023_0 decomp: 905026 len: 905030 to MEMORY
2015-03-23 05:39:59,025 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000015_0 sent hash and received reply
2015-03-23 05:39:59,026 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000015_0 decomp: 909786 len: 909790 to MEMORY
2015-03-23 05:39:59,028 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913668 bytes from map-output for attempt_1427088391284_0001_m_000017_0
2015-03-23 05:39:59,028 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913668, inMemoryMapOutputs.size() -> 5, commitMemory -> 3642315, usedMemory ->8185601
2015-03-23 05:39:59,028 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6fca0960:13562 freed by fetcher#4 in 1276ms
2015-03-23 05:39:59,028 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-acca09a3:13562 with 1 to fetcher#4
2015-03-23 05:39:59,028 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-acca09a3:13562 to fetcher#4
2015-03-23 05:39:59,030 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904273 bytes from map-output for attempt_1427088391284_0001_m_000046_0
2015-03-23 05:39:59,030 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904273, inMemoryMapOutputs.size() -> 6, commitMemory -> 4555983, usedMemory ->8185601
2015-03-23 05:39:59,105 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6bca0964:13562 freed by fetcher#2 in 330ms
2015-03-23 05:39:59,105 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b8ca09b7:13562 with 1 to fetcher#2
2015-03-23 05:39:59,105 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b8ca09b7:13562 to fetcher#2
2015-03-23 05:39:59,112 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910533 bytes from map-output for attempt_1427088391284_0001_m_000009_0
2015-03-23 05:39:59,112 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910533, inMemoryMapOutputs.size() -> 7, commitMemory -> 5460256, usedMemory ->8185601
2015-03-23 05:39:59,112 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-04c90a0b:13562 freed by fetcher#3 in 337ms
2015-03-23 05:39:59,112 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5eca0951:13562 with 1 to fetcher#3
2015-03-23 05:39:59,112 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5eca0951:13562 to fetcher#3
2015-03-23 05:39:59,118 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000002_0 sent hash and received reply
2015-03-23 05:39:59,118 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000002_0 decomp: 906267 len: 906271 to MEMORY
2015-03-23 05:39:59,120 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905026 bytes from map-output for attempt_1427088391284_0001_m_000023_0
2015-03-23 05:39:59,120 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905026, inMemoryMapOutputs.size() -> 8, commitMemory -> 6370789, usedMemory ->9091868
2015-03-23 05:39:59,120 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-92ca099d:13562 freed by fetcher#1 in 234ms
2015-03-23 05:39:59,120 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9aca0995:13562 with 1 to fetcher#1
2015-03-23 05:39:59,120 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9aca0995:13562 to fetcher#1
2015-03-23 05:39:59,126 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000028_0 sent hash and received reply
2015-03-23 05:39:59,127 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000028_0 decomp: 913382 len: 913386 to MEMORY
2015-03-23 05:39:59,168 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906267 bytes from map-output for attempt_1427088391284_0001_m_000002_0
2015-03-23 05:39:59,168 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906267, inMemoryMapOutputs.size() -> 9, commitMemory -> 7275815, usedMemory ->10005250
2015-03-23 05:39:59,169 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5eca0951:13562 freed by fetcher#3 in 57ms
2015-03-23 05:39:59,169 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-beca09b1:13562 with 1 to fetcher#3
2015-03-23 05:39:59,169 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-beca09b1:13562 to fetcher#3
2015-03-23 05:39:59,201 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000010_0 sent hash and received reply
2015-03-23 05:39:59,201 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000010_0 decomp: 910495 len: 910499 to MEMORY
2015-03-23 05:39:59,202 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913382 bytes from map-output for attempt_1427088391284_0001_m_000028_0
2015-03-23 05:39:59,202 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913382, inMemoryMapOutputs.size() -> 10, commitMemory -> 8182082, usedMemory ->10915745
2015-03-23 05:39:59,203 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9aca0995:13562 freed by fetcher#1 in 83ms
2015-03-23 05:39:59,203 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-97ca0998:13562 with 1 to fetcher#1
2015-03-23 05:39:59,203 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-97ca0998:13562 to fetcher#1
2015-03-23 05:39:59,270 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000016_0 sent hash and received reply
2015-03-23 05:39:59,271 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000016_0 decomp: 911265 len: 911269 to MEMORY
2015-03-23 05:39:59,298 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000044_0 sent hash and received reply
2015-03-23 05:39:59,599 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000041_0 sent hash and received reply
2015-03-23 05:39:59,601 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910495 bytes from map-output for attempt_1427088391284_0001_m_000010_0
2015-03-23 05:39:59,601 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000044_0 decomp: 908207 len: 908211 to MEMORY
2015-03-23 05:39:59,605 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910495, inMemoryMapOutputs.size() -> 11, commitMemory -> 9095464, usedMemory ->12735217
2015-03-23 05:39:59,606 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-beca09b1:13562 freed by fetcher#3 in 437ms
2015-03-23 05:39:59,606 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ccca09c3:13562 with 1 to fetcher#3
2015-03-23 05:39:59,606 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ccca09c3:13562 to fetcher#3
2015-03-23 05:39:59,607 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000041_0 decomp: 894980 len: 894984 to MEMORY
2015-03-23 05:39:59,686 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000018_0 sent hash and received reply
2015-03-23 05:39:59,687 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000018_0 decomp: 910106 len: 910110 to MEMORY
2015-03-23 05:39:59,700 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911265 bytes from map-output for attempt_1427088391284_0001_m_000016_0
2015-03-23 05:39:59,700 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911265, inMemoryMapOutputs.size() -> 12, commitMemory -> 10005959, usedMemory ->14540303
2015-03-23 05:39:59,701 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-acca09a3:13562 freed by fetcher#4 in 672ms
2015-03-23 05:39:59,701 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-dfca09d0:13562 with 1 to fetcher#4
2015-03-23 05:39:59,701 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-dfca09d0:13562 to fetcher#4
2015-03-23 05:39:59,704 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909786 bytes from map-output for attempt_1427088391284_0001_m_000015_0
2015-03-23 05:39:59,784 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909786, inMemoryMapOutputs.size() -> 13, commitMemory -> 10917224, usedMemory ->14540303
2015-03-23 05:39:59,794 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6cca0963:13562 freed by fetcher#5 in 910ms
2015-03-23 05:39:59,794 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-95ca099a:13562 with 1 to fetcher#5
2015-03-23 05:39:59,794 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-95ca099a:13562 to fetcher#5
2015-03-23 05:39:59,793 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 894980 bytes from map-output for attempt_1427088391284_0001_m_000041_0
2015-03-23 05:39:59,788 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000031_0 sent hash and received reply
2015-03-23 05:39:59,796 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000031_0 decomp: 910898 len: 910902 to MEMORY
2015-03-23 05:39:59,796 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 894980, inMemoryMapOutputs.size() -> 14, commitMemory -> 11827010, usedMemory ->15451201
2015-03-23 05:39:59,797 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b8ca09b7:13562 freed by fetcher#2 in 691ms
2015-03-23 05:39:59,797 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-0bc90a04:13562 with 1 to fetcher#2
2015-03-23 05:39:59,797 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-0bc90a04:13562 to fetcher#2
2015-03-23 05:39:59,825 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910106 bytes from map-output for attempt_1427088391284_0001_m_000018_0
2015-03-23 05:39:59,825 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910106, inMemoryMapOutputs.size() -> 15, commitMemory -> 12721990, usedMemory ->15451201
2015-03-23 05:39:59,825 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ccca09c3:13562 freed by fetcher#3 in 219ms
2015-03-23 05:39:59,825 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f4ca09fb:13562 with 1 to fetcher#3
2015-03-23 05:39:59,826 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f4ca09fb:13562 to fetcher#3
2015-03-23 05:39:59,828 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908207 bytes from map-output for attempt_1427088391284_0001_m_000044_0
2015-03-23 05:39:59,828 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908207, inMemoryMapOutputs.size() -> 16, commitMemory -> 13632096, usedMemory ->15451201
2015-03-23 05:39:59,829 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-97ca0998:13562 freed by fetcher#1 in 625ms
2015-03-23 05:39:59,829 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c2ca09cd:13562 with 1 to fetcher#1
2015-03-23 05:39:59,829 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c2ca09cd:13562 to fetcher#1
2015-03-23 05:39:59,834 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000037_0 sent hash and received reply
2015-03-23 05:39:59,834 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000037_0 decomp: 912826 len: 912830 to MEMORY
2015-03-23 05:39:59,835 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000014_0 sent hash and received reply
2015-03-23 05:39:59,836 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000014_0 decomp: 911735 len: 911739 to MEMORY
2015-03-23 05:39:59,837 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000025_0 sent hash and received reply
2015-03-23 05:39:59,843 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000025_0 decomp: 907667 len: 907671 to MEMORY
2015-03-23 05:39:59,886 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000019_0 sent hash and received reply
2015-03-23 05:39:59,887 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000019_0 decomp: 909293 len: 909297 to MEMORY
2015-03-23 05:39:59,927 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910898 bytes from map-output for attempt_1427088391284_0001_m_000031_0
2015-03-23 05:39:59,927 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910898, inMemoryMapOutputs.size() -> 17, commitMemory -> 14540303, usedMemory ->19092722
2015-03-23 05:39:59,927 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-dfca09d0:13562 freed by fetcher#4 in 226ms
2015-03-23 05:39:59,927 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-09c90a06:13562 with 1 to fetcher#4
2015-03-23 05:39:59,927 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-09c90a06:13562 to fetcher#4
2015-03-23 05:39:59,970 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000047_0 sent hash and received reply
2015-03-23 05:39:59,971 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000047_0 decomp: 895943 len: 895947 to MEMORY
2015-03-23 05:39:59,975 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907667 bytes from map-output for attempt_1427088391284_0001_m_000025_0
2015-03-23 05:39:59,975 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907667, inMemoryMapOutputs.size() -> 18, commitMemory -> 15451201, usedMemory ->19988665
2015-03-23 05:39:59,976 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c2ca09cd:13562 freed by fetcher#1 in 147ms
2015-03-23 05:39:59,976 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b3ca09bc:13562 with 1 to fetcher#1
2015-03-23 05:39:59,976 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b3ca09bc:13562 to fetcher#1
2015-03-23 05:39:59,982 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909293 bytes from map-output for attempt_1427088391284_0001_m_000019_0
2015-03-23 05:39:59,982 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909293, inMemoryMapOutputs.size() -> 19, commitMemory -> 16358868, usedMemory ->19988665
2015-03-23 05:39:59,982 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-0bc90a04:13562 freed by fetcher#2 in 185ms
2015-03-23 05:39:59,982 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ddca09d2:13562 with 1 to fetcher#2
2015-03-23 05:39:59,982 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ddca09d2:13562 to fetcher#2
2015-03-23 05:39:59,984 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912826 bytes from map-output for attempt_1427088391284_0001_m_000037_0
2015-03-23 05:39:59,984 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912826, inMemoryMapOutputs.size() -> 20, commitMemory -> 17268161, usedMemory ->19988665
2015-03-23 05:39:59,985 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-95ca099a:13562 freed by fetcher#5 in 191ms
2015-03-23 05:39:59,985 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5fca0950:13562 with 1 to fetcher#5
2015-03-23 05:39:59,985 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5fca0950:13562 to fetcher#5
2015-03-23 05:39:59,986 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000040_0 sent hash and received reply
2015-03-23 05:39:59,990 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000040_0 decomp: 898030 len: 898034 to MEMORY
2015-03-23 05:39:59,991 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911735 bytes from map-output for attempt_1427088391284_0001_m_000014_0
2015-03-23 05:39:59,991 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911735, inMemoryMapOutputs.size() -> 21, commitMemory -> 18180987, usedMemory ->20886695
2015-03-23 05:39:59,992 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f4ca09fb:13562 freed by fetcher#3 in 167ms
2015-03-23 05:40:00,026 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5dca0952:13562 with 1 to fetcher#3
2015-03-23 05:40:00,026 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5dca0952:13562 to fetcher#3
2015-03-23 05:40:00,026 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000035_0 sent hash and received reply
2015-03-23 05:40:00,029 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000035_0 decomp: 906888 len: 906892 to MEMORY
2015-03-23 05:40:00,065 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000008_0 sent hash and received reply
2015-03-23 05:40:00,068 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000008_0 decomp: 912431 len: 912435 to MEMORY
2015-03-23 05:40:00,085 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000036_0 sent hash and received reply
2015-03-23 05:40:00,166 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 898030 bytes from map-output for attempt_1427088391284_0001_m_000040_0
2015-03-23 05:40:00,167 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000036_0 decomp: 914704 len: 914708 to MEMORY
2015-03-23 05:40:00,168 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 898030, inMemoryMapOutputs.size() -> 22, commitMemory -> 19092722, usedMemory ->23620718
2015-03-23 05:40:00,168 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b3ca09bc:13562 freed by fetcher#1 in 192ms
2015-03-23 05:40:00,169 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bdca09b2:13562 with 1 to fetcher#1
2015-03-23 05:40:00,169 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bdca09b2:13562 to fetcher#1
2015-03-23 05:40:00,174 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 895943 bytes from map-output for attempt_1427088391284_0001_m_000047_0
2015-03-23 05:40:00,174 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 895943, inMemoryMapOutputs.size() -> 23, commitMemory -> 19990752, usedMemory ->23620718
2015-03-23 05:40:00,174 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-09c90a06:13562 freed by fetcher#4 in 247ms
2015-03-23 05:40:00,174 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-d7ca09d8:13562 with 1 to fetcher#4
2015-03-23 05:40:00,174 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-d7ca09d8:13562 to fetcher#4
2015-03-23 05:40:00,182 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000000_0 sent hash and received reply
2015-03-23 05:40:00,211 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000000_0 decomp: 905474 len: 905478 to MEMORY
2015-03-23 05:40:00,216 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912431 bytes from map-output for attempt_1427088391284_0001_m_000008_0
2015-03-23 05:40:00,216 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912431, inMemoryMapOutputs.size() -> 24, commitMemory -> 20886695, usedMemory ->24526192
2015-03-23 05:40:00,217 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5dca0952:13562 freed by fetcher#3 in 191ms
2015-03-23 05:40:00,217 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-64ca096b:13562 with 1 to fetcher#3
2015-03-23 05:40:00,217 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-64ca096b:13562 to fetcher#3
2015-03-23 05:40:00,224 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906888 bytes from map-output for attempt_1427088391284_0001_m_000035_0
2015-03-23 05:40:00,224 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906888, inMemoryMapOutputs.size() -> 25, commitMemory -> 21799126, usedMemory ->24526192
2015-03-23 05:40:00,226 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ddca09d2:13562 freed by fetcher#2 in 244ms
2015-03-23 05:40:00,226 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-08c90a07:13562 with 1 to fetcher#2
2015-03-23 05:40:00,226 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-08c90a07:13562 to fetcher#2
2015-03-23 05:40:00,227 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000001_0 sent hash and received reply
2015-03-23 05:40:00,230 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000001_0 decomp: 908000 len: 908004 to MEMORY
2015-03-23 05:40:00,258 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000045_0 sent hash and received reply
2015-03-23 05:40:00,258 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000045_0 decomp: 905392 len: 905396 to MEMORY
2015-03-23 05:40:00,259 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905474 bytes from map-output for attempt_1427088391284_0001_m_000000_0
2015-03-23 05:40:00,259 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905474, inMemoryMapOutputs.size() -> 26, commitMemory -> 22706014, usedMemory ->26339584
2015-03-23 05:40:00,259 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bdca09b2:13562 freed by fetcher#1 in 91ms
2015-03-23 05:40:00,259 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c5ca09ca:13562 with 1 to fetcher#1
2015-03-23 05:40:00,260 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c5ca09ca:13562 to fetcher#1
2015-03-23 05:40:00,261 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000027_0 sent hash and received reply
2015-03-23 05:40:00,264 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000027_0 decomp: 911751 len: 911755 to MEMORY
2015-03-23 05:40:00,267 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914704 bytes from map-output for attempt_1427088391284_0001_m_000036_0
2015-03-23 05:40:00,267 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914704, inMemoryMapOutputs.size() -> 27, commitMemory -> 23611488, usedMemory ->27251335
2015-03-23 05:40:00,272 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5fca0950:13562 freed by fetcher#5 in 287ms
2015-03-23 05:40:00,274 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-62ca096d:13562 with 1 to fetcher#5
2015-03-23 05:40:00,274 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-62ca096d:13562 to fetcher#5
2015-03-23 05:40:00,275 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908000 bytes from map-output for attempt_1427088391284_0001_m_000001_0
2015-03-23 05:40:00,275 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908000, inMemoryMapOutputs.size() -> 28, commitMemory -> 24526192, usedMemory ->27251335
2015-03-23 05:40:00,276 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-64ca096b:13562 freed by fetcher#3 in 59ms
2015-03-23 05:40:00,276 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-53ca095c:13562 with 1 to fetcher#3
2015-03-23 05:40:00,276 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-53ca095c:13562 to fetcher#3
2015-03-23 05:40:00,281 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000007_0 sent hash and received reply
2015-03-23 05:40:00,282 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000007_0 decomp: 907546 len: 907550 to MEMORY
2015-03-23 05:40:00,282 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000012_0 sent hash and received reply
2015-03-23 05:40:00,283 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000012_0 decomp: 912141 len: 912145 to MEMORY
2015-03-23 05:40:00,286 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911751 bytes from map-output for attempt_1427088391284_0001_m_000027_0
2015-03-23 05:40:00,286 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911751, inMemoryMapOutputs.size() -> 29, commitMemory -> 25434192, usedMemory ->29071022
2015-03-23 05:40:00,287 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-d7ca09d8:13562 freed by fetcher#4 in 113ms
2015-03-23 05:40:00,287 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f2ca09fd:13562 with 1 to fetcher#4
2015-03-23 05:40:00,287 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f2ca09fd:13562 to fetcher#4
2015-03-23 05:40:00,292 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000043_0 sent hash and received reply
2015-03-23 05:40:00,292 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000043_0 decomp: 901482 len: 901486 to MEMORY
2015-03-23 05:40:00,336 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912141 bytes from map-output for attempt_1427088391284_0001_m_000012_0
2015-03-23 05:40:00,336 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912141, inMemoryMapOutputs.size() -> 30, commitMemory -> 26345943, usedMemory ->29972504
2015-03-23 05:40:00,337 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-53ca095c:13562 freed by fetcher#3 in 61ms
2015-03-23 05:40:00,337 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-51ca095e:13562 with 1 to fetcher#3
2015-03-23 05:40:00,337 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-51ca095e:13562 to fetcher#3
2015-03-23 05:40:00,370 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000006_0 sent hash and received reply
2015-03-23 05:40:00,371 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000006_0 decomp: 907911 len: 907915 to MEMORY
2015-03-23 05:40:00,376 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907546 bytes from map-output for attempt_1427088391284_0001_m_000007_0
2015-03-23 05:40:00,376 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907546, inMemoryMapOutputs.size() -> 31, commitMemory -> 27258084, usedMemory ->30880415
2015-03-23 05:40:00,377 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-62ca096d:13562 freed by fetcher#5 in 102ms
2015-03-23 05:40:00,377 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8aca0985:13562 with 1 to fetcher#5
2015-03-23 05:40:00,377 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8aca0985:13562 to fetcher#5
2015-03-23 05:40:00,385 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 901482 bytes from map-output for attempt_1427088391284_0001_m_000043_0
2015-03-23 05:40:00,385 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 901482, inMemoryMapOutputs.size() -> 32, commitMemory -> 28165630, usedMemory ->30880415
2015-03-23 05:40:00,385 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f2ca09fd:13562 freed by fetcher#4 in 98ms
2015-03-23 05:40:00,385 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-a8ca09a7:13562 with 1 to fetcher#4
2015-03-23 05:40:00,385 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-a8ca09a7:13562 to fetcher#4
2015-03-23 05:40:00,414 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000034_0 sent hash and received reply
2015-03-23 05:40:00,415 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000034_0 decomp: 909120 len: 909124 to MEMORY
2015-03-23 05:40:00,415 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000039_0 sent hash and received reply
2015-03-23 05:40:00,416 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000039_0 decomp: 898240 len: 898244 to MEMORY
2015-03-23 05:40:00,436 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905392 bytes from map-output for attempt_1427088391284_0001_m_000045_0
2015-03-23 05:40:00,436 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905392, inMemoryMapOutputs.size() -> 33, commitMemory -> 29067112, usedMemory ->32687775
2015-03-23 05:40:00,436 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-08c90a07:13562 freed by fetcher#2 in 210ms
2015-03-23 05:40:00,436 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-e3ca09ec:13562 with 1 to fetcher#2
2015-03-23 05:40:00,436 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-e3ca09ec:13562 to fetcher#2
2015-03-23 05:40:00,530 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907911 bytes from map-output for attempt_1427088391284_0001_m_000006_0
2015-03-23 05:40:00,531 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000003_0 sent hash and received reply
2015-03-23 05:40:00,607 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000003_0 decomp: 909956 len: 909960 to MEMORY
2015-03-23 05:40:00,608 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907911, inMemoryMapOutputs.size() -> 34, commitMemory -> 29972504, usedMemory ->33597731
2015-03-23 05:40:00,608 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-51ca095e:13562 freed by fetcher#3 in 271ms
2015-03-23 05:40:00,608 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8eca0981:13562 with 1 to fetcher#3
2015-03-23 05:40:00,608 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8eca0981:13562 to fetcher#3
2015-03-23 05:40:00,610 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 898240 bytes from map-output for attempt_1427088391284_0001_m_000039_0
2015-03-23 05:40:00,610 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 898240, inMemoryMapOutputs.size() -> 35, commitMemory -> 30880415, usedMemory ->33597731
2015-03-23 05:40:00,610 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8aca0985:13562 freed by fetcher#5 in 233ms
2015-03-23 05:40:00,610 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-82ca098d:13562 with 1 to fetcher#5
2015-03-23 05:40:00,610 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-82ca098d:13562 to fetcher#5
2015-03-23 05:40:00,898 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000020_0 sent hash and received reply
2015-03-23 05:40:00,899 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000020_0 decomp: 907540 len: 907544 to MEMORY
2015-03-23 05:40:00,900 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000005_0 sent hash and received reply
2015-03-23 05:40:00,901 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909120 bytes from map-output for attempt_1427088391284_0001_m_000034_0
2015-03-23 05:40:00,901 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000005_0 decomp: 905158 len: 905162 to MEMORY
2015-03-23 05:40:00,902 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909120, inMemoryMapOutputs.size() -> 36, commitMemory -> 31778655, usedMemory ->35410429
2015-03-23 05:40:00,903 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c5ca09ca:13562 freed by fetcher#1 in 644ms
2015-03-23 05:40:00,903 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bcca09b3:13562 with 1 to fetcher#1
2015-03-23 05:40:00,903 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bcca09b3:13562 to fetcher#1
2015-03-23 05:40:00,904 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000033_0 sent hash and received reply
2015-03-23 05:40:00,905 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000033_0 decomp: 911422 len: 911426 to MEMORY
2015-03-23 05:40:00,943 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000042_0 sent hash and received reply
2015-03-23 05:40:00,949 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000042_0 decomp: 901389 len: 901393 to MEMORY
2015-03-23 05:40:00,957 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911422 bytes from map-output for attempt_1427088391284_0001_m_000033_0
2015-03-23 05:40:00,957 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911422, inMemoryMapOutputs.size() -> 37, commitMemory -> 32687775, usedMemory ->37223240
2015-03-23 05:40:00,958 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-82ca098d:13562 freed by fetcher#5 in 348ms
2015-03-23 05:40:00,959 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905158 bytes from map-output for attempt_1427088391284_0001_m_000005_0
2015-03-23 05:40:00,965 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909956 bytes from map-output for attempt_1427088391284_0001_m_000003_0
2015-03-23 05:40:00,965 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-7eca0971:13562 with 1 to fetcher#5
2015-03-23 05:40:00,990 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909956, inMemoryMapOutputs.size() -> 38, commitMemory -> 33599197, usedMemory ->37223240
2015-03-23 05:40:00,991 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-a8ca09a7:13562 freed by fetcher#4 in 605ms
2015-03-23 05:40:00,991 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-00c90a0f:13562 with 1 to fetcher#4
2015-03-23 05:40:00,991 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-00c90a0f:13562 to fetcher#4
2015-03-23 05:40:00,992 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905158, inMemoryMapOutputs.size() -> 39, commitMemory -> 34509153, usedMemory ->37223240
2015-03-23 05:40:00,992 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8eca0981:13562 freed by fetcher#3 in 384ms
2015-03-23 05:40:00,992 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-cfca09c0:13562 with 1 to fetcher#3
2015-03-23 05:40:00,992 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-cfca09c0:13562 to fetcher#3
2015-03-23 05:40:00,999 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000029_0 sent hash and received reply
2015-03-23 05:40:01,000 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000029_0 decomp: 910259 len: 910263 to MEMORY
2015-03-23 05:40:01,000 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-7eca0971:13562 to fetcher#5
2015-03-23 05:40:01,012 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 901389 bytes from map-output for attempt_1427088391284_0001_m_000042_0
2015-03-23 05:40:01,012 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 901389, inMemoryMapOutputs.size() -> 40, commitMemory -> 35414311, usedMemory ->38133499
2015-03-23 05:40:01,013 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bcca09b3:13562 freed by fetcher#1 in 109ms
2015-03-23 05:40:01,013 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b5ca09ba:13562 with 1 to fetcher#1
2015-03-23 05:40:01,013 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b5ca09ba:13562 to fetcher#1
2015-03-23 05:40:01,040 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910259 bytes from map-output for attempt_1427088391284_0001_m_000029_0
2015-03-23 05:40:01,040 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910259, inMemoryMapOutputs.size() -> 41, commitMemory -> 36315700, usedMemory ->38133499
2015-03-23 05:40:01,041 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-cfca09c0:13562 freed by fetcher#3 in 48ms
2015-03-23 05:40:01,041 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-02c90a0d:13562 with 1 to fetcher#3
2015-03-23 05:40:01,041 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-02c90a0d:13562 to fetcher#3
2015-03-23 05:40:01,042 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000024_0 sent hash and received reply
2015-03-23 05:40:01,042 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000032_0 sent hash and received reply
2015-03-23 05:40:01,043 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000032_0 decomp: 913635 len: 913639 to MEMORY
2015-03-23 05:40:01,044 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000024_0 decomp: 908824 len: 908828 to MEMORY
2015-03-23 05:40:01,050 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000004_0 sent hash and received reply
2015-03-23 05:40:01,050 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000004_0 decomp: 905714 len: 905718 to MEMORY
2015-03-23 05:40:01,055 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000026_0 sent hash and received reply
2015-03-23 05:40:01,056 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000026_0 decomp: 911057 len: 911061 to MEMORY
2015-03-23 05:40:01,106 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905714 bytes from map-output for attempt_1427088391284_0001_m_000004_0
2015-03-23 05:40:01,106 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905714, inMemoryMapOutputs.size() -> 42, commitMemory -> 37225959, usedMemory ->41772729
2015-03-23 05:40:01,107 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-02c90a0d:13562 freed by fetcher#3 in 65ms
2015-03-23 05:40:01,107 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-03c90a0c:13562 with 1 to fetcher#3
2015-03-23 05:40:01,107 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-03c90a0c:13562 to fetcher#3
2015-03-23 05:40:01,139 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911057 bytes from map-output for attempt_1427088391284_0001_m_000026_0
2015-03-23 05:40:01,139 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911057, inMemoryMapOutputs.size() -> 43, commitMemory -> 38131673, usedMemory ->41772729
2015-03-23 05:40:01,139 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b5ca09ba:13562 freed by fetcher#1 in 127ms
2015-03-23 05:40:01,139 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-81ca098e:13562 with 1 to fetcher#1
2015-03-23 05:40:01,139 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-81ca098e:13562 to fetcher#1
2015-03-23 05:40:01,144 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000030_0 sent hash and received reply
2015-03-23 05:40:01,144 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000030_0 decomp: 907698 len: 907702 to MEMORY
2015-03-23 05:40:01,151 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908824 bytes from map-output for attempt_1427088391284_0001_m_000024_0
2015-03-23 05:40:01,151 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908824, inMemoryMapOutputs.size() -> 44, commitMemory -> 39042730, usedMemory ->42680427
2015-03-23 05:40:01,151 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-7eca0971:13562 freed by fetcher#5 in 161ms
2015-03-23 05:40:01,158 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=94&map=attempt_1427088391284_0001_m_000022_0 sent hash and received reply
2015-03-23 05:40:01,158 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000022_0 decomp: 911563 len: 911567 to MEMORY
2015-03-23 05:40:01,163 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907698 bytes from map-output for attempt_1427088391284_0001_m_000030_0
2015-03-23 05:40:01,163 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907698, inMemoryMapOutputs.size() -> 45, commitMemory -> 39951554, usedMemory ->43591990
2015-03-23 05:40:01,163 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-81ca098e:13562 freed by fetcher#1 in 24ms
2015-03-23 05:40:01,177 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911563 bytes from map-output for attempt_1427088391284_0001_m_000022_0
2015-03-23 05:40:01,177 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911563, inMemoryMapOutputs.size() -> 46, commitMemory -> 40859252, usedMemory ->43591990
2015-03-23 05:40:01,177 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-03c90a0c:13562 freed by fetcher#3 in 70ms
2015-03-23 05:40:01,213 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907540 bytes from map-output for attempt_1427088391284_0001_m_000020_0
2015-03-23 05:40:01,213 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907540, inMemoryMapOutputs.size() -> 47, commitMemory -> 41770815, usedMemory ->43591990
2015-03-23 05:40:01,214 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-e3ca09ec:13562 freed by fetcher#2 in 777ms
2015-03-23 05:40:01,243 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913635 bytes from map-output for attempt_1427088391284_0001_m_000032_0
2015-03-23 05:40:01,243 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913635, inMemoryMapOutputs.size() -> 48, commitMemory -> 42678355, usedMemory ->43591990
2015-03-23 05:40:01,244 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2015-03-23 05:40:01,245 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-00c90a0f:13562 freed by fetcher#4 in 254ms
2015-03-23 05:40:01,249 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 48 in-memory map-outputs and 0 on-disk map-outputs
2015-03-23 05:40:01,278 INFO [main] org.apache.hadoop.mapred.Merger: Merging 48 sorted segments
2015-03-23 05:40:01,279 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 48 segments left of total size: 43591510 bytes
2015-03-23 05:40:12,046 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 48 segments, 43591990 bytes to disk to satisfy reduce memory limit
2015-03-23 05:40:12,047 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 43591900 bytes from disk
2015-03-23 05:40:12,048 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2015-03-23 05:40:12,048 INFO [main] org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2015-03-23 05:40:12,053 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 43591886 bytes
2015-03-23 05:40:12,093 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-03-23 05:40:12,614 INFO [main] org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2015-03-23 05:40:12,616 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2015-03-23 05:40:26,108 INFO [main] HiBench.HiveData$GenerateRankingsReducer: pid: 94, 0 erros, 0 missed
2015-03-23 05:40:26,797 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1427088391284_0001_r_000094_0 is done. And is in the process of committing
2015-03-23 05:40:26,861 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1427088391284_0001_r_000094_0 is allowed to commit now
2015-03-23 05:40:26,878 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1427088391284_0001_r_000094_0' to hdfs://172.31.17.135:8120/HiBench/Hive/Input-comp/rankings/_temporary/1/task_1427088391284_0001_r_000094
2015-03-23 05:40:26,941 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1427088391284_0001_r_000094_0' done.
2015-03-23 05:40:26,942 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ReduceTask metrics system...
2015-03-23 05:40:26,943 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system stopped.
2015-03-23 05:40:26,943 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system shutdown complete.
