2015-03-23 05:39:55,137 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-03-23 05:39:55,731 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-03-23 05:39:55,731 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system started
2015-03-23 05:39:55,827 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2015-03-23 05:39:55,828 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1427088391284_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@7abbee99)
2015-03-23 05:39:56,361 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2015-03-23 05:39:57,674 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /tmp/hadoop-ubuntu/nm-local-dir/usercache/ubuntu/appcache/application_1427088391284_0001
2015-03-23 05:40:03,266 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2015-03-23 05:40:05,365 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2015-03-23 05:40:05,481 INFO [main] org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@502f719a
2015-03-23 05:40:05,655 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=1503238528, maxSingleShuffleLimit=375809632, mergeThreshold=992137472, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-03-23 05:40:05,665 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000011_0 Thread started: EventFetcher for fetching Map Completion Events
2015-03-23 05:40:05,715 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-daca09d5:13562 with 1 to fetcher#4
2015-03-23 05:40:05,715 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-daca09d5:13562 to fetcher#4
2015-03-23 05:40:05,747 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9cca0993:13562 with 1 to fetcher#5
2015-03-23 05:40:05,755 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9cca0993:13562 to fetcher#5
2015-03-23 05:40:05,758 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bdca09b2:13562 with 1 to fetcher#2
2015-03-23 05:40:05,758 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bdca09b2:13562 to fetcher#2
2015-03-23 05:40:05,758 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6cca0963:13562 with 1 to fetcher#3
2015-03-23 05:40:05,759 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6cca0963:13562 to fetcher#3
2015-03-23 05:40:05,759 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-dfca09d0:13562 with 1 to fetcher#1
2015-03-23 05:40:05,759 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-dfca09d0:13562 to fetcher#1
2015-03-23 05:40:05,819 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000011_0: Got 48 new map-outputs
2015-03-23 05:40:06,421 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000038_0 sent hash and received reply
2015-03-23 05:40:06,426 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000031_0 sent hash and received reply
2015-03-23 05:40:06,426 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000015_0 sent hash and received reply
2015-03-23 05:40:06,430 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000031_0 decomp: 910337 len: 910341 to MEMORY
2015-03-23 05:40:06,456 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000000_0 sent hash and received reply
2015-03-23 05:40:06,457 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000000_0 decomp: 912278 len: 912282 to MEMORY
2015-03-23 05:40:06,457 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000015_0 decomp: 910583 len: 910587 to MEMORY
2015-03-23 05:40:06,458 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000038_0 decomp: 909657 len: 909661 to MEMORY
2015-03-23 05:40:06,464 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000013_0 sent hash and received reply
2015-03-23 05:40:06,465 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000013_0 decomp: 905905 len: 905909 to MEMORY
2015-03-23 05:40:06,520 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905905 bytes from map-output for attempt_1427088391284_0001_m_000013_0
2015-03-23 05:40:06,526 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909657 bytes from map-output for attempt_1427088391284_0001_m_000038_0
2015-03-23 05:40:06,554 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910337 bytes from map-output for attempt_1427088391284_0001_m_000031_0
2015-03-23 05:40:06,555 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910337, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->4548760
2015-03-23 05:40:06,558 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909657, inMemoryMapOutputs.size() -> 2, commitMemory -> 910337, usedMemory ->4548760
2015-03-23 05:40:06,560 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-daca09d5:13562 freed by fetcher#4 in 845ms
2015-03-23 05:40:06,560 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-a8ca09a7:13562 with 1 to fetcher#4
2015-03-23 05:40:06,560 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-a8ca09a7:13562 to fetcher#4
2015-03-23 05:40:06,562 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-dfca09d0:13562 freed by fetcher#1 in 803ms
2015-03-23 05:40:06,562 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-0bc90a04:13562 with 1 to fetcher#1
2015-03-23 05:40:06,562 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-0bc90a04:13562 to fetcher#1
2015-03-23 05:40:06,563 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905905, inMemoryMapOutputs.size() -> 3, commitMemory -> 1819994, usedMemory ->4548760
2015-03-23 05:40:06,565 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9cca0993:13562 freed by fetcher#5 in 811ms
2015-03-23 05:40:06,566 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-33ca093c:13562 with 1 to fetcher#5
2015-03-23 05:40:06,566 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-33ca093c:13562 to fetcher#5
2015-03-23 05:40:06,566 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910583 bytes from map-output for attempt_1427088391284_0001_m_000015_0
2015-03-23 05:40:06,566 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910583, inMemoryMapOutputs.size() -> 4, commitMemory -> 2725899, usedMemory ->4548760
2015-03-23 05:40:06,566 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6cca0963:13562 freed by fetcher#3 in 808ms
2015-03-23 05:40:06,566 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c5ca09ca:13562 with 1 to fetcher#3
2015-03-23 05:40:06,567 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c5ca09ca:13562 to fetcher#3
2015-03-23 05:40:06,568 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912278 bytes from map-output for attempt_1427088391284_0001_m_000000_0
2015-03-23 05:40:06,568 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912278, inMemoryMapOutputs.size() -> 5, commitMemory -> 3636482, usedMemory ->4548760
2015-03-23 05:40:06,569 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bdca09b2:13562 freed by fetcher#2 in 811ms
2015-03-23 05:40:06,569 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ccca09c3:13562 with 1 to fetcher#2
2015-03-23 05:40:06,569 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ccca09c3:13562 to fetcher#2
2015-03-23 05:40:06,581 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000019_0 sent hash and received reply
2015-03-23 05:40:06,582 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000019_0 decomp: 908035 len: 908039 to MEMORY
2015-03-23 05:40:06,583 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000011_0 sent hash and received reply
2015-03-23 05:40:06,626 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000003_0 sent hash and received reply
2015-03-23 05:40:06,627 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000018_0 sent hash and received reply
2015-03-23 05:40:06,628 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000011_0 decomp: 912815 len: 912819 to MEMORY
2015-03-23 05:40:06,630 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000018_0 decomp: 915426 len: 915430 to MEMORY
2015-03-23 05:40:06,630 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000034_0 sent hash and received reply
2015-03-23 05:40:06,631 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000034_0 decomp: 914204 len: 914208 to MEMORY
2015-03-23 05:40:06,633 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000003_0 decomp: 909379 len: 909383 to MEMORY
2015-03-23 05:40:06,685 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912815 bytes from map-output for attempt_1427088391284_0001_m_000011_0
2015-03-23 05:40:06,685 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912815, inMemoryMapOutputs.size() -> 6, commitMemory -> 4548760, usedMemory ->9108619
2015-03-23 05:40:06,686 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-33ca093c:13562 freed by fetcher#5 in 120ms
2015-03-23 05:40:06,686 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-62ca096d:13562 with 1 to fetcher#5
2015-03-23 05:40:06,686 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-62ca096d:13562 to fetcher#5
2015-03-23 05:40:06,689 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908035 bytes from map-output for attempt_1427088391284_0001_m_000019_0
2015-03-23 05:40:06,690 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908035, inMemoryMapOutputs.size() -> 7, commitMemory -> 5461575, usedMemory ->9108619
2015-03-23 05:40:06,691 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-0bc90a04:13562 freed by fetcher#1 in 127ms
2015-03-23 05:40:06,691 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8fca0980:13562 with 1 to fetcher#1
2015-03-23 05:40:06,692 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8fca0980:13562 to fetcher#1
2015-03-23 05:40:06,694 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000007_0 sent hash and received reply
2015-03-23 05:40:06,695 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000007_0 decomp: 909035 len: 909039 to MEMORY
2015-03-23 05:40:06,721 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 915426 bytes from map-output for attempt_1427088391284_0001_m_000018_0
2015-03-23 05:40:06,722 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 915426, inMemoryMapOutputs.size() -> 8, commitMemory -> 6369610, usedMemory ->10017654
2015-03-23 05:40:06,722 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ccca09c3:13562 freed by fetcher#2 in 153ms
2015-03-23 05:40:06,722 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-08c90a07:13562 with 1 to fetcher#2
2015-03-23 05:40:06,722 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-08c90a07:13562 to fetcher#2
2015-03-23 05:40:06,728 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000021_0 sent hash and received reply
2015-03-23 05:40:06,729 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000021_0 decomp: 909415 len: 909419 to MEMORY
2015-03-23 05:40:06,730 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000045_0 sent hash and received reply
2015-03-23 05:40:06,732 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000045_0 decomp: 900826 len: 900830 to MEMORY
2015-03-23 05:40:06,793 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909379 bytes from map-output for attempt_1427088391284_0001_m_000003_0
2015-03-23 05:40:06,794 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909379, inMemoryMapOutputs.size() -> 9, commitMemory -> 7285036, usedMemory ->11827895
2015-03-23 05:40:06,794 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-a8ca09a7:13562 freed by fetcher#4 in 235ms
2015-03-23 05:40:06,795 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-53ca095c:13562 with 1 to fetcher#4
2015-03-23 05:40:06,795 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-53ca095c:13562 to fetcher#4
2015-03-23 05:40:06,798 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909035 bytes from map-output for attempt_1427088391284_0001_m_000007_0
2015-03-23 05:40:06,798 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909035, inMemoryMapOutputs.size() -> 10, commitMemory -> 8194415, usedMemory ->11827895
2015-03-23 05:40:06,799 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-62ca096d:13562 freed by fetcher#5 in 113ms
2015-03-23 05:40:06,799 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-acca09a3:13562 with 1 to fetcher#5
2015-03-23 05:40:06,799 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-acca09a3:13562 to fetcher#5
2015-03-23 05:40:06,801 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914204 bytes from map-output for attempt_1427088391284_0001_m_000034_0
2015-03-23 05:40:06,801 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914204, inMemoryMapOutputs.size() -> 11, commitMemory -> 9103450, usedMemory ->11827895
2015-03-23 05:40:06,801 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c5ca09ca:13562 freed by fetcher#3 in 234ms
2015-03-23 05:40:06,801 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ddca09d2:13562 with 1 to fetcher#3
2015-03-23 05:40:06,801 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ddca09d2:13562 to fetcher#3
2015-03-23 05:40:06,803 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000012_0 sent hash and received reply
2015-03-23 05:40:06,975 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909415 bytes from map-output for attempt_1427088391284_0001_m_000021_0
2015-03-23 05:40:06,976 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000035_0 sent hash and received reply
2015-03-23 05:40:06,978 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000012_0 decomp: 911221 len: 911225 to MEMORY
2015-03-23 05:40:06,982 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000035_0 decomp: 913958 len: 913962 to MEMORY
2015-03-23 05:40:06,983 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909415, inMemoryMapOutputs.size() -> 12, commitMemory -> 10017654, usedMemory ->13653074
2015-03-23 05:40:06,984 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8fca0980:13562 freed by fetcher#1 in 292ms
2015-03-23 05:40:06,984 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-cfca09c0:13562 with 1 to fetcher#1
2015-03-23 05:40:06,984 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-cfca09c0:13562 to fetcher#1
2015-03-23 05:40:07,032 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911221 bytes from map-output for attempt_1427088391284_0001_m_000012_0
2015-03-23 05:40:07,033 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911221, inMemoryMapOutputs.size() -> 13, commitMemory -> 10927069, usedMemory ->13653074
2015-03-23 05:40:07,033 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-53ca095c:13562 freed by fetcher#4 in 239ms
2015-03-23 05:40:07,034 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-7eca0971:13562 with 1 to fetcher#4
2015-03-23 05:40:07,034 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-7eca0971:13562 to fetcher#4
2015-03-23 05:40:07,035 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913958 bytes from map-output for attempt_1427088391284_0001_m_000035_0
2015-03-23 05:40:07,035 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913958, inMemoryMapOutputs.size() -> 14, commitMemory -> 11838290, usedMemory ->13653074
2015-03-23 05:40:07,036 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ddca09d2:13562 freed by fetcher#3 in 235ms
2015-03-23 05:40:07,036 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-04c90a0b:13562 with 1 to fetcher#3
2015-03-23 05:40:07,036 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-04c90a0b:13562 to fetcher#3
2015-03-23 05:40:07,049 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000016_0 sent hash and received reply
2015-03-23 05:40:07,051 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000029_0 sent hash and received reply
2015-03-23 05:40:07,051 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000016_0 decomp: 912612 len: 912616 to MEMORY
2015-03-23 05:40:07,052 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000029_0 decomp: 908168 len: 908172 to MEMORY
2015-03-23 05:40:07,060 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 900826 bytes from map-output for attempt_1427088391284_0001_m_000045_0
2015-03-23 05:40:07,060 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 900826, inMemoryMapOutputs.size() -> 15, commitMemory -> 12752248, usedMemory ->15473854
2015-03-23 05:40:07,060 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-08c90a07:13562 freed by fetcher#2 in 339ms
2015-03-23 05:40:07,061 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-51ca095e:13562 with 1 to fetcher#2
2015-03-23 05:40:07,062 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-51ca095e:13562 to fetcher#2
2015-03-23 05:40:07,064 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000009_0 sent hash and received reply
2015-03-23 05:40:07,065 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000009_0 decomp: 912766 len: 912770 to MEMORY
2015-03-23 05:40:07,098 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000006_0 sent hash and received reply
2015-03-23 05:40:07,099 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000006_0 decomp: 908587 len: 908591 to MEMORY
2015-03-23 05:40:07,113 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912766 bytes from map-output for attempt_1427088391284_0001_m_000009_0
2015-03-23 05:40:07,113 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912766, inMemoryMapOutputs.size() -> 16, commitMemory -> 13653074, usedMemory ->17295207
2015-03-23 05:40:07,113 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-04c90a0b:13562 freed by fetcher#3 in 77ms
2015-03-23 05:40:07,113 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-d7ca09d8:13562 with 1 to fetcher#3
2015-03-23 05:40:07,113 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-d7ca09d8:13562 to fetcher#3
2015-03-23 05:40:07,115 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908168 bytes from map-output for attempt_1427088391284_0001_m_000029_0
2015-03-23 05:40:07,115 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908168, inMemoryMapOutputs.size() -> 17, commitMemory -> 14565840, usedMemory ->17295207
2015-03-23 05:40:07,116 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-cfca09c0:13562 freed by fetcher#1 in 132ms
2015-03-23 05:40:07,116 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5fca0950:13562 with 1 to fetcher#1
2015-03-23 05:40:07,116 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5fca0950:13562 to fetcher#1
2015-03-23 05:40:07,141 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000024_0 sent hash and received reply
2015-03-23 05:40:07,142 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000024_0 decomp: 909010 len: 909014 to MEMORY
2015-03-23 05:40:07,146 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000027_0 sent hash and received reply
2015-03-23 05:40:07,147 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000027_0 decomp: 910441 len: 910445 to MEMORY
2015-03-23 05:40:07,158 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912612 bytes from map-output for attempt_1427088391284_0001_m_000016_0
2015-03-23 05:40:07,159 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912612, inMemoryMapOutputs.size() -> 18, commitMemory -> 15474008, usedMemory ->19114658
2015-03-23 05:40:07,159 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-acca09a3:13562 freed by fetcher#5 in 360ms
2015-03-23 05:40:07,159 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-95ca099a:13562 with 1 to fetcher#5
2015-03-23 05:40:07,159 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-95ca099a:13562 to fetcher#5
2015-03-23 05:40:07,160 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000036_0 sent hash and received reply
2015-03-23 05:40:07,161 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000036_0 decomp: 911061 len: 911065 to MEMORY
2015-03-23 05:40:07,202 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910441 bytes from map-output for attempt_1427088391284_0001_m_000027_0
2015-03-23 05:40:07,203 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910441, inMemoryMapOutputs.size() -> 19, commitMemory -> 16386620, usedMemory ->20025719
2015-03-23 05:40:07,204 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-d7ca09d8:13562 freed by fetcher#3 in 90ms
2015-03-23 05:40:07,204 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5dca0952:13562 with 1 to fetcher#3
2015-03-23 05:40:07,204 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5dca0952:13562 to fetcher#3
2015-03-23 05:40:07,206 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908587 bytes from map-output for attempt_1427088391284_0001_m_000006_0
2015-03-23 05:40:07,206 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908587, inMemoryMapOutputs.size() -> 20, commitMemory -> 17297061, usedMemory ->20025719
2015-03-23 05:40:07,206 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-51ca095e:13562 freed by fetcher#2 in 144ms
2015-03-23 05:40:07,206 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b8ca09b7:13562 with 1 to fetcher#2
2015-03-23 05:40:07,206 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b8ca09b7:13562 to fetcher#2
2015-03-23 05:40:07,215 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000008_0 sent hash and received reply
2015-03-23 05:40:07,216 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000037_0 sent hash and received reply
2015-03-23 05:40:07,221 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000037_0 decomp: 915316 len: 915320 to MEMORY
2015-03-23 05:40:07,224 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000041_0 sent hash and received reply
2015-03-23 05:40:07,252 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000008_0 decomp: 909297 len: 909301 to MEMORY
2015-03-23 05:40:07,299 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911061 bytes from map-output for attempt_1427088391284_0001_m_000036_0
2015-03-23 05:40:07,300 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911061, inMemoryMapOutputs.size() -> 21, commitMemory -> 18205648, usedMemory ->22748301
2015-03-23 05:40:07,300 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5fca0950:13562 freed by fetcher#1 in 184ms
2015-03-23 05:40:07,300 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b3ca09bc:13562 with 1 to fetcher#1
2015-03-23 05:40:07,300 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b3ca09bc:13562 to fetcher#1
2015-03-23 05:40:07,301 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909010 bytes from map-output for attempt_1427088391284_0001_m_000024_0
2015-03-23 05:40:07,302 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909010, inMemoryMapOutputs.size() -> 22, commitMemory -> 19116709, usedMemory ->22748301
2015-03-23 05:40:07,302 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-7eca0971:13562 freed by fetcher#4 in 268ms
2015-03-23 05:40:07,302 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bcca09b3:13562 with 1 to fetcher#4
2015-03-23 05:40:07,302 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bcca09b3:13562 to fetcher#4
2015-03-23 05:40:07,303 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000041_0 decomp: 897969 len: 897973 to MEMORY
2015-03-23 05:40:07,307 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000040_0 sent hash and received reply
2015-03-23 05:40:07,308 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000042_0 sent hash and received reply
2015-03-23 05:40:07,359 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000040_0 decomp: 898630 len: 898634 to MEMORY
2015-03-23 05:40:07,360 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000042_0 decomp: 903247 len: 903251 to MEMORY
2015-03-23 05:40:07,363 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 915316 bytes from map-output for attempt_1427088391284_0001_m_000037_0
2015-03-23 05:40:07,363 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 915316, inMemoryMapOutputs.size() -> 23, commitMemory -> 20025719, usedMemory ->24550178
2015-03-23 05:40:07,364 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-95ca099a:13562 freed by fetcher#5 in 205ms
2015-03-23 05:40:07,364 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-64ca096b:13562 with 1 to fetcher#5
2015-03-23 05:40:07,364 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-64ca096b:13562 to fetcher#5
2015-03-23 05:40:07,399 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000001_0 sent hash and received reply
2015-03-23 05:40:07,401 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000001_0 decomp: 912488 len: 912492 to MEMORY
2015-03-23 05:40:07,409 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 903247 bytes from map-output for attempt_1427088391284_0001_m_000042_0
2015-03-23 05:40:07,409 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 903247, inMemoryMapOutputs.size() -> 24, commitMemory -> 20941035, usedMemory ->25462666
2015-03-23 05:40:07,410 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bcca09b3:13562 freed by fetcher#4 in 108ms
2015-03-23 05:40:07,410 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-03c90a0c:13562 with 1 to fetcher#4
2015-03-23 05:40:07,410 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-03c90a0c:13562 to fetcher#4
2015-03-23 05:40:07,444 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 898630 bytes from map-output for attempt_1427088391284_0001_m_000040_0
2015-03-23 05:40:07,444 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 898630, inMemoryMapOutputs.size() -> 25, commitMemory -> 21844282, usedMemory ->25462666
2015-03-23 05:40:07,445 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909297 bytes from map-output for attempt_1427088391284_0001_m_000008_0
2015-03-23 05:40:07,445 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909297, inMemoryMapOutputs.size() -> 26, commitMemory -> 22742912, usedMemory ->25462666
2015-03-23 05:40:07,445 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5dca0952:13562 freed by fetcher#3 in 240ms
2015-03-23 05:40:07,449 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-00c90a0f:13562 with 1 to fetcher#3
2015-03-23 05:40:07,449 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-00c90a0f:13562 to fetcher#3
2015-03-23 05:40:07,451 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b3ca09bc:13562 freed by fetcher#1 in 151ms
2015-03-23 05:40:07,451 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9aca0995:13562 with 1 to fetcher#1
2015-03-23 05:40:07,451 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9aca0995:13562 to fetcher#1
2015-03-23 05:40:07,452 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000022_0 sent hash and received reply
2015-03-23 05:40:07,453 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000022_0 decomp: 908710 len: 908714 to MEMORY
2015-03-23 05:40:07,457 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000032_0 sent hash and received reply
2015-03-23 05:40:07,462 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000032_0 decomp: 911614 len: 911618 to MEMORY
2015-03-23 05:40:07,477 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000028_0 sent hash and received reply
2015-03-23 05:40:07,502 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912488 bytes from map-output for attempt_1427088391284_0001_m_000001_0
2015-03-23 05:40:07,507 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912488, inMemoryMapOutputs.size() -> 27, commitMemory -> 23652209, usedMemory ->27282990
2015-03-23 05:40:07,507 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-64ca096b:13562 freed by fetcher#5 in 144ms
2015-03-23 05:40:07,507 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b5ca09ba:13562 with 1 to fetcher#5
2015-03-23 05:40:07,508 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b5ca09ba:13562 to fetcher#5
2015-03-23 05:40:07,511 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000028_0 decomp: 906156 len: 906160 to MEMORY
2015-03-23 05:40:07,513 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911614 bytes from map-output for attempt_1427088391284_0001_m_000032_0
2015-03-23 05:40:07,513 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911614, inMemoryMapOutputs.size() -> 28, commitMemory -> 24564697, usedMemory ->28189146
2015-03-23 05:40:07,514 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-00c90a0f:13562 freed by fetcher#3 in 64ms
2015-03-23 05:40:07,514 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-82ca098d:13562 with 1 to fetcher#3
2015-03-23 05:40:07,514 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-82ca098d:13562 to fetcher#3
2015-03-23 05:40:07,532 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000033_0 sent hash and received reply
2015-03-23 05:40:07,534 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000033_0 decomp: 912564 len: 912568 to MEMORY
2015-03-23 05:40:07,534 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000026_0 sent hash and received reply
2015-03-23 05:40:07,535 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000026_0 decomp: 913176 len: 913180 to MEMORY
2015-03-23 05:40:07,536 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 897969 bytes from map-output for attempt_1427088391284_0001_m_000041_0
2015-03-23 05:40:07,536 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 897969, inMemoryMapOutputs.size() -> 29, commitMemory -> 25476311, usedMemory ->30014886
2015-03-23 05:40:07,536 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b8ca09b7:13562 freed by fetcher#2 in 330ms
2015-03-23 05:40:07,536 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-02c90a0d:13562 with 1 to fetcher#2
2015-03-23 05:40:07,536 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-02c90a0d:13562 to fetcher#2
2015-03-23 05:40:07,544 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908710 bytes from map-output for attempt_1427088391284_0001_m_000022_0
2015-03-23 05:40:07,544 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908710, inMemoryMapOutputs.size() -> 30, commitMemory -> 26374280, usedMemory ->30014886
2015-03-23 05:40:07,545 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-03c90a0c:13562 freed by fetcher#4 in 135ms
2015-03-23 05:40:07,545 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8aca0985:13562 with 1 to fetcher#4
2015-03-23 05:40:07,545 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8aca0985:13562 to fetcher#4
2015-03-23 05:40:07,547 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000004_0 sent hash and received reply
2015-03-23 05:40:07,547 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000004_0 decomp: 907519 len: 907523 to MEMORY
2015-03-23 05:40:07,586 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000039_0 sent hash and received reply
2015-03-23 05:40:07,587 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000039_0 decomp: 897781 len: 897785 to MEMORY
2015-03-23 05:40:07,596 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906156 bytes from map-output for attempt_1427088391284_0001_m_000028_0
2015-03-23 05:40:07,596 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906156, inMemoryMapOutputs.size() -> 31, commitMemory -> 27282990, usedMemory ->31820186
2015-03-23 05:40:07,597 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9aca0995:13562 freed by fetcher#1 in 146ms
2015-03-23 05:40:07,597 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c2ca09cd:13562 with 1 to fetcher#1
2015-03-23 05:40:07,597 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c2ca09cd:13562 to fetcher#1
2015-03-23 05:40:07,602 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913176 bytes from map-output for attempt_1427088391284_0001_m_000026_0
2015-03-23 05:40:07,603 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913176, inMemoryMapOutputs.size() -> 32, commitMemory -> 28189146, usedMemory ->31820186
2015-03-23 05:40:07,604 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b5ca09ba:13562 freed by fetcher#5 in 97ms
2015-03-23 05:40:07,604 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-97ca0998:13562 with 1 to fetcher#5
2015-03-23 05:40:07,605 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-97ca0998:13562 to fetcher#5
2015-03-23 05:40:07,608 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000025_0 sent hash and received reply
2015-03-23 05:40:07,640 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907519 bytes from map-output for attempt_1427088391284_0001_m_000004_0
2015-03-23 05:40:07,640 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907519, inMemoryMapOutputs.size() -> 33, commitMemory -> 29102322, usedMemory ->31820186
2015-03-23 05:40:07,641 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-02c90a0d:13562 freed by fetcher#2 in 104ms
2015-03-23 05:40:07,641 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-81ca098e:13562 with 1 to fetcher#2
2015-03-23 05:40:07,641 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-81ca098e:13562 to fetcher#2
2015-03-23 05:40:07,643 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000025_0 decomp: 908702 len: 908706 to MEMORY
2015-03-23 05:40:07,646 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000030_0 sent hash and received reply
2015-03-23 05:40:07,647 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000030_0 decomp: 912656 len: 912660 to MEMORY
2015-03-23 05:40:07,664 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000044_0 sent hash and received reply
2015-03-23 05:40:07,667 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000044_0 decomp: 902917 len: 902921 to MEMORY
2015-03-23 05:40:07,734 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912656 bytes from map-output for attempt_1427088391284_0001_m_000030_0
2015-03-23 05:40:07,736 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908702 bytes from map-output for attempt_1427088391284_0001_m_000025_0
2015-03-23 05:40:07,737 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908702, inMemoryMapOutputs.size() -> 34, commitMemory -> 30009841, usedMemory ->34544461
2015-03-23 05:40:07,737 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c2ca09cd:13562 freed by fetcher#1 in 140ms
2015-03-23 05:40:07,737 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-e3ca09ec:13562 with 1 to fetcher#1
2015-03-23 05:40:07,737 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-e3ca09ec:13562 to fetcher#1
2015-03-23 05:40:07,736 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 897781 bytes from map-output for attempt_1427088391284_0001_m_000039_0
2015-03-23 05:40:07,738 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 897781, inMemoryMapOutputs.size() -> 35, commitMemory -> 30918543, usedMemory ->34544461
2015-03-23 05:40:07,739 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8aca0985:13562 freed by fetcher#4 in 194ms
2015-03-23 05:40:07,739 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-beca09b1:13562 with 1 to fetcher#4
2015-03-23 05:40:07,739 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-beca09b1:13562 to fetcher#4
2015-03-23 05:40:07,735 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912564 bytes from map-output for attempt_1427088391284_0001_m_000033_0
2015-03-23 05:40:07,740 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912564, inMemoryMapOutputs.size() -> 36, commitMemory -> 31816324, usedMemory ->34544461
2015-03-23 05:40:07,740 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-82ca098d:13562 freed by fetcher#3 in 226ms
2015-03-23 05:40:07,740 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f2ca09fd:13562 with 1 to fetcher#3
2015-03-23 05:40:07,740 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f2ca09fd:13562 to fetcher#3
2015-03-23 05:40:07,740 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912656, inMemoryMapOutputs.size() -> 37, commitMemory -> 32728888, usedMemory ->34544461
2015-03-23 05:40:07,779 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-81ca098e:13562 freed by fetcher#2 in 139ms
2015-03-23 05:40:07,779 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8eca0981:13562 with 1 to fetcher#2
2015-03-23 05:40:07,780 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8eca0981:13562 to fetcher#2
2015-03-23 05:40:07,781 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000020_0 sent hash and received reply
2015-03-23 05:40:07,782 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000020_0 decomp: 908803 len: 908807 to MEMORY
2015-03-23 05:40:07,783 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000010_0 sent hash and received reply
2015-03-23 05:40:07,784 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000010_0 decomp: 913483 len: 913487 to MEMORY
2015-03-23 05:40:07,793 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000005_0 sent hash and received reply
2015-03-23 05:40:07,793 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000005_0 decomp: 906174 len: 906178 to MEMORY
2015-03-23 05:40:07,801 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000043_0 sent hash and received reply
2015-03-23 05:40:07,802 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000043_0 decomp: 903490 len: 903494 to MEMORY
2015-03-23 05:40:07,842 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913483 bytes from map-output for attempt_1427088391284_0001_m_000010_0
2015-03-23 05:40:07,842 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913483, inMemoryMapOutputs.size() -> 38, commitMemory -> 33641544, usedMemory ->38176411
2015-03-23 05:40:07,843 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-beca09b1:13562 freed by fetcher#4 in 104ms
2015-03-23 05:40:07,844 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-92ca099d:13562 with 1 to fetcher#4
2015-03-23 05:40:07,844 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-92ca099d:13562 to fetcher#4
2015-03-23 05:40:07,848 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908803 bytes from map-output for attempt_1427088391284_0001_m_000020_0
2015-03-23 05:40:07,849 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908803, inMemoryMapOutputs.size() -> 39, commitMemory -> 34555027, usedMemory ->38176411
2015-03-23 05:40:07,849 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-e3ca09ec:13562 freed by fetcher#1 in 112ms
2015-03-23 05:40:07,849 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5eca0951:13562 with 1 to fetcher#1
2015-03-23 05:40:07,849 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5eca0951:13562 to fetcher#1
2015-03-23 05:40:07,849 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000023_0 sent hash and received reply
2015-03-23 05:40:07,850 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000023_0 decomp: 906965 len: 906969 to MEMORY
2015-03-23 05:40:07,886 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000002_0 sent hash and received reply
2015-03-23 05:40:07,887 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000002_0 decomp: 907858 len: 907862 to MEMORY
2015-03-23 05:40:07,895 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906174 bytes from map-output for attempt_1427088391284_0001_m_000005_0
2015-03-23 05:40:07,895 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906174, inMemoryMapOutputs.size() -> 40, commitMemory -> 35463830, usedMemory ->39991234
2015-03-23 05:40:07,896 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8eca0981:13562 freed by fetcher#2 in 116ms
2015-03-23 05:40:07,900 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6fca0960:13562 with 1 to fetcher#2
2015-03-23 05:40:07,900 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6fca0960:13562 to fetcher#2
2015-03-23 05:40:07,902 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 902917 bytes from map-output for attempt_1427088391284_0001_m_000044_0
2015-03-23 05:40:07,902 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 902917, inMemoryMapOutputs.size() -> 41, commitMemory -> 36370004, usedMemory ->39991234
2015-03-23 05:40:07,903 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-97ca0998:13562 freed by fetcher#5 in 298ms
2015-03-23 05:40:07,903 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f4ca09fb:13562 with 1 to fetcher#5
2015-03-23 05:40:07,903 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f4ca09fb:13562 to fetcher#5
2015-03-23 05:40:07,905 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906965 bytes from map-output for attempt_1427088391284_0001_m_000023_0
2015-03-23 05:40:07,905 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906965, inMemoryMapOutputs.size() -> 42, commitMemory -> 37272921, usedMemory ->39991234
2015-03-23 05:40:07,905 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-92ca099d:13562 freed by fetcher#4 in 61ms
2015-03-23 05:40:07,905 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-09c90a06:13562 with 1 to fetcher#4
2015-03-23 05:40:07,905 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-09c90a06:13562 to fetcher#4
2015-03-23 05:40:07,935 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907858 bytes from map-output for attempt_1427088391284_0001_m_000002_0
2015-03-23 05:40:07,936 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907858, inMemoryMapOutputs.size() -> 43, commitMemory -> 38179886, usedMemory ->39991234
2015-03-23 05:40:07,936 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5eca0951:13562 freed by fetcher#1 in 87ms
2015-03-23 05:40:07,936 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6bca0964:13562 with 1 to fetcher#1
2015-03-23 05:40:07,936 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6bca0964:13562 to fetcher#1
2015-03-23 05:40:07,938 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000014_0 sent hash and received reply
2015-03-23 05:40:07,939 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000014_0 decomp: 909621 len: 909625 to MEMORY
2015-03-23 05:40:07,942 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000046_0 sent hash and received reply
2015-03-23 05:40:07,943 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000046_0 decomp: 905575 len: 905579 to MEMORY
2015-03-23 05:40:07,944 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000017_0 sent hash and received reply
2015-03-23 05:40:07,944 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000017_0 decomp: 910835 len: 910839 to MEMORY
2015-03-23 05:40:07,945 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 903490 bytes from map-output for attempt_1427088391284_0001_m_000043_0
2015-03-23 05:40:07,945 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 903490, inMemoryMapOutputs.size() -> 44, commitMemory -> 39087744, usedMemory ->42717265
2015-03-23 05:40:07,945 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f2ca09fd:13562 freed by fetcher#3 in 206ms
2015-03-23 05:40:07,983 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=11&map=attempt_1427088391284_0001_m_000047_0 sent hash and received reply
2015-03-23 05:40:07,984 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000047_0 decomp: 896793 len: 896797 to MEMORY
2015-03-23 05:40:07,992 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909621 bytes from map-output for attempt_1427088391284_0001_m_000014_0
2015-03-23 05:40:07,992 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909621, inMemoryMapOutputs.size() -> 45, commitMemory -> 39991234, usedMemory ->43614058
2015-03-23 05:40:07,992 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f4ca09fb:13562 freed by fetcher#5 in 89ms
2015-03-23 05:40:07,998 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905575 bytes from map-output for attempt_1427088391284_0001_m_000046_0
2015-03-23 05:40:07,998 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905575, inMemoryMapOutputs.size() -> 46, commitMemory -> 40900855, usedMemory ->43614058
2015-03-23 05:40:07,999 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6bca0964:13562 freed by fetcher#1 in 63ms
2015-03-23 05:40:08,029 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 896793 bytes from map-output for attempt_1427088391284_0001_m_000047_0
2015-03-23 05:40:08,029 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 896793, inMemoryMapOutputs.size() -> 47, commitMemory -> 41806430, usedMemory ->43614058
2015-03-23 05:40:08,029 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-09c90a06:13562 freed by fetcher#4 in 124ms
2015-03-23 05:40:08,098 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910835 bytes from map-output for attempt_1427088391284_0001_m_000017_0
2015-03-23 05:40:08,098 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910835, inMemoryMapOutputs.size() -> 48, commitMemory -> 42703223, usedMemory ->43614058
2015-03-23 05:40:08,099 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2015-03-23 05:40:08,100 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6fca0960:13562 freed by fetcher#2 in 199ms
2015-03-23 05:40:08,105 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 48 in-memory map-outputs and 0 on-disk map-outputs
2015-03-23 05:40:08,152 INFO [main] org.apache.hadoop.mapred.Merger: Merging 48 sorted segments
2015-03-23 05:40:08,153 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 48 segments left of total size: 43613578 bytes
2015-03-23 05:40:17,424 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 48 segments, 43614058 bytes to disk to satisfy reduce memory limit
2015-03-23 05:40:17,425 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 43613968 bytes from disk
2015-03-23 05:40:17,427 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2015-03-23 05:40:17,427 INFO [main] org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2015-03-23 05:40:17,455 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 43613954 bytes
2015-03-23 05:40:17,464 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-03-23 05:40:18,020 INFO [main] org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2015-03-23 05:40:18,022 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2015-03-23 05:40:32,230 INFO [main] HiBench.HiveData$GenerateRankingsReducer: pid: 11, 0 erros, 0 missed
2015-03-23 05:40:32,566 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1427088391284_0001_r_000011_0 is done. And is in the process of committing
2015-03-23 05:40:32,608 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1427088391284_0001_r_000011_0 is allowed to commit now
2015-03-23 05:40:32,668 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1427088391284_0001_r_000011_0' to hdfs://172.31.17.135:8120/HiBench/Hive/Input-comp/rankings/_temporary/1/task_1427088391284_0001_r_000011
2015-03-23 05:40:32,816 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1427088391284_0001_r_000011_0' done.
2015-03-23 05:40:32,819 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ReduceTask metrics system...
2015-03-23 05:40:32,820 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system stopped.
2015-03-23 05:40:32,820 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system shutdown complete.
