2015-03-23 05:39:58,603 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-03-23 05:39:59,224 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-03-23 05:39:59,224 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system started
2015-03-23 05:39:59,268 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2015-03-23 05:39:59,268 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1427088391284_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@482dbdc2)
2015-03-23 05:39:59,694 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2015-03-23 05:40:01,496 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /tmp/hadoop-ubuntu/nm-local-dir/usercache/ubuntu/appcache/application_1427088391284_0001
2015-03-23 05:40:05,720 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2015-03-23 05:40:07,677 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2015-03-23 05:40:07,794 INFO [main] org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5daef86b
2015-03-23 05:40:07,883 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=1503238528, maxSingleShuffleLimit=375809632, mergeThreshold=992137472, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-03-23 05:40:07,891 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000052_0 Thread started: EventFetcher for fetching Map Completion Events
2015-03-23 05:40:07,965 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-daca09d5:13562 with 1 to fetcher#5
2015-03-23 05:40:07,965 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-daca09d5:13562 to fetcher#5
2015-03-23 05:40:07,967 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9cca0993:13562 with 1 to fetcher#1
2015-03-23 05:40:07,967 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9cca0993:13562 to fetcher#1
2015-03-23 05:40:07,969 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6fca0960:13562 with 1 to fetcher#4
2015-03-23 05:40:07,969 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6fca0960:13562 to fetcher#4
2015-03-23 05:40:07,974 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8fca0980:13562 with 1 to fetcher#2
2015-03-23 05:40:07,974 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8fca0980:13562 to fetcher#2
2015-03-23 05:40:07,975 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-33ca093c:13562 with 1 to fetcher#3
2015-03-23 05:40:07,975 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-33ca093c:13562 to fetcher#3
2015-03-23 05:40:08,063 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000052_0: Got 48 new map-outputs
2015-03-23 05:40:08,608 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000038_0 sent hash and received reply
2015-03-23 05:40:08,615 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000017_0 sent hash and received reply
2015-03-23 05:40:08,611 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000021_0 sent hash and received reply
2015-03-23 05:40:08,611 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000011_0 sent hash and received reply
2015-03-23 05:40:08,622 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000011_0 decomp: 908645 len: 908649 to MEMORY
2015-03-23 05:40:08,623 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000017_0 decomp: 912559 len: 912563 to MEMORY
2015-03-23 05:40:08,624 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000021_0 decomp: 913332 len: 913336 to MEMORY
2015-03-23 05:40:08,624 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000038_0 decomp: 911253 len: 911257 to MEMORY
2015-03-23 05:40:08,682 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000013_0 sent hash and received reply
2015-03-23 05:40:08,684 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000013_0 decomp: 909477 len: 909481 to MEMORY
2015-03-23 05:40:08,692 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911253 bytes from map-output for attempt_1427088391284_0001_m_000038_0
2015-03-23 05:40:08,724 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908645 bytes from map-output for attempt_1427088391284_0001_m_000011_0
2015-03-23 05:40:08,696 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913332 bytes from map-output for attempt_1427088391284_0001_m_000021_0
2015-03-23 05:40:08,724 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911253, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->4555266
2015-03-23 05:40:08,729 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913332, inMemoryMapOutputs.size() -> 2, commitMemory -> 911253, usedMemory ->4555266
2015-03-23 05:40:08,729 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8fca0980:13562 freed by fetcher#2 in 755ms
2015-03-23 05:40:08,730 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ddca09d2:13562 with 1 to fetcher#2
2015-03-23 05:40:08,730 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ddca09d2:13562 to fetcher#2
2015-03-23 05:40:08,731 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908645, inMemoryMapOutputs.size() -> 3, commitMemory -> 1824585, usedMemory ->4555266
2015-03-23 05:40:08,731 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-33ca093c:13562 freed by fetcher#3 in 756ms
2015-03-23 05:40:08,732 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f2ca09fd:13562 with 1 to fetcher#3
2015-03-23 05:40:08,732 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f2ca09fd:13562 to fetcher#3
2015-03-23 05:40:08,733 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-daca09d5:13562 freed by fetcher#5 in 768ms
2015-03-23 05:40:08,734 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-08c90a07:13562 with 1 to fetcher#5
2015-03-23 05:40:08,734 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-08c90a07:13562 to fetcher#5
2015-03-23 05:40:08,736 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000035_0 sent hash and received reply
2015-03-23 05:40:08,737 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000035_0 decomp: 910333 len: 910337 to MEMORY
2015-03-23 05:40:08,738 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000043_0 sent hash and received reply
2015-03-23 05:40:08,781 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000043_0 decomp: 903955 len: 903959 to MEMORY
2015-03-23 05:40:08,818 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000045_0 sent hash and received reply
2015-03-23 05:40:08,818 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000045_0 decomp: 905514 len: 905518 to MEMORY
2015-03-23 05:40:08,823 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909477 bytes from map-output for attempt_1427088391284_0001_m_000013_0
2015-03-23 05:40:08,824 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909477, inMemoryMapOutputs.size() -> 4, commitMemory -> 2733230, usedMemory ->7275068
2015-03-23 05:40:08,824 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9cca0993:13562 freed by fetcher#1 in 857ms
2015-03-23 05:40:08,824 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6cca0963:13562 with 1 to fetcher#1
2015-03-23 05:40:08,824 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6cca0963:13562 to fetcher#1
2015-03-23 05:40:08,827 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910333 bytes from map-output for attempt_1427088391284_0001_m_000035_0
2015-03-23 05:40:08,827 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910333, inMemoryMapOutputs.size() -> 5, commitMemory -> 3642707, usedMemory ->7275068
2015-03-23 05:40:08,827 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ddca09d2:13562 freed by fetcher#2 in 97ms
2015-03-23 05:40:08,827 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-00c90a0f:13562 with 1 to fetcher#2
2015-03-23 05:40:08,827 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-00c90a0f:13562 to fetcher#2
2015-03-23 05:40:08,831 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000015_0 sent hash and received reply
2015-03-23 05:40:08,831 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000015_0 decomp: 912394 len: 912398 to MEMORY
2015-03-23 05:40:08,852 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000032_0 sent hash and received reply
2015-03-23 05:40:08,853 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000032_0 decomp: 913677 len: 913681 to MEMORY
2015-03-23 05:40:08,858 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905514 bytes from map-output for attempt_1427088391284_0001_m_000045_0
2015-03-23 05:40:08,858 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905514, inMemoryMapOutputs.size() -> 6, commitMemory -> 4553040, usedMemory ->9101139
2015-03-23 05:40:08,859 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-08c90a07:13562 freed by fetcher#5 in 125ms
2015-03-23 05:40:08,859 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bdca09b2:13562 with 1 to fetcher#5
2015-03-23 05:40:08,859 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bdca09b2:13562 to fetcher#5
2015-03-23 05:40:08,873 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000000_0 sent hash and received reply
2015-03-23 05:40:08,873 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000000_0 decomp: 906252 len: 906256 to MEMORY
2015-03-23 05:40:08,902 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912394 bytes from map-output for attempt_1427088391284_0001_m_000015_0
2015-03-23 05:40:08,902 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912394, inMemoryMapOutputs.size() -> 7, commitMemory -> 5458554, usedMemory ->10007391
2015-03-23 05:40:08,902 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6cca0963:13562 freed by fetcher#1 in 78ms
2015-03-23 05:40:08,902 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-0bc90a04:13562 with 1 to fetcher#1
2015-03-23 05:40:08,902 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-0bc90a04:13562 to fetcher#1
2015-03-23 05:40:08,906 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913677 bytes from map-output for attempt_1427088391284_0001_m_000032_0
2015-03-23 05:40:08,906 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913677, inMemoryMapOutputs.size() -> 8, commitMemory -> 6370948, usedMemory ->10007391
2015-03-23 05:40:08,906 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-00c90a0f:13562 freed by fetcher#2 in 79ms
2015-03-23 05:40:08,906 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8aca0985:13562 with 1 to fetcher#2
2015-03-23 05:40:08,906 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8aca0985:13562 to fetcher#2
2015-03-23 05:40:08,910 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 903955 bytes from map-output for attempt_1427088391284_0001_m_000043_0
2015-03-23 05:40:08,910 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 903955, inMemoryMapOutputs.size() -> 9, commitMemory -> 7284625, usedMemory ->10007391
2015-03-23 05:40:08,910 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f2ca09fd:13562 freed by fetcher#3 in 179ms
2015-03-23 05:40:08,910 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-03c90a0c:13562 with 1 to fetcher#3
2015-03-23 05:40:08,910 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-03c90a0c:13562 to fetcher#3
2015-03-23 05:40:08,913 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912559 bytes from map-output for attempt_1427088391284_0001_m_000017_0
2015-03-23 05:40:08,913 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912559, inMemoryMapOutputs.size() -> 10, commitMemory -> 8188580, usedMemory ->10007391
2015-03-23 05:40:08,914 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6fca0960:13562 freed by fetcher#4 in 945ms
2015-03-23 05:40:08,914 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5dca0952:13562 with 1 to fetcher#4
2015-03-23 05:40:08,914 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5dca0952:13562 to fetcher#4
2015-03-23 05:40:08,925 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000008_0 sent hash and received reply
2015-03-23 05:40:08,926 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000008_0 decomp: 905980 len: 905984 to MEMORY
2015-03-23 05:40:08,934 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000039_0 sent hash and received reply
2015-03-23 05:40:08,934 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000039_0 decomp: 898286 len: 898290 to MEMORY
2015-03-23 05:40:08,963 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000022_0 sent hash and received reply
2015-03-23 05:40:09,119 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000019_0 sent hash and received reply
2015-03-23 05:40:09,121 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906252 bytes from map-output for attempt_1427088391284_0001_m_000000_0
2015-03-23 05:40:09,122 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000022_0 decomp: 908432 len: 908436 to MEMORY
2015-03-23 05:40:09,125 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906252, inMemoryMapOutputs.size() -> 11, commitMemory -> 9101139, usedMemory ->12720089
2015-03-23 05:40:09,125 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bdca09b2:13562 freed by fetcher#5 in 266ms
2015-03-23 05:40:09,126 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-97ca0998:13562 with 1 to fetcher#5
2015-03-23 05:40:09,126 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-97ca0998:13562 to fetcher#5
2015-03-23 05:40:09,127 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000019_0 decomp: 911195 len: 911199 to MEMORY
2015-03-23 05:40:09,156 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000044_0 sent hash and received reply
2015-03-23 05:40:09,157 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000044_0 decomp: 899972 len: 899976 to MEMORY
2015-03-23 05:40:09,165 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905980 bytes from map-output for attempt_1427088391284_0001_m_000008_0
2015-03-23 05:40:09,165 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905980, inMemoryMapOutputs.size() -> 12, commitMemory -> 10007391, usedMemory ->14531256
2015-03-23 05:40:09,165 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5dca0952:13562 freed by fetcher#4 in 251ms
2015-03-23 05:40:09,165 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b3ca09bc:13562 with 1 to fetcher#4
2015-03-23 05:40:09,165 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b3ca09bc:13562 to fetcher#4
2015-03-23 05:40:09,171 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000040_0 sent hash and received reply
2015-03-23 05:40:09,173 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000040_0 decomp: 901940 len: 901944 to MEMORY
2015-03-23 05:40:09,185 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 899972 bytes from map-output for attempt_1427088391284_0001_m_000044_0
2015-03-23 05:40:09,185 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 899972, inMemoryMapOutputs.size() -> 13, commitMemory -> 10913371, usedMemory ->15433196
2015-03-23 05:40:09,186 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-97ca0998:13562 freed by fetcher#5 in 60ms
2015-03-23 05:40:09,186 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-e3ca09ec:13562 with 1 to fetcher#5
2015-03-23 05:40:09,186 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-e3ca09ec:13562 to fetcher#5
2015-03-23 05:40:09,189 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 898286 bytes from map-output for attempt_1427088391284_0001_m_000039_0
2015-03-23 05:40:09,189 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 898286, inMemoryMapOutputs.size() -> 14, commitMemory -> 11813343, usedMemory ->15433196
2015-03-23 05:40:09,189 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8aca0985:13562 freed by fetcher#2 in 283ms
2015-03-23 05:40:09,214 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5fca0950:13562 with 1 to fetcher#2
2015-03-23 05:40:09,214 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5fca0950:13562 to fetcher#2
2015-03-23 05:40:09,220 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 901940 bytes from map-output for attempt_1427088391284_0001_m_000040_0
2015-03-23 05:40:09,220 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 901940, inMemoryMapOutputs.size() -> 15, commitMemory -> 12711629, usedMemory ->15433196
2015-03-23 05:40:09,220 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b3ca09bc:13562 freed by fetcher#4 in 55ms
2015-03-23 05:40:09,220 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bcca09b3:13562 with 1 to fetcher#4
2015-03-23 05:40:09,220 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bcca09b3:13562 to fetcher#4
2015-03-23 05:40:09,224 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000020_0 sent hash and received reply
2015-03-23 05:40:09,224 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000020_0 decomp: 914533 len: 914537 to MEMORY
2015-03-23 05:40:09,226 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000042_0 sent hash and received reply
2015-03-23 05:40:09,227 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000042_0 decomp: 900116 len: 900120 to MEMORY
2015-03-23 05:40:09,267 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 900116 bytes from map-output for attempt_1427088391284_0001_m_000042_0
2015-03-23 05:40:09,267 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 900116, inMemoryMapOutputs.size() -> 16, commitMemory -> 13613569, usedMemory ->17247845
2015-03-23 05:40:09,268 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bcca09b3:13562 freed by fetcher#4 in 48ms
2015-03-23 05:40:09,268 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-92ca099d:13562 with 1 to fetcher#4
2015-03-23 05:40:09,270 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-92ca099d:13562 to fetcher#4
2015-03-23 05:40:09,276 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000023_0 sent hash and received reply
2015-03-23 05:40:09,276 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000023_0 decomp: 911263 len: 911267 to MEMORY
2015-03-23 05:40:09,288 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000036_0 sent hash and received reply
2015-03-23 05:40:09,289 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000036_0 decomp: 906510 len: 906514 to MEMORY
2015-03-23 05:40:09,289 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911195 bytes from map-output for attempt_1427088391284_0001_m_000019_0
2015-03-23 05:40:09,289 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911195, inMemoryMapOutputs.size() -> 17, commitMemory -> 14513685, usedMemory ->19065618
2015-03-23 05:40:09,289 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-0bc90a04:13562 freed by fetcher#1 in 387ms
2015-03-23 05:40:09,289 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-beca09b1:13562 with 1 to fetcher#1
2015-03-23 05:40:09,290 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-beca09b1:13562 to fetcher#1
2015-03-23 05:40:09,293 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914533 bytes from map-output for attempt_1427088391284_0001_m_000020_0
2015-03-23 05:40:09,293 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914533, inMemoryMapOutputs.size() -> 18, commitMemory -> 15424880, usedMemory ->19065618
2015-03-23 05:40:09,294 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-e3ca09ec:13562 freed by fetcher#5 in 108ms
2015-03-23 05:40:09,294 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-d7ca09d8:13562 with 1 to fetcher#5
2015-03-23 05:40:09,294 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-d7ca09d8:13562 to fetcher#5
2015-03-23 05:40:09,322 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000010_0 sent hash and received reply
2015-03-23 05:40:09,323 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000010_0 decomp: 911651 len: 911655 to MEMORY
2015-03-23 05:40:09,327 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000027_0 sent hash and received reply
2015-03-23 05:40:09,329 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000027_0 decomp: 912480 len: 912484 to MEMORY
2015-03-23 05:40:09,330 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911263 bytes from map-output for attempt_1427088391284_0001_m_000023_0
2015-03-23 05:40:09,330 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911263, inMemoryMapOutputs.size() -> 19, commitMemory -> 16339413, usedMemory ->20889749
2015-03-23 05:40:09,339 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-92ca099d:13562 freed by fetcher#4 in 69ms
2015-03-23 05:40:09,339 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-09c90a06:13562 with 1 to fetcher#4
2015-03-23 05:40:09,339 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-09c90a06:13562 to fetcher#4
2015-03-23 05:40:09,371 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000047_0 sent hash and received reply
2015-03-23 05:40:09,372 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000047_0 decomp: 896596 len: 896600 to MEMORY
2015-03-23 05:40:09,376 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911651 bytes from map-output for attempt_1427088391284_0001_m_000010_0
2015-03-23 05:40:09,376 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911651, inMemoryMapOutputs.size() -> 20, commitMemory -> 17250676, usedMemory ->21786345
2015-03-23 05:40:09,376 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-beca09b1:13562 freed by fetcher#1 in 87ms
2015-03-23 05:40:09,376 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-51ca095e:13562 with 1 to fetcher#1
2015-03-23 05:40:09,376 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-51ca095e:13562 to fetcher#1
2015-03-23 05:40:09,381 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912480 bytes from map-output for attempt_1427088391284_0001_m_000027_0
2015-03-23 05:40:09,381 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912480, inMemoryMapOutputs.size() -> 21, commitMemory -> 18162327, usedMemory ->21786345
2015-03-23 05:40:09,381 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-d7ca09d8:13562 freed by fetcher#5 in 87ms
2015-03-23 05:40:09,381 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c2ca09cd:13562 with 1 to fetcher#5
2015-03-23 05:40:09,383 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906510 bytes from map-output for attempt_1427088391284_0001_m_000036_0
2015-03-23 05:40:09,383 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906510, inMemoryMapOutputs.size() -> 22, commitMemory -> 19074807, usedMemory ->21786345
2015-03-23 05:40:09,383 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5fca0950:13562 freed by fetcher#2 in 169ms
2015-03-23 05:40:09,383 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c5ca09ca:13562 with 1 to fetcher#2
2015-03-23 05:40:09,383 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c5ca09ca:13562 to fetcher#2
2015-03-23 05:40:09,384 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000006_0 sent hash and received reply
2015-03-23 05:40:09,385 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c2ca09cd:13562 to fetcher#5
2015-03-23 05:40:09,389 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000025_0 sent hash and received reply
2015-03-23 05:40:09,391 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000025_0 decomp: 909758 len: 909762 to MEMORY
2015-03-23 05:40:09,441 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000034_0 sent hash and received reply
2015-03-23 05:40:09,467 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000006_0 decomp: 908244 len: 908248 to MEMORY
2015-03-23 05:40:09,470 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000034_0 decomp: 910652 len: 910656 to MEMORY
2015-03-23 05:40:09,481 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909758 bytes from map-output for attempt_1427088391284_0001_m_000025_0
2015-03-23 05:40:09,481 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909758, inMemoryMapOutputs.size() -> 23, commitMemory -> 19981317, usedMemory ->24514999
2015-03-23 05:40:09,481 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c2ca09cd:13562 freed by fetcher#5 in 98ms
2015-03-23 05:40:09,482 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5eca0951:13562 with 1 to fetcher#5
2015-03-23 05:40:09,482 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5eca0951:13562 to fetcher#5
2015-03-23 05:40:09,486 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000002_0 sent hash and received reply
2015-03-23 05:40:09,490 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 896596 bytes from map-output for attempt_1427088391284_0001_m_000047_0
2015-03-23 05:40:09,491 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000002_0 decomp: 909808 len: 909812 to MEMORY
2015-03-23 05:40:09,491 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 896596, inMemoryMapOutputs.size() -> 24, commitMemory -> 20891075, usedMemory ->25424807
2015-03-23 05:40:09,492 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-09c90a06:13562 freed by fetcher#4 in 152ms
2015-03-23 05:40:09,492 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-64ca096b:13562 with 1 to fetcher#4
2015-03-23 05:40:09,492 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-64ca096b:13562 to fetcher#4
2015-03-23 05:40:09,494 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908432 bytes from map-output for attempt_1427088391284_0001_m_000022_0
2015-03-23 05:40:09,494 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908432, inMemoryMapOutputs.size() -> 25, commitMemory -> 21787671, usedMemory ->25424807
2015-03-23 05:40:09,494 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-03c90a0c:13562 freed by fetcher#3 in 584ms
2015-03-23 05:40:09,494 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9aca0995:13562 with 1 to fetcher#3
2015-03-23 05:40:09,495 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9aca0995:13562 to fetcher#3
2015-03-23 05:40:09,527 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000001_0 sent hash and received reply
2015-03-23 05:40:09,527 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000001_0 decomp: 908018 len: 908022 to MEMORY
2015-03-23 05:40:09,538 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909808 bytes from map-output for attempt_1427088391284_0001_m_000002_0
2015-03-23 05:40:09,538 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909808, inMemoryMapOutputs.size() -> 26, commitMemory -> 22696103, usedMemory ->26332825
2015-03-23 05:40:09,538 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5eca0951:13562 freed by fetcher#5 in 56ms
2015-03-23 05:40:09,538 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-62ca096d:13562 with 1 to fetcher#5
2015-03-23 05:40:09,539 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-62ca096d:13562 to fetcher#5
2015-03-23 05:40:09,544 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000007_0 sent hash and received reply
2015-03-23 05:40:09,546 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000007_0 decomp: 912025 len: 912029 to MEMORY
2015-03-23 05:40:09,548 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908018 bytes from map-output for attempt_1427088391284_0001_m_000001_0
2015-03-23 05:40:09,548 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908018, inMemoryMapOutputs.size() -> 27, commitMemory -> 23605911, usedMemory ->27244850
2015-03-23 05:40:09,548 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-64ca096b:13562 freed by fetcher#4 in 56ms
2015-03-23 05:40:09,548 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-95ca099a:13562 with 1 to fetcher#4
2015-03-23 05:40:09,548 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-95ca099a:13562 to fetcher#4
2015-03-23 05:40:09,586 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000028_0 sent hash and received reply
2015-03-23 05:40:09,587 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000028_0 decomp: 915232 len: 915236 to MEMORY
2015-03-23 05:40:09,592 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000037_0 sent hash and received reply
2015-03-23 05:40:09,592 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000037_0 decomp: 909383 len: 909387 to MEMORY
2015-03-23 05:40:09,594 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912025 bytes from map-output for attempt_1427088391284_0001_m_000007_0
2015-03-23 05:40:09,594 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912025, inMemoryMapOutputs.size() -> 28, commitMemory -> 24513929, usedMemory ->29069465
2015-03-23 05:40:09,594 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-62ca096d:13562 freed by fetcher#5 in 56ms
2015-03-23 05:40:09,594 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f4ca09fb:13562 with 1 to fetcher#5
2015-03-23 05:40:09,595 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f4ca09fb:13562 to fetcher#5
2015-03-23 05:40:09,600 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000014_0 sent hash and received reply
2015-03-23 05:40:09,601 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000014_0 decomp: 912777 len: 912781 to MEMORY
2015-03-23 05:40:09,635 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908244 bytes from map-output for attempt_1427088391284_0001_m_000006_0
2015-03-23 05:40:09,635 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908244, inMemoryMapOutputs.size() -> 29, commitMemory -> 25425954, usedMemory ->29982242
2015-03-23 05:40:09,636 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-51ca095e:13562 freed by fetcher#1 in 260ms
2015-03-23 05:40:09,640 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-02c90a0d:13562 with 1 to fetcher#1
2015-03-23 05:40:09,640 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-02c90a0d:13562 to fetcher#1
2015-03-23 05:40:09,644 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910652 bytes from map-output for attempt_1427088391284_0001_m_000034_0
2015-03-23 05:40:09,644 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910652, inMemoryMapOutputs.size() -> 30, commitMemory -> 26334198, usedMemory ->29982242
2015-03-23 05:40:09,644 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c5ca09ca:13562 freed by fetcher#2 in 261ms
2015-03-23 05:40:09,644 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-81ca098e:13562 with 1 to fetcher#2
2015-03-23 05:40:09,644 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-81ca098e:13562 to fetcher#2
2015-03-23 05:40:09,645 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000004_0 sent hash and received reply
2015-03-23 05:40:09,646 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000004_0 decomp: 906134 len: 906138 to MEMORY
2015-03-23 05:40:09,653 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000030_0 sent hash and received reply
2015-03-23 05:40:09,654 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000030_0 decomp: 912050 len: 912054 to MEMORY
2015-03-23 05:40:09,657 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912777 bytes from map-output for attempt_1427088391284_0001_m_000014_0
2015-03-23 05:40:09,657 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912777, inMemoryMapOutputs.size() -> 31, commitMemory -> 27244850, usedMemory ->31800426
2015-03-23 05:40:09,657 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f4ca09fb:13562 freed by fetcher#5 in 63ms
2015-03-23 05:40:09,657 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-04c90a0b:13562 with 1 to fetcher#5
2015-03-23 05:40:09,657 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-04c90a0b:13562 to fetcher#5
2015-03-23 05:40:09,699 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912050 bytes from map-output for attempt_1427088391284_0001_m_000030_0
2015-03-23 05:40:09,699 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912050, inMemoryMapOutputs.size() -> 32, commitMemory -> 28157627, usedMemory ->31800426
2015-03-23 05:40:09,700 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-81ca098e:13562 freed by fetcher#2 in 56ms
2015-03-23 05:40:09,700 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b5ca09ba:13562 with 1 to fetcher#2
2015-03-23 05:40:09,700 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b5ca09ba:13562 to fetcher#2
2015-03-23 05:40:09,700 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906134 bytes from map-output for attempt_1427088391284_0001_m_000004_0
2015-03-23 05:40:09,700 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906134, inMemoryMapOutputs.size() -> 33, commitMemory -> 29069677, usedMemory ->31800426
2015-03-23 05:40:09,701 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-02c90a0d:13562 freed by fetcher#1 in 60ms
2015-03-23 05:40:09,701 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-dfca09d0:13562 with 1 to fetcher#1
2015-03-23 05:40:09,701 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-dfca09d0:13562 to fetcher#1
2015-03-23 05:40:09,733 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000031_0 sent hash and received reply
2015-03-23 05:40:09,734 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000031_0 decomp: 908221 len: 908225 to MEMORY
2015-03-23 05:40:09,746 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000026_0 sent hash and received reply
2015-03-23 05:40:09,746 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000026_0 decomp: 910485 len: 910489 to MEMORY
2015-03-23 05:40:09,751 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908221 bytes from map-output for attempt_1427088391284_0001_m_000031_0
2015-03-23 05:40:09,751 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908221, inMemoryMapOutputs.size() -> 34, commitMemory -> 29975811, usedMemory ->33619132
2015-03-23 05:40:09,751 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-dfca09d0:13562 freed by fetcher#1 in 50ms
2015-03-23 05:40:09,751 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-82ca098d:13562 with 1 to fetcher#1
2015-03-23 05:40:09,752 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000009_0 sent hash and received reply
2015-03-23 05:40:09,940 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-82ca098d:13562 to fetcher#1
2015-03-23 05:40:09,943 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000009_0 decomp: 909797 len: 909801 to MEMORY
2015-03-23 05:40:09,946 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000033_0 sent hash and received reply
2015-03-23 05:40:09,947 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000033_0 decomp: 909289 len: 909293 to MEMORY
2015-03-23 05:40:09,992 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910485 bytes from map-output for attempt_1427088391284_0001_m_000026_0
2015-03-23 05:40:09,993 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910485, inMemoryMapOutputs.size() -> 35, commitMemory -> 30884032, usedMemory ->35438218
2015-03-23 05:40:09,993 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b5ca09ba:13562 freed by fetcher#2 in 293ms
2015-03-23 05:40:09,993 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-53ca095c:13562 with 1 to fetcher#2
2015-03-23 05:40:09,993 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-53ca095c:13562 to fetcher#2
2015-03-23 05:40:10,003 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909383 bytes from map-output for attempt_1427088391284_0001_m_000037_0
2015-03-23 05:40:10,003 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909383, inMemoryMapOutputs.size() -> 36, commitMemory -> 31794517, usedMemory ->35438218
2015-03-23 05:40:10,005 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-95ca099a:13562 freed by fetcher#4 in 457ms
2015-03-23 05:40:10,005 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-a8ca09a7:13562 with 1 to fetcher#4
2015-03-23 05:40:10,006 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-a8ca09a7:13562 to fetcher#4
2015-03-23 05:40:10,007 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000012_0 sent hash and received reply
2015-03-23 05:40:10,011 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 915232 bytes from map-output for attempt_1427088391284_0001_m_000028_0
2015-03-23 05:40:10,011 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 915232, inMemoryMapOutputs.size() -> 37, commitMemory -> 32703900, usedMemory ->35438218
2015-03-23 05:40:10,011 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9aca0995:13562 freed by fetcher#3 in 517ms
2015-03-23 05:40:10,012 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b8ca09b7:13562 with 1 to fetcher#3
2015-03-23 05:40:10,012 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b8ca09b7:13562 to fetcher#3
2015-03-23 05:40:10,013 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000012_0 decomp: 912220 len: 912224 to MEMORY
2015-03-23 05:40:10,014 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909797 bytes from map-output for attempt_1427088391284_0001_m_000009_0
2015-03-23 05:40:10,014 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909797, inMemoryMapOutputs.size() -> 38, commitMemory -> 33619132, usedMemory ->36350438
2015-03-23 05:40:10,014 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-04c90a0b:13562 freed by fetcher#5 in 357ms
2015-03-23 05:40:10,014 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ccca09c3:13562 with 1 to fetcher#5
2015-03-23 05:40:10,014 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ccca09c3:13562 to fetcher#5
2015-03-23 05:40:10,015 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000003_0 sent hash and received reply
2015-03-23 05:40:10,015 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909289 bytes from map-output for attempt_1427088391284_0001_m_000033_0
2015-03-23 05:40:10,015 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909289, inMemoryMapOutputs.size() -> 39, commitMemory -> 34528929, usedMemory ->36350438
2015-03-23 05:40:10,016 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-82ca098d:13562 freed by fetcher#1 in 76ms
2015-03-23 05:40:10,016 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8eca0981:13562 with 1 to fetcher#1
2015-03-23 05:40:10,017 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000003_0 decomp: 907065 len: 907069 to MEMORY
2015-03-23 05:40:10,044 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8eca0981:13562 to fetcher#1
2015-03-23 05:40:10,053 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000005_0 sent hash and received reply
2015-03-23 05:40:10,054 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000005_0 decomp: 909560 len: 909564 to MEMORY
2015-03-23 05:40:10,059 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912220 bytes from map-output for attempt_1427088391284_0001_m_000012_0
2015-03-23 05:40:10,059 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912220, inMemoryMapOutputs.size() -> 40, commitMemory -> 35438218, usedMemory ->38167063
2015-03-23 05:40:10,060 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-53ca095c:13562 freed by fetcher#2 in 66ms
2015-03-23 05:40:10,060 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-acca09a3:13562 with 1 to fetcher#2
2015-03-23 05:40:10,060 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-acca09a3:13562 to fetcher#2
2015-03-23 05:40:10,093 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000041_0 sent hash and received reply
2015-03-23 05:40:10,093 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000041_0 decomp: 899814 len: 899818 to MEMORY
2015-03-23 05:40:10,094 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000016_0 sent hash and received reply
2015-03-23 05:40:10,094 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000016_0 decomp: 909722 len: 909726 to MEMORY
2015-03-23 05:40:10,102 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909560 bytes from map-output for attempt_1427088391284_0001_m_000005_0
2015-03-23 05:40:10,103 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909560, inMemoryMapOutputs.size() -> 41, commitMemory -> 36350438, usedMemory ->39976599
2015-03-23 05:40:10,103 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8eca0981:13562 freed by fetcher#1 in 59ms
2015-03-23 05:40:10,103 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6bca0964:13562 with 1 to fetcher#1
2015-03-23 05:40:10,103 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6bca0964:13562 to fetcher#1
2015-03-23 05:40:10,107 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907065 bytes from map-output for attempt_1427088391284_0001_m_000003_0
2015-03-23 05:40:10,107 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907065, inMemoryMapOutputs.size() -> 42, commitMemory -> 37259998, usedMemory ->39976599
2015-03-23 05:40:10,107 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-a8ca09a7:13562 freed by fetcher#4 in 102ms
2015-03-23 05:40:10,107 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-7eca0971:13562 with 1 to fetcher#4
2015-03-23 05:40:10,108 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-7eca0971:13562 to fetcher#4
2015-03-23 05:40:10,110 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000046_0 sent hash and received reply
2015-03-23 05:40:10,111 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000046_0 decomp: 902749 len: 902753 to MEMORY
2015-03-23 05:40:10,147 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 899814 bytes from map-output for attempt_1427088391284_0001_m_000041_0
2015-03-23 05:40:10,147 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 899814, inMemoryMapOutputs.size() -> 43, commitMemory -> 38167063, usedMemory ->40879348
2015-03-23 05:40:10,147 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b8ca09b7:13562 freed by fetcher#3 in 135ms
2015-03-23 05:40:10,148 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-cfca09c0:13562 with 1 to fetcher#3
2015-03-23 05:40:10,148 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-cfca09c0:13562 to fetcher#3
2015-03-23 05:40:10,156 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000018_0 sent hash and received reply
2015-03-23 05:40:10,166 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 902749 bytes from map-output for attempt_1427088391284_0001_m_000046_0
2015-03-23 05:40:10,166 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 902749, inMemoryMapOutputs.size() -> 44, commitMemory -> 39066877, usedMemory ->40879348
2015-03-23 05:40:10,167 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6bca0964:13562 freed by fetcher#1 in 64ms
2015-03-23 05:40:10,172 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000024_0 sent hash and received reply
2015-03-23 05:40:10,173 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000024_0 decomp: 910789 len: 910793 to MEMORY
2015-03-23 05:40:10,179 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000018_0 decomp: 909246 len: 909250 to MEMORY
2015-03-23 05:40:10,180 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909722 bytes from map-output for attempt_1427088391284_0001_m_000016_0
2015-03-23 05:40:10,180 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909722, inMemoryMapOutputs.size() -> 45, commitMemory -> 39969626, usedMemory ->42699383
2015-03-23 05:40:10,181 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-acca09a3:13562 freed by fetcher#2 in 121ms
2015-03-23 05:40:10,194 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=52&map=attempt_1427088391284_0001_m_000029_0 sent hash and received reply
2015-03-23 05:40:10,195 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000029_0 decomp: 915051 len: 915055 to MEMORY
2015-03-23 05:40:10,215 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 915051 bytes from map-output for attempt_1427088391284_0001_m_000029_0
2015-03-23 05:40:10,215 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 915051, inMemoryMapOutputs.size() -> 46, commitMemory -> 40879348, usedMemory ->43614434
2015-03-23 05:40:10,216 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-cfca09c0:13562 freed by fetcher#3 in 68ms
2015-03-23 05:40:10,266 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910789 bytes from map-output for attempt_1427088391284_0001_m_000024_0
2015-03-23 05:40:10,266 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910789, inMemoryMapOutputs.size() -> 47, commitMemory -> 41794399, usedMemory ->43614434
2015-03-23 05:40:10,267 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-7eca0971:13562 freed by fetcher#4 in 160ms
2015-03-23 05:40:10,340 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909246 bytes from map-output for attempt_1427088391284_0001_m_000018_0
2015-03-23 05:40:10,340 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909246, inMemoryMapOutputs.size() -> 48, commitMemory -> 42705188, usedMemory ->43614434
2015-03-23 05:40:10,340 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2015-03-23 05:40:10,341 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ccca09c3:13562 freed by fetcher#5 in 327ms
2015-03-23 05:40:10,349 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 48 in-memory map-outputs and 0 on-disk map-outputs
2015-03-23 05:40:10,417 INFO [main] org.apache.hadoop.mapred.Merger: Merging 48 sorted segments
2015-03-23 05:40:10,418 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 48 segments left of total size: 43613954 bytes
2015-03-23 05:40:19,463 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 48 segments, 43614434 bytes to disk to satisfy reduce memory limit
2015-03-23 05:40:19,464 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 43614344 bytes from disk
2015-03-23 05:40:19,465 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2015-03-23 05:40:19,465 INFO [main] org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2015-03-23 05:40:19,470 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 43614330 bytes
2015-03-23 05:40:19,503 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-03-23 05:40:20,165 INFO [main] org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2015-03-23 05:40:20,167 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2015-03-23 05:40:34,291 INFO [main] HiBench.HiveData$GenerateRankingsReducer: pid: 52, 0 erros, 0 missed
2015-03-23 05:40:34,553 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1427088391284_0001_r_000052_0 is done. And is in the process of committing
2015-03-23 05:40:34,729 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1427088391284_0001_r_000052_0 is allowed to commit now
2015-03-23 05:40:34,751 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1427088391284_0001_r_000052_0' to hdfs://172.31.17.135:8120/HiBench/Hive/Input-comp/rankings/_temporary/1/task_1427088391284_0001_r_000052
2015-03-23 05:40:34,824 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1427088391284_0001_r_000052_0' done.
2015-03-23 05:40:34,825 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ReduceTask metrics system...
2015-03-23 05:40:34,827 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system stopped.
2015-03-23 05:40:34,827 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system shutdown complete.
