2015-03-23 05:39:58,254 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-03-23 05:39:58,807 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-03-23 05:39:58,808 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system started
2015-03-23 05:39:58,828 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2015-03-23 05:39:58,828 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1427088391284_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@7abbee99)
2015-03-23 05:39:59,483 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2015-03-23 05:40:00,853 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /tmp/hadoop-ubuntu/nm-local-dir/usercache/ubuntu/appcache/application_1427088391284_0001
2015-03-23 05:40:07,411 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2015-03-23 05:40:09,574 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2015-03-23 05:40:09,742 INFO [main] org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5acb5e96
2015-03-23 05:40:09,813 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=1503238528, maxSingleShuffleLimit=375809632, mergeThreshold=992137472, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-03-23 05:40:09,896 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000009_0 Thread started: EventFetcher for fetching Map Completion Events
2015-03-23 05:40:09,946 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b8ca09b7:13562 with 1 to fetcher#5
2015-03-23 05:40:09,946 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-beca09b1:13562 with 1 to fetcher#4
2015-03-23 05:40:09,947 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-beca09b1:13562 to fetcher#4
2015-03-23 05:40:09,948 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5eca0951:13562 with 1 to fetcher#3
2015-03-23 05:40:09,948 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5eca0951:13562 to fetcher#3
2015-03-23 05:40:09,948 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6cca0963:13562 with 1 to fetcher#2
2015-03-23 05:40:09,948 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6cca0963:13562 to fetcher#2
2015-03-23 05:40:09,948 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8aca0985:13562 with 1 to fetcher#1
2015-03-23 05:40:09,949 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8aca0985:13562 to fetcher#1
2015-03-23 05:40:09,949 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b8ca09b7:13562 to fetcher#5
2015-03-23 05:40:10,004 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000009_0: Got 48 new map-outputs
2015-03-23 05:40:10,573 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000015_0 sent hash and received reply
2015-03-23 05:40:10,580 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000039_0 sent hash and received reply
2015-03-23 05:40:10,580 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000010_0 sent hash and received reply
2015-03-23 05:40:10,576 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000002_0 sent hash and received reply
2015-03-23 05:40:10,587 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000002_0 decomp: 905184 len: 905188 to MEMORY
2015-03-23 05:40:10,589 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000039_0 decomp: 895669 len: 895673 to MEMORY
2015-03-23 05:40:10,589 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000010_0 decomp: 911737 len: 911741 to MEMORY
2015-03-23 05:40:10,589 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000015_0 decomp: 913859 len: 913863 to MEMORY
2015-03-23 05:40:10,626 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000041_0 sent hash and received reply
2015-03-23 05:40:10,635 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000041_0 decomp: 899893 len: 899897 to MEMORY
2015-03-23 05:40:10,638 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911737 bytes from map-output for attempt_1427088391284_0001_m_000010_0
2015-03-23 05:40:10,648 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 895669 bytes from map-output for attempt_1427088391284_0001_m_000039_0
2015-03-23 05:40:10,678 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 895669, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->4526342
2015-03-23 05:40:10,681 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8aca0985:13562 freed by fetcher#1 in 733ms
2015-03-23 05:40:10,681 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ddca09d2:13562 with 1 to fetcher#1
2015-03-23 05:40:10,681 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ddca09d2:13562 to fetcher#1
2015-03-23 05:40:10,683 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911737, inMemoryMapOutputs.size() -> 2, commitMemory -> 895669, usedMemory ->4526342
2015-03-23 05:40:10,683 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-beca09b1:13562 freed by fetcher#4 in 736ms
2015-03-23 05:40:10,685 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-cfca09c0:13562 with 1 to fetcher#4
2015-03-23 05:40:10,685 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-cfca09c0:13562 to fetcher#4
2015-03-23 05:40:10,687 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905184 bytes from map-output for attempt_1427088391284_0001_m_000002_0
2015-03-23 05:40:10,687 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905184, inMemoryMapOutputs.size() -> 3, commitMemory -> 1807406, usedMemory ->4526342
2015-03-23 05:40:10,687 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5eca0951:13562 freed by fetcher#3 in 739ms
2015-03-23 05:40:10,687 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-0bc90a04:13562 with 1 to fetcher#3
2015-03-23 05:40:10,687 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-0bc90a04:13562 to fetcher#3
2015-03-23 05:40:10,689 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000035_0 sent hash and received reply
2015-03-23 05:40:10,693 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000019_0 sent hash and received reply
2015-03-23 05:40:10,694 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000019_0 decomp: 906858 len: 906862 to MEMORY
2015-03-23 05:40:10,696 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 899893 bytes from map-output for attempt_1427088391284_0001_m_000041_0
2015-03-23 05:40:10,696 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 899893, inMemoryMapOutputs.size() -> 4, commitMemory -> 2712590, usedMemory ->5433200
2015-03-23 05:40:10,696 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b8ca09b7:13562 freed by fetcher#5 in 750ms
2015-03-23 05:40:10,696 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b5ca09ba:13562 with 1 to fetcher#5
2015-03-23 05:40:10,696 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b5ca09ba:13562 to fetcher#5
2015-03-23 05:40:10,742 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000029_0 sent hash and received reply
2015-03-23 05:40:10,744 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000035_0 decomp: 913304 len: 913308 to MEMORY
2015-03-23 05:40:10,771 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913859 bytes from map-output for attempt_1427088391284_0001_m_000015_0
2015-03-23 05:40:10,771 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913859, inMemoryMapOutputs.size() -> 5, commitMemory -> 3612483, usedMemory ->7259802
2015-03-23 05:40:10,772 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6cca0963:13562 freed by fetcher#2 in 824ms
2015-03-23 05:40:10,772 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9aca0995:13562 with 1 to fetcher#2
2015-03-23 05:40:10,772 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9aca0995:13562 to fetcher#2
2015-03-23 05:40:10,774 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000026_0 sent hash and received reply
2015-03-23 05:40:10,774 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000026_0 decomp: 907510 len: 907514 to MEMORY
2015-03-23 05:40:10,782 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000028_0 sent hash and received reply
2015-03-23 05:40:10,782 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000028_0 decomp: 910978 len: 910982 to MEMORY
2015-03-23 05:40:10,783 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000029_0 decomp: 913298 len: 913302 to MEMORY
2015-03-23 05:40:10,822 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907510 bytes from map-output for attempt_1427088391284_0001_m_000026_0
2015-03-23 05:40:10,822 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907510, inMemoryMapOutputs.size() -> 6, commitMemory -> 4526342, usedMemory ->9078290
2015-03-23 05:40:10,822 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b5ca09ba:13562 freed by fetcher#5 in 126ms
2015-03-23 05:40:10,822 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5fca0950:13562 with 1 to fetcher#5
2015-03-23 05:40:10,822 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5fca0950:13562 to fetcher#5
2015-03-23 05:40:10,825 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913304 bytes from map-output for attempt_1427088391284_0001_m_000035_0
2015-03-23 05:40:10,825 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913304, inMemoryMapOutputs.size() -> 7, commitMemory -> 5433852, usedMemory ->9078290
2015-03-23 05:40:10,826 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ddca09d2:13562 freed by fetcher#1 in 145ms
2015-03-23 05:40:10,826 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5dca0952:13562 with 1 to fetcher#1
2015-03-23 05:40:10,826 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5dca0952:13562 to fetcher#1
2015-03-23 05:40:10,837 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000008_0 sent hash and received reply
2015-03-23 05:40:10,838 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000008_0 decomp: 909738 len: 909742 to MEMORY
2015-03-23 05:40:10,842 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000036_0 sent hash and received reply
2015-03-23 05:40:10,843 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000036_0 decomp: 915268 len: 915272 to MEMORY
2015-03-23 05:40:10,874 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913298 bytes from map-output for attempt_1427088391284_0001_m_000029_0
2015-03-23 05:40:10,874 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913298, inMemoryMapOutputs.size() -> 8, commitMemory -> 6347156, usedMemory ->10903296
2015-03-23 05:40:10,875 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-cfca09c0:13562 freed by fetcher#4 in 190ms
2015-03-23 05:40:10,875 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bcca09b3:13562 with 1 to fetcher#4
2015-03-23 05:40:10,875 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bcca09b3:13562 to fetcher#4
2015-03-23 05:40:10,876 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910978 bytes from map-output for attempt_1427088391284_0001_m_000028_0
2015-03-23 05:40:10,876 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910978, inMemoryMapOutputs.size() -> 9, commitMemory -> 7260454, usedMemory ->10903296
2015-03-23 05:40:10,876 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9aca0995:13562 freed by fetcher#2 in 104ms
2015-03-23 05:40:10,876 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c5ca09ca:13562 with 1 to fetcher#2
2015-03-23 05:40:10,876 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c5ca09ca:13562 to fetcher#2
2015-03-23 05:40:10,906 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909738 bytes from map-output for attempt_1427088391284_0001_m_000008_0
2015-03-23 05:40:10,906 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909738, inMemoryMapOutputs.size() -> 10, commitMemory -> 8171432, usedMemory ->10903296
2015-03-23 05:40:10,906 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5dca0952:13562 freed by fetcher#1 in 80ms
2015-03-23 05:40:10,906 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-09c90a06:13562 with 1 to fetcher#1
2015-03-23 05:40:10,906 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-09c90a06:13562 to fetcher#1
2015-03-23 05:40:10,908 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000042_0 sent hash and received reply
2015-03-23 05:40:10,909 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000042_0 decomp: 900318 len: 900322 to MEMORY
2015-03-23 05:40:10,917 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000034_0 sent hash and received reply
2015-03-23 05:40:11,099 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000047_0 sent hash and received reply
2015-03-23 05:40:11,100 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 915268 bytes from map-output for attempt_1427088391284_0001_m_000036_0
2015-03-23 05:40:11,100 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906858 bytes from map-output for attempt_1427088391284_0001_m_000019_0
2015-03-23 05:40:11,102 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 900318 bytes from map-output for attempt_1427088391284_0001_m_000042_0
2015-03-23 05:40:11,102 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000034_0 decomp: 910718 len: 910722 to MEMORY
2015-03-23 05:40:11,106 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 915268, inMemoryMapOutputs.size() -> 11, commitMemory -> 9081170, usedMemory ->12714332
2015-03-23 05:40:11,106 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5fca0950:13562 freed by fetcher#5 in 284ms
2015-03-23 05:40:11,106 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6bca0964:13562 with 1 to fetcher#5
2015-03-23 05:40:11,106 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6bca0964:13562 to fetcher#5
2015-03-23 05:40:11,108 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000047_0 decomp: 891621 len: 891625 to MEMORY
2015-03-23 05:40:11,109 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 900318, inMemoryMapOutputs.size() -> 12, commitMemory -> 9996438, usedMemory ->13605953
2015-03-23 05:40:11,109 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906858, inMemoryMapOutputs.size() -> 13, commitMemory -> 10896756, usedMemory ->13605953
2015-03-23 05:40:11,110 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-0bc90a04:13562 freed by fetcher#3 in 422ms
2015-03-23 05:40:11,110 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8eca0981:13562 with 1 to fetcher#3
2015-03-23 05:40:11,110 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8eca0981:13562 to fetcher#3
2015-03-23 05:40:11,111 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bcca09b3:13562 freed by fetcher#4 in 235ms
2015-03-23 05:40:11,111 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f4ca09fb:13562 with 1 to fetcher#4
2015-03-23 05:40:11,111 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f4ca09fb:13562 to fetcher#4
2015-03-23 05:40:11,115 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000005_0 sent hash and received reply
2015-03-23 05:40:11,116 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000005_0 decomp: 905312 len: 905316 to MEMORY
2015-03-23 05:40:11,146 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000014_0 sent hash and received reply
2015-03-23 05:40:11,154 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000014_0 decomp: 909077 len: 909081 to MEMORY
2015-03-23 05:40:11,164 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910718 bytes from map-output for attempt_1427088391284_0001_m_000034_0
2015-03-23 05:40:11,164 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910718, inMemoryMapOutputs.size() -> 14, commitMemory -> 11803614, usedMemory ->15420342
2015-03-23 05:40:11,165 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c5ca09ca:13562 freed by fetcher#2 in 289ms
2015-03-23 05:40:11,165 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-97ca0998:13562 with 1 to fetcher#2
2015-03-23 05:40:11,165 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-97ca0998:13562 to fetcher#2
2015-03-23 05:40:11,191 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 891621 bytes from map-output for attempt_1427088391284_0001_m_000047_0
2015-03-23 05:40:11,191 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 891621, inMemoryMapOutputs.size() -> 15, commitMemory -> 12714332, usedMemory ->15420342
2015-03-23 05:40:11,192 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-09c90a06:13562 freed by fetcher#1 in 285ms
2015-03-23 05:40:11,192 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-03c90a0c:13562 with 1 to fetcher#1
2015-03-23 05:40:11,192 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-03c90a0c:13562 to fetcher#1
2015-03-23 05:40:11,195 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905312 bytes from map-output for attempt_1427088391284_0001_m_000005_0
2015-03-23 05:40:11,195 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905312, inMemoryMapOutputs.size() -> 16, commitMemory -> 13605953, usedMemory ->15420342
2015-03-23 05:40:11,195 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8eca0981:13562 freed by fetcher#3 in 85ms
2015-03-23 05:40:11,195 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-02c90a0d:13562 with 1 to fetcher#3
2015-03-23 05:40:11,195 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-02c90a0d:13562 to fetcher#3
2015-03-23 05:40:11,198 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000022_0 sent hash and received reply
2015-03-23 05:40:11,199 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000022_0 decomp: 910878 len: 910882 to MEMORY
2015-03-23 05:40:11,201 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000004_0 sent hash and received reply
2015-03-23 05:40:11,202 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000004_0 decomp: 908560 len: 908564 to MEMORY
2015-03-23 05:40:11,207 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000044_0 sent hash and received reply
2015-03-23 05:40:11,208 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000044_0 decomp: 901941 len: 901945 to MEMORY
2015-03-23 05:40:11,212 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909077 bytes from map-output for attempt_1427088391284_0001_m_000014_0
2015-03-23 05:40:11,212 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909077, inMemoryMapOutputs.size() -> 17, commitMemory -> 14511265, usedMemory ->18141721
2015-03-23 05:40:11,212 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f4ca09fb:13562 freed by fetcher#4 in 101ms
2015-03-23 05:40:11,213 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8fca0980:13562 with 1 to fetcher#4
2015-03-23 05:40:11,213 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8fca0980:13562 to fetcher#4
2015-03-23 05:40:11,245 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000021_0 sent hash and received reply
2015-03-23 05:40:11,245 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000046_0 sent hash and received reply
2015-03-23 05:40:11,246 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000046_0 decomp: 904787 len: 904791 to MEMORY
2015-03-23 05:40:11,252 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000021_0 decomp: 910184 len: 910188 to MEMORY
2015-03-23 05:40:11,253 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908560 bytes from map-output for attempt_1427088391284_0001_m_000004_0
2015-03-23 05:40:11,253 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908560, inMemoryMapOutputs.size() -> 18, commitMemory -> 15420342, usedMemory ->19956692
2015-03-23 05:40:11,253 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-02c90a0d:13562 freed by fetcher#3 in 58ms
2015-03-23 05:40:11,253 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f2ca09fd:13562 with 1 to fetcher#3
2015-03-23 05:40:11,253 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f2ca09fd:13562 to fetcher#3
2015-03-23 05:40:11,291 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910878 bytes from map-output for attempt_1427088391284_0001_m_000022_0
2015-03-23 05:40:11,291 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910878, inMemoryMapOutputs.size() -> 19, commitMemory -> 16328902, usedMemory ->19956692
2015-03-23 05:40:11,292 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-03c90a0c:13562 freed by fetcher#1 in 99ms
2015-03-23 05:40:11,294 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-53ca095c:13562 with 1 to fetcher#1
2015-03-23 05:40:11,294 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-53ca095c:13562 to fetcher#1
2015-03-23 05:40:11,298 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 901941 bytes from map-output for attempt_1427088391284_0001_m_000044_0
2015-03-23 05:40:11,298 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 901941, inMemoryMapOutputs.size() -> 20, commitMemory -> 17239780, usedMemory ->19956692
2015-03-23 05:40:11,299 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-97ca0998:13562 freed by fetcher#2 in 134ms
2015-03-23 05:40:11,299 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-a8ca09a7:13562 with 1 to fetcher#2
2015-03-23 05:40:11,299 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-a8ca09a7:13562 to fetcher#2
2015-03-23 05:40:11,305 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000012_0 sent hash and received reply
2015-03-23 05:40:11,306 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000003_0 sent hash and received reply
2015-03-23 05:40:11,309 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000012_0 decomp: 910437 len: 910441 to MEMORY
2015-03-23 05:40:11,336 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000003_0 decomp: 908959 len: 908963 to MEMORY
2015-03-23 05:40:11,337 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000043_0 sent hash and received reply
2015-03-23 05:40:11,338 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910184 bytes from map-output for attempt_1427088391284_0001_m_000021_0
2015-03-23 05:40:11,338 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910184, inMemoryMapOutputs.size() -> 21, commitMemory -> 18141721, usedMemory ->21776088
2015-03-23 05:40:11,339 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8fca0980:13562 freed by fetcher#4 in 127ms
2015-03-23 05:40:11,339 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-d7ca09d8:13562 with 1 to fetcher#4
2015-03-23 05:40:11,339 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-d7ca09d8:13562 to fetcher#4
2015-03-23 05:40:11,341 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000043_0 decomp: 904996 len: 905000 to MEMORY
2015-03-23 05:40:11,345 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000027_0 sent hash and received reply
2015-03-23 05:40:11,368 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904787 bytes from map-output for attempt_1427088391284_0001_m_000046_0
2015-03-23 05:40:11,369 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000027_0 decomp: 911926 len: 911930 to MEMORY
2015-03-23 05:40:11,407 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904787, inMemoryMapOutputs.size() -> 22, commitMemory -> 19051905, usedMemory ->23593010
2015-03-23 05:40:11,408 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6bca0964:13562 freed by fetcher#5 in 301ms
2015-03-23 05:40:11,408 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6fca0960:13562 with 1 to fetcher#5
2015-03-23 05:40:11,408 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6fca0960:13562 to fetcher#5
2015-03-23 05:40:11,413 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000017_0 sent hash and received reply
2015-03-23 05:40:11,414 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000017_0 decomp: 912470 len: 912474 to MEMORY
2015-03-23 05:40:11,436 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910437 bytes from map-output for attempt_1427088391284_0001_m_000012_0
2015-03-23 05:40:11,437 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910437, inMemoryMapOutputs.size() -> 23, commitMemory -> 19956692, usedMemory ->24505480
2015-03-23 05:40:11,440 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-53ca095c:13562 freed by fetcher#1 in 146ms
2015-03-23 05:40:11,440 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-64ca096b:13562 with 1 to fetcher#1
2015-03-23 05:40:11,441 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-64ca096b:13562 to fetcher#1
2015-03-23 05:40:11,474 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912470 bytes from map-output for attempt_1427088391284_0001_m_000017_0
2015-03-23 05:40:11,474 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912470, inMemoryMapOutputs.size() -> 24, commitMemory -> 20867129, usedMemory ->24505480
2015-03-23 05:40:11,475 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000001_0 sent hash and received reply
2015-03-23 05:40:11,475 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000001_0 decomp: 906778 len: 906782 to MEMORY
2015-03-23 05:40:11,476 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6fca0960:13562 freed by fetcher#5 in 68ms
2015-03-23 05:40:11,476 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ccca09c3:13562 with 1 to fetcher#5
2015-03-23 05:40:11,476 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ccca09c3:13562 to fetcher#5
2015-03-23 05:40:11,483 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000018_0 sent hash and received reply
2015-03-23 05:40:11,483 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000018_0 decomp: 910083 len: 910087 to MEMORY
2015-03-23 05:40:11,486 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911926 bytes from map-output for attempt_1427088391284_0001_m_000027_0
2015-03-23 05:40:11,486 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911926, inMemoryMapOutputs.size() -> 25, commitMemory -> 21779599, usedMemory ->26322341
2015-03-23 05:40:11,486 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-d7ca09d8:13562 freed by fetcher#4 in 147ms
2015-03-23 05:40:11,486 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-95ca099a:13562 with 1 to fetcher#4
2015-03-23 05:40:11,486 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-95ca099a:13562 to fetcher#4
2015-03-23 05:40:11,487 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904996 bytes from map-output for attempt_1427088391284_0001_m_000043_0
2015-03-23 05:40:11,488 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904996, inMemoryMapOutputs.size() -> 26, commitMemory -> 22691525, usedMemory ->26322341
2015-03-23 05:40:11,488 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f2ca09fd:13562 freed by fetcher#3 in 235ms
2015-03-23 05:40:11,488 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bdca09b2:13562 with 1 to fetcher#3
2015-03-23 05:40:11,488 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bdca09b2:13562 to fetcher#3
2015-03-23 05:40:11,518 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000037_0 sent hash and received reply
2015-03-23 05:40:11,518 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000037_0 decomp: 914242 len: 914246 to MEMORY
2015-03-23 05:40:11,522 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908959 bytes from map-output for attempt_1427088391284_0001_m_000003_0
2015-03-23 05:40:11,522 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908959, inMemoryMapOutputs.size() -> 27, commitMemory -> 23596521, usedMemory ->27236583
2015-03-23 05:40:11,523 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-a8ca09a7:13562 freed by fetcher#2 in 224ms
2015-03-23 05:40:11,523 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-08c90a07:13562 with 1 to fetcher#2
2015-03-23 05:40:11,523 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-08c90a07:13562 to fetcher#2
2015-03-23 05:40:11,527 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000000_0 sent hash and received reply
2015-03-23 05:40:11,528 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000000_0 decomp: 907447 len: 907451 to MEMORY
2015-03-23 05:40:11,529 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906778 bytes from map-output for attempt_1427088391284_0001_m_000001_0
2015-03-23 05:40:11,529 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906778, inMemoryMapOutputs.size() -> 28, commitMemory -> 24505480, usedMemory ->28144030
2015-03-23 05:40:11,529 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-64ca096b:13562 freed by fetcher#1 in 89ms
2015-03-23 05:40:11,529 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-00c90a0f:13562 with 1 to fetcher#1
2015-03-23 05:40:11,529 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-00c90a0f:13562 to fetcher#1
2015-03-23 05:40:11,534 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000032_0 sent hash and received reply
2015-03-23 05:40:11,535 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000032_0 decomp: 910336 len: 910340 to MEMORY
2015-03-23 05:40:11,575 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910083 bytes from map-output for attempt_1427088391284_0001_m_000018_0
2015-03-23 05:40:11,575 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910083, inMemoryMapOutputs.size() -> 29, commitMemory -> 25412258, usedMemory ->29054366
2015-03-23 05:40:11,579 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ccca09c3:13562 freed by fetcher#5 in 103ms
2015-03-23 05:40:11,580 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-e3ca09ec:13562 with 1 to fetcher#5
2015-03-23 05:40:11,580 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-e3ca09ec:13562 to fetcher#5
2015-03-23 05:40:11,584 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907447 bytes from map-output for attempt_1427088391284_0001_m_000000_0
2015-03-23 05:40:11,584 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907447, inMemoryMapOutputs.size() -> 30, commitMemory -> 26322341, usedMemory ->29054366
2015-03-23 05:40:11,585 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bdca09b2:13562 freed by fetcher#3 in 96ms
2015-03-23 05:40:11,585 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-92ca099d:13562 with 1 to fetcher#3
2015-03-23 05:40:11,585 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-92ca099d:13562 to fetcher#3
2015-03-23 05:40:11,590 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000023_0 sent hash and received reply
2015-03-23 05:40:11,591 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000023_0 decomp: 912097 len: 912101 to MEMORY
2015-03-23 05:40:11,605 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000020_0 sent hash and received reply
2015-03-23 05:40:11,606 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000020_0 decomp: 907992 len: 907996 to MEMORY
2015-03-23 05:40:11,635 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912097 bytes from map-output for attempt_1427088391284_0001_m_000023_0
2015-03-23 05:40:11,635 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912097, inMemoryMapOutputs.size() -> 31, commitMemory -> 27229788, usedMemory ->30874455
2015-03-23 05:40:11,635 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-92ca099d:13562 freed by fetcher#3 in 50ms
2015-03-23 05:40:11,636 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-acca09a3:13562 with 1 to fetcher#3
2015-03-23 05:40:11,636 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-acca09a3:13562 to fetcher#3
2015-03-23 05:40:11,637 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000045_0 sent hash and received reply
2015-03-23 05:40:11,638 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000045_0 decomp: 902686 len: 902690 to MEMORY
2015-03-23 05:40:11,641 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000016_0 sent hash and received reply
2015-03-23 05:40:11,642 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000016_0 decomp: 914773 len: 914777 to MEMORY
2015-03-23 05:40:11,656 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914242 bytes from map-output for attempt_1427088391284_0001_m_000037_0
2015-03-23 05:40:11,656 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914242, inMemoryMapOutputs.size() -> 32, commitMemory -> 28141885, usedMemory ->32691914
2015-03-23 05:40:11,685 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-95ca099a:13562 freed by fetcher#4 in 199ms
2015-03-23 05:40:11,685 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c2ca09cd:13562 with 1 to fetcher#4
2015-03-23 05:40:11,688 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c2ca09cd:13562 to fetcher#4
2015-03-23 05:40:11,693 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000025_0 sent hash and received reply
2015-03-23 05:40:11,694 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000025_0 decomp: 914687 len: 914691 to MEMORY
2015-03-23 05:40:11,699 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 902686 bytes from map-output for attempt_1427088391284_0001_m_000045_0
2015-03-23 05:40:11,699 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 902686, inMemoryMapOutputs.size() -> 33, commitMemory -> 29056127, usedMemory ->33606601
2015-03-23 05:40:11,699 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-08c90a07:13562 freed by fetcher#2 in 176ms
2015-03-23 05:40:11,699 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b3ca09bc:13562 with 1 to fetcher#2
2015-03-23 05:40:11,699 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b3ca09bc:13562 to fetcher#2
2015-03-23 05:40:11,705 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000040_0 sent hash and received reply
2015-03-23 05:40:11,961 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910336 bytes from map-output for attempt_1427088391284_0001_m_000032_0
2015-03-23 05:40:11,962 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000040_0 decomp: 898834 len: 898838 to MEMORY
2015-03-23 05:40:11,963 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910336, inMemoryMapOutputs.size() -> 34, commitMemory -> 29958813, usedMemory ->34505435
2015-03-23 05:40:11,963 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-00c90a0f:13562 freed by fetcher#1 in 434ms
2015-03-23 05:40:11,963 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-51ca095e:13562 with 1 to fetcher#1
2015-03-23 05:40:11,963 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-51ca095e:13562 to fetcher#1
2015-03-23 05:40:11,965 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914773 bytes from map-output for attempt_1427088391284_0001_m_000016_0
2015-03-23 05:40:11,965 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914773, inMemoryMapOutputs.size() -> 35, commitMemory -> 30869149, usedMemory ->34505435
2015-03-23 05:40:11,966 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-acca09a3:13562 freed by fetcher#3 in 330ms
2015-03-23 05:40:11,966 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-daca09d5:13562 with 1 to fetcher#3
2015-03-23 05:40:11,966 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-daca09d5:13562 to fetcher#3
2015-03-23 05:40:11,971 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000038_0 sent hash and received reply
2015-03-23 05:40:11,972 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000038_0 decomp: 909524 len: 909528 to MEMORY
2015-03-23 05:40:11,979 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000006_0 sent hash and received reply
2015-03-23 05:40:11,980 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000006_0 decomp: 904088 len: 904092 to MEMORY
2015-03-23 05:40:12,015 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914687 bytes from map-output for attempt_1427088391284_0001_m_000025_0
2015-03-23 05:40:12,015 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914687, inMemoryMapOutputs.size() -> 36, commitMemory -> 31783922, usedMemory ->36319047
2015-03-23 05:40:12,015 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c2ca09cd:13562 freed by fetcher#4 in 330ms
2015-03-23 05:40:12,015 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-04c90a0b:13562 with 1 to fetcher#4
2015-03-23 05:40:12,016 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-04c90a0b:13562 to fetcher#4
2015-03-23 05:40:12,019 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 898834 bytes from map-output for attempt_1427088391284_0001_m_000040_0
2015-03-23 05:40:12,019 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 898834, inMemoryMapOutputs.size() -> 37, commitMemory -> 32698609, usedMemory ->36319047
2015-03-23 05:40:12,019 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b3ca09bc:13562 freed by fetcher#2 in 320ms
2015-03-23 05:40:12,019 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9cca0993:13562 with 1 to fetcher#2
2015-03-23 05:40:12,019 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9cca0993:13562 to fetcher#2
2015-03-23 05:40:12,021 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000009_0 sent hash and received reply
2015-03-23 05:40:12,023 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000009_0 decomp: 912408 len: 912412 to MEMORY
2015-03-23 05:40:12,028 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909524 bytes from map-output for attempt_1427088391284_0001_m_000038_0
2015-03-23 05:40:12,028 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909524, inMemoryMapOutputs.size() -> 38, commitMemory -> 33597443, usedMemory ->37231455
2015-03-23 05:40:12,028 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-daca09d5:13562 freed by fetcher#3 in 62ms
2015-03-23 05:40:12,028 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-7eca0971:13562 with 1 to fetcher#3
2015-03-23 05:40:12,028 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-7eca0971:13562 to fetcher#3
2015-03-23 05:40:12,058 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000013_0 sent hash and received reply
2015-03-23 05:40:12,059 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000013_0 decomp: 911081 len: 911085 to MEMORY
2015-03-23 05:40:12,063 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000024_0 sent hash and received reply
2015-03-23 05:40:12,064 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000024_0 decomp: 912766 len: 912770 to MEMORY
2015-03-23 05:40:12,083 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907992 bytes from map-output for attempt_1427088391284_0001_m_000020_0
2015-03-23 05:40:12,083 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907992, inMemoryMapOutputs.size() -> 39, commitMemory -> 34506967, usedMemory ->39055302
2015-03-23 05:40:12,083 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-e3ca09ec:13562 freed by fetcher#5 in 503ms
2015-03-23 05:40:12,083 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-dfca09d0:13562 with 1 to fetcher#5
2015-03-23 05:40:12,084 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-dfca09d0:13562 to fetcher#5
2015-03-23 05:40:12,086 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912408 bytes from map-output for attempt_1427088391284_0001_m_000009_0
2015-03-23 05:40:12,087 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912408, inMemoryMapOutputs.size() -> 40, commitMemory -> 35414959, usedMemory ->39055302
2015-03-23 05:40:12,088 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-04c90a0b:13562 freed by fetcher#4 in 73ms
2015-03-23 05:40:12,088 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-33ca093c:13562 with 1 to fetcher#4
2015-03-23 05:40:12,088 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-33ca093c:13562 to fetcher#4
2015-03-23 05:40:12,091 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000031_0 sent hash and received reply
2015-03-23 05:40:12,092 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000031_0 decomp: 908081 len: 908085 to MEMORY
2015-03-23 05:40:12,098 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000011_0 sent hash and received reply
2015-03-23 05:40:12,099 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000011_0 decomp: 910777 len: 910781 to MEMORY
2015-03-23 05:40:12,129 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912766 bytes from map-output for attempt_1427088391284_0001_m_000024_0
2015-03-23 05:40:12,130 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912766, inMemoryMapOutputs.size() -> 41, commitMemory -> 36327367, usedMemory ->40874160
2015-03-23 05:40:12,131 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-7eca0971:13562 freed by fetcher#3 in 103ms
2015-03-23 05:40:12,131 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-62ca096d:13562 with 1 to fetcher#3
2015-03-23 05:40:12,132 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-62ca096d:13562 to fetcher#3
2015-03-23 05:40:12,138 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911081 bytes from map-output for attempt_1427088391284_0001_m_000013_0
2015-03-23 05:40:12,138 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911081, inMemoryMapOutputs.size() -> 42, commitMemory -> 37240133, usedMemory ->40874160
2015-03-23 05:40:12,139 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9cca0993:13562 freed by fetcher#2 in 120ms
2015-03-23 05:40:12,139 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-81ca098e:13562 with 1 to fetcher#2
2015-03-23 05:40:12,139 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-81ca098e:13562 to fetcher#2
2015-03-23 05:40:12,146 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908081 bytes from map-output for attempt_1427088391284_0001_m_000031_0
2015-03-23 05:40:12,146 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908081, inMemoryMapOutputs.size() -> 43, commitMemory -> 38151214, usedMemory ->40874160
2015-03-23 05:40:12,147 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000030_0 sent hash and received reply
2015-03-23 05:40:12,147 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000030_0 decomp: 911061 len: 911065 to MEMORY
2015-03-23 05:40:12,150 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-dfca09d0:13562 freed by fetcher#5 in 67ms
2015-03-23 05:40:12,150 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-82ca098d:13562 with 1 to fetcher#5
2015-03-23 05:40:12,175 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000007_0 sent hash and received reply
2015-03-23 05:40:12,175 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000007_0 decomp: 904924 len: 904928 to MEMORY
2015-03-23 05:40:12,177 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910777 bytes from map-output for attempt_1427088391284_0001_m_000011_0
2015-03-23 05:40:12,178 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904088 bytes from map-output for attempt_1427088391284_0001_m_000006_0
2015-03-23 05:40:12,179 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-82ca098d:13562 to fetcher#5
2015-03-23 05:40:12,179 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904088, inMemoryMapOutputs.size() -> 44, commitMemory -> 39059295, usedMemory ->42690145
2015-03-23 05:40:12,179 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-51ca095e:13562 freed by fetcher#1 in 216ms
2015-03-23 05:40:12,179 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910777, inMemoryMapOutputs.size() -> 45, commitMemory -> 39963383, usedMemory ->42690145
2015-03-23 05:40:12,180 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-33ca093c:13562 freed by fetcher#4 in 92ms
2015-03-23 05:40:12,185 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=9&map=attempt_1427088391284_0001_m_000033_0 sent hash and received reply
2015-03-23 05:40:12,187 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000033_0 decomp: 911769 len: 911773 to MEMORY
2015-03-23 05:40:12,196 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911061 bytes from map-output for attempt_1427088391284_0001_m_000030_0
2015-03-23 05:40:12,197 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911061, inMemoryMapOutputs.size() -> 46, commitMemory -> 40874160, usedMemory ->43601914
2015-03-23 05:40:12,198 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-81ca098e:13562 freed by fetcher#2 in 59ms
2015-03-23 05:40:12,236 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911769 bytes from map-output for attempt_1427088391284_0001_m_000033_0
2015-03-23 05:40:12,236 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911769, inMemoryMapOutputs.size() -> 47, commitMemory -> 41785221, usedMemory ->43601914
2015-03-23 05:40:12,237 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-82ca098d:13562 freed by fetcher#5 in 87ms
2015-03-23 05:40:12,245 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904924 bytes from map-output for attempt_1427088391284_0001_m_000007_0
2015-03-23 05:40:12,245 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904924, inMemoryMapOutputs.size() -> 48, commitMemory -> 42696990, usedMemory ->43601914
2015-03-23 05:40:12,245 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2015-03-23 05:40:12,246 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-62ca096d:13562 freed by fetcher#3 in 115ms
2015-03-23 05:40:12,251 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 48 in-memory map-outputs and 0 on-disk map-outputs
2015-03-23 05:40:12,298 INFO [main] org.apache.hadoop.mapred.Merger: Merging 48 sorted segments
2015-03-23 05:40:12,299 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 48 segments left of total size: 43601434 bytes
2015-03-23 05:40:21,575 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 48 segments, 43601914 bytes to disk to satisfy reduce memory limit
2015-03-23 05:40:21,576 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 43601824 bytes from disk
2015-03-23 05:40:21,578 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2015-03-23 05:40:21,578 INFO [main] org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2015-03-23 05:40:21,583 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 43601810 bytes
2015-03-23 05:40:21,607 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-03-23 05:40:22,164 INFO [main] org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2015-03-23 05:40:22,166 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2015-03-23 05:40:36,228 INFO [main] HiBench.HiveData$GenerateRankingsReducer: pid: 9, 0 erros, 0 missed
2015-03-23 05:40:36,520 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1427088391284_0001_r_000009_0 is done. And is in the process of committing
2015-03-23 05:40:36,561 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1427088391284_0001_r_000009_0 is allowed to commit now
2015-03-23 05:40:36,573 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1427088391284_0001_r_000009_0' to hdfs://172.31.17.135:8120/HiBench/Hive/Input-comp/rankings/_temporary/1/task_1427088391284_0001_r_000009
2015-03-23 05:40:36,631 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1427088391284_0001_r_000009_0' done.
2015-03-23 05:40:36,660 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ReduceTask metrics system...
2015-03-23 05:40:36,661 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system stopped.
2015-03-23 05:40:36,661 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system shutdown complete.
