2015-03-23 05:39:58,690 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-03-23 05:39:59,230 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-03-23 05:39:59,230 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system started
2015-03-23 05:39:59,323 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2015-03-23 05:39:59,323 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1427088391284_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@482dbdc2)
2015-03-23 05:40:00,003 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2015-03-23 05:40:01,533 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /tmp/hadoop-ubuntu/nm-local-dir/usercache/ubuntu/appcache/application_1427088391284_0001
2015-03-23 05:40:06,890 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2015-03-23 05:40:09,016 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2015-03-23 05:40:09,215 INFO [main] org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@a3768da
2015-03-23 05:40:09,312 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=1503238528, maxSingleShuffleLimit=375809632, mergeThreshold=992137472, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-03-23 05:40:09,345 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000070_0 Thread started: EventFetcher for fetching Map Completion Events
2015-03-23 05:40:09,398 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-daca09d5:13562 with 1 to fetcher#5
2015-03-23 05:40:09,398 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-daca09d5:13562 to fetcher#5
2015-03-23 05:40:09,435 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9cca0993:13562 with 1 to fetcher#1
2015-03-23 05:40:09,436 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9cca0993:13562 to fetcher#1
2015-03-23 05:40:09,437 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6fca0960:13562 with 1 to fetcher#4
2015-03-23 05:40:09,437 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6fca0960:13562 to fetcher#4
2015-03-23 05:40:09,437 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8fca0980:13562 with 1 to fetcher#2
2015-03-23 05:40:09,437 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8fca0980:13562 to fetcher#2
2015-03-23 05:40:09,438 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-33ca093c:13562 with 1 to fetcher#3
2015-03-23 05:40:09,438 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-33ca093c:13562 to fetcher#3
2015-03-23 05:40:09,486 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1427088391284_0001_r_000070_0: Got 48 new map-outputs
2015-03-23 05:40:10,125 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000021_0 sent hash and received reply
2015-03-23 05:40:10,127 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000011_0 sent hash and received reply
2015-03-23 05:40:10,127 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000017_0 sent hash and received reply
2015-03-23 05:40:10,127 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000038_0 sent hash and received reply
2015-03-23 05:40:10,128 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000013_0 sent hash and received reply
2015-03-23 05:40:10,136 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000038_0 decomp: 910945 len: 910949 to MEMORY
2015-03-23 05:40:10,137 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000011_0 decomp: 910313 len: 910317 to MEMORY
2015-03-23 05:40:10,139 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000017_0 decomp: 909457 len: 909461 to MEMORY
2015-03-23 05:40:10,139 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000021_0 decomp: 910728 len: 910732 to MEMORY
2015-03-23 05:40:10,140 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000013_0 decomp: 910198 len: 910202 to MEMORY
2015-03-23 05:40:10,236 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910728 bytes from map-output for attempt_1427088391284_0001_m_000021_0
2015-03-23 05:40:10,243 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910313 bytes from map-output for attempt_1427088391284_0001_m_000011_0
2015-03-23 05:40:10,242 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910945 bytes from map-output for attempt_1427088391284_0001_m_000038_0
2015-03-23 05:40:10,249 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910945, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->4551641
2015-03-23 05:40:10,251 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910728, inMemoryMapOutputs.size() -> 2, commitMemory -> 910945, usedMemory ->4551641
2015-03-23 05:40:10,252 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8fca0980:13562 freed by fetcher#2 in 815ms
2015-03-23 05:40:10,253 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bdca09b2:13562 with 1 to fetcher#2
2015-03-23 05:40:10,253 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bdca09b2:13562 to fetcher#2
2015-03-23 05:40:10,254 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910313, inMemoryMapOutputs.size() -> 3, commitMemory -> 1821673, usedMemory ->4551641
2015-03-23 05:40:10,255 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-33ca093c:13562 freed by fetcher#3 in 817ms
2015-03-23 05:40:10,255 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-9aca0995:13562 with 1 to fetcher#3
2015-03-23 05:40:10,255 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-9aca0995:13562 to fetcher#3
2015-03-23 05:40:10,257 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-daca09d5:13562 freed by fetcher#5 in 858ms
2015-03-23 05:40:10,257 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-beca09b1:13562 with 1 to fetcher#5
2015-03-23 05:40:10,260 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-beca09b1:13562 to fetcher#5
2015-03-23 05:40:10,287 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000028_0 sent hash and received reply
2015-03-23 05:40:10,288 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000028_0 decomp: 908901 len: 908905 to MEMORY
2015-03-23 05:40:10,292 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000010_0 sent hash and received reply
2015-03-23 05:40:10,313 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000010_0 decomp: 912282 len: 912286 to MEMORY
2015-03-23 05:40:10,347 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000000_0 sent hash and received reply
2015-03-23 05:40:10,347 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000000_0 decomp: 904485 len: 904489 to MEMORY
2015-03-23 05:40:10,348 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909457 bytes from map-output for attempt_1427088391284_0001_m_000017_0
2015-03-23 05:40:10,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909457, inMemoryMapOutputs.size() -> 4, commitMemory -> 2731986, usedMemory ->7277309
2015-03-23 05:40:10,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6fca0960:13562 freed by fetcher#4 in 912ms
2015-03-23 05:40:10,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5eca0951:13562 with 1 to fetcher#4
2015-03-23 05:40:10,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5eca0951:13562 to fetcher#4
2015-03-23 05:40:10,356 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000002_0 sent hash and received reply
2015-03-23 05:40:10,357 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000002_0 decomp: 904854 len: 904858 to MEMORY
2015-03-23 05:40:10,365 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910198 bytes from map-output for attempt_1427088391284_0001_m_000013_0
2015-03-23 05:40:10,365 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910198, inMemoryMapOutputs.size() -> 5, commitMemory -> 3641443, usedMemory ->8182163
2015-03-23 05:40:10,365 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9cca0993:13562 freed by fetcher#1 in 929ms
2015-03-23 05:40:10,365 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-82ca098d:13562 with 1 to fetcher#1
2015-03-23 05:40:10,365 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-82ca098d:13562 to fetcher#1
2015-03-23 05:40:10,368 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908901 bytes from map-output for attempt_1427088391284_0001_m_000028_0
2015-03-23 05:40:10,368 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908901, inMemoryMapOutputs.size() -> 6, commitMemory -> 4551641, usedMemory ->8182163
2015-03-23 05:40:10,368 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-9aca0995:13562 freed by fetcher#3 in 113ms
2015-03-23 05:40:10,369 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-97ca0998:13562 with 1 to fetcher#3
2015-03-23 05:40:10,369 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-97ca0998:13562 to fetcher#3
2015-03-23 05:40:10,399 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000044_0 sent hash and received reply
2015-03-23 05:40:10,400 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000033_0 sent hash and received reply
2015-03-23 05:40:10,402 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000044_0 decomp: 910310 len: 910314 to MEMORY
2015-03-23 05:40:10,403 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000033_0 decomp: 906481 len: 906485 to MEMORY
2015-03-23 05:40:10,412 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912282 bytes from map-output for attempt_1427088391284_0001_m_000010_0
2015-03-23 05:40:10,412 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912282, inMemoryMapOutputs.size() -> 7, commitMemory -> 5460542, usedMemory ->9998954
2015-03-23 05:40:10,413 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-beca09b1:13562 freed by fetcher#5 in 154ms
2015-03-23 05:40:10,413 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6cca0963:13562 with 1 to fetcher#5
2015-03-23 05:40:10,413 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6cca0963:13562 to fetcher#5
2015-03-23 05:40:10,418 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904854 bytes from map-output for attempt_1427088391284_0001_m_000002_0
2015-03-23 05:40:10,418 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904854, inMemoryMapOutputs.size() -> 8, commitMemory -> 6372824, usedMemory ->9998954
2015-03-23 05:40:10,418 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5eca0951:13562 freed by fetcher#4 in 69ms
2015-03-23 05:40:10,443 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-92ca099d:13562 with 1 to fetcher#4
2015-03-23 05:40:10,443 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-92ca099d:13562 to fetcher#4
2015-03-23 05:40:10,453 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906481 bytes from map-output for attempt_1427088391284_0001_m_000033_0
2015-03-23 05:40:10,453 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906481, inMemoryMapOutputs.size() -> 9, commitMemory -> 7277678, usedMemory ->9998954
2015-03-23 05:40:10,453 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-82ca098d:13562 freed by fetcher#1 in 88ms
2015-03-23 05:40:10,453 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-51ca095e:13562 with 1 to fetcher#1
2015-03-23 05:40:10,453 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-51ca095e:13562 to fetcher#1
2015-03-23 05:40:10,462 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000023_0 sent hash and received reply
2015-03-23 05:40:10,462 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000023_0 decomp: 914561 len: 914565 to MEMORY
2015-03-23 05:40:10,467 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000006_0 sent hash and received reply
2015-03-23 05:40:10,468 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000015_0 sent hash and received reply
2015-03-23 05:40:10,661 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000006_0 decomp: 909463 len: 909467 to MEMORY
2015-03-23 05:40:10,664 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000015_0 decomp: 913047 len: 913051 to MEMORY
2015-03-23 05:40:10,711 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914561 bytes from map-output for attempt_1427088391284_0001_m_000023_0
2015-03-23 05:40:10,723 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914561, inMemoryMapOutputs.size() -> 10, commitMemory -> 8184159, usedMemory ->12736025
2015-03-23 05:40:10,724 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-92ca099d:13562 freed by fetcher#4 in 281ms
2015-03-23 05:40:10,724 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ddca09d2:13562 with 1 to fetcher#4
2015-03-23 05:40:10,724 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ddca09d2:13562 to fetcher#4
2015-03-23 05:40:10,728 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910310 bytes from map-output for attempt_1427088391284_0001_m_000044_0
2015-03-23 05:40:10,728 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910310, inMemoryMapOutputs.size() -> 11, commitMemory -> 9098720, usedMemory ->12736025
2015-03-23 05:40:10,729 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-97ca0998:13562 freed by fetcher#3 in 360ms
2015-03-23 05:40:10,729 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-08c90a07:13562 with 1 to fetcher#3
2015-03-23 05:40:10,729 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-08c90a07:13562 to fetcher#3
2015-03-23 05:40:10,730 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913047 bytes from map-output for attempt_1427088391284_0001_m_000015_0
2015-03-23 05:40:10,730 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913047, inMemoryMapOutputs.size() -> 12, commitMemory -> 10009030, usedMemory ->12736025
2015-03-23 05:40:10,756 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904485 bytes from map-output for attempt_1427088391284_0001_m_000000_0
2015-03-23 05:40:10,756 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904485, inMemoryMapOutputs.size() -> 13, commitMemory -> 10922077, usedMemory ->12736025
2015-03-23 05:40:10,756 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bdca09b2:13562 freed by fetcher#2 in 504ms
2015-03-23 05:40:10,756 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-81ca098e:13562 with 1 to fetcher#2
2015-03-23 05:40:10,757 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-81ca098e:13562 to fetcher#2
2015-03-23 05:40:10,758 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6cca0963:13562 freed by fetcher#5 in 345ms
2015-03-23 05:40:10,758 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-6bca0964:13562 with 1 to fetcher#5
2015-03-23 05:40:10,758 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-6bca0964:13562 to fetcher#5
2015-03-23 05:40:10,762 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000030_0 sent hash and received reply
2015-03-23 05:40:10,764 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000030_0 decomp: 909503 len: 909507 to MEMORY
2015-03-23 05:40:10,766 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000035_0 sent hash and received reply
2015-03-23 05:40:10,767 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000035_0 decomp: 911494 len: 911498 to MEMORY
2015-03-23 05:40:10,772 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000045_0 sent hash and received reply
2015-03-23 05:40:10,774 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000045_0 decomp: 903746 len: 903750 to MEMORY
2015-03-23 05:40:10,779 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909463 bytes from map-output for attempt_1427088391284_0001_m_000006_0
2015-03-23 05:40:10,779 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909463, inMemoryMapOutputs.size() -> 14, commitMemory -> 11826562, usedMemory ->15460768
2015-03-23 05:40:10,780 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-51ca095e:13562 freed by fetcher#1 in 327ms
2015-03-23 05:40:10,780 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-0bc90a04:13562 with 1 to fetcher#1
2015-03-23 05:40:10,780 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-0bc90a04:13562 to fetcher#1
2015-03-23 05:40:10,785 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000046_0 sent hash and received reply
2015-03-23 05:40:10,786 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000046_0 decomp: 901837 len: 901841 to MEMORY
2015-03-23 05:40:10,822 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909503 bytes from map-output for attempt_1427088391284_0001_m_000030_0
2015-03-23 05:40:10,822 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909503, inMemoryMapOutputs.size() -> 15, commitMemory -> 12736025, usedMemory ->16362605
2015-03-23 05:40:10,828 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-81ca098e:13562 freed by fetcher#2 in 72ms
2015-03-23 05:40:10,830 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-53ca095c:13562 with 1 to fetcher#2
2015-03-23 05:40:10,830 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-53ca095c:13562 to fetcher#2
2015-03-23 05:40:10,834 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911494 bytes from map-output for attempt_1427088391284_0001_m_000035_0
2015-03-23 05:40:10,835 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911494, inMemoryMapOutputs.size() -> 16, commitMemory -> 13645528, usedMemory ->16362605
2015-03-23 05:40:10,835 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ddca09d2:13562 freed by fetcher#4 in 111ms
2015-03-23 05:40:10,836 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-bcca09b3:13562 with 1 to fetcher#4
2015-03-23 05:40:10,836 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-bcca09b3:13562 to fetcher#4
2015-03-23 05:40:10,863 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 903746 bytes from map-output for attempt_1427088391284_0001_m_000045_0
2015-03-23 05:40:10,864 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 903746, inMemoryMapOutputs.size() -> 17, commitMemory -> 14557022, usedMemory ->16362605
2015-03-23 05:40:10,864 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-08c90a07:13562 freed by fetcher#3 in 135ms
2015-03-23 05:40:10,864 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-dfca09d0:13562 with 1 to fetcher#3
2015-03-23 05:40:10,865 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-dfca09d0:13562 to fetcher#3
2015-03-23 05:40:10,870 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000012_0 sent hash and received reply
2015-03-23 05:40:10,870 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000012_0 decomp: 914030 len: 914034 to MEMORY
2015-03-23 05:40:10,872 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000042_0 sent hash and received reply
2015-03-23 05:40:10,874 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000042_0 decomp: 900959 len: 900963 to MEMORY
2015-03-23 05:40:10,878 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000031_0 sent hash and received reply
2015-03-23 05:40:10,880 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000031_0 decomp: 910073 len: 910077 to MEMORY
2015-03-23 05:40:10,912 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000019_0 sent hash and received reply
2015-03-23 05:40:10,913 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000019_0 decomp: 905246 len: 905250 to MEMORY
2015-03-23 05:40:10,939 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 901837 bytes from map-output for attempt_1427088391284_0001_m_000046_0
2015-03-23 05:40:10,939 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 901837, inMemoryMapOutputs.size() -> 18, commitMemory -> 15460768, usedMemory ->19992913
2015-03-23 05:40:10,939 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-6bca0964:13562 freed by fetcher#5 in 181ms
2015-03-23 05:40:10,939 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-e3ca09ec:13562 with 1 to fetcher#5
2015-03-23 05:40:10,940 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-e3ca09ec:13562 to fetcher#5
2015-03-23 05:40:10,974 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 900959 bytes from map-output for attempt_1427088391284_0001_m_000042_0
2015-03-23 05:40:10,977 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000020_0 sent hash and received reply
2015-03-23 05:40:10,977 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 900959, inMemoryMapOutputs.size() -> 19, commitMemory -> 16362605, usedMemory ->19992913
2015-03-23 05:40:10,978 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-bcca09b3:13562 freed by fetcher#4 in 143ms
2015-03-23 05:40:10,978 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-a8ca09a7:13562 with 1 to fetcher#4
2015-03-23 05:40:10,978 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-a8ca09a7:13562 to fetcher#4
2015-03-23 05:40:10,981 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000020_0 decomp: 912163 len: 912167 to MEMORY
2015-03-23 05:40:10,983 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910073 bytes from map-output for attempt_1427088391284_0001_m_000031_0
2015-03-23 05:40:10,983 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910073, inMemoryMapOutputs.size() -> 20, commitMemory -> 17263564, usedMemory ->20905076
2015-03-23 05:40:10,984 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-dfca09d0:13562 freed by fetcher#3 in 119ms
2015-03-23 05:40:10,984 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c2ca09cd:13562 with 1 to fetcher#3
2015-03-23 05:40:10,984 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c2ca09cd:13562 to fetcher#3
2015-03-23 05:40:10,988 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905246 bytes from map-output for attempt_1427088391284_0001_m_000019_0
2015-03-23 05:40:10,988 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905246, inMemoryMapOutputs.size() -> 21, commitMemory -> 18173637, usedMemory ->20905076
2015-03-23 05:40:10,988 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-0bc90a04:13562 freed by fetcher#1 in 208ms
2015-03-23 05:40:10,988 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-04c90a0b:13562 with 1 to fetcher#1
2015-03-23 05:40:10,988 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-04c90a0b:13562 to fetcher#1
2015-03-23 05:40:10,995 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000025_0 sent hash and received reply
2015-03-23 05:40:10,996 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000025_0 decomp: 909096 len: 909100 to MEMORY
2015-03-23 05:40:11,004 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000009_0 sent hash and received reply
2015-03-23 05:40:11,005 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914030 bytes from map-output for attempt_1427088391284_0001_m_000012_0
2015-03-23 05:40:11,005 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914030, inMemoryMapOutputs.size() -> 22, commitMemory -> 19078883, usedMemory ->21814172
2015-03-23 05:40:11,006 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-53ca095c:13562 freed by fetcher#2 in 177ms
2015-03-23 05:40:11,006 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-62ca096d:13562 with 1 to fetcher#2
2015-03-23 05:40:11,006 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-62ca096d:13562 to fetcher#2
2015-03-23 05:40:11,032 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000003_0 sent hash and received reply
2015-03-23 05:40:11,034 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000003_0 decomp: 907229 len: 907233 to MEMORY
2015-03-23 05:40:11,086 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000009_0 decomp: 910521 len: 910525 to MEMORY
2015-03-23 05:40:11,092 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000007_0 sent hash and received reply
2015-03-23 05:40:11,093 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000007_0 decomp: 906192 len: 906196 to MEMORY
2015-03-23 05:40:11,130 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909096 bytes from map-output for attempt_1427088391284_0001_m_000025_0
2015-03-23 05:40:11,133 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909096, inMemoryMapOutputs.size() -> 23, commitMemory -> 19992913, usedMemory ->24538114
2015-03-23 05:40:11,133 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c2ca09cd:13562 freed by fetcher#3 in 149ms
2015-03-23 05:40:11,133 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5dca0952:13562 with 1 to fetcher#3
2015-03-23 05:40:11,133 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5dca0952:13562 to fetcher#3
2015-03-23 05:40:11,151 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 912163 bytes from map-output for attempt_1427088391284_0001_m_000020_0
2015-03-23 05:40:11,151 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 912163, inMemoryMapOutputs.size() -> 24, commitMemory -> 20902009, usedMemory ->24538114
2015-03-23 05:40:11,152 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-e3ca09ec:13562 freed by fetcher#5 in 213ms
2015-03-23 05:40:11,152 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910521 bytes from map-output for attempt_1427088391284_0001_m_000009_0
2015-03-23 05:40:11,160 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 906192 bytes from map-output for attempt_1427088391284_0001_m_000007_0
2015-03-23 05:40:11,161 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-5fca0950:13562 with 1 to fetcher#5
2015-03-23 05:40:11,161 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-5fca0950:13562 to fetcher#5
2015-03-23 05:40:11,200 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000008_0 sent hash and received reply
2015-03-23 05:40:11,201 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000008_0 decomp: 905872 len: 905876 to MEMORY
2015-03-23 05:40:11,201 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910521, inMemoryMapOutputs.size() -> 25, commitMemory -> 21814172, usedMemory ->25443986
2015-03-23 05:40:11,201 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-04c90a0b:13562 freed by fetcher#1 in 213ms
2015-03-23 05:40:11,202 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b8ca09b7:13562 with 1 to fetcher#1
2015-03-23 05:40:11,202 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b8ca09b7:13562 to fetcher#1
2015-03-23 05:40:11,203 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 906192, inMemoryMapOutputs.size() -> 26, commitMemory -> 22724693, usedMemory ->25443986
2015-03-23 05:40:11,203 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-62ca096d:13562 freed by fetcher#2 in 197ms
2015-03-23 05:40:11,203 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8eca0981:13562 with 1 to fetcher#2
2015-03-23 05:40:11,203 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8eca0981:13562 to fetcher#2
2015-03-23 05:40:11,207 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000005_0 sent hash and received reply
2015-03-23 05:40:11,208 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000005_0 decomp: 904764 len: 904768 to MEMORY
2015-03-23 05:40:11,212 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000041_0 sent hash and received reply
2015-03-23 05:40:11,213 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000041_0 decomp: 902685 len: 902689 to MEMORY
2015-03-23 05:40:11,217 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907229 bytes from map-output for attempt_1427088391284_0001_m_000003_0
2015-03-23 05:40:11,220 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907229, inMemoryMapOutputs.size() -> 27, commitMemory -> 23630885, usedMemory ->27251435
2015-03-23 05:40:11,221 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-a8ca09a7:13562 freed by fetcher#4 in 243ms
2015-03-23 05:40:11,221 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-ccca09c3:13562 with 1 to fetcher#4
2015-03-23 05:40:11,221 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-ccca09c3:13562 to fetcher#4
2015-03-23 05:40:11,230 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000018_0 sent hash and received reply
2015-03-23 05:40:11,232 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000018_0 decomp: 909006 len: 909010 to MEMORY
2015-03-23 05:40:11,235 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904764 bytes from map-output for attempt_1427088391284_0001_m_000005_0
2015-03-23 05:40:11,235 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904764, inMemoryMapOutputs.size() -> 28, commitMemory -> 24538114, usedMemory ->28160441
2015-03-23 05:40:11,260 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8eca0981:13562 freed by fetcher#2 in 57ms
2015-03-23 05:40:11,261 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-64ca096b:13562 with 1 to fetcher#2
2015-03-23 05:40:11,261 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-64ca096b:13562 to fetcher#2
2015-03-23 05:40:11,266 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 902685 bytes from map-output for attempt_1427088391284_0001_m_000041_0
2015-03-23 05:40:11,266 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 902685, inMemoryMapOutputs.size() -> 29, commitMemory -> 25442878, usedMemory ->28160441
2015-03-23 05:40:11,266 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b8ca09b7:13562 freed by fetcher#1 in 64ms
2015-03-23 05:40:11,266 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f2ca09fd:13562 with 1 to fetcher#1
2015-03-23 05:40:11,266 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f2ca09fd:13562 to fetcher#1
2015-03-23 05:40:11,270 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000001_0 sent hash and received reply
2015-03-23 05:40:11,271 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000001_0 decomp: 910240 len: 910244 to MEMORY
2015-03-23 05:40:11,272 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000036_0 sent hash and received reply
2015-03-23 05:40:11,273 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000036_0 decomp: 908072 len: 908076 to MEMORY
2015-03-23 05:40:11,325 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910240 bytes from map-output for attempt_1427088391284_0001_m_000001_0
2015-03-23 05:40:11,325 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910240, inMemoryMapOutputs.size() -> 30, commitMemory -> 26345563, usedMemory ->29978753
2015-03-23 05:40:11,326 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-64ca096b:13562 freed by fetcher#2 in 66ms
2015-03-23 05:40:11,326 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-95ca099a:13562 with 1 to fetcher#2
2015-03-23 05:40:11,326 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-95ca099a:13562 to fetcher#2
2015-03-23 05:40:11,329 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 908072 bytes from map-output for attempt_1427088391284_0001_m_000036_0
2015-03-23 05:40:11,329 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 908072, inMemoryMapOutputs.size() -> 31, commitMemory -> 27255803, usedMemory ->29978753
2015-03-23 05:40:11,329 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5fca0950:13562 freed by fetcher#5 in 168ms
2015-03-23 05:40:11,330 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-acca09a3:13562 with 1 to fetcher#5
2015-03-23 05:40:11,330 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-acca09a3:13562 to fetcher#5
2015-03-23 05:40:11,356 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000037_0 sent hash and received reply
2015-03-23 05:40:11,357 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000037_0 decomp: 911325 len: 911329 to MEMORY
2015-03-23 05:40:11,363 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905872 bytes from map-output for attempt_1427088391284_0001_m_000008_0
2015-03-23 05:40:11,363 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905872, inMemoryMapOutputs.size() -> 32, commitMemory -> 28163875, usedMemory ->30890078
2015-03-23 05:40:11,364 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-5dca0952:13562 freed by fetcher#3 in 230ms
2015-03-23 05:40:11,364 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-02c90a0d:13562 with 1 to fetcher#3
2015-03-23 05:40:11,364 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-02c90a0d:13562 to fetcher#3
2015-03-23 05:40:11,367 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000016_0 sent hash and received reply
2015-03-23 05:40:11,371 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000016_0 decomp: 914778 len: 914782 to MEMORY
2015-03-23 05:40:11,374 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 909006 bytes from map-output for attempt_1427088391284_0001_m_000018_0
2015-03-23 05:40:11,374 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 909006, inMemoryMapOutputs.size() -> 33, commitMemory -> 29069747, usedMemory ->31804856
2015-03-23 05:40:11,375 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-ccca09c3:13562 freed by fetcher#4 in 153ms
2015-03-23 05:40:11,375 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b3ca09bc:13562 with 1 to fetcher#4
2015-03-23 05:40:11,375 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b3ca09bc:13562 to fetcher#4
2015-03-23 05:40:11,380 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000040_0 sent hash and received reply
2015-03-23 05:40:11,382 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000040_0 decomp: 903097 len: 903101 to MEMORY
2015-03-23 05:40:11,383 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000043_0 sent hash and received reply
2015-03-23 05:40:11,384 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000043_0 decomp: 904351 len: 904355 to MEMORY
2015-03-23 05:40:11,385 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000004_0 sent hash and received reply
2015-03-23 05:40:11,387 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000004_0 decomp: 905674 len: 905678 to MEMORY
2015-03-23 05:40:11,478 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911325 bytes from map-output for attempt_1427088391284_0001_m_000037_0
2015-03-23 05:40:11,482 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911325, inMemoryMapOutputs.size() -> 34, commitMemory -> 29978753, usedMemory ->34517978
2015-03-23 05:40:11,484 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-95ca099a:13562 freed by fetcher#2 in 157ms
2015-03-23 05:40:11,484 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-03c90a0c:13562 with 1 to fetcher#2
2015-03-23 05:40:11,484 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-03c90a0c:13562 to fetcher#2
2015-03-23 05:40:11,518 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 905674 bytes from map-output for attempt_1427088391284_0001_m_000004_0
2015-03-23 05:40:11,518 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 905674, inMemoryMapOutputs.size() -> 35, commitMemory -> 30890078, usedMemory ->34517978
2015-03-23 05:40:11,519 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-02c90a0d:13562 freed by fetcher#3 in 155ms
2015-03-23 05:40:11,519 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-d7ca09d8:13562 with 1 to fetcher#3
2015-03-23 05:40:11,519 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-d7ca09d8:13562 to fetcher#3
2015-03-23 05:40:11,521 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000022_0 sent hash and received reply
2015-03-23 05:40:11,522 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000022_0 decomp: 907710 len: 907714 to MEMORY
2015-03-23 05:40:11,531 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 903097 bytes from map-output for attempt_1427088391284_0001_m_000040_0
2015-03-23 05:40:11,531 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 903097, inMemoryMapOutputs.size() -> 36, commitMemory -> 31795752, usedMemory ->35425688
2015-03-23 05:40:11,531 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b3ca09bc:13562 freed by fetcher#4 in 156ms
2015-03-23 05:40:11,531 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-8aca0985:13562 with 1 to fetcher#4
2015-03-23 05:40:11,531 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-8aca0985:13562 to fetcher#4
2015-03-23 05:40:11,533 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000027_0 sent hash and received reply
2015-03-23 05:40:11,533 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000027_0 decomp: 910961 len: 910965 to MEMORY
2015-03-23 05:40:11,564 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 914778 bytes from map-output for attempt_1427088391284_0001_m_000016_0
2015-03-23 05:40:11,564 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 914778, inMemoryMapOutputs.size() -> 37, commitMemory -> 32698849, usedMemory ->36336649
2015-03-23 05:40:11,565 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-acca09a3:13562 freed by fetcher#5 in 235ms
2015-03-23 05:40:11,565 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-cfca09c0:13562 with 1 to fetcher#5
2015-03-23 05:40:11,565 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-cfca09c0:13562 to fetcher#5
2015-03-23 05:40:11,572 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000029_0 sent hash and received reply
2015-03-23 05:40:11,573 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000029_0 decomp: 910081 len: 910085 to MEMORY
2015-03-23 05:40:11,574 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 904351 bytes from map-output for attempt_1427088391284_0001_m_000043_0
2015-03-23 05:40:11,574 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 904351, inMemoryMapOutputs.size() -> 38, commitMemory -> 33613627, usedMemory ->37246730
2015-03-23 05:40:11,574 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f2ca09fd:13562 freed by fetcher#1 in 308ms
2015-03-23 05:40:11,575 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-7eca0971:13562 with 1 to fetcher#1
2015-03-23 05:40:11,575 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-7eca0971:13562 to fetcher#1
2015-03-23 05:40:11,609 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000039_0 sent hash and received reply
2015-03-23 05:40:11,609 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#4 about to shuffle output of map attempt_1427088391284_0001_m_000039_0 decomp: 899663 len: 899667 to MEMORY
2015-03-23 05:40:11,617 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910961 bytes from map-output for attempt_1427088391284_0001_m_000027_0
2015-03-23 05:40:11,618 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910961, inMemoryMapOutputs.size() -> 39, commitMemory -> 34517978, usedMemory ->38146393
2015-03-23 05:40:11,618 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-d7ca09d8:13562 freed by fetcher#3 in 99ms
2015-03-23 05:40:11,618 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-09c90a06:13562 with 1 to fetcher#3
2015-03-23 05:40:11,618 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-09c90a06:13562 to fetcher#3
2015-03-23 05:40:11,625 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910081 bytes from map-output for attempt_1427088391284_0001_m_000029_0
2015-03-23 05:40:11,625 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910081, inMemoryMapOutputs.size() -> 40, commitMemory -> 35428939, usedMemory ->38146393
2015-03-23 05:40:11,625 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-cfca09c0:13562 freed by fetcher#5 in 61ms
2015-03-23 05:40:11,625 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-f4ca09fb:13562 with 1 to fetcher#5
2015-03-23 05:40:11,625 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-f4ca09fb:13562 to fetcher#5
2015-03-23 05:40:11,628 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000024_0 sent hash and received reply
2015-03-23 05:40:11,629 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000024_0 decomp: 911061 len: 911065 to MEMORY
2015-03-23 05:40:11,657 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000014_0 sent hash and received reply
2015-03-23 05:40:11,658 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000014_0 decomp: 913424 len: 913428 to MEMORY
2015-03-23 05:40:11,662 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000047_0 sent hash and received reply
2015-03-23 05:40:11,663 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#3 about to shuffle output of map attempt_1427088391284_0001_m_000047_0 decomp: 895419 len: 895423 to MEMORY
2015-03-23 05:40:11,663 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 907710 bytes from map-output for attempt_1427088391284_0001_m_000022_0
2015-03-23 05:40:11,664 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 907710, inMemoryMapOutputs.size() -> 41, commitMemory -> 36339020, usedMemory ->40866297
2015-03-23 05:40:11,664 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-03c90a0c:13562 freed by fetcher#2 in 180ms
2015-03-23 05:40:11,664 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-00c90a0f:13562 with 1 to fetcher#2
2015-03-23 05:40:11,664 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-00c90a0f:13562 to fetcher#2
2015-03-23 05:40:11,717 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 913424 bytes from map-output for attempt_1427088391284_0001_m_000014_0
2015-03-23 05:40:11,717 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 913424, inMemoryMapOutputs.size() -> 42, commitMemory -> 37246730, usedMemory ->40866297
2015-03-23 05:40:11,718 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-f4ca09fb:13562 freed by fetcher#5 in 92ms
2015-03-23 05:40:11,718 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-b5ca09ba:13562 with 1 to fetcher#5
2015-03-23 05:40:11,718 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-b5ca09ba:13562 to fetcher#5
2015-03-23 05:40:11,728 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000026_0 sent hash and received reply
2015-03-23 05:40:11,730 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#5 about to shuffle output of map attempt_1427088391284_0001_m_000026_0 decomp: 910443 len: 910447 to MEMORY
2015-03-23 05:40:11,734 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911061 bytes from map-output for attempt_1427088391284_0001_m_000024_0
2015-03-23 05:40:11,734 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911061, inMemoryMapOutputs.size() -> 43, commitMemory -> 38160154, usedMemory ->41776740
2015-03-23 05:40:11,736 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-7eca0971:13562 freed by fetcher#1 in 162ms
2015-03-23 05:40:11,736 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning i-c5ca09ca:13562 with 1 to fetcher#1
2015-03-23 05:40:11,737 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of 1 to i-c5ca09ca:13562 to fetcher#1
2015-03-23 05:40:11,735 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 895419 bytes from map-output for attempt_1427088391284_0001_m_000047_0
2015-03-23 05:40:11,738 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 895419, inMemoryMapOutputs.size() -> 44, commitMemory -> 39071215, usedMemory ->41776740
2015-03-23 05:40:11,738 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-09c90a06:13562 freed by fetcher#3 in 120ms
2015-03-23 05:40:11,765 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 899663 bytes from map-output for attempt_1427088391284_0001_m_000039_0
2015-03-23 05:40:11,765 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 899663, inMemoryMapOutputs.size() -> 45, commitMemory -> 39966634, usedMemory ->41776740
2015-03-23 05:40:11,766 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-8aca0985:13562 freed by fetcher#4 in 235ms
2015-03-23 05:40:11,781 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000034_0 sent hash and received reply
2015-03-23 05:40:11,782 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1427088391284_0001_m_000034_0 decomp: 910079 len: 910083 to MEMORY
2015-03-23 05:40:11,788 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910443 bytes from map-output for attempt_1427088391284_0001_m_000026_0
2015-03-23 05:40:11,788 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910443, inMemoryMapOutputs.size() -> 46, commitMemory -> 40866297, usedMemory ->42686819
2015-03-23 05:40:11,789 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-b5ca09ba:13562 freed by fetcher#5 in 71ms
2015-03-23 05:40:11,790 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: for url=13562/mapOutput?job=job_1427088391284_0001&reduce=70&map=attempt_1427088391284_0001_m_000032_0 sent hash and received reply
2015-03-23 05:40:11,791 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 about to shuffle output of map attempt_1427088391284_0001_m_000032_0 decomp: 911003 len: 911007 to MEMORY
2015-03-23 05:40:11,810 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 911003 bytes from map-output for attempt_1427088391284_0001_m_000032_0
2015-03-23 05:40:11,811 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 911003, inMemoryMapOutputs.size() -> 47, commitMemory -> 41776740, usedMemory ->43597822
2015-03-23 05:40:11,811 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-00c90a0f:13562 freed by fetcher#2 in 147ms
2015-03-23 05:40:11,845 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 910079 bytes from map-output for attempt_1427088391284_0001_m_000034_0
2015-03-23 05:40:11,846 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 910079, inMemoryMapOutputs.size() -> 48, commitMemory -> 42687743, usedMemory ->43597822
2015-03-23 05:40:11,846 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2015-03-23 05:40:11,847 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: i-c5ca09ca:13562 freed by fetcher#1 in 111ms
2015-03-23 05:40:12,005 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 48 in-memory map-outputs and 0 on-disk map-outputs
2015-03-23 05:40:12,027 INFO [main] org.apache.hadoop.mapred.Merger: Merging 48 sorted segments
2015-03-23 05:40:12,055 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 48 segments left of total size: 43597342 bytes
2015-03-23 05:40:21,778 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 48 segments, 43597822 bytes to disk to satisfy reduce memory limit
2015-03-23 05:40:21,779 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 43597732 bytes from disk
2015-03-23 05:40:21,784 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2015-03-23 05:40:21,784 INFO [main] org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2015-03-23 05:40:21,822 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 43597718 bytes
2015-03-23 05:40:21,847 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-03-23 05:40:22,287 INFO [main] org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2015-03-23 05:40:22,289 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2015-03-23 05:40:36,870 INFO [main] HiBench.HiveData$GenerateRankingsReducer: pid: 70, 0 erros, 0 missed
2015-03-23 05:40:37,215 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1427088391284_0001_r_000070_0 is done. And is in the process of committing
2015-03-23 05:40:37,281 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1427088391284_0001_r_000070_0 is allowed to commit now
2015-03-23 05:40:37,292 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1427088391284_0001_r_000070_0' to hdfs://172.31.17.135:8120/HiBench/Hive/Input-comp/rankings/_temporary/1/task_1427088391284_0001_r_000070
2015-03-23 05:40:37,380 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1427088391284_0001_r_000070_0' done.
2015-03-23 05:40:37,384 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ReduceTask metrics system...
2015-03-23 05:40:37,386 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system stopped.
2015-03-23 05:40:37,387 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system shutdown complete.
