Repo: fairseq
Filename: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/projects/fairseq/tests/test_multi_corpus_sampled_dataset.py
ClassName: TestMultiCorpusSampledDataset
Testname: test_multi_corpus_sampled_dataset_weighted_sample
Params: num_epochs,47,20,ParamType.ITER,1000
Assertion: self.assertLess(abs(sample_from_first_ds_percentage - expected_sample_from_first_ds_percentage),0.01,)
Original runtime: 1.524
>>Getting original runtime
Optimization Iteration: 1
Running with params: {'num_epochs': 1000}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_1000
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 28...
Iter 29...
Timings: Avg: 1.954, Max: 2.0, Min: 1.91
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.954, Max: 2.0, Min: 1.91
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.954
Score: 1.954
Best-score: 1.954
Best-param: {'num_epochs': 1000}
>>Setting original runtime to 1.954
Optimization Iteration: 1
Running with params: {'num_epochs': 990}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_990
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 18...
Iter 19...
Iter 17...
Iter 20...
Iter 22...
Iter 21...
Iter 23...
Iter 24...
Iter 25...
Iter 27...
Iter 26...
Iter 29...
Iter 28...
Timings: Avg: 1.9416666666666667, Max: 1.9800000000000002, Min: 1.8599999999999999
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.9416666666666667, Max: 1.9800000000000002, Min: 1.8599999999999999
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.9416666666666667
Score: 1.9416666666666667
Best-score: 1.9416666666666667
Best-param: {'num_epochs': 990}
Optimization Iteration: 2
Running with params: {'num_epochs': 380}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_380
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Timings: Avg: 1.7226666666666663, Max: 1.7799999999999998, Min: 1.6400000000000001
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.7226666666666663, Max: 1.7799999999999998, Min: 1.6400000000000001
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.7226666666666663
Score: 1.7226666666666663
Best-score: 1.7226666666666663
Best-param: {'num_epochs': 380}
Optimization Iteration: 3
Running with params: {'num_epochs': 280}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_280
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Timings: Avg: 1.6656666666666666, Max: 1.71, Min: 1.56
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.6656666666666666, Max: 1.71, Min: 1.56
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.6656666666666666
Score: 1.6656666666666666
Best-score: 1.6656666666666666
Best-param: {'num_epochs': 280}
Optimization Iteration: 4
Running with params: {'num_epochs': 140}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_140
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 29...
Iter 27...
Iter 28...
Timings: Avg: 1.7026666666666668, Max: 1.7599999999999998, Min: 1.4900000000000002
Passed tests : 0
Failed tests : 30
Half of samples failed, exiting...
All-Passed: False
Probabilty of failure: 1.0
Runtime: 10000.0
Score: 3.4028234663852886e+38
Best-score: 1.6656666666666666
Best-param: {'num_epochs': 280}
Optimization Iteration: 5
Running with params: {'num_epochs': 150}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_150
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Timings: Avg: 1.6223333333333336, Max: 1.6600000000000001, Min: 1.54
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.6223333333333336, Max: 1.6600000000000001, Min: 1.54
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.6223333333333336
Score: 1.6223333333333336
Best-score: 1.6223333333333336
Best-param: {'num_epochs': 150}
Optimization Iteration: 6
Running with params: {'num_epochs': 370}
Higher iteration... returning...
Optimization Iteration: 7
Running with params: {'num_epochs': 130}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_130
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 25...
Iter 24...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Timings: Avg: 1.6039999999999996, Max: 1.65, Min: 1.54
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.6039999999999996, Max: 1.65, Min: 1.54
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.6039999999999996
Score: 1.6039999999999996
Best-score: 1.6039999999999996
Best-param: {'num_epochs': 130}
Optimization Iteration: 8
Running with params: {'num_epochs': 260}
Higher iteration... returning...
Optimization Iteration: 9
Running with params: {'num_epochs': 970}
Higher iteration... returning...
Optimization Iteration: 10
Running with params: {'num_epochs': 290}
Higher iteration... returning...
Optimization Iteration: 11
Running with params: {'num_epochs': 260}
Higher iteration... returning...
Optimization Iteration: 12
Running with params: {'num_epochs': 120}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_120
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Timings: Avg: 1.6126666666666665, Max: 1.65, Min: 1.46
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.6126666666666665, Max: 1.65, Min: 1.46
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.6126666666666665
Score: 1.6126666666666665
Best-score: 1.6039999999999996
Best-param: {'num_epochs': 130}
Optimization Iteration: 13
Running with params: {'num_epochs': 300}
Higher iteration... returning...
Optimization Iteration: 14
Running with params: {'num_epochs': 160}
Higher iteration... returning...
Optimization Iteration: 15
Running with params: {'num_epochs': 220}
Higher iteration... returning...
Optimization Iteration: 16
Running with params: {'num_epochs': 170}
Higher iteration... returning...
Optimization Iteration: 17
Running with params: {'num_epochs': 150}
Optimization Iteration: 18
Running with params: {'num_epochs': 730}
Higher iteration... returning...
Optimization Iteration: 19
Running with params: {'num_epochs': 400}
Higher iteration... returning...
Optimization Iteration: 20
Running with params: {'num_epochs': 270}
Higher iteration... returning...
Optimization Iteration: 21
Running with params: {'num_epochs': 100}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_100
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Timings: Avg: 1.7016666666666669, Max: 1.74, Min: 1.5799999999999998
Passed tests : 0
Failed tests : 30
Half of samples failed, exiting...
All-Passed: False
Probabilty of failure: 1.0
Runtime: 10000.0
Score: 3.4028234663852886e+38
Best-score: 1.6039999999999996
Best-param: {'num_epochs': 130}
Optimization Iteration: 22
Running with params: {'num_epochs': 100}
Optimization Iteration: 23
Running with params: {'num_epochs': 120}
Optimization Iteration: 24
Running with params: {'num_epochs': 200}
Higher iteration... returning...
Optimization Iteration: 25
Running with params: {'num_epochs': 120}
Optimization Iteration: 26
Running with params: {'num_epochs': 500}
Higher iteration... returning...
Optimization Iteration: 27
Running with params: {'num_epochs': 120}
Optimization Iteration: 28
Running with params: {'num_epochs': 200}
Higher iteration... returning...
Optimization Iteration: 29
Running with params: {'num_epochs': 110}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614559/assert_54582980_110
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 28...
Iter 29...
Timings: Avg: 1.5946666666666667, Max: 1.63, Min: 1.5
Passed tests : 30
Failed tests : 0
Converged: True
Convergence score: 0.0
Evaluating 30 values out of 30
Overall-timings: Avg: 1.5946666666666667, Max: 1.63, Min: 1.5
Variance (0.0) too small, using delta distribution
Probability of fail : 0.0
All-Passed: True
Probabilty of failure: 0.0
Runtime: 1.5946666666666667
Score: 1.5946666666666667
Best-score: 1.5946666666666667
Best-param: {'num_epochs': 110}
Optimization Iteration: 30
Running with params: {'num_epochs': 180}
Higher iteration... returning...
Optimization Iteration: 31
Running with params: {'num_epochs': 110}
Optimization Iteration: 32
Running with params: {'num_epochs': 100}
Optimization Iteration: 33
Running with params: {'num_epochs': 550}
Higher iteration... returning...
Optimization Iteration: 34
Running with params: {'num_epochs': 210}
Higher iteration... returning...
Optimization Iteration: 35
Running with params: {'num_epochs': 980}
Higher iteration... returning...
Optimization Iteration: 36
Running with params: {'num_epochs': 360}
Higher iteration... returning...
Optimization Iteration: 37
Running with params: {'num_epochs': 140}
Optimization Iteration: 38
Running with params: {'num_epochs': 110}
Optimization Iteration: 39
Running with params: {'num_epochs': 240}
Higher iteration... returning...
Optimization Iteration: 40
Running with params: {'num_epochs': 780}
Higher iteration... returning...
Optimization Iteration: 41
Running with params: {'num_epochs': 180}
Higher iteration... returning...
Optimization Iteration: 42
Running with params: {'num_epochs': 480}
Higher iteration... returning...
Optimization Iteration: 43
Running with params: {'num_epochs': 140}
Optimization Iteration: 44
Running with params: {'num_epochs': 110}
Optimization Iteration: 45
Running with params: {'num_epochs': 320}
Higher iteration... returning...
Optimization Iteration: 46
Running with params: {'num_epochs': 240}
Higher iteration... returning...
Optimization Iteration: 47
Running with params: {'num_epochs': 150}
Optimization Iteration: 48
Running with params: {'num_epochs': 130}
Optimization Iteration: 49
Running with params: {'num_epochs': 640}
Higher iteration... returning...
Optimization Iteration: 50
Running with params: {'num_epochs': 170}
Higher iteration... returning...
Upto iteration 50: {'num_epochs': 110.0}
Optimization Iteration: 51
Running with params: {'num_epochs': 130}
Optimization Iteration: 52
Running with params: {'num_epochs': 330}
Higher iteration... returning...
Optimization Iteration: 53
Running with params: {'num_epochs': 420}
Higher iteration... returning...
Optimization Iteration: 54
Running with params: {'num_epochs': 110}
Optimization Iteration: 55
Running with params: {'num_epochs': 880}
Higher iteration... returning...
Optimization Iteration: 56
Running with params: {'num_epochs': 160}
Higher iteration... returning...
Optimization Iteration: 57
Running with params: {'num_epochs': 250}
Higher iteration... returning...
Optimization Iteration: 58
Running with params: {'num_epochs': 190}
Higher iteration... returning...
Optimization Iteration: 59
Running with params: {'num_epochs': 220}
Higher iteration... returning...
Optimization Iteration: 60
Running with params: {'num_epochs': 290}
Higher iteration... returning...
Optimization Iteration: 61
Running with params: {'num_epochs': 100}
Optimization Iteration: 62
Running with params: {'num_epochs': 160}
Higher iteration... returning...
Optimization Iteration: 63
Running with params: {'num_epochs': 110}
Optimization Iteration: 64
Running with params: {'num_epochs': 130}
Optimization Iteration: 65
Running with params: {'num_epochs': 440}
Higher iteration... returning...
Optimization Iteration: 66
Running with params: {'num_epochs': 110}
Optimization Iteration: 67
Running with params: {'num_epochs': 140}
Optimization Iteration: 68
Running with params: {'num_epochs': 120}
Optimization Iteration: 69
Running with params: {'num_epochs': 110}
Optimization Iteration: 70
Running with params: {'num_epochs': 150}
Optimization Iteration: 71
Running with params: {'num_epochs': 100}
Optimization Iteration: 72
Running with params: {'num_epochs': 170}
Higher iteration... returning...
Optimization Iteration: 73
Running with params: {'num_epochs': 120}
Optimization Iteration: 74
Running with params: {'num_epochs': 220}
Higher iteration... returning...
Optimization Iteration: 75
Running with params: {'num_epochs': 100}
Optimization Iteration: 76
Running with params: {'num_epochs': 130}
Optimization Iteration: 77
Running with params: {'num_epochs': 350}
Higher iteration... returning...
Optimization Iteration: 78
Running with params: {'num_epochs': 110}
Optimization Iteration: 79
Running with params: {'num_epochs': 190}
Higher iteration... returning...
Optimization Iteration: 80
Running with params: {'num_epochs': 150}
Optimization Iteration: 81
Running with params: {'num_epochs': 160}
Higher iteration... returning...
Optimization Iteration: 82
Running with params: {'num_epochs': 630}
Higher iteration... returning...
Optimization Iteration: 83
Running with params: {'num_epochs': 140}
Optimization Iteration: 84
Running with params: {'num_epochs': 110}
Optimization Iteration: 85
Running with params: {'num_epochs': 120}
Optimization Iteration: 86
Running with params: {'num_epochs': 100}
Optimization Iteration: 87
Running with params: {'num_epochs': 200}
Higher iteration... returning...
Optimization Iteration: 88
Running with params: {'num_epochs': 270}
Higher iteration... returning...
Optimization Iteration: 89
Running with params: {'num_epochs': 180}
Higher iteration... returning...
Optimization Iteration: 90
Running with params: {'num_epochs': 130}
Optimization Iteration: 91
Running with params: {'num_epochs': 240}
Higher iteration... returning...
Optimization Iteration: 92
Running with params: {'num_epochs': 140}
Optimization Iteration: 93
Running with params: {'num_epochs': 170}
Higher iteration... returning...
Optimization Iteration: 94
Running with params: {'num_epochs': 300}
Higher iteration... returning...
Optimization Iteration: 95
Running with params: {'num_epochs': 220}
Higher iteration... returning...
Optimization Iteration: 96
Running with params: {'num_epochs': 150}
Optimization Iteration: 97
Running with params: {'num_epochs': 380}
Optimization Iteration: 98
Running with params: {'num_epochs': 210}
Higher iteration... returning...
Optimization Iteration: 99
Running with params: {'num_epochs': 100}
Optimization Iteration: 100
Running with params: {'num_epochs': 550}
Higher iteration... returning...
Upto iteration 100: {'num_epochs': 110.0}
Breaking...
{'num_epochs': 110.0}
Best score: 1.5946666666666667
Repeated params: 38
Trials: 101
Best param {'num_epochs': 110.0}
Reduction: 18.389628113271918%
Speedup: 1.225334448160535x
Optimizer time: 19.869420289993286
