Repo: fairseq
Filename: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/projects/fairseq/tests/test_lstm_jitable.py
ClassName: TestJitLSTMModel
Testname: test_assert_jit_vs_nonjit_
Params: iter,93,15,ParamType.ITER,100
Assertion: self.assertTensorEqual(result[1], scripted_result[1])
Original runtime: 6.827
>>Getting original runtime
Optimization Iteration: 1
Running with params: {'iter': 100}
Logdir: /mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/logs/optim_1597614526_fairseq/run_1597614586/assert_46289221_100
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.203333333333332, Max: 7.35, Min: 7.01
Passed tests : 30
Failed tests : 0
Converged: False
Convergence score: 27.355559036282546
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 15...
Iter 13...
Iter 16...
Iter 14...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 7...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.209666666666667, Max: 7.35, Min: 7.01
Passed tests : 60
Failed tests : 0
Converged: False
Convergence score: 24.908885250258415
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 19...
Iter 18...
Iter 20...
Iter 21...
Iter 24...
Iter 22...
Iter 25...
Iter 23...
Iter 26...
Iter 27...
Iter 28...
Iter 29...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.216777777777777, Max: 7.35, Min: 7.01
Passed tests : 90
Failed tests : 0
Converged: False
Convergence score: 26.02520180280191
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.2210833333333335, Max: 7.3500000000000005, Min: 7.01
Passed tests : 120
Failed tests : 0
Converged: False
Convergence score: 23.51001234795318
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 22...
Iter 20...
Iter 21...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 28...
Iter 29...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.237133333333334, Max: 7.4399999999999995, Min: 6.98
Passed tests : 150
Failed tests : 0
Converged: False
Convergence score: 13.1872988780116
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 9...
Iter 8...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.235222222222221, Max: 7.4399999999999995, Min: 6.98
Passed tests : 180
Failed tests : 0
Converged: False
Convergence score: 19.559704585695847
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 28...
Iter 29...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.234523809523809, Max: 7.4399999999999995, Min: 6.98
Passed tests : 210
Failed tests : 0
Converged: False
Convergence score: 15.059355409459275
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 24...
Iter 22...
Iter 25...
Iter 23...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.229791666666666, Max: 7.4399999999999995, Min: 6.97
Passed tests : 240
Failed tests : 0
Converged: False
Convergence score: 12.581427032131083
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 28...
Iter 29...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.235111111111111, Max: 7.4399999999999995, Min: 6.97
Passed tests : 270
Failed tests : 0
Converged: False
Convergence score: 17.441103582648495
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.245133333333333, Max: 7.44, Min: 6.97
Passed tests : 300
Failed tests : 0
Converged: False
Convergence score: 24.620041986086186
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 27...
Iter 25...
Iter 29...
Iter 28...
Iter 26...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.2464545454545455, Max: 7.44, Min: 6.97
Passed tests : 330
Failed tests : 0
Converged: False
Convergence score: 28.07500812974786
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 26...
Iter 25...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.245416666666667, Max: 7.44, Min: 6.88
Passed tests : 360
Failed tests : 0
Converged: False
Convergence score: 22.853659386072014
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.246230769230769, Max: 7.44, Min: 6.88
Passed tests : 390
Failed tests : 0
Converged: False
Convergence score: 19.345606395292652
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.242357142857143, Max: 7.44, Min: 6.88
Passed tests : 420
Failed tests : 0
Converged: False
Convergence score: 19.569689969075633
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.240866666666667, Max: 7.44, Min: 6.88
Passed tests : 450
Failed tests : 0
Converged: False
Convergence score: 19.97264112522853
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 25...
Iter 22...
Iter 26...
Iter 23...
Iter 24...
Iter 29...
Iter 28...
Iter 27...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.240083333333333, Max: 7.44, Min: 6.73
Passed tests : 480
Failed tests : 0
Converged: False
Convergence score: 19.97264112522853
Continuing to next batch...
Launching 30 jobs, 30 in parallel
Iter 0...
Iter 1...
Iter 2...
Iter 3...
Iter 4...
Iter 5...
Iter 6...
Iter 7...
Iter 8...
Iter 9...
Iter 10...
Iter 11...
Iter 12...
Iter 13...
Iter 14...
Iter 15...
Iter 16...
Iter 17...
Iter 18...
Iter 19...
Iter 20...
Iter 21...
Iter 22...
Iter 23...
Iter 24...
Iter 25...
Iter 26...
Iter 27...
Iter 29...
Iter 28...
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Not found ::: 
Timings: Avg: 7.237215686274509, Max: 7.44, Min: 6.73
Passed tests : 510
Failed tests : 0
Converged: False
Convergence score: 18.797158109959252
Continuing to next batch...
Evaluating 286 values out of 510
Overall-timings: Avg: 7.237215686274509, Max: 7.44, Min: 6.73
Exception>>>
Traceback (most recent call last):
  File "optimizer.py", line 59, in run_optimizer
    enable_opt=True, enable_timeout=False)
  File "/mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/src/strategy/bayesopt.py", line 30, in __init__
    self.get_original_runtime()
  File "/mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/src/strategy/bayesopt.py", line 126, in get_original_runtime
    result = self.compute_score(p_dict)
  File "/mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/src/strategy/bayesopt.py", line 66, in compute_score
    output = self.runner.run_with_param(params, self.best_runtime if self.enable_timeout and self.best_runtime < np.inf else None, self.best_runtime)
  File "/mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/src/runner/StocRunner.py", line 207, in run_with_param
    prob=compareDist.evaluate()
  File "/mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/CompareDistribution.py", line 217, in evaluate
    p = self.check_assert_equal(actual, expected)
  File "/mnt/batch/tasks/workitems/opt1/job-1/Task_fairseq/wd/borntobeflaky/tool/CompareDistribution.py", line 253, in check_assert_equal
    if not np.equal(np.ndarray.flatten(np.array(actual[i])), np.ndarray.flatten(np.array(expected[i]))).all():
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Optimizer time: 163.8667995929718
