/gpfs/projects/LynchGroup/spacewhale/git_spacewhale/spacewhale/shell_scripts
/gpfs/projects/LynchGroup/spacewhale
Sun Jul  7 22:04:58 EDT 2019
## NOW TRAINING RESNET-34, LR=0.0008
######################################################################################################
WELCOME TO SPACEWHALE!
######################################################################################################
We will now train your model.. please be patient
Using resnet32 Your trained model will be named resnet32_full32_lr0008
------------------------------------------------------------------------------
<torch.utils.data.dataloader.DataLoader object at 0x2aaafcd13a20>
Your dataset size is: 12545
You have 2 classes in your dataset
------------------------------------------------------------------------------
Labels for the dataset are:
water = 0
whale = 1
------------------------------------------------------------------------------
Data loaded into gpu
------------------------------------------------------------------------------
Epoch 0/23
----------
train Loss: 0.5572 Acc: 0.7107 Err: 0.2892
TP: 4919.0000  TN: 3997.0000  FP: 2266.0000  FN: 1362.0000
Epoch 1/23
----------
train Loss: 0.4382 Acc: 0.7956 Err: 0.2043
TP: 5574.0000  TN: 4407.0000  FP: 1865.0000  FN: 698.0000
Epoch 2/23
----------
train Loss: 0.4071 Acc: 0.8147 Err: 0.1852
TP: 5643.0000  TN: 4578.0000  FP: 1754.0000  FN: 569.0000
Epoch 3/23
----------
train Loss: 0.3775 Acc: 0.8274 Err: 0.1725
TP: 5847.0000  TN: 4533.0000  FP: 1696.0000  FN: 468.0000
Epoch 4/23
----------
train Loss: 0.3629 Acc: 0.8383 Err: 0.1616
TP: 5776.0000  TN: 4741.0000  FP: 1595.0000  FN: 432.0000
Epoch 5/23
----------
train Loss: 0.3656 Acc: 0.8410 Err: 0.1589
TP: 6006.0000  TN: 4544.0000  FP: 1601.0000  FN: 393.0000
Epoch 6/23
----------
train Loss: 0.3499 Acc: 0.8469 Err: 0.1530
TP: 5972.0000  TN: 4652.0000  FP: 1563.0000  FN: 357.0000
Epoch 7/23
----------
train Loss: 0.3399 Acc: 0.8502 Err: 0.1497
TP: 5848.0000  TN: 4818.0000  FP: 1571.0000  FN: 307.0000
Epoch 8/23
----------
train Loss: 0.3303 Acc: 0.8551 Err: 0.1448
TP: 5945.0000  TN: 4782.0000  FP: 1524.0000  FN: 293.0000
Epoch 9/23
----------
train Loss: 0.3314 Acc: 0.8555 Err: 0.1444
TP: 5983.0000  TN: 4749.0000  FP: 1506.0000  FN: 306.0000
Epoch 10/23
----------
train Loss: 0.3377 Acc: 0.8497 Err: 0.1503
TP: 5915.0000  TN: 4744.0000  FP: 1547.0000  FN: 338.0000
Epoch 11/23
----------
train Loss: 0.3234 Acc: 0.8551 Err: 0.1448
TP: 5918.0000  TN: 4809.0000  FP: 1504.0000  FN: 313.0000
Epoch 12/23
----------
train Loss: 0.3226 Acc: 0.8583 Err: 0.1416
TP: 5971.0000  TN: 4797.0000  FP: 1473.0000  FN: 303.0000
Epoch 13/23
----------
train Loss: 0.3299 Acc: 0.8507 Err: 0.1492
TP: 5933.0000  TN: 4739.0000  FP: 1556.0000  FN: 316.0000
Epoch 14/23
----------
train Loss: 0.3296 Acc: 0.8568 Err: 0.1431
TP: 6029.0000  TN: 4720.0000  FP: 1513.0000  FN: 282.0000
Epoch 15/23
----------
train Loss: 0.3184 Acc: 0.8603 Err: 0.1397
TP: 5994.0000  TN: 4798.0000  FP: 1468.0000  FN: 284.0000
Epoch 16/23
----------
train Loss: 0.3206 Acc: 0.8574 Err: 0.1425
TP: 5990.0000  TN: 4766.0000  FP: 1479.0000  FN: 309.0000
Epoch 17/23
----------
train Loss: 0.3119 Acc: 0.8617 Err: 0.1382
TP: 6008.0000  TN: 4802.0000  FP: 1441.0000  FN: 293.0000
Epoch 18/23
----------
train Loss: 0.3148 Acc: 0.8619 Err: 0.1381
TP: 5999.0000  TN: 4813.0000  FP: 1456.0000  FN: 276.0000
Epoch 19/23
----------
train Loss: 0.3148 Acc: 0.8628 Err: 0.1371
TP: 5955.0000  TN: 4869.0000  FP: 1458.0000  FN: 262.0000
Epoch 20/23
----------
train Loss: 0.3211 Acc: 0.8568 Err: 0.1432
TP: 6002.0000  TN: 4746.0000  FP: 1456.0000  FN: 340.0000
Epoch 21/23
----------
train Loss: 0.3082 Acc: 0.8657 Err: 0.1342
TP: 6099.0000  TN: 4761.0000  FP: 1391.0000  FN: 293.0000
Epoch 22/23
----------
train Loss: 0.3104 Acc: 0.8645 Err: 0.1354
TP: 5974.0000  TN: 4871.0000  FP: 1436.0000  FN: 263.0000
Epoch 23/23
----------
train Loss: 0.3185 Acc: 0.8578 Err: 0.1421
TP: 5949.0000  TN: 4812.0000  FP: 1470.0000  FN: 313.0000
-----------------------------------------------------------
Training complete in 77m 20s
-----------------------------------------------------------
Sun Jul  7 23:22:38 EDT 2019
## Now training ResNet-34, LR= 0.0007
######################################################################################################
WELCOME TO SPACEWHALE!
######################################################################################################
We will now train your model.. please be patient
Using resnet32 Your trained model will be named resnet32_full32_lr0007
------------------------------------------------------------------------------
<torch.utils.data.dataloader.DataLoader object at 0x2aaafcd0fa58>
Your dataset size is: 12545
You have 2 classes in your dataset
------------------------------------------------------------------------------
Labels for the dataset are:
water = 0
whale = 1
------------------------------------------------------------------------------
Data loaded into gpu
------------------------------------------------------------------------------
Epoch 0/23
----------
train Loss: 0.5621 Acc: 0.7026 Err: 0.2973
TP: 4857.0000  TN: 3957.0000  FP: 2298.0000  FN: 1432.0000
Epoch 1/23
----------
train Loss: 0.4375 Acc: 0.7993 Err: 0.2006
TP: 5635.0000  TN: 4392.0000  FP: 1801.0000  FN: 716.0000
Epoch 2/23
----------
train Loss: 0.4041 Acc: 0.8141 Err: 0.1858
TP: 5633.0000  TN: 4580.0000  FP: 1722.0000  FN: 609.0000
Epoch 3/23
----------
train Loss: 0.3919 Acc: 0.8245 Err: 0.1754
TP: 5818.0000  TN: 4525.0000  FP: 1713.0000  FN: 488.0000
Epoch 4/23
----------
train Loss: 0.3710 Acc: 0.8361 Err: 0.1638
TP: 5834.0000  TN: 4655.0000  FP: 1638.0000  FN: 417.0000
Epoch 5/23
----------
train Loss: 0.3608 Acc: 0.8403 Err: 0.1596
TP: 5869.0000  TN: 4673.0000  FP: 1613.0000  FN: 389.0000
Epoch 6/23
----------
train Loss: 0.3466 Acc: 0.8498 Err: 0.1501
TP: 5961.0000  TN: 4700.0000  FP: 1574.0000  FN: 309.0000
Epoch 7/23
----------
train Loss: 0.3493 Acc: 0.8423 Err: 0.1576
TP: 5948.0000  TN: 4619.0000  FP: 1648.0000  FN: 329.0000
Epoch 8/23
----------
train Loss: 0.3269 Acc: 0.8589 Err: 0.1410
TP: 6037.0000  TN: 4738.0000  FP: 1507.0000  FN: 262.0000
Epoch 9/23
----------
train Loss: 0.3281 Acc: 0.8548 Err: 0.1451
TP: 5983.0000  TN: 4741.0000  FP: 1528.0000  FN: 292.0000
Epoch 10/23
----------
train Loss: 0.3186 Acc: 0.8611 Err: 0.1389
TP: 6071.0000  TN: 4731.0000  FP: 1473.0000  FN: 269.0000
Epoch 11/23
----------
train Loss: 0.3248 Acc: 0.8570 Err: 0.1429
TP: 5961.0000  TN: 4790.0000  FP: 1497.0000  FN: 296.0000
Epoch 12/23
----------
train Loss: 0.3350 Acc: 0.8555 Err: 0.1444
TP: 6060.0000  TN: 4672.0000  FP: 1498.0000  FN: 314.0000
Epoch 13/23
----------
train Loss: 0.3197 Acc: 0.8608 Err: 0.1391
TP: 5989.0000  TN: 4810.0000  FP: 1484.0000  FN: 261.0000
Epoch 14/23
----------
train Loss: 0.3231 Acc: 0.8571 Err: 0.1428
TP: 5960.0000  TN: 4792.0000  FP: 1491.0000  FN: 301.0000
Epoch 15/23
----------
train Loss: 0.3335 Acc: 0.8506 Err: 0.1493
TP: 5992.0000  TN: 4679.0000  FP: 1541.0000  FN: 332.0000
Epoch 16/23
----------
train Loss: 0.3252 Acc: 0.8569 Err: 0.1430
TP: 5951.0000  TN: 4799.0000  FP: 1492.0000  FN: 302.0000
Epoch 17/23
----------
train Loss: 0.3185 Acc: 0.8633 Err: 0.1366
TP: 6067.0000  TN: 4763.0000  FP: 1438.0000  FN: 276.0000
Epoch 18/23
----------
train Loss: 0.3258 Acc: 0.8568 Err: 0.1432
TP: 6026.0000  TN: 4722.0000  FP: 1487.0000  FN: 309.0000
Epoch 19/23
----------
train Loss: 0.3089 Acc: 0.8640 Err: 0.1359
TP: 6012.0000  TN: 4827.0000  FP: 1446.0000  FN: 259.0000
Epoch 20/23
----------
train Loss: 0.3193 Acc: 0.8606 Err: 0.1393
TP: 6065.0000  TN: 4731.0000  FP: 1477.0000  FN: 271.0000
Epoch 21/23
----------
train Loss: 0.3136 Acc: 0.8613 Err: 0.1386
TP: 6027.0000  TN: 4778.0000  FP: 1453.0000  FN: 286.0000
Epoch 22/23
----------
train Loss: 0.3216 Acc: 0.8590 Err: 0.1409
TP: 6019.0000  TN: 4757.0000  FP: 1501.0000  FN: 267.0000
Epoch 23/23
----------
train Loss: 0.3178 Acc: 0.8613 Err: 0.1386
TP: 6032.0000  TN: 4773.0000  FP: 1420.0000  FN: 319.0000
-----------------------------------------------------------
Training complete in 77m 20s
-----------------------------------------------------------
Mon Jul  8 00:40:14 EDT 2019
## Now training ResNet-34, LR= 0.0006
######################################################################################################
WELCOME TO SPACEWHALE!
######################################################################################################
We will now train your model.. please be patient
Using resnet32 Your trained model will be named resnet32_full32_lr0006
------------------------------------------------------------------------------
<torch.utils.data.dataloader.DataLoader object at 0x2aaafcd0fac8>
Your dataset size is: 12545
You have 2 classes in your dataset
------------------------------------------------------------------------------
Labels for the dataset are:
water = 0
whale = 1
------------------------------------------------------------------------------
Data loaded into gpu
------------------------------------------------------------------------------
Epoch 0/23
----------
train Loss: 0.5728 Acc: 0.6937 Err: 0.3062
TP: 4843.0000  TN: 3860.0000  FP: 2355.0000  FN: 1486.0000
Epoch 1/23
----------
train Loss: 0.4581 Acc: 0.7831 Err: 0.2168
TP: 5437.0000  TN: 4387.0000  FP: 1898.0000  FN: 822.0000
Epoch 2/23
----------
train Loss: 0.4198 Acc: 0.8045 Err: 0.1955
TP: 5606.0000  TN: 4486.0000  FP: 1811.0000  FN: 641.0000
Epoch 3/23
----------
train Loss: 0.3958 Acc: 0.8230 Err: 0.1769
TP: 5815.0000  TN: 4510.0000  FP: 1711.0000  FN: 508.0000
Epoch 4/23
----------
train Loss: 0.3850 Acc: 0.8304 Err: 0.1695
TP: 5858.0000  TN: 4559.0000  FP: 1659.0000  FN: 468.0000
Epoch 5/23
----------
train Loss: 0.3736 Acc: 0.8340 Err: 0.1659
TP: 5834.0000  TN: 4629.0000  FP: 1666.0000  FN: 415.0000
Epoch 6/23
----------
train Loss: 0.3621 Acc: 0.8389 Err: 0.1610
TP: 5815.0000  TN: 4709.0000  FP: 1603.0000  FN: 417.0000
Epoch 7/23
----------
train Loss: 0.3453 Acc: 0.8478 Err: 0.1521
TP: 6001.0000  TN: 4635.0000  FP: 1586.0000  FN: 322.0000
Epoch 8/23
----------
train Loss: 0.3302 Acc: 0.8582 Err: 0.1417
TP: 5984.0000  TN: 4782.0000  FP: 1472.0000  FN: 306.0000
Epoch 9/23
----------
train Loss: 0.3331 Acc: 0.8521 Err: 0.1479
TP: 5927.0000  TN: 4762.0000  FP: 1525.0000  FN: 330.0000
Epoch 10/23
----------
train Loss: 0.3340 Acc: 0.8536 Err: 0.1464
TP: 5914.0000  TN: 4794.0000  FP: 1499.0000  FN: 337.0000
Epoch 11/23
----------
train Loss: 0.3363 Acc: 0.8502 Err: 0.1497
TP: 5893.0000  TN: 4773.0000  FP: 1532.0000  FN: 346.0000
Epoch 12/23
----------
train Loss: 0.3294 Acc: 0.8528 Err: 0.1472
TP: 5930.0000  TN: 4768.0000  FP: 1513.0000  FN: 333.0000
Epoch 13/23
----------
train Loss: 0.3323 Acc: 0.8534 Err: 0.1465
TP: 6065.0000  TN: 4641.0000  FP: 1525.0000  FN: 313.0000
Epoch 14/23
----------
train Loss: 0.3247 Acc: 0.8583 Err: 0.1417
TP: 5948.0000  TN: 4819.0000  FP: 1487.0000  FN: 290.0000
Epoch 15/23
----------
train Loss: 0.3157 Acc: 0.8588 Err: 0.1411
TP: 5888.0000  TN: 4886.0000  FP: 1509.0000  FN: 261.0000
Epoch 16/23
----------
train Loss: 0.3276 Acc: 0.8564 Err: 0.1435
TP: 5937.0000  TN: 4807.0000  FP: 1485.0000  FN: 315.0000
Epoch 17/23
----------
train Loss: 0.3199 Acc: 0.8619 Err: 0.1381
TP: 5937.0000  TN: 4875.0000  FP: 1444.0000  FN: 288.0000
Epoch 18/23
----------
train Loss: 0.3213 Acc: 0.8612 Err: 0.1387
TP: 5939.0000  TN: 4865.0000  FP: 1442.0000  FN: 298.0000
Epoch 19/23
----------
train Loss: 0.3212 Acc: 0.8593 Err: 0.1406
TP: 6017.0000  TN: 4763.0000  FP: 1457.0000  FN: 307.0000
Epoch 20/23
----------
train Loss: 0.3309 Acc: 0.8524 Err: 0.1475
TP: 5917.0000  TN: 4776.0000  FP: 1546.0000  FN: 305.0000
Epoch 21/23
----------
train Loss: 0.3245 Acc: 0.8595 Err: 0.1404
TP: 5957.0000  TN: 4826.0000  FP: 1462.0000  FN: 299.0000
Epoch 22/23
----------
train Loss: 0.3153 Acc: 0.8627 Err: 0.1373
TP: 5977.0000  TN: 4845.0000  FP: 1423.0000  FN: 299.0000
Epoch 23/23
----------
train Loss: 0.3195 Acc: 0.8611 Err: 0.1389
TP: 6037.0000  TN: 4765.0000  FP: 1417.0000  FN: 325.0000
-----------------------------------------------------------
Training complete in 77m 8s
-----------------------------------------------------------
Mon Jul  8 01:57:43 EDT 2019
