MBPP+ coverage
============================================================

EvalPlus package version (pip show inside container): 0.4.0.dev2
EvalPlus Docker image tag: ganler/evalplus:v0.3.1
Docker image digest: sha256:26b118098bef281fe8dfe999bf05f1d5b45374b4e6c00161ec0f30592aef4740
MBPP+ dataset version: v0.2.0
MBPP+ tasks total: 378
Original study tasks: MBPP IDs 11-510 (500 tasks)
Intersection size: 224

Note: EvalPlus MBPP+ uses the sanitized MBPP dataset. Its base tests therefore do not necessarily reproduce the classifications obtained from the original full MBPP test_list used in the study.

Intersection task IDs:
11,12,14,16,17,18,19,20,56,57,58,59,61,62,63,64,65,66,67,68,69,70,71,72,74,75,77,79,80,82,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,108,109,111,113,116,118,119,120,123,124,125,126,127,128,129,130,131,132,133,135,137,138,139,140,141,142,145,160,161,162,165,166,167,168,170,171,172,222,223,224,226,227,230,232,233,234,235,237,238,239,240,242,244,245,247,250,251,252,253,255,256,257,259,260,261,262,264,265,266,267,268,269,270,271,272,273,274,276,277,278,279,280,281,282,283,284,285,286,287,290,292,293,294,296,297,299,300,301,305,306,308,309,310,311,312,388,389,390,391,392,394,395,397,398,404,405,406,409,410,412,413,414,415,418,419,420,421,422,424,425,426,427,428,429,430,432,433,435,436,437,439,440,441,445,446,447,448,450,451,453,454,455,456,457,458,459,460,462,463,465,468,470,471,472,473,474,475,476,477,478,479
