Commercialized transgenic traits, maize productivity and yield risk

111 1. Zerbino, D.r. et al. Science 336, 179–182 (2012). 2. Tan, T.W. et al. BMC Genomics 10 (suppl 3), S36 (2009). 3. Wollman, r. & Stuurman, N. J. Cell Sci. 120, 3715– 3722 (2007). 4. Joyce, A. & palsson, B. Nat. Rev. Mol. Cell Biol. 7, 198–210 (2006). 5. Arnaout, r. et al. PLoS ONE 6, e22365 (2011). 6. Brown, B., chui, M. & Manyika, J. McKinsey Q. 4, 24–35 (2011). 7. Taylor, c.r. Am. Econ. Rev. 85, 872–890 (1995). 8. Scotchmer, S. Innovation and Incentives (MIT press, cambridge, MA; 2004). 9. Levine, D.K. Science 323, 1296–1297 (2009). 10. Terwiesch, c. & Xu, Y. Manage. Sci. 54, 1529–1543 (2008). 11. Travis, J. Science 319, 1750–1752 (2008). 12. Howe, J. Crowdsourcing (crown Books, New York; 2008). 13. eiben, c.B. et al. Nat. Biotechnol. 30, 190–192 (2012). 14. Khatib, F. et al. Proc. Natl. Acad. Sci. USA 108, 18949–18953 (2011). 15. cooper, S. et al. Nature 466, 756–760 (2010). 16. Khatib, F. et al. Nat. Struct. Mol. Biol. 18, 1175–1177 (2011). 17. Scudellari, M. Nat. Med. 18, 326 (2012). 18. Jeppesen, L. & Lakhani, K.r. Organ. Sci. 21, 1016– 1033 (2010). 19. Boudreau, K.J., Lacetera, N. & Lakhani, K.r. Manage. Sci. 57, 843–863 (2011). 20. Marbach, D. et al. Nat. Methods 9, 796–804 (2012). 21. Boyd, S.D. et al. Sci. Transl. Med. 1, 12ra23 (2009). 22. Weinstein, J.A. et al. Science 324, 807–810 (2009). 23. robins, H.S. et al. Sci. Transl. Med. 2, 47ra64 (2010). 24. Jung, D. et al. Annu. Rev. Immunol. 24, 541–570 (2006). 25. Altschul, S.F. et al. J. Mol. Biol. 215, 403–410 (1990). 26. Brochet, X. et al. Nucleic Acids Res. 36, W503–W508 (2008). 27. Hong, L. & page, S. Proc. Natl. Acad. Sci. USA 101, 16385–16389 (2004). and collected data. R.A.A. identified and codeveloped the immunogenomics problem, tested the submissions and helped write the manuscript. M.L. and L.B. codeveloped the problem statement and test data. P.-R.L. analyzed and categorized all submission data and helped write the manuscript.

Although the solvers were virtually devoid of domain-specific knowledge, abstracting the problem into general algorithmic and mathematical terms allowed a wide range of nondomain experts to address an important, complex problem. These contestants brought to the problem whatever skills and expertise they had or could find, probably yielding a far more diverse toolkit than would be available locally, and generated substantial diversity in technical approaches. Accessing such diversity may be particularly important, as big-data biomedical analytics is a rapidly evolving field in which it is difficult to know a priori the kind, quality and breadth of expertise needed to produce an effective solution.
In summary, we show that a prizebased contest on a commercial platform can effectively recruit skilled individuals to apply their knowledge to a big-data biomedical problem. Deconstruction and transformation of problems for a heterogeneous solver community coupled with adequate data to produce and validate results can support solution diversity and minimize the risk of suboptimal solutions that may arise from limited searches. In addition to the benefits of generating new knowledge, this strategy may be particularly useful in situations where the computational or algorithmic problem, or potentially any science problem, represents a barrier to rapid progress but where finding the solution is not itself the major thrust of the investigator's scientific effort. The America Competes Act passed by the US Congress provides funding agencies with the authority to administer their own prize-based contests and paves the way for establishing how grant recipients might access commercial prize platforms to accelerate their own research.

ACKNOWLEDGMENTS
Harvard Business School's Division of Research and Faculty Development funded the prize money and supported K.R.L., E.L., C.B. and A.M. K.R.L. and K.J.B. were also supported by the NASA Tournament Lab, which is funded by the NASA Human Exploration and Operations Mission Directorate. E.C.G. was supported by the Harvard Clinical and Translational Science Center (NCRR 5UL1RR025758 and 5UL1RR025758-S), and R.A.A. was supported by the Klarman Family Foundation. P.-R.L. was supported by US National Defense Science and Engineeering and US National Science Foundation graduate research fellowships.

AuThOr CONTribuTiONS
K.R.L., K.J.B. and E.C.G. conceived, designed and executed the experiment, analyzed data and wrote the manuscript. C.B. and A.M. contributed to experimental design. E.L. executed the experiment To the Editor: Maize expressing different versions of Bacillus thuringiensis toxins (Bt), 5 enolphyruvylshikimate-3 phosphate synthase (EPSPS) and phosphinothricin acetyl transferase alone or in combination are part of the current wave of agricultural technological change. We analyzed grain yield data from annual field experiments during 1990-2010 in Wisconsin to test hypotheses that maize expressing these transgenic traits alone or in combination (stacked) has greater productivity (as measured by the mean harvested yield) and lower production risk (as measured by the variance, skewness and kurtosis of harvested yield). Compared with conventional hybrids, the impact of transgenic traits (both single and stacked traits) on mean yield ranges from -12.2 to +6.5 bushels per acre. This shows that reducing yield risk is an important source of benefits of transgenic technology, especially for stacked traits. These benefits are estimated to be equivalent to a yield increase of 0.8-4.2 bushels per acre. We found evidence for gene interactions ('yield drag' and 'event lag' effects) that can reduce yield.
The past century has seen marked increases in maize productivity. Average US maize yields increased from 72 to 153 bushels per acre from 1970 to 2010 (ref. 1). Genetic selection has contributed to advances in maize productivity in recent decades 2,3 . Over the past 15 years, productivity gains have been complemented by rapid adoption of transgenic hybrids in the United States (and elsewhere) 2,3 . Rapid adoption of transgenic maize by US farmers suggests that the technology benefits them. Yet documenting the nature and sources of these benefits has been challenging 4,5 . There is some evidence of delayed yield increase due to 'yield lag' and yield drag associated with transgenes 5 . Agricultural production is also subject to substantial risk from unpredictable weather and pest damage. Transgenic crops have been argued to help reduce agricultural production risk, thus motivating insurance companies to At planting time, weather and pest damages are typically unknown and generate production uncertainty among farmers. Maize yield varies with managerial inputs (including both conventional genetic materials and transgenic materials) and other noncontrollable inputs (such as weather effects).
Transgenic technology provides new opportunities for advancing agricultural productivity and enhancing food security around the world [6][7][8][9] . Our objective was to measure the effect of transgenic traits that confer pest and/or herbicide resistance on grain yield and risk by analyzing yield data from field experiments over the period 1990-2010 in Wisconsin. We document how patented transgenes affect maize grain yield and its variability. We examined two hypotheses: (i) transgene interactions affect maize productivity and (ii) transgenic technology lowers grain yield risk, as measured by the variance, skewness and kurtosis of maize yield. In general, reducing variance and increasing skewness are seen as desirable: they lead to lower risk exposure (from a lower variance) and lower exposure to unfavorable events located in the lower tail of the yield distribution (as captured by a higher skewness). Decreasing kurtosis may be desirable to the extent that it means a lower exposure to rare events in the tails of the yield distribution. Our analysis documents the extent to which transgenes contribute to reducing yield loss and risk exposure (data are described in Box 1).
The estimation of mean yield shows that, over the past two decades, maize yield has increased on average by 1.81 bushels per acre per year (Supplementary Table 1). As we expected, there are substantial maize productivity differences across sites, reflecting the effects of agroclimatic conditions. In general, maize yield decreases northward owing to the effects of a shorter growing season.
The effects of transgenic hybrids on yield vary with the type of transgenic traits ( Table 1). For single-transgenic-trait hybrids compared with conventional seeds, average yield is greater by 6.54 bushels per acre for maize expressing Bt against European corn borer (Ostrinia nubilalis; ECB) and by 5.76 for maize expressing phosphinothricin acetyl transferase conferring glufosinate tolerance (GFT). But average yield is lower by 5.98 bushels per acre for EPSPS glyphosate tolerant (GT) maize and by 12.22 bushels per acre for maize expressing single-trait Bt against corn rootworm (Diabrotica sp.; CRW). Several stacked hybrids show no statistical difference in mean yield compared with conventional hybrids: ECB-CRW, GT-CRW, ECB-GT-GFT, ECB-CRW-GFT and ECB-GT-CRW-GFT. However, the other stacked hybrids have significant effects on mean yield: ECB-GT (+3.47 bushels per acre), ECB-GFT (+3.13 bushels per acre) and ECB-CRW-GT (-1.57 bushels per acre).
Overall, the ECB trait has the most favorable effect on mean yield: +6.54 bushels per acre for ECB, +3.47 bushels per acre for ECB-GT and +3.13 bushels per acre for ECB-GFT. ECB gene technology effectively controls the European corn borer and limits its adverse effects on maize yield. Yet, with the exception of the ECB trait, we were surprised not to find strongly positive transgenic yield effects. First, this could be explained in part by yield losses located in the lower tail of the distribution (downside risk). The kurtosis measures the thickness in the tails of the distribution of maize yield, which is relevant to the extent that the yield distribution departs from a normal distribution (which has 'thin tails').
We first specified and estimated a regression model for mean yield as a function of location and managerial inputs. The error term was then used to estimate the variance, skewness and kurtosis by regressing the square, cube and fourth power of the error term, respectively, against location and managerial inputs [10][11][12] . The results provide a basis for evaluating productivity effects of genetically engineered genes as they affect maize yield under risk. Details on the statistical approach are in Supplementary Methods.
In general, the mean, variance, skewness and kurtosis of yield vary with management choices, including transgenes. We examined the presence of possible interaction effects among genes, including interactions between transgenes and the basic germplasm, and interactions among stacked transgenes. Transgenes can also affect variance, skewness and kurtosis of yield. Noting that both germplasm and transgenes change over time, our analysis focused on comparing yield performance across hybrid types: conventional, single transgenic traits and stacked transgenic traits. explanatory variables include transgenes, location, time and other managerial factors. The location variable captures site-specific factors affecting maize yield (for example, soil, weather and test site management). Statistical estimates are in Supplementary Table 1. The transgenic variables are dummy variables set equal to 1 when a given transgene is present, and set to zero otherwise. For conventional hybrids, all transgenic dummy variables are set equal to zero. In the presence of multiple transgenes, stacking dummies are set equal to 1 when a particular set of stacked genes are present and set to zero otherwise.

Box 1 Data used in analysis
c o r r e S p o N D e N c e npg event lag effect was first negative, but then turned positive (after 7 years for GT and after 10 years for ECB). This indicates that the effects of transgenes on maize yield depend on interactions between transgenes and the underlying germplasm. We interpret our finding as indirect evidence that plant breeders try to minimize adverse interaction effects between transgenes and the germplasm over time. Yet such gene interactions vary with each transgene. For ECB-CRW, GT-CRW, ECB-GFT and ECB-GFT-CRW hybrids, the event lag effects are negative and significant. In contrast, for ECB-CRW-GT and the ECB-GT-CRW-GFT hybrids, the event lag effects are positive and significant. And for single-GFT, ECB-GT and ECB-GT-GFT hybrids, there is no statistical evidence of event lag effects. In these cases, interaction effects between transgenes and the underlying germplasm seem negligible.
Next, we examined the effects of transgenic technology on production risk for maize. Transgenic crops have been argued to lower yield loss and reduce risk exposure 5 (for example, by reducing risk of yield damage from pest infestation). Our analysis shows that transgenic hybrids have significant effects on yield risk (Supplementary Table 1). Transgenic technology affects the variance of maize yield. The variance of transgenic seeds is always lower than that of conventional seeds ( Table 1). This suggests that transgenic hybrids tend to reduce yield risk, which is consistent with earlier findings 12 . Even so, these effects vary across transgenic hybrids. Stacked transgenes tend to be more effective in reducing variance. For example, the reduction in variance is much larger for ECB-CRW than for ECB or CRW. There seems to be substantial synergy among transgenes in reducing risk exposure. This finding indicates that in certain cases transgenic technology can help farmers reduce their risk exposure. In particular, transgenic technology provides new options for mitigating the effects of increased weather uncertainty (for example, owing to climate change).
For event lag effects, the longer the GT trait has been introduced, the lower the yield variance, whereas the effect is opposite for the ECB trait (Supplementary Table 1). However, the quadratic term time lag effect for the ECB trait is negative, showing that variance will eventually decrease for that type of hybrid. The variance-reducing effects of the ECB trait seem stronger in the longer term. The event lag effect on yield variance for the ECB-CRW stacked event is also negative.
The skewness and kurtosis of maize yield show that transgenic traits can affect the production and selection of better cultivars by geneticists and plant breeders.
Lower performance may also be caused by a time lag in the development of hybrids and evolving progress in refining the effectiveness of transgenic technologies (Supplementary Table 1). These lags are measured by the number of years since the first introduction of a given event for each specific trait or system of traits (stacks). We evaluated the effect of these lags on maize yield and call them event lag effects. Such effects may develop if transgenes interact with genetic material in the germplasm. Again, gene interactions can be reduced by replacement of low-yielding cultivars with better ones by geneticists and plant breeders. Event lag effects may develop depending on where transgenes are inserted in the germplasm, the quality of the germplasm and the success of transgene transfer. For example, improved production of transgenic cultivars may generate positive event lag effects if transgenes and the basic seed germplasm complement each other over time.
We found evidence of event lag effects, although they vary by trait and by stacking. Nonlinear lag effects were found for events in the market for many years. Five trait(s) systems have been in the market for >10 years (Supplementary Table 1): GT, ECB, GFT, ECB-GT and ECB-GFT. A quadratic time lag term (in addition to a linear term) was included for these transgenic trait(s) systems. For example, for GT and ECB hybrids, the drag effects, which may occur when there are negative interaction effects among transgenes (as discussed above). Second, our analysis focuses only on yield effects, and neglects possible benefits of transgenic technology associated with reduced input use (for example, pesticides and tillage).
We also analyzed the joint effects of stacked transgenes on mean yield. Without gene interactions, the effect of stacked genes would be equal to the sum of the corresponding single gene effects. Alternatively, with gene interactions, the effect of stacked genes would differ from the sum of single gene effects. We found strong evidence of gene interactions among transgenic traits when they are stacked. We found negative and significant interaction effects for four different stacking schemes: ECB-GT; ECB-GFT; ECB-GT-GFT; and ECB-GT-CRW-GFT (Supplementary Table 1). For these stacking schemes, the mean yield increase from stacked seeds is smaller than the sum of the mean yield increases obtained from corresponding single genes ( Table 1). Positive and significant interaction effects were found when ECB and CRW were stacked. This indicates that Bt transgenes interact with each other as they affect maize yield. Although the identification of gene interactions in maize is not new 10,11 , the evidence of negative interaction effects among transgenes suggests that transgenic hybrids can perform more poorly than conventional hybrids. Yet, such gene interactions can be reduced over time by  Table 1 and were evaluated using an arbitrary scenario of maize grown in 2005 in Lancaster, Wisconsin, without irrigation or insecticide application, and with no tillage practice in either fall or spring. All other relevant variables were set at the mean or median. results comparing yield differences across hybrid types were not affected by these choices, because our analysis was based on pooled data across all sites and was done by evaluating yield differences across hybrid types, controlling for location and time. The standard errors used for the significance tests were bootstrapped. *, P < 0.1; **, P < 0.05; ***, P < 0.01 (t-test). b For transgenic hybrids, number of observations, mean yield, variance in yield, skewness of yield and kurtosis of yield are expressed as difference between that of transgenic variety and that of the conventional benchmark. c See Table 2 for a more intuitive evaluation of estimated effects on variance, skewness and kurtosis. c o r r e S p o N D e N c e npg hybrids ( Table 2). These effects come from reductions in all three components of the cost of risk, but the reduction in variance dominates that of skewness and kurtosis. This indicates that Bt and EPSPS transgenic hybrids help reduce farmers' exposure to risk. However, these effects vary with transgenic hybrids. Stacked transgenes reduce the cost of risk more than single transgenes ( Table 2). Given the current move toward widespread use of stacked genes, transgenic technology is becoming more effective in reducing the cost of risk exposure in maize production. Transgenic maize can generate sizable risk benefits. For example, compared with conventional hybrids, ECB-CRW stacked genes reduce the cost of risk by an equivalent yield increase of 4.19 bushels per acre ( Table 2). For ECB-CRW-GFT and ECB-CRW-GT, the reduction is 3.23 and 3.06 bushels per acre, respectively. And for ECB-GT, GT-CRW and ECB-GT-CRW-GFT, the reduction is ~2.4 bushels per acre. Some of these estimates are larger than the corresponding estimates related to mean effects ( Table 1).
In conclusion, our results show how transgenic technology can improve farmers' ability to deal with a risky environment. The availability of this technology seems important given current concerns about the effects of climate change on production uncertainty in agriculture.

ACKNOWLEDGMENTS
This work was support in part by a US Department of Agriculture Hatch grant and a US Department of Agriculture AFRI grant.

AuThOr CONTribuTiONS
G.S. and J.C. designed the study and analyzed the data. J.L. conducted the field tests, collected, processed and distribution of maize yield (beyond their effects on variance). We need to go beyond mean and variance in the analysis of yield risk. Yield skewness varies with seed types. It is negative and statistically significant for conventional seeds. In this case, yield distribution is skewed to the left (indicating a greater exposure to losses and downside risk). This shows that maize yield is not normally distributed [13][14][15][16] . Transgenes affect yield skewness and can reduce exposure to downside risk (by increasing skewness compared with conventional hybrids; Table 1). These effects are notable for GT, CRW, ECB-CRW, ECB-GT, GT-CRW and ECB-CRW-GT. Thus in certain cases, transgenic technology can contribute to reducing exposure to downside risk, indicating that transgenic technology can help farmers reduce their exposure to unfavorable events and mitigate the effects of adverse weather shocks.
Transgenic hybrids have lower kurtosis compared with conventional hybrids ( Table 1); therefore transgenic technology also contributes to reducing rare events in the tails of the maize yield distribution. This feature is also desirable. Although climate change may increase the likelihood of these rare events, transgenic technology can help mitigate their effects on maize yield.
How important are the effects of genetically engineered genes on yield risk? To evaluate such effects, we measure the cost of risk (called the risk premium in economics; Supplementary Methods). The cost of risk is defined as the number of bushels of maize per acre a farmer is willing to give up to replace a risky yield with mean yield. The cost of risk depends on the farmer's degree of risk aversion. We considered a scenario in which the farmer exhibits constant relative risk aversion, with a relative risk aversion parameter equal to 3 (moderate risk aversion). The estimated costs of risk associated with different transgenic hybrids are reported in Table 2. The total cost of risk is decomposed into three components: variance, skewness and kurtosis.
In general, the total cost of risk is 2-4% of expected yield. We analyzed the extent of risk exposure in maize production. First, most of the cost of risk comes from the variance component. For example, the total cost of risk for conventional hybrids is 6.36 bushels per acre; the variance component accounts for 90% of it (5.72 bushels per acre) and the skewness and kurtosis components account for about 5% each.
Second, all transgenic hybrids decrease the cost of risk compared with conventional