CASE TEACHING NOTES
for
"Experimental Design and Statistical Analysis:
Bt Corn, Lignin, and ANOVAs"

by
Eric Ribbens
Department of Biological Sciences
Western Illinois University
European corn borer

INTRODUCTION / BACKGROUND

This case was designed as an assignment for a senior-level plant ecology course and a graduate level biostatistics course.  It could be used in other types of courses, such as plant anatomy, plant physiology, soil ecology, agriculture, or genetics.  The case is based on a paper about the lignin content of genetically modified corn published by Saxena and Stotzky in the September 2001 issue of the American Journal of Botany.  The article is available to subscribers at http://www.amjbot.org/ and there is also an option to purchase a single online "reprint."  Most students find the subject interesting and relevant, and discussing the article provides students with experience in examining experimental design and interpretation as well as viewing statistics as a tool used in experimental design and analysis.  The case requires one to two hours of class time, and can be used to produce a written assignment from each student.

Objectives

My primary objectives when I teach this case are for the students to:

In addition, this case could be used to support a number of secondary objectives, including:  the environmental consequences of the widespread use of Bt-corn; the ethics of using genetically transformed organisms when all effects are not understood; lignin as an important chemical in plant-animal interactions and in soil ecology; and genetics, particularly the development of genetic transformations.

This case fits nicely within my goals for two courses I teach.  In plant ecology, an upper-level undergraduate course, I want my students to think about the kinds of questions that plant ecologists ask, to gain experience in designing, analyzing, and reporting ecological investigations, and to develop skills of evaluating original research publications.  I have discovered that many students are reluctant to challenge what they see in print and, unless they are carefully guided through the process of evaluating these papers, tend to read them too superficially.  In my biostatistics course, which is required for all of our master's degree students, one of my goals is to have students understand that statistical analysis is simply a tool to aid in the process of conducting research.  I want them to discover that there should be a natural interplay between the design and the analysis of research projects.

I present this case midway through my plant ecology course, mostly because in the earlier weeks I want to maximize our use of course time to get outside while plants are still active and easily studied.  In plant ecology this is the second of four cases that I use, and we have already extensively discussed experimental design and the importance of a focused research question in the context of our lab experiences.  In my biostatistics course, I use this case after I have taught the concept of two-way and three-way ANOVAs.  Then, when given the data, the students are required to reanalyze the experiment and write a paper presenting their conclusions.  In plant ecology, while we have already used t-tests and calculated means and standard deviations, I focus less on the statistical analysis and more on the overall experimental design.  In both courses I want my students to see how a research paper reflects the process of doing science and to realize that they can intelligently read original research documents and ask legitimate questions about the manuscript.

In both courses I begin by making it clear that this paper is not selected as a particularly good or bad example of research; rather, it is an interesting recent paper that is accessible to graduate and advanced undergraduate readers.  I also remind them that there are gaps in the information reported, so if in answering one of my questions they include a statement that they need more information, that is not necessarily wrong.  Finally, I tell them that this paper raises important societal issues.  In the United States alone, Bt-modified corn is already widely planted—20 million acres in 1999 according to the U.S. Environmental Protection Agency.

CLASSROOM MANAGEMENT / BLOCKS OF ANALYSIS

This case is designed so that students receive the material sequentially in four parts.  I hand out Part I, which consists of the abstract of the paper and questions 1-5, a few days before we discuss the case in class and ask the students to write out answers to the questions.  Alternatively, Part I could be given to students at the beginning of the class.  I begin the class by discussing the questions, with a particular emphasis on questions 4 and 5.  Students are then given Part II, which consists of a description of methods and questions 6-9, and an in-class discussion of this second set of questions ensues.  Part III, consisting of the graphs I have created and questions 10 and 11a or 11b, should follow.  Then, after discussing the graphical results, students receive Tables 1 and 2 and questions 12-18, which make up Part IV.  I usually hand out Part IV at the end of the first class (50 minutes) and then spend the next class discussing questions 12-18.  There are several options for student assignments after all of the questions have been discussed.

Because my courses tend to be small (six to 10 students), I use the discussion method and have found that our discussions work well if we simply follow the questions attached at the end of each part of the case.  In a larger course some of these questions could effectively be discussed in small groups.  Not all of the questions are equally important or equally appropriate to all courses and teachers should modify or skip questions as needed.  For example, questions 13-16 may expect more statistical skills than many undergraduate students have, and Question 16 is specifically designed for my biostatistics course.  Furthermore, this case can be extended in many different directions.  There are important ecological questions about using Bt-corn that can be evaluated.  A plant anatomy or physiology course could examine the structure of lignin more comprehensively and obtain corn stalks for a lab exercise in which they use the same techniques to assess the location and concentration of lignin.  Similarly, this case could be a springboard for courses in genetics, agriculture, plant-animal interactions, or soil ecology.

Because this case is designed for advanced courses and contains some rather technical questions (e.g., what is SEM and why is it an inappropriate statistic to use), effectively teaching the case requires careful thought on the part of the instructor to avoid frustrating the students unduly.  As an advocate of student-centered instruction, I prefer to have students find the answers on their own, but of course that means the students must be given more time to work on the case.

With these notes I have included two sets of references, one on Bt and one on lignin, which I hand out to the students with Part I of the case.  While the focus of this case is not specifically on either topic, students often feel more comfortable with the case if they know more about these two important underlying components, and these handouts help them to answer questions 1 and 2.

Questions 3-11 do not require any external material to answer, but students will handle them most effectively if the concepts of a research question and an experimental design have already been discussed.  Surprisingly, many of my seniors and graduate students have had little exposure to these aspects of the scientific method, perhaps because most lab experiences move briskly into the details of conducting the lab.

Questions 12-14 and 16 are more statistically technical.  My biostatistics students have been prepared to handle these questions, but my plant ecology students have not.  In that class I lead the students through the questions much more intensively.  For example, question 12 asks about SEM.  In biostatistics, if my students cannot answer this question, I suggest that they pull out their copy of Zar and find out what the formula for SEM is.  In plant ecology, I will write the formula on the board.  I need to ask some probing questions of the students in both courses to get them to understand the importance of sample size in this formula, but once they understand that the value of the formula shifts as sample size increases even if the standard deviation is a constant, they are then ready to explore why this is undesirable.

Question 15 can be answered by both sets of students, but my graduate students are able to explore this question more comprehensively.

Question 17 requires some botanical background.  Usually I have several botanically oriented students in class, so I can ask them to explain the importance of vascular bundles for photosynthesis in C4 plants.  The botanists should also be familiar with the concept that larger diameter xylem strands are more likely to cavitate, but may need me to ask about the differences between spring and summer wood before they recall this.

Question 18 is perhaps the most difficult because it is so broad.  Depending on the course, I often assign it as a follow-up writing topic, in which case I expect students will spend some time reading about lignin, Bt, and genetic engineering.  Usually a reminder that I will not accept vague unsupported generalizations is sufficient.

Below I highlight in greater detail important components that should come out in response to the questions posed in the case.

1.  What does the Bt in Bt-corn represent and why was it used to genetically modify corn plants?

Bt refers to a set of toxic proteins, called Bt-delta endotoxins, which are produced by the bacteria Bacillus thuringiensisBt has been widely used in organic gardening as a "natural" pesticide and one of the concerns about using Bt-engineered corn is that the widespread use of plants expressing these proteins (in 1999, 26% of U.S. corn fields were planted in Bt-engineered varieties) is likely to result in some insect pests evolving resistance to these toxins.  The ability of corn borers to develop such a resistance has already been demonstrated.  Alstad et al. (1997) found that after seven to nine generations of selection for Bt-resistance in European corn borers raised in a laboratory, they evolved a moderate resistance to the toxin, needing up to 60% more toxin to achieve a 50% mortality rate.
Several different seed companies have developed Bt-modified hybrids.  According to Ric Bessin ( http://www.uky.edu/Agriculture/Entomology/entfacts/fldcrops/ef118.htm), "Each of the transgenic events listed includes the insertion of a Bt gene, a promoter gene, and a marker gene (to allow corn breeders to know which plants have the new genetic material).  The promoter gene allows the Bt gene to be turned on and different promoter genes may allow the Bt toxin to be expressed at different times of the year or different parts of the plant.  The promoter used with Event 176 is different from the promoter used with the Bt 11 and MON 810 events.  Differences in insertion packages and insertion events translate into real differences in corn borer control in the field.  For example, the 176 event (Knockout and NatureGard) is designed to have maximum effectiveness against first generation corn borers and [is designed to] not have the Bt endotoxin expressed in the grain, while the YieldGard is designed to provide high levels of control throughout the plant and is effective against first and second generation corn borers."  Furthermore, note that Saxena and Stotzky only examined varieties engineered to produce the cry1A(b) protein; other seed companies, such as DeKalb, have used other Bt-proteins.
It is important to note that when Bt-engineered corn varieties are produced, more genetic material is inserted than just the gene to express the endotoxin.  You may want to ask students what the relationship is between Bt and lignin; the answer is that we don't know.  As Ric Bessin points out (see the previous paragraph), a promoter sequence is also inserted which determines when the gene is expressed, a marker gene to permit easy identification by geneticists is inserted, and there may be sequences that cause the gene to be rapidly replicated in a bacteria before it is transferred into the corn plant.
Note:  Below is a list of resources containing information about Bt that might be useful either for course preparation or as a handout for students to enable them to more efficiently discover the answers themselves.

2.  What is lignin?  Why is it an important plant chemical?

Lignin is an important chemical secreted by plant cells into cell walls, especially secondary cell walls.  It adds strength to the cell walls and is not permeable to water.  Once constructed, it is extremely difficult to break down because only a few species of fungi possess the necessary enzymes to degrade lignin.  Thus, plant tissue with lignin in it tends to be sturdier and more resistant to decay than tissue without lignin.
The presence or absence of lignin can be determined by staining thin sections, and there are well-developed laboratory protocols for determining the concentration of lignin in plant tissue.  Saxena and Stotzky (2001) wrote:  "Uniform free-hand sections of fresh corn stems between the 3rd and 4th node from the surface of the soil (thickness was [~] 11 mm for plants grown in the plant growth room and [~] 18 mm for field-grown plants) were examined for lignin by fluorescence microscopy at 400 nm (Hu et al., 1999) and by staining with 0.01% toluidine blue (Sylvester and Ruzin, 1994).  The content of lignin of the same portion of the stems (oven-dried, ground, and passed through an 80-mesh sieve) was determined by the acetyl bromide method (Hatfield et al., 1999)."
Note:  Below is a list of resources containing information about lignin that might be useful either for course preparation or as a handout for students to enable them to more efficiently discover the answers themselves.

3.  What was their research question?

This is a deceptively simple question.  Students are likely to overgeneralize, giving answers such as "They wanted to know whether Bt-modified corn is safe."  Based only on the abstract, a better answer would be "They wanted to determine whether the lignin content of all varieties of Bt-corn is higher than it is in the non-Bt-corn isolines." You may need to discuss the difference between the implications of research and the actual question that a research project addresses.
Students are not likely to know that an isoline refers to the non-transformed hybrid used in the transformation (iso = same, line refers to the genetic lineage).  Some students may have only a vague understanding of the concept of a genetic transformation, and often they have exaggerated notions of what can and cannot be done in genetic transformations.  You may want to cover this with Ric Bessin's description (given above).

4.  Based on the abstract above, reconstruct their experimental approach.  What did they do?  How did they do it?  What did they compare, and how do those comparisons help them answer their research question?

This question is extremely important.  Depending on course needs, you may prefer to ask students to develop a written analysis of this and the next question before beginning this discussion.  It should be thoroughly discussed in class.  I suggest that you begin by reading through each sentence in the abstract, making sure that everyone knows what is meant by terms such as transformed or isolines.  Then list on the chalkboard all of the pieces of information we are given that are relevant to methods (i.e., 10 varieties, growth room and field tests, etc.).  Then ask students to articulate a methodology or have them work on a methods section in small groups.  Do not expect that everyone will agree on one common methodology!  It can be very interesting to examine the diversity of ideas people have about a research project's methods.  In my opinion, the role of the instructor at this point in the case is to focus the discussion on the abstract—what do we know from the abstract?
From the abstract, students should be able to state that the authors looked for lignin and performed chemical assays to calculate the actual concentration of lignin in 10 corn hybrids with three different types of genetically engineered Bt expression.  The statement that lignin was significantly higher in the engineered corn implies that they tested comparable varieties of non-Bt corn as well.  You will probably need to explain what isolines are and why using isolines is an important piece of the experimental design.  Finally, students should be able to determine that corn plants were grown in two different environments, the growth room and in a field trial.
End this part of the discussion by examining the connection between their methods and their research question.  How does each piece of what they did contribute to answering the question?

5.  What questions do you have about their research protocol that the abstract does not answer?

It is equally fascinating to identify missing information.  For example, were the growth-room pots all the same size?  Were the different varieties planted in the field randomly located, or planted in distinct clumps or rows?  What can be reasonably inferred?  What important pieces of information are missing?  It may be useful to discuss what the point of an abstract is, its summary nature, to reduce student frustration about the "missing" pieces of knowledge.

6.  Based on the abstract above and on this description of the methods, modify your reconstruction of their experimental approach.  What did they do?  How did they do it?

This question is as important as questions 4 and 5, but should not be considered until those questions have been thoroughly explored.  I like to emphasize that they modify their reconstruction in order to emphasize the connection between the abstract and the rest of the paper, but an alternative approach is to ask them to reconstruct the experiment based only on the methods and then compare their two reconstructions.
The methods section provides several important missing pieces of information, such as the information that the growth room was kept at 26 degrees C with a 12-hour daylight cycle.  Students should realize that the experiments are not exactly equivalent because 10 varieties of corn were tested in the growth room but only eight varieties were tested in the field.  We also learn from the methods that field-grown plants were irrigated but not fertilized, that the sections analyzed were taken by freehand from the same node in each stem, and that the field specimens produced thicker sections.  We discover that during the chemical analysis of lignin the samples were oven-dried, ground, and run through a sieve.  You may want to spend some time discussing why it is important to remove water and whether or not the sieving of the specimen could be a source of bias or error.  Normally plant ecologists use dry weight (the weight after water is removed) because the amount of water can vary substantially and because water yields no nutritional benefit.  Sieving might be a source of error if it means that larger particles were removed from the analysis.  However, probably what occurred was that sieving was used to ensure that all particles were small (i.e., particles that did not fit through the sieve were reground) in order to maximize the consistency of the chemical analysis.

7.  What questions do you have about their research protocol that the abstract and methods sections do not answer?

Equally interesting are all of the pieces of information that we are not given.  Were the pots and plants in the field randomly located or grouped by variety?  How big is the sample size?  Does lignin content vary from location to location in the stem, and if so, should that be incorporated into the experimental design?  Why did they sample after 90 days?  How big were the pots in the greenhouse?  Did they fertilize?  When there were three stems per pot, did they sample each stem?
In my opinion, it is also worth pointing out that the authors may be overstating their results in the abstract.  Recall that in the abstract they wrote "Chemical analysis confirmed that the lignin content of all hybrids of Bt corn, whether grown in a plant growth room or in the field, was significantly higher (33-97% higher) than that of their respective non-Bt isolines."  However, they never tested the lignin content of the entire plant; rather, they tested the lignin content of the stem between the 3rd and 4th node and are inferring that the concentrations found there are representative of concentrations throughout the entire plant.  Furthermore, they are generalizing from a sample taken at one point during the life cycle of the plant.  These may seem to be minor points, and the inferences are of course reasonable, but the fact remains that at this point their data show only that after a certain number of days in the two environments in which they grew corn the lignin concentration in the stem between the 3rd and 4th node was higher.  Generalizations to the entire plant, all growing conditions, all soil types, etc., are not only reasonable but an essential component of science, but we must always remember the difference between evidence and inference.  Covering this point teaches your students to think critically, which is an essential component of scientific philosophy.
As the instructor, you must be sensitive to the biases of your class.  Some students, for example, are excessively reluctant to challenge or question any aspect of the research design while others seem to believe that they could have done it much better.  Occasionally my graduate students try to build grandiose multifactorial experimental designs in which they plan to simultaneously manipulate soil, water, fertilizer, European corn borers, and take repeated samples from multiple locations while still maintaining high sample sizes.  If this happens it is worth discussing the importance of setting boundaries so that the project can be completed and the realities that limited funds impose on research.  It is important that you have established a level of trust between you and your students so that they do not feel they must try to identify the magic answer that makes you happy.

8.  Does it matter that the average thickness of the stem sections differed between the growth room and the field-grown plants?  Should lignin content be reported as an absolute value, or as a percentage of the total sampled biomass?

Probably it doesn't matter much that the field-grown sections are thicker.  However, it is critical that the lignin content be reported as a percentage of the total biomass because the research question here is not whether a larger piece of tissue has more lignin, but if the plant changes the amount of lignin it is manufacturing to support a given amount of tissue.

9.  Why did they test differences in lignin content in both the growth room and in the field?  Why might the two experiments yield different results?  What can they conclude about lignin based on performing both of these experiments that they could not conclude with a growth room test alone, or with a field test alone?

To the extent that the patterns are the same, we can conclude that any differences in the production of lignin between modified and non-modified corn are likely to occur in a wider variety of environmental conditions.  For example, environmental conditions in the field are much more varied than conditions in the lab; if lignin production is in part driven by responses to stress, then the field concentrations should be different from the growth room.  Likewise, in the field Bt-corn may grow larger and be healthier than non-Bt corn due to its ability to repel European corn borers.  It might be useful to have your students identify possible patterns of results and determine how different categories of results should be interpreted.

10.  Why did they test multiple varieties of corn?

Again, testing multiple varieties produces a study with more widely applicable results.  If they had only examined one variety of Bt-corn, we would guess that similar results might also be found in other varieties of engineered corn, but this would remain an inference.  Some students may need to explore the concepts of proof versus inference.  If I prove that members of the local football team lose weight on a 3000-calorie diet, I might infer that all students would lose weight on a 3000-calorie diet.  However, I must be aware that differences between my sample and the entire population may exist, and if they exist they may cause differences in the results I am attempting to predict.  This may be true even if there are no sources of bias in the sample design.
Secondly, because the different varieties of corn tested cover three different types of genetic modification, the authors will be able to evaluate whether all of these different types of genetic engineering are equal in their effects on lignin production.  Note however that there is a problem here, because there is no non-Bt-corn isoline for Event 176.  This is brought out more fully in Question 12.

There are several different ways to handle Question 11.  I present two options.  Option (a) uses a series of graphs designed to illustrate the potential interactions between the different sets of independent variables (Bt-corn vs. non-Bt-corn and field vs. growth room trials) for each isoline.  Option (b) uses a single bar graph, which makes it more difficult to see the interactions but easier to compare the different isolines.  Note that depending on how this case is taught, an additional option is to use these different graphs to explore how graphical presentations influence the interpretation of the results.  Give the students the results table and ask them to construct the bar graph before returning to class, or give them both the six graphs and the bar graph and compare them.

11.  (a. and b.)  Based on the graphs (note:  graphs are based on data in Tables 1 and 2 of the paper but are not from the paper), what do you conclude about differences in the % lignin content between the growth room and the field?  Between Bt-corn and non-Bt-corn?  Between the six varieties of corn?  Is it appropriate to make generalizations, or is there evidence that generalizations cannot be supported by the data?

I hand out these graphs and the questions immediately after we finish discussing the additional information that the methods section gives us.  My biostatistics students should be familiar with this method of presenting means in a two-way experiment, but my plant ecology students need me to explain what the graphs show.  The title of each graph is the name of the company and the name of the hybrid variety.  The vertical axis is the % of lignin.  The horizontal axis is really a dichotomy:  Bt-corn vs. non-Bt-corn.  There are two lines on each graph, one connecting the means for the growth-room experiment and one connecting the means for the field-grown experiment.  If there are no differences between the two environments, then the two lines should overlap completely.  If there are no differences between the two varieties of corn, then the lines should be horizontal.  The steeper the slope of the line, the stronger the effect of variety, and the farther apart the two lines are, the stronger the effect of environment.  Finally, if the lines are not parallel then there is an interaction between variety and environment (i.e., it is invalid to make generalizations about the effect of environment or the effect of variety without referencing the specific conditions of the other variable).  An interesting option is to hand out the graphs without the interconnected lines and ask the students to draw the lines; this engages them more into the concept that the graphical presentation attempts to make.
In all six graphs the lines slope down to the right, indicating that the concentration of lignin in Bt-corn is higher than in non-modified corn.  In N7590 there probably is no difference between the two growing environments, but in the other five graphs the field-grown corn is slightly to substantially higher in lignin content.  Interestingly, in the DeKalb and perhaps the Pioneer varieties the concentration of lignin in the growth-room increases more between Bt-corn and non-Bt-corn than it does in the field.  Finally, within a given environment the amount of lignin in the modified corn seems to be more variable across the different varieties tested than does the amount of lignin in the non-modified corn.
The bar graph actually contains more information than the six pairwise graphs because standard deviations are included to indicate the amount of variation.  Explain to the students that standard deviations are not 95% confidence intervals around the mean, but do efficiently describe the amount of variability.  These standard deviations are quite small given the small sample size, which means that we can have more confidence that the differences between means are not just due to sampling error.  Students should be able to see that in general the field-grown corn had higher lignin levels than the growth room corn, that the genetically modified corn had substantially higher lignin levels than the unmodified partner, and that in some cases the amount of difference between the field trial and the growth room trial is bigger for one member of the isoline than for the other.
You may want to end this discussion by examining the words used to describe the patterns.  What do we mean by "probably," "perhaps," "is higher"? This language goes directly to the heart of experimental methodology:  we examine a sample so that we can make inferences about the population, and statistical analysis enables us to move away from vague language like "probably" to language like "Pioneer produced more lignin than DeKalb (p=.03)."  Statistics do not make decisions for us, but they do provide quantitative information to enable us to make more accurate decisions.

12.  Why do you think they tested different varieties of corn in the growth room vs. in the field?

This is an important but unanswered question.  A careful examination of Tables 1 and 2 reveals that the Maximizer variety apparently has no non-Bt isoline.  In three varieties tests were conducted in the growth room but not in the field, and in one variety tests were conducted in the field but not in the growth room.  In my opinion, this should have been explained in the methods or the beginning of the results section.  It does not make sense to have deliberately designed the experiment with these omissions.  It is possible that the growth room did not have enough room to permit all of the varieties to be tested there, and it is possible that there were crop failures; if so, this should be mentioned.  Ask students why it might be important to know if crop failures occurred.
The real experimental design can be summarized as a four-way design, with type of genetic modification, modified vs. non-modified varieties, growth room vs. field test, and multiple varieties of corn.  It may be helpful to build a table depicting the overall experimental design, so I have included one as an example.  In this table I have assigned each variety of corn a different letter and have included the mean lignin content for the treated group.
Bt11
  non-Bt corn Bt-corn
growth roomA: N7590 = 4.8
B: N67-H6 = 3.9
C: N3030 = 4.4
D: NC4880 = 3.4
E: NK4640Bt = 3.2
A: N7590Bt = 7.2
B: N67-T4 = 6.3
C: N3030Bt = 7.0
D: NC4990Bt = 6.6
E: NK4640Bt = 6.3
field testA: N7590 = 5.2
B: N67-H6 = 4.5
D: NC4880 = 4.8
E: NK4640Bt = 4.9
A: N7590Bt = 7.4
B: N67-T4 = 7.1
D: NC4990Bt = 7.7
E: NK4640Bt = 7.9


176
 non-Bt cornBt corn
growth room F: Maximizer = 4.0
field test F: Maximizer = 5.9


MON810
 non-Bt cornBt corn
growth roomG: P3223 = 3.2
H: DK647 = 4.4
I: DK679 = 3.8
J: DK626 = 3.2
G: P31B13 = 6.0
H: DK647Bty = 6.2
I: DK679Bty = 6.6
J: DK626Bty = 6.1
field testG: P3223 = 4.9
J: DK626 = 5.1
K: P32P75 = 4.8
G: P31B13 = 7.0
J: DK626Bty = 6.8
K: P32P76 = 6.8

Questions 13-17 focus on the statistical analysis of the data.  I use these questions in both my plant ecology course and my biostatistics course, but I expect much deeper discussions from my biostatistics students than from my plant ecology students. Depending on the needs of the instructor, some or all of these questions may be omitted, modified, or used more as teaching tools than as student-driven discussions.

13.  Examine Tables 1 and 2.  What does SEM mean in the title?  How is SEM determined?  Why is SEM a poor measure of variability?  What should have been reported instead and why?

This may seem like a technical point, but it is important because the use of SEM is less informative than other measures of variability.  SEM is the Standard Error of the Mean (see Zar 1999, Biostatistical Analysis, p. 77).  It is calculated by dividing the standard deviation of the sample by the square root of the sample size.  SEM specifically refers to the normal distribution of means, and because it is highly dependent on sample size (as sample size increases SEM decreases) it cannot and should not be used as a measure of sample variability or to infer variability within a population.  It is highly preferable to report the standard deviation or 95% confidence intervals.  Furthermore, in this paper the sample sizes are never stated, so it is impossible even to calculate the standard deviations by reversing the equation.

14.  What statistical test do you think they used to produce the P values?  What do the P values mean?  What information should have been reported but is missing?

Probably a series of t-tests were conducted, each test comparing differences in the concentration of lignin for one isoline in one environment.  A P value is a measure of the probability that the two samples are taken from two different populations.  In other words, the smaller the value of P, the more confidently we can say that there is a real difference between the two types of corn within that particular environment.  P values range from 0 to 1, and most scientists are unwilling to say there is a difference between two populations unless the P value is smaller than 0.05.
The authors should have reported what the statistical test was, what computer program was used to calculate the test, and at a minimum the sample size.  Ideally the actual t value would be reported as well.  Dr. Stotzky has told me that they used a Student's t-test for all of the statistical analyses and that the sample size was five or six corn stalks for each of the varieties tested (Stotzky, personal communication).

15.  Overall, do you think there are differences between the three different types of genetic modifications (Bt11, 176, MON810)?  What evidence can you identify to support your conclusion?

There are several ways this question can be approached.  Some students may decide to average the lignin content of each of the three different types of transformations.  If they do this, it is preferable that they keep the two types of growth conditions separate.  Other students will just look for trends:  "The Bt11 varieties seem to have higher lignin contents than the other transformations, ranging from 6.3 to 7.2 (growth room) and 7.1 to 7.9 (field), whereas MON810 ranged from 6.0 to 6.6 (growth room) and 6.8 to 7.0 (field), and 176 was even lower, with 4.0 and 5.9%."
Note that there is a very serious problem with interpreting the Event 176 results because there is only one replicate and no isoline.  Thus, only very general conclusions should be made about Event 176.  It is interesting that the Bt-corn produced by Event 176 is closer to the non-Bt-corn isolines of the other varieties than it is to the other Bt-corn varieties.  This is a point that should particularly be explored in courses examining the consequences of genetic transformations.
Another issue to consider in analysis is whether the differences in lignin content in the non-transformed isolines should be considered.  For example, while N7590Bt is higher than N67-T4, the non-transformed N7590 isoline is also higher than N67-T6.  While the differences in the nontransformed varieties are substantially smaller than the differences between transformations, it appears that significant differences in lignin content do exist between the nontransformed varieties.
A good way to get students to move beyond vague qualitative assertions is to require them to build a mathematical model of this experiment.  These models can take different forms depending on how much information is condensed, but a typical one might be:
 
% lignin = A + B if field grown + C if Event 176 or D if Event B11 or E if Event Bt11 + error
 
where A, B, C, D, and E are quantifiable parameters.
This type of approach forces more rigorous analysis of the results, emphasizes how the results could be used predictively, and is a springboard for discussing error, particularly in comparing different models.  For example, the model
 
% lignin = A + B if field grown + C if genetically transformed + error
 
will have a larger error term, but is simpler and more general.  In my biostatistics course we have already built many models and my students are familiar with the concept that models can maximize at best only two of the three attributes of models (precision, accuracy, and generality).
In my biostatistics course, I tell them that Saxena and Stotzky (2001) wrote:  "There was a significantly higher lignin content (P <0.002) in plants transformed by event Bt11 (7.4 + 0.10 and 6.7 + 0.12% for field- and growth room-grown plants, respectively) than by event MON810 (6.9 + 0.07 and 6.2 + 0.10% for field- and growth room-grown plants, respectively).  The lignin content of the only available hybrid transformed by Event 176 was lower than that of hybrids transformed by events Bt11 and MON810."  I then go on to ask them how these P values were obtained (unclear, but probably one-way ANOVAs), and I ask them how the average numbers were calculated.  With a little encouragement, students quickly realize that these averages are not just the averages of the means reported for each experiment (i.e., the mean of the Bt11 transformations in the field is 7.525, not 7.4).  This leads us to a discussion about sample sizes probably being unequal, which is generally not recommended for ANOVAs.  Finally, we discuss how this question should be answered statistically (probably as a set of 2-way ANOVAs).

16.  What statistical test should have been used on the six varieties of corn for which they have both modified and non-modified varieties tested in both the field and the growth room?

I use this question primarily in my biostatistics course.  A two-way ANOVA for each of the six varieties is the most appropriate and powerful test.  I ask my students to explain why this is true, and make sure that the concept of interactions between transformation and environment is discussed.  It is helpful to refer back to the page of graphs to discuss how the graphs depict possible interactions, especially for DeKalb DK626.

17.  The authors also report that "The average diameter of the vascular bundle and surrounding lignified cells in Bt corn was 21.5 ± 0.84 um, whereas that of non-Bt corn was 12.4 ± 1.14 um."  Is this useful information?  If so, how?  What are possible implications?

The bundle sheath cells are the primary site of photosynthesis in leaves.  Of course, these data are given for the stems, not the leaves.  However, it is possible that the genetically modified corn has a larger photosynthesis apparatus.  Second, larger diameter xylem cells are more likely to cavitate, so the genetically modified varieties may be more susceptible to drought-induced stress.  Third, these physiological changes suggest a variety of possible mechanistic explanations of the genetic modification causing an increase in lignin.  Presently we do not know why lignin increases, only that it does.  The most satisfactory explanations must include a clear understanding of how the genetic change alters the physiology of lignin production.  For example, perhaps the engineered plants, because they are resistant to corn borers, simply grow larger; larger plants need to invest more in structural support.

18.  What kinds of ecological implications might modifications in lignin content have?  What research should be done next?  What suggestions do you have for other experiments and comparisons that should be made to improve this research paper?

I always like to ask my students what research should be done next.  In fact, this question can be a useful follow-up essay assignment.  Future research questions could include examining the varieties that should have been included to make it a complete design, sampling lignin from multiple places within the corn plant, contrasting these plants grown in a low-European corn borer density field with results from a field heavily infested with corn borers, examining lignin concentrations in the corn grain itself, and testing for differences in decomposition rates.
There are many possible implications.  First, lignin slowly decomposes, so we can reasonably expect that the residency time of corn biomass in soil will increase.  Second, cattle and other animals fed corn silage may receive less nutritional value from lignin-enhanced silage and may feed preferentially on silage with a lower lignin content if it is offered.  Third, we do not know how much the Bt toxin is retained in corn biomass with high lignin contents, but if the toxin is present in the corn biomass, then an increased residency time in soil may result in an increased accumulation of the toxin in the soil.  Fourth, the widespread use of Bt-engineered corn may cause some species of insect pests to evolve a resistance to the CryIA proteins, which could be economically devastating to the organic farming industry which relies heavily on using Bacillus thuringensis as a "natural" pesticide.  Fifth, it is interesting to note that Bt-corn seed is more expensive to purchase than non-Bt-corn.  Therefore, if the concentrations of European corn borers are low, farmers will lose money and Bt-engineered corn should be planted only when concentrations of corn borers are expected to be fairly high.  European corn borers tend to go through a seven-year cycle, with losses during peak infestations of as much as 20% of the harvest (see http://www.dekalb.com/dktraits/BTcorn.htm).  However, planting Bt-corn during years when corn borers are not abundant will not be economically sensible.  Ric Bessin has an excellent webpage, located at http://www.uky.edu/Agriculture/Entomology/entfacts/fldcrops/ef118.htm, which includes a worksheet on this subject and an extended discussion of costs and benefits from a farmer's viewpoint.  Finally, although we know nothing about the concentration of lignin in the leaves, if leaf concentrations are higher, then animals which eat the leaves may experience reduced digestibility, longer developmental time frames, and other negative effects.
It is worth examining the paragraph by Saxena and Stotzky on this issue, so I include it below, with the references they cite.
Saxena and Stotzky (2001) wrote:  "Any modifications in lignin content could have ecological implications (Halpin et al., 1994).  For example, the increase in lignin content in Bt corn may be beneficial, as it can provide greater resistance to attack by second-generation European Corn Borer (Ostrander and Coors, 1997), reduce susceptibility to molds (Masoero et al., 1999), and retard litter degradation and decomposition by microbes (Reddy, 1984; Tovar-Gomez et al., 1997).  The addition of biomass from Bt corn to soil resulted in a significantly lower gross metabolic activity (i.e., CO2 evolution) in soil than did the addition of non-Bt corn (S. Flores, D. Saxena, and G. Stotzky, New York University, unpublished data; Stotzky, 2000), which may be beneficial, as the organic matter derived from Bt corn may persist and accumulate longer and at higher levels in soil, thereby improving soil structure and reducing erosion (James et al., 1998), or it may be detrimental, as the longer persistence of the biomass of Bt corn may extend the time that the toxin is present in soil and, thereby, may enhance the hazard to nontarget organisms and result in the selection and enrichment of toxin-resistant target insects.  Moreover, lignin is relatively indigestible and reduces the ability of herbivores to digest plant material, and its increase in forages might affect rates of feeding and population dynamics of defoliators (Barriere and Argillier, 1993; Jung and Allen, 1995; Gardner et al., 1999).  Additional studies are necessary to clarify the environmental impacts of a higher lignin content, especially in Bt corn, as about 8.1 million hectares (20 million acres) of Bt corn (26% of total corn acreage) were planted in the United States alone in 1999 (U.S. Environmental Protection Agency, 2000).

Other Options

This case provides a framework that can be modified to fit the needs of a particular course.  I have used it, as described, to meet the needs of two different courses I teach.  I am always looking for writing assignments, so in my biostatistics course I end the case by handing out a dataset (available for download in Excel format here), which I have obtained from the authors.  I expect my students to analyze the six varieties with the appropriate statistical tests, build quantitative models of the effects of each variable, draw their own conclusions, and write a formal analysis of the data.  In my plant ecology course, I have asked students to produce a written analysis of the first five questions, giving them the first piece of the case several days before we begin to discuss it.  Other instructors of other courses might find it useful to assign reaction papers or to use this case as a springboard into the many other aspects of biology which this case touches upon (soil ecology, genetic engineering, social and ethical implications of science, plant physiology, etc.).  For example, a fascinating question that could be asked is:

12b.  What different events can occur when genes (including the Bt genes) are inserted into a plant's chromosome?  Where do the Bt genes become inserted?  How could the insertion site affect the production of lignin?

Finally, it may be worthwhile to ask students to reflect on their experiences as they analyzed this case.  Many students do not realize what they have learned unless you build in a reflection component that forces them to examine themselves.  I find that this type of self reflection is particularly valuable in defusing the possible consequences of frustrations they may feel as they struggled with the case and in helping them to see how they have accomplished the learning objectives.  One way that I have built in this reflective component is to ask them to spend a few minutes responding in writing to this statement:

"I want my students to see how a research paper reflects the process of doing science and to realize that they can intelligently read original research documents and ask legitimate questions about the manuscript.  How has this case helped you understand the process of doing science, and how has it helped you to build confidence in your ability to read original manuscripts?"

Because my courses are small, my students and I have been able to reach a fairly high level of trust by the time I teach this case.  Therefore, another option I have used is to collect their final writing assignments and ask a few questions.  "What was the most frustrating piece of this case?" Usually they say that they felt I expected them to know too much.  I get responses such as "I really don't know much about genetic engineering, so I didn't feel comfortable answering questions about their data." I can then go on to point out that it is not only ok that they don't completely understand the topic, but expected; in fact, I don't know much about genetic engineering either.  Often they will say that they felt uncomfortable with the statistics, which gives me the opportunity to point out that statistics are nothing more than a measuring tool we can hold up against the data to evaluate the probability and magnitude of effects, but that the writers and the readers both are required to decide for themselves whether the differences between the samples indicate real differences between the populations.

In particular, one of my goals in this reflective process is to defuse any feelings that they are not able to explore primary literature or that in science the answers must exist somewhere.  One problem of the three-hour canned lab experience is that we condition our students to expect that every research project should produce significant differences with easily understandable causes and effects; then, when they encounter this case, our lack of knowledge about why lignin increases, why the researchers didn't test an Event 176 isoline, or why there are differences in lignin content between different non-transformed hybrids is a source of confusion and frustration, when it should be a source of inspiration for future research ideas.

REFERENCES

Information on Lignin:

Information on Bt:

Literature Cited:

Acknowledgements:  This case was developed with support from The Pew Charitable Trusts.  It is based on a 2001 article written by Saxena and Stotzky entitled "Bt Corn Has a Higher Lignin Content than Non-Bt Corn," which appeared in the American Journal of Botany (vol. 88, no. 9, pp. 1704-1706).  The author wishes to express his gratitude to Drs. Saxena and Stotzky, who were particularly helpful in clarifying questions the author had and providing him with a copy of the original data.

Go back to the case


Image Credit:  European corn borer, Ostrinia nubilalis, an insect species introduced to North America and belonging to the family Pyralidae in the order Lepidoptera.  Photo by Keith Weller, from the collection of the USDA Agricultural Research Service.

Date Posted:  10/03/02 nas
This file is also available in Adobe Portable Document Format (PDF).  PDF Version


SEND Comments HOME READ Comments click left to send comments, middle to go home, right to read comments