Organic chemistry is and always has been a difficult course to master for most students. A student’s success in a chemistry course is strongly correlated to on-task studying of the material through solving homework problems.(1) With large undergraduate class sizes, such as the ones at the University of Cincinnati, hand grading of homework problems and providing students with valuable feed-back is impractical. Online homework overcomes most of these obstacles. The computer software acts as the instructor, providing the correct answers, guidance, and tutorials.(2) Furthermore, the software is a diagnostic tool for the instructor to monitor individual and overall performance. Students appreciate the immediate feedback provided by the software, which reinforces the topic being taught and fosters a more positive study attitude.(3, 4) However, it has been found that most students need to receive points toward their final scores before they will complete online homework problems.(3) Over the past decade, many studies have shown a positive relationship between online homework and a student’s total points/course grade.(1-6) In general, the total points/course grade for the students who perform well on the online homework is higher than for students who do not complete online homework assignments. In addition, those students who complete online homework on average do better on examinations in comparison to those students who do only written homework.(7)
For our organic courses, taught by four professors over the past eight years, we observed the same positive relationship between online homework and examination performance as determined by the examination averages. However, looking more deeply into these data, we unexpectedly discovered that excellent performance on online homework is not a predictor of examination performance. Specifically, we find that half of the students who earn the majority of points on the online homework are significantly below average on the examinations. In addition, we find that students who combined traditional pencil–paper problem solving with the online software performed better than students who solely use the online homework package.
Method—Online Homework Setup
For those only familiar with online homework packages for a general chemistry course, please be aware that online homework for organic chemistry is somewhat different. Similar to general chemistry problems, the organic package contains multiple-choice questions, but the possible algorithmic variations of a single problem are much more limited. In addition, unlike the general chemistry problems, online homework for organic chemistry contains a large number of problems that require using a drawing program to show the product of a reaction or to show the arrows for a mechanism.
The organic professors at the University of Cincinnati have used Connect or OWL as homework packages. These programs were chosen, in part, because they are integrated with course textbooks and contain a subset of the back of the chapter problems. Connect and OWL are both interactive homework packages providing students with feedback, guidance, and tutorials. The problems are parametrized such that if a problem is repeated, it is a new problem that investigates the same concept with different values or molecules. For Connect, we also provided LearnSmart, which is an interactive tool that adaptively assesses a student’s knowledge and confidence level of organic topics. Both Connect(7) and OWL(8) have been shown to improve student performances in organic chemistry courses.
Regardless of which package was used by us, online homework accounted for about 15% of the total points for the course. For the online homework, we assigned 15 problem sets with questions based on the material discussed in each hour and a half class period. To earn all the possible points, a student needs to solve around 160 problems per semester. The remaining points in the course were generally divided into quizzes worth between 0% and about 15%, two or three examinations worth a total of 40–55%, and a final examination worth about 30% of the total points. The examinations use pencil and paper and do not contain multiple-choice questions. A full range of typical organic chemistry questions are asked including topics such as nomenclature, physical and chemical properties of molecules, stereochemistry, products from reactions, synthesis, and mechanisms.
The authors believe that online homework is a very promising pedagogical tool. Because we are strong proponents of homework problems as an instructional tool for organic chemistry, students are given a large number of attempts and are given feedback after each attempt. We realize that this method of administering the online homework enables a student to acquire all the homework points through guessing. On the other hand, we felt that limiting the number of attempts would be more stressful by treating homework as an examination rather than a learning experience. Our approach is supported by a recent study that found students were most appreciative of the immediate feedback, the opportunity of multiple attempts, and the ease of the Connect program when solving online homework.(7) In two of our large classes (approximately 350 students), we discovered that most students completed each problem set using only two to three attempts (the average is 2.2 ± 0.4 attempts per problem set). Furthermore, we found that most students spent a significant amount of time working homework problems, apparently in an attempt to master organic chemistry.
In this study, we use Pearson correlation coefficients to measure the linear correlation between the percentage of examination points versus the percentage of online homework completed. A statistical analysis was performed on students’ use of the Connect program (averaged time spent actively working on the problems and the averaged number of attempts) to determine if there was a significant difference in the methods used by students to complete the online homework assignments.
For the past two academic years, we used Connect. For our courses, a total of 1143 students completed organic chemistry. As can be seen in Table 1, we compare the percentages of online homework completed to examination scores. We do not compare percentages of online homework completed to the total points because the total points include the online homework, and thus, bias the results. We also exclude quiz scores since some courses did not give quizzes. As the data show, those students who earn 90% or more of the points for the online homework did better on the examinations than the class as a whole (56% vs 48% of total examination points, Table 1). In contrast, students who complete less than 50% of the online homework generally do more poorly (31% vs 48% of total examination points, Table 1). The large standard deviation of approximately 17% makes the difference in these numbers less relevant. Because of this rather large standard deviation for the averages, we also looked at the median value and obtained similar percentages. Thus, our findings are consistent with previous literature studies; those students who do well on the online homework, on average, do better on the examinations than the class as a whole.
When the examination percentage versus the online homework percentage is plotted (Figure 1), we find it is true that students who perform poorly on the online homework also perform poorly on the examinations. However, we find no relationship between online homework points and examination scores for those students who earn most of the online homework points (Figure 1, right graph). The Pearson correlation coefficient drops from 0.405 when comparing all students to 0.205 when comparing only students who solved at least 90% of the online problems. Put another way, if a student does poorly on the online homework, he or she will likely perform poorly on the examinations. However, if a student does really well on the online homework, one cannot predict how well that student will perform on the examinations.
Figure 1. Connect percentage versus examination percentage. The left graph is all the students, and the right graph is those students who earned 90% or more of the online homework points. The correlation coefficient for the left graph is 0.450 and for the right graph is 0.205.
Next, we wanted to determine whether this was a general trend or something specific to the Connect package we are currently using. In previous years, we used the OWL homework software. Because we used it for a longer period of time, we have more data. As can be seen in Table 1, the general trend is the same as seen with Connect. The higher the percentage of points earned for the online homework, the higher the examination average or median; however, the same high standard deviation is obtained. Plotting these data, as shown in Figure 2, shows that students who perform poorly on the online homework perform poorly on the examinations, but once again, there is no correlation between completing 90% of the OWL homework and performing well on the examinations.
Figure 2. OWL percentage versus examination percentage. The left graph is all the students, and the right graph is those students who earned 90% or more of the online homework points. The correlation coefficient for the left graph is 0.443 and for the right graph is 0.115.
One could readily argue that the lack of correlation between online homework performance and examination performance is due to the fact that the problems given on the examinations are substantially different than the ones given online. We believe this is not true. As discussed previously, the problems given on the examinations are standard organic chemistry questions, and most of the examination problems are very similar to the homework problems. As an aid to the students to help them study, they are told at the beginning of each course that examination problems are very similar to the online homework problems. Furthermore, the scores reported were obtained from multiple classes taught by educators with different teaching styles.
To further explore the possibility that there is a disconnection between our examination questions and the online questions, we compared the online homework scores with the percentage of questions answered correctly on the ACS standardized examination (Table 1 and Figure 3). The same trend as discussed earlier was found with these standardized examinations—there is no relationship between doing well on the online homework, using either Connect or OWL, and how a student will perform on the ACS examination. (Because not all classes gave the ACS examination, we have fewer data points.)
Figure 3. Online homework percentage versus ACS standardized exam percentage. The left graph is for Connect, and the right-hand graph is for OWL. The correlation coefficient for the left graph is 0.337 and for the right graph is 0.402.
Table 1. Examination Average versus Online Homework Data
Because the graphs and the correlation coefficients are so similar, to expedite the discussion of the data, in Figure 4, we have combined the Connect (Figure 1) and OWL (Figure 2) data and have arbitrarily divided the graph into four quadrants of equal size. The students in Quadrant 2 (Q2) are generally well motivated, well prepared, and most likely would excel in organic chemistry independently of the style of homework. Notice that there are virtually no students in Quadrant 4 (Q4), namely, those students who do poorly on the online homework, but do well on the examinations.
Figure 4. Homework percentage versus examination percentage. The data shown in Figure 1 and in Figure 2 have been combined, and the graph has been divided into four quadrants of equal size.
The students in Quadrant 3 (Q3) are the “nonparticipants”. They do not complete the homework assignments and apparently do not study for examinations. This group is the main reason why the examination average or median for those who do online homework is higher than the examination average or median for the class as a whole.
The final group of students lies in Quadrant 1 (Q1). They score well on the homework assignments but perform poorly on the examinations. As seen with the entire population of students, there is no correlation between percentage of examination scores and percentage of online homework completed (Pearson correlation coefficient = 0.224). These students, who comprise approximately 40% of the class, would benefit the most from a well-crafted homework system. Most educators likely would agree that teaching students on a one-on-one basis is ideal. Current online homework programs provide this service conveniently because students can work a problem multiple times with constant feedback. So why are there so many students in Q1?
Because of the nature of the material in organic chemistry, the algorithmic variations for a problem are much more limited than for general chemistry problems. Thus, an unmotivated student may just randomly click responses until the correct answer is obtained. If one considers that the solution for many problems requires using a template to draw structures or to provide mechanistic arrows, the likelihood that students randomly provide correct answers is diminished. Another possibility, of course, is that a classmate simply provides the answers. In other words, these students are apathetic; they do not care about learning the material, only in obtaining the points.
Purchasing expensive textbooks is, for the most part, a necessary evil.
After all, it’s not easy to make a textbook. Writing a college-level text requires a significant time commitment from a number of very well-educated and intelligent people – people whose time is valuable, and who must be compensated well for their time. Besides the actual writers, it takes an enormous amount of editing to make sure every little detail of the book is correct.
Of course, this assumes the textbook itself is high-quality. The only thing worse than dropping $200 on a textbook is dropping $200 on a shitty textbook.
While it may be there isn’t too much to do about high textbook prices for the time being (I’d love to be proven wrong), there is a worrying new trend that is making it even more difficult to purchase textbooks without breaking the bank: a required online portion of a course.
At first, this doesn’t sound like an inherently bad idea. After all, technology is becoming an ever more important part of our lives, and classrooms need to take advantage of these new opportunities. However, there is a right and wrong way to go about doing so.
Certain classes at this university require students to use an online program to complete and submit their homework — for instance, Accounting 100, Finance 100 Economics 101 and Physics 202. There are certainly some advantages to online homework, like faster feedback and much less work for professors and teaching assistants. However, at what expense? For many of these programs, there are two ways to gain access: either purchase the textbook new or purchase an access code for the program separately. In some cases, access to the online program can cost more than $100 – as much or more than the new textbook itself.
Considering that online coursework programs are priced in such a ludicrous manner, it becomes clear this is nothing more than an attempt by textbook companies to quash the used textbook market.
But why would professors choose to use these types of programs? After all, professors have been through their fair share of school. They understand how budget-busting purchasing textbooks can be. While I’m not a mind reader, allow me to hazard a few guesses.
First, they might not have considered the issue. The textbook publisher tells them the online portion of the course comes with all (new) textbooks, so it won’t cost the students anything extra – which is true only if students never buy used books.
Second, they might be trying to make their lives easier. Especially in large lectures, grading homework from a few hundred students can be an awful lot of work.
If the second reason does play into professors’ decision to use these sorts of online programs, then they either don’t understand or willfully ignore the interests of their students. If that’s the case, then professors are essentially charging students in order to decrease their own workload – in a large lecture, we could be talking on the order of tens of thousands of dollars for a single class. College is already expensive enough, thank you very much.
Besides the simple monetary aspect of the issue, there’s also the reality these programs often aren’t that good. Any sort of automated grading system leaves little room for questions gauging actual comprehension and understanding, rather than rote memorization and computation.
The classroom as we know it needs to continue to evolve in order to keep up with the 21st century. There are a variety of ways to go about doing this – some better than others. There are definitely ways to update the educational experience without allowing textbook publishers to destroy the used textbook market.
If professors are using these sorts of programs out of ignorance, then they need to realize what they’re doing to their students. If they’re using them because they don’t want to grade homework assignments, then they need to seriously reevaluate their priorities as an educator.
Joe Timmerman ([email protected]) is a sophomore majoring in math and economics.