Effect of K-12 Calculator Usage on College Grades

W. Stephen Wilson

(410) 516-7413

(410) 516-5549 FAX

Department of Mathematics

wsw@math.jhu.edu

Daniel Q. Naiman

Department of Mathematical Sciences

Daniel.Naiman@jhu.edu

Johns Hopkins University

3400 N. Charles Street

Baltimore, MD 21218, USA

Abstract

We find that students in the big mathematics service courses at the Johns Hopkins University who were encouraged to use calculators in K-12 have somewhat lower grades than those who weren't.

Effect of K-12 Calculator Usage on College Grades

Results

We collected various information from the students in the large multi-section mathematics courses at the Johns Hopkins University in the fall of 2002. We asked students if parents ever helped them with mathematics and if calculator usage was "emphasized and encouraged" in K-12. We put this together with their grades in these courses and their SAT mathematics and verbal scores. Regression analysis led to the following fit to the data:

grade = constant + parents .21547 -calc .23239 + msat .00350 + vsat .00057.

Here, parents is 1 if parents ever helped and 0 if not; calc is 1 if calculators were encouraged and emphasized and 0 if not. msat and vsat are the mathematics and verbal SAT I scores respectively. The grades are on the usual 4.0 scale. We did the usual 3.3 for a B+, 3.0 for a B, 2.7 for a B-, etc. The constant isn't relevant.

Let's first look at the verbal SAT score. 200 points would raise the grade by .1, which isn't much. Worse, the p-value is .32 so even this isn't significant. This is a disappointing result as anecdotal evidence suggested there might be something interesting here.

The question about parent's help was motivated by the many reform mathematics programs that instruct children to not talk about mathematics with their parents. Its p-value is .01 and the standard deviation is .08349. This question is fraught with difficulties because of the socio-economic issues that could underlie it, so we don't want to push this.

The real heart of the matter for us is the combination of the mathematics SAT and the question about calculator usage. We are not that familiar with the literature but what we have seen suggests that SAT scores are not that good at predicting grades in college. Although the SAT score does have some input here (the p-value is .00000 and the standard deviation is .00066) it would take 286 points to account for a full letter grade and the entire spread of our mathematics SAT scores is only 280.

The -.23239 contribution from calculator usage has a p-value of .00377 and a standard deviation of .07992. It may not seem like much but with plus and minus grades it suggests that a fair number of students are a grade notch lower. Our students care about each step in the grades. This requires 66 points on the mathematics SAT to make up for it, another thing our students care about.

We did this without the SAT scores first and since we are unfunded we thought we would stop there. However, it turned out to be much easier than expected to get the scores.

Calculators clearly have a place in the classroom but we suspect that they are sometimes misused. Our students get quite upset when we show them these statistics.

Background

In the fall of 2002 a questionnaire was handed out with the final exams in most of the large multi-section service courses in the mathematics department at the Johns Hopkins University.

The courses surveyed were as follows: 110.105, Introduction to Calculus, our pre-calculus course. This course is actually quite small but since it has two sections it was included. 110.106-7, Calculus I and II for Biological and Social Sciences. 110.108 Calculus I for Physical Sciences and Engineering. Regrettably, the Calculus II course in this sequence did not get surveyed because of a breakdown in communication. 110.201, Linear Algebra. 110.202, Calculus III - Calculus of Several Variables. We overlooked the higher level multi-section differential equations course.

There were a total of 776 students who finished the surveyed courses with a grade. Of these, we had signed, complete, surveys from 663 of these students. A number of students handed in the survey but didn't put their names on it so we couldn't check their grades. And, of course, some students just didn't do the survey. Other students gave ambiguous answers. Of these 663 students we had SAT scores for 607 of them. Johns Hopkins University doesn't require the SAT for admission (but most students take it).

Calculators were allowed on exams for only one of the surveyed courses (107). Most faculty at Johns Hopkins University don't feel the need for calculators in basic mathematics courses.

The department has a grading policy that tries to keep grades in all of these courses within a certain range. Thus the percentage of A grades, B grades, etc, doesn't vary dramatically from one course to the next.

The survey

The first question on the survey asked the student to circle one: "In K-12, your parents (or tutors) helped you with mathematics" (1) regularly, (2) a little, or (3) not at all. We combined the first two answers together for our analysis.

The second question was: "In K-12, calculator usage was" (1) emphasized and encouraged, (2) was taught but not pushed, or (3) not much at all. Here we combined the last two answers.

In addition we asked a couple more questions that weren't very good. Things we would like to know but find it difficult to find out are what kind of geometry course a student has had in high school and what texts they used for algebra. The impact of these on college performance would be very interesting but this is not information that can be extracted from students unless you are willing to take "it was a blue book" as an answer.

More numbers

Doing the numbers on individual courses (without the verbal SAT) doesn't always work out because of the smaller numbers of students. In only one case does the calculator coefficient turn up positive. That is in the smallest (15 survey respondents) course (105), and the p-value is a whopping .80. The largest (184 respondents) course (106) had a p-value of .50. For the next two courses (in size) not only is the result significant but the coefficient is quite large. For 202 (with 136 respondents) the coefficient is -0.49447 with p-value .02172. In 108 (130 respondents) the coefficient is -0.43561 with p-value .00485. For these two courses this is nearly half a letter grade. For the other two courses, 201 (77 respondents) and 107 (65 respondents), the coefficient is negative but the p-value is high. In all cases the p-value for the parent coefficient is high. The only close case is 202 where the coefficient is .35088 and the p-value is 0.08269.

The numbers for the parents-calculator scores are: {0,0}, 133; {0,1}, 73; {1,0}, 206; and {1,1}, 195.

We have various correlations coefficients: Calc-grade, -0.13; Calc-MSAT, -0.11; Grade-MSAT, 0.24; Parent-Calc, 0.13; Parent-grade, 0.08; Parent-MSAT, -0.06.

The standard deviation for the MSAT is 64 points. The standard deviation for the grades was 1.00, exactly one letter grade.

Suggestions

It would be nice to refine the questions somewhat and do this again, but it isn't likely that we will. In particular, this phrasing of the calculator question allows the student to decide what happened in K-12. We doubt if any of these students had calculators emphasized and encouraged all the way through. Likewise, they clearly took it to mean more than just a year to get them ready for the SAT exams. If we informally ask a class if calculator usage was emphasized and encouraged for 1 or more years in K-12 then the the percentage nearly doubles from the 44% we got for our question. Thus, it would be nice to refine this question significantly asking for a variety of details about lower and middle school, etc.

The majority of high school graduates now go on to college. Thus most K-12 programs are college preparatory by default. The success of mathematics programs in K-12 should be judged by how well students do in college. If this were our line of business, and we had significant funding, then we could think of lots of studies to do to try to answer the question: What should K-12 do to make a successful college student?