Effects at 10 Schools
Effects of HighScope Early Elementary Curriculum Support on School Achievement and Reducing Discipline Referrals
By Lawrence J. Schweinhart, President, and Charles Smith, Director of Youth Development Group
HighScope Educational Research Foundation
September 20, 2001
Abstract
This time-series study with matched comparison schools found evidence that the supported use of the HighScope model improved performance on grade 4 achievement tests at 10 schools. From the baseline year to the last year of HighScope support, 6 schools increased the percent of students passing state tests by a mean of 11 percentage points in reading (p < .01) and 26 points in mathematics (p < .001), exceeding comparison-school increases by a mean of 10 percentage points in reading and 18 in mathematics. At the 3 sites with relevant data, even stronger results were found for low-income students. At 2 HighScope schools, standardized achievement test means increased a mean of 8 normal-curve-equivalent (NCE) points in reading and 6 points in mathematics. In the 1 to 5 years since HighScope Educational Research Foundation support ended but the model continued, compared to the baseline, the 2 HighScope schools with relevant data improved a mean of 33 percentage points in reading and 51 points in mathematics, outdistancing comparison schools by a mean of 20 points in reading and 47 points in mathematics. At the site reporting standardized achievement NCE scores, compared to the final year of support, the HighScope school improved an additional 21 points in reading and 14 in mathematics, exceeding the additional increases of the comparison school by 19 NCE points in reading and 6 in mathematics. In addition, the supported use of the HighScope model at 4 schools decreased discipline referrals by 48 percent. A survey of 28 teachers' perceptions about HighScope support at 3 HighScope schools found that 76 percent of the teachers were positive about the HighScope effort and only 4 percent were negative.
Study design
This study examines improvements in school achievement at 10 schools attributable to training support in the HighScope curriculum framework by staff of the HighScope Educational Research Foundation. Table 1 identifies the characteristics of the study sites —context, years during which HighScope Educational Research Foundation provided the school with support for the model, number of years of support, years since support was provided, and tests used in the comparisons. Of the 10 sites, 6 were urban, 2 were suburban, and 2 were rural. HighScope Educational Research Foundation provided them with curriculum support for a mean of 2.9 years at some time between 1991 and 2001. Seven of the sites reported percentages passing state tests — 4 in Michigan, 2 in Wisconsin, and 1 in Texas. The 3 other sites — 1 each in Nebraska, Wisconsin, and California — reported NCE means for standardized tests.
|
Table 1
|
|
Characteristics of the Study Sites
|
|
Site
|
Context
|
Period of HighScope Support
|
Years of HighScope Support
|
Years Since HighScope Support
|
Tests Used
|
|
1 |
Urban |
1991 to 1996
|
5
|
5
|
Michigan Educational Assessment Program |
|
2 |
Suburban |
1997 to 2000
|
3
|
1
|
" |
|
3 |
Suburban |
1997 to 2000
|
3
|
1
|
" |
|
4 |
Urban |
2000 to 2001
|
1
|
0
|
" |
|
5 |
Rural |
1998 to 2001
|
3
|
0
|
Wisconsin Knowledge and Concepts Examination |
|
6 |
Urban |
1994 to 1997
|
3
|
4
|
Texas Assessment of Academic Skills |
|
7 |
Rural |
1998 to 2001
|
3
|
0
|
Metropolitan Achievement Test |
|
8 |
Urban |
1999 to 2000
|
1
|
1
|
Terra Nova Test of Basic Skills |
|
9 |
Urban |
1995 to 1998
|
3
|
3
|
Wisconsin Knowledge and Concepts Examination |
|
10 |
Urban |
1993 to 1997
|
4
|
4
|
Stanford Achievement Test |
At each of the schools using HighScope, grade 4 achievement-test statistics are available from the baseline year and the last year of HighScope Educational Research Foundation support, showing how much test scores improved while HighScope Educational Research Foundation provided support. Of the 8 sites with baseline data, 6 used state achievement tests with percentages passing statistics available at state education agency websites, and 2 used standardized achievement tests with NCE mean scores available from the school districts. At 7 of the 8 sites, achievement-test statistics were also available for comparison schools, nominated by the principals of the HighScope schools because of their demographic similarity. This comparison offers a more stringent test of the effect of HighScope support on test scores by removing from the comparison any effects of changes in the tests and external conditions during the intervening period. However, it also decreases the difference between the schools and underestimates the HighScope effect to the extent that the comparison school had engaged in any school improvement effort of its own.
Comparisons of percentages passing state tests in Table 2 were tested for statistical significance by chi-squared analyses. While such analyses involved different tests at different sites, the comparisons at each site were with the same test. Three such comparisons of percents passing the tests were carried out: HighScope schools in the final year of support versus the baseline year, HighScope schools versus the comparison schools in the baseline year, and HighScope school versus the comparison schools in the final year of support. Note that the statistical procedures could not directly test the statistical significance of the relative gains of the two groups of schools. Nor was it possible, because only mean scores were available to us, to test the statistical significance of group differences in NCE mean scores on standardized achievement tests.
Table 2 presents the improvements in percentages of students passing state tests given at the end of grade 4 at 6 HighScope schools from the baseline year before HighScope support began to the end of the period of HighScope support. This comparison offers a fair estimate of the contribution of HighScope to school improvement, in that school improvement is the principal reason for schools to improve on this measure. /p>
-
In reading, 11 percentage points more students passed the test after HighScope support than at baseline (68% vs. 57%, χ2 = 8.11, p < .01).
-
In mathematics, 26 percentage points more students passed the test after HighScope support than at baseline (71% vs. 45%, χ2 = 47.66, p < .001).
Table 2 also compares the percentage of students passing state tests at HighScope schools with the matched comparison schools at the baseline and end of the time of support to the HighScope schools.
-
In reading after HighScope support, 10 percentage points more HighScope than comparison students passed the test (68% vs. 58%, χ2 = 7.60, p < .01), up from virtually no group difference at baseline (56.8% vs. 57.3%, χ2 = 0.03, p > .10).
-
In mathematics after HighScope support, 18 percentage points more HighScope than comparison students passed the test, catching up with and slightly surpassing them (71% vs. 65%, χ2 = 3.18, p < .10), overcoming a 12-percentage-point disadvantage to the comparison students at baseline (45% vs. 57%, χ2 = 11.93, p < .001). Even though the final status of HighScope schools was not significantly greater than that of comparison schools, HighScope schools did overcome the significant baseline advantage of the comparison schools.
|
Table 2
|
Percent Passing Grade 4 State Tests at Baseline and After HighScope Support
|
|
Site |
# of Cases
|
HighScope School
|
Comparison School
|
Difference Between Increases
|
|
Baseline Year
|
Last Year of Support
|
Increase
|
|
Baseline Year
|
Last Year
|
Increase
|
|
Reading
|
|
1 |
92
|
25.2%
|
55.4%
|
30.2
|
|
20.6%
|
38.9%
|
18.3
|
11.9
|
|
2 |
328
|
41.7%
|
47.5%
|
5.8
|
|
41.7%
|
47.1%
|
5.4
|
0.3
|
|
3 |
44
|
47.7%
|
55.9%
|
8.2
|
|
75.4%
|
75.0%
|
-0.4
|
8.6
|
|
4 |
105
|
37.2%
|
41.7%
|
4.5
|
|
38.7%
|
42.3%
|
3.6
|
0.9
|
|
5 |
100
|
72.0%
|
93.0%
|
21.0
|
|
79.0%
|
92.0%
|
13.0
|
8.0
|
|
6 |
183
|
77.5%
|
91.0%
|
13.5
|
|
88.7%
|
88.9%
|
-0.2
|
13.7
|
|
Meana |
142
|
56.8%
|
67.5%
|
10.7
|
|
57.3%
|
58.3%
|
1.0
|
9.7
|
|
Mathematics
|
|
1 |
96
|
20.0%
|
66.1%
|
46.1
|
|
36.7%
|
63.9%
|
27.2
|
18.9
|
|
2 |
323
|
47.5%
|
62.5%
|
15.0
|
|
47.0%
|
57.4%
|
10.4
|
4.6
|
|
3 |
44
|
40.9%
|
73.5%
|
32.6
|
|
85.4%
|
85.0%
|
-0.4
|
33.0
|
|
4 |
105
|
55.8%
|
45.8%
|
-10.0
|
|
40.3%
|
40.4%
|
0.1
|
-9.9
|
|
5 |
108
|
50.0%
|
66.0%
|
16.0
|
|
52.0%
|
73.0%
|
21.0
|
-5.0
|
|
6 |
183
|
58.0%
|
91.0%
|
33.0
|
|
83.0%
|
77.8%
|
-5.2
|
38.2
|
|
Meana |
143
|
45.0%
|
70.7%
|
25.7
|
|
56.9%
|
65.1%
|
8.2
|
17.5
|
|
aMeans are calculated by numbers of cases per group and cannot be calculated from the information presented in this table. |
Table 3 presents mean NCE scores for the two sites that reported findings for standardized achievement tests given at grade 4. NCE scores are similar to percentiles except that they have been adjusted to fit a normal distribution. On reading after HighScope support, the mean NCE score of HighScope school students was 8 points higher than at baseline (51 vs. 43). On mathematics after HighScope support, the score was 6 points higher than at baseline (57 vs. 51). At the site that had a matched comparison school, in reading, the mean NCE score increase of HighScope students was 10 points more than that of the comparison students, going from a 4-point disadvantage to a 6-point advantage. In mathematics, the mean NCE score increase of HighScope students was 5 points more than that of the comparison students, going from a 1-point advantage to a 6-point advantage.
|
Table 3
|
|
Mean NCE Scores on Grade 4 Standardized Tests at Baseline and After HighScope Support
|
|
Site |
# of Cases
|
HighScope School
|
|
Comparison School
|
Difference Between Increases
|
|
Baseline Year
|
Last Year of Support
|
Increase
|
|
Baseline Year
|
Last Year
|
Increase
|
|
Reading
|
|
7 |
44
|
40.2
|
50.8
|
10.6
|
|
44.2
|
45.3
|
1.1
|
9.5
|
|
8 |
44
|
46.6
|
51.3
|
4.7
|
|
-
|
-
|
-
|
-
|
|
Meana |
44
|
43.4
|
51.1
|
7.7
|
|
-
|
-
|
-
|
-
|
|
Mathematics
|
|
7 |
44
|
49.4
|
58.0
|
8.6
|
|
48.4
|
52.0
|
3.6
|
5.0
|
|
8 |
44
|
53.1
|
55.9
|
2.8
|
|
-
|
-
|
-
|
-
|
|
Meana |
44
|
51.2
|
57.0
|
5.7
|
|
-
|
-
|
-
|
-
|
Table 4 presents the improvements in percentages of low-income students passing state tests given at the end of grade 4 at 3 of the 6 HighScope schools from the baseline year to the last year of HighScope support.
-
In reading after HighScope support, 16 percentage points more students passed the test than at baseline (85% vs. 70%, χ2 = 11.02, p < .01).
-
In mathematics after HighScope support, 27 percentage points more students passed the test than at baseline (86% vs. 59%, χ2 = 28.51, p < .001).
Table 4 also compares the percentage of low-income students passing state tests at HighScope schools with their matched comparison schools, at the baseline year and at the last year of the period of support to the HighScope schools. At HighScope and comparison schools combined, the number of students compared was about two-thirds of the total number of students taking the test.
-
In reading after HighScope support, 16 percentage points more HighScope than comparison low-income students passed the test (85% vs. 69%, χ2 = 9.64, p < .01), up from virtually no group difference at baseline (69.5% vs. 69.0%, χ2 = 0.01, p > .10).
-
In mathematics after HighScope support, 29 percentage points more HighScope than comparison low-income students passed the test (86% vs. 66%, χ2 = 15.77, p < .001), overcoming an 8-percentage-point disadvantage at baseline (59% vs. 67%, χ2 = 1.52, p > .10).
|
|
|
Percent of Low-Income Students Passing Grade 4 State Tests at Baseline and After HighScope Support
|
|
Site
|
# of Cases
|
HighScope School
|
|
Comparison School
|
Difference Between Increases
|
|
Baseline Year
|
Last
Year of Support
|
Increase
|
|
Baseline
Year
|
Last
Year
|
Increase
|
|
Reading
|
|
1 |
98
|
55.4%
|
72.9%
|
17.5
|
|
38.9%
|
41.0%
|
2.1
|
15.4
|
|
5 |
38
|
71.0%
|
95.0%
|
24.0
|
|
54.0%
|
84.0%
|
30.0
|
-6.0
|
|
6 |
145
|
79.0%
|
90.6%
|
11.6
|
|
87.9%
|
85.3%
|
2.6
|
9.0
|
|
Meana |
94
|
69.5%
|
85.2%
|
15.7
|
|
69.0%
|
69.4%
|
0.4
|
15.3
|
|
Mathematics
|
|
1 |
98
|
66.1%
|
84.7%
|
18.6
|
|
63.9%
|
48.7%
|
-15.2
|
33.8
|
|
5 |
38
|
42.0%
|
60.0%
|
18.0
|
|
38.0%
|
67.0%
|
29.0
|
-11.0
|
|
6 |
129
|
58.1%
|
94.3%
|
36.2
|
|
75.8%
|
76.5%
|
0.7
|
35.5
|
|
Meana |
88
|
58.9%
|
85.8%
|
26.9
|
|
66.7%
|
64.6%
|
-2.1
|
29.0
|
|
aMeans are calculated by numbers of cases per group and cannot be calculated from the information presented in this table. |
Table 5 presents school performance on grade 4 achievement tests for 4 HighScope schools and their matched comparison schools in the time since HighScope support ended, at the end of the period of HighScope support and this year — 5 years later for site 1, 4 years later for sites 6 and 10, and 3 years later for site 9. These statistics show how much test scores at HighScope schools continued to improve during the years after HighScope provided support but the model continued to be practiced, an indicator of how well the school staff remained motivated in their effective use of HighScope in the aftermath of direct support. The findings for percentages of students passing state tests are as follows:
-
In reading this year, 89 percent of the students in HighScope schools passed the test, 9 percentage points more than the 80 percent at the end of HighScope support (χ2 = 6.39, p < .05) and for 2 of the schools, a total of 33 percentage points from baseline to this year (χ2 = 27.52, p < .01).
-
In mathematics this year, 86 percent of the students in HighScope schools passed the test, 10 percentage points more than the 76 percent at the end of HighScope support (χ2 = 6.84, p < .01) and for 2 of the schools, a total of 51 percentage points from baseline to this year (χ2 = 77.32, p < .001).
Table 5 also presents the percentage of students passing state tests at the HighScope schools in comparison with the matched comparison schools for these HighScope schools, at the end of HighScope support and this year.
-
In reading as of this year, the 3 HighScope schools had improved by 6 percentage points more than their comparison schools since HighScope support ended (89% vs. 82%, χ2 = 4.14, p < .05), making the total improvement for 2 of them 20 percentage points more than the comparison schools (χ2 = 7.44, p < .01).
-
In mathematics this year, the 3 HighScope schools had improved by 12 percentage points more than the comparison schools since HighScope support ended (86% vs. 74%, χ2 = 8.29, p < .01), making the total improvement for 2 of them 47 percentage points more than the comparison schools (χ2 = 19.05, p < .001).
Table 5 also presents mean NCE scores for one site's standardized achievement tests give at grade 4. On reading this year, the mean NCE score of students at the HighScope school was 43 — 14 points higher than at the end of HighScope support. On mathematics this year, the score was 47 — 21 points higher than at the end of HighScope support. Versus the comparison school, the HighScope school improved another 19 points in reading and 6 points in mathematics in the time since HighScope support ended.
|
Table 5
|
|
Grade 4 Achievement Test Performance from Last Year of HighScope Support to This Year
|
|
Site
|
# of Cases
|
HighScope School
|
|
Comparison School
|
Difference Between Increases
|
|
Last Year of Support
|
This Year
|
Increase
|
|
Last Year
|
This Year
|
Increase
|
|
Percent Passing State Test
|
|
Reading
|
|
1 |
98
|
55.4%
|
72.9%
|
17.5
|
|
38.9%
|
41.0%
|
2.1
|
15.4
|
|
6 |
179
|
91.0%
|
95.3%
|
4.3
|
|
88.9%
|
90.5%
|
1.6
|
2.7
|
|
9 |
136
|
83.0%
|
96.0%
|
13.0
|
|
88.0%
|
93.0%
|
5.0
|
8.0
|
|
Meana |
141
|
79.7%
|
88.8%
|
9.1
|
|
78.4%
|
81.5%
|
3.1
|
6.0
|
|
Mathematics
|
|
1 |
98
|
66.1%
|
84.7%
|
18.6
|
|
63.9%
|
48.7%
|
-15.2
|
33.8
|
|
6 |
179
|
91.0%
|
95.5%
|
4.5
|
|
77.8%
|
82.3%
|
4.5
|
0.0
|
|
9 |
136
|
57.0%
|
69.0%
|
12.0
|
|
71.0%
|
80.0%
|
9.0
|
3.0
|
|
Meana |
140
|
75.6%
|
85.8%
|
10.2
|
|
72.7%
|
74.1%
|
-1.7
|
11.9
|
|
Site 10: Mean NCE Score on Stanford Achievement Test
|
|
Reading |
87
|
29.0
|
43.0
|
14.0
|
|
55.0
|
50.0
|
-5.0
|
19.0
|
|
Math |
96
|
26.0
|
47.0
|
21.0
|
|
39.0
|
54.0
|
15.0
|
6.0
|
|
aMeans are calculated by numbers of cases per group and cannot be calculated from the information presented in this table. |
Table 6 presents the number of discipline referrals at four HighScope schools during the period of HighScope support. Although only some data points are available, the table shows that discipline referrals have decreased 48 percent at these schools during the time of HighScope support. Three of the schools have these data only for 2 years; at the one site with data for 3 years, the decrease in discipline referrals was 79 percent — 37 percent from the first to the second year, and another 66 percent from the second to the third year.
|
Table 6
|
|
Discipline Referrals During Period of HighScope Support
|
|
School
|
Baseline
Year
|
First
Year
|
Second
Year
|
Third
Year
|
Decrease
|
|
3
|
-
|
2,734
|
1,722
|
585
|
78.6%
|
|
4
|
364
|
249
|
-
|
-
|
31.6%
|
|
5
|
-
|
-
|
100a
|
73a
|
27.0%
|
|
7
|
-
|
1,035
|
477
|
-
|
53.9%
|
|
Meana
|
-
|
-
|
-
|
-
|
47.8%
|
|
aSpring semester only. |
Table 7 presents the results of a survey of 28 teachers' perceptions about HighScope support at 3 HighScope schools — 33 items addressing HighScope's responsibility and flexibility, commitment to the reform model, training quality, model implementation, materials, distance support and networks, standards, local support, and classroom management. With all items positively worded, means of 31 percent of the teachers strongly agreed, 46 percent agreed, 12 percent were neutral, 3 percent disagreed, 1 percent, strongly disagreed, and 8 percent did not know how to respond. Put another way, 76 percent of the teachers were positive about the HighScope effort and only 4 percent were negative.
|
Table 7
|
|
Teachers' Perceptions of HighScope Support at Three Schools
|
|
Item
|
Strongly Agree
|
Agree
|
Neutral
|
Disagree
|
Strongly disagree
|
Don't Know
|
|
HighScope's Responsiveness and Flexibility
|
|
HighScope has successfully adjusted its services to my school's particular circumstances. |
21%
|
64%
|
11%
|
0%
|
0%
|
4%
|
|
HighScope achieved a good match between the trainer and the school. |
54%
|
43%
|
4%
|
0%
|
0%
|
0%
|
|
HighScope adequately supports teachers at different levels of experience and expertise. |
21%
|
64%
|
11%
|
0%
|
0%
|
4%
|
|
Commitment to the Reform Model
|
|
At our earliest meetings, achieving buy-in from school staff was a high priority to the HighScope consultants. |
32%
|
43%
|
4%
|
0%
|
4%
|
18%
|
|
HighScope trainers were able to bring skeptical staff into a productive relationship with the school reform effort. |
18%
|
39%
|
21%
|
7%
|
4%
|
11%
|
|
HighScope trainers were able to bring administrators into a productive relationship with the school reform effort. |
36%
|
50%
|
7%
|
0%
|
0%
|
7%
|
|
As a result of our work with HighScope, parents feel more empowered to be present and active in the school. |
14%
|
46%
|
21%
|
0%
|
0%
|
18%
|
|
Training Quality
|
|
HighScope has offered enough training to successfully implement the model in most of my school's classrooms. |
25%
|
68%
|
0%
|
4%
|
4%
|
0%
|
|
The training provided me with solid grounding in the theory and research upon which the model is based. |
43%
|
54%
|
4%
|
0%
|
0%
|
0%
|
|
The training provided staff with an opportunity to integrate the model with the existing curriculum in our school. |
36%
|
57%
|
4%
|
4%
|
0%
|
0%
|
|
HighScope trainers have adequate experience at the elementary level. |
39%
|
57%
|
4%
|
0%
|
0%
|
0%
|
|
HighScope trainers are flexible and knowledgeable enough to respond to school-specific issues during the training. |
32%
|
61%
|
7%
|
0%
|
0%
|
0%
|
|
Model Implementation
|
|
As a result of adopting the model, our staff talk more about how to do high-quality instruction. |
15%
|
54%
|
21%
|
4%
|
0%
|
7%
|
|
The in-class observation-feedback process is effective. |
29%
|
46%
|
14%
|
0%
|
7%
|
4%
|
|
HighScope has an effective method for monitoring the level of implementation in classrooms. |
14%
|
43%
|
25%
|
4%
|
0%
|
14%
|
|
Staff who make an effort receive enough encouragement from trainers to get the model implemented in their classrooms. |
36%
|
57%
|
4%
|
0%
|
0%
|
4%
|
|
Materials
|
|
HighScope materials are up to date. |
43%
|
36%
|
14%
|
0%
|
4%
|
4%
|
|
HighScope provided the knowledge needed to implement the model, given the materials that were available to us at our school. |
36%
|
57%
|
7%
|
0%
|
0%
|
0%
|
|
HighScope trainers informed staff about non-HighScope materials that also support the model. |
39%
|
36%
|
11%
|
0%
|
0%
|
14%
|
|
Distance Support and Networks
|
|
School staff had adequate support from HighScope via phone, e-mail, and website during the contract period. |
14%
|
39%
|
29%
|
4%
|
0%
|
14%
|
|
HighScope provides opportunities (not necessarily funding) to meet outer teachers and see other schools that are implementing the model. |
21%
|
43%
|
21%
|
4%
|
0%
|
11%
|
|
Standards
|
|
The HighScope model can be successfully linked to state standards and benchmarks. |
50%
|
43%
|
4%
|
0%
|
0%
|
4%
|
|
HighScope training dedicated sufficient time to aligning the new model with state standards, benchmarks, and achievement tests. |
50%
|
25%
|
11%
|
0%
|
4%
|
11%
|
|
Local Support
|
|
Staff have had adequate support from the school principal to meet implementation goals. |
61%
|
39%
|
0%
|
0%
|
0%
|
0%
|
|
School staff have had adequate support from district/agency level administrators to meet implementation goals. |
25%
|
25%
|
14%
|
25%
|
7%
|
4%
|
|
Adequate out-of-class time was provided to get the most out of classroom feedback sessions with the trainer. |
36%
|
36%
|
14%
|
7%
|
4%
|
4%
|
|
Sufficient materials have been made available to implement the HighScope model in classrooms. |
36%
|
39%
|
14%
|
4%
|
7%
|
0%
|
|
School support staff have been asked to participate in the training or given guidance on how to adopt the model to their own roles. |
43%
|
46%
|
4%
|
4%
|
0%
|
4%
|
|
Classroom Management
|
|
Since implementing the HighScope model, I have fewer out-of-class disciplinary referrals. |
18%
|
46%
|
25%
|
4%
|
0%
|
7%
|
|
By keeping my students more engaged in their work, use of the HighScope model has reduced discipline issues in my classroom. |
18%
|
50%
|
21%
|
4%
|
0%
|
7%
|
|
In our school, classrooms with the highest level of implementation have the fewest discipline problems. |
25%
|
21%
|
14%
|
0%
|
0%
|
39%
|
|
It is important for staff to develop patterns of collaboration and decision-making that parallel the student model. |
36%
|
57%
|
4%
|
0%
|
0%
|
4%
|
|
Student attendance has improved since we began using the HighScope model. |
4%
|
25%
|
14%
|
4%
|
0%
|
54%
|
|
Note. The number of cases equals 28 for all items listed. |
|
|
