Credit: Randall Munroe - xkcd.com |
Roughly speaking, the effect sizes used in Visible Learning are the amount of improvement a student would make in a year scaled to one standard deviation on a standardized test. By mapping all effects onto a common effect size scale you can compare the relative value of different techniques and theories.
Among Hattie's observations is the following:
Almost everything works. Ninety percent of all effect sizes in education are positive. Of the ten percent that are negative, about half are "expected" (e.g., effects of disruptive students); thus about 95 percent of all things we do have a positive influence on achievement. When teachers claim that they are having a positive effect on achievement or when a policy improves achievement this is almost a trivial claim: virtually everything works. One only needs a pulse and we can improve achievement. (Hattie, Visible Learning, p. 15)On Hattie's scale, a child simply living for a year with no schooling achieves an effect size of 0.15. "Maturation alone can account for much of the enhancement of learning." Being present in a classroom with a teacher results in effect sizes between 0.15 and 0.40. So, for an innovation to be interesting, it must result in an effect size substantially higher than 0.40. (Hattie, p. 16).
From the book, here are some selected influences with their rank and effect sizes.
Rank | Domain | Influence | Effect Size |
---|---|---|---|
1 | Student | Self-report grades | 1.44 |
2 | Student | Piagetian programs | 1.28 |
3 | Teaching | Providing formative evaluation | 0.90 |
4 | Teacher | Micro teaching | 0.88 |
5 | School | Acceleration | 0.88 |
6 | School | Classroom behavioral | 0.80 |
7 | Teaching | Comprehensive interventions for learning disabled | 0.77 |
8 | Teacher | Teacher clarity | 0.75 |
9 | Teaching | Reciprocal teaching | 0.74 |
10 | Teaching | Feedback | 0.73 |
11 | Teacher | Teacher-student relationships | 0.72 |
22 | Curricula | Phonics instruction | 0.60 |
25 | Teaching | Study skills | 0.59 |
29 | Teaching | Mastery learning | 0.58 |
31 | Home | Home environment | 0.57 |
32 | Home | Socioeconomic status | 0.57 |
42 | School | Classroom management | 0.52 |
45 | Home | Parental involvement | 0.51 |
51 | Student | Motivation | 0.48 |
56 | Teacher | Quality of teaching | 0.44 |
59 | School | School size | 0.43 |
62 | Teaching | Matching style of learning | 0.41 |
81 | Student | Drugs (e.g. for ADHD) | 0.33 |
91 | School | Desegregation | 0.28 |
92 | School | Mainstreaming | 0.28 |
100 | Teaching | Individualized instruction | 0.23 |
106 | School | Class size | 0.21 |
107 | School | Charter schools | 0.20 |
129 | Curricula | Whole language | 0.06 |
133 | School | Open vs. traditional | 0.01 |
134 | School | Summer vacation | -0.09 |
135 | Home | Welfare policies | -0.12 |
136 | School | Retention | -0.16 |
137 | Home | Television | -0.18 |
138 | School | Mobility | -0.34 |
There's a ton of stuff to chew on here. Far more than I can do justice in a blog post. Hattie has between one half and five pages for each of the 138 effects and there is nuance that the numbers don't capture. I'll just make a few observations:
- The top five influences all involve adapting the experience according to individual student needs.
- Charter schools, something I favor, have an unimpressive effect size of 0.20. But charters were intended to enable experimentation. So we should expect them to average similar to conventional public schools but with a much larger standard deviation. Recent studies seem to confirm that expectation. And studies are starting to identify what factors distinguish the high-performing charters from other schools.
- Smaller schools help somewhat while the impact of smaller classes is minimal. That's probably because most small-class initiatives dilute their impact by with a consequential reduction in teacher experience.
- Feedback loops, among my favorite topics, appear at #10 with an effect size of 0.73.
- Home and socioeconomic status have a huge impact. But other factors are bigger so it should be possible to overcome the achievement gap in the school.
- Phonics Instruction has an effect size of 0.60 while Whole Language has one tenth that effect. There's much to be said for Whole Language and I tend to agree with its constructivist roots but not at the expense of phonics.
Of course, the observation that nearly everything works doesn't eliminate the other perils of false positive science and the correlation/causation fallacy. All three of these make it possible to latch on to ones's favorite intervention while claiming to be evidence driven. To defend against this, we must seek 2-5 times improvement in learning performance and replicable results. It also helps to be careful, honest and humble.
I'm really glad you gave me a summary of this research. It makes me wonder how many things that have been touted as "proven in the classroom" were just marginal improvements. I'm really glad someone is quantifying the effects of different influences.
ReplyDelete