Failure to Disrupt, by Reich

Wednesday October 28, 2020

As is especially salient during a pandemic, "technology alone can't transform education." Reich's book nicely wraps up the most recent edtech hype cycle, which I was more optimistic about at the time. Reich thinks we have to keep tinkering toward utopia.

I mostly agree with Reich, except that now I tend to put more weight on knowledge à la Hirsch, and I still have hope for the application of tools like Anki in educational settings.

cover

Chapter 1: Instructor-Guided Learning at Scale: Massive Open Online Courses

"To a first approximation, there are only two primary approaches to teaching and learning. As Plutarch wrote in "On Listening" in the first century CE, "Education is not the filling of a pail, but the lighting of a fire."" (page 22)

Reich then identifies "instructionism" and "social constructivism" as names for these approaches, associated with Edward L. Thorndike and John Dewey, respectively.

One thing I thought I noticed in edtech over the years was a trend in which companies that focused on algorithms (like Knewton) moved toward being more like content companies over time. In a sense they started as pail companies, but found people wouldn't buy empty pails. Might be related to the adoption problem.

This is a fun metaphor to play with. I think it might be better as "not the stacking of wood, but the lighting of a fire." When a fire is burning, you want to keep getting more wood. The wood is largely the same in both approaches; the difference is the motivation or application around it.

"... teaching how to reason from evidence is one of the main purposes of higher education, ..." (page 27)

"People develop self-regulated learning strategies through direct instruction and practice, often through a long apprenticeship in formal educational systems." (page 36)

"I have jokingly summarized the bulk of MOOC learning research as proving Reich's Law: "People who do stuff do more stuff, and people who do stuff do better than people who don't do stuff."" (page 39)

See also:

This is a slight contradiction to the above in that it says some stuff is better than other stuff, but still a nice reference:

"It turns out that one can have terabytes of data about what people do online and very little understanding of what changes inside their heads." (page 39)

"Since we know that most people who register for a course drop out soon after enrollment, we start our courses with short "capsule" units that summarize the most important ideas of the course." (page 41)

"Social inequality is a tenacious feature of educational systems." (page 42)

Chapter 2: Algorithm-Guided Learning at Scale: Adaptive Tutors and Computer-Assisted Instruction

"... a feature of logistic curves is that when plotted against log-transformed x and y axes, the S-curve becomes a linear straight line." (page 60)

This doesn't seem true. Maybe it means that with a log y axis, the left and right sides are each approximately linear? I tweeted at Reich about this.

"Reading instructors sometimes discuss a transition, which happens in about the third grade, from learning to read–learning how to decode the sounds and meaning of text–to reading to learn–using reading to advance content knowledge." (pages 63-64)

"Ritter's research team looked at how much adaptive mastery learning teachers actually allowed in their classes, and they found that some teachers assigned work in Carnegie Learning in such a way that it wasn't really personalized–these teachers required students to work on problem sets related to the topics being taught at that moment to the whole class. By contrast, other teachers allowed students to work at their own pace, even if this meant that some students were still doing practice problems on topics that might have been covered in class weeks earlier. Ritter's team found that overall learning gains were higher in the classes where students were allowed more opportunities to move at their own pace; in other words, the teachers who used Carnegie Learning as intended had more learning gains in their classrooms than did the teachers who kept their students moving in lockstep." (page 70)

Makes me think about whole-group single track, tracking, personalized via computer, tutoring, and how these are thought about in different ways...

Chapter 3: Peer-Guided Learning at Scale: Networked Learning Communities

"The consensus about the great potential of personalized learning depended on a stark disagreement about what the term actually meant." (page 77)

Roughly, this is referring to personalizing pace (go through the same material but at whatever speed you like) versus personalizing content (choose your own area of interest, pursue it however you want).

"... a shift in identity where the learning activity is a part of who they are rather than something that they do ..." (page 91)

On page 95, a reference to Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching:

"The advantage of guidance begins to recede only when learners have sufficiently high prior knowledge to provide “internal” guidance." (from the abstract)

"Put simply, people have a limited working memory, and when learners allocate that working memory to solving a problem, they often are not permanently encoding learning about the patterns and practices that let them solve that type of problem." (page 95)

Reich raises the issue of people choosing to get involved in bad stuff like hate groups. So total freedom in what to get into isn't necessarily great.

"Somehow, millions of people around the world find ways to teach and learn with one another online without any formal training and sometimes without any organization, and yet when designers try to create these kinds of environments with intention, they encounter substantial challenges with motivation and comprehension." (page 102)

Is he deliberately avoiding the obvious interpretation from selection/observation bias? In a whole world, some people have motivation and comprehension to do great things. If you have a fixed group of students that isn't skimming those people, many won't. It seems highly likely that peer-guided learning at scale has the same issue as other MOOCs, in that the people who benefit are overwhelmingly people who are already doing really well to begin with.

Thinking about the various ways to do whole class together vs. totally personalized/independent learning... When I was teaching, "differentiation" was the popular term for customizing class activities for students who were struggling or way ahead...

I think really, there's some stuff that we want all students to learn, and there are learning experiences that make sense to do with a whole class: stuff about working in groups, communication, etc. It would be neat to have more explicit separation between a small whole-group set of content, and scaffolding for students to study more things on their own.

Chapter 4: Testing the Genres of Learning at Scale: Learning Games

The difficulty of transfer (mentioned page 109) makes me wonder to what extent transfer is related to general intelligence - or whether they might even be largely the same thing.

The referenced Critical Thinking: Why Is It So Hard to Teach? (web, more complete PDF) is quite good.

"A more pessimistic view built upon the research on transfer is that people who spend a lot of time playing Minecraft will primarily learn about playing Minecraft." (page 119)

But: some reasonable notes on, for example, the general strategy of knowing to look things up online.

"Domain-independent skills are slightly useful in lots of different domains but not deeply useful in any particular domain." (page 120)

"an edtech Matthew effect is quite common: that new technologies disproportionately benefit learners with the financial, social, and technical capital to take advantage of new innovations." (page 125)

This Zoombinis game seems kind of neat? Maybe mathier puzzles than other puzzle games? Or not? Shrug?

Chapter 5: The Curse of the Familiar

"The dilemma that Resnick and colleagues face as Scratch is widely adopted in schools is that a variety of typical school structures are inimical to the pedagogical philosophy behind Scratch." (page 134)

I think Reich is neglecting something that Scratch has in common with most edtech, and which he comments on for other types: Learning with Scratch depends hugely on the student being prepared for and/or having characteristics that align with learning with Scratch.

"Another approach from the Scratch team has been to develop decks of physical coding cards ..." (page 138)

Published by No Starch.

"... Getting Unstuck, a twenty-one-day email-based course during which participants received daily creative challenges, ..." (page 140)

From Harvard. Aha! They have strategies for getting unstuck!

"Education technology is a good field for those who see themselves as patient optimists." (page 147)

Chapter 6: The Edtech Matthew Effect

"Sociologist Tressie McMillan Cottom argues that technologists often imagine their students as "roaming autodidacts," which she describes as "a self-motivated, able learner that is simultaneously embedded in technocratic futures and disembedded from place, culture, history, and markets."" (page 159)

Chapter 7: The Trap of Routine Assessment

"Autograders excel at assessing routine tasks. These are exactly the kinds of tasks that we no longer need humans to do." (page 171)

This is incomplete. There are lots of basic things that humans need to be able to do that can be assessed automatically. But I agree that higher-level skills are difficult to assess automatically.

Reich makes an example of a plumber who installs a water heater with an LED panel and then says on page 175: "What would it be like to explain to the young men in that apprenticeship program that fifty years later, plumbers would be computer programmers?" This isn't the case. The plumber who in fact installed his water heater didn't do any programming. This was a bad example for the point I think Reich hoped to make.

"... the "reification fallacy," when we uncritically believe that something's name accurately represents what that thing actually is." (page 176)

Somewhat unusual way of explaining this. Interesting.

"Computers can mostly assess what computers are good at doing, and these are things we do not need humans to do in the labor market." (page 197)

I already commented on this, but I'll comment again: I think he's ignoring the many things that humans should be able to do that computers can also do. I'm thinking mostly of simple recall, where knowing something by heart (immediately) is way better than being able to look it up (if you even know that it exists, to look up).

Chapter 8: The Toxic Power of Data and Experiments

Mostly, I think Reich is pulling punches in this chapter. He doesn't go where I thought he was going to: He doesn't really address possible abuses of student data or the negative effects that can arise (intentionally or accidentally) from data and experiment.

"If online learning leads to better outcomes from students, it will be through the amalgamation of a thousand or ten thousand studies like this one that incrementally accrete a knowledge base about effective online learning." (page 223)

I think Reich takes his "tinkering" a little too far sometimes. If what's really needed is a bigger leap, ten thousand incremental experiments won't necessarily get there.

Conclusion: Preparing for the Next Learning-at-Scale Hype Cycle

"In 1913, Thomas Edision declared that the age of books was about to give way to the age of motion pictures. He told an interviewer, "Books will soon be obsolete in the public schools. Scholars will be instructed through the eye. It is possible to teach every branch of human knowledge with the motion picture. Our school system will be completely changed inside of ten years." When Edison's ten-year prediction failed to come to pass, he simple gave himself more time. In 1923, speaking before the Federal Trade Commission, Edison explained, "I made an experiment with a lot of [motion] pictures to teach children chemistry. I got twelve children and asked them to write down what they had learned, from the [motion] pictures. I was amazed that such a complicated subject as chemistry was readily grasped by them to a large extent through [motion] pictures. The parts of the [motion] pictures they did not understand I did over and over again until they finally understood the entire [motion] picture. I think motion pictures have just started and it is my opinion that in 20 years children will be taught through [motion] pictures and not through textbooks."" (pages 229-230)

It was worth reading the book just for these quotes, for this video-hater.

Quote Investigator has the receipts, as usual. And you can read the newspaper for the second quote.

Reich mention's Khan's 2011 Let's Use Video to Reinvent Education on page 231 as well.

"The students who tend to thrive in these kinds of environments are those who have already demonstrated academic proficiency, since most people develop self-regulated learning through an apprenticeship in the formal education system." (page 235)

(As also seen on page 36.)

"... step change is what continuous, incremental change looks like from a distance." (page 245)