Rapid-iteration teaching, and the evolution of education

This is a guest post written by Marcus Phillips. Marcus is a lead instructor and cofounder at Hack Reactor, “The CS Degree for the 21st Century”

Teaching has been trapped in old, slowly evolving models for hundreds of years now. Suddenly, those models are being completely rethought out on the cutting edge, thanks to recent leaps in technology and high-speed communication. Agile methodologies, borrowed from fields like software engineering, are acting as templates for how these old education models can be improved. These same agile methodologies are being used to quickly experiment on our ideas about how people learn, allowing for faster re-examination of teaching practices than ever before.

At Hack Reactor, we adhere to the idea of “rapid-iteration teaching”, which describes a philosophy of data- and outcome-driven thinking that you can apply to every level of your teaching. Put simply, this paradigm prescribes the use of frequent experimentation, and rapid re-investment of your findings back into the class. This technique has precedents in countless other fields, resembling historical shifts like:

So why aren’t we using science to improve our teaching? Even though so many disciplines have benefited from an increase of critical inquiry, the field of education receives an inordinately limited degree of such rigor. This has historically resulted in slow-moving cycles of improvement that leave students and educators mired in outdated traditions and superstitions, many of which actually impede us on the way to that “aha moment”.

Perhaps the important aspects of education are especially subtle, and more resistant to scrutiny. Perhaps we are not careful to reward and respect our educators when they achieve the sorts of breakthroughs that could change everything. It may even be that pedagogical discoveries are happening every day, but bureaucracy and counter-productive systems of improvement make incorporating those discoveries slow or completely impractical. Whatever the reasons, it seems clear to many of us that our current education system is struggling to keep up with the needs and changes of our growingly complex body of knowledge and an ever-more-scientific world.

Let’s dive into the details of how this methodology plays out in practice. You can apply any degree of scientific rigor you like, but at its core this idea prescribes that you should do two things:

Measure the result of your teaching as frequently as possible, and
Make immediate changes in your approach as a result of your findings.

An implied third step is that you should repeat this cycle as quickly as possible, in an effort to improve your teaching at maximum speed. Whether you go so far as to do careful mathematical analysis that accounts

Rapid-iteration in the classroom

Let’s look on a micro level at how the notion of rapid-iteration teaching dramatically improves the learning outcomes of students. In a typical lecture at Hack Reactor, we use real-time data collection methods to measure student uptake of the ideas we’re presenting to them. This even includes mechanisms for knowing whether or not a point made in lecture has been heard and understood by the whole room. In order to achieve our high standard for speed and consistency of learning, we can’t accept the traditional, one-way, “shout and hope it sticks” model of lecturing. Instead, we actively measure the class’s ability to apply a newly presented idea—about every few minutes, on average. This allows for immediate pivots, repetition, or omission of some part of the lecture, depending on the real, measured needs of the learners. Unlike most lectures I can recall attending, no one in the audience at Hack Reactor expects to skulk off afterwards, reeling from unaddressed confusion and anxiety, vowing to mine their textbook for more context. Everyone remains in lock step, because we’re actually asking them how well we’ve explained things so

far.

This turns out to be one of the biggest leaps forward we see in teaching efficacy, and when you consider the status quo, it becomes obvious why. Looking back on your own classroom experiences, you can probably remember a teacher speaking in front of the room about something you either didn’t understand at all, or had already known before you sat down. In fact, the entire class might have been feeling exactly the same, and none of them would have a way of finding that out. But whether the material being presented was too obvious or not clear enough, the audience’s opportunities to steer the lecture back on track were pretty limited, if they existed at all.

Astonishingly, this experience is pretty universal among students, but almost no one feels comfortable speaking up about what they need in order to make use of a lecture. Inevitably, every listener in the room will miss something along the way. They’ll tune out for a minute due to boredom or confusion, or simply while they take a note and put on a sweatshirt. This tendency is natural and predictable, and it’s a shame that most lectures aren’t tuned to accommodate it. Learning requires repetition, and if a one-way lecturer doesn’t happen to guess the precise amount of repetition necessary, listeners will spend the last three quarters of the talk lost, trying to regain their conceptual footing. You can easily verify this at the next lecture you give by picking someone at random (who doesn’t volunteer to answer), and posing them a simple quiz question on their understanding of a sentence from 45 sentences earlier. Even in a room full of geniuses, I’ve found that the majority of the responses will surprise you with how different their interpretation is from the intended message. As a speaker, you must always be respectful of the fact that tracking on your words is an extremely difficult, messy task, and that speaking unequivocally is just as hard.

But why aren’t lecturers measuring student understanding as they go, and why don’t students demand that they do it? It’s likely that a lifetime of conditioning has taught us A) classrooms are structured for one-way speech, so get used to it, and B) looking dumb is a supreme and ever-present danger. The widespread insecurity we can observe among listeners—who worry that their own question might be completely irrelevant—is really quite fair, given the format of most lectures. Listeners have no visibility into how similar their own confusion might be to that of the rest of their peers. As a result, most lectures are impervious to criticism or improvement, since they and the audience have such limited access to any direct feedback about whether their ideas are actually taking root. In a one-way lecture, whatever content a lecturer supposes in advance will be useful is exactly what gets delivered the day of—with almost no checks, balances, or verifications! This would make for pretty deplorable science, but it passes as pretty standard teaching, and it’s the students who have to make up the gap. Students are usually expected to do a lot of followup research after a one-way lecture, using a pantheon of even more rigid sources of information found at the library. For all but the top 5% of the class who already understood the framework of what was going to be taught, one-way lectures offer countless opportunities to stumble and lose the throughline, at which point the only useful strategy left to a listener is in making lists of jargon words to be looked up later, and hope that the explanations found elsewhere ring a bell.

By contrast, lecturers at Hack Reactor train themselves to pause after every new concept is delivered, and get estimates from the students about how comfortable they feel applying those ideas. And we don’t stop there; we know that listeners can be shy or overconfident, so once the audience claims a certain level of comfort with the material, we do light-weight testing by asking students to apply their understanding to simple scenarios, mid-lecture. Lastly, we know that such an interactive form of lecturing can require more bravery, attentiveness, and trust from students than most of them are used to needing in a classroom, so we do a lot of foundational work in advance to ensure they feel comfortable and safe participating without being judged.

Rapid-iteration in the class materials

Online teaching platforms and hyper-light, in-person programs like ours present exciting new opportunities to gather user experience data and study the learning process up close. Consequently, we’re able to tweak our teaching materials more frequently and more intentionally than traditional settings have ever been able to. At Hack Reactor, we do a “sprint reflection” with our students every two days and incorporate findings back into the program immediately. This rapid iteration loop has been either unavailable or unutilized in more traditional education systems that rely primarily on the occasional standardized test for data about student understanding. Interpreting such coarse measurements correctly and making sound judgements about how students are confused would be challenging, to put it lightly. Furthermore, even when an exceptional teacher in a traditional institution takes care to measure students closely, and notices a real opportunity to improve the materials, they’re often handcuffed from doing so by the sheer size and bureaucracy of their institution.

We rely on our lightweight nature to help us hone the curriculum much faster. One of our sources of rich feedback about student experience comes from our classroom management software, which provides instant visibility into how our curriculum is landing with students, and what sticking points they’re encountering. A big part of our software is a real-time help system, which allows any student to get in-person answers to any sort of question. When a student hits a sticking point of any sort that hasn’t been planned for and explained in advance, they just click the “help” button, and an instructor finds them at their desk. On average, we can respond to student help requests in under a minute, and instructors stay there until the student can independently explain their plan for resolving the problem. The end result is that the student can continue moving at top speed, and the school gets a very clear view into how the project or the support materials could be improved. On a regular basis, the instructors discuss the patterns observed in student help requests, and use those findings to make the curriculum even clearer and more effective.

Feedback goes both ways, though, so we’ve also written sophisticated testing software to increase the amount and quality of feedback we give our students. When a student makes an error solving a Hack Reactor project, they don’t have to wait days or weeks while someone grades it—our software does instant analysis of their project and reports any findings immediately. Then, teachers go back through the projects looking for any additional issues that the computer might have missed. In the process, the teacher has an opportunity to give the student higher-level advice than they would have if the computer wasn’t catching the more basic errors, and they can even improve the computer program so it will detect those higher-level issues in the future. Furthermore, these projects and assessments are a constant part of the curriculum, with students getting new feedback about their work on an hourly basis. Contrast this with the traditional paradigm, where

new grades and feedback on student work are offered weekly in the best case scenario, and semesterly in the worst.

Rapid-iteration in the industry

At the macro level, we should be careful how we decide what “effective” training is, and how we’re judging the different types of programs that we participate in. We should especially value data about student outcomes, as a compass for whether a program is achieving its goals. After all, verifying that you successfully communicated an idea to a listener isn’t too useful, if it turns out the idea was wrong. And even if your ideas were right, did learning them provide your students with the kind of value they actually wanted? The entirety of a course might be best evaluated in terms of the learning outcomes and practical outcomes that it actually leads to. As an administrator, to design a program without keeping constant watch on these outcomes would be flying blind. At Hack Reactor, we keep a close eye on bottom lines like student satisfaction, job placement rates, and starting salaries, and we think all programs should hold themselves to such practical standards. Traditional institutions of learning have come under a lot of scrutiny regarding their price, as well as the cost-benefit analysis of attending them—which could indicate that they’ve lost touch with the outcomes people really want to attain.

So it’s not just lectures, teaching materials, and support systems that need the kind of constant monitoring and tweaking that we practice at H/R. The concept of rapid-iteration teaching even applies to whole institutions. In order to make the kinds of timely changes necessary at every level, students and educators around the world should focus closely the boots-on-the-ground data regarding “who really learned what?” Want to give an effective talk? Measure your listeners’ understanding as you go. Want your students to get a ton out of your materials? Give them frequent updates on how they’re doing, so they can adjust on a dime. Want to design a fantastic institution of learning that changes peoples’ lives tremendously? Measure the outcomes, and pour over them looking for ways you might improve them. Measure these things frequently, and then really do something with the results. But if you don’t bother measuring a thing, you won’t have any hope of improving it.

Conclusion

There have always been ways for schools to improve and streamline their teaching—and the best, most inspired teachers always have been innovators. But their great ideas spread slowly, and are easily drowned out by the noise of industrial-scale education systems. Recent explosions of teachable technologies and information require our systems of teaching to evolve faster than ever, just to keep up.

Lucky for all of us, the internet, new innovations in educational models, and novel modes of communication are allowing great ideas in education to rise above the noise, making possible a pace of learning that has never been seen before. There are countless new formats out there—from online learning platforms to crowd-funded apprenticeship programs. At Hack Reactor, we’re honored to be leading the charge on the rapid-iteration, detail-oriented, in-person class format. We’re researching and implementing the most effective teaching techniques we can find, and using result-driven data to power every decision we make.

—

Learn more about Marcus Phillips, CTO and Head of Instruction at Hack Reactor

Learn more about Hack Reactor.

Learn more about the Twitter Chat on this topic on 2/12/2014.