Fear of failure

Jonathan Rasmussen in Love it when you're proven wrong argues for the benefits of being proven wrong. As a strong advocate of learning from failure, I can't argue with the benefits.

Furthermore, fear of failure is a big problem in agile teams. It occurred in 17 of 17 organizations surveyed in People over Process (PDF) by Conboy et al. in the July/August 2011 IEEE Software. [One of the best such survey articles I've seen by the way. Seek it out for that and the other 8 challenges they discovered.]

But I don't think seeing the learning benefits addresses the real problem. The big negatives of being wrong are social -- embarrassment, loss of face, loss of future ability to influence decisions, etc. The individual cognitive gains of learning from failure are swamped by the perceived social costs.

That's why to enable learning from failure in my courses, I have most learning occur in private extended homework activities and one-on-one in-depth critiquing exchanges.

Getting team members to embrace individual failure without negative consequences is really hard. If only we could get developer teams to emulate improv groups. In a profile in the Northwestern alumni magazine, Stephen Colbert described the event that clinched for him why he wanted to work in comedy and improv, not serious theater.

I saw someone fail onstage — terribly, massively fail onstage,” Colbert recalls. “And we backstage laughed so hard at this woman’s failure, and our laughter was so joyful and not derisive. I remember turning to a friend of mine, Dave Razowsky, and we threw our arms out wide and hugged each other in laughter and literally fell to the ground in each other’s arms over the joy of that failure.

5 Whys, Harder than it looks, Part 1: What's a Failure?

An exercise I gave my software project management class at Northwestern this year is doing 5 Whys process improvement analysis, as in this video and essay from Eric Ries. I also strongly recommend Tony Ford's experience report and example analysis.

There were three parts to the exercise:

• identifying some key failures
• developing a 5 Whys causal chain for each failure
• proposing process improvements to address each step in each chain

I showed the Ries video, assigned the Ries essay, and then had them do similar analyses on several failures in the project they had just finished. I did these as critiqued exercises, figuring that it would take a few iterations to get right. I was right about the need for iterations, way off on the "few" part. Each part has been quite hard for most of the students.

First up:

What is a 5YA failure?

A project has lots of failures, but only some are appropriate for a 5 Whys analysis. Both Ries and Ford uses examples involving some form of web site failure. Here are some of the things my students submitted and the critiques they got. There were many similar examples of each type of problem.

• Our team had to implement image upload three times

I classify ones like this as as "bad choices led to rework." Clearly something you'd like to avoid in the future, but is it a 5YA (5 Why Appropriate) failure? I say no, because there's no user story failure. A 5YA failure is a user failure, like "the web site crashed" or "the search function returned 'page not found' errors." Without knowing what user stories were impacted and how (broken, missing, slow, ....), you have no idea how to prioritize the importance of the failure and therefore how much effort to expend in trying to avoid it in the future. Furthermore, making things run smoother is much less motivating than avoiding embarrassing public failures.

• Our client changed the design of the website twice

There were a lot of these "requirement changes led to rework." This has three fundamental things wrong with it. The first problem is the same as the previous example. There's no specific user story impact. Second, this is not a developer failure. You can't fix what you can't control. What you need to fix is how your process handles the things you don't control. It's not a 5YA failure if your cloud service dies. It is a 5YA failure if you have no backup. Third, and most important, this may not even be a failure at all. Iterating on designs is a good thing, if driven by actual usage. The whole point of agile is that it's impossible to get it all correct up front. The corollary is that sometimes we'll get it wrong. That's not a failure. That's life.

• XXX was working on a fork of the repository so almost none of his code got integrated

I leave what's wrong here as an exercise for the reader. It's one of the points already made.

• The code XXX submitted for "index" view of the "projects" controller didn't work
• Buggy and totally non-working code was checked in

These are just way too broad and non-specific. "Didn't work" doesn't distinguish "nothing happened" from "wrong results" from "error page appeared." The devil is in the details. Ignore those details and the devil remains. Trying to fix the problem "code is buggy" leads to ineffective measures like "test more!" and expensive ineffective measures like "everyone goes to Java camp this summer!" How the user story actually breaks leads to very different analyses, responses and priorities.

A 5YA failure is a bug report

My final recommendation for my class was that a 5YA failure should be writable as a bug report. That is, it should be put in the form "when a [user-type] does [action], [failure event] occurs." Failure events should distinguish between nothing happening, the wrong things being return, errors being returned, and so on.

Agile Education and the CS Principles Course

One of the things about the Agile movement that attracted me from the beginning was how proponents would go back to the core goals of development, like "getting something working that the client really needs," and from that develop practices that turned standard development idea on their head. That resonated with my own experiences in how to attack problems like natural language understanding and the design of learning environments.

I recently gave an internal "brown bag" talk on how several of these upside-down Agile practices could be fruitfully applied to educational design and academic research.

Then I happened across the AP CS Principles course, and specifically their learning outcomes document (PDF). I have great respect for the goal, the people involved, and the effort invested, but that "requirements" document is wrong in so many ways that I felt a need to try and articulate why.

My own approach to designing courses and other learning experiences is based on the 6-question framework developed by Alex Kass, Ray Bareiss, Danny Edelson and myself in response to the many flawed designs we'd seen (or created) over the previous decade at the Institute for the Learning Sciences at Northwestern.

Question #1 in that framework, "What mistakes are people making that matter, and who cares?" is the most critical. It defines what you need to teach. It's where at least half the designs we saw failed, including the CS Principles learning outcomes. Pull out just the statements about what a student will learn to do. Here are two:

• "The student can analyze the effects of computing within economic, social, and cultural contexts."
• "The student can use abstractions and models to solve computational problems and analyze systems."

Try and find any outcomes that a student would say "I can't do that but I'd love to learn how!"

In case it's not clear why this matters, read the UW CSE120 blog on a pilot run of this course. Look where student engagement occurred. Without engagement, nothing else happens when it comes to learning

It's easy to criticize. That's the point of this blog. But it's not enough to criticize. What would a better answer look like? What would happen if I applied the 6-question framework to the course with the same intent as the CS Principles course?

That's my plan for the next several blog posts. Stay tuned.

The Next Revolution in Software Development

Don't miss the boat!

Training seminars in preparation now for the next paradigm shift. More extreme than extreme programming!

I refer of course to the outsourcing of development to even lower life forms than programmers: Algae Software Development.

Alistair Cockburn says: "I am really a blue algae, not a blue-green algae."

Learn how to do rapid web application prototyping in the C Web Engine Emulation Development kit, or C Weed.

Learn what it means to be really off-shore.

Become a Certified Scum Master today!