Designing CS101: Failure-Driven Course Design

In Agile Education and the CS Principles Course, I criticized learning objectives like:

• "The student can analyze the effects of computing within economic, social, and cultural contexts." (Objective 4)
• "The student can use abstractions and models to solve computational problems and analyze systems." (Objective 5)

These appear to have no answer to Question #1 in the 6-question framework: What mistakes are people making that matter, and who cares? What goes wrong when people don't or can't do Objective 4? Outside of essays and exam questions in the CS Principles Course, where does this skill matter? Is there some subgroup for whom this skill is critical? Who and when?

With Objective 5, I can think of some people to whom it might matter: computer scientists and software developers. There are two problems though. First, this is a much narrower group than the CS Principles Course is being developed for. Second, I can only say "might" because I need to see typical examples of developers not using abstractions and models, and what the negative consequences are. What's the evidence that this occurs often enough to be of concern, and is harmful when it does occur?

All of the CS Principles learning objectives have this problem, except those directly about programming, such as

• The student can create a computational artifact through programming.

But the point of CS Principles was that there's more to CS that people should know besides programming.

So can we answer Question #1? Let's step back, as good Agile developers do, and ask "what's the goal of the CS Principles project?" Like many committee and community-built curricula, a number of goals have accrued. A summary of them can be found in the Open Letter to the Computer Science community.

One impetus certainly is the feeling by many in CS that most people, including students, teachers, parents, and so on, think computer science means programming, programming is boring, and therefore students choose not to pursue a career in computer science.

That feels like a real mistake, but Question #1 also asks who cares and why? I can think of 3 major groups:

• employers
• the government
• students

Employers care because they need good developers. But that's back to programming plus other developer skills. Not where the CS Principles initiative arose from.

The government, especially NSF, cares because they want the US to maintain a lead in research in new forms of information technology. That's good. A CS Principles Course about the big ideas, where they came from and where new ones are coming from, would help address a failure they care about.

Students should care. It's in their own best interests to know whether CS is a career they might actually enjoy and do well in.

So we can argue for a course on how to be a software developer or a computer scientist. But the proponents of CS Principles have a broader aim. Peter Denning in a number of essays on the Great Ideas in CS, Jeannette Wing in various essays and talks on Computational Thinking, and others, have said that a big mistake students make is not knowing how to approach problems in science, engineering, and social issues, from a computational perspective.

The relevant parts in the Open Letter are these two bullet points:

• With a concepts-rich curriculum that emphasizes computational thinking and problem solving students taking the course will be better prepared for most careers, given the role that computing plays in most sectors of American life.
• The population at large -- starting with present day high school students -- will become more knowledgeable and cognizant of computing and computational phenomena.

I'm unconvinced. Students certainly need to be tech-savvy. But they are, often more so, when it comes to social media, than most of the faculty in CS departments. But do they really need computational thinking? Where exactly, outside of programming and CS research, does it matter if someone doesn't apply computational thinking?

I can easily see "here's why it matters" arguments for subareas of computational thinking for subgroups of students. Students in biology should know about multi-agent simulations. Students in cognitive psychology should know about cognitive modeling. There are motivating examples of muddled thinking failures in those and other fields due to the inability to think computationally.

But I'm unable to see why poets, playwrights, and plumbers need to know about computational thinking. Without convincing examples of "why does this failure matter?" a course design is doomed. Students will view CS Principles as yet another irrelevant rite of passage, like learning the parts of a neuron, the lifecycle of stars, the dynasties of China, and so on.

So, I'll stick with the answer "students who don't know what CS is about miss out on a chance to find a career they'll really enjoy in software development or computer science." Phew! One question down, five more to go.

Agile Education and the CS Principles Course

One of the things about the Agile movement that attracted me from the beginning was how proponents would go back to the core goals of development, like "getting something working that the client really needs," and from that develop practices that turned standard development idea on their head. That resonated with my own experiences in how to attack problems like natural language understanding and the design of learning environments.

I recently gave an internal "brown bag" talk on how several of these upside-down Agile practices could be fruitfully applied to educational design and academic research.

Then I happened across the AP CS Principles course, and specifically their learning outcomes document (PDF). I have great respect for the goal, the people involved, and the effort invested, but that "requirements" document is wrong in so many ways that I felt a need to try and articulate why.

My own approach to designing courses and other learning experiences is based on the 6-question framework developed by Alex Kass, Ray Bareiss, Danny Edelson and myself in response to the many flawed designs we'd seen (or created) over the previous decade at the Institute for the Learning Sciences at Northwestern.

Question #1 in that framework, "What mistakes are people making that matter, and who cares?" is the most critical. It defines what you need to teach. It's where at least half the designs we saw failed, including the CS Principles learning outcomes. Pull out just the statements about what a student will learn to do. Here are two:

• "The student can analyze the effects of computing within economic, social, and cultural contexts."
• "The student can use abstractions and models to solve computational problems and analyze systems."

Try and find any outcomes that a student would say "I can't do that but I'd love to learn how!"

In case it's not clear why this matters, read the UW CSE120 blog on a pilot run of this course. Look where student engagement occurred. Without engagement, nothing else happens when it comes to learning

It's easy to criticize. That's the point of this blog. But it's not enough to criticize. What would a better answer look like? What would happen if I applied the 6-question framework to the course with the same intent as the CS Principles course?

That's my plan for the next several blog posts. Stay tuned.

Critique-based Assessment

I said that in my programming courses (and others), I'd rather critique than grade, because grades are such a bad idea.

[And I'm not alone. Alfie Kohn's nice post references similar criticisms as far back as 1933!]

But schools want grades, and therefore so do students, so how do I assign grades at the end of a term?

Here's how my critique-based courses work. Instead "do it once, get a grade, repeat," the process is "do, review, re-do until mastered, then move on." More specifically,

• Students select and work on assignments (exercises, projects, ...).
• Students submit their best solutions to each problem.
• I critique each solution, pointing out weaknesses and areas for improvement.
• Unless the critiques are non-trivial, the student needs to re-do and re-submit. This happens for most initial submissions.

Quite quickly students begin separating out. Some zip through the early easy stuff, some need a lot of tries to get the basics down. In a standard course, the latter students would be forced to face more and more complex problems with incomplete skills, while the former students would be bored waiting for the challenging stuff.

At the end of the term (and usually at one or two mid-points), I calculate grades based on

• progress, as measured by the number, range, and complexity of assignments completed to this point,
• mastery, as measured by the lack of serious basic critiques in the solutions initially submitted, and
• effort, as measured by the number of submissions and re-submissions.

During the course I get to focus on what I care about: what problems are students having and what advice should they be given. There's no repeated reduction of wildly different points into singular point values. As a bonus, I get a detailed current picture of what students are having the most trouble with.

When I describe this to other faculty, the usual response is "whoa, that must take forever!" Indeed, without some kind of tool support, it would not be feasible to give detailed feedback like this to a large class, and it would be very time-consuming and error-prone to review and evaluate each student's history of interactions for an entire class.

Fortunately, we have the technology. Next time...

Critiques, not Grades

When teaching programming, I critique code, I don't grade it. Despite their ubiquity, grades have many problems. In no particular order, here are some big ones:

• For any solution complex enough to be worth evaluating, a single number is way too inadequate a measure of quality.
• Calculating a weighted average of inadequate individual grades to get a final grade makes things even worse.
• Who would you rather have working for you: someone who got 5 programs 80% done, or 4 out of 5 programs 100% done?
• A grade provides no useful feedback to the student on what was wrong and how to do better.
• Grading requires repeated arbitrary decisions about the relative demerits of misleading function names, bad indentation, failure to use standard library functions, and so on, when all I really care about is whether is something worth fixing or just noting.
• The students tune their behavior to optimize their grade based on whatever arbitrary rules I developed.

But if I don't grade, only critique, how do I figure out a final grade? Next time...

10 Reasons Why Accessor Functions are Better than Global Variables

Novices often read that accessor functions, such as lineWidth() and setLineWidth(), are better than global variables, such as LINE_WIDTH. But they usually only see one or two reasons why, hardly enough to overcome the clear simplicity of using variables.

So let me try to put the nail in this particular coffin with 10 different reasons.

By global variables here, I include things like Lisp and C global variables, and things like public C++ data members and Java instance variables. By accessor functions, I mean things like Lisp and C functions, things like public C++ and Java getters and setters, and things like properties that some languages support that are "variable-like" in syntax but "function-like" in actual semantics.

1. Readability

To make it clear that some variable is global, most languages have some naming convention to make them stand out, e.g., asterisks as in *current-color* in Lisp, and upper-case as in MAX_DATA_LENGTH in C++ and Java. But such conventions clutter up code badly. [When an experienced Lisper sees code littered with starred variables, her first reaction is "Asterisks! The gall!".]

Accessor functions however are just regular functions and need no such special naming.

2. Read-only access

Often there is global information that the user should be able to access, but not change. By defining and making public only a reader function, you can prevent users from changing information that can not or should not be changed.

3. Write-only access

While less common, sometimes there is information that needs to be specified that should not be readable, by some users at least. changePassword(old, new) comes to mind, here.

4. Uniform access to variable-like data

There's more to a computer than CPU and memory. There are clocks, I/O ports, system host information, and so on. Accessors like currentTime() offer a uniform way to access such information, without worrying about whether such information is stored in a memory location or derived from some other source.

5. Scoped values

Consider output line width. When we set it to 50 characters, do we mean that to apply to all output, including output to files, or just to output to the screen or to some window on the screen? With accessors, it's easy to extend the API
to allow values to be scoped appropriately, e.g., setLineWidth(debugOut, 50), without creating a slew of specialized variables.

6. Safe assignment

Consider line width again. What happens if someone says (setq *line-width* nil) or setLineWidth(-10)? With variables, the chances are nothing happens until later when printing is attempted, at which point an error occurs. With an accessor function, bad values can be caught and prevented or replaced at assignment time.

7. Assignment with special values

Consider line width once more. With an accessor like set-line-width, we can extend the values it accepts to include things like named standardized values, such as (set-line-width :wide) or setLineWidth(WIDE), where "wide"
is not the actual value but a key to look up some standard default value.

8. Assignment by example

Consider date formats. There are many ways dates can be printed: full month names vs. abbreviated month names versus digital months, two-digit vs. four-digit years, month-day-year vs. day-month-year, hyphens vs slashes vs. spaces and commas, etc. A clever way of making this complex combination of choices simple to specify is to allow the user to given an example date, e.g.,
setDateFormat("2/31/95"). This isn't the actual value stored, but a
value that can be used to construct the actual format object.

9. Assignment by parts

Consider default pathnames, e.g., the default pathname for the compiled
binary form of a source code file, or an XSLT-transformed XML file.
Even though it makes sense to store this internally as one pathname, it makes more sense to allow the user to modify pieces of it without worrying about the other parts, e.g., (set-module-binary-pathname :directory "Lisp:").

10. Traceable access

Many development environments, especially for high-level languages, don't make it easy to trace when someone gets or sets the value of a global variable. This is no problem with accessor functions.

The Next Revolution in Software Development

Don't miss the boat!

Training seminars in preparation now for the next paradigm shift. More extreme than extreme programming!

I refer of course to the outsourcing of development to even lower life forms than programmers: Algae Software Development.

Alistair Cockburn says: "I am really a blue algae, not a blue-green algae."

Learn how to do rapid web application prototyping in the C Web Engine Emulation Development kit, or C Weed.

Learn what it means to be really off-shore.

Become a Certified Scum Master today!

Early Client Value is more than just something that works

I'm currently teaching agile development in two very different contexts. One is a course for mid-level managers, most with no software experience. I'm also coaching a team of computer science and business students, doing a project in a course on innovation and entrepreneurship. In both courses, I talk about the need to deliver early client value via slices (my shorthand image for minimal marketable release).

Or, as I usually put it: How soon will you be able to say "Even if they cancel us, we delivered value?"

I contrast developing slices with a "foundation up" approach that works on infrastructure first, rather than some end-to-end deliverable.

Both groups got the idea of getting something working that the client can use, but both made the same mistake in their release and iteration planning. Their first releases were systems that supported basic operations like adding, deleting and displaying items. For a hypothetical system to manage software purchases, users could add records of things purchased and query along various parameters. For a social network site project, users could upload pictures for sharing.

These are end-to-end, and they do make sure things are working, but they are not valuable. These things can be done with existing freely available tools. If the project died tomorrow, the client would not have anything they didn't have already. These are minimal releases, but they are not marketable releases.

Every system has a reason for being, a need that's currently not met. Early value means something that makes some visible progress towards meeting that need. The software purchasing system was originally envisioned as a way to reduce costs by seeing the best prices that divisions in a company had managed to get recently. Therefore, an initial release and early iterations should focus on displaying data, not entering it, and demonstrating possible savings. The social network site first release should show something about sharing pictures or other media not currently possible or easy in existing sites.

Note that in both cases, it may turn out that the original vision is infeasible or much harder than thought. Better to know that early. And, in the purchasing project, the initial release might not need to have any user interface for adding purchase records at all. A table from an existing purchasing database generated by a manual query may provide enough real historical data for the system to fulfill its original mission well enough for a marketable release.