Critique-based Assessment

I said that in my programming courses (and others), I'd rather critique than grade, because grades are such a bad idea.

[And I'm not alone. Alfie Kohn's nice post references similar criticisms as far back as 1933!]

But schools want grades, and therefore so do students, so how do I assign grades at the end of a term?

Here's how my critique-based courses work. Instead "do it once, get a grade, repeat," the process is "do, review, re-do until mastered, then move on." More specifically,

• Students select and work on assignments (exercises, projects, ...).
• Students submit their best solutions to each problem.
• I critique each solution, pointing out weaknesses and areas for improvement.
• Unless the critiques are non-trivial, the student needs to re-do and re-submit. This happens for most initial submissions.

Quite quickly students begin separating out. Some zip through the early easy stuff, some need a lot of tries to get the basics down. In a standard course, the latter students would be forced to face more and more complex problems with incomplete skills, while the former students would be bored waiting for the challenging stuff.

At the end of the term (and usually at one or two mid-points), I calculate grades based on

• progress, as measured by the number, range, and complexity of assignments completed to this point,
• mastery, as measured by the lack of serious basic critiques in the solutions initially submitted, and
• effort, as measured by the number of submissions and re-submissions.

During the course I get to focus on what I care about: what problems are students having and what advice should they be given. There's no repeated reduction of wildly different points into singular point values. As a bonus, I get a detailed current picture of what students are having the most trouble with.

When I describe this to other faculty, the usual response is "whoa, that must take forever!" Indeed, without some kind of tool support, it would not be feasible to give detailed feedback like this to a large class, and it would be very time-consuming and error-prone to review and evaluate each student's history of interactions for an entire class.

Fortunately, we have the technology. Next time...

Critiques, not Grades

When teaching programming, I critique code, I don't grade it. Despite their ubiquity, grades have many problems. In no particular order, here are some big ones:

• For any solution complex enough to be worth evaluating, a single number is way too inadequate a measure of quality.
• Calculating a weighted average of inadequate individual grades to get a final grade makes things even worse.
• Who would you rather have working for you: someone who got 5 programs 80% done, or 4 out of 5 programs 100% done?
• A grade provides no useful feedback to the student on what was wrong and how to do better.
• Grading requires repeated arbitrary decisions about the relative demerits of misleading function names, bad indentation, failure to use standard library functions, and so on, when all I really care about is whether is something worth fixing or just noting.
• The students tune their behavior to optimize their grade based on whatever arbitrary rules I developed.

But if I don't grade, only critique, how do I figure out a final grade? Next time...

10 Reasons Why Accessor Functions are Better than Global Variables

Novices often read that accessor functions, such as lineWidth() and setLineWidth(), are better than global variables, such as LINE_WIDTH. But they usually only see one or two reasons why, hardly enough to overcome the clear simplicity of using variables.

So let me try to put the nail in this particular coffin with 10 different reasons.

By global variables here, I include things like Lisp and C global variables, and things like public C++ data members and Java instance variables. By accessor functions, I mean things like Lisp and C functions, things like public C++ and Java getters and setters, and things like properties that some languages support that are "variable-like" in syntax but "function-like" in actual semantics.

1. Readability

To make it clear that some variable is global, most languages have some naming convention to make them stand out, e.g., asterisks as in *current-color* in Lisp, and upper-case as in MAX_DATA_LENGTH in C++ and Java. But such conventions clutter up code badly. [When an experienced Lisper sees code littered with starred variables, her first reaction is "Asterisks! The gall!".]

Accessor functions however are just regular functions and need no such special naming.

2. Read-only access

Often there is global information that the user should be able to access, but not change. By defining and making public only a reader function, you can prevent users from changing information that can not or should not be changed.

3. Write-only access

While less common, sometimes there is information that needs to be specified that should not be readable, by some users at least. changePassword(old, new) comes to mind, here.

4. Uniform access to variable-like data

There's more to a computer than CPU and memory. There are clocks, I/O ports, system host information, and so on. Accessors like currentTime() offer a uniform way to access such information, without worrying about whether such information is stored in a memory location or derived from some other source.

5. Scoped values

Consider output line width. When we set it to 50 characters, do we mean that to apply to all output, including output to files, or just to output to the screen or to some window on the screen? With accessors, it's easy to extend the API
to allow values to be scoped appropriately, e.g., setLineWidth(debugOut, 50), without creating a slew of specialized variables.

6. Safe assignment

Consider line width again. What happens if someone says (setq *line-width* nil) or setLineWidth(-10)? With variables, the chances are nothing happens until later when printing is attempted, at which point an error occurs. With an accessor function, bad values can be caught and prevented or replaced at assignment time.

7. Assignment with special values

Consider line width once more. With an accessor like set-line-width, we can extend the values it accepts to include things like named standardized values, such as (set-line-width :wide) or setLineWidth(WIDE), where "wide"
is not the actual value but a key to look up some standard default value.

8. Assignment by example

Consider date formats. There are many ways dates can be printed: full month names vs. abbreviated month names versus digital months, two-digit vs. four-digit years, month-day-year vs. day-month-year, hyphens vs slashes vs. spaces and commas, etc. A clever way of making this complex combination of choices simple to specify is to allow the user to given an example date, e.g.,
setDateFormat("2/31/95"). This isn't the actual value stored, but a
value that can be used to construct the actual format object.

9. Assignment by parts

Consider default pathnames, e.g., the default pathname for the compiled
binary form of a source code file, or an XSLT-transformed XML file.
Even though it makes sense to store this internally as one pathname, it makes more sense to allow the user to modify pieces of it without worrying about the other parts, e.g., (set-module-binary-pathname :directory "Lisp:").

10. Traceable access

Many development environments, especially for high-level languages, don't make it easy to trace when someone gets or sets the value of a global variable. This is no problem with accessor functions.

The Next Revolution in Software Development

Don't miss the boat!

Training seminars in preparation now for the next paradigm shift. More extreme than extreme programming!

I refer of course to the outsourcing of development to even lower life forms than programmers: Algae Software Development.

Alistair Cockburn says: "I am really a blue algae, not a blue-green algae."

Learn how to do rapid web application prototyping in the C Web Engine Emulation Development kit, or C Weed.

Learn what it means to be really off-shore.

Become a Certified Scum Master today!

Early Client Value is more than just something that works

I'm currently teaching agile development in two very different contexts. One is a course for mid-level managers, most with no software experience. I'm also coaching a team of computer science and business students, doing a project in a course on innovation and entrepreneurship. In both courses, I talk about the need to deliver early client value via slices (my shorthand image for minimal marketable release).

Or, as I usually put it: How soon will you be able to say "Even if they cancel us, we delivered value?"

I contrast developing slices with a "foundation up" approach that works on infrastructure first, rather than some end-to-end deliverable.

Both groups got the idea of getting something working that the client can use, but both made the same mistake in their release and iteration planning. Their first releases were systems that supported basic operations like adding, deleting and displaying items. For a hypothetical system to manage software purchases, users could add records of things purchased and query along various parameters. For a social network site project, users could upload pictures for sharing.

These are end-to-end, and they do make sure things are working, but they are not valuable. These things can be done with existing freely available tools. If the project died tomorrow, the client would not have anything they didn't have already. These are minimal releases, but they are not marketable releases.

Every system has a reason for being, a need that's currently not met. Early value means something that makes some visible progress towards meeting that need. The software purchasing system was originally envisioned as a way to reduce costs by seeing the best prices that divisions in a company had managed to get recently. Therefore, an initial release and early iterations should focus on displaying data, not entering it, and demonstrating possible savings. The social network site first release should show something about sharing pictures or other media not currently possible or easy in existing sites.

Note that in both cases, it may turn out that the original vision is infeasible or much harder than thought. Better to know that early. And, in the purchasing project, the initial release might not need to have any user interface for adding purchase records at all. A table from an existing purchasing database generated by a manual query may provide enough real historical data for the system to fulfill its original mission well enough for a marketable release.

The Cardinal Rule of Functions

Functions should be:

• short
• easy to understand
• well-named
• reusable
• maintainable

All these desirable properties follow from obeying the Cardinal Rule of Functions.

One Function to a Function

That is, the code implementing a function should have one and only one task to do. Example tasks are:

• Calculate: using primitive operators, calculate a value; this includes arithmetic, string operations, list building, and so on
• Read or write: get data from or send data to files or users
• Parse or format: convert text into structured data or construct text representations of structured data
• Combine: apply several other functions to a set of arguments
• Dispatch: decide which of several other functions to apply to a set of arguments
• Iterate: repeatedly apply another function to a list of argument values

Notice that the latter tasks correspond to classic control structures (composition of function calls, conditional branching, and iteration). A symptom that the rule has been broken is when several of these control structures appear in one function. Another symptom is when you have trouble keeping a function to less than a dozen lines of code, and/or its lines to less than 72 characters.
Another good sign that the rule has been broken is difficulty naming the function clearly, or using names like scan-and-compress-and-print.

Names (first in a series)

It's hard to over-emphasize the importance names bring to the maintainability of code. They're the first thing a maintainer sees, and they're often the last thing a novice code thinks about. Textbooks provide notoriously bad examples that students follow in the real code.

If you haven't already, spend some time at Roedy Green's How to write unmaintainable code. Note that the longest essay there is on naming.

Unfortunately, there is no formula for generating names, just some good heuristics, and some major "don't"'s, like names that are

• cryptic, e.g., bltx
  
  
• vague, e.g., data
  
  
• verbose, e.g., first-item-to-delete
  
  
• misleading, e.g., num for something that holds a
  list of numbers
  

Good names are:

• short
  
• plain English or conventional
  
• accurate
  

A function name should

• say what a function returns or does, e.g., get-test,
  never how it does it, e.g., search-recursively
• be either a verb-object, e.g., get-test, or
  qualified-verb, e.g., string-sort
• be semantic for application-specific functions, e.g.,
  remove-test-case
• be generic for general utilities,
  e.g., reverse-sequence

Much more to come on this topic.

Addendum: Just to reinforce the importance of naming, see
page 5 of Where the Wild Bugs Are.