Today’s XKCD webcomic entertained me far beyond the allowable limits.
We are no longer friends, indeed.
-
Controlled Flight Into Terrain by Irving Reid is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.
Align Your Metrics
Tuesday, June 26, 2007
An unusually long link-excavation strikes pay dirt: Greg points through Jorge Aranda to Ed Yourdon who, in the course of an article well worth reading, happens to point to an interview with Linda Rising (also worth your time) at InfoQ. The interview was conducted by Deborah Hartmann, a Toronto-based agile development consultant who co-authored a paper at the agile2006 conference called “Appropriate Agile Metrics: Using Metrics and Diagnostics to Deliver Business Value” (.pdf)
And there was the gold.
It’s a short paper; take the time to go read it. It’s a much better version of the argument I tried to make a few weeks ago about making sure that you understand what you’re measuring and why.
Hartmann describes a process based on a single central metric, which must be the best possible (at the time) approximation of the business value generated by the effort and capital invested. Of course, like all things agile, this metric must be continually reassessed and improved.
Everything else you might measure, including things like agile iteration velocity, must be taken as short term “diagnostics” that are only meaningful in their immediate context, and only applicable to the overall goal of maximizing the business value metric.
The “Metric / Diagnostic Evaluation Checklist” on page 4 is worth the paper it’s printed on and then some. If my requisition for a stapler ever goes through, I’ll be attaching copies of that to anyone from the Current Employer’s Project Management Office who stands still for long enough…
What Did the Identity Provider Know?
Wednesday, June 20, 2007
…And when did it know it?
Kim Cameron sums up the reasons why we need to understand the technical possibilities for how digital identity information can affect privacy; in short, we can’t make good policy if we don’t know how this stuff actually works.
But I want to call out one assertion he (and he’s not the only one) makes:
 First, part of what becomes evident is that with browser-based technologies like Liberty, WS-Federation and OpenID,  NO collusion is actually necessary for the identity provider to “see everythingâ€.
The identity provider most certainly does not “see everything”. The IP sees which RPs you initiate sessions with and, depending on configuration, has some indication of how long those sessions last. Granted, that is *a lot* of information, but it’s far from “everything”. The IP must collude with the RPs to get any information about what you did at the RP during the session.
Correlating Identities over Time
Monday, June 18, 2007
Kim Cameron responds to Paul Madsen responding to Kim Cameron, and I wonder what it is about Canadians and identity…
Paul points out that Kim is missing one possible form of collusion, in which a single site correlates repeated visits from an individual even though they don’t know that individual’s name. Kim, in response, asks:
But I have to admit that I have not personally been that interested in the use case of presenting “managed assertions†to amnesiac web sites. In other words, I think the cases where you would want a managed identity provider for completely amnesiac interactions are fairly few and far between. (If someone wants to turn me around me in this regard I’m wide open.)
It turns out that Shibboleth, in particular, has a very clear requirement for this use case. FERPA requires that educational institutions disclose the least possible information about students, staff and faculty to their partners. The example I heard, back in the early days of SAML, was of an institution that had a contract with an on-line case law research provider such that anyone affiliated with the law school at that institution could look up cases.
In this case, the “managed identity provider” (representing the educational institution) needs to assert that the person visiting right now is affiliated with the law school. However, the provider has no need to know anything more than that, and therefore the institution has a responsibility under FERPA to not give the provider any extra information. “The person looking up Case X right now is the same person who looked up Case Y last week” is one of the pieces of information the institution shouldn’t share with the provider.
Software Testing Cheat Sheet
Sunday, June 17, 2007
Adam Goucher, who escaped the Previous Employer before I did, links to an excellent reference on software testing: Quality Tree Software‘s Test Heuristics Cheat Sheet. Many thanks to Elisabeth Hendrickson for sharing this with us.
I’m planning to print out a bunch of copies and staple one onto each member of my development team.
Fun introductions to RSS and Wikis
Thursday, June 7, 2007
Bokardo posted links to a couple of good videos by Lee Lefever at Common Craft: RSS in Plain English and Wikis in Plain English.
I’m pretty sure most of my readers are well beyond the intro level with RSS, but they’re worth watching just for the nifty lo-fi presentation style.
Battle Scars
Wednesday, June 6, 2007
I don’t think my rides are nearly as intense as Larry’s, but today at least I can boast about earning some (mild) scars. Here’s my route.
It was a running-around morning; I had an early appointment up on St. Clair Avenue and then I had to go down to Toronto General to get my tuberculosis test read. Being a programmer doesn’t exempt me from the disease and vaccination screening (and N95 mask fitting) that every employee in the health system gets.
I say “up on St. Clair” because Lake Ontario is a vestige of a larger lake, Lake Iroquois, that existed during the ice age. St. Clair is at the top of the north shore of that lake, while “lower” Toronto is on the old lake bed. It’s not like a real mountain or anything; next time I’ll try to remember my GPS to prove how much of a wimp I am.
Anyhow, after work it was my turn to get the kids out of storage and tow them home in the trailer. They’re currently rebuilding the streetcar tracks on Dundas St. through downtown Toronto, so traffic is a nightmare. We usually take the bike lane along Gerrard Street, even though the pavement is pretty rough, because there’s enough room to get past cars with the trailer.
Today, just before I got to Yonge St. a woman walked off the sidewalk into the bike lane without looking. Rang my bell, braked hard, and almost missed her, but the end of my handlebar just hooked her and flipped me onto my side. Very mild road rash on one elbow and knee, and smug satisfaction about having the kids in a trailer (which stayed upright) instead of a jump-seat.
Why Aren’t We Using Electronic Health Records?
Tuesday, May 29, 2007
Greg sent me a link to a CBC.ca story: Doctors too slow to embrace electronic health records.
My first reaction was “Of course doctors don’t want to use electronic health record systems. Have you looked at them? They’re nasty!”
Now, physicians are notorious for resisting change, but we need to step back and think about why they might be resisting this particular change. I see some seeds of it in the CBC article. Almost all of the touted benefits are about making health care professionals do their job better. That’s an important goal, but the only way you’re going to get widespread buy-in is if you also make their job easier. If there was ever an area that’s ripe for user-centric design, it’s this one.
Manage or Measure?
Monday, May 28, 2007
One of the rants that has been bubbling in my head for a while is about the current obsession with “metrics” in management circles. I usually blame this on MBA culture, though I must admit that I don’t have direct evidence to blame the business schools.
The oft-heard mantra is “If you can’t measure it, you can’t manage it.” I think this is missing a first, critical step:
If you don’t understand it, you can’t measure it.
Of course, you can try to measure something you don’t understand, but chances are (a) you don’t measure what you think you are measuring, (b) your attempt at measurement alters the system you’re trying to measure, and (c) the result of your measurement is too noisy to draw valid conclusions.
Joel Spolsky’s recent article “Typical Econ 101 Management” led me to re-read his excellent articles on management techniques:
The Command and Control Management Method,
The Econ 101 Management Method, and The Identity Management Method. What Joel calls Econ 101 Management is exactly the sort of poor measurement I’m referring to, applied to the context of getting individuals to do The Man’s bidding.
The Previous Employer demonstrated a couple of interesting variants on the measurement problem. The first one came in the form of “industry benchmarks”. We looked at what a bunch of other companies spent on various business functions, at a macroscopic level. For example, we spent X% of revenue on IT, while “competitors” spent X’%; we spent Y% on R&D, versus Y’%. Every place where our spend was higher than our competitors was taken as an opportunity for wide scale budget cuts.
Now, I think it’s a great idea to look around for good ideas on how to run your business. The problem comes in when you turn that comparison-with-a-competitor into a metric that applies to one specific component of your business. Unless your middle managers have both great backbones and great understanding of the overall business, they’re going to break a lot of things to make that metric turn green on your executive dashboard. Next year you’re going to discover that you’re lagging the competitor on some other benchmark (like employee morale…) and you’ll have to invent another metric to try and manage that, and your managers will break other things to turn the new metric green, and you’ll wonder why all the productive employees are leaving.
The other object lesson from the Previous Employer was around the software development life cycle and defect tracking. Automated processes scrape the defect tracking database for all products and produce various statistics, such as the rate of new defect arrivals, time to address open defects, level of customer impact, etc. The best part of these statistics is that they were used to apply competing metrics to different groups: Developers need to keep the defect arrival rate low, or they’re not allowed to ship to customers (because a high defect arrival rate indicates unstable code). At the same time, the Product Test group needs to file a certain number of defects per tester per week (because a low rate indicates they’re ineffective). This led to an unhealthy divide between developers and testers as we fought over when and how to track defects, with each side trying to game the process so that they wouldn’t look bad on the monthly management charts.
And yet, when it comes right down to it, I’m a big fan of the Scientific Method. That means I do want to measure the effectiveness of both broad scale and fine grain activities; how else can we tell whether we’re improving? The key is the understanding, all the way down the management hierarchy. A measurement isn’t a business outcome. You should never ask your organization to change a measurement (either explicitly, or implicitly by rewarding based on the measurement). Always ask people to change the business outcome, explain why you think the measurement is a reasonable approximation of the business outcome, and watch very carefully to make sure that decisions are directed toward the business outcome, not the measurement. And in the spirit of Continuous Improvement, constantly review your metrics to make sure they really are driving toward better business outcomes.
Guns Don’t Kill People, People Kill People
Sunday, May 20, 2007
Diane’s comment on my previous post is bang on (groan…). With the exception of a couple of sign changes, “Guns don’t…” is an excellent analogy for what I’m talking about.
While I was ruminating on how to flesh out the analogy, Larry stepped in and got to the point. There’s a growing “user-centric” movement in software development, and for those in the loop, Hugh’s “…it’s what the user does” cartoon is all about the subtext: Software doesn’t exist in a vacuum.
It’s too easy for people who spend all their time building software to think that the software has some sort of intrinsic value. Now to some extent, it can; a well crafted algorithm is beautiful to those who can read it. However, just as with some modern art and music, the audience that can appreciate the beauty is pretty small. For the majority of us who toil in ones and zeros our intended audience isn’t trained in the theory of computation, and there’s no reason they should be.
I’ll stretch the music analogy a little farther, and hope it doesn’t break. Most music isn’t written to be appreciated by professors of musicology, it’s written to be danced to, or to enhance the drama of a movie, or some other relatively mundane goal. Similarly, almost all software is created to be used: To entertain, to educate, or to enable.
And this is where the “Guns don’t…” analogy fits (with those sign changes I mentioned). Of course it’s people who do the killing. However, a firearm is a very effective enabling tool. If we have a goal to reduce people-killing-people, taking away that enabling tool may be a very effective approach. With software, of course it’s about the user. However, software is typically a very poor enabling tool. If we have a goal to improve people-being-entertained-or-educated-or-enabled, we need to make good tools to support said people.