Transcript of “Two Revolutions” (given at RailsConf 2007)
Hi, some of you may be surprised to find a talk purporting to be about XP here at Rails Conf, but I’m hoping you’ll see that its worth looking at, and it might provide some context for whats going in in the Rails community today.
I’m a little nervous, and that might make me talk fast. If that happens, someone please, feel free to ask me to slow down. I’m aware that you all have accents and that might make it difficult to understand me at times.
So. Whats the this talk really all about ? The abstract mentioned XP and Rails and what one might learn from the other, and thats what I thought I might talk about back in…whenever it was the abstracts were submitted, But in the intervening months I’ve had 3 attempts at writing about this and I’m still not entirely sure I’ve articulated what I’m trying to say.
I think that I have to admit defeat and accept that this is like an essay question. A pub discussion. I’m hoping that what you might take away from this session is something to think about, rather than something to implement.
So let me start at the beginning.
First off, can I get an idea of the audience. How many of you know what XP is ? Read a book ? Attended a course or conference ? Worked in an XP team ?
In 1999 I was working for a Panasonic research lab in Edinburgh. I was building authoring tools for the likes of Universal Studios to use to build titles for what would eventually become DVD. We were using Visual C++, and eventually Java and we had no real process to speak of. It was a research lab, which meant we had a long time to meander through the applications, and it was a pretty egalitarian team, which meant there wasn’t an architect trying to tell us how to build stuff.
In September of that year, a team was formed to build a Java tool for authoring. It was a pretty small project by our standards, and we had a team of 4. The nominal team lead, Andy, was a friend of mine from college, and was a bit of a programmer savant. We liked to say that English was his third language, after C++ and Java. Andy had been doing some reading (on Wards WIki) about this new methodology called Extreme Programming and asked us if we’d give it a try. Cutting a long story short, we successfully delivered bug-free tested, running software every two weeks to the engineering division in Japan and we were all instant converts.
In December of 2000, we convinced the lab to send us to Chicago for a week to attend the first ever public training course on Extreme Programming - XPImmersion run by ObjectMentor. Present were the big names in XP - Kent Beck, Ron Jeffries, Martin Fowler, Uncle Bob Martin, and a large part of the attendees were pretty much converts anyway..it felt more like a conference than a course, in some ways. Some of these attendees would go on to become pretty big names in the XP community in their own right - Francesco Cirillo, Mike Hill, Mark Windholtz.
I left OWL in April 2000 to join the inevitable dot-com startup. My main reason for joining was that the founder was looking to use XP for the software and wanted someone with experience and the ability to teach not just the programmers, but also our customers and even the company lawyer what it would mean to deliver software in this way.
After I’d been there for nearly a year, I got an email from one of those guys I mentioned earlier - Mike Hill. He’d since joined ObjectMentor as a trainer and consultant and asked if I was interested in doing the same. I flew across to Chicago again in February 2001 to teach a class (my interview). In April I left the startup and joined OM.
After a year at OM I tried going independent for a while, and eventually joined ThoughtWorks, again hired because of my Agile and XP skills (I cheekily used Martin Fowler as a reference). I was with TW for four years in the UK and Australia, and I left in 2005 to join a team of friends using XP to build software for JPMorgan in Glasgow.
Through all of this time, (8 years now) I’d been training and consulting and programming with companies large and small, from a little charity in south east england to Coca-Cola in Australia, I was also increasingly involved in conferences, mailing lists, pub discussion sessions, user groups. I remember when the folks came back from Snowbird to tell us that ‘LightWeight methods’ were henceforth going to be called Agile methods and I was one of the first independent signatories of the Agile Manifesto.
In short, it wasn’t just my job - I threw myself into XP’s culture wholesale, because I liked the message, I liked the results, and mostly I like the people.
So… what about Ruby? Well I bought a first printing of the first edition of the pickaxe back in 2000/2001 mostly on the strength of ‘The Pragmatic Programmer’. I tried it briefly, but I don’t think I was ready. I was still primarily a Java programmer, and though I’d started to read a little Smalltalk, I wasn’t ready to think about a dynamic language. I put the book back on the shelf beside the equally good-intentioned but unread Smalltalk books and got on with teaching folks about JUnit.
I started to hear more and more about Ruby from the likes of Ron Jeffries, Martin Fowler and Dave Thomas, but it wasn’t until Rails in, I think early 2005, that I found something I could get my teeth into. Webapps, I understood.
In the intervening years, I had switched to a Macintosh and had a child, and it’s hard to say which had a larger impact on my approach to programming :-) Whatever the causes, I was ready to hear it, and I jumped wholesale into the culture - buying books, attending conferences, playing with new features, but never (it must be noted) actually building a production app. Through Chad Fowler, who I met as a student on one of my teams at an XPImmersion course I was teaching, I’ve been introduced a whole lot of people, and a whole lot of culture.
My team at JPMorgan, the XP team, uses Java for our production application, but we *all* use Ruby everywhere we can, from WATIR for UI testing, to custom stuff for DB testing and capacity planning exercises. We also have a Rails app which reads our Apache server logs and converts them into JMeter scripts so we have server regression tests which mimic production usage. We were all XP nuts, and now we’re all Ruby nuts.
What, it seems again, is key is that while I like the message, and the results, I also like the people.
So, when the time came to submit to RailsConf, I couldn’t help but think of the parallels. I noted down a few thoughts or observations and without thinking to hard I wrote these.
Firstly, Both XP and Rails are about more than just software. One of the next descriptions I heard of XP was that it was making the world safe for programmers, and reversing nearly 100 years of Fred Taylor’s “Scientific Management” theory that separates those who do the work from those who decide how the work is best to be done. With Rails, stemming from Ruby I guess„ the word thats heard over and over again is Fun.
Second, an old pre-XP quote from a paper written by Ralph Johnson, based on work done by him, and by Ward and Kent. The best frameworks emerge through refactoring, not up-front design, Build a real thing, build another, extract whats common. Don’t try to guess. XP was extracted from the C3 project, Rails from Basecamp.
Third, both XP and Rails centre around the singular vision of a single individual, and his goals and design decisions are key.
Fourth, both XP and Rails can be seen as an extreme over-reaction to what went before. XP is very much the anti-RUP, although we always knew that RUP was not the enemy. Rails almost defines itself by being not-J2EE.
Fifth, both XP c2000 and Rails c2007 are pushy and abrasive in their language. They are teenagers, stating their beliefs and intentions without embarrassment or the tact and guile of more mature heads.
Sixth, Alpha Geeks are drawn to this stuff. The same type of people who were interested in XP in 2000 are interested in Rails now.
Seventh. Less. XP was all about simplicity and less process. High Discipline, but low ceremony. Rails of course stands for Less Software and High Productivity.
Lastly, almost as an afterthought, but included here because it really did occur to me when I was listing this stuff. TW are all over this like a rash. Perhaps Embracing, perhaps Embrace-and-extending, but certainly here. Part of that is covered by the alpha geeks, and the teenage language, because TW is certainly very full of alpha geeks with that kind of teenage swagger. But I thought it bore talking about because it signals a large consulting company staking some serious dollars on this as a message.
Now when I wrote this list, I imagined my talk would really just enumerate these things, and talk a bit about each of them, but I found that I was particularly drawn to the notion of them being more than software. So I scrapped the original idea and started working on version one of the talk.
Rails as Movement.
Thinking about these things as more than software can seem weird since XP is a process, and Rails is a framework. But just as anyone who says that football is only a game is missing the point, anyone who thinks that XP is just a process, or Rails is just a framework is forcing himself to ignore the thousands of people who work in and around it, the whole culture of teams using it and the ideas and opinions it’s use encourages. Metal’s not just music…it’s a way of life.
Back in the days of XP, a criticism regularly levelled against us ‘true believers’ was that XP was a religion and we were zealots. Running on faith and trying to convert the world to our own brand of dogma. This was regularly debated on mailing lists, and at conferences. Religion’s bad, right. I don;t mean generally, I mean as applied here. The connotations are of blind faith. Of brainwashing and no-one would want to own up to that, right ?
The religion label is most dangerous to XP and to Rails because it allows the audience to switch off and reject any of the benefits of each approach because of a perception of the speaker as simply parroting dogma. If it’s a religion that must mean it’s all about faith, and belief, and not about science and concrete practical effects.
But Bill Caputo, in a blog post titles “okay, its a religion” had this to say:
"I now see religion as the energy that builds around a good idea as [it] fires other people’s imaginations and contributes to their own success. As it grows and spreads, this energy (i.e. the idea) is amplified, distorted, enshrined, preserved, dissipated, combined, and ultimately re-expressed — then the cycle starts all over again."
So this is a new strategy - capitulation. Accept its a religion, but change the definition of religion to remove the negative connotations. So in this context, I started to look at Rails as a religion. Some of the things on the list I showed earlier apply here - cult of personality, the extraction creation myth, and particularly the “reaction to what went before”. But I’ll come to that in a moment,
Despite Bill’s ‘redefinition’ of religion, I’m still not comfortable applying that label to XP or to Rails. Kind of like BDD guys feel about the word ‘test’. The word is too overloaded to be easily redefined. Now, I think Bill has nailed what it was I was trying to say with ‘more than just software’, and his ‘energy building’ applies well to XP and to what I wanted to say about Rails, but I tried to think of a different word I could use to sum up this notion of Bill’s, without using religion. And what I ended up sticking with, is ‘movement’.
Here’s a definition of movement from Apple’s dictionary
“a group of people working together to advance their shared political, social, or artistic ideas”
How’s that ? I think that sums up pretty nicely what I’m getting at. Political, social or artistic. I’d say that XP and Rails had or have aspirations in all these areas.
But back to that bit about a reaction to what went before. With my Caputo quote, and my new word to describe it, what could I say about XP and Rails ?
The most important thing out of this first draft, that I kept for now, was Context. A lot of religions (as opposed to faiths) arise because of people finding themselves in a particular setting. The faith of Judaeism (i.e believing in God) didn’t arise particularly from spending a long time in the desert, but there’s a good chance that many of the practices (e.g. circumcision and not eating pork) did. Chad mentioned to me that Hindus sprinkle water over their food, which although associated with religion, actually has origins which are simply to do with keeping flies away.
Similarly, as far as XP’s ‘faith’ is concerned, we’ve been wanting to build software better for ever, but I’d argue that the specific practices of XP really caught on due to the prevalence of RUP and heavy design methodologies at the time.
XP benefited greatly from having an ‘enemy’. Something it very much *wasn’t*.
By having something to kick against, XP pulled the lever entirely in the opposite direction. Farther than it really needed to go, but far enough that it could slip back a couple of notches and still be an improvement. In the beginning we said “40 hour week”. Later on we said “Energised Team”. In the beginning we said, “Customer and Programmer”. Later on we started to acknowledge a role for management and QA.
This is sometimes characterised now, by folks inside the XP community, as a kind of angry teenage rebellion. A blunt kind of politics. And now, with the mellowing of age, less extreme tactics are required. A blurring of the hard lines. An easing of restrictions.
This may be true, but I’d not be embarrassed by our teenage outbursts. I’d argue, only by starting from the most extreme position, and compromising where necessary, do you make the real gains. If instead we’d started with RUP and dropped the Use Case diagrams, we’d not have gotten very far. We stripped the process back to nothing, and only added things that really added value. Like clean installing my laptop, and copying back the stuff I need, instead of just trashing stuff, I end up with way more free space. I managed to get rid of all the stuff I didn’t even know I had.
Rails too is a pretty extreme reaction to J2EE. Spring and it’s cohorts are the more measured response. Lighter weight certainly than full-on J2EE / EJBs but in trying to shift the lever a couple of notches, they’ve maybe only moved by one in real terms. Rails, however, has pulled the lever all the way over. I’d expect that, like XP, in the next year or two, it’ll start to seem less extreme than it does now, and more stuff will be added back in. Yet even though it might feel like backsliding to the hardcore, in real terms the lever has moved more than it ever would have without the angry teenage rebellion and arrogance of youth.
Now, this sounds like a pretty interesting talk, I think, but to be honest, you got the nest fo it right there, and I realised I couldn’t fill 45 minutes from this angle. So I started to seek some help from my friend Stuart, who used to be a university lecturer. I talked and talked about what I felt and what I was trying to communicate with this talk, and I ended up realising that the talk I *actually* wanted to give was probably a much more personal one than this first draft.
So onto version two.
While I was working on this talk, an old mailing list haunt awoke in my inbox after 3 years of silence and inactivity. xp-coaches, a yahoo group, was formed as a place for a bunch of ex-ObjectMentors and their close and trusted friends to hang out and bitch about stuff. Not as exclusive as the shadowy, rumoured xp-leadership list, but cutting out the noise of the extremeprogramming group. Through its lifetime, its been host to many discussions pro and con about Xp, about Agile and about programming in general.
The topic that reawakened the list was the question of whether XP Agile had killed XP. In essence, we ‘won’ in that Agile is now on everyone lips, but in doing so have we diluted XP to the point of irrelevance ?
One of the first things that really drew me to XP was the balance between the abstract notion hat I talked about in terms of being a movement., and the very, very concrete and specific. Then the 2nd Edition of XP Explained came out, and the few of left who still said we were Extreme programmers, instead of Agile Practitioners got ready to rally round the flag again.
But we were disappointed. Gone was 40 hour week. In it’s place was ‘energised team’. Gone was ‘on-site customer’ - in it’s place was ‘whole team’. 12 practices had grown to over 30.
When a few of us tried to engage Kent on this, his response was a sensible one. The absolutist language of the teenager is often necessary to get a point across clearly. This is useful in the early stages of a ‘movement’. Once the attention has been created though, is time to tone down and appeal to a larger market. It’s time to ‘grow up’.
As I said, this sounds fairly sensible, but I and a few of my XP friends don’t feel we needed to grow up. I’m currently working in a team inside an investment bank, doing XP by-the-book (the first book that is). We didn’t have to tone down the message, or adopt a less absolutist message, or start using the word ‘agile’ instead of extreme. We get to use XP because we demonstrate results. We deliver running, tested features, bug-free, every two weeks. The permanent guys on the team get an extra monthly payment for carrying the support cellphone, which theoretically could go off any time with a support issue for our app. I think it’s rung about twice in 3 years and then it was just to figure out to restart a crashed tomcat instance. It’s essentially free money, and we like to think of it as a performance bonus for delivering zero defect code.
Thats the kind of results that we deliver from the teenage , non-compromise version of XP.
Here’s a quote from an english comedian, Stephen Fry. It’s from his autobiography, and is part of letter he wrote to himself when he was 16. It was headed “not to be opened until I am 25”.
I know what you will think when you read this. You will be embarrassed. You will scoff and sneer. Well I tell you now that everything I feel now, everything I am now, is truer and better than anything I shall ever be. Ever. This is me now, the real me. Every day that I grow away from the me that is writing this now is a betrayal and a defeat….
WHAT I AM NOW IS ME, WHAT I WILL BE IS A LIE.
Now there’s a very real fear in my mind that feeling like this makes me a bit sad. Kind of stupid, 35 year old yearning for those good old college days with your buddies. A mid-life crisis if you will.
But on balance, I think there is something genuine here.
I really do feel that when we were, metaphorically, teenagers we changed the world. The lever that I talked about before in terms of the context couldn’t have been pulled as far over as it could be without that teenage language, and I think it;s mistake to say that we can drop that now and let the lever slide all the way back. couple of notches, yes, maybe. But ‘Agile’ is a step too far. It doesn’t mean anything any more.
Here’s another quote, this time from an Apple ad campaign in 97/98.
Here’s to the crazy ones.
You can praise them, disagree with them, quote them, disbelieve them, glorify or vilify them. About the only thing you can’t do is ignore them.
Because they change things.
Maybe they have to be crazy. Because only the people who are crazy enough to think they can change the world, are the ones who do.
See? now we’re getting somewhere. Okay, its a bit schmaltzy, but it rings true for me on a fundamental level. I think it’s only through that crazy teenage language that we really can change things. Make software orders of magnitude better. XP was Opionated Process.
So, what about Rails. Well, I don’t know about you, but I don’t think we have a shortage of pushy, absolutist statements and teenage-style language in the Rails community.
From side-by-side comparisons of Java and Ruby code, through the components/engines debate and WS-DeathStar. From the infamous canada-on-rails slide, to the communities extreme over-reaction to Dave Thomas’s ‘enterprise-friendly’ keynote and recent twitter debate ‘Rails’, and one man in particular, are not for compromising. Not for growing up.
Now, there may or may not be older, wiser heads who think this is a problem. There may be those who think that toning it down will bring wider acceptance. And this may really be good for Ruby. More acceptance, more usage, all good things. For Ruby.
But for as long as Rails remains the pushy teenager. The outsider. It is free to shake things up. To push the dials to 11. To pull the lever all the way over. And this, the second version of the talk, concludes with call to arms. Don’t Grow Up. Change the world.
So. I was feeling pretty pleased with how I’d discovered this core of passion, this thing that was really at the heart of my attraction to XP and to Rails. I knew that what I wanted to say was ‘don’t let what happened to XP happen to you’. I knew that there was a personal tale here of feeling that what I thought was a group of like-minded individuals had ‘grown up’ and left me, still feeling like a teenager, still wanting to change the world.
Last week I sent this version of the talk to Dave Hoover, some of you may know Dave from Obtiva. I worked with him at ThoughtWorks.
Dave pretty much told me I was full of crap. All that had happened was that when one revolution had run its course, I’d found another, which would, in turn, run its course.
His position was that XP didn’t succeed because it toned down. It toned down because it succeeded. And inevitable the same would happen to Rails. If it succeeded in replacing the mainstream, the opinionated stance would fade. Without an enemy to push against, there’s no need to violently differentiate yourself.
At first this made me sad, Because I realised he was right. But then it made me happy, because I realised he was right. This leads to version 3 of the talk. Which I only really wrote up on the plane, and have decided to call…
How many people have heard of the Ba’h’ai faith ?
Okay, so when I was younger, I dabbled in a few big-R religions. I never really found one that stuck, but at the core of the big four or five religions is a fairly cross-cultural message - “Love God, treat the world and other people with respect - do unto others, in effect”. Each of the major world religions, however, layers a variety of practices, rules, a responsibilities onto this core. Each has a different perspective, a different approach to supporting that core belief.
The Ba’ha’is are a little different.
The Ba’hais believe in the same god as Christians, Jews and Muslims. But where they differ is that they believe in what the call the ‘changeless faith of god’ and the ‘oneness of religion’. That every so often, God sends what they call a ‘Manifestation’.
This is different from a prophet, or a priest. They believe that the manifestation is God in earthly form. Jesus was a manifestation, they believe, as was Abraham, and Moses. But also Zoroaster, Krishna, Buddha and Mohammed. Each of these people was divine, not just blessed by God, but actually Gods physical manifestation on Earth.
Each time God sent a manifestation, he brought with him a different ‘religion’. A different set of codes and practices, appropriate for the time and place of the manifestation, to try and bring the human race closer the idea of Unity - all loving god, all loving each other.
So this is pretty key. In essence they’re saying that God is trying over and over to help us get it right. trying a different approach, a different metaphor, a different way of saying the same thing.
So. How can we apply this to XP and Rails. What does this have to do with ‘Two Revolutions’. Well, Everything.
Because it’s not just 2 revolutions, its many. The same type of people, in some cases, even the same people. Have ridden the waves of revolution through Abstract Data Types, Objects, Patterns. Through Lisp and Smalltalk and Ruby and Rails. Through Extreme Programming and Less Software. All of these things are different - some are ways of analysing the problem, or of designing or coding a solution, or of managing that process of turning a thought into a piece of software.
But all of them share stuff as well. They work in a context, reacting to whats gone before, they attract a core of smart, passionate people - alpha geeks. They’re too slow, too weird, to crazy to really work. They’re just for hackers or for toy projects.
But some of them succeed. Who would argue against Objects now ? Who hasn’t heard of Design Patterns ? Who would stand up and seriously advocate a phased approach to software over an iterative approach ? Some don’t succeed, they just stay niche. But the core of people will jump from one revolution to the next, always looking for that essence, that little glimpse of something larger, a movement, a new way to bring us closer to the idea of better software.
Each time something new comes along we’ll ride it as long as its making things better, and when its gone as far as it can, and attacking the ‘problem’ from that angle isn;t helping any more, we’ll look for another angle. Objects and Patterns gone as far as they can - no more gains to be made on ‘design’, lets switch to ‘process’. XP take process as far as it can, lets switch to languages and frameworks. Ruby and Rails taken us as far as we can go there, lets switch to something else. Maybe Continuations are whats next? Maybe its Erlang?
Or maybe its going to be back to process, or design, or analysis.
Here’s a last quote - from Ward Cunningham in May 2007
We have to assume that there are variables, dimensions even, that we don’t yet address. You can be sure that long term success will not come from repeating our own dogma again, louder.
In order to advance the art, we must always be looking for a new perspective, be aware of when our revolution has run its course, and try and keep an eye on when the next one kicks off.