A Conversation I’d Like to Have

3 03 2012

Do you ever talk to yourself while you’re driving alone, playacting how things might go if you were able to say what you wanted to the people to whom you wanted to say them?

I do.  It doesn’t usually do me any good, but I do it anyway.  Especially these days, since I have a much longer commute than I used to have.  For example:

“Would you stop screwing around with your Agile teams that way?  You can’t just keep breaking them up and stirring them around and re-forming them and expect any decent velocity from them!

“People are not commodities.  They’re not interchangeable.  They’re unique individuals, with unique aptitudes and unique skills and unique strengths and unique weaknesses. Put a small number of them on a team and leave them there, and they’ll learn all those things about each other and grow to be able to combine their strengths and cancel out their weaknesses.

“A brand-new, immature Agile team that has only one or two projects under its belt isn’t particularly valuable, true, except that it has the potential to grow into an experienced, mature Agile team, which is particularly valuable.

“For example, here’s a challenge. I am under no illusion that you will take me up on this challenge, but I’m not bluffing. Take Kevin and DJ and Gabbie and me, and put us on a team.  I don’t mean add us to an existing team: I mean make a new team out of us.  Give us two of your better QA folks—say Matt and Kay—and a product owner who’s in a bind.  We will blow the frickin’ doors off anything you folks have ever seen in this company before.

“Why—because we’re superintelligent?  Hey: you know me well enough by now to say definitively and with confidence that I, at least, am certainly not superintelligent.  Because we’re expert programmers?  No.  Because we’re an experienced, mature Agile team.

“We know each other; we’ve worked with each other long enough that each of us knows how the others work.  And we won’t stand for any frickin’ bullcrap.  Our overriding concern is not following IT’s established rules and processes: our overriding concern is creating business value for our product owner as quickly and efficiently as humanly possible.  If your IT people support us, we’ll work with them.  If they get in our way, we’ll bypass them.  We will have something in production by the end of the second week, come hell or high water, and at the end of every week thereafter.

“But I warn you: after experiencing a team like ours, that product owner will be forever corrupted.  He’s no longer going to be interested in waiting for approvals from committees of architects or waiting for monthly releases or waiting for weeks of big design up front to be complete…or really in waiting for much of anything, for that matter.  He’s going to want what he wants, and he’s going to want it right now, and if you tell him you can’t give it to him on his time schedule he’s going to point to us and say, ‘They can.'”


“Yeah…listen, I heard something a little disturbing yesterday.  Do I understand correctly that you have an application you want us to put into production?  Because we haven’t heard anything about this application: it’s not in our process or on our schedule.”

“No, you don’t understand correctly.  The application is already in production—has been for almost two months now.  We hit 200 simultaneous users last week.”

“[indulgent chuckle] I’m afraid your information is mistaken.  I would certainly know if your application was in production, because—as you know—I oversee the deployment process; and I haven’t put it in production.”

“It’s your information that’s mistaken.  We talked to you folks—what, ten weeks ago or so now?—about our application, and you told us we’d have to write and submit a proposal to the proper committee and have it discussed and approved at the next quarterly meeting, and we simply didn’t have time for that; so we hired a hosting service in the Cloud and deployed it there.”

“No—you see, in order to do that you’d have to…wait.  Wait.  What?!”

“I told my team that I needed the application in production as soon as possible.  They talked to you guys, then decided it was possible to put it into production sooner than you could, so that’s what they did.  They’re Agile, you know.”

“But you’re not allowed to do that!  We don’t host our code on third-party servers, we host it on our own servers, under our control.  You’re going to have to take it down and have us put it up on our farm.”

“What did you say to me?  I’m not allowed?  Listen, let’s talk for a minute about what I am, rather than what I’m not.  I am the business.  You’re IT.  I’m your reason for being.  You exist to serve me, not to tell me what I’m allowed to do or give me orders.  My team is leading me to really understand that for the first time.

“But look, I’ll tell you what.  You’ve got a proposed requirement for our application.  Fine.  Write me up a story card, and I’ll put it in the backlog. I’ll prioritize it among all the others written by my team and me.  If you get fast enough to give me the same level of service my team gives me with our hosting service, or if the application gets mature enough that we change it so seldom that your latency doesn’t really matter, your card might get somewhere near the top of the backlog.”

“It’s not a good idea to talk to me that way.  Since the application didn’t go through our established processes, we can simply choose not to take the responsibility for supporting it.”

“We’re not interested in having you support it.  My team is doing a fine job supporting it themselves.  As a matter of fact, the defect count is so low there hasn’t been much in the way of support to do, anyhow.”

“They didn’t consult with any of our senior architects.”

“Or the junior ones either—yes, I know.  That’s one of the reasons they were able to move so quickly.  Another is that they wrote it in Scala, which enabled them to use a couple of new frameworks they say really gave them a boost.”

“Scala?  Scala?!  We don’t support Scala!”

“Yeah, we noticed.  You don’t support git or IntelliJ or Jenkins either.”

“We can’t maintain that—none of our maintenance people know any of that stuff.”

“We’re not interested in having you maintain it.  My team is going to maintain it.  They’re the ones who wrote it, after all—it makes sense for them to maintain it.”

“Well, okay, what are you interested in having us do?”

“Hey, I didn’t call you; you called me.  You just go on doing whatever it is you do—make a rule, require an approval, establish a process; I’m just guessing here.  I probably won’t have a lot of time to talk to you for awhile: I’ve got about two more iterations of stories on this project, and then I’m going to have my team start another one.  I’ll tell you about it if you like, but I’m not interested in having you support it or anything.  My guess is that we’ll have it serving users before you could have gotten it through your process.

“This Agile stuff is really cool.  You should try it sometime.”


Should Agile Testers Code?

13 01 2012

This morning at CodeMash, I had an argument over breakfast with Jeff “@chzy” Morgan (more commonly known as Cheezy) of LeanDog about Agile testers.  After some advancing and retreating around the battlefield, it became clear that we seemed to generally agree about almost everything regarding how Agile teams should be run, except for this one issue.

The story test that every story card should produce when it first arrives in the Dev In Progress column: should that story test be written by a developer or by a tester?

Cheezy says a tester should write it; I say a developer should write it (after a detailed conversation with a tester or product owner, of course).

I’m going to reproduce here, as well as I can remember them, the salient points of the argument, and I’d love to have people having more experience with testing than I have address them in the comments.

Before I say what I’m going to say, though, I’d like to say what I’m not going to say.  As a developer, I’m not going to disrespect testers.  I’m not going to imply that coding is an exalted, morally superior activity that should be undertaken only by developers, since developers are the moral superiors of testers.  I live in awe of good Agile testers, and–after having tried once or twice–I have no lingering illusions that I could do what they do.  Agile testers have brought my cheese in out of the wind so many times that I’ll be indebted to them for a long time.

Essentially, my arguments will all boil down to this: people should do what they’re good at.

Finally: I’m going to do my best to be fair to Cheezy’s arguments as he presented them.  Cheezy, if you read this, feel free to comment publicly or privately about anything I get wrong, and I’ll post an update.


Cheezy says that since testers will almost always know more about the problem domain than developers will, it is the testers who should have the responsibility of translating the requirements from story-card language into executable language, like Cucumber over Ruby.

I’ve been on teams before where this was advocated.  It sounds good, both to Cheezy and to me.  What I find in the real world, though, is that writing good tests is absolutely the hardest technical part of a developer’s job.  It can sometimes take me a couple of days to get a story test written and failing for the right reasons, given all the investigation and forgotten last-minute spiking that has to go into it; once I do that, the code to make it pass is generally much easier.

So in this case you’re not just saddling testers with development work: you’re saddling them with the very hardest kind of development work.  For this, it’s not good enough to take an excellent tester and teach him a little coding.  You have to actually make him into a real honest-to-goodness developer as well as a tester, if he’s going to be successful.

I was on a team once with a tester who was really good at writing story tests.  In his case, if I remember correctly, he was writing easyb tests and backing them up with Java.  As a matter of fact, he liked it so much that he ended up crossing over and becoming a developer instead of a tester.  I was rotated off the project shortly after that, but I imagine that with both testing and development skills, he went far and fast.  If he did so, however, it was as a developer, not as a tester.

The rest of the testers on that team didn’t really cotton well to easyb or Java.  They struggled at it for awhile, frequently resorting to the developers for help, and eventually stopped writing automated tests altogether, whereupon the task fell to…that’s right, the developers.

To support Cheezy’s side of the argument, I should say that I was also on a team that used Fitnesse over Java for ATDD, and on this team the tester really did write the acceptance tests in Fitnesse, and it worked out great.  On my side of the argument, though, the Fitnesse tests had to be supported by fixtures written by the developers, and it’s somewhat of a stretch to call filling out a Fitnesse wiki table “coding.”

Cheezy says that the most dangerous part of the software development process is the place where requirements get translated from English to code; therefore it’s always safest for that to happen all inside one brain, rather than having it depend on one person understanding another person’s explanation.

I agree that that would be the best place for it if that kind of thing worked; but in my experience, in a large percentage of cases, the tester gets stuck while writing the code–remember, this is hard work, not easy work–and enlists a developer to help him.  When that happens, the developer has to look at the tester’s existing code and figure out what he’s trying to do–error-prone–talk to the tester and figure out what he wants to do–error-prone–and guide him through the rest of the process of getting his thoughts down in code–again, error-prone.

I don’t know of a way that we can eliminate errors in translation.  We can reduce them by listening really, really carefully and asking lots of questions; that’s the way Agile developers are trained to operate.  But eliminate?  You tell me.

There’s another issue here too that didn’t come up in our discussion, but which I’d like to raise here.

Many times, especially when a card represents a foray into an as-yet-unexplored wing of the project, there will be spikes preceding it to discover things like, “Once I’ve sent a set of instructions over the network to that remote node, how do I get it to tell me what it’s done in response to them?” or “How do I take control of this legacy subsystem so that I can make it behave as my tests need it to?”

Those spikes won’t be completed by testers: they’ll be completed by developers.  Hopefully they’ll be completed by the same developers who will eventually play the cards they precede.  Having those developers employ the technology they’ve just spiked out is certainly less error-prone than having them explain it to testers so that the testers can write the story tests (quite possibly in a completely different language from that of the spike) that use it.

Cheezy says that having testers automate their testing frees them from regression testing so that they can do the most valuable thing testers do: manual exploratory testing.  I agree that exploratory testing is the crowning glory of an Agile tester; I agree that manual regression testing is worse than scrubbing the gymnasium floor with a toothbrush.  But my experience with testers writing automated tests is that they spend so much time struggling with the code that it cuts into their exploratory time just as much if not more.

And as for manual regression testing, nobody should have to do that–except perhaps after a system-wide refactoring, like moving from a SQL database to a No-SQL database or switching Web frameworks, where the vulnerabilities move around.  When a tester discovers a defect during his exploratory testing, a developer needs to write a test proving that defect doesn’t exist and then make that test pass, leaving it as a permanent feature of the test suite, so that the tester never has to regress that problem again.

Cheezy pointed out that it’s very expensive when cards move backwards on the wall, and opined that having testers write story tests could eliminate that.  I’m skeptical, but if Cheezy–if anybody–can come up with a way to eliminate having cards move backwards on the wall, I suggest that the very next step should be getting rid of testing altogether: why would it be needed if a card never moves backward anyway?

A final point I’d like to make is that I’ve found that one of the reasons testers are so valuable is that they don’t instinctively think like developers; they think like users.  Making them code would force them to think like developers at least a little–that is, enough to be able to do the hardest thing developers do.  Hopefully it wouldn’t pull them completely over to the Dark Side, but in the final analysis I think I really would rather have them stare blankly at me as I yammer on about some developer excuse and then say, “But it doesn’t work.”  If they start accepting my excuses because they understand my viewpoint, that’s no good for anybody.

Thanks again, Cheezy, for the discussion this morning; it has helped me think about Agile testing in new ways.  Perhaps your comments and others will help me further.

Programming Language Blues

16 11 2011

I spent some time recently putting together a list of pros and cons for various programming languages for a large (thousands of devs) Java shop that tests after in Ruby. (Ruby 1.8, not JRuby.) We’re bucking for approval to write a greenfield project in something more modern than Java.

The project will require high-volume data processing.  There’s a big relational database, many MQ queues, and web services galore.  We’ll be modifying a couple of existing web applications, but not writing any from scratch–at least not of any significant size.

Several folks on Twitter have requested to see the list; here it is. If you read carefully, you may detect the barest hints of bias.  Enjoy.

Java Advantages:

1. Lots of people know Java.

2. Extensive installed base of support in the form of software libraries.

Java Disadvantages:

1. Difficult to learn in a modern Agile TDD environment, because it predates many of the concepts on which Agile and TDD are based.

2. Very verbose, compared to modern languages. Example:

def ageMap = [Kurt: 29, Dan: 45]
val ageMap = Map ("Kurt" -> 29, "Dan" -> 45)
(let [age_map {:Kurt 29 :Dan 45}])
ageMap = {'Kurt' => 29, 'Dan' => 45}
let ageMap = Map.fromList [("Kurt", 29), ("Dan", 45)]
Map<String, Integer> ageMap = new HashMap<String, Integer> ();
ageMap.put ("Kurt", 29);
ageMap.put ("Dan", 45);

3. Weighted down with obsolete concepts the rest of the community has moved away from, like checked exceptions and static typing without type inference.

4. Glaring lack of closures.

5. Compulsively imperative syntax in a season where the industry is of necessity moving toward functional programming.

6. Concurrency primitives are very low-level; hence non-trivial concurrent programming is nearly impossible to get right and nearly impossible to fix if it’s not gotten right.

7. JUnit, the most popular testing framework, is definitely showing its age alongside more modern rspec-like frameworks; Java syntax pretty much precludes anything rspec-like that’s not distracting kludgework.

Groovy Advantages:

1. Dynamic typing makes writing powerful tests very easy.

2. Almost a superset of Java; easy to learn for Java programmers. (Most .java files will compile and run without modification as .groovy files.)

3. Receptive to a functional style of coding.

4. Much more concise than Java; therefore less error-prone.

5. Modern testing frameworks already exist for Groovy.

6. JVM language; compiles to .class files just like Java; instantly and completely compatible with all existing Java libraries and frameworks.

7. Healthy and growing Groovy community.

Groovy Disadvantages:

1. Over-use of dynamic typing can make program code difficult to read.

2. Dynamic typing means the compiler can offer much less help than in a statically-typed language.

3. Dynamic typing means that deciding at runtime what a given method call means is a non-trivial operation; performance is lower than statically-typed Java, for which the meaning of a method call is always clear.

4. IDE plugins for Groovy are notoriously bad, both because Groovy’s dynamic nature fundamentally limits the amount of help they can provide, and because they’re immature software. IntelliJ IDEA has the best Groovy plugin, but it’s still bad compared to the worst Java plugin. (Update: I’ve been informed that the Groovy plugin for the SpringSource Tool Suite version of Eclipse is much improved in its newest version; I haven’t been in STS for a number of months.)

5. No better than Java at concurrency.

Scala Advantages:

1. Statically typed, so that the compiler can give significant help and performance is at the same level as Java.

2. Much more powerful and flexible type system than Java’s.

3. Smart type inference engine, so that types are not overspecified and change is easier.

4. Syntax is more logically consistent than Java’s; easier to learn for beginning developers than Java.

5. Tremendously powerful language in general; ease of writing domain-specific languages in Scala means that it can be made even more powerful in particular areas of interest.

6. Functional programming is built into the language, although imperative programming is made simple as well.

7. Significantly more concise than Java.

8. Features such as implicits provide many of the advantages of a dynamic language without the disadvantages.

9. High-level concurrency primitives–actors, in particular–are built right into the language.

10. XML is native syntax in Scala: you can write XML literals in Scala without putting them in strings.

11. Enough like Java that Java programmers can quickly learn the Java-like imperative side of Scala and become productive, then absorb the goodness of the language more slowly.

12. Modern rspec-like testing frameworks exist for Scala–for example, ScalaTest.

13. JVM language; compiles to .class files just like Java; instantly and completely compatible with all existing Java libraries and frameworks.

14. A prospective hire who knows Scala is likely to be a significantly better developer than a prospective hire who only knows Java.

15. If you spec a project in Scala, and word gets out that you’re becoming a Scala shop, throngs of said significantly better developers will be pounding on your doors to get away from J2EE.

16. Large and growing Scala community; may be the fourth most popular language behind Java, C#, and Ruby.

Scala Disadvantages:

1. Powerful syntax requires powerful compiler; compile time can be twice or three times as slow as Java.

2. Scala programmers don’t grow on trees (although they’re not that hard to create), and Scala shops attract the ones who already exist.

3. Scala IDE plugins are, like Groovy plugins, still immature; but Scala plugins can provide more assistance than Groovy plugins because Scala is statically typed.

Clojure Advantages:

1. By far the coolest language in common use today, except for J and maybe Erlang.

2. Almost completely functional; provides future-proofing against the coming wide swing toward functional programming, and eliminates much of the hassle associated with writing tests for imperative languages.

3. More flexible and powerful than an object-oriented language–especially a partially-OO language like Java; can be made to act OO for people who are uncomfortable outside OO.

4. Much more concise than Java.

5. Optionally-typed; dynamic typing when flexibility is needed, static typing when performance is needed.

6. The testing framework distributed with the language itself, clojure.test, already beats JUnit.

7. High-level concurrency primitives–agents, futures, promises, and software transactional memory, in particular–are built right into the language.

8. JVM language; compiles to .class files just like Java; completely compatible with all existing Java libraries and frameworks.

9. Small but growing Clojure community.

Clojure Disadvantages:

1. Syntax is wildly different from anything most Java programmers have seen before.

2. Imperative programmers will be uncomfortable at first with the functional style of coding that Clojure pushes heavily.

3. Clojure programmers don’t grow on trees, and are not easily created unless they already have a passion for learning.

4. Lisps have always had academic interest, but every Lisp so far has faded eventually into obscurity in the business world. If ever a Lisp could succeed, it seems Clojure has the best chance; but Clojure may end up fading too.

Ruby Advantages:

1. Dynamic typing makes writing powerful tests very easy.

2. Receptive to a functional style of coding.

3. Much more concise than Java; therefore less error-prone.

4. The original rspec was written for Ruby; JUnit & Co. cough and choke in its dust.

5. Ruby community is the third largest after Java and C#. Ruby programmers are easy to find.

Ruby Disadvantages:

1. Syntax is different enough from Java to make for a fairly steep learning curve (steeper than Scala, not as steep as Clojure).

2. Ruby is notoriously slow, especially in its own virtual machine.

3. Not a JVM language; no access to existing Java libraries or frameworks; if used with JVM languages, much code will need to be duplicated.

4. Dynamic typing means the compiler can offer much less help than in a statically-typed language.

5. Ruby is no better than Java at concurrency.

Haskell Advantages:

1. Pure functional language–or as pure as it gets, anyway. All data is immutable. Easy to test, easy to understand, easy to reason about.

2. Statically typed, with a type system so powerful that Haskell programmers claim, with some justification, that the compiler can find all your bugs before you even run the code.

3. Powerful high-level semantics offer both conciseness and descriptiveness.

4. Performance is high.

Haskell Disadvantages:

1. Syntax and semantics are both wildly different from anything most Java programmers have seen before.

2. Imperative programmers will be uncomfortable at first with the functional style of coding.

3. Haskell programmers don’t grow on trees, and are not easily created unless they already have a passion for learning.

4. Not a JVM language; no access to existing Java libraries or frameworks; if used with JVM languages, much code will need to be duplicated.

Cage Match

14 10 2011

As I’m currently on the bench between engagements, my company sent me out to interview for a gig with a small local company that creates third-from-the-bottom-tier websites for startups.

Let me explain what I mean.

A bottom-tier website, in the definition I’ve chosen to use here, is one that serves up nothing but static data: that is, it’ll have a front page with your company’s name, logo, contact information, and a blurb or two about the business you’re in, recommendations from previous clients, and so on, but it’s essentially just a point of presence on the Web that gets you into search-engine results and not much more.

A next-to-bottom-tier website is one that has some functionality behind it, but that has been created without any programming expertise using one of a hosting company’s standard creation wizards.  Maybe there’s a user database and a shopping cart and an RSS feed, but they’re all standard building blocks hooked together in a standard way by the site owner without any build-time interaction from developers.

A third-from-the-bottom-tier website, like the ones this company creates, has the same active building blocks, but there’s also a comparatively small amount of real custom business logic involved that has to be coded by real developers.

The operative issue for this company is that since the startups (two- and three-person companies) it works for don’t tend to have big budgets, it has to complete these simple websites quickly–sometimes on the order of just hours, more often in a month or less, occasionally as long as three months–so that it can afford to hire developers who are capable of doing the work.

To that end, the company depends heavily on Spring Roo and several custom plugins that let it generate most of the code for a new website in just a few minutes so that they can spend the rest of the time on cosmetics and business logic.

So I was interviewing with the CTO, and I spent some time arguing with him about whether his small budgets and reduced cycle times would be better served by an Agile process or by the compressed-SDLC/waterfall process he was using.

I’d like to explore some of these questions a little further here, but I’m hampered by two things.  First, there’s my obvious prejudice in favor of Agile.  Second, while the CTO has obviously read a fair amount about Agile, he has never had any direct experience with it, leading to certain misconceptions that couldn’t be addressed in the space of a job interview.  Perhaps interested readers can compensate for these failings, where they appear, in the comments.

Issue: No interaction with the customer between requirements specification and demo of finished product

CTO’s position My position
The customers, by and large, are not technical people and don’t care about technical issues. We’re the technical people. If we run into requirements that are not completely specified, we use our heads to make the best guess and build it in, and the customers are generally fine with that once they see it. Technical people should decide technical issues; but non-technical issues are not really their forte. If a non-technical requirement isn’t completely specified, it’s probably because the specifier didn’t think about it especially hard; perhaps if he had, he would have uncovered completely new requirements without which the project will be significantly poorer.
If we show the site to the customers before it’s done, they’ll want all kinds of other stuff too. They’ll creep the requirements, we won’t get finished by our deadline, and we won’t get paid. Creeping requirements are a good thing, not a bad thing. It means the finished product will come closer to meeting the customers’ needs. Also: more requirements, more money. Don’t deal in time: always work on what the customer thinks is most important right now, and let them stop paying you when the money is worth more to them than the next feature down the priority list.
Our customers are tiny two- and three-person companies. They don’t have time to be constantly babysitting us: they’ve got more important things to do. If we send them a question requesting clarification of a requirement, it may take them two or three days to respond, and their response may just immediately prompt another question. Our schedule doesn’t allow us that kind of time to be waiting around. It’s true that an Agile development process requires high-bandwidth communication with the customer. I feel like this one’s a good point. My best response would be: which is worse, having a feature be blocked for awhile, during which time you work on another feature, or wasting time and funds developing the wrong thing?

Issue: The first time a product sees production is after it’s completely finished.

CTO’s position My position
Almost all our websites use exactly the same components, and we deploy them all the same way on the same hosting service. We know the route to production very well, and have it mostly automated; there’s no reason for us to deploy more often. It’s a good point. There are two reasons for pushing to production early: first, to get a handle on the process; and second, so that your customers can see the product early. You’ve already gotten the first taken care of, and you’re intentionally eliminating the second.

Issue: Project is not deployable until development is complete

CTO’s position My position
We work under very tight schedules; we don’t have the time to spend repeatedly making the code deployable. If you had continuous integration running automated tests, you could easily fix it so that your code was deployable whenever CI was green.
Even if we could get quick at it, there’s no need to deploy early. If you had constantly-deployable code, then customers who ran out of money early could have at least something for the money they had already spent, rather than nothing.  A customer who has run out of money is an unhappy customer, but a customer who has run out of money and who gets nothing for the investment he’s already made is an unhappier customer.
That would require customers to be constantly looking at the product.  Our customers don’t have time for that: their days are full of appointments. Then they’re accustomed to appointments.  One of those appointments could be a weekly demo of the product, closely guided and sharply focused just on what’s been developed that week.  You’d be done in half an hour.  You could leave that version of the code running over the next week on a private server so that they could check it out on their own whenever they had time–even from their smartphones.

Issue: Automated test coverage is very low

CTO’s position My position
We work under very tight schedules: we don’t have time to spend writing tests. Tests make development faster, not slower. They don’t make you type code faster, but they make you think more about what you are typing, and they almost always force you to get it right the first time, so that the time spent later stepping through with the debugger is eliminated.
Some of our developers write tests; others don’t. We don’t enforce a rule; we’re most interested in having them do whatever makes them go fastest. What’s fastest is a Very High Test Coverage environment where everyone can instantly see all the effects of every change. It’s not faster if all you count is the time from when the source file is created to the moment the developer first checks it in as finished; but it’s significantly faster if you wait to stop the clock until it’s really finished, including all the bugfixes made necessary in the new code and in existing and future code it touches.
If you wait to write tests until after the code is already developed, then you know A) what needs testing, and B) how to test it. But I already know what needs testing even before I turn on the machine: everything. And I typically find it quite difficult and time-consuming to figure out how to write tests for code that’s already in existence, written without regard to testability, than I do to simply demand each bit of code into existence with a failing test. [Thanks to Justin Searls for that terminology.] That code is going to be more testable, therefore more accessible, therefore more loosely coupled, therefore better, therefore more understandable, therefore quicker to develop around and to refactor.
A mass of automated tests will make it painful and slow to refactor existing code. Quite the contrary: any significant refactoring of any significant chunk of code absolutely depends for its timely success on VHTC!  Without tests, you’ll be finding subtle bugs in your refactoring for months.  Refactoring is precisely where VHTC comes into its own and shows its real value.

Maybe the tests you’ve seen aren’t the kinds of tests I’m talking about.

Issue: Pairing is rare

CTO’s position My position
We pair occasionally, but we only pair senior developers with each other, and only under emergency circumstances, and generally for long periods–perhaps 12 hours. Hmm. Maybe you and I mean different things by “pairing.”

Issue: Change is expensive

CTO’s position My position
The way we’ve been doing it works for us, even if it outrages your sensibilities. We have a lot of happy customers and we’ve made a lot of money. We’re going to keep going in the direction we’re going at least for another year. Can’t argue with success. You’ll be fine as long as some hungry Agile startup competitor doesn’t come along and eat your lunch.
We’re not yet a big company. Agile would be pretty much completely new to us. It’d take us awhile to learn it; during that time our velocity would be lower, our customers would be less happy, and we would make less money, as well as having the added expense of courses or consultants or coaches or whatever we chose to train us. We can’t afford that now. Maybe after that year. Uh…that is…it’s…I mean…ah…

Sorry, I got nothin’.

Mixing Coaching and Delivery

10 10 2011

After a year and a half of exploring what happens when you try to mix coaching and delivery, I have formed some fairly strong opinions on the subject.

In more than one way, it’s like grinding up an entire bicycle into a barrel of steel filings and then eating them with mustard.

Is it possible? I don’t know. But I’m pretty sure that even if it is possible, it’s not a good idea.

When a coaching-plus-delivery coach is pairing with a client developer and they encounter something the client developer doesn’t understand, the coach has three choices.

A) Be in slow, pedagogical, I-don’t-care-how-many-iterations-this-one-point-card-takes coaching mode, and explain in full detail, making sure the coachee gets it and maybe even doing some spikes on the side to demonstrate concepts.

B) Explain what you’re doing as you bang out the solution, allowing enough drag to answer a few questions, but maybe slowing down only to half speed and not getting distracted from the task at hand.

C) Say, “You should have learned this when I explained it to you last week” and just abandon the client developer, leaving him in the dust in order to crank out delivery.

Choice A has definite value, as it’s the best learning environment there is and builds proficiency in the client developer if the coach is any good. This increased proficiency is the client’s reward for paying confiscatory prices for developer coaches.

Choice C has definite value, as it produces velocity and at least one useful piece of information about the client developer: whether he shrugs, gives up, and goes out to have a smoke, or whether he grabs hold, hangs on, and guts it out immediately, then goes home and sticks his nose in the books to catch up. The client buys unusually high velocity, plus information about whether his developers have what it takes to operate at that level or not.

Choice B has no real value. For a long time I thought it did, but not after long and bitter experience with it. Explanations of that sort seem to have little traction on real-world clients in real-world situations, and they serve chiefly to further frustrate the coach as he abrades his velocity yet again for what he has learned is no good purpose. The client buys low velocity and unimproved proficiency, and the consulting firm looks bad.

We need to keep our coaches choosing Choice A and Choice C appropriately, and rescue them (and our clients, and our companies) from Choice B.

How do we keep our folks in Choices A and C and rescue them from Choice B?

I think we need a changeover point, or at the least a well-defined set of changeover points. An individual coaching engagement should look broadly like this:

1) First, I coach you. Every time a basic question comes up, I’m always in Choice A mode, never Choice C. Make the wrong decision and we’ll have a pleasant, humorous, unhurried talk about why the right decision is better.

2) Now, some well-defined milestone comes up–one that cannot be postponed, at least not indefinitely.

3) Then, I depend on you for delivery. Every basic question automatically gets a Choice C answer. Make the wrong decision and I take the keyboard from you and do it right. Maybe you can cut it; maybe you can’t.

4) Finally (if you can cut it), I trust you to make the right decisions on your own as you crank out the delivery, and I move on to another developer or another team or another engagement.

I should note here that when I say “basic question”, I mean not, “Why are you using a map there instead of a list?” but “Mock? What’s a mock?”

If we have no official changeover point, or if we try to install one but get no support from management, then every coach is constantly having to make judgment calls among Choices A, B, and C, and since he’s always going to be wrong by somebody’s lights for choosing A or C, he’s going to choose B most often, and everybody will suffer.

If we can get a management-supported changeover point, then there’s never any question between Choice A and Choice C for the coach, and he never has to get anywhere near Choice B.

If a manager hassles him about Choice A, he can defend himself: “You agreed that we weren’t going to start seeing real delivery until the changeover point. You want velocity this iteration? Fine, I can give it to you; but it’s going to push the changeover point out an iteration.”

If a client developer hassles him about Choice C, he can defend himself: “Hey, you’ve been through the changeover point. You should know this stuff. Are you a software developer or a princess?”

Another reason for management support: if we get management support for the changeover point, the implication the client developers will take away is that after the changeover point, developers who can’t make the grade may get scraped off. From that implication will come questions like, “Say, can we pick a card with lots of mocking in it this time? I’m still a little weak on mocking, and my changeover point is coming up next week.”

“Changeover point” is pretty vague and ambiguous. What do I mean?

Specifically, I mean a changeover point between coaching and delivery. However, in the real world things are a little more complex.

A client developer has to learn a number of discrete kinds of things during a Pillar coaching engagement. For example, at a recent client, a bunch of Java developers and a few Perl developers had to learn Groovy and Grails. They have to understand the concept of a development environment with CI as its nerve center. They have to learn how to write automated tests in general; then they have to learn how to write each different kind of automated test. They probably have to learn to use new technologies, like DBMaintain or maven or git. They have to learn to do little-design-up-front from the perspective of TDD, which is different from the way you think about design when you’re not testing. They have to learn how to pair, and get over the ego- and agoraphobia-related aversions almost everyone has. They have to learn to think in terms of business value rather than functionality. They have to learn various code smells and how to avoid them, and various code patterns and how to apply them.

And any halfway-competent developer coach is going to be able to at least double the length of this list. It’s unrealistic to expect anyone to be a tyro in all this stuff up to a given point, and then to be an expert in all of it.

So instead of one changeover point, the optimal situation would probably be to have more than one, where you have to concentrate on the basic stuff to pass the first, more advanced stuff to pass the next, and so on.

For example, I really wish we could have seen how BJ Allmon’s plan for a recent client would have worked out.

On one of the teams there, we decided that a new client developer entering the project ought to be able to run a one-point card all by herself (except for very minimal help, on the order of “Hey, you misspelled that!”) from beginning to end, including any defects, after three weeks of coaching. That requires a certain level of aptitude in a few fairly low-level skills, and the developer can get a definite feel for the skills required by going through some representative one-point cards with her coach.

And we decided that that same developer, after six weeks on the team, should be able to run a three-point card all by herself with the same very minimal level of help. Three-point cards require both different skills and more aptitude in already-acquired skills.

We never got to see how this would have worked in real life; but I had high expectations for it. Something like this: “Bill is still an Egg, because he just started, so we’ll be coaching him exclusively. Mary passed her one-point test, so she’s a Tadpole. We’ll coach her too, but harder. Jeff, on the other hand, is through his three-point test and a full-grown Bullfrog. He’s a hair-on-fire high-velocity delivery hound, and solidly in Choice C territory.”

I’m going to recommend that something like this be part of any future coaching engagements I’m part of.

A Parable of Agile Transformation

10 10 2011

“We want you to help us get to the point where we can compete in the Indianapolis 500.”

“Do you want to win it?”

“Well, sure, I guess that’d be nice eventually and all, but for the time being let’s just look at becoming competitive, shall we?”

“Sure, sure.  All right.  Well, first, let’s have a look here at the kind of car you’ll need.  This, for example, is the car that Jacques Lazier drove in 2007.”

“Heh.  That’s pretty funny.  No, we’re pretty happy with the vehicle we have.”

“You’ll never make it with an 18-wheeler.”

“Well, that’s what you’re here for.  Our 18-wheeler can carry 80,000 pounds of cargo.  How much can your funny-looking little car carry?”

“That’s not the point.  The Indy 500 is not about carrying as much weight as you can, it’s about carrying as little weight as you can.  The less weight, the better.”

“But look how small it is!  Where’s the sleeper cab?”

“There’s no sleeper cab.  There’s no second driver.  Indy races aren’t long enough to need more than one driver; they’re over in a few hours.”

“I don’t like the fact that it only has four wheels.  Our truck has duallies for a reason: if we lose a tire miles from help, we can slow down and run on its dual partner until we get somewhere where we can get the tire changed.”

“An Indy car is never more than one lap from a complete full-service rebuild shop, and it goes fast enough that it can coast into the pit with a flat tire or a blown engine.  And anyway, the tires don’t have a chance to get old; we change them at least every hour.”

“Yeah, and that’s another thing.  Reliability.  Our big rumbly low-RPM diesel engine is just better than your little snarly thing: we haven’t had to do any major service on it since the 1980s!”

“But there’s no need for that kind of reliability in the engine of an Indy car: it’s torn down and completely rebuilt after every few hours of operation!”

“And what’s with those stupid-looking wings on the car?  Those’ve gotta be just needless expense.”

“You’ve never experienced the need for a wing, because your truck can’t go fast enough to use one.  But when you get up to Indy-car speeds, you have to have wings because if you don’t you’ll go flying off into the grandstand.”

“But your car isn’t even street-legal!  Where are the headlights?  We’d end up in trouble with the law with that thing!”

“The Indianapolis Motor Speedway is a private track, not a public road.  Street laws don’t apply.  Besides, the Indy 500 is always held in the daytime.  Headlights would be useless extra weight.”

“Well, mostly we were hoping that you could modify our truck so that we could push the accelerator down past level with the floor–you know, to make it go faster.”

Get All These Documents Approved and We’ll Put You Down for Next Quarter’s Release

10 10 2011

How does Agile fit into a company where getting your data supplier to add a field to the data being supplied takes a month?  Where you share read-only access to the development environment with several other teams, and you have to fill out a service ticket to have a change made to it?  Where new technologies and tools have to be approved by department heads before they can be used?  Where development machines cannot be reconfigured by developers, but must be maintained by tech-support personnel?  Where your production hosting service requires six weeks of notice before a production release, and it takes two weeks to assemble the documents describing that release and two more weeks to get them approved?

It doesn’t.  Obviously.  There’s no succession of small changes you can make to a company culture like that that will result in the company becoming Agile.  You can make your team completely test-driven with continuous pairing and everything automated through CI and the most beautiful retrospectives in the industry, and the best it’ll do is get you out to the line you have to stand in a lot quicker.  The line will be just as long and just as slow, and by the time your carefully prioritized stories finally get out to the customer, many of them won’t be of use to him anymore.

That sort of a culture results from layer after layer of process and ceremony being added to catch bad, buggy code before it hits the customer.  By the time it begins to constrain the company enough that it hollers for help, though, the process will be ingrained enough that the people ensnared in it won’t hear or won’t believe that a healthy Agile process produces code that isn’t bad or buggy in the first place.  They may actually be so riddled with Stockholm Syndrome that they like the process and believe the claims that it keeps them safe.

In a case like this, I believe you can’t change the process; you have to bypass the process.  Get a “hall pass” from a powerful ally in the company–perhaps the CIO or the CTO–that gives permission for this one team and this one project to do only what is physically necessary to get a bare-bones, hello-world version of this project into production just as fast as possible.

Hack together your own temporary data-translation code to give you what will eventually come from your data supplier when they get around to it.  (When they do, rip out the hack and use their more complete, professional solution.)  Bonus: if you test-drive the hack, you’ll have automated validation of their solution, when it comes, and if they like they’ll be able to look at your tests for a rigorous definition of what you want them to do.

Send somebody to Best Buy to pick up some machines to turn into pairing stations.  Download whatever you want onto them.  Decide among yourselves whether you’ll even tell the tech-support people about them.  Probably you won’t want to tell the department heads about them, at least not until you’ve gone too far to go back.

Set up your own development environment in a virtual-machine appliance and install it in VirtualBox or VMWare on each development machine and CI. (That idea comes from Justin Searls.) Have your sponsor tell the DBAs to provide you with a JDBC connection (or equivalent) and the necessary firewall holes and leave you the heck alone.

Deploy your code into production in the Cloud.  Provide your own production support for it: if you’re test-driving correctly and deploying automatically, there should be precious little production support to do.

Do whatever you have to do, but get into production!  Make a serious attempt by the end of the first week, so that you can have the bugs worked out and succeed by the end of the second week.  Thereafter, release to production every week at the latest and preferably more often–maybe much more often.

Users will love this–even if your application is really ugly at first.  Product owners will love it, because the short feedback loop will provide lots of guidance and energize the developers to produce massive volumes of value in amazingly short time.  By the time the rest of the IT crowd shambles together to assemble against you, you’ll have a lot of people on your side.

And once you’ve got that first project essentially finished, you will be in a very good position.  First, you’ll have established a proven way that things can demonstrably be done: the IT establishment won’t be able to call it hypothetical pie in the sky.  Second, if you’ve done it right, your project will be under budget, under deadline, and fraught by few or no defects–and the code will likely be the best code anyone in that company’s ever seen.  Finally, you will have presented all the forces for transformation with a terrific argument against all the forces of status quo.  All the product owners will be saying, “I want my product done the new way, not the old way.”  The developers will be saying, “That way looks like a whole lot more fun than this way.”  The CFO will be saying, “You’re telling me that switching everything over to the new way will save us seven million dollars a year?!”  The customers will be saying, “Wow: they sure got a lot more responsive and attentive all of a sudden.  Wait ’til I tell my friends!”

The IT establishment may hate you for that, sure enough; but if they do, they won’t be around long enough for it to be a problem.