To Agile or not to Agile?

"Prejudices are what fools use for reason." - Voltaire

I've never much liked Voltaire's philosophy, but he has a knack for quotes. I like this one in particular, because its witty enough -- and ambigious enough -- to be used in nearly every situation about anything you disagree with at all. Kind of like Agile.

Some background

What is Agile? In a sentance: "Agile software development is a conceptual framework for undertaking software engineering projects that embraces and promotes evolutionary change throughout the entire life-cycle of the project." (wikipedia) It achieves its agility through some very cool (and very non-conventional) processes like frequent deliveries, test driven development, pair programming and continual integration. There is much less emphasis on documentation and more emphasis on face-to-face communication than traditional development. It emphasizes short meetings, a client representative and some other very cool ideas. When used appropriately, it really is great. But it can get out of hand.

the problem: its just plain cool

Don't get me wrong, I've been a fan of the Agile concepts for a long time, but lately the amount of hype, and the associated ambiguity, has me frustrated. Decisions become emotional because Agile is "cool" (it is cool, by the way, but that doesnt mean its the best fit for everything). This is, as I see it, the same thing as a prejudice: when one becomes blind to other ideas, regardless of their merit, simply because they are foreign. Worse, it can take on the mindset of a crusade where the valiant (the developers) battle the heathens (the business), which, as I hope you can already tell, is exactly what a profitable tech team should avoid.

the answer: weigh costs against benefits

The key to getting the most out of Agile, not to mention the other 50 years of thinking about software process that preceded it, is to consider when it is best used to solve a problem, and when other methods or processes might be better suited. In process development, you almost always solve problems through trade-offs: when done well, you trade things you dont need for the things you do. As with any trade off, you have to know what you're losing and what youre gaining.

In the next set of blog entries, we'll begin to look at the decision factors for where and when to introduce (or restrict) agile processes in your organization.

Incentive Compensation, Part II

This one is simple: we're structuring a formal incentive compensation plan where I work and there is, not surprisingly, some disagreement internally.

I believe that most people, and likely most comp plans, would tend toward the individual: reward people based on their contribution right? I believe that is a flawed view because, at least in our firm, it is cooperation that gets the job done. Individual productivity has some merit, of course, but, as a web company, we do not make money from beatiful designs or lines of code. Instead, we profit through the value we provide. We need to encourage that teamwork and not, as I pointed out in the last blog, fodder for their resume or portfolio.

We do, however, have to counterbalance people who might slack while others work hard. To do so, we create demand for efficient workers by rewarding profitable projects, and split the rewards based on demand for that employees services, which we can also measure relatively easily.

So for us, the measure becomes (in order of priority):

1. Total company profit (measure of long term value to our customers)


2. Membership on profitable projects (measure of short term value)


3. Relative utilization (demand for individual services)

All in a nice little package. With some proper reporting and communication, these will go far to incent people to work in their own and the greater (company) good. More on this as we have some results.

What does Jamestown Teach us about Incentive Compensation?

This weekend, Jamestown, Virgina celebrates its 400th birthday. For those of you that don't remember your grade school history, Jamestown was a settlement in the Virgina colony, which was privately funded and run by the Virgina Company under a charter granted by King James I of England. Jamestown, as an investment, was a terrible one -- it lost tons of money and, obviously much worse, lives. Neither the addition of a desirable cash crop (tobacco), slaves, or fines for bad behavior seemed to help until, in 1623, the colony established private property rights. With the advent of private property, the company aligned rewards with behavior which led to success for the investors and colonists of the struggling Virgina Company. In this article, I'll point out a few parallels between the group behavior in 17th century Virigina and your technical department, and point out why aligning rewards with desired behavior is so important.

Jamestown was worse than your tech department

Life in Jamestown was bad. Very bad. 80% of the settlement died in the winter of 1609-1610 and, worse, some attribute the massive loss of life to lazy colonists (Morgan). Upon his arrival less than 2 years after the devastating winter, their new leader, Sir Thomas Dale found the colonists at "their daily and usuall workes, bowling in the streetes" (Morgan). Dale put laws into place that forced them to grow corn so they didn't die. The system of punishments was overbearing by most accounts; it helped, but not enough. It was not until the advent of private property in 1623 that the starvation had been reversed (Boettke). With private property, the colonists could see a direct correlation between the quality of their work and their personal gain. With incentives properly aligned with the goals of the company (the settlement was, after all, an investment by a group of individuals) the settlement thrived.

Let's take a minute to process this: people were dying and it took someone, a very mean person by most accounts, to force people to plant food instead of going bowling. To survive. It is estimated that the colonists worked, on average, about 3-4 hours a day (Morgan) even though if they just worked a little more, lives would have been saved. Even the religiously based Plymouth colony in Massachusetts had similar experiences until they instituted private property (Bethell). One thing is very clear from these examples: even the best intentioned have difficulty working together for the greater good when not properly incented to do so.

my tech department isnt lazy, they work a lot

But are they working for you or themselves? A problem in this field, as I'm sure in many, is that there is greater incentive to work for your next job than your current one. With technology, it is often what you can list on your resume that will land you your next job and not what you've done for your employer, in terms of their success. Of the hundreds of resumes I've reviewed, rarely have I seen anything related to company performance. As a Programmer, or even a higher level position like System Architect, why bother? The recruiters do keyword searches, and many technical managers barely, if at all, understand the technologies used in their departments. In short, there is a very good chance that your company is paying people to work inefficiently so they can keep their resume current.

How Do I Know if they're doing it the right way?

That's tough; you probably won't. You you may, as I did, catch someone doing something really stupid like using web services to do post a form to itself, but removing that contractor doesn't do much overall. The problem is deeper than that: it is too expensive to review everything people do -- you must give them a good reason to work in the interests of the company.

You may want to respond to inefficient workers like the Virginia company first did: get tough. Getting tough, as the Virigia company soon learned, doesn't work either. In less than 3 months after he instituted his harsh laws, Sir Thomas Dale was removed from office (Morgan). Even if you do last longer than Dale did, you run the risk that your best team members may leave for greener pastures (many left Jamestown for other settlements to the north). In short, heavy handed dicipline is a no-win option.

Develop appropriate Metrics and Incentives

The best way, overall, is to make sure that the technical team is measured -- and rewarded -- based on their impact on the company. Further, this reward must exceed that of jumping ship with an extra buzzword in their pocket. If you can't measure the benefit of the technical team on the company's overall operations -- if there is no benefit to having the tech team -- then find one soon or quit. CEOs are getting smarter about technology and if you can't find the value, they probably can't either. The mystery that existed in prior decades of the technical genius is over; now you've got to prove your value or be risk being replaced.

One way to further encourage good behavior is to neutralize the benefits of bad behavior. Rather than let new technologies run wild and impact project success, develop a routine training and evaluation program where people are encouraged to learn a technology that may be relevant to something you're already doing. Budget time to investigate emerging technologies, and determine which can directly benefit the business. The advantages will be more than just a happy tech team; if done properly, you increase shareholder value. Happy shareholders and a happy, productive and efficient team: what could be better?

References

Bethell, Thomas. "How Private Property Saved the Pilgrims." Hoover Digest Online (1999) http://www.hoover.org/publications/digest/3507051.html

Boettke, Peter. "The Role of Private Property in a Free Society." Virginia Viewpont. http://www.virginiainstitute.org/viewpoint/2005_04_2.html

Diamond, Sigmund. "From Organization to Society: Virginia in the Seventeenth Century." The American Journal of Sociology. March, 1958.

Morgan, Edmund S. "The Labor Problem at Jamestown, 1607-18." American Historical Review. 1971. pages 595-611

Scarce Resources: Part II

With the last post, I described a common problem in most any technical team: scarce resources. You'll hear, in virtually every technical team anywhere, "we don't have enough resources" and they're almost always talking about people. They're correct, but not always in the way they think.

Are we doing the right things?

Let's start with something simple: requirements documentation. Most people would agree you need requirements in some form, although the specific methods and extent has been debated for nearly as long as software has existed. How do we know enough is enough? How much is too much?

The answer, it turns out, is simple; we simply ask ourselves, during the process: does what I'm doing improve the project or could the time be better spent somewhere else? Does adding this sentance make the specification more clear, or does it serve to confuse? If I only had another hour, would I continue to add more and more detail on how the login function works, or could I better spend that hour detailing a much higher risk requirement, more likely to require clarification?

These sound like simple questions, but even with experienced professionals, they are rarely asked. I once remember reviewing a project requirements specification that spent three pages documenting a login function that had already been built and deployed on another project. We knew how to build it. The client knew how it worked. The time could have been much better spent elsewhere.

How much did it cost?

In the above example, we spent our time and resources detailing a login page that added nearly no value. Let's assume it took 4 hours to add the extra unnessasary documentation. What was the cost? It was not, as first suggested to me, $400 (4 hours x internal loaded resource cost of $100/hr). Let's assume we spent those 4 hours doing the best available alternative: talking to the client to clarify and detail the riskiest point of the project. Had that prevented only one complex defect from being injected during the requirements stage, we might have saved days of work in testing, removing and re-deploying the defect. Our mis-spent 4 hours had, in effect, cost us no less than 24 hours fixing, 3 hours testing, and an hour of deployment time, for a grand total of $2800 (28 hours x $100/hr). When we factor in indirect and intangible costs like client perception (an inhibitor to future work), client meetings to discuss and diagnose the defect, and potential delays to the schedule, we see that our 3 page login spec was quite costly indeed.

28 Hours for a Single Defect: Isn't that a bit much?

Not at all. Barry Boehm first suggested that defects injected in the requirements stage can cost between 10 and 100 times the cost to fix if found in later phases (post launch, for example) in his landmark book, Software Engineering Economics (published in 1981). While that number has been hotly contested, my experience is that the concept, if not the precise numbers, are generally correct. I suspect most software engineers can think of more extreme cases with much larger numbers than I've used as a ballpark here.

In Summary

Focus throughout the project on the best available activities is an important part of keeping a project high quality and on schedule for a given budget. It is no less important for the requirements team (as illustrated here) as it is for later project participants: architects, programmers, quality assurance and, of course, project managers.

More on those in a future entry.

What are your scarce resources?

If you are a PMI certified project manager, you may have caught a quote of mine in the cover story (available in PDF form), where I laid out a simple -- but often forgotten -- principle: "every firm is short on some resource -- cash, employees, executive mindshare, etc. -- so you've got to pick the best opportunity." This idea of scarce resources is the foundation of Neoclassical economics, not to mention a very important priciple in technical design and project management, but somehow the importance of the idea is often ignored.

This will be the first in a series of blog entries about the importance of considering scarce resources. In a recent article, I gave some interview tips, so lets take hiring as the starting point.

Scarce resource: In House Talent

Consider a case in point: at a former company, we had several seasonal accounts, one of which could, at a moments notice, double or triple the resources required, then needs could drop by the same or greater amount. At another company, outsourcing might have been a good option, but several factors prevented it in this scenario: culture, contractual obligations, and the almost patented ambiguity that define technical specifications at advertising-based firms. To be successful, something had to change, but what?

Take Inventory: What do you have and what do you need?

Let's say that its May now. We know that the we've got a launch soon, so we'll probably have slightly more technical resources than we need just after the launch. We don't know what is coming in when, but a lot of it will come in with almost no time to respond.

At first glance, one might assume the scarce resource is programming talent, but more than enough talent exists in our market at prices we were willing to pay. Our scarce resource, the thing we could not afford to spend, was time of the senior people during busy season.

Thinking in terms of exactly what we need, something obvious emerges: the contractors we need are not at all like our full time employees. We need contractors who can do a narrow set of things, without a lot of direction, and do it fast -- but how do we find them reliably?

The solution: Allocate Scarce Resources Appropriately

Here we have at least a few options:

1. Begin looking for people now using the existing process: Use a lot of senior resource time now, and potentially a lot during busy season


2. Hire later using an abbreviated version of our existing interview process: Use very little senior resources up front, but potentially huge abouts during busy season if the contractors are sub-standard.


3. Hire later using a completely new specialized interview process:Use a lot of senior resources now (when we have them) to define the test, but, if done well, very little when they are most in demand.

The third option, keeping in mind which resources are scarce, is clearly the best option. Here is what we did:


We established specialized interview roles for team members that would work with the contractors, develope a practical test that simulated the working environment and tested ONLY the key skills they would need during the period. In under half a day, we were able to accurately access, compare, and make offers. Many candidates were objectively weeded out in the first half hour, minimizing our time spent evaluating unsuitable candidates, allowing more time to discern the good from the great.

Several years later, the creative work on the account has since moved on, but the technical work remains at my old firm, where they continue to use the same process successfully today.

In Summary

You obviously can't solve every problem by an analysis of scarce resources (risk is a major factor), but it is far too often removed from the decision process entirely -- especially in determining things like which projects to take on (the subject of this the May edition of PM Network) and when evaluating overall technical strategy.

Recommended Related Readings

A few books I thought about while writing this blog entry. I think you'll enjoy any and all.

Fisher, Robert. Getting to Yes: Negotiating Agreement Without Giving In (1992)
People negotiate all the time. This is the classic text on how to make lasting and successful negotations by understanding what you -- and they -- really want. If you enjoy this, I would also recommend the Harvard Program on Negotiation Newsletter.

Nalebuff, Barry and Ian Ayres. Why Not?: How to Use Everyday Ingenuity to Solve Problems Big And Small (2006)
A very entertaining look at 4 seperate methods of solving problems. This book may turn the way you solve problems -- and the way you eat bananas -- upside down.

Polya, G. How to Solve It (1945)
Intended for mathematics educators and students, this books shows how to solve virtually any problem. A new edition came out in 2004, which I have not read, but I assume it is as good as the original. Also quite different in content and approach to Why Not?, both have similar underlying messages about the way to solve problems.

Skousen, Mark. The Making of Modern Economics: The Lives and Ideas of the Great Thinkers (2001)
A brilliant and highly entertaining history of economics, the perculiar yet fascinating lives of its major contributors, and the evolution of its major schools of thought.

welcome!

Hello Everyone!

My name is Steven Mocarski and I'm the CTO at Singularity Design, a growing center city Philadelphia web design and development company. I've spent my career using technology to help companies:

• reach consumers (like I did with Campbell's Soup Company),


• work more efficiently with partners and customers (like I did with Hunter Douglas)


• communicate internally (like I did with the NFL and the US Senate).

Over the last 10 years, I've worked in 4 different countries to help a wide range of companies from startups like Alliegience Technology Partners to media giants like Martha Stewart Omnimedia to Fortune 500 companies like IBM, Hewlett Packard , and Wyeth. I've seen sparkling successes and spectacular failures; I'm here to try to help my readers find a way to learn from all of it.

Some of you who know me might be expecting detailed technical commentary on software development languages, tools, or process -- you wont find that here; I'll leave that to people like my former employees Steve Eichert and David Solivan. I'm hoping to take a business focused stance, and only dive into the details when its nessasary to prove or disprove a given concept or theory. As my former boss and mentor used to say, "...or not." If that doesnt prove valuable, we'll find a path that does.

Enjoy!