Project Design - A Call for Action

Juval Lowy, October 2013

What is Project Design

The software development industry has grown to accept architecture and system design as a required part of every software effort. Even approaches that formally eschewed any amount of initial design now acknowledge the need to invest up-front in architecture. There are proven methodologies of accomplishing architecture even in the face of unknown and shifting requirements. We learned how to encapsulate change and design a robust system, without knowing the last detail of the design or the code itself.

Much as you to design the software system, you must design the project to build the system: from accurately calculating the planned duration and cost, to devising several good execution options, scheduling resources, and even validating your plan, to ensure it is sensible and feasible, and that your team can deliver with acceptable risk, on the schedule and budget.

It's About Time

As software engineering is coming of age, there is a natural convergence with other more traditional engineering disciplines. Some 20 years ago software architects were practically non-existing, yet today they are commonplace. But software architects are nothing more than the traditional system engineer in other domains. Even Agile as a methodology has strong correlation to lean manufacturing and even TQM. It should not be a surprise to see project design making a début in the software world - after all, the other engineering disciplines have been designing their projects for decades. The techniques of software project design are specific to software projects, but in the abstract there is nothing new - these are the same ideas that engineers have been using in classic engineering projects since the 1950's. To use an analogy, if I ask you to design a system that is maintainable, reusable, extensible, secure (or safe), and of high quality, you cannot tell if I am taking about a mechanical system or a software system. In fact, the approach, the Zen of achieving these goals is the same in software as it is in traditional engineering.

Much the same way, if I ask you to design a project that will comply with a set budget and deadline, within acceptable risk, be traceable and manageable, you cannot tell if I am talking about a bridge or an ERP system. And not surprisingly, in the abstract, the techniques for designing both types of projects are identical.

Common Misconceptions

There are several misconceptions regarding project design. The first is that it is part of project management. In fact, project design is not project management. The correct relationship between project design and project management is what architecture is to programming. Furthermore, devising a good project design stems from your system architecture and doing it properly is a hard-core engineering task – designing the project. This design task requires both the project manager and the architect to work closely together to determine the best overall plan.

The second misconception is that project design contradicts Agile-like development processes. Nothing could be further from the truth. Project design, like architecture, is not mutually-exclusive with Agile. We know that you can design a system in a preliminary sprint focused on architecture, even though you don’t know exactly what you are going to build. You can do the same with project design. Note that both architecture and project design are activities, while Agile is a development process.

The third misconception is that project design is waterfall-like. In reality, project design is the opposite of the waterfall, because it forces you to face head-on the realities of interleaving all the various activities in the project, across developers, completion phases, iterations and milestones, and design the project as you are going to build it, as opposed to a fictional waterfall.

The last misconception is that project design is time consuming, and under an aggressive deadline you can't afford it. Don’t be concerned. A seasoned architect can design the project in a matter of a few days for each given set of planning assumptions.

Project Design and Project Sanity

Project design allows you to shed light on dark corners, and have up-front visibility on the true scope of the project. Project design forces managers to think through work before it begins, to recognize unsuspected relationships and limits, to represent all activities, and to recognize several options for building the system. It allows the organization to determine whether it even wants to get the project done. After all, if the true cost and duration will exceed the acceptable limits, why start the work in the first place, only to have the project canceled once you run out of money or time.

As such, once project design is in place, you eliminate the commonplace gambling, death marches, the wishful thinking and the horrendously expensive trial and errors. A well-designed project also lays the foundation for enabling managers to evaluate and think through impact of a change once work commences, and then to assess the impact of the change request on the schedule and the budget. This enables the project manger and the architect to keep the project on time, all the time.

Assembly Instructions

Yet there is much more to project design than proper decision making. The project design also serves as the system assembly instructions. To use an analogy, would you buy a well-designed IKEA set without the assembly instructions? Regardless of how comfortable or convenient the furniture is, you will recoil at the mere though of trying to guess where each of the hundreds of pins, bolts, screws, and plates go, and in which order.

Your software system is significantly more complex, and yet architects presume developers and projects managers can just go about assembling the system, figuring it out as they go along, without mistakes. While that is possible, it is clearly not the most efficient way of assembling the system. What project design produces is the project plan, much as the system design activity produces the architecture. If the architecture is the "what", the project plan is the "how" - your system assembly instructions. The very nature of producing the assembly instructions is yet another reason why project design is an engineering task rather than a project management one.

A System of Options

What may not be obvious from the discussion so far is that a given project design is not a single pair of schedule and cost. It is a very different project if you only have one developer or six, if you are allowed to engage sub-contractors at key phases, if you try to build the system as fast as you can or with the least possible cost, or if you try to avoid risks and maximize the probability for success. When you design a project you must provide management with several options trading cost, schedule and risk, allowing management and decision makers to choose up-front the solution that best fits their needs and expectations. Providing options in project design is in fact the key to success. Clearly, if time is of no consequence then you should build project at lowest cost. If cost is of no consequence then you must build the project in the fastest way possible. But in reality, cost and schedule always matter, and the best solution is found between these two extremes. Finding a balanced solution and even an optimal solution is a highly engineered design task. I say it is engineered because engineering is all about tradeoffs and accommodating reality. With project design, there is no single correct solution even for the same constraints, much as there are several possible design approaches for any system. It is the task of the project manager and the architect to narrow this spectrum to several good options for management to choose from. Specifically, when you design a project you must provide with reasonable certainty design options such as:

A normal solution not subjected to constraints
The most economical way to meet the deadline
The fastest way to deliver on set cost
A constrained solution with limited resources
How these options vary with regard to quantified risk

Often you will have to even recommend the best overall plan across possible architectures, schedules, cost, resources and risk.

If you don't provide these options, you will have none to blame but yourself. How common it is for the architect to labor on the design of the system, present it to management, only to have them ordain "You have a year and four developers". Any correlation between that year and the four developers and what it really takes to deliver the system is accidental, and so are your chances of success. But if the same architect were to present the same architecture, accompanied by 3-4 project design options, all of them doable, but reflecting different trade of cost, schedule and risk? Now the discussion revolves around which of these options to choose out of the menu. I call this strategy "Cake or Pie". Failure however, is not on the menu.

Call for Action

I wrote this page so far extolling the virtues of project design, and yet I have said nothing about how to go about doing it. That is deliberate - there is no way I can do project design justice in three pages. But it takes only a few days to learn some basic skills of project design, enough to make a huge difference. The same is true with software architecture - armed with basic understanding of encapsulation and decoupling you can do wonders. On the other hand, the body of knowledge of project design is as wide and deep as that of system architecture. It also takes a commensurable amount of time and experience to do well and fast, just as with system architecture.

Consequently I frequently witness a cognitive bias in our industry: most people assume that because they can't do project design or even worse, because they have never seen it done properly, then it can't be done. Let me assert here and now that is nonsense. The same could have been said about distributed systems architecture twenty years ago. There are plenty of well-designed software projects. I have personally spent comparable time over the years with the IDesign customers on their system architecture and the project design supporting it. I have educated and mentored hundreds of architects and project leads on project design as well as my own techniques and breakthroughs. The results speak for themselves, with success story after success story. Don't be fooled by the common cognitive bias. Remember, that the absence of evidence is not evidence of absence. Do invest in learning more and then mastering the crucial act of project design. It will set your career on a different trajectory, restore confidence between managers, developers and customers, improve communication all around, reduce overall tension, and greatly increase your chance of success.