Agile—the Pros and Cons

John M. Nicholson

There is a movement within the Department of Defense (DoD) acquisitions community to become more “Agile” and to field capability more quickly. Achieving this goal requires that the defense acquisition community become more risk tolerant.

Currently, there is a disconnect between the leadership of the DoD and the rest of the acquisition community. That is, DoD acquisition leadership desires to field capabilities more quickly and with more agility, but the middle ranks of the acquisition community seem resistant to the Agile paradigm shift. This disconnect creates a dangerous path to travel for those in the DoD acquisition community trying to plan and execute projects in a more streamlined and agile framework.

But what causes this disconnect? Let’s explore some observations from the perspective of a project lead engineer working on a U.S. Air Force (USAF) sustainment contract for Air Force Space Command (AFSPC) on several Agile projects. The project experience is drawn from software-centric, 1- to 2-year efforts involving 10 to 20 technical staff members—the ideal project category for a mainstream Agile methodology.

Why Go Agile?

The Agile methodology offers numerous advantages over the traditional “waterfall” approach to development. The primary weakness of the traditional approach to development is the assumption that, after the requirements are specified and the project is planned, nothing will change. The traditional development process has no inherent mechanism for dealing with uncertainty. Even worse, the project requirements, schedule and budget are all cemented in place when we know the very least—at the beginning of the project or before it begins—and there is no effective mechanism for accommodating these inevitable changes.

An organic culture means decentralized decision making, less adherence to a chain of command, more autonomy at the worker level, less task specialization and more focus around self-discipline than process discipline.

One way to deal with the inevitable uncertainty and change associated with engineering development is to embrace the change. That is, to admit that change is inevitable and to structure the development methodology and framework to and team to accommodate those changes. This is what Agile development does that traditional processes do not. Agile methods acknowledge that requirements will change and timelines and budgets often shrink. The Agile methodology is structured to accommodate these changes by having the flexibility to modify scope to meet these changes. This allows the Agile project to field capability—possibly less capability than initially expected—despite changes to the requirements, cost or schedule. It usually is preferred to deliver partial capability after a financial investment rather than the “all-or-nothing” approach in the traditional process.

Two Definitions of Discipline

Interestingly, the major strengths of Agile are also among the reasons that it meets resistance in the acquisition community. Often, acquisition managers are not trained on Agile methods and are not comfortable with ambiguity associated with potentially frequent scope changes. To compound the problem, Agile often de-emphasizes formal verification, endless planning, comprehensive documentation and formal processes. That is, Agile methods de-emphasize aspects of the traditional process that the traditional DoD acquisition approach was built around. This change in emphasis is often interpreted and critiqued as having a lack of discipline. This concept of “discipline” has been a major focal point in the debate between so-called Agilists and traditionalists.

However, this disagreement is somewhat of a miscommunication. Traditionalists believe that the lack of formal methods and process adherence is a fatal flaw in the Agile methodology. On the other hand, Agilists believe that formal processes and documentation de-emphasize the self-discipline required for engineering development. That is, one side describes discipline as “process discipline” and the other side describes discipline as “self-discipline.” In reality, both are needed for effective engineering development. Agile methodologies are not absent of process and process discipline, they just use a different approach. Reviews, for example, are conducted on Agile projects, they are just pushed lower in the organization and are accomplished more organically. In fact, many Agile practitioners are process zealots and are fanatical (including the author) about the processes used on their project.

Organizational Limitations

There are two aspects of the organization that cause disconnects between the DoD acquisition leadership and the people in the program offices that execute defense acquisition work. The first cause for this disconnect is so-called “organizational inertia.” In other words, the organization resists change. One reason for this is that the processes and guidance that exist within the organization are built around the current way of doing business. For example, current organizational guidance has specific instructions for performing System Requirements Review (SRR) and then controlling the requirements baseline after through a disciplined change control process. This process is deliberately established to prevent the sort of scope flexibility that Agile methodologies strive to achieve. In some ways, Agile methods contradict the existing way of doing business. Advocates of the existing approach therefore have precedence over the advocates for process change—it’s too radical in some ways. It’s also a bad idea in some cases. However, the existing DoD framework is inflexible and does not condone the concept of Agile.

The second organizational cause of disconnects between DoD acquisition leadership and their program office staff is the organizational structure of many of these organizations. The paperwork-based, process-oriented, traditional waterfall model of engineering development fits nicely into bureaucracies. Bureaucratic organizational cultures notoriously lack innovation and are often at the other end of the culture spectrum of team-oriented, organic organizations that most Agilists operate within. These organic cultures are somewhat foreign to the DoD. Organic culture does not mean wearing flip-flops, playing ping-pong and bringing your dog to work. An organic culture means decentralized decision making, less adherence to a chain of command, more autonomy at the worker level, less task specialization and more focus around self-discipline than process discipline.

The DoD has been in the acquisition business a long time—the acquisition workforce has organized itself in a specific way, establishing process and guidance around the traditional waterfall engineering model and creating an organizational culture that working as a whole creates a formidable amount of friction in implementing Agile processes. Perhaps the most difficult obstacle is the radically different organizational culture established in most Agile commercial organizations and the DoD. For example, adherence to the chain of command is accepted without question in the DoD. Two side effects of the existing DoD culture that stifle the adoption of Agile methods—and innovation in general—are that the punishment for failure outweighs the reward for success in many of our organizations and that this results in a severe aversion to taking risks.

Striving Toward Predictability

Most bureaucracies, including the DoD acquisition community, have established a culture that strives for predictability. Bureaucracies endeavor to improve efficiency and optimize the steady-state. This goal for efficiency and predictability is a reason why the waterfall model is so appealing. The waterfall model fixes the scope, baselines a project plan and then executes as closely as possible to that plan. Indeed, this process discipline often leads to better cost and schedule control on a project.

However, this never-ending quest for efficiency comes at a price—well several prices, actually. First, improving costs and schedule control is accomplished by slavishly following a plan and predetermined scope. Often, requirements change because the project team misunderstood the user needs in the beginning and/or the user changed its mind as the project progressed or the operational environment changed during the project. Regardless of that, the requirements very often changed for a good reason. Therefore, failing to change them degrades the quality or usefulness of the product being developed. Agile methods are one approach to deal with these inevitable changes.

Perhaps more important, the drive for efficiency hampers innovation. That is, innovation involves taking calculated risks. We must try new things in order to innovate. Trying new things requires the possibility of failing. Optimizing the steady-state, by definition, discourages change. Change is not predictable. Therefore, optimization of the steady-state increases competitiveness over the short term at the expense of innovation. Over the long term, organizations driven by efficiency increasingly become obsolete by failing to evolve due to lack of innovation.

Striving to maximize efficiency puts the emphasis on predictability rather than innovation. In this organizational culture, little value is placed on innovation. In some ways, it is more desirable to fail predictably then to succeed unpredictably. Bureaucratic organizations often punish failure more than they reward success. Therefore, in order to innovate, one must take great personal risk to their professional reputation and credibility within their organization.

Contracting Constraints

Regardless of how innovative program offices become, they will be limited by the rules under the Federal Acquisition Regulation. No matter how innovative and Agile organizations become, they will still be bound to execute to contracts that include a statement of work with a fixed scope that the contractor is required to meet—a scope that cannot be easily modified. This also is intended to drive predictability into the process. However, it can serve to stifle project Agility by limiting the flexibility of programs to execute projects.

If the DoD desires to become more innovative and Agile in its methods, the programs’ contracting rules need to be updated. Contracting mechanisms and incentives should examined with a view of maximizing capabilities delivered to the warfighter rather than maximizing predictability in the acquisition process. This is what it means to be truly Agile.

Conclusion

Many DoD acquisition leaders call for their programs to become more Agile and innovative. They want to deliver capabilities to the warfighters more quickly and cheaply. They wish to take more risk and are willing to accept less-than-perfect solutions. This allows more rapid feedback and provides opportunities to improve through incremental capability enhancements.

To achieve this goal, however, leaders must be willing to take action beyond proclaiming “we are going to adopt Agile.” Leaders must be willing to look deeper. They must be willing consider their definition of discipline and evaluate their processes and regulations and be willing to change their organizational structure and help their staffs overcome inertia built into the DoD acquisition system. Leaders must be willing to take a hard look at the organizational culture established in the program offices they lead and decide if they provide employees with enough incentives to innovate. Do they seek to optimize the steady state or to innovate and push the envelope? Are they willing to take more risks? Are they willing to look at the contracting process and be innovative regarding how contractors are incentivized to maximize capability to the warfighter?

In order to become truly Agile, we must seek to encourage organizational change and incentivize those early adopters and project innovators to evolve the DoD acquisition process without having to put their own careers on the line. This involves accepting greater risk and the possibility of failure. Agile methods are not for the faint of heart; they involve ambiguity and uncertainty and are not predictable. They are often less efficient. But they often are faster and more effective than traditional processes. They often provide great value for those who are willing to embrace the culture change and seek innovation over predictability. Making the Agile movement more than just a fad in defense acquisitions is the required culture shift. The question is: Can we take the leap? 

Read the full issue of

Defense AT&L