U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock () or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Simulations Vs. Case Studies

Breadcrumb

  1. Home
  2. Simulations Vs. Case Studies
Simulations Vs. Case Studies

In the Navy, we used to say that there are three types of officers: those who make things happen, those who watch things happen and those who say, “What happened?”
We can no longer afford “Number 3.” We must support those who make things happen and give the right analytical tools to those assigned to watch things happen; so that everybody knows “what happened.”

In 2017, I wrote an article for Defense AT&L magazine titled “Tabletop Exercises—An Affordable ‘Value-Add’ in the Acquisition Process,” in which I recommended using tabletop exercises when wargames are too hard to develop, fund or schedule. In this current article, I suggest not only that the tabletop is superior to the case study, but that, by applying basic risk, probability and gaming simulation, developers can create a meaningful tabletop or wargame where only a case study previously existed. Unlike the “here it is, read it” structure of the case study, tabletops and wargames are iterative processes, wherein the players work through timely scenarios. The dynamic structure of the simulations helps the players not only to arrive at (hopefully correct) decisions, but to work through decision processes and learn about the effects of those decisions.

Unlike stand-alone case studies, simulations come to life by providing the following:

  • An actual, iterative sequence of events
  • Player participation in the play of the game
  • Immediate (albeit artificial) feedback
  • Critical analyses, versus simple data review
  • New theories for testing—and their implications
  • An early appreciation for the fog of war


Since our college days at least, we have worked with case studies. What Business Administration student could ever forget the Acme Widget Company? Case studies were a great way for inexperienced kids in classrooms to stretch their new brain muscles around problems long past but often repeated. These days, there is only limited value to gaining proficiency with what already happened. We need to go further. Certainly, we need to learn from the past, but we need to take away what is important and apply it optimally to challenges yet to come—and do so quickly.

The Department of Defense (DoD) should take whatever worthwhile case study and history data it can extract from the past and use it to inform and predict the future—discarding what it doesn’t need and modeling the rest to enhance what it does need.

Defining Terms

A case study is a process or record of research in which detailed consideration is given to developing a particular person, group or situation over a period of time—or a particular instance of something used or analyzed in order to illustrate a thesis or principle. The problem with case studies is that often they (at best) stop short of providing a productive mental exercise, or (at worst) leave users with unproven and likely erroneous root causes and conclusions—already arrived at. Data collected from case studies can provide a starting place and (perhaps) a working hypothesis for simulations, and, possibly tabletops and wargames.

A simulation is an imitation of a situation or process, or the action of pretending; deception, or the production of a computer model of something, especially for the purpose of study, analysis and prediction. Many of the criticisms directed at military simulations result from an incorrect application of them as a predictive and analytical tool. Basic simulations tend to produce three sets of results: a best, intermediate and worst-case outcome. It is not my intention here to bury the reader in algorithms and formulae, linear programming, Monte Carlo or the theory of games. The best approach is the one that gets actionable answers without scaring away the participants. We’ll stay at a 50,000 feet elevation for now.

Outcomes supplied by models rely on human interpretation and therefore should not be regarded as providing “gospel truth.”

In a tabletop exercise key personnel who are assigned high-level roles and responsibilities are brought together to deliberate various simulated emergency or rapid response situations. Tabletop exercises (conducted in conference rooms) are often the first opportunity that participants from different commands have to meet and gain appreciation for each other’s capabilities and shortcomings.

A wargame is a type of warfare modeling, including simulation, campaign and systems analysis, and military exercises; and a simulated battle or campaign to test military concepts and uses. Wargames normally are conducted in dedicated facilities with officers acting as opposing staffs; and with actual force members participating. And the games are refereed by umpires.

Each of the four activities as defined has, to varying degrees: objectives, a scenario, and data. That much is basic, and you need it to be informed. However, to analyze and predict, you also need models, rules and players. And analysts. Like umpires, analysts are vital. Analyze the past and predict the future—or else don’t waste your time.

Figure 1 describes where the case study fits in dynamic, predictive modeling and simulation. It reminds the reader that the case study, however informative, is a “done deal” unless and until it becomes part of a greater enterprise.

Figure 2, from my previous article, describes the dynamic nature of a process worthy of the time spent on it. A case study would appear in the “Research” bubble as essentially a finished product—informative but inert. Simulation begins in the Integration bubble. The constant churning of ideas (clockwise arrows) and feedback (counterclockwise arrows) takes developers well past case studies and into tabletops or wargames.

Political-Military Simulations

(Exciting Stuff)
Military simulations (wargames) are models in which theories of warfare can be tested and refined without the need for actual hostilities. They exist in many different forms, with varying degrees of realism. In recent times, their scope has widened to include not only military but also political and social factors. Political-military simulations remain widely used today. Often, modern simulations are concerned not with a potential war between superpowers but more with international cooperation, the rise of global terrorism and smaller brushfire conflicts such as those in Kosovo, Bosnia, Sierra Leone and Sudan.

For years, there have been many charges that computerized models are unrealistic and slanted toward particular outcomes. Critics point to the case of military contractors seeking to sell a weapons system. For obvious reasons of cost, weapons systems are extensively computer modeled. Without testing of its own, the DoD may need to rely largely on the manufacturer’s own models configured to show weapons systems under ideal conditions, with actual operational effectiveness turning out to be less than stated.
Human error is another factor that can render a model/simulation invalid. A programming error (a guided missile cruiser consistently steaming at 70 knots) can produce outrageously incorrect outcomes. Human factors, such as training, expertise and morale frequently lead to programming snags and complications.

Available intelligence (or the lack thereof) brings its own set of snags and complications. Modelers simply may not know accurately the capabilities of opposing forces.
Ideally, political-military simulations should be as realistic as possible—that is, so designed as to provide measurable, repeatable results that can be confirmed by observation of real-world events. This is especially true for simulations that are random in nature (called “stochastic”), as they are used in a manner that is intended to produce useful, predictive outcomes. Any user of simulations must always bear in mind that the simulations are, however, only an approximation of reality and, hence, only as accurate as the model itself.
Disaster preparedness simulation can replicate emergency situations, train first responders, and develop concepts of operation. Disaster preparedness simulation can involve training on how to handle terrorist attacks, natural disasters such as hurricanes, pandemic outbreaks or other life-threatening emergencies.

Management Simulations

Even short of simulating combat situations, (i.e., in actual tabletops and wargames) simulation has contributions to offer in Finance, Project Management, Training, Risk Analysis and Management, Needs Analysis, Supply Chain Management, and general decision making. Basically, any tasks requiring:

  • Evaluation of strategies and core values
  • Life-cycle product or system management
  • Identification and evaluation of alternative approaches
  • Analysis and quantification of strategies, goals and objectives
  • Database development and data analysis
  • Identifying potential synergies and innovations
  • Metrics and measures of effectiveness
  • Assignment responsibilities
  • Performance-based contract administration
  • Actionable courses of corrective action

Project management simulation, for example, is often used for present and future project managers in the private sector. In some cases, simulations are used for “what-if” analyses and for supporting decision making in real projects. The simulation often is conducted using specific software. It also often is used to analyze and evaluate planned and existing projects. The goal of the simulation is to show the user the different possible outcomes of his or her decisions, along with the probability of each outcome. Simulation helps in reducing the project risk and in choosing the optimal approach. In a typical simulation, the project is first modeled with a software tool and use of uncertain variables. A simulation then is run to check the different possible outcomes and their probability as a result of different inputs for the uncertain variables.

The use of simulation throughout a product’s life cycle, especially at the earlier concept and design stages, offers possible benefits, ranging from direct cost reductions as in reduced prototyping and shorter time to service use and better performing products with longer service lives.

Continuous Improvement

Every modern management program, regardless of purpose or focus, must be executed with a continuous improvement mindset; and simulations provide managers with sneak peeks into continuous improvement innovations and opportunities. If a model does not add value and include continuous improvement, it’s not ready for use.
Unlike when dealing with opposing forces, modelers of management-related simulations will likely possess robust data and a high degree of situational awareness—making their validity and contribution greater.

Validation

In the development of simulations, validation is the process of testing a model by supplying it with historical data and comparing its output with the known historical result. If a model can reliably reproduce known results, it is considered to be validated and assumed to be capable of providing predictive outputs within a reasonable degree of uncertainty.

Progressively Gaining Robustness, Payback and “Value Add”

The payback from each of the four approaches described (case study, simulation, tabletop and wargame), like the approaches themselves, directly reflects the level of preparation and execution. Figure 3 and Table 1 demonstrate how progressing from stand-alone case studies, through wargames by the accelerated employment of simulation, takes developers from information of the past to prediction of the future, with attendant increases in utility, comprehensiveness and realism.

Planners and project managers need more than historic information and anecdotal evidence. They need the capability to subject that information to critical analysis, while all the time honing their own analytical skills and professional competence. They need to develop actionable findings and predictions, and to validate them.


Table 1, in applying Figure 3, describes how developers can predict and validate their concepts by increasing the robustness of the effort.
Start with the column marked “Areas for Improvement” which lists the focus areas (feel free to put in your own). Then do the following:

  • Apply mission-centric goals and objectives.
  • Gather available, useful data.
  • Determine what needs to be verified and validated, looking always for areas of potential synergy and innovation.
  • Consider the entire life cycle of involved systems and equipment, including training simulators.
  • Develop tentative conclusions for testing.
  • And then:
  • Develop models and simulations.
  • Work them in realistic scenarios with all positions filled by experts.
  • Develop and/or identify decision points.
  • Identify actionable intelligence for realistic prediction and decision making.
  • Develop feedback loops.

You have now developed the foundation for (at least) a comprehensive tabletop or (at best) a comprehensive wargame.

Summary

Decision making in DoD requires simulations, tabletops and wargames; they must be structured to allow players to make decisions and to measure and predict the impacts of those decisions. DoD needs to embrace simulation not only for realistic warfare planning and training but for responsive project management.
In the classroom, simulations can become a “living” textbook addressing challenges in the present and creating/recreating the curriculum of the future and a vital and indispensable part of the DoD acquisition processes.
In the game room, simulations scrutinize ideas and theories, assessing and predicting outcomes with minimal time and funding requirements with the goals of preserving life and fulfilling the mission. 


Razzetti, a retired U.S. Navy captain, is a management consultant, auditor, and military analyst, and a frequent contributor to both Defense Acquisition and the former Defense AT&L magazines. He is the author of five management books, including Hardening By Auditing—A Handbook for Measurably and Immediately Improving the Security Management of Any Organization, and he has served on the advisory boards of two business schools.

The author can be reached at [email protected].


subscribe