Predictive Modeling in Naval Supply Systems

LCDR Duncan R. Ellis, USN

Naval Supply Systems Command, Weapon Systems Support (NAVSUP WSS), is the Navy’s Program Support Inventory Control Point (PSICP) for a globally dispersed force consisting of nearly 300 deployable ships, 92 submarines, and 3,700 aircraft. WSS manages a total inventory of $34 billion, processing more than 500,000 yearly demands from Navy, Marine Corps, Joint and Allied Forces customers. To ensure the operations readiness of this diverse and distributed customer base, NAVSUP WSS must determine what materiel to buy, how much to buy, when to buy it and where to place it. Each of these problems is complicated—doing all four well is a formidable challenge.

As part of its reform effort, and in coordination with the Office of the Secretary of Defense (OSD) Comprehensive Inventory Management Improvement Plan (CIMIP), NAVSUP has been heavily focused on efforts to improve all areas of its inventory planning. One particular focus area is improving how the command informs wholesale inventory policy decisions—the backbone of the Navy supply system.

NAVSUP WSS manages a diverse wholesale inventory profile primarily made up of depot level repairable (DLR) items. Repairable items represent a particularly difficult challenge for inventory managers due to the complex nature of managing a pipeline of items that are often repaired and reissued. To construct inventory policy for repairable items, inventory managers must consider several factors, including:

• Customer demand for the item, which is often determined as part of a forecasting process.
• Return rates for the carcasses of items that fail. This rate represents the likelihood that once a failure is experienced, the broken item, referred to as a carcass, is returned as retrograde for repair.
• Repair rates for carcasses that are returned. This rate represents the likelihood that a damaged item can be successfully repaired.
• Lead times for repairing an item, whether done at a Navy Depot or by a commercial vender. Inventory managers also must consider lead times involved in procuring a new item from a vendor, which is required to augment pipeline losses from failed repair or missed returns, as well as provide additional assets if demand for the item increases.

To manage this complex process, NAVSUP WSS uses Navy Enterprise Resource Planning (ERP). Within the Supply Chain Management (SCM) Module of ERP, the Distributed Requirements Planning (DRP) engine takes input from forecasting, augmented with expertise from the command’s inventory planners, and creates a 5-year “buy/repair” plan for each item. The “buy/repair” plan aims to keep inventory levels at wholesale above an average minimum target level, referred to as a Planned Minimum Safety Stock Level, or sometimes simply as a safety stock. These Planned Minimum Safety Stock Levels, established for every item, are derived through sophisticated optimization models that balance wholesale sustainment objectives against capital inventory budget constraints.

Before discussing any specific approach to tackling the task of setting policy for wholesale safety stocks, it’s worth taking a moment to discuss analytics and modeling in general. These often are described on a continuum, ranging from descriptive models, then predictive, and finally prescriptive.

Descriptive analytics show organizations what has happened in the past. Enterprise metrics briefs often are primarily descriptive, with managers meeting periodically to review past performance. Good descriptive analytics are critical for organizations to assess the impacts of their business decisions.

Predictive models show what might happen. Rather than simply describing past behavior, these models lend insight into possible futures. While predictive modeling is useful for inventory models, often the sheer magnitude of the decision space makes it infeasible to choose enterprise level policy by simply looking at raw results from predictive models. The goal is to build prescriptive models.

A prescriptive model informs “what to do” given specific predictions. In the context of wholesale policy, the desire is for a model that can prescribe recommended safety stock recommendations across large numbers of items.

WIOM is able to reduce period-to-period item level churn from more than 50 percent of line items to between 10 percent and 20 percent of line items, with simulation testing showing no detriment to existing customer service levels.

Tackling complex business problems is not new to NAVSUP WSS. To support inventory modeling across the NAVSUP enterprise, the command has engaged with the Navy Postgraduate School (NPS) Operations Research Department in a long-term partnership that started in 2014 with development of the Wholesale Inventory Optimization Model (WIOM) that currently is in production, setting safety levels on more than 50,000 NAVSUP WSS items. WIOM recommends safety levels that meet fill rate objectives and reduce “inventory policy churn” while ensuring that capital inventory costs are constrained.

Inventory policy churn control, as implemented in WIOM, was a feature not found on the previous commercial inventory model. Churn results when model recommendations are not consistent from period to period. While it is desirable that models react to new information, long procurement and repair times reward stability in investment decisions. WIOM is able to reduce period-to-period item level churn from more than 50 percent of line items to between 10 percent and 20 percent of line items, with simulation testing showing no detriment to existing customer service levels.

Following the success of WIOM, the suite of NAVSUP WSS/NPS models was expanded to provide recommendations for Navy-owned consumable materiel procured from the Defense Logistics Agency. The Site Level Inventory Optimization Model (SIOM) sets order quantity and order level recommendations for consumable materiel held at locations around the world, while reducing inventory policy churn.

The most recent NAVSUP WSS/NPS developed model is the Naval Aviation Readiness Based Sparing Model (NAVARM). This model will set recommended max stock levels for repairable and consumable aviation materiel on carriers, large deck amphibious ships, air stations and smaller pack-up-kits. Similar to both SIOM and WIOM, NAVARM includes an inventory policy churn control mechanism.

Although the WIOM model has performed well since its production implementation in 2017, NAVSUP WSS maintains an aggressive culture of continuous improvement. This drives the command to continuously assess current models and processes, always seeking to make improvements. NAVSUP WSS models are developed using a spiral development model in which lessons from the current version of the model are used to inform future variants.

A particular challenge in managing NAVSUP WSS materiel is the low demand for many of the items. While the command manages a diverse and large portfolio of items, the individual demand for most items is quite low. Within any 3-year window, more than 30 percent of all repairable aviation demand is unique to a single fiscal year, and for maritime items that number is as high as 50 percent. Management of items where the intermittency between customer requisitions is often measured in years means that there might not be enough historical data to produce reliable and robust forecasts for any particular item’s demand, repair success rate, carcass return rate or replenishment lead time. Unlike a consumer retail organization that has the ability to drop poor performers from its portfolio, NAVSUP WSS must strive to support every customer requirement, regardless of how infrequent. This support is critical to maintaining the operational readiness of our deployed forces.

Forecasts created for low-demand materiel items are subject to large degrees of error. However, it is important not to fall into the trap of thinking that these errors are a result of deficiencies in the forecasting methodologies. They simply highlight the real difficulty of planning for items with little transactional history, especially when the limited set of historical demand suggests highly variable inter-arrival and replenishment times. However, even though an item is not “forecastable,” the command is not absolved of the responsibility to set coherent inventory policy and provide the best possible support to the fleet.

Forecasting is a critical intermediate function the output of which is used to inform other planning processes. Forecast model output for inventory planners, often expressed as materiel arrival rates or replenishment times, are widely used as inputs for inventory optimization models. This means that any error or bias in the forecast will result in suspect inventory policy recommendations.

Consider a typical paradigm for many current models. First, historical transactional data, perhaps customer orders or lead times, is used to create forecasts. Traditionally these forecasts are expressed as a point estimate, or single number. Sometimes a variance, or sense of uncertainty, accompanies the forecast. These forecasts values are often used to select mathematical probability functions that are then used in the underlying “engines” of the models to produce predictions. This closed-form forecast-based paradigm can be problematic for many reasons.

For items with a great deal of transactional history, the forecasting process as outlined above reduces complex transactional data to only one or two values, such as a point estimate and a variance. Often these numbers are insufficient to explain complex historical patterns. Forecasts produced from items with too little historical data often provide an unwarranted confidence in predictions. Finally, plugging these forecasts into functions, as mentioned above, requires myriad mathematical assumptions that often do not hold up well.

Recent improvements in computational power, including the availability of large computing clusters, mean that model builders can explore new architectures that were not feasible over the last decade. One alternative to the closed-form forecast-based approach described above is to leverage the power of simulation.

Simulation models are widely used and accepted in many areas and industries, as in exploring factory production facilities or evaluating queuing and service systems. Military applications of simulations run the range from highly detailed kinetic simulations of weapon systems to broad campaign level models. Properly built, a simulation model can provide enormous insight into the behavior of incredibly complex systems. To be reliable, a simulation must accurately describe the essential parts of the system and then be supplied with sufficient input data. As output, a simulation can provide a robust set of measures and metrics that can be used to inform decisions makers.

WIOM 5.0 Model Overview

WSS is currently working on the next generation of the WIOM model that couples a sophisticated inventory simulation with a flexible optimization model to set more reliable safety stock levels across a wider range of materiel than is possible with current tools. Figure 1 shows the new model’s high-level architecture. The major components are connected to a large relational database that houses not only input and output data but also model parameters and metric definitions. This one-stop shop for both model data and parameters simplifies the archiving of model results and ensures that runs are reproducible and auditable. The architecture borrows best practices from software engineering, allowing for a modular and extensible package in which individual components are interfaced to produce a modeling tool set.

Transactional data from the Navy ERP system is input and processed into the model. This input data includes more than a million past records of item orders, repair lead times, procurement lead times, as well as item-level detail like price, shelf life and budget group.

For each item, a series of safety stock decisions will be evaluated. Each item and candidate safety stock level will be run through a simulation multiple times. During each simulation, descriptive metrics will be calculated, including, but not limited to, item fill rates, back orders and contracting wait times. Conducting a large number of simulations for each of the candidate safety stock solutions allows for the model to measure variation across all metrics of interest. While previous NAVSUP models, including the current generation of WIOM, only provide a limited set of output metrics, the flexible nature of simulation allows the model builder to measure almost any desired parameter. This flexibility makes simulation models inherently more extensible. When future processes change, simulation models can quickly adapt. And simulation outputs can produce volumes of transactional data, allowing managers and decision makers to access high levels of detail regarding a particular inventory policy. This might include detailed assessment of inventory levels over time, contracting workload, or backorders.

Results from the WIOM simulation will be tabulated and the results uploaded into the database. The user then has the option to use this data to construct an optimization run. WIOM 5.0 will produce a flexible optimization architecture that allows the user wide flexibility as to what objectives and constraints should be used in prescribing recommended inventory levels. For example, a user might wish to produce a set of recommended safety levels that maximize the demand weighted fill-rate of items, while minimizing the fill-rate variance. They also might choose to minimize churn, while imposing a hard constraint of less than $1 billion in safety level inventory. WIOM 5.0 allows weights to be established at the item or metric level, so that the model user can weigh both the relative importance of items in the model run, as well as the relative importance of metrics used in the optimization. Once the optimization is complete, the user will be able to view summary reports, assess the output data and export it for load back in the Navy ERP production system.

The NAVSUP WSS WIOM development team is targeting model delivery, testing and production integration for completion this year. Representing the future of Navy wholesale inventory modeling, the capability provided by this architecture will support increased Navy readiness and lethality while establishing NAVSUP WSS and NPS on the leading edge of applied advanced analytic techniques.

Read the full issue of
Defense Acquisition magazine

ELLIS is an operations research analyst for Weapons Systems Support at the Naval Supply Systems Command in Mechanicsburg, Pennsylvania.

The author can be contacted at [email protected].

The views expressed in this article are those of the author alone and not the Department of Defense. Reproduction or reposting of articles from Defense Acquisition magazine should credit the authors and the magazine.