U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock () or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

  1. Home
  2. Communities of Practice
  3. Manufacturing and Quality
CommentCommunity

Manufacturing and Quality

The Manufacturing and Quality (M&Q) Community of Practice (CoP) is a compilation of policy, guidance, processes, references, resources, and tools. for completing M&Q activities across the DoD system acquisition life cycle. The community managers will work periodically to update this CoP based on current policy, guidance, tools, best practices and lessons learned. The complexities of various production environments, from low volume (job shop) to high volume (continuous flow), make the management of M&Q functions especially challenging. This CoP is dedicated to providing M&Q and other technical personnel with access to the knowledge needed, when needed.  

 

 

 

 

 

 

 

 

30
Members

Views

Pinned Content

COMMUNITY / Manufacturing and Quality
Systems Engineering Process Support
COMMUNITY / Manufacturing and Quality
Contracting for Manufacturing and Quality
COMMUNITY / Manufacturing and Quality
Advanced Manufacturing / Additive Manufacturing
COMMUNITY / Manufacturing and Quality
Industrial Base / Supply Chain Management
COMMUNITY / Manufacturing and Quality
Quality Assurance and Control
COMMUNITY / Manufacturing and Quality
Manufacturing Management and Risk Assessment
COMMUNITY / Manufacturing and Quality
Workforce Development, Training and Education
COMMUNITY / Manufacturing and Quality
Manufacturing and Quality (M&Q) Body of Knowledge (BoK)

Community Contacts

COMMUNITY / Manufacturing and Quality
George A Noyes III - Community Leader
COMMUNITY / Manufacturing and Quality
Robert Arthur - Community Owner
COMMUNITY / Manufacturing and Quality
Tim Mead - Community Leader

Feed / Manufacturing and Quality

Resource / Manufacturing and Quality
Systems Engineering Process Support
View Resource

Systems Engineering is a disciplined approach for the specification, design, development, realization, technical management, operation, and retirement of a weapon system. SE is an interdisciplinary and collaborative effort requiring close interaction with many disciplines to include operations, maintenance, logistics, test, production, quality, etc. The practice of SE is composed of 16 processes: 8 technical processes and 8 technical management processes. These 16 processes provide a structured approach to increasing the technical maturity of a system, increasing the likelihood that the capability being developed balances mission performance with cost, schedule, risks, and design considerations. The DoD Systems Engineering model is located below, and M&Q personnel need to support these activities and processes.

 Systems Engineering Process | www.dau.edu

Source: DoD Systems Engineering Guidebook https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

Eight Technical Processes that may require M&Q participation:

  • Stakeholder Requirements Definition, Systems Engineering Guide, 4.2.1 
  • Requirements Analysis, Systems Engineering Guide, 4.2.2
  • Architecture Design, Systems Engineering Guide, 4.2.3
  • Implementation, Systems Engineering Guide, 4.2.4
  • Integration, Systems Engineering Guide, 4.2.5
  • Verification, Systems Engineering Guide, 4.2.6
  • Validation, Systems Engineering Guide, 4.2.7
  • Transition, Systems Engineering Guide, 4.2.8

Eight Technical Management Processes that may require M&Q participation:

  • Technical Planning, Systems Engineering Guidebook, 4.1.1  
  • Decision Analysis, Systems Engineering Guidebook,4.1.2
  • Technical Assessment, Systems Engineering Guidebook, 4.1.3  
  • Requirements Management, Systems Engineering Guidebook, 4.1.4   
  • Risk Management, Systems Engineering Guidebook, 4.1.5  
  • Configuration Management, Systems Engineering Guidebook, 4.1.6  
  • Technical Data Management, Systems Engineering Guidebook, 4.1.7  
  • Interface Management, Systems Engineering Guidebook, 4.1.8  

The industry standard for systems engineering is ISO/IEC/IEEE 15288, “Systems and Software Engineering–System Life Cycle Processes has a slightly different list of technical and technical management processes, which is described in the graphic below:

Technical Management Processes

  • Project Planning
  • Project Assessment and Control 
  • Decision Management 
  • Risk Management 
  • Configuration Management
  • Information Management
  • Measurement 
  • Quality Assurance

Technical Processes 

  • Business or Mission Analysis
  • Stakeholder Needs and Requirements Definition
  • System Requirements Definition
  • Architecture Definition
  • Design Definition
  • System Analysis
  • Implementation
  • Integration
  • Verification
  • Transition
  • Validation
  • Operation
  • Maintenance 
  • Disposal

Source: DoD Best Practices for Using Systems Engineering Standards ((ISO/IEC/IEEE 15288, IEEE 15288.1, and IEEE 15288.2) on Contracts for Department of Defense Acquisition Programs

This resource page will focus on the following Pinned Content:

  • Systems Engineering Resources and Guidance
  • Systems Engineering Tools and Checklist
  • Technical Reviews and Audits
  • Producibility Engineering 
  • Key Characteristics 
  • Producibility Best Practices 

DoD Systems Engineering Guidebook https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

DAU Continuous Learning Modules and other training: 

Systems Engineering Plan (SEP) 

The Systems Engineering Plan (SEP) is a planning and management tool that defines the methods used for implementing all system requirements having technical content, technical staffing, and technical management. The SEP is specific to a program and is an important tool for managing complex technology-based system developments. The SEP should be tailored to meet program needs and objectives.

The purpose of the Systems Engineering Plan (SEP) is to help Program Managers develop, communicate, and manage the overall systems engineering (SE) approach that guides all technical activities of the program. The SEP documents key technical risks, processes, resources, metrics, SE products, and completed and scheduled SE activities. The SEP is a living document that should be updated as needed to reflect the program’s evolving SE approach and/or plans and current status.

The SEP describes the integration of SE activities with other program management and control efforts, including the Acquisition Strategy, Integrated Master Plan (IMP), Work Breakdown Structure (WBS), Integrated Master Schedule (IMS), Risk Management Plan, Technical Performance Measures (TPMs) and other documentation fundamental to successful program execution. The SEP also describes the program’s technical requirements, engineering resources and management, and technical activities and products as well as the planning, timing, conduct, and success criteria of event-driven SE technical reviews throughout the acquisition life cycle.

The SEP is the program’s blueprint for the conduct, management, and control of all technical activities, the SEP captures decisions made during the technical planning process and communicates objectives and guidance to program personnel and other stakeholders. The SEP should define the “who, what, when, why, and how” of the SE approach and should include the following: 

  • The program organization with roles and responsibilities, authority, accountability, and staffing resources.
  • Key activities, resources, tools, and events that support execution of the SE technical processes and technical management processes.
  • The event-driven technical reviews to be conducted and the approach to technical reviews based on successful completion of key activities as opposed to calendar-based deadlines.
  • The approach for how requirements and technical performance trade-offs are balanced within the larger program scope to deliver operationally effective, suitable, and affordable systems.
  • The identification of key design considerations and criteria.
  • The use of and employment of modular design.
  • The identification of how manufacturing and quality planning will be incorporated into the SEP and systems engineering processes.
  • The identification of and management of Key Performance Parameters (KPPs) and Key System Attributes (KSAs) and development of a prototyping strategy to ensure system requirements can be met within cost and schedule constraints.
  • The program’s strategy for identifying, prioritizing, and selecting the set of TPMs and metrics (TPMM) should provide sufficient insight into the technical progress and program risks.

The Systems Engineering Plan Outline:

1 Introduction
2 Program Technical Definition
   2.1 Requirements Development
   2.2 Architectures and Interface Control
   2.3 Specialty Engineering
   2.4 Modeling Strategy
   2.5 Design Considerations
   2.6 Technical Certifications
3 Program Technical Management
   3.1 Technical Planning
         3.1.1 Technical Schedule
         3.1.2 Maturity Assessment Planning
         3.1.3 Technical Structure and Organization
   3.2 Technical Tracking
         3.2.1 Technical Risk, Issue, and Opportunity Management
         3.2.2 Technical Performance Measures
         3.2.3 Reliability and Maintainability Engineering
         3.2.4 Manufacturing and Quality Engineering
         3.2.5 Human Systems Engineering
         3.2.6 System Safety
         3.2.7 Corrosion Prevention and Control
         3.2.8 Software Engineering
         3.2.9 Technology Insertion and Refresh
         3.2.10 Configuration and Change Management
         3.2.11 Technical Data Management
         3.2.12 System Security Engineering
         3.2.13 Technical Reviews, Audits and Activities
Appendix A - Acronyms
Appendix B - Item Unique Identification Implementation Plan
Appendix C - Agile and Development Security and Operations Software Development Metrics
Appendix D - Concept of Operations Description
Appendix E - Digital Engineering Implementation Plan
References

SEP Guidance and other Resources: 

DAU Systems Engineering Plan training videos:

  • Several videos are available via a web search 
Systems Engineering Resources and Guidance
Systems Engineering Tools and Checklist 
A Comprehensive List of Tools to Aid the Engineering Community 

This document identifies tools that could be used to help manage DoD acquisition technical, business, and management processes to include many manufacturing and quality activities. Most of these tools support systems engineering technical and technical management processes, but a few can be used to support business processes such as cost estimating, contract language, acquisition strategies, etc. Some tools could be used by a contractor, some by government personnel and some tools can bd used by many different people in many functional specialties. Most tools are available from multiple on-line sources, some tools may need to be purchased to use, many are free. 

Note: You can find more information on the tools listed below with a search of the web. 

Tools (listed in alphabetical order include):

  • 3Ps - Production Preparation Process
  • 5S’s (Sort, Straighten, Shine, Standardize, Sustain
  • 5-Whys
  • 7 - Basic Tools for Quality Improvement (includes the following which are also discussed separately):
    • Cause and Effect Diagram
    • Check Sheet
    • Control Chart
    • Histogram
    • Pareto Chart
    • Run Chart
    • Scatter Diagram
    • Stratification or Flowchart or 
  • 7 - Management and Planning Tools, or Advanced Tools for Quality Improvement (includes the following which are also discussed separately):
    • Affinity Diagram
    • Relations Diagram or Interrelationship Diagraph
    • Tree Diagram
    • Matrix Diagram
    • Matrix Data Analysis
    • Arrow Diagram
    • Process Decision Program Chart (PDPC)
  • 8D/PSP (Eight Disciplines/Problem Solving Process)
  • A3 Problem Solving Chart
  • Acceptable Quality Levels (AQL)
  • Acceptance Sampling 
  • Active Risk Manager (ARM)
  • Affinity Diagram
  • Advanced Product Quality Planning (APQP) Core Tools
  • Arrow Diagram (Chart)
  • AS6500 Manufacturing Management System (MMS)
  • AS9100 Advanced Quality Management Systems (QMS)
  • AS9103 Variation Management of Key Characteristics
  • AS9110 Maintenance
  • AS9120 Distributors
  • AS9102 FAI
  • AS9115 Software QA
  • AS9131 N/C Document
  • AS9132 Marking
  • AS9133 Supplier QA
  • AS 9137 AQAP Align
  • AS 9138 Statistical Process Control
  • AS9162 Self Verification
  • Axiomatic Design 
  • Balanced Scorecard
  • Baldrige Performance Excellence Criteria
  • Bekidou Rate
  • Benchmarking
  • Bill of Materials (BOM) 
  • Bone Diagram
  • Bottleneck Analysis
  • Box and Whisker Plot
  • Bubble Chart
  • Capacity Matrix
  • Capacity Analysis
  • Cause and Effect Diagram (Fishbone or IshIkawa)
  • Cause and Effect Matrix
  • CFMEA – Concept Failure, Mode and Effect Analysis
  • Check Sheet
  • Chokko Rate
  • Computer-Aided Design (CAD)
  • Computer-Aided Manufacturing (CAM)
  • Computer Aided Process Planning (CAPP)
  • Computer-Aided Three-Dimensional Interactive Application (CATIA)
  • Computer Integrated Manufacturing (CIM)
  • Consensogram
  • Contingency Planning
  • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart -U-chart for Attribute Data
    • Control Chart - NP-chart for Attribute Data
    • Control Chart - P-chart for Attribute Data
    • Control Chart - X-barR chart for Variable Data (measurable)
    • Control Chart - X-bar-S chart for Variable Data
    • Control Chart - X-MR/I-MR chart for Variable Data
  • Correlation Chart (Scatter Diagram)
  • Cost/Benefit Analysis
  • Cost of Quality Analysis
  • Cost of Quality (COQ)
  • Cost Modeling (Estimating)
  • Criteria Testing
  • Critical Chain Project Management 
  • Critical Design Review Checklist (DoD)
  • Critical Path/PERT
  • Critical to the Customer (CTC)
  • Critical to Quality (CTQ) Tree
  • Customer Contingency Table
  • DCOV – Define, Characterize, Optimize and Verify
  • Deming Cycle or Wheel (PDCA)
  • Departmental Purpose Analysis
  • Design of Experiments (DoE)
  • DFMEA - Design Failure Mode and Effects Analysis 
  • DFMA - Design for Manufacturing and Assembly 
  • Design to Cost (DTC)
  • DFSS - Design for Six Sigma 
  • DMAIC - Define, Measure, Analyze, Improve and Control
  • DMADV (see DCOV)
  • Domainal Mapping
    • Factory Modeling and Simulation
    • Producibility Analysis & Ergonomics
    • Process Planning 
    • Production Planning & Scheduling
    • Line Balancing & Bottleneck Analysis
    • Capacity Planning
    • Predictive Analytics & Optimization
    • Facility Planning, Layout and Design
    • Virtual Factory Mock-up
  • Failure Mode and Effects Analysis (FMEA)
  • Fault Tree Analysis 
  • First Article Inspection
  • First Article Testing 
  • Flow Chart or Process Flow Chart
  • Force Field Analysis
  • Gage R&R Studies
  • Gantt Chart
  • Histogram (Frequency or Bar Chart)
  • Hoshin Kanri (Quality Policy Deployment) 
  • Interrelationship Diagraph (also see Relations Diagram, or Network Diagram)
  • ISO 9001 Quality Management Systems (QMS) 
  • Kano Model
  • KJ model - Kawalota Jiro (see affinity diagram)
  • Lead Time Analysis
  • Learning Curve 
  • Learning Curve Analysis
  • Line of Balance (LOB)
  • Taguchi Loss Function
  • Lotus Diagram
  • Manufacturing Cost Estimating
  • Manufacturing Plan 
  • Manufacturing Readiness Assessment (MRA)
  • Manufacturing Readiness Level (MRL) Criteria
  • Matrix Diagram
  • Matrix Data Analysis Diagram
  • Measurement Systems Analysis (MSA) 
  • MIL-HDBK-896A Manufacturing and Quality Program
  • Multi-Vari Charts
  • Nominal Group Technique
  • One Piece Flow
  • Operations Process Chart
  • Overall Equipment Effectiveness (OEE)
  • Pareto Charts (Template)
  • Part-Family Analysis
  • Paynter Chart
  • P-Diagram or Parameter Design 
  • PERT Chart (Program Evaluation Routine Technique)
  • PFMEA – Process Failure, Mode and Effect Analysis
  • Pie Chart
  • Preliminary Design Review Checklist (DoD) 
  • Preliminary Hazards List (PHL)
  • Process Capability Studies (Cp and Cpk)
  • Process Performance Studies (Pp and Ppk)
  • Process Decision Program Chart (PDPC)
  • Producibility Analysis/Assessments
  • Producibility Assessment Worksheet
  • Producibility Engineering and Planning (PEP) Program
  • Production Part Approval Process
  • Production Readiness Review (PRR) Checklist
  • Programmatic Evaluation of ESOH (PESHE)
  • Pugh Matrix
  • Quadrant Chart
  • Quality Function Deployment (QFD)
  • Queuing Theory/Waiting Line Analysis
  • Radar Chart
  • Rational DOORS
  • Relation Diagram
  • Reliability Growth Analysis
  • Requirements Verification (Traceability) Matrix (RVM)
  • Risk Management Assessment Tool 
  • Route Sheet
  • Run Chart
  • Scatter Diagram (Mind Mapping)
  • SIPOC – System, Input, Process, Output and Customer
  • Six Sigma
  • SMART – Specific, Measurable, Attainable, Resources, Time
  • Spaghetti Diagram
  • Spider Diagram
  • Stratification
  • Statistical Process Control (SPC)
  • Supply Chain Management Risk Assessment
  • Swim Lane Chart (sometimes called a Deployment Flow Chart)
  • SWOT Model (Strength, Weaknesses, Opportunities and Threats
  • Systems Engineering Plan (SEP) 
  • Takt Time Analysis
  • Technical Risk Identification and Mitigation System (TRIMS)
  • Technology Readiness Level (TRL) Checklist
  • Theory of Constraints Analysis
  • Throughput Analysis Tool
  • Throughput Accounting
  • Tolerance Analysis 
  • Tolerance Design 
  • Total Productive Maintenance (TPM) 
  • Trade Studies/Analysis
  • Transition to Production (Willoughby Templates)
  • Tree Diagram
  • TRIZ Matrix 
  • Value Stream Mapping (VSM) 
  • Variability Reduction Program
  • Venn Diagram
  • Work Center
  • Work Measurement
  • X-Matrix
  • Yamazumi Chart

Note: You can find a lot of additional information by googling the tool, by visiting a number of different academic sites, or professional organizations, or by visiting various Communities of Practice (CoPs).

Quality of Design / Quality by Design / Quality Engineering 

Quality By Design – Based on Juran Institute Processes

Quality by Design (Quality of Design or Concurrent Engineering) is the process of creating a design using multidisciplinary teams (IPTs) to conduct conceptual thinking, product design, and production planning all at one time. Quality by Design was developed by Dr. J. M. Juran who believed that quality should be built into a product from the beginning, and that most quality issues are related to the product's original design. 

Quality by Design differs from traditional sequential engineering practices and uses a five step process of Define, Discover, Design, Develop, and Deliver. Each process step has a focus and utilizes a variety of advanced quality and technical tools to meet customer requirements (satisfies the customer) often focusing on the eight dimensions of quality to create a design that is optimized for performance, cost and schedule. Quality by Design should require input from many different technical teams to include manufacturing and quality. It is the job of systems/design engineers to create the design, and it is the role of these technical personnel is to “influence the design” for producibility, manufacturability, reliability and maintainability, testability, and sustainability.

Define

Describe in general terms what the product is and what set of customers it is intended to serve. Establish the team, identify the customers and stakeholders, set goals, and create plans:

  • Operational Needs and Requirements
  • Sustainment Requirements
  • IOC Date
  • Weapon System Performance 
  • Cost (Procurement and Sustainment)
Discover

Discover the exact needs of the customer expressed in terms of the benefit to the customer. Collect and prioritize customer needs and translate those needs and create:

 

  • Work Breakdown Structure (WBS)
  • Functional Allocation 
  • Allocated Baseline
  • Measures of Effectiveness (MoE)
  • Key Performance Parameters (KPP) 
  • Measures of Performance (MoP)
  • Technical Performance Measures (TPM)
  • “Critical to Quality” measures. 

 

Develop planning worksheets for:

 

  • Customer requirements
  • Allocate Functions and Features
  • Allocate Measures and Goals (MoE, KPP, MoP, TPM and Critical to Quality
  • Detailed design features and goals 
  • Manufacturing Process features and goals 
  • Manufacturing Control features and goals

 
Design

Design a product that will meet those needs better than competitors’ and preceding products. Establish high-level product features and goals: (functional design), then develop detailed product features and goals (detailed design), optimize design features, set and approve final design.

 

  • Product Baseline
  • Functional
  • Allocated Design
  • Detail Design 
  • Design reviews and approvals
  • Identify and manage key and critical characteristics
  • Producibility Engineering
  • Trade Studies and Trade-off Analysis
  • Design Reviews
  • Above activities may require the use of the following advanced engineering techniques:
    • Design of Experiments 
    • Quality Function Deployment
    • Design for Manufacturing and Assembly (DFMA)
    • Design for X
    • Parameter Desing 

    • Tolerance Design 

       
Develop

Produce or manufacture the product by identifying, managing, controlling manufacturing processes and control.

 

  • Create manufacturing process planning (route sheets, flow diagrams, etc.)
  • Create assembly charts and operations process charts
  • Understand and control manufacturing processes:
    • Set goals for processes
    • Develop control plans
    • Establish audit procedures
    • Capture feedback and continuously improve
  • Manage capacity, identify bottlenecks:
    • Theory of Constraints
    • Optimize throughput
  • Optimize processes: 
    • Lean
    • Six Sigma
    • Standard Stable Processes
    • Identify and manage critical process parameters
    • Demonstrate process capability
    • Optimize process capability 
  • Manage Quality: 
    • Establish a Quality Management System
    • Identify critical quality attributes
    • Control variation
    • Statistical Process Controls
  • Manage Supply Chain 
Deliver

Plan for the transfer of the product to the customer, then measure and manage customer satisfaction.

 

  • Conforms to requirements
  • Performs as expected
  • Reliable product
  • On-time
  • Defect free
Technical Reviews and Audits

For DoD systems development, a properly tailored series of technical reviews and audits provide key points throughout the system development to evaluate significant achievements and assess technical maturity and risk. Technical reviews of program progress should be event driven and conducted when the system under development meets the review entrance criteria as documented in the SEP. An associated activity is to identify technical risks associated with achieving entrance criteria at each of these points. SE is an event-driven process based on successful completion of key events as opposed to arbitrary calendar dates. As such, the SEP should clarify the timing of events in relation to other SE and program events.

Technical reviews of program progress should be event driven and conducted when the system under development meets the review entrance criteria as documented in the program’s Systems Engineering Plan (SEP). An associated activity is to identify technical risks associated with achieving entrance criteria at each of these points (see the DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs). Systems Engineering (SE) is an event-driven process based on successful completion of key events as opposed to arbitrary calendar dates. As such, the SEP should clarify the timing of events in relation to other SE and program events. While the initial SEP and Integrated Master Schedule (IMS) have the expected occurrence in the time of various milestones (such as overall system Critical Design Review (CDR)), the plan should be updated to reflect changes to the actual timing of SE activities, reviews and decisions. Figure 3-1 of the SE Guidebook provides the end-to-end perspective and the integration of SE technical reviews and audits across all AAF pathways. Technical reviews should be tailored appropriately for other acquisition pathway.

The DoD Systems Engineering Guidebook provides guidance on support of the following technical reviews and audits https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

  • Alternative Systems Review (ASR): Is a technical review that assesses the preliminary materiel solutions that have been developed during the MSA Phase. The review ensures that one or more proposed materiel solution(s) have the best potential to be cost-effective, affordable, operationally effective, and suitable, and can be developed to provide a timely solution at an acceptable level of risk to satisfy the capabilities listed in an ICD.  The ASR helps the PM and Systems Engineer ensure that further engineering and technical analysis needed to draft the system performance specification is consistent with customer needs. https://acqnotes.com/acqnote/tasks/alternative-systems-review-2 
  • System Requirements Review (SRR): Is a formal review conducted to ensure that system requirements have been completely and properly identified and that a mutual understanding exists between the government and the contractor. It ensures that the system under review can proceed into initial systems development and that all system and performance requirements derived from the ICD or draft CDD are defined and testable, and are consistent with cost, schedule, risks, technology readiness, and other system constraints. https://acqnotes.com/acqnote/acquisitions/system-requirements-review-srr 
  • System Functional Review (SFR): Is a technical review to ensure that the system’s functional baseline is established and can satisfy the requirements of the ICD or draft CDD within the currently allocated budget and schedule. It also determines whether the system’s lower-level performance requirements are fully defined and consistent with the system concept and whether lower-level systems requirements trace to top-level system performance requirements. https://acqnotes.com/acqnote/acquisitions/system-functional-review 
  • Preliminary Design Review (PDR): Is a technical assessment that establishes the Allocated Baseline of a system to ensure a system is operationally effective.  A PDR is conducted before the start of detailed design and is the first opportunity for the Government to observe the Contractor’s hardware and software designs.  This review assesses the allocated design documented in subsystem product specifications for each configuration item in the system. It ensures that each function in the Functional Baseline has been allocated to one or more system configuration items. A PDR is required by statute for all Major Defense Acquisition Programs (MDAPs). https://acqnotes.com/acqnote/acquisitions/preliminary-design-review 
  • Critical Design Review (CDR): Is a multi-disciplined independent technical assessment to ensure that a system can proceed into fabrication, demonstration, and test and meet stated performance requirements within cost, schedule, and risk.  A successful CDR is predicated upon a determination that the detailed design satisfies the CDD.  Multiple CDRs may be held for key Configuration Items (CI) and/or at each subsystem level, culminating in a system-level CDR. https://acqnotes.com/acqnote/acquisitions/critical-design-review 
  • System Verification Review (SVR): Is a product and process assessment to ensure the system under review can proceed into LRIP and FRP within cost, schedule, risk, and other system constraints during the EMD Phase. It assesses the system functionality and determines if it meets the functional requirements in the CDD documented in the functional baseline. The SVR establishes and verifies final product performance and provides inputs to the CPD. https://acqnotes.com/acqnote/acquisitions/system-verification-review-svr#google_vignette 
  • Functional Configuration Audit (FCA): Examines the functional characteristics of the configured product. It verifies that the product has met the requirements specified in its Functional Baseline documentation approved at the PDR and CDR.  The FCA reviews the configuration item’s test and analysis data to validate that the intended function meets the system performance specification. The audit is more systems engineering-focused than program management official auditing. https://acqnotes.com/acqnote/tasks/functional-configuration-audit-2 
  • Production Readiness Review (PRR): Assesses a program to determine if the design is ready for production. It evaluates if the prime contractor and major subcontractors have accomplished adequate production planning without incurring unacceptable risks that will breach thresholds of schedule, performance, cost, or other established criteria. https://acqnotes.com/acqnote/acquisitions/production-readiness-review 
  • Physical Configuration Audit (PCA): Is a formal technical review that determines if the configuration of a system or item has met its documented requirements to establish a product baseline. The Milestone Decision Authority (MDA) receives proof from a successful PCA that the product design is stable, the capability satisfies end-user needs, and the production risks are tolerably low. https://acqnotes.com/acqnote/tasks/physical-configuration-auditaudit 
  • Independent Technical Risk Assessment (ITRA): is a formal review that is independent of the program office and is conducted in advance of milestone and production decisions in order to provide senior leaders with an independent review of a programs technical risks, including the maturity of critical technologies and manufacturing processes,

DAU Continuous Training Modules and other training:

Guidance and other resources:

Digital Engineering

Digital Engineering (DE), Digital Twin, Digital Threads, and Digital Models: 

Digital Engineering: Digital Engineering can be used to support all of the Systems Engineering functions. MIL-HDBK-539 Digital Engineering and Modeling Practices defines digital engineering as “an integrated, computation-based approach that uses authoritative sources of system data and models as a continuum across disciplines to support lifecycle activities.” DE is a cutting-edge approach that uses authoritative sources of system data and models throughout the development and life of a system. Digital engineering harnesses computational technology, modeling, analytics, and data sciences to update traditional systems engineering practices. In the face of increasing global challenges and dynamic threat environments, digital engineering is a necessary practice to support acquisition.

DoDI 5000.97-Digital Engineering calls for the use of digital engineering methodologies, technologies and practices across the life cycle of defense acquisition programs.  Further, the document:

  •  Mandates the incorporation of digital engineering for all new programs (exceptions can be granted by decision authority)
  •  Directs Components to use digital engineering practices in requirements, cost, business and sustainment.
  • Calls for the replacement of documents with the use of digital models as the primary means of communicating system information. 
  • During program planning and contracting, requires that appropriate data rights be obtained.
  • Singles out DAU as providing workforce training on digital engineering.

The diagram below, from the instruction, captures the digital Engineering framework:

The figure describes the DoD Digital Engineering Ecosystem

Digital Twin: Every product produced, and every process executed is unique. There can be thousands, of key input variables used to describe products, assets, entire lines, and processes.  A Digital Twin is a virtual replica of a physical object, manufacturing process, or plant that is designed to capture, map, and structure process variables into a continuously updated database. This database can be used by an organization to monitor, analyze, design, or optimize the process without having to go out into the field or do costly trials on the physical equipment. By making this data more readily available in a digital environment, teams can use data in other applications, models, or third-party programs to make meaningful discoveries.

Digital Thread: A digital thread is a framework that connects data about a product throughout its lifecycle, from design to disposal. It uses a variety of technologies, including computer-aided design (CAD), product lifecycle management (PLM), and Internet of Things (IoT) sensors, to collect and share data. A manufacturing digital thread is designed to expand upon a digital twin. Put simply, the digital thread captures digital twin data as they evolve. As manufacturers evolve their processes and their digital twins adapt to new settings or recipe changes, the digital thread encapsulates the link between these evolutions. 

The digital thread should seamlessly advance the controlled interplay of technical data, software, information, and knowledge in the digital engineering ecosystem. Digital threads are used to connect authoritative data and orchestrate digital models and information across a system’s life cycle. The digital thread informs decision makers throughout a system’s life cycle by providing the capability to access, integrate, and transform data into actionable information. The digital thread should support feedback over the life cycle.

Example Data Elements or Digital Artifacts

Engineering Design Data

Technical Product Data

Manufacturing and Quality Data

Enterprise Data
  • Producibility Analysis
  • CAD Data
  • Modeling and Simulation Data
  • Special Tooling Data
  • Special Inspection Equipment Data
  • Digital Drawings
  • Digital Models
  • Specifications
  • Standards
  • Critical Manufacturing Processes
  • Configuration Data
  • Interface Management Data
  • Analytical Data
  • Bill of Material
  • Manufacturing Floor Layout
  • Production Line Data
  • Pilot Line Data
  • LRIP/FRP Data
  • Industrial Engineering Data
  • Production Data 
  • Machining Instructions
  • Customer Demand Data
  • Rates and Quantities Data
  • Work Breakdown Structure Data
  • Supplier Data
  • FRACAS Data 
  • Test Plans
  • Schedules
  • Product Support Strategy

Digital Models Include:

  • Requirements Model
  • Structural Model
  • Functional Model
  • Architecture Model
  • Business Process Model 
  • Enterprise Model
  • Human Performance Model
  • Product Life Cycle Model 

DAU Continuous Learning Modules and other training: 

Guidance and other Resources:

Digital Engineering Body of Knowledge (DEBoK) 

The Digital Engineering Body of Knowledge (DEBoK) will serve as a reference for the DoD engineering community to use in implementing digital engineering practices starting with systems engineering and expanding to specific disciplines, engineering domains and specialty areas. The BoK will store collective data, information and knowledge on digital engineering. Members of the government, industry and academia working within this space will be able to contribute to the DEBoK and build their digital engineering solutions based on collective knowledge. Access the DoD DE BoK briefing at https://ndiastorage.blob.core.usgovcloudapi.net/ndia/2021/systems/Wed_23770_Zimmerman_Davidson_Salvatore.pdf 

As a best practice, when conducting early M&Q engineering analysis, the technical team should consider DE principles, methods, and tools. DE best practices and tools are defined in the DE Body of Knowledge (DEBoK). https://de-bok.org/ 

The DEBoK is also available to DoD Common Access Card users at the Defense Technical information Center (DTIC) website: 

https://www.dodtechipedia.mil/dodwiki/pages/viewpage.action?page Id=760447627 

Digital Engineering Strategy and other Resources 

DoD’s Digital Engineering Strategy provides guiding principles and promotes consistency in engineering processes through the use and reuse of digital tools, models, and curated data throughout the program’s life cycle. As a best practice, the technical team should consider M&Q digital data requirements (e.g. factory floor modeling and simulation, digital technical data packages and work instructions, digital data in supply chains) during early establishment and development of the digital thread.

Digital Engineering: An integrated digital approach that uses authoritative sources of systems’ data and models as a continuum across disciplines to support lifecycle activities from concept through disposal. Access to Digital Engineering Fundamentals can be found at the following urls:

https://www.cto.mil/wp-content/uploads/2023/06/Dig-Eng-Fundamentals-2022.pdf 

https://ac.cto.mil/wp-content/uploads/2019/06/DE-Fundamentals.pdf 

Digital Engineering Ecosystem: The interconnected infrastructure, environment, and methodology (process, methods, and tools) used to store, access, analyze, and visualize evolving systems’ data and models to address the needs of the stakeholders. End-to-end digital enterprise. 

Digital Artifact: An artifact produced within, or generated from, the digital engineering ecosystem. These artifacts provide data for alternative views to visualize, communicate, and deliver data, information, and knowledge to stakeholders.

Guidance and other Resources:

Modeling and Simulation (M&S) 

A model is a physical, mathematical, or logical representation of a system, entity, phenomenon, or process. Manufacturing models include plant diagrams, flow charts, 5Ms chart, 

A simulation is the implementation of a model over time, showing how the model works, and can be live, virtual, or constructive. Manufacturing simulations include 

The use of models and simulations in engineering is well recognized. Simulation technology is an essential tool for engineers in all application domains. A digital model represents an actual or conceptual system that involves physics, mathematics, or logical expressions. A simulation is a method for implementing a model over time. Together models and simulations allow the Department to vet potential requirements prior to the Request for Proposal release, assess engineering change orders or program upgrades, etc. M&S can be used to assess and optimize resource usage, examine process changes, support supply-chain management routing and inventory quantities, business decisions, etc.

Models and simulations are SE tools used by multiple functional area disciplines. Models, simulations, data, and other artifacts should be developed and used in a well-defined and controlled engineering ecosystem to support an effort’s reuse of the information across the life cycle of activities. Models, simulations, data, and artifacts should be integrated, managed, and controlled to ensure that the products maintain consistency with the system and external dependencies and provide a comprehensive view of the effort and increase efficiency and confidence throughout the project’s life span.

Systems Engineering process related tools:

Note: Each of these technical and technical management processes have commercial software tools that can be used to support these processes. A web search starting with Model-Based Systems Engineering (MBSE) tools should provide many links.

Systems Engineering Technical Processes  Tool Capabilities and Features
Stakeholder Requirements Definition
  • Assists in capturing and identifying stakeholder requirements
  • Assists in analyzing and maintaining stakeholder requirements
Requirements Analysis
  • Assists in requirements definition and decomposition 
  • Interfaces with architecting tools 
  • Supports requirements validation
Architecture Design
  • Assists in development of functional and physical architectures 
  • Provides traceability among system elements 
  • Supports multiple views
Implementation 
  • Assists in development of the system design, prototypes and alternate solutions 
  • Assists in realization of the system, system elements and enabling system elements
Integration 
  • Assists in integration-planning activities 
  • Assists in assembling lower-level system elements into successively higher-level system elements 
  • Provides analysis and simulation capability
Verification
  • Assists in determining the system and system elements performance as designed through demonstration, examination, analysis and test
Validation
  • Assists in determining, the effectiveness, suitability and survivability of the system in meeting end-user needs
Transition 
  • Assists in planning and executing delivery and deploying of the system to the end user for use in the operational environment
Systems Engineering Technical Management Processes  
Decision Analysis
  • Assists in trade-off analysis  Provides optimization and sensitivity analysis capability 
  • Assists in recording, tracking, evaluating, and reporting decision outcomes
Technical Planning 
  • Assists in planning and scheduling activities 
  • Assists in resource planning, tracking, and allocation  Facilitates cost estimation
Technical Assessment 
  • Assists in tracking, measuring, and assessing metrics 
  • Assists in metric collection
Requirements Management Provides requirements bi-directional traceability capability  Provides requirements flow-down capability  Tracks requirements changes
Risk Management
  • Assists in risk, issue, and opportunity planning, identification, analysis, mitigation/management and monitoring
Configuration Management
  • Assists in the identification of configuration items 
  • Assists in baseline/version control of all configuration items 
  • Assists in ensuring configuration baselines and changes are identified, recorded, evaluated, approved, incorporated and verified
Technical Data Management
  • Assists in identification of data requirements 
  • Assists in recording and managing data rights 
  • Assists in storage, maintenance, control, use and exchange of data including digital artifacts 
  • Assists in document preparation, update, and analysis
Interface Management 
  • Assists in capturing system internal and external interfaces and their requirement specifications 
  • Assists in assessing compliance of interfaces among system elements of the system or systems of systems 
  • Produces a view of interface connectivity
   

MBSE Software Tools:

  • Requirements Management software provides a single, centralized platform to store, organize, and manage requirements, which enables better collaboration and communication among team members and stakeholders. Traceability provides end-to-end traceability between requirements, system elements, and their associated models, which ensures consistency throughout the development process and simplifies change management. Examples include Visure Requirements Platform, Siemens Teamcenter, and Sparx Systems Enterprise Architect to name a few. Listing these tools here is not an endorsement of these tools.
  • Product Data Management (PDM) software manages design and engineering files such as CAD models and manufacturing instructions allowing teams to collaborate across concurrent design environments. PDM is mostly used by manufacturing companies to control product data from design to production. This type of software is beneficial for designers creating the initial specifications of a new product and production managers following manufacturing instructions. There are approximately 57 PDM software tool available to engineers. Examples include Autodesk Vault, Teamcenter, and Solidworks to name a few. Listing these tools here is not an endorsement of these tools.
  • Product Lifecycle Management (PLM) software manages all of the information and processes at every step of a product or service lifecycle across globalized supply chains. Today’s PLM software provides the foundation and intersection of critical, cradle-to-grave product lifecycle processes woven with real-time data from technologies, such as the Internet of Things (IoT), artificial intelligence (AI), and machine learning (ML). Global organizations are leveraging what emerged as a “digital thread” to change how they design, manufacture, and service products. There are dozens of PLM software tools available to engineers. Examples include SAP PLM, Oracle Agile, and Aras PLM to name a few. Listing these tools here is not an endorsement of these tools.
  • Ergonomic Design and Simulation software allows production engineers to design workstations and plant features while focusing on the man-machine interface in order to ensure safety of the worker, while providing for comfort, ease of use, productivity and performance. Examples include Delmia Ergonomic Workstations, Semins Human-centered design and simulation, ERGOMIX, and others. Listing these tools here is not an endorsement of these tools.
  • Producibility Analysis software allows design engineers, along with other technical personnel, to design product that promotes the ease of fabrication and assembly thus reducing production time, while increasing reliability. See DFMA for more information. Examples include Solidworks DFMXpress, DFMPro, DFMA software by Boothroyd Dewhurst, and others.  Listing these tools here is not an endorsement of these tools.
  • Validation and Verification Support software supports the validation and verification of requirements by linking them to test cases, test results, and other verification artifacts, ensuring that the system meets its intended purpose and satisfies stakeholder needs. Examples include Visure, ANSYS SCADE Suite, Simulink, and others. Listing these tools here is not an endorsement of these tools.
  • Change Management software provides efficient change management features such as version control, change tracking, and impact analysis, helping teams manage changes to requirements and their corresponding models effectively. Examples include Visure, Siemens Teamcenter, Topcased, and others. Listing these tools here is not an endorsement of these tools. 

    • Note: There are many software vendors that offer these tools that are mostly used by government contractors.  You can find more information on these software tools through a web search.

DAU Continuous Learning Modules and other training:

Guidance and other Resources:

M&S and DE Guidance and other Resources 
Producibility Engineering 

Producibility can be defined as “the measure of the relative ease of manufacturing.” That is, you can manufacture a part out of inexpensive material, using unskilled workers, simple tools, and manufacture it in a very short time. 

The terms "producibility" and "manufacturability" are often used interchangeably. The DoD Producibility and Manufacturability Engineering Guide distinguishes between producibility and manufacturability as distinct but complimentary and sometimes overlapping concepts.

Producibility is a "design" consideration to facilitate the ease of manufacture, that is, designing a product in a way so it is relatively easy to manufacture. That includes any technique that helps to improve the designs efficient and ability to be produced (see producibility tools below). Development teams should consider producibility during system development and design following detailed design guidelines and producibility principles. 

Manufacturability is a "factory floor or manufacturing operations," consideration used to enhance the ease of manufacture by developing and implementing efficient manufacturing processes. This includes and best practices like the use of Lean/Six Sigma, Theory of Constraints, Process Failure Modes and Effects Analysis, continuous process improvement, and others.

Note: Producibility Analysis is a requirement of MIL-HDBK-896, para. 6.2.1, and is addressed in the DoD Systems Engineering Guidebook. See Producibility Best Practices for more information on Producibility Engineering. 

Listed below are some things to consider during the design process: 

Producibility Resources

Producibility Tools (most will require a search of the web as there are no DoD links for these and there are dozens of links for each of the tools listed).

  • Design for Manufacturing (DFM)
  • Design for Assembly (DFA)
  • Design for Manufacturing and Assembly (DFMA) 
  • Process Failure Modes Effects Analysis (PFMEA)
  • Design for Ergonomics (DFE): 
  • Design for Reliability (DFR): 
  • Design for Maintainability (DFM): 
  • Design for Sustainability (DFS): 
  • Design for Quality (DFQ): 
  • Design for Supply Chain: 
  • Design for Safety (DFS): 
  • Producibility Assessment Worksheet 

The graphic below illustrated the improvements from producibility initiatives.

Note: You can find more information on these producibility tools through a web search.

DAU Continuous Learning Modules and training:

Guidance and other Resources:

Key Characteristics 

A key characteristic is a feature whose variation has the greatest impact on the fit, performance (function), or service life of the finished product from the perspective of the customer. In other words, it is a product or process characteristic that if you deviate from the target value, there is a high loss function the further from the target value you get, and it will cost you (see graphic below). Most characteristics have a low loss function and thus do not need the same management attention as does a Key or Critical characteristic. Thus, when you deviate for the target value on a characteristic that has a low loss function, there is not a significant impact to fit, function or service life.  However, Key Characteristics are the vital few characteristics that must be identified and managed in order to avoid this high loss function. 

Note: A KC can be either a product, process, or service KC. 

The management of KCs includes:

  • Identifying product characteristics of the design which most influence fit, performance or reliability
    • This will require the use of one or more of the approaches listed below
  • Supporting the mapping of product characteristics to production processes
  • Enabling the balancing of product design requirements with manufacturing process capabilities 
  • Enabling the development of the required process controls for production.

Engineers have used a wide variety of tools or approaches for identifying KCs with the identification process beginning in the design stage. There are objective and subjective approaches that may be used to help identify and manage KCs to include:

  • Quality Function Deployment (QFD)
  • Taguchi experimentation
  • Capability Analysis
  • Design Failure Mode and Effects Analysis (DFMEA)
  • Process Failure Mode and Effects Analysis (PFMEA)
  • Statistical Process Control (SPC)
  • Design Verification
  • Control Plans
  • Statistical analysis of yield 
  • Process Flow Charts
  • Advanced Product Quality Planning (APQP)
  • Production Part Approval Process (PPAP)
  • Field Reporting and Corrective Action System (FRACAS)
  • Reliability data from similar products.

Watch the Ford Batavia Transmission Quality study at the url listed below for an excellent example of a company that was facing a serious manufacturing problem with building transmissions that were reliable, and Ford used some of the tools listed above to identify and manage key characteristics and get control of their manufacturing problems that helped them to achieve cost, schedule and reliability goals. What Ford found was that out of thousands of measurement characteristics, only four were significant to achieving their manufacturing goals. This profound knowledge then allowed them to control their processes. Note that the film is quite old but one of the best examples available. 

The Batavia Movie https://www.youtube.com/watch?v=uAfUOfSY-S0 

NOTE: Technical personnel should also control the quality of parts designated as Critical Safety Items (CSIs) or Critical Application Items. In addition to managing KCs technical personnel need to manage special characteristics which AIAG divides into two categories critical characteristics and significant characteristics. You can find more information on these tools through a web search.

  • Special Characteristics: Product characteristics or manufacturing process parameters that can affect safety or compliance with regulations, fit, function, or performance.
  • Significant Characteristics: Characteristics that are important to the customer or final client, but do not have a direct impact on the safety, performance or functionality of the product, but can still affect customer satisfaction.
  • Critical Characteristics: Special characteristics that are crucial for the safety, performance, or functionality of the product. These characteristics have a direct impact on quality, reliability, and safety. If critical characteristics are not identified and managed, they could result in serious consequences. 
  • Source: IATF 16949:2016 clause 8.3.3.3 Special Characteristics 
  • These characteristics can be identified and managed using the tools and approaches listed above.

Below is a notional chart depicting the activities that should be accomplished during each planning and execution phase of development and production.

Key Characteristics Guidance can be found in: 

Producibility Best Practices 

Producibility analysis. Producibility should be considered as a part of design trade studies. The role of design trade studies in the manufacturing development process is to achieve a product design that effectively balances the system design with cost, schedule and performance elements to minimize total program risk. Institutionalizing producibility as part of the design trade study process is essential to an overall goal of affordable weapon system acquisition. Another excellent source for information on producibility programs is the Navy’s NAVSO P-3687, “Producibility System Guidelines.” This guide recommends a 5-step process: 1. establish a producibility infrastructure, 2. determine process capabilities, 3. address producibility during conceptual design, 4. address producibility during detailed design, and 5. measure producibility.

Producibility Best Practice tools include: the following and may require a web search to gather more information:

  • Quality Function Deployment (QFD): A structured process and set of tools that can be used to tools used to effectively define customer requirements and convert them into detailed engineering specifications and plans to produce the products that fulfill those requirements. QFD is used to translate customer requirements (or VOC) into measurable design targets and drive them from the assembly level down through the sub-assembly, component and production process levels.
  • Concurrent Engineering (CE): I more of an approach to engineering than a specific tool. Concurrent engineering can be used to reduce product development time while reducing costs and improving quality and reliability by concurrently and systematically product design along with associated manufacturing, quality and other processes.
  • Integrated Product and Process Development (IPPD): Is a DoD management technique that simultaneously integrates all essential acquisition activities through the use of Integrated Product Teams (IPTs) to optimize design, manufacturing, and supportability processes. IPPD facilitates meeting cost and performance objectives from product concept through production, including field support. It evolved in industry as an outgrowth of efforts such as Concurrent Engineering to improve customer satisfaction and competitiveness in a global economy.
  • Integrated Product Teams (IPT): An Integrated Product Team (IPT): Is a team composed of representatives from appropriate functional disciplines working together to build successful programs, identify and resolve issues, and make sound and timely recommendations to facilitate decision-making. IPTs are used in complex development programs/projects for review and decision-making. The emphasis of the IPT is on the involvement of all Stakeholders (users, customers, management, developers, contractors) in a collaborative forum.
  • Taguchi/Robust Design/Parameter Design: Is s a powerful statistical method to produce high quality product and optimize the process design problems in a cost-efficient way by reducing process variation through robust design of experiments. An experimental design is used to identify and exploit the interactions between control and noise factors. Once the significant factors have been identified and their control settings established the resultant product will be optimized by designing quality into the product and processes.
  • Taguchi Loss Function: Is a graphical technique to show how an increase in variation from the target value, on key characteristics, can have an exponential impact on cost, reliability and customer dissatisfaction. Traditional quality looks at product quality as either good or bad, that is it either meets the spec or does not. While this may be true for many characteristics, it is not true for key characteristics. 

  • Modeling and Simulation M&S): Manufacturing simulation is the use of computer modeling to virtually test manufacturing methods and procedures – including processes such as production, assembly, inventory, and transportation. Simulation software can be used to predict the performance of a planned manufacturing system and to compare solutions for any problems discovered in the system's design. This makes manufacturing simulation a significantly competitive capability - allowing manufacturers to test a range of scenarios before buying tooling, reserving capacity, or coordinating other expensive production resources. By using simulation software to determine exactly what is needed, the manufacturer can avoid problems during production while also reducing scrap and rework. Various types of Factory Modeling and Simulation tools currently available include, but are not limited to the following areas:

    •    Producibility Analysis and Ergonomics
    •    Process Planning
    •    Production Planning and Scheduling
    •    Line Balancing and Bottleneck Analysis
    •    Capacity Planning
    •    Predictive Analytics and Optimization
    •    Facility Planning, Layout and Design
    •    Virtual Factory Mock-up

  • Model Based Engineering (MBE): Uses annotated digital three-dimensional (3D) models of a product and relevant production equipment and processes as the authoritative information source for all activities in that product’s lifecycle including relevant production equipment and processes. MBE is an integral part of the technical baseline that evolves throughout the acquisition life cycle.
  • Model Based Systems Engineering (MBSE): Is a systems engineering methodology that focuses on creating and exploiting domain models as the primary means of information exchange between engineers, rather than on document-based information exchange. MBSE is generally defined as a formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases. MBSE uses models as an integral part of the technical baseline, which includes the requirements, analysis, design, implementation, and verification of a capability, system, and/or product throughout the acquisition life cycle
  • Computer Aided Design (CAD): Is the process of digitally creating design simulations of products in 2D or 3D, complete with scale, precision, and physics properties, to optimize and perfect the design – often in a collaborative manner – before manufacturing. The use of digital data allows various engineering functions to share, review, simulate, and edit technical data and allow organizations to introduce new product quickly.
  • Computer Aided Manufacturing (CAM): Involves the use of digital data, software and computer-controlled factory machinery to create products with a high quality by automating and optimizing manufacturing processes. CAM is used to either create new or improve upon existing manufacturing setups to boost efficiency and reduce wastage. It does so by expediting the manufacturing process and tooling and reducing energy requirements. The final results have a high degree of consistency, quality, and accuracy.
  • Note: You can find more information on these producibility tools through a web search.
Resource / Manufacturing and Quality
Quality Assurance and Control
View Resource

Quality Assurance (QA) is a broad and organizational-wide system for managing and improving quality. Quality assurance includes the development and implementation of planned and systematic activities in a quality system to ensure that the quality requirements for a product or service are fulfilled. QA focuses on the entire quality system including suppliers and ultimate consumers of the product or service. It includes all activities designed to produce products and services of appropriate quality. QA begins before a product is made or before a project is even started.

Quality Control (QC) Is a subset of quality assurance and refers to the activities used during the production of a product that are designed to verify that the product meets the customer's requirement. QC focuses on the process of producing the product or service with the intent of eliminating problems that might result in defects. QC begins as the product is being produced.

This resource page will focus on the following topic areas: 

  • Quality Management Systems (QMS)
  • Product Quality Control
  • Process Capability and Control 
  • Supplier Quality Management 
  • Quality of Design / Quality Engineering 
  • Advanced Product Quality Planning
  • Measurement Systems Analysis (MSA)
  • Continuous Process Improvement (Leam/Six Sigma/TOC)
  • Quality Policy and Guidance
  • Quality Tools and Checklist
  • A Comprehensive List of Tools to Improve Manufacturing and Quality  

FAR 46.202 Types of Contract Quality Requirements: 

  • 46.202-1 Contracts for commercial items. The Government shall rely on contractors’ existing QA systems as a substitute for Government inspection and testing before tender for acceptance unless customary market practices for the commercial item includes in-process inspection.
  • 46.202-2 Government reliance on inspection by contractor. The Government shall rely on the contractor to accomplish all inspection and testing needed to ensure that supplies or services acquired conform to contract quality requirements before they are tendered to the Government. The Government shall not rely on contractor inspection if the Government has a need to test the supplies or services in advance of their tender for acceptance, or to pass judgment upon the adequacy of the contractor’s internal work processes.
  • 46.202-3 Standard inspection requirements. Requires the contractor to provide and maintain an inspection system that is acceptable to the Government;

     -  Give the Government the right to make inspections and tests while work is in process; and

     -  Require the contractor to keep complete, and make available to the Government, records of its inspection work.

  • 46.202-4 Higher-level contract quality requirements.

    (a) Requiring compliance with higher-level quality standards is appropriate in solicitations and contracts for complex or critical items or when the technical requirements of the contract require—

           (1) Control of such things as work operations, in-process controls, and inspection; or

           (2) Attention to such factors as organization, planning, work instructions, documentation control, and advanced metrology.

    (b) When the contracting officer finds it is in the Government’s interest to require that higher-level quality standards be maintained. The contracting officer shall indicate in the clause which higher-level quality standards will satisfy the Government’s requirement. Examples of higher-level quality standards are ISO 9001, 9002, or 9003; ANSI/ISO/ASQ Q9001-2000; ANSI/ASQC Q9001, Q9002, or Q9003; QS-9000; AS-9000; ANSI/ASQC E4; and ANSI/ASME NQA-1.

DFAR related Quality Clauses:

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

Note: DCMA has many quality related Continuous Learning Modules available in DAU's iCatalog

Quality Management System (QMS)

A Quality Management System (QMS) is about quality assurance. A QMS is a clearly defined set of processes and procedure that are formalized in documents that outline processes, procedures, and responsibilities for ensuring products or services consistently meet customer and regulatory requirements. Quality Assurance is about "how" a product is made and focuses on processes and building in quality and preventing defects before they happen. A Quality Management System is a requirement of ISO 9001 and AS9100.  ISO 9001 can be considered the baseline QMS while AS9100 is used for developing and advanced QMS. 

MIO-Q-9858 was the first standard that addressed an organizations QMS. This standard was 1st released in 1959 by the U.S. Department of Defense to cover U.S. defense contracts. This standard was later revised to become the NATO Allied Quality Assurance Publications (AQAP in 1969. Then in 1987 the Intenational Organization for Standards created the ISO 9001 series based on MIL-Q-9858 and NATO AQAP.

The ISO series of documents outlines the requirements for a quality management system:

  • ISO 900: Identifies the vocabulary and fundamentals for a QMS to include the seven quality management principles 
  • ISO 9001: Details the requirements organizations need to meet in order to meet the standard and become certified
  • ISO 9002: Provides for guidance for the implementation of ISO 9001
  • ISO 9004: Provides guidance for achieving sustained success through the application of a program of evaluation and continuous performance improvement 

ISO 9001 Quality Assurance Process Model:    

  • Requirements (Customers and Relevant Interested Parties)
  • Leadership
  • Planning
  • Support Processes
  • Operational Planning
  • Performance Evaluation 
  • Customer Satisfaction 

ISO 9001 QMS Principles:

  • Customer Focus   
  • Leadership
  • Engagement of People
  • Process Approach
  • Improvement
  • Evidence Based Decision Making
  • Relationship Management

The AS9100 series of documents outlines the requirements for a QMS within the aerospace, space, and defense industries. AS9100 constitutes and "advanced quality system" that goes well beyond ISO 9001 adding increased focus on:

  • A clear understanding of the Organizations Context
  • Risk-Based Management
  • Process Approach integrated with business processes
  • Emphasis on Change Management  
  • Introduction of Knowledge Management 
  • Explicit Performance Evaluation Requirements

AS9100 Process Model:

  • Organization and it's context
  • Customer Requirements
  • Interested Parties Needs and Expectations
  • Leadership
  • Planning
  • Support
  • Operations
  • Performance Evaluation 
  • Results
  • Customer Satisfaction 

AS9100 series of documents include:

  • AS 9100: Identifies the aerospace requirements for a QMS that demonstrates an organization's ability to provide products that meet statutory and regulatory requirements  
  • AS9101: Identifies the requirements for auditing aviation, space, and defense organizations against the 9100 family of standards 
  • AS9102: Identifies the requirements for First Article Inspection to ensure that a new product or part meet all requirements through production part verification 
  • AS9103: Identifies the requirement to plan for and manage Key Characteristics and any variation of those characteristics 
  • AS9110: Identifies the requirements for a QMS at aviation maintenance organizations and is based on ISO 9001, but includes additional requirements for aviation maintenance and airworthiness 
  • AS9120: Identifies the requirements for a QMS at stock distributors and is based on ISO 9001 but includes additional requirements. Stock distributors include organizations that resell, distribute, and warehouse parts for aerospace industries.
  • AS9131: Identifies requirements for the uniform identification, documentation, and management of nonconforming that requires formal decisions 
  • AS9134: Identifies the requirements for managing risk in the supply chain on both new and existing suppliers
  • AS9146: Identifies the requirements for the prevention of Foreign Object Damage for organizations involved in the design, development, delivery, and post-delivery provisions for maintenance, spares, and other materials
  • AS9015: Identifies the requirements for the delegation of product verification activities at an organization's suppliers

  • AS5553: Identifies the requirements for the management of electrical, electronic, and electromechanical parts to avoid the introduction of counterfeit parts into any aviation, space, and defense assemblies   

The QMS is used to:

  • Manage product and process quality 
  • Reduce the cost of poor quality (scrap, rework, repair, etc.)
  • Make better decisions based on statistical and other data
  • Engage in a process of continuous improvement  

ISO 9001 and AS9100 requirements include:

  • Leadership and Commitment
  • Risk Identification, Management and Opportunities 
  • Quality Planning 
  • Support Functions (Manpower, Facilities, etc.)
  • Operation and Production Control 
    • Design and Development
    • Release of Product
    • Control of Nonconforming Material 
    • Internal Audits and Management Review
  • Performance Evaluation (Monitoring and Measuring)
  • Quality Goals and Continuous Improvement

QMS documentation includes:

  • Policies
  • Procedures
  • Quality Manual
  • Training Materials
  • Work Instructions
  • Audit Forms
  • Process Maps
  • Control Plans

DAU Continuous Learning Modules (DCMA):

Guidance, Resources, and Tools: 

Product Quality Control

The American National Standards Institute (ANSI) and the American Society for Quality (ASQ) define quality as follows: "the totality of features and characteristics of a product or service that bears on its ability to satisfy given needs."

The American Society for Quality (ASQ) states that "quality can have two meanings: 1) the characteristics of a product or service that bear on its ability to satisfy stated or implied needs; 2) a product or service free of deficiencies." 

For many years American companies viewed quality as "conformance to requirements." This means that if a product is "in spec" it is deemed good, and if it is "out of spec" it is deemed defective. This remained true for many years until Japanese companies began taking away market share from many U.S. companies, and they did this by using techniques that they learned from American quality guru's (Deming, Juran, Crosby, Shewhart, et. al.). The Japanese view quality as "uniformity," or very little variation in the product. They achieved this by continuously improving product and processes leading to six sigma quality levels (see graphic below).

FAR 46.105 Contractor Responsibilities directs that the contractor is responsible under the contract for:

  • Controlling the quality of supplies and services 

  • Tendering to the Government supplies and services that conform to contract requirements

  • Ensuring the quality of their suppliers and vendors

  • Maintaining evidence that the supplies and services conform to contract requirements 

  • Providing to the Government that evidence

The Government has the right to perform contract quality assurance at such times and places as may be deemed necessary to determine that the supplies and services conform to contract requirements.

FAR Part 52.246 Contractor Inspection Requirements: The Contractor is responsible for performing or having performed all inspections and tests necessary to substantiate that the supplies or services furnished under this contract conform to contract requirements, including any applicable technical requirements for specified manufacturers’ parts. 

Product Quality requires an inspection and test system that is used to substantiate that the product meets the contract specifications. Dr. Juran would define product quality as "fitness for use, " while Phillip Crosby would say "conformance to requirements." The inspection system should include requirements for: 

  • Inspection System

  • Inspection and Test Status

  • Control of Nonconforming Material

  • Corrective Action

  • Document Controls

  • Control of Quality Records 

  • Process Controls

  • Inspection and Testing Criteria

  • Control of Measuring and Test Equipment 

Quality Control is a sub-set of quality assurance and is about the quality pf the product as determined by inspection and/or testing and is an aspect of quality management that consists of inspection, testing and quality measurements that verifies that the product deliverables conform to specification, is fit for purpose and meet stakeholder’s expectations. Quality control techniques are varied and driven by the nature of the product. Product inspections and tests that are done to check whether a product meets its specification is the most obvious form of QC. The inspection and test methods used depends on the technical nature of the product being developed. These methods could include product and process inspection, First Article Inspection/First Article Testing, Production Lot Testing, Production Part Approval, Qualification Testing, and Production Qualification Testing. Quality control begins at the supplier's, includes receiving inspection, in-process inspection, final inspection and testing, and customer satisfaction reporting.

First Article Inspection (FAI)/First Article Testing (FAR): FAI is a physical audit (see AS9100, AS9102 and DCMA Manual 2101-01). FAI conducted to ensure that the product meets contractual requirements and is a dimensional and qualitative inspection of a part or assembly to ensure the part or assembly fully conforms to technical drawings, specifications, customer envelope, and interface dimensions. A flow chart will be provided to validate the capability and stability of each process step. In addition, Key characteristics and critical characteristics shall be identified to ensure that they are validated during the FAI. First Article Inspections also verify production processes by examining work instructions, routing sheets, quality plans. FAIs also include reviews of in-process, acceptance testing procedures, and results. These tests may be witnessed by DCMA Quality Assurance Representatives (QAR) and/or government acquisition personnel.

Production Lot Testing: The purpose of production lot testing (PLT) is to validate quality conformance of products prior to lot acceptance. Product specialist will review the ESA testing requirements for completeness, accuracy, and applicability; coordinate any changes with the ESA; and enter the testing requirements in the material master. The test indicates that the manufacturer’s ability to create a consistent product within prescribed tolerances (quality). These tests may be witnessed by DCMA Quality Assurance Representatives (QAR) and/or government acquisition personnel.

Production Part Approval Process: Production Part Approval Process (PPAP) is used to accept an item from a manufacturing process. The purpose is to determine if engineering design records, functional, and specification requirements are understood and if the manufacturer's process has the capability to produce product consistently and continuously. The PPA provides the parts characteristics, part sample size, documentation, and requirements based on AF's needs for assessing the manufacturers' product. Varying degrees of requirements may be needed to demonstrate the manufacturing capability. These tests may be witnessed by DCMA Quality Assurance Representatives (QAR) and/or government acquisition personnel.

Qualification Testing: Qualification testing is a series of simulated operational, environmental, and endurance tests that prove the design "should" hold up and perform adequately in the field. There are no guarantees since the field can produce multiple environments and simultaneous stresses that are either impossible to produce in a test environment or too expensive to do in a test environment.

Production Qualification Testing: A technical test completed prior to the Full-Rate Production (FRP) decision to ensure the effectiveness of the manufacturing process, equipment, and procedures. This testing also provides data for the independent evaluation required for materiel release so the evaluator can address the materiel's adequacy with respect to the stated requirements. These tests are conducted on a number of random samples from the first production lot and are repeated if the process or design is changed significantly and when a second or alternative source is brought online.

Process Capability and Control 

Roles of Manufacturing:

  • Influence the design for producibility and manufacturability 
  • Prepare for production (planning)
  • Execute the production plan 
    • Reflect the design intent
    • Use repeatable processes
    • Focus on Continuous Process Improvement 

Goals of Manufacturing:

  • Deliver uniform, defect-free product 
  • Product provides consistent performance
  • Product is affordable 

One of the major goals of manufacturing is to provide the customer with “uniform, defect free product that has consistent performance and is affordable. M&Q personnel should support the assessment of manufacturing processes in order to determine if those processes are capability and in control. 

Process capability and control is a requirement of AS6500 Manufacturing Management Program standard, AS9100 quality standards, AS9003 Variation Management of Key Characteristics, and AS9138 Quality Management Systems Statistical Product Acceptance Requirements. These standards require a process control plan to describe activities that will demonstrate process capabilities. Process capability clarifies the inherent process variability of a given characteristic or process. A capability study is used to assess the ability of a process to meet a drawing/specification requirement. Typical measures include process capability (Cp/Cpk) and process performance (Pp/Ppk); X bar and R charts; control charts; and other statistical analysis tools. 

Process Capability and Control: Requires an analysis of the risks that the manufacturing processes are able to reflect the design intent (repeatability and affordability) of key characteristics.

Process Capability (Cp) is a statistical analysis or measurement of a process’s capacity or ability to produce product that meets specifications (within tolerance) and to manufacture parts repeatedly within technical specifications. Process capability is measured using capability indices (Cp, Cpk, Pp, and Ppk) which compare the processes variability (voice of the process) to the tolerance limits (voice of the customer). The data is used to calculate the standard deviation or sigma value, we can calculate Cp (Process Capability), Cpk (Process Capability Index), or Pp (Preliminary Process Capability), and Ppk (Preliminary Process Capability Index) to determine how our process is operating. Histograms are often used to analyze process capability to determine if the process is capable or not capable, and to the degree of capability often with a goal of 6 sigma for key and critical characteristics.  

Process control is a way to monitor and manage manufacturing processes using statistical process control to produce uniform, defect-free products. Statistical Process control and control charts were developed by Walter Shewhart at Bell Labs in the early 1920"s. Process control requires the management of process input variables, the identification and management of product deviations or variability from technical requirements, and the modification of process to ensure future production processes perform as expected. A process is in statistical control when all special causes of variation have been removed. Control charts are used to determine if a process is in control or not. 

An in-control process has many benefits:

  • Variability is known and thus scrap and rework can be estimated
  • Process settings can be identified and optimized during production
  • Tolerance design can be optimized to ensure drawings include appropriate tolerances 
  • Customers are better satisfied 

Process Capability and Control Guidance, Tools, and other Resources 

Supplier Quality Management 

Supply Chain: A supply chain includes the flow of material, information, and funds from raw material to the end customer and return if necessary. A supply chain includes all organizations and processes involved in the creation and delivery of products and services to the end customer. Creating faster flow results in:

  • Customer gets product faster (customer order cycle time)
  • Money moves faster (cash-to-cash cycle time)
  • Inventory is lower (cost are lower)

Supply chain quality management is the process of developing and executing a supplier quality program that ensures that products are delivered on-time, to the right place, in the right count and condition, at the agreed upon price, and in time to meet the customers’ requirements (production schedule).  A supply chain can be defined as a network of people, organizations, resources, activities and technology involved in the creation, production and delivery of a product. A supply chain includes the flow of material from a source to a destination. The Association for Operations Management (APICS) defines supply chain management (SCM) as the "design, planning, execution, control, and monitoring of supply chain activities with the objective of creating net value, building a competitive infrastructure, leveraging worldwide logistics, synchronizing supply with demand and measuring performance globally."

DoD contractors often invoke AS9120, Quality Management Systems - Requirements for Aviation, Space and Defense Distributors on their contracts. Detailed supplier requirements often are detailed under the Shewhart Cycle or Deming Cycle which includes Plan-Do-Check-Act (PDCA) processes. 

  • Plan 
    • Context of the Organization 
    • Leadership and Commitment
    • Planning 
    • Support 
  •  Do
    • Operational Planning
    • Product Requirements 
    • Production Control 
    • Control of Nonconforming Material
    • Release of Product 
  • Check 
    • Customer Satisfaction
    • Monitoring, Measuring, and Analysis
    • Internal Audits
  • Act
    • Nonconforming Material and Corrective Action 
    • Continuous Improvement

Supplier quality management begins early in product design and development and continues throughout the life cycle of the system or product. Supplier quality goes beyond lowest price to include identifying “best value” subcontractors and vendors that have a history of providing quality products and services, with low nonconformance rates and rapid response to problems. 

Quality Auditing 

Quality Auditing required by:

  • ISO 9001, Section 9.2 Internal Audit
  • ISO/OEC 17021
  • AS9100, Section 9.2 Audit
    • These documents require contractors and organizations to conduct internal audits
  • AS9104 Requirements for Aerospace Quality Management System Certification/Registrations Programs
  • AS9133 Qualification Procedure for Aerospace Standard Parts 
    • These documents require contractors to control the quality of their subcontractors/vendors 

Audit Activities:

  • Audit:  A planned, independent and documented assessment to determine whether agreed upon requirements are being met.
    • Compliance with written requirements
    • Effectiveness of written requirements in meeting basic management controls.
  • Compliance Audit:  Conformance to rules.
  • Management Audit:  Effectiveness of rules in achieving desired results.

Audit Requirements:

  • Identify program requirements
  • Identify audit criteria 
  • Plan, establish, implement and maintain an audit program 
  • Identify the frequency, methods, responsibilities, planning requirements and reporting
  • Identify consideration the results of previous audits
  • Define the audit criteria and scope for each audit
  • Identify the audit team and audit training 
  • Conduct fair and detailed audits to ensure objectivity and the impartiality 
  • Ensure that the results of the audits are reported to relevant management
  • Take correction and corrective actions 
  • Follow up to ensure closure of audit findings
  • Retain audit documentation 

Audit Types:

  • Process Audit: An audit conducted to verify that the processes are working within established limits and is producing conforming product. It evaluates an operation or method against predetermined instructions or standards to measure conformance to these standards and the effectiveness of the instructions. A process audit may:
    • Check conformance to defined requirements such as time, accuracy, temperature, pressure, composition, responsiveness, amperage, and component mixture.
    • Examine the resources (equipment, materials, people) applied to transform the inputs into outputs, the environment, the methods (procedures, instructions) followed, and the measures collected to determine process performance.
    • Check the adequacy and effectiveness of the process controls established by procedures, work instructions, flowcharts, and training and process specifications.
  • Product audit: An audit that examines a product to ensure it conforms to requirements (i.e., specifications, performance standards, and customer requirements).
  • System audit: An audit conducted on the quality management system. It can be described as a documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the system are appropriate and effective and have been developed, documented, and implemented in accordance and in conjunction with specified requirements.
    • A quality management system audit evaluates an existing quality management program to determine its conformance to company policies, contract commitments, and regulatory requirements.
    • Similarly, an environmental system audit examines an environmental management system, a food safety system audit examines a food safety management system, and safety system audits examine the safety management system
  • A first-party audit is performed within an organization to measure its strengths and weaknesses against its own procedures or methods and/or against external standards adopted by (voluntary) or imposed on (mandatory) the organization. A first-party audit is an internal audit conducted by auditors who are employed by the organization being audited but who have no vested interest in the audit results of the area being audited.
  • A second-party audit is an external audit performed on a supplier by a customer or by a contracted organization on behalf of a customer. A contract is in place, and the goods or services are being, or will be, delivered. Second-party audits are subject to the rules of contract law, as they are providing contractual direction from the customer to the supplier. Second-party audits tend to be more formal than first-party audits because audit results could influence the customer’s purchasing decisions.
  • A third-party audit is performed by an audit organization independent of the customer-supplier relationship and is free of any conflict of interest. Independence of the audit organization is a key component of a third-party audit. Third-party audits may result in certification, registration, recognition, an award, license approval, a citation, a fine, or a penalty issued by the third-party organization or an interested party.

Audit Report Components:

  • Executive Summary
  • Positive Practice Statements
  • Observations
  • Findings 

Quality Audit Objectives:

  • Ensure compliance to requirements
  • Improve performance, streamline processes and lower costs 
  • Meet and improve customer satisfaction 
  • Find and mitigate quality deviations 
  • Continuously improve 

DAU Continuous Learning Modules:

Quality of Design / Quality by Design / Quality Engineering 

Quality of design or quality by design is a deliberate and structured process for designing, developing and producing new products or improved products in a way to ensure customer satisfaction.  When an organization is designing a product the design team should be a multi-functional team utilizing a concurrent engineering approach that includes subject matter experts from design, materials, manufacturing, quality, reliability, sustainment, etc., in order to ensure that the product being developed achieves design, manufacturing, and sustainment efficiency while meeting customer requirements. 

Quality begins with the design:

  • Quality of design is the level of quality or features the producer is intending to deliver to the customer.
  • Requires the producer to understand the customer’s requirements, what is “Critical to Customer (CTC).”
  • Poor design contributes up to 40% of all quality problems, and between 60% and 80% of all costs are fixed at the design stage.
  • Lack of concentration on quality of design results in time consuming and costly reworking of products, processes, and design plans.

Quality By Design – Based on Juran Institute Processes

Quality by Design (Quality of Design or Concurrent Engineering) is the process of creating a design using multidisciplinary teams (IPTs) to conduct conceptual thinking, product design, and production planning all at one time. Quality by Design was developed by Dr. J. M. Juran who believed that quality should be built into a product from the beginning, and that most quality issues are related to the product's original design. 

Quality by Design differs from traditional sequential engineering practices and uses a five step process of Define, Discover, Design, Develop, and Deliver. Each process step has a focus and utilizes a variety of advanced quality and technical tools to meet customer requirements (satisfies the customer) often focusing on the eight dimensions of quality to create a design that is optimized for performance, cost and schedule. Quality by Design should require input from many different technical teams to include manufacturing and quality. It is the job of systems/design engineers to create the design, and it is the role of these technical personnel is to “influence the design” for producibility, manufacturability, reliability and maintainability, testability, and sustainability.

Define

Describe in general terms what the product is and what set of customers it is intended to serve. Establish the team, identify the customers and stakeholders, set goals, and create plans:

  • Operational Needs and Requirements
  • Sustainment Requirements
  • IOC Date
  • Weapon System Performance 
  • Cost (Procurement and Sustainment)
Discover

Discover the exact needs of the customer expressed in terms of the benefit to the customer. Collect and prioritize customer needs and translate those needs and create:

 

  • Work Breakdown Structure (WBS)
  • Functional Allocation 
  • Allocated Baseline
  • Measures of Effectiveness (MoE)
  • Key Performance Parameters (KPP) 
  • Measures of Performance (MoP)
  • Technical Performance Measures (TPM)
  • “Critical to Quality” measures. 

 

Develop planning worksheets for:

 

  • Customer requirements
  • Allocate Functions and Features
  • Allocate Measures and Goals (MoE, KPP, MoP, TPM and Critical to Quality
  • Detailed design features and goals 
  • Manufacturing Process features and goals 
  • Manufacturing Control features and goals

 
Design

Design a product that will meet those needs better than competitors’ and preceding products. Establish high-level product features and goals: (functional design), then develop detailed product features and goals (detailed design), optimize design features, set and approve final design.

 

  • Product Baseline
  • Functional
  • Allocated Design
  • Detail Design 
  • Design reviews and approvals
  • Identify and manage key and critical characteristics
  • Producibility Engineering
  • Trade Studies and Trade-off Analysis
  • Design Reviews
  • Above activities may require the use of the following advanced engineering techniques:
    • Design of Experiments 
    • Quality Function Deployment
    • Design for Manufacturing and Assembly (DFMA)
    • Design for X
    • Parameter Desing 

    • Tolerance Design 

       
Develop

Produce or manufacture the product by identifying, managing, controlling manufacturing processes and control.

 

  • Create manufacturing process planning (route sheets, flow diagrams, etc.)
  • Create assembly charts and operations process charts
  • Understand and control manufacturing processes:
    • Set goals for processes
    • Develop control plans
    • Establish audit procedures
    • Capture feedback and continuously improve
  • Manage capacity, identify bottlenecks:
    • Theory of Constraints
    • Optimize throughput
  • Optimize processes: 
    • Lean
    • Six Sigma
    • Standard Stable Processes
    • Identify and manage critical process parameters
    • Demonstrate process capability
    • Optimize process capability 
  • Manage Quality: 
    • Establish a Quality Management System
    • Identify critical quality attributes
    • Control variation
    • Statistical Process Controls
  • Manage Supply Chain 
Deliver

Plan for the transfer of the product to the customer, then measure and manage customer satisfaction.

 

  • Conforms to requirements
  • Performs as expected
  • Reliable product
  • On-time
  • Defect free

The Eight Dimensions of Quality 

Design Dimension Description Example 
Performance Will the product do the intended job?  What are it’ primary operating requirements or functions? GT automobile can go from 0-60 in under 4.5 seconds
Features Secondary performance characteristics, bells and whistles Automobile has heated and cooled seats
Reliability How it performs over time, mean time between failure Service at 5,000-mile intervals, expected life 200,000 miles. Reliability rating of 3.8 out of 5.0
Durability Products expected life Comes with a 60,000-mile warranty
Serviceability  How easy it is to maintain and repair, mean time to repair, total ownership cost 5-year cost for maintenance $4500, repairs $900
Conformance The degree to which a product conforms to the specifications Under 80 defect parts per million (DPM)
Aesthetics The appearance of the product, visual appeal, brand recognition  Looks fast while standing still
Reputation  Customer has a positive or negative experience J.D. Power's Award for Excellence

Quality by Design (Quality of Design) was first created by the quality guru Joseph M. Juran in his journal "Juran on Quality by Design." include prioritize a user-centered approach, that thoroughly understand user needs, implements a systematic design approach with iteration and feedback loops. 

Define
  • Identify the customers 
  • Identify product goals 
  • Establish the team
  • Construct high-level flow
  • Create plans 
Discover
  • Identify the needs of the customer 
  • Prioritize customer needs
Design
  • Identify product features (functional design)
Develop

Develop Process

  • Collect known baselines

Develop Process Controls 

  • Identify control subjects
Deliver
  • Plan for transfer to the customer

Quality by design may use some of the following tools:

  • Quality Function Deployment (QFD), this tool is probably the best tool for achieving Qualituy by Design
  • Design for Manufacturing and Assembly
  • Process Mapping 
  • Design Failure Modes and Effects Analysis (DFMEA)
  • Process Failure Modes and Effects Analysis (PFMEA)
  • Process Capability and Control

Note: You can find more information on these Quality of Design tools with a web search.

The role of manufacturing in this process is to "influence the design for producibility," which is the relative ease of fabrication and assembly. A better way to look at it is below:

Quality by Design is a component of the systems engineering process and an integral part of product and process verification and validation. It changes the way manufacturers approach all process design, process qualification, and process verification through the entire lifecycle of the product. 

Producibility is a design accomplishment resulting from a coordinated effort by design engineering and all the functional engineering specialties to create a functional hardware design that optimizes the ease and economy of fabrication, assembly, inspection, test, and acceptance of the hardware without sacrificing desired function, performance, or quality. Producibility is considered one of the most important determinants of product cost as producibility or lack thereof impacts both the product and the sustainment or life cycle cost.

Advanced Product Quality Planning (APQP) 

APQP is a structured approach to product and process design. This framework consists of a standardized ser of quality requirements (e.g. AS9145) that enable a supplier to design a product that satisfies the customer. The APQP approach consists of five steps or phases:

  • Plan and Define: Establishes the framework for the project and product and links customer requirements through product and resource planning focusing on the Voice of the Customer.
  • Product Design and Development: Part of the systems engineering process. Starts with the translation of requirements into the design and includes setting goals for design, reliability, manufacturing and quality. 
    • Identify Key Characteristics 
    • Identify Production Processes
    • Conduct Design Risk Analysis 
    • Conduct DFMEA
  • Process Design and Development: Part of the systems engineering process. Develops the manufacturing processes needed for production and identifies tools to aid in Producibility and Manufacturing Engineering:
    • Design Verification Plan and Report
    • Process Flow Charts
    • DFMEA/PFMEA
    • Product and Process Characteristics 
    • Pre-Launch Control Plans
  • Product and Process Validation: Demonstrates manufacturing and assembly processes to ensure that the given design and production processes can achieve customer requirements. 
    • Trial Runs
    • First Article Inspection 
    • Production Control Plans
    • Measurement System Analysis
  • Production: Conducts fabrication and assembly and fielding using process controls, production part approval, and uses feedback to continuously improve. 

There are five basic Core Tools detailed in separate guideline handbooks, including Advanced Product Quality Planning (APQP).  The other Core Tools are:

  • Failure Mode and Effects Analysis (FMEA)
  • Measurement Systems Analysis (MSA)
  • Statistical Process Control (SPC)
  • Production Part Approval Process (PPAP)

Resources and Guidance: 

Measurement Systems Analysis (MSA)

A measurement system has been described as a system of related measures that enables the quantification of particular characteristics. It can also include a collection of gages, fixtures, software and personnel required to validate a particular unit of measure or make an assessment of the feature or characteristic being measured. Variation often comes from a manufacturing process, but there are other sources of variation in a measurement process can include the following:

  • Process – test method, specification
  • Personnel – the operators, their skill level, training, etc.
  • Tools / Equipment – gages, fixtures, test equipment used and their associated calibration systems
  • Items to be measured – the part or material samples measured, the sampling plan, etc.
  • Environmental factors – temperature, humidity, etc.

Measurement System Analysis (MSA) is used to determine the suitability of a measurement system for use. It is crucial to have a well-functioning measurement system so that the data collected is accurate and precise. There are many factors to consider when conducting a measurement system analysis. This paper will discuss the importance of Measurement System Analysis and how to go about completing one.

MSA is a process used to evaluate the suitability of a measuring system for use. A measuring system can be any combination of a transducer, signal conditioner, display, recorder, or data acquisition system used to obtain a measurement. A measuring system is suitable if it meets the required technical performance specifications. MSA is used to identify and quantify the sources of variation in a measuring system.

Discussion on Metrology and Calibration:  

Government standards describe the requirement to provide a means for calibration and measurement traceability of all system, subsystem, and equipment. Standards define the requirement for establishing measurement traceability from actual system level measurements to the National Institute of Standards and Technology (NIST) or other approved measurement sources. 

Calibration defined: The set of operations that establish, under specified conditions, the relationship between values of quantities indicated by a measuring instrument or measuring system, or values represented by a material measure or a reference material, and the corresponding values realized by standards. 

Contractors shall establish and maintain a system for the calibration of all measurement and test equipment (M&TE) and measurement standards-used in fulfillment of contractual requirements. The calibration system shall be coordinated with the contractor’s inspection or quality control system and shall be designed to provide adequate accuracy in use of M&TE and measurement standards. All M&TE and measurement standards applicable to the contract, whether used in the contractor’s plant or at another source, shall be subject to such control as is necessary to assure conformance of supplies and services to contractual requirements. The calibration system shall provide for the prevention of inaccuracy by ready detection of deficiencies and timely positive action for their correction. The contractor shall make objective evidence of accuracy conformance readily available to the Government representative.

 Calibration requirements include:

  • Calibration system description 
  • Identification of acceptance criteria
  • Measurement traceability 
  • Environmental controls 
  • Storage and handling of measurement equipment
  • Intervals of calibration 
  • Calibration procedures documented 
  • Out-if-tolerance conditions and corrective action
  • Adequacy of the calibration system (accurate and precise)
  • Calibration sources/instruments
  • Calibration records 
  • Calibration status/labeling
  • Control of subcontractor calibration 

MIL-STD-1839D, DoD Standard Practice Calibration and Measurement System Requirements https://quicksearch.dla.mil/Transient/9050F2CC275C438888825B265D4FB4ED.pdf

MIL-HDBK-1839, Calibration and Measurement Requirements Handbook file:///C:/Users/ganoy/Downloads/MIL-HDBK-1839A.PDF

See Air Force Manual 21-113, Air Force Metrology and Calibration Program Management https://static.e-publishing.af.mil/production/1/af_a4/publication/afman21-113/dafman21-113.pdfDCMA CALIBRATION/CONTROL OF MEASURING & TEST EQUIPMENT CHECKLIST https://www.pdrep.csd.disa.mil/pdrep_files/documents/docs/QPR/04-QPR-Calibration-Control_MeasuringTestEquipment_July2019.docx

Discussion on Gage R&R Studies:

Gage Repeatability and Reproducibility (R&R) studies is a methodology for assessing the repeatability and reproducibility of the measurement system and the amount of variation in that system. These studies are a type of Measurement System Analysis. It is a study to understand how much of the process variation is caused by the measurement system itself as compared to the total variation. The study collects a series of measurements under consistent operating conditions over a set period of time with the intent of verifying that the output is the same as the input.

Measurement variation looks at two important factors. These are repeatability and reproducibility. 

  • Repeatability: Repeatability is the variation between successive measurements of the same part or trait by the same person using the same gage. In other words, how much variation do we see in measurements taken by the same person, on the same part, using the same tool?
  • Reproducibility: Reproducibility is the difference in the average of the measurements made by different people using the same instrument when measuring the identical characteristics on the same part. In other words, how much variation do we see in measurements taken by different people on the same part using the same tool?

There are many sources of measurement error:

  • Manpower: The ability of operators to achieve the same measurement. Sometimes different operators interpret measurement instructions differently or have different ability to measure and the result is a lack of reproducibility. 
  • Methods: The inspection/test methods and how the test fixtures are set up, used and the data recorded. Sometimes the measurement instructions are unclear or can be interpreted differently leading to measurement error.
  • Materials: The parts or specimens that are being measured may have inherent or natural variability that could lead to measurement error.
  • Measurement Instruments: Includes the instrument or gage that is being used to make measurements along with any fixtures, or other setup devices. Sometimes the equipment or instrument being used is not calibrated often enough, or there is wear and tear on the equipment, or the instrument itself lacks the precision needed for the measurements being taken, all can lead to measurement error.
  • Specification or Technical Requirement: Measurement values are compared to the specification or requirement. The tolerance of the specification is an important factor in assessing the measurement system.
  • Environmental Factors: Factory environmental settings can often vary during the day, or seasons. The effects of variation in temperature, humidity, cleanliness, lighting, etc., can lead to measurement error.
Continuous Process Improvement - Lean - Six Sigma - Theory of Constraints

CPI is an integrated system of improvement that focuses on doing the right things right. It is also an enterprise-wide "way of thinking" for achieving lower cost, shorter lead times, and higher quality. As a way of thinking, CPI is relevant to any process, regardless of complexity or relative importance. CPI provides an ongoing focus on enhancing the satisfaction of the Warfighter's needs. CPI can seek "incremental" improvement over time or "breakthrough" improvement all at once. Delivery (customer valued) processes are constantly evaluated and improved in the light of their efficiency, effectiveness and flexibility.

DoD CPI is a strategic approach for developing a culture of continuous improvement in the areas of reliability, process cycle times, costs in terms of less total resource consumption, quality, and productivity. In DoD, CPI comprises the application of a broad range of tools and methods, such as Lean, Six Sigma, and Theory of Constraints (TOC).

The role of the program manager (PM) is to direct the development, production, and initial deployment of a new defense system. This must be done within limits of cost, schedule, and performance, and as approved by the program manager's acquisition executive. The CPI tools outlined in this chapter can be used to support the achievement of these capabilities. A program manager should be able to:

  • Define quality and identify the various forms and structures associated with quality
  • Describe a few of the more significant quality initiatives
  • Identify several continuous process improvement tools
  • Describe the connection between quality and reliability/maintainability (R&M)
  • Describe how quality can be addressed in contract language

Major techniques used in a CPI program include:

  • Lean:  Focuses on eliminating waste or muda
  • Six Sigma: Focuses on variation and the reduction of variation on Key and Critical Characteristics
  • Theory of Constraints: Focuses on bottlenecks and the flow of material 
  • Note: Each of these techniques requires many pages of description, and there is a lot of material available on the web for each of them.

DAU Continuous Learning Modules:

Resources and other Guidance:

Lean, Six Sigma, and Theory of Constraints all have a different approach to quality. Each have a different focus, a different set of assumptions, and a different primary effect. These tools, while different can work together to improve overall product quality. See the graphic below that illustrates these differences. 

DAU Continuous Learning Modules and training:

Lean
The term “Lean” was coined by Womack, Jones and Roos in “The Machine that Changed the World” to describe the Toyota Production System (TPS) as developed by Shigeo Shingo. Toyota, using lean production, is able to reduce cost by eliminating waste, which reduces flow time and increase efficiency. They found that “Lean production” is different from craft and mass production and has the following characteristics:
  • Manpower is skilled and flexible, and are considered valuable and key resources
  • Machines, tools and methods are all flexible 
  • Suppliers are considered valuable and key and are treated as such by establishing good relationships
  • Quality is based on reducing waste and continuously improving 
  • Metrics are satisfied customers and high profits 
  • Inventories are kept low, often using a Just-in-Time delivery system
  • Flexible production, can produce a wide variety of products at either high or low volumes 
The Toyota Production System (TPS) focuses on five basic elements:
  • The use of standard/stable processes (work instructions) and a trained, multi-skilled workforce
  • The use of continuous improvement (Kaizen) focusing on perfection
  • The use of Just-in-Time (JIT) inventories to create continuous flow using a pull system based on a balanced workload 
  • Jidoka, also known as autonomation, which stops machines automatically when a defect is detected allowing for quicker responses to quality problems
  • A focus on customer satisfaction 
 
Lean Principles include:
  • Define Value: Form the customers perspective and express value in terms of specific product of service.
  • Map all of the steps:
    -  Value Added Steps
    -  Non-Value-Added Steps
  • Create Flow:  Processes must flow continuously from end-to-end.  Remove activities that impede flow.
  • Establish a Pull: Nothing moves until the customer (downstream) creates an actual demand that then drives production.
  • Continuously Improve: Until there is the complete elimination of all waste, and all remaining activities create value.
 
Lean does not call themselves Lean, they use the Toyota Production System (TPS) to achieve higher quality, lower cost, shorter cycle and throughput times, and higher customer satisfaction. Toyota has identified the seven forms of waste that cost time and money as:
  • Excess Transportation 
  • Excess Inventory
  • Excess Motion 
  • Excess Waiting 
  • Over-Production 
  • Over-Processing
  • Defects
 
DAU Continuous Learning Modules:
Guidance and other Resources:
Six Sigma

While Lean focuses on eliminating waste to improve, six sigma focuses on reducing waste by focusing on the reduction of variation. Six Sigma uses the DMAIC process for achieving quality goals. The DMAIC process is the disciplined methodology of defining, measuring, analyzing, improving and controlling the quality in every one of the Company’s products, processes and transactions-with the ultimate goal of virtually eliminating all defects. Often the goal of a quality program is to achieve six sigma outcomes where there are only 3.4 defects in a million opportunities on key and critical characteristics.  

Note: As you move from 3 sigma to 6 sigma your number of waste drops from 66,800 defects per million opportunities (DPMO) to only 3.4 DPMO. It cost a lot of money inspecting product to find those 66,800 defective units, and even with 100% inspection you will not find them all. With only 3.4 DPMO you should not inspect and that will reduce your cost of quality (COQ).

The main tool for achieving six sigma is DMAIC. 

  • Define what is important to the customer 
  • Measure how well you are currently doing 
  • Analyze what is wrong and where variation occurs
  • Improve by addressing root causes of problems 
  • Control to hold the gains made 

DMAIC utilizes many tools to help them achieve quality goals to include the following:

Note: You can find more information on Six Sigma and the DMAIC process with a web search.

DAU Continuous Learning Modules:
Theory of Constraints (TOC)

Theory of Constraints (TOC) is a systems thinking process that helps organizations improve by identifying and eliminating bottlenecks in their processes. All organizations and systems have constraints or bottlenecks that causes the organization to lose productivity and efficiency. The constraint or bottleneck controls and limits the capacity of a plant TOC uses a five-step problem solving methodology to identify and correct the bottleneck.

TOC is called a Thinking process asks three questions that are essential to improvement:

  • What needs to be changed
  • What should it be changed to
  • What action will cause the change
 
TOC Metrics identify three measures that are used to measure performance and guide management decisions:
  • Throughput
  • Investment 
  • Operating Expenses 
 
TOC operates on the following principles utilizing the Drum-Buffer-Rope process:
  • Bottlenecks govern both throughput and inventories
  • An hour lost at a bottleneck is an hour lost for the entire system
  • An hour saved at a non-bottleneck is an illusion of efficiency
  • It is ok to let a non-bottleneck to sit idle
  • It is never ok to let a bottleneck sit idle 
  • Balance flow not capacity 
TOC follows a five-step process called focusing steps:
  1. Identify the bottleneck
  2. Exploit the bottleneck
  3. Subordinate everything else to the bottleneck 
  4. Elevat the bottleneck 
  5. Repeat the above steps with the next bottleneck
Note: You can find more information on TOC with a search of the web. 
Quality Resources and Guidance

Note: All AS documents can be obtained at https://www.sae.org/standards/content/as9100/ 

Quality Tools and Checklist

The Tools and Checklist below relate to ISO 9001 and AS9100:

The Tools and Checklist below relate to the Seven Basic Quality Tools. Nine tools are presented as there is some disagreement on which are the seven. The first seven have been identified by ASQ as the 7 Quality Tools and these tools were developed at Toyota by Kaoru Ishikawa: All of these tools with definitions and templates can be found at https://asq.org/quality-resources/seven-basic-quality-tools 

  • Cause and Effect Diagram: Is a tool for identifying the potential cause of a related effect. Sometimes referred to as the Ishikawa or Fishbone diagram.
  • Check Sheet: Provides a count or tally of quantitative or qualitative data allowing you to analyze the data in order to identify defects in a process or product. 
  • Control Chart: Are used to determine whether a process will produce a product or service that has consistent and measurable properties. 
  • Histogram: A type of bar chart that depicts the frequency of data. This chart provides the easiest way to evaluate the distribution of data. 
  • Pareto Chart: A type of a histogram chart that is used to identify problems and then prioritize the problems to be solved. Prioritization is based on the 80/20 rule.
  • Scatter Diagram: Are used to study and identify the relationship between two different sets of variables.  
  • Stratification: Is a pictorial diagram that is used to sort data, objects, and people into distinct groups in order to determine patterns.
  • Flow Chart: Is a pictorial representation that captures the sequence of steps in a process, or system in a way that highlights the connections or relationships in that process.
  • Run Chart: Is a graph of data that is plotted over time. This chart allows you to identify patterns or trends in a process.

The Tools and Checklist below relate to the Seven Quality Management Tools: All of these tools with definitions and templates can be found at  https://asq.org/quality-resources/new-management-planning-tools 

  • Affinity Diagram:  Is an analytical tool that organizes many ideas into subgroups based on their "natural" relationships. This diagram is often developed using brainstorming techniques and is sometimes referred to as the KJ Method.
  • Interrelationship Diagram or Diagraph: Is a visual tool that identifies and depicts cause-and-effect relationships between factors and helps analyze the natural links between different aspects of a complex situation. 
  • Tree Diagram: Depicts the hierarchy of tasks and subtasks by breaking down tasks from broad categories into finer and finer levels of detail, helping to move step-by-step thinking from generalities to specifics.
  • Matrix Diagram: Provides a method for analyzing and displaying relationships between different data sets (two, three or more) and depicts the relationship between groups of information and can provide information about the relationship, such as its strength, the roles played by various individuals, or measurements.
  • Matrix Data Analysis: Is used to analyze and display relationships between data sets by classifying items by two major characteristics that are common to all items and then plotting each item on an x-y chart. 
  • Arrow Diagram: Is used to depict the order or sequence of tasks in a project or process and the interconnectivity of those tasks, the best schedule for the entire project, and potential scheduling and resource problems and their solutions.
  • Process Decision Program Chart: Provides a systematic means for identifying errors in a plan, and to identify what might go wrong in a plan under development.

Note: You can find more information on these tools with a web search. Also note, that there are literally hundreds of quality tools thar are available on the web to assist managers in managing projects and processes. 

A Comprehensive List of Tools to Assist Quality Managers 

This document identifies tools that could be used to help manage DoD acquisition technical, business, and management processes to include many manufacturing and quality activities. Most of these tools support systems engineering technical and technical management processes, but a few can be used to support business processes such as cost estimating, contract language, acquisition strategies, etc. Some tools could be used by a contractor, some by government personnel and some tools can bd used by many different people in many functional specialties. Most tools are available from multiple on-line sources, some tools may need to be purchased to use, many are free. 

Note: You can find more information on the tools listed below with a search of the web. 

Tools (listed in alphabetical order include):

  • 3Ps - Production Preparation Process
  • 5S’s (Sort, Straighten, Shine, Standardize, Sustain
  • 5-Whys
  • 7 - Basic Tools for Quality Improvement (includes the following which are also discussed separately):
    • Cause and Effect Diagram
    • Check Sheet
    • Control Chart
    • Histogram
    • Pareto Chart
    • Run Chart
    • Scatter Diagram
    • Stratification or Flowchart or 
  • 7 - Management and Planning Tools, or Advanced Tools for Quality Improvement (includes the following which are also discussed separately):
    • Affinity Diagram
    • Relations Diagram or Interrelationship Diagraph
    • Tree Diagram
    • Matrix Diagram
    • Matrix Data Analysis
    • Arrow Diagram
    • Process Decision Program Chart (PDPC)
  • 8D/PSP (Eight Disciplines/Problem Solving Process)
  • A3 Problem Solving Chart
  • Acceptable Quality Levels (AQL)
  • Acceptance Sampling 
  • Active Risk Manager (ARM)
  • Affinity Diagram
  • Advanced Product Quality Planning (APQP) Core Tools
  • Arrow Diagram (Chart)
  • AS6500 Manufacturing Management System (MMS)
  • AS9100 Advanced Quality Management Systems (QMS)
  • AS9103 Variation Management of Key Characteristics
  • AS9110 Maintenance
  • AS9120 Distributors
  • AS9102 FAI
  • AS9115 Software QA
  • AS9131 N/C Document
  • AS9132 Marking
  • AS9133 Supplier QA
  • AS 9137 AQAP Align
  • AS 9138 Statistical Process Control
  • AS9162 Self Verification
  • Axiomatic Design 
  • Balanced Scorecard
  • Baldrige Performance Excellence Criteria
  • Bekidou Rate
  • Benchmarking
  • Bill of Materials (BOM) 
  • Bone Diagram
  • Bottleneck Analysis
  • Box and Whisker Plot
  • Bubble Chart
  • Capacity Matrix
  • Capacity Analysis
  • Cause and Effect Diagram (Fishbone or IshIkawa)
  • Cause and Effect Matrix
  • CFMEA – Concept Failure, Mode and Effect Analysis
  • Check Sheet
  • Chokko Rate
  • Computer-Aided Design (CAD)
  • Computer-Aided Manufacturing (CAM)
  • Computer Aided Process Planning (CAPP)
  • Computer-Aided Three-Dimensional Interactive Application (CATIA)
  • Computer Integrated Manufacturing (CIM)
  • Consensogram
  • Contingency Planning
  • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart -U-chart for Attribute Data
    • Control Chart - NP-chart for Attribute Data
    • Control Chart - P-chart for Attribute Data
    • Control Chart - X-barR chart for Variable Data (measurable)
    • Control Chart - X-bar-S chart for Variable Data
    • Control Chart - X-MR/I-MR chart for Variable Data
  • Correlation Chart (Scatter Diagram)
  • Cost/Benefit Analysis
  • Cost of Quality Analysis
  • Cost of Quality (COQ)
  • Cost Modeling (Estimating)
  • Criteria Testing
  • Critical Chain Project Management 
  • Critical Design Review Checklist (DoD)
  • Critical Path/PERT
  • Critical to the Customer (CTC)
  • Critical to Quality (CTQ) Tree
  • Customer Contingency Table
  • DCOV – Define, Characterize, Optimize and Verify
  • Deming Cycle or Wheel (PDCA)
  • Departmental Purpose Analysis
  • Design of Experiments (DoE)
  • DFMEA - Design Failure Mode and Effects Analysis 
  • DFMA - Design for Manufacturing and Assembly 
  • Design to Cost (DTC)
  • DFSS - Design for Six Sigma 
  • DMAIC - Define, Measure, Analyze, Improve and Control
  • DMADV (see DCOV)
  • Domainal Mapping
    • Factory Modeling and Simulation
    • Producibility Analysis & Ergonomics
    • Process Planning 
    • Production Planning & Scheduling
    • Line Balancing & Bottleneck Analysis
    • Capacity Planning
    • Predictive Analytics & Optimization
    • Facility Planning, Layout and Design
    • Virtual Factory Mock-up
  • Failure Mode and Effects Analysis (FMEA)
  • Fault Tree Analysis 
  • First Article Inspection
  • First Article Testing 
  • Flow Chart or Process Flow Chart
  • Force Field Analysis
  • Gage R&R Studies
  • Gantt Chart
  • Histogram (Frequency or Bar Chart)
  • Hoshin Kanri (Quality Policy Deployment) 
  • Interrelationship Diagraph (also see Relations Diagram, or Network Diagram)
  • ISO 9001 Quality Management Systems (QMS) 
  • Kano Model
  • KJ model - Kawalota Jiro (see affinity diagram)
  • Lead Time Analysis
  • Learning Curve 
  • Learning Curve Analysis
  • Line of Balance (LOB)
  • Taguchi Loss Function
  • Lotus Diagram
  • Manufacturing Cost Estimating
  • Manufacturing Plan 
  • Manufacturing Readiness Assessment (MRA)
  • Manufacturing Readiness Level (MRL) Criteria
  • Matrix Diagram
  • Matrix Data Analysis Diagram
  • Measurement Systems Analysis (MSA) 
  • MIL-HDBK-896A Manufacturing and Quality Program
  • Multi-Vari Charts
  • Nominal Group Technique
  • One Piece Flow
  • Operations Process Chart
  • Overall Equipment Effectiveness (OEE)
  • Pareto Charts (Template)
  • Part-Family Analysis
  • Paynter Chart
  • P-Diagram or Parameter Design 
  • PERT Chart (Program Evaluation Routine Technique)
  • PFMEA – Process Failure, Mode and Effect Analysis
  • Pie Chart
  • Preliminary Design Review Checklist (DoD) 
  • Preliminary Hazards List (PHL)
  • Process Capability Studies (Cp and Cpk)
  • Process Performance Studies (Pp and Ppk)
  • Process Decision Program Chart (PDPC)
  • Producibility Analysis/Assessments
  • Producibility Assessment Worksheet
  • Producibility Engineering and Planning (PEP) Program
  • Production Part Approval Process
  • Production Readiness Review (PRR) Checklist
  • Programmatic Evaluation of ESOH (PESHE)
  • Pugh Matrix
  • Quadrant Chart
  • Quality Function Deployment (QFD)
  • Queuing Theory/Waiting Line Analysis
  • Radar Chart
  • Rational DOORS
  • Relation Diagram
  • Reliability Growth Analysis
  • Requirements Verification (Traceability) Matrix (RVM)
  • Risk Management Assessment Tool 
  • Route Sheet
  • Run Chart
  • Scatter Diagram (Mind Mapping)
  • SIPOC – System, Input, Process, Output and Customer
  • Six Sigma
  • SMART – Specific, Measurable, Attainable, Resources, Time
  • Spaghetti Diagram
  • Spider Diagram
  • Stratification
  • Statistical Process Control (SPC)
  • Supply Chain Management Risk Assessment
  • Swim Lane Chart (sometimes called a Deployment Flow Chart)
  • SWOT Model (Strength, Weaknesses, Opportunities and Threats
  • Systems Engineering Plan (SEP) 
  • Takt Time Analysis
  • Technical Risk Identification and Mitigation System (TRIMS)
  • Technology Readiness Level (TRL) Checklist
  • Theory of Constraints Analysis
  • Throughput Analysis Tool
  • Throughput Accounting
  • Tolerance Analysis 
  • Tolerance Design 
  • Total Productive Maintenance (TPM) 
  • Trade Studies/Analysis
  • Transition to Production (Willoughby Templates)
  • Tree Diagram
  • TRIZ Matrix 
  • Value Stream Mapping (VSM) 
  • Variability Reduction Program
  • Venn Diagram
  • Work Center
  • Work Measurement
  • X-Matrix
  • Yamazumi Chart

Note: You can find a lot of additional information by googling the tool, by visiting a number of different academic sites, or professional organizations, or by visiting various Communities of Practice (CoPs).

Resource / Manufacturing and Quality
Workforce Development, Training and Education
View Resource

DoD workforce development, training, and education is concerned about DoD organizations having the right personnel that possess the necessary education, skills, and abilities to support acquisition and sustainment programs across a broad spectrum of industries and sectors, and technical, business, and academic competencies.

Department of Defense (DoD) Systems Engineering and Architecture (SE&A) partners with the Military Services and defense agencies to identify workforce challenges and champion initiatives to ensure DoD maintains its advantage in Warfighter readiness in a rapidly evolving technological environment. Cross-cutting workforce initiatives focus on building technical capability and capacity to support current and future leadership priorities in four main categories: Forecast Future Talent Needs, Strengthen Talent Pipeline, Advance Our Workforce Skills, and Close Capability Gaps.

Image removed.

The Defense Acquisition University (DAU) is one of the primary providers of acquisition training and you can access their services from their Homepage for access to your training, to apply for training, and access to web events:  https://www.dau.edu/?tour 

Apply for a Course https://www.dau.edu/training/apply-for-a-course 

This resource page will focus on the following Pinned Content:

  • Access to Specific DAU sites:
  • Access to Specific DAU courses:
  • Other Training and Education Opportunities 
  • Recommended Reading List 

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

DoD Workforce Development Resources 

DOD Workforce Development Resources can be found at https://www.cto.mil/sea/workforce/ 

  • DoD Advanced Technical Degree Guidebook 
  • DoD Engineers Make a Difference
  • DoD Civilian Careers: Engineering 
  • DoDI 1400.25, Vol. 250 DoD Civilian Strategic Human Capital Planning 
  • DoDI 5000.66. Defense Acquisition Workforce Education, Training, Experience, and Career Development Program 
  • Digital Engineering Workforce Plan
  • Acquisition Workforce Credentials
Access to specific DAU sites:
Access to specific DAU courses:

Engineering and Technical Management personnel can take any of the following courses:

Foundational: https://www.dau.edu/functional-areas/engineering-and-technical-management?field_level_id_value=2 

ACQ 1010 Fundamentals of Systems Acquisition Management https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12339 

ENG 101 Systems Engineering Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=2005 

ETM 1010 Leading Change Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12576 

ETM 1020 Mission and Systems Thinking Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12548 

ETM 1030 Requirements Definition and Analysis https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12577 

ETM 1040 Technical Management Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12573 

ETM 1050 Design Considerations Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12635 

ETM 1060 Product Realization Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12563 

ETM 1070 Digital Literacy Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12633 

ETM 1080 Software Literacy Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12631 

ETM 1090 Technical Perspectives on Defense Contracting Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12634 

Practitioner: https://www.dau.edu/functional-areas/engineering-and-technical-management?field_level_id_value=2 

ETM 2010V Leading Change for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12570 

ETM 2020V Mission and Systems Thinking for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12568 

ETM 2030V Requirements Definition and Analysis for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12569 

ETM 2040V Technical Management for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12571 

ETM 2050V Desing Considerations for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12553 

ETM 2060V Product Realization for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12572 

ETM 2070V Digital Literacy for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12564 

ETM 2080M Software Literacy for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12584 

ETM 2090V Technical Perspectives on Defense Contracting for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12560 

Continuous Learning Modules at the Continuous Learning Center: https://www.dau.edu/continuous-learning-center 

CLE 001 Value Engineering https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=251 

CLE 003 Technical Reviews https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=274 

CLE 004 Introduction to Lean Enterprise Concepts https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=262 

CLE 007 Lean Six Sigma for Manufacturing https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=256 

CLE 008 Six Signa Concepts and Processes https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=309 

CLE 015 Continuous Process Improvement Familiarization https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=263 

CLE 017 Technical Planning https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=322 

CLE 019 Modular Open Systems Approach https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12258 

CLE 021 Technology Readiness Assessments https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=323 

CLE 026 Trade Studies https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=438 

CLE 028 Market Research for Engineering and Technical Personnel https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=67 

CLE 035 Introduction to Probability and Statistics https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=334 

CLE 036 Engineering Change Proposals for Engineers https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=470 

CLE 064 Standardization in the Acquisition Life Cycle https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1734 

CLE 065 Standardization Documents https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1785 

CLE 066 Systems Engineering for Systems of Systems https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1790 

CLE 068 Intellectual Property and Data Rights https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1911 

CLE 069 Technology Transfer https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2002 

CLE 070 Corrosion and Polymeric Coatings https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1956 

CLE 074 Cybersecurity Throughout DoD Acquisition https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2048 

CLE 075 Introduction to DoD Cloud Computing https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2117 

CLE 076 Introduction to Agile Software Acquisition https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2147 

CLE 077 Defense Business Systems (DBS) Acquisition https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2152 

CLE 078 Software Acquisition for the Program Office Workforce https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12249 

CLE 079 Chemical, Biological, Radiological, and Nuclear (CBRN) Survivability https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12174 

CLE 080 SCRM for Information and Communications Technology https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12181 

CLE 084 Models, Simulations, and Digital Engineering https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12176 

CLE 085 Scientific Test and Analysis Techniques om T&E https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12241 

CLE 301 Reliability and Maintainability https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=275 

CLE 201 ISO 9000 https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=336 

CLL 032 Preventing Counterfeit Electronic Parts from Entering the DoD Supply System https://icatalog.dau.edu/mobile/CLModuleDetails.aspx?id=1729 

CLL 062 Counterfeit Prevention Awareness https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=1931 

 CMQ 100 Quality Assurance Basics https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=2021 

CMQ 230 Quality Control Graphics and Charting https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=2047 

CLC 042 Predictive Analysis and Quality Assurance https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=338 

CMQ 101 Government Contract Quality Assurance (GCQA) https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2003 

CMQ 211 Quality Management System (QMS) Auditor https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12307 

ENG 0720 ISO 9000 https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12654 

CMQ 200 Statistical Sampling https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2024 

CMQ 1310 Data Collection and Analysis https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=13344 

CMQ 231 Data Collection and Analysis Application https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2054 

ETM 1060 Product Realization Fundamentals https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12563 

ETM 2060 Product Realization for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12572 

CME 103 Manufacturing and Delivery Surveillance https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12606 

CME 130 Surveillance Implications of Manufacturing and Subcontractor Management https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2077 

CME 230 Production Planning and Control (PP&C) https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2076 

CLE 004 Introduction to Lean Enterprise Concepts https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=262 

CLE 007 Lean Six Sigma for Manufacturing https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=256 

CLE 008 Six Sigma: Concepts and Processes https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=309 

ETM 2090V Technical Perspective on Defense Contracting for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12560 

CLE 001 Value Engineering https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=251 

LOG 0390 Additive Manufacturing Overview https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12618 

LOG 0400 Additive Manufacturing Case Studies https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12885 

LOG 0640 DMSMS, What the PM Needs to Do and Why https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12513 

LOG 0650 DMSMS Fundamentals https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12509 

LOG 0660 DMSMS Executive Overview https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12498 

LOG 0670 DMSMS Research Essentials https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12508 

Other Training and Education Opportunities:
  • Rapid eLearning Courses
  • LIFT IGNITE Program offers a 2-year program focusing on materials science and advanced manufacturing. You can find more information at https://lift.technology/wp-content/uploads/2020/08/Ignite-Brochure-9.25.18.pdf 
  • MIT EdX offers over 350 courses on manufacturing and supply chain. You can find out more at  https://www.edx.org/search?q=manufacturing+ 
  • NetFlex Flex Factor is associated with Manufacturing USA is an outreach, recruitment, and STEM education program designed to familiarize K-12 students with advanced manufacturing technology, entrepreneurship, and the education and career pathways that can lead to a STEM career. You can find out more at https://www.nextflex.us/ewd/flexfactor/ 
  • MxD (Manufacturing x Digital) advances economic prosperity and national security by strengthening U.S. manufacturing competitiveness through technology innovation, workforce development, and cybersecurity preparedness. MxD Learn is testing a three-year digital manufacturing curriculum and developing a separate four-week awareness program for high school students. https://www.mxdusa.org/focus-areas/workforce-development/ 
  • America Makes ACADEMI is a comprehensive set of immersive training experiences with advanced AM curriculum, delivered in an intense, hands-on environment, integrating skills from multiple disciplines employed in Design for Additive Manufacturing (DfAM) processes. https://www.americamakes.us/academi/ 
  • Udemy Manufacturing Operations, Planning, Management and Control https://www.udemy.com/topic/manufacturing/ 
    • 14 sections (Operations Management, Systems Desing and Capacity, Facility Layout, Forecasting Demand, Developing and Designing Product, Material Management, etc.) over 100 lectures 
  • Note: There are literally thousands of technical/trade schools, undergraduate and graduate programs that cover many M&Q skills, and higher level education in Industrial Engineering, Manufacturing Engineering, Quality Engineering, and Quality Management for M&Q personnel. 
Recommended Reading List

A Study of the Toyota Production System, Shingo, 1989

Becoming Lean: Inside Stories of US Manufacturers, by Liker, 1997

Conquering Complexity in Your Business, George, 2004

Creating a Level Pull, Smalley, 2004

Creating Continuous Flow, Rother and Harris

Creating Quality: Process Design for Results, Kolarik McGraw-Hill, 1999

Critical Chain, Eliyahu Goldratt, 1997

Factory Physics, Hopp and Spearman, 2001

Guide to Quality Control, Ishikawa, 1990

How Digital is Your Business?, Slywotzky, Adrian J. Crown Business, 2000.

Implementing Six Sigma, Breyfogle, 1999

It's Not Luck, Eliyahu Goldratt

Japanese Manufacturing Techniques, Richard Schonberger, 1982

Juran's Quality Handbook, 5th Edition, Juran, 1998

Lean Assembly, Baudin, 2004

Lean Logistics, Baudin, 2004

Lean Manufacturing: A Plant Flow Guide, Allen, Robinson and Steward, 2001

Lean Production Simplified, Dennis, 2002

Lean Six Sigma, George 2002

Lean Six Sigma for Service, George, 2003

Lean Thinking: Banish Waste and Create Wealth in Your Corporation, Womack, James P. Simon and Schuster, 2nd Edition, 2003.

Learning to See, Rother and Shook,

Let's Fix It: Overcoming the Crisis in Manufacturing, Richard Schonberger, 2001

Making Materials Flow, Harris and Wilson, 2003

Managing the Design Factory, Reinertsen, 1997

Manufacturing Planning and Control for Supply Chain Management, Jacobs, Berry, Whybark and Vollmann, Certification Edition, 

Manufacturing at Warp Speed, Schragenheim and Dettmer, 2000

Manufacturing Survival, Williams, 1995

Operations Management for Competitive Advantage, Chase, Jacob, and Aquilano, 11th Edition

Out of the Crisis, Deming 

Powered by Honda, Nelson, Moody and Mayo, 1998

Putting 5S to Work, Hirano, 1993

Quality Function Deployment: Integrating Customer Requirements Into Product Design, Akao 

Quality Function Deployment, Bossert

Quality, Productivity and Competitive Position,

Quality is Free, Crosby 

Quality, Productivity and Competitive Position, Deming

Quality Without Tears, Crosby 

Seeing the Whole, Jones and Womack

Six Sigma, Harry and Schroeder, 2000

The Certified Quality Engineer Handbook, Benbow, 2002

The Certified Quality Manager Handbook, Okes, 2001

The Complete Lean Enterprise: Value Stream Mapping for Administrative and Office Processes, Keyte and Locher

The Evolution of a Manufacturing System at Toyota, Fujimoto, 1999

The Goal, Goldratt and Cox, 1984

The Lean Design Guidebook, Mascitelli, 2004

The Lean Design Solution, Huthwaite, 2004

The Machine that Changed the World, Womack, Jones and Roos, 1990

The New Manufacturing Challenge, Kiyroshi Suzaki, 1987

The Purchasing Maching, Nelson, Moody and Stenger, 2001

The Six Sigma Handbook, Pyzdek, 2003

The Six Sigma Way, Pande, 2000

Toyota Production System:  Beyond Large Scale Production, Ohno, 1988 

The Toyota Way, Liker, 2004

Toyota Production System, Monden, 1993

Unleashing the Killer App, Downes and Mui. Harvard Business School Press, 1998. Hall Europe, 1998

Value Stream Management for the Lean Office, Tapping and Shuker, 2003

What is Lean Six Sigma, George, Rowlands and Kastle, 2004

Resource / Manufacturing and Quality
Manufacturing Management and Risk Assessment
View Resource

Manufacturing Management (MM) is concerned with the conversion of raw materials and/or components into products or finished goods. This conversion is accomplished through a series of manufacturing procedures and processes. Manufacturing management includes such major functions as manufacturing planning, cost estimating and scheduling, engineering, fabrication and assembly, installation and checkout, demonstration and testing, product assurance, and shipment. Manufacturing considerations can begin as early as pre-MSA in which technical managers (system engineers, manufacturing, quality, etc.) assess the "manufacturing feasibility" associated with the current product or manufacturing approach.

Programs that require manufacturing will need to support manufacturing planning and control activities and may require that a manufacturing management system be put in place to support planned activities. The use of a comprehensive manufacturing management system will support the timely development, production, modification, fielding, and sustainment of affordable products by managing manufacturing risks and issues throughout the program life cycle. Meeting this objective is accomplished by including best practices and standards (i.e., AS6500, Manufacturing Management Program) in contracts with industry.

Per MIL-HNBK-896, states that "Manufacturing management system. SAE AS6500 requirements stipulate contractors should have an overall manufacturing management system that documents organizational responsibilities for each requirement in the standard. Refer to Section 6.4, Manufacturing planning for additional information on documented manufacturing plans."

Manufacturing Planning: The purpose of manufacturing planning is the identification of resources and integration into a structure that provides the capability to achieve production objectives. Manufacturing planning should include:

  • A Manufacturing Strategy
  • A Manufacturing Management Program (per AS6500 and MIL-HDBK-896)
  • A Manufacturing Plan
  • Material Management System (Material Requirements Planning)
  • Manufacturing Resource Planning, Scheduling, and Execution 
  • Workforce Plan 
  • Facilities and Tooling Planning, Scheduling, and Execution
  • Manufacturing requirements in contracts
  • Appropriate agreements with other agencies (e.g., DCMA)
  • Manufacturing assessments to support program decision points and major design reviews
  • Manufacturing cost estimating and cost analysis 
  • Manufacturing metrics and reviews at a frequency commensurate with manufacturing risks
  • Manufacturing System Verification
  • Manufacturing Surveillance 
  • Manufacturing risk assessment and management 
  • Smart Shutdown

Manufacturing Operations Management includes manufacturing planning and control systems. These planning and control systems are often managed using Manufacturing Resource Planning (MRP) software. These manufacturing planning and control systems have a three-tier planning hierarchy as listed below:

  • Front End: Sets the strategy and vision for the organization or program and creates the following long-range plans:
    • Demand Management Assessment and Planning
    • Capacity Planning 
      • Resource Planning 
      • Rough-cut Capacity Planning 
    • Material Planning 
      • Sales and Operations Planning
      • Master Production Scheduling
  • Engine: Takes the Front-End plans and develops more detailed plans that are aimed at mid-range planning timeframe
    • Detailed Capacity Planning
    • Detailed Material Planning
      • Material and Capacity Plans 
    • Distribution Requirements Planning (DRP)
  • Back End: This is the execution plans for production and supply chain operations and is based on the period of production associated with a particular product (hours, days, weeks, etc.).
    • Shop floor systems plan
      • Shop floor scheduling and control plans
    • Supplier systems plan
      • Vendor scheduling and control plans

Resource Pages will focus on the following Pinned Content:

  • MIL-HDBK-896 Manufacturing Management Program Guide
  • Production Part Approval Process (PPAP)
  • Manufacturing Key Performance Indicators (KPIs)
  • Manufacturing Management Tools and Resources 
  • Facilities Management / tooling and Test Equipment
  • Environment, Safety and Occupational Health (ESOH)
  • DMSMS/Obsolescence 
  • Corrosion Control 
  • Counterfeit Parts 
  • Manufacturing Workforce
  • Manufacturing Risk Identification 
  • MRL Resources 
  • DoD Technical Reviews and Audits 

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

Manufacturing Strategy and Plan

The program office shall develop and document the Manufacturing Plan/Strategy for the program. This can be in a standalone Manufacturing Plan or, as a minimum, documented in the Systems Engineering Plan (SEP), Life Cycle Sustainment Plan (LCSP), or Life Cycle Management Plan (LCMP). The program office may require the contractor to develop a Manufacturing Plan and contractor format is encouraged. If a contractor Manufacturing Plan is required, it should include the following elements:

  • Develop Manufacturing and Quality (M&Q) inputs to the Systems Engineering Plan (SEP)
    • Technical Schedule 
    • Technical Risk and Opportunity Management
    • Technical Program Office organization - Technical IPT
    • Engineering team identified 
    • Develop Technical Performance Measures and Metrics
  • Document the Quality Strategy in the Program Quality Plan 
  • Define manufacturing and quality assurance (M&Q) contractual requirements 
    • Specific M&Q requirements (AS6500 and AS9100
    • M&Q SOW inputs
    • Section L&M inputs
    • Develop M&Q Inputs to Award Fee/Incentive Fee plans
  • Develop IMP/IMS Entry and Exit requirements 
    • Anticipated manufacturing schedule
    • Identification of Program and Technical Reviews
    • M&Q reviews (MRL/PRR)
  • M&Q plans should include:
    • M&Q design considerations 
    • Producibility approach
    • Manufacturing process flow
    • Applicable elements of MIL-HDBK-896
    • Plans for KCs and Variability Reduction
    • Supplier management processes
    • Facility, tooling, test equipment, and manpower requirements
    • Plans for readiness assessments
    • Manufacturing process verification plans
    • Production rate analysis
    • Product acceptance strategy
    • First Article Inspections 
    • M&Q modeling and simulation models identified 
  • M&Q Supply Chain considerations 
  • Develop; DCMA Interface Agreements 

Guidance and other resources:

Manufacturing Planning and Control/Manufacturing Execution

Manufacturing Planning and Control (MPC) includes all of the systems and processes needed to design, develop, produce, and deliver a product to the customer, on time, with the right count, and right performance, all as specified in the contract. The system is concerned about planning and controlling all of the elements of manufacturing processes (manpower, machines, materials, methods, and measurements). Planning considerations include identifying and managing costs, availability, and process properties.  

Planning begins with the development of a Master Production Schedule (MPS) which drives the development of many lower-level detailed plans.  The MPS breaks down the production plan to show the quantity of each item to be made over a specific time period. The MPS takes into account the demand signal (contract for acquisition personnel) identifying what is needed (requirement), when it is needed, and where it is needed. The MPS also takes into account the Strategic or Operations Plan and the Resource Plan. The MPS is often managed by an integrated suite of software tools (MRP, MRP II, or ERP) that help users to manage their business and technical processes.

Material and Capacity Plans evolve from the MPS and detailed capacity plans and detailed material plans. Material planning begins with a Material Requirements Plan (MRP) that is used to help manage material requirements and inventory during the manufacturing process and provides a mechanism so that organizations can purchase raw materials and components in time to support production. Capacity plans are developed to determine if current production capabilities have the capacity to support the planned production rates and quantities based on labor, machines, facilities, etc. 

The Material Execution System (MES) is designed for managing, monitoring, and executing manufacturing processes in real time. The MES helps with:

  • Production Scheduling
  • Quality Management 
  • Inventory Management
  • Maintenance Management 
  • Work Order Management 

The MES works within the MRP/ERP systems used to manage major organizational functions, and the PLM system used to manage the lifecycle of the product from concept to production, sustainment, and to final disposal (see graphic below).

Manufacturing Execution Systems (MES) as a software suite that manages, tracks, and documents all manufacturing processes and the suite of tools is integrated to into a digital framework under Enterprise Resource Planning and Product Lifecycle Management. MES collects real-time manufacturing data from many sources and provides them to management of action. MES provides for the following:

  • Data Collection, Integration, Analysis, and Reporting:
  • Production Scheduling:
  • Work Order Management: 
  • Manpower and Machine Integration:
  • Material and Inventory Management:
  • Quality Management: 

Note: The DoD has not published many guides, references, resources for manufacturing planning and control as it is seen mainly as a contractor responsibility, with the exceptions of government production and maintenance facilities. The best references are college textbooks. see below for a couple of recommendations:

  • Operations Management, Chasse, Jacobs and Aquilano 
  • Factory Physics, Hopp and Spearman

Note: DAU offers a 3-day Supply Chain Management workshop several times a year and this workshop covers the following major manufacturing topics:

  • Manufacturing Production and Control
  • Forecasting Demand 
  • Demand Planning and Management 
  • Manufacturing and Material Planning 
  • Capacity Planning and Management 
  • Production and Control (Shop Floor Control)
  • Distribution

https://www.dau.edu/blogs/workshops-workshops-and-more-workshops 

DAU Continuous Learning Module:

Guidance and other resources:

Manufacturing Cost Estimating and Budgeting

Manufacturing Cost Estimating describes the nature and structure of manufacturing costs, the various techniques used to estimate cost those costs. Manufacturing and Quality (M&Q) personnel need to be able to support the following cost and budgeting activities:

  • Creation of cost estimates that are based on a cost estimating process and various cost estimating techniques:
    • Independent Cost Estimate (ICE) 
    • DoD Component Cost Estimate
    • Program Office Estimate (POE)
    • Dod Component Cost Position
    • Cost Capability Analysis (CCA) 
    • Independent Government Cost Estimate (IGCE)
    • Should Cost Estimate
  • DoD 5000.4-M, Cost Analysis Guidance and Procedures, identifies four major analytical methods or cost estimating techniques used to develop cost estimates for acquisition programs:
    • Analogy: The analogy method estimates the cost of a new item by starting with the cost of a similar existing item, then modifying this cost to take into account the differences between the old item and the new item.
    • Parametric (Statistical): The parametric (statistical) method uses regression analysis of a database of several similar systems to develop cost estimating relationships.
    • Engineering (Bottoms Up): The engineering method builds an estimate from the "bottom up" by analyzing the individual elements of the WBS.
    • Actual Costs: The actual cost method estimates the cost of future system units based on data from earlier/previous units, prototypes, or production lots of that same system (not a similar system).
  • Assessment of manufacturing and quality costs is a requirement of DoDI 5000.04, Cost and Software Data Reporting (CSDR) which requires program management offices (PMS’s) for “managing, overseeing, and executing funding (either appropriated funding or working capital funds) for developing, procuring (either initial procurement or procurement of spares or replacement parts), testing and evaluating, or sustaining a DoD acquisition program at any phase of the lifecycle.” Reporting those cost may require the following reports:
    • Integrated {Program Management Data Analysis Report (IPMDAR) 
    • Earned Value Management (EVM) analysis 
  • The identification manufacturing and quality cost risks and mitigation of cost growth on the program. The growth in cost was often matched by a slip in schedule and higher development risk. Cost growth is often linked to the lack of technical maturity, lack of design maturity, and lack of production maturity. 
  • Development of learning curves. The learning curve (cost improvement curve, or experience curve) is a well-known approach to modeling the effect of quantity on cost. Learning curve theorizes that people and organizations learn to do things more efficiently when performing repetitive tasks. The use of learning curves supports both the development of good cost estimates, but also the management of costs once the program is underway.

The cost to manufacture a weapon system or equipment results from a combination of the design, the physical facility, and the five M's (manpower, materials, methods, measurements, and machines) used to build the design and the management efficiency of the operation. As such, the manufacturing cost for a product should be viewed within the context of the factory in which the product will be built. Three other very significant cost factors will need to be identified to support the estimating activity, and these are rate, quantity and efficiency.

You will need to have a basic understanding of several accounting terms, especially as they relate to the manufacturing environment, if you are to understand manufacturing costs. These terms include:

  • Fixed Cost: A cost that does not chance with the production rate.
  • Variable Cost: Cost that do change with the production rate (materials).
  • Direct Cost: Cost that can be identified to a specific final cost objective>
  • Indirect Cost: Cost that cannot be identified to a specific cost element.
  • Recurring Cost: Cost that occur on a regular basis (labor, materials, etc.)
  • Nonrecurring Cost: Cost that occur only one time (travel, installation of equipment, etc.)

A classic division of manufacturing cost is between direct and indirect costs. Costs can also be described as fixed, or variable based on their behavior as production volume changes within broad limits. Finally, costs can be described as nonrecurring or recurring depending on when and how often costs are accumulated. Finally, costs can be described in multiple terms, thus materials could be both a direct and a variable cost.

DAU Cost Courses and other Videos:

Two important manufacturing cost tools are:

  • Learning Curve: Learning curve theorizes that people and organizations learn to do things more efficiently when performing repetitive tasks. The more often the task is performed or repeated, the more efficient the worker becomes and the less time it takes to perform those tasks. There is a usable pattern to the learning. And that pattern is different for different conditions. For that reason, a number of different learning curves have been developed. Learning curves are generally drawn showing that as the number of units produced doubles, the unit cost decreases in a predictable pattern. Learning that results in productivity and efficiency improvement can be attributed to:
    • Worker Learning 
    • Supervisor Learning
    • Applying 5S's to Workstations (Sort, Straighten, Shine, Standardize, and Sustain)
    • Tooling Improvements
    • Design/Producibility Improvements
    • Improved Work Methods
    • Improved Planning and Control
    • Increased Lot Sizes
    • Reduction in Waste (7 sources of waste in Lean are excess transportation, inventory, motion, waiting, over-production, over-processing, defects)
    • Improved Operation Sequencing and Synchronization 
    • Application of Lean and Six Sigma Activities 

Learning Curve Guidance and other Resources:

DAU Teaching Note: Application of Learning Curve Theory to Systems Acquisition

  • Work Measurement: Work Measurement is a labor standard based on a time and motion study that is used to measure and manage worker efficiency. Work measurement involves applying techniques to determine the time a qualified worker needs to complete a given task at a specific level of performance. The defined rate of working is the amount of product or work that can be produced by a qualified worker, working at a normal pace, in a normal space, using specific tools and processes.
    • Work Measurement Techniques:
      • Analytical Estimating
      • Historical Data 
      • Time Study
      • Predetermined Motion Time System
      • Work Sampling
      • Synthesis Method
    • Main Objectives of Work Measurement:
      • Identify and eliminate lost or non-productive time
      • Establish standard times for accomplishing a specific job
      • Measure performance against a defined standard
      • Establish operating goals and objectives
      • Improve performance against and productivity

Manufacturing Budgeting

Preparation of a manufacturing and quality budget requires Services and Agencies develop Program Objective Memorandums (POMs) to identify and request resources (money) to acquire capabilities and perform operations. The POM is part of the Programming Phase of the Program, Planning, Budget, and Execution (PPBE) process. 

In addition to developing POM inputs M&Q personnel need to focus on developing budgets that support various manufacturing and quality investments and operating expenses for the coming period and phase. Budgets should include an investment strategy that includes long lead funding for capital equipment, facilities, new processes, new materials, workforce development, sustainable manufacturing, supply chains, ManTech, continuous process improvements, and digital engineering efforts such as Industry 4.0 capabilities, etc.). 

Guidance and other Resources:

MIL-HDBK-896 Manufacturing Management Program Guide Requirements 

AS6500 and MIL-HDBK-896 requirements stipulate contractors should have an overall manufacturing management system that documents organizational responsibilities for each requirement in the standard. (MIL-HDBK-896 para 6.1)

AS6500 and MIL-HDBK-896A address many requirements including:

Manufacturing Planning (MIL-HDBK-896, PARA 6.4): Manufacturing plans should describe how their manufacturing management system meets the intent and requirements of the standard. The program office should require a deliverable manufacturing plan. Table III provides a list of topics to be addressed in the plan and includes the items listed below. 

  • Manufacturing System Verification
  • Facilities 
  • Tooling and Test Equipment
  • Manpower and Skills
  • Capacity Analysis
  • Capability Analysis 
  • Key Characteristics and Variability Reduction 
  • Process Capability and Control 
  • Supply Chain Management 
  • Modeling and Simulation 
  • Cost Estimating and Analysis 

Design Analysis (MIL-HDBK-896, PARA 6.2): Requires that producibility be considered as a part of design trade studies with manufacturing and quality engineers participating in the various systems engineering processes. The following is a list of important Design Analysis considerations:

  • Producibility Analysis: Requires that producibility be considered as a part of design studies. (MIL-HDBK-896, para. 6.2.1)
  • Key Characteristics (KCs) and processes: The identification of key product characteristics and key production process capabilities is a basic engineering task essential to successful manufacturing development. (MIL-HDBK-896, para. 6.2.2)
  • Design and Process Analysis: Requires the use of both Design Failure Modes and Effects Analysis (DFMEA) and Process Failure Modes and Effects Analysis (PFMEA) to identify and prevent failures early in the design and manufacturing processes. (MIL-HDBK-896, para. 6.2.3)

Manufacturing Risk Identification (MIL-HDBK-896, PARA 6.3): Manufacturing risk evaluations and assessments are performed as part of defense acquisition programs for oversight and risk assessment and come in a variety of forms (e.g. Production Readiness Reviews, Manufacturing Management/Production Capability Reviews, etc.). These evaluations and assessments are used to identify and manage risks as programs transition through the various acquisition phases and are often performed in support of program reviews and technical audits. These evaluations and assessments include:

  • Manufacturing Feasibility Assessment 
  • Manufacturing Readiness Level (MRL) Assessment
  • Production Readiness Review 

Manufacturing Operations Management (MIL-HDBK-896, PARA 6.5): Manufacturing Operations includes many functions and activities, to include:

  • Production Scheduling and Control
  • Process Planning and Control
  • Manufacturing Surveillance 
  • Manufacturing Data Analysis
  • Process Capability and Control 
  • Continuous Process Improvement 
  • Variability Reduction 
  • Measurement System Analysis 
  • Production Process Verification
  • First Article Inspection (FAI)/First Article Testing (FAT)
  • Supplier Management 
  • Cost Estimating and Cost Assessment

Integrated Master Plan (IMP) Entry Criteria (MIL-HDBK-896, PARA 6.6): The Integrated Master Plan (IMP) is an event-based, top-level plan consisting of a hierarchy of Program Events.  Each event is decomposed into specific accomplishments and each specific accomplishment is decomposed into specific Criteria.  The IMP is ultimately used to develop a time-based Integrated Master Schedule to show a networked, multi-layered schedule showing all the detailed tasks required to accomplish the work effort contained in the IMP. The IMP and IMS are related to the Work Breakdown Structure (WBS). The IMP provides a Program Manager (PM) with a systematic approach to planning, scheduling, and execution.  Includes manufacturing and quality entry criteria for many of the major life cycle milestones and design reviews:

  • Production Cost Estimates
  • Producibility (activities by phase)
  • Industrial Base Considerations 
  • Material Concerns (maturity, availability, etc.)
  • Technology Goals and ManTech studies
  • Supplier Goals
  • Production Demonstrations 
Production Part Approval Process (PPAP)

PPAP is structured process for new or revised parts, or parts produced from new or significantly revised production methods.

The Production Part Approval Process (PPAP) handbook is an industry standard that outlines the process to demonstrate engineering design and product specifications are met by the supplier’s manufacturing process. Through PPAP, suppliers and customers agree upon the requirements needed to obtain approval of supplier manufactured parts. Applicable to all parts and commodities, PPAP principles help reduce delays and non-conformances during part approval by providing a consistent approval process.

PPAP defines the approval process for new or revised parts, or parts produced from new or significantly revised production methods. The PPAP process consists of 18 elements that may be required for approval of production level parts. Not all of the elements are required for every submission. There are five generally accepted PPAP submission levels. The PPAP manual contains detailed information, guidelines and sample documents useful for completing the process requirements. The resulting PPAP submission provides the evidence that the supplier has met or exceeded the customer’s requirements, and the process is capable of consistently reproducing quality parts.

PPAPs 18 Elements:

  • Design Documentation (Records): Includes customer and supplier drawings and models. The documentation should include a copy of the purchase order.
  • Engineering Change Documents: Is required for a change to a part or product and provides a detailed description of changes of parts from previous revisions called Engineering Change Notice.
  • Customer Engineering Approval: Includes customer approval of sample production parts when required.
  • Design Failure Mode and Effects Analysis (DFMEA): Is an examination of the design risk by assessing the possible failure modes and their effects on the product or customer and the probability of occurrence. 
  • Process Flow Diagrams: Diagrams all the steps in manufacturing process from start to finish and includes components, measurement, and inspection.
  • Process Failure Mode and Effects Analysis (PFMEA): Identifies all of the possible failures within the manufacturing process for a specific product and includes a prediction of a potential process failure that could occur during production.
  • Control Plan: Identifies and details the preventive measures designed to mitigate possible PMFEAs and how quality will be implemented to ensure a stable, capable, and reliable process.
  • Measurement System Analysis (MSA): Documents the specifications and details of all equipment that will be used in the manufacturing process, and the conformance to ISO or TS standard. MSA usually includes Gage R&R studies on measurement equipment used to assess the impact on key and critical characteristics to control repeatability and reproducibility and confirmation that gages are calibrated to measure these characteristics to control measurement bias.
  • Dimensional Results: Are used to validate the measurements on drawing packages to ensure that they are correct. This includes a list of every dimension noted on the ballooned drawing or model with pass/fail assessment.
  • Design Verification Plan and Report (DVP&R): Provides Material / Performance Test Results which includes a summary of every test performed on the part, usually in the form of DVP&R (Design Verification Plan and Report).
  • Initial Process Studies: Documents all processes that will be used in the fabrication and assembly of a product and shows that critical processes are reliable. Includes SPC (statistical process control) charts.
  • Qualified Laboratory Documentation: Provides industry certifications for any lab that participated in validation testing or any offsite contracted test facilities that were used for validation or material testing.
  • Appearance Approval Report (AAR): Provides verification that the customer has approved the appearance of the product including color, texture, fit, and more.
  • Sample Production Parts: Are sample production parts that are sent to the customer for approval and can be stored at the customers site or supplier's site after the product development is complete.
  • Master Sample: Is the final sample of a part or product that has been signed off by customer and stored at the supplier.
  • Checking Aids: Isa detailed list of all tools used to inspect, test, and measure parts. This list should include the calibration schedule and frequency for the tool.
  • Records of Customer-Specific Requirements: Is a list of customer’s specific requirements for PPAP process.
  • Part Submission Warrant (PSW): Is a summary of entire PPAP submission.

Resources and Guidance:

Manufacturing Key Performance Indicators (KPIs)

What is a manufacturing Key Performance Indicator (KPI)?

A manufacturing KPI is a SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) metric that is used to track and improve the quality of production related activities. 

Many world-class manufacturing organizations measure their manufacturing performance in order to make sound business decisions and improve speed and quality. Digital technologies often help them capture this information and display that information on KRP dashboards. The following manufacturing KPIs contain both lagging and leading performance indicators and highlights that might be critical to your program. 

A key enabler for capturing this digital manufacturing information is the companies Enterprise Resource Program (ERP) that is a suite of software programs that support the automation of many business and manufacturing functions. Such software programs include SAP, Oracle, NetSuite, and others. This is NOT an endorsement of any of these tools. 

Manufacturing KPIs include: 

Customer: Measures how satisfied your customer is with your performance>

  • Customer Fill Rate: Measures the ratio of the number of orders delivered compared to the number of orders placed. 
  • On-Time delivery rate: Measures the ratio of product delivered on time compared to the total number of products delivered. 
  • Lead Time: The number of days it takes for a customer to receive an order, which includes order processing time + production time + delivery time. 
  • Customer Satisfaction Index: Is a percentage of the number customers who said that they were very or extremely satisfied with the order, divided by the number of customer surveys, and multiplied by 100.
  • Perfect Order Rate:  Is a percentage of times that customers receive the right order, at the right time, and to the right requirements.
  • On-Time Delivery to Commit:  Measures the percentage of time that the organization delivers the product on the schedule according to the contract or purchase order. 
  • Manufacturing Cycle Time: Measures the time it takes for an organization to produce a product from the time the order is released to production and completes production. 
  • Time to Make Changeovers: Measures the time it takes for an organization to switch a manufacturing line from making one product to making a different product.

Design: Measures how well the final design satisfies the customers performance and affordability requirements.

  • Drawing Release Rate: Measures the progress of designing the product.
  • Configuration Change Management: Measures the status and accounting of configuration changes, configuration verification, and configuration audits.
  • Hardware Qualification Testing: Measures hardware capability when meeting anticipated environmental and operational conditions.
  • Producibility: Is a measure of the relative ease of fabrication and assembly of the designed product.
  • Design Maturity: Is a measure of the design nearing completion counting the number of Class 1 and Class 2 changes vs. planned.

Quality: Measure important quality indicators of your ability to produce uniform, defect-free product.

  • 1st Pass Yield: Is a measure of the percentage of products that are manufactured without defects and to specifications the first time through the manufacturing process.
  • Cost of Quality (CoQ): The cost associated with appraisal and prevention, both internal and external. The cost of producing product that fails to meet requirements includes:
    • Scrap, Rework, and Repair
    • Waivers and Deviations
    • Failure Analysis
    • Planning Errors
    • Drawing Errors
    • Excess Inventory
  • Out of Station Work: Is a measure of work required on a product after it passes through a workstation. 
  • Quality Deficiency Reports: Is a measure of the number of quality problems requiring reporting. 
  • Material Review Board Actions: Is a measure of the number of actions going to the MRB, too many indicates quality problems. 
  • Defects per Million Opportunities (DPMO): Is a Six Sigma/DMAIC measure that identifies high performing quality targets.
  • Customer Rejects/Returns: Measures how many times customers rejects a product or request returns of products based on receipt a nonconforming product that is not conforming to requirements.
  • Supplier Quality Incoming: Measures the percentage of good product coming into receiving inspection from a given supplier.
  • Internal Audits: A systematic evaluation of an organizations Quality Management System to assess compliance, identify problems and implement corrective action. 
  • Warranty Claims and Costs:  Measures the costs associated with product problems after delivery. Cost can be 1.5-4% of sales.

Efficiency: Measures the use of plant, machinery, and equipment.

  • Throughput: Measures how much product is being produced on a machine, line, unit, or plant over a specified period of time (hour, day, week, month, etc.)
  • Capacity Utilization: Measures the utilization of a workstation given the relationship between actual output and the design capacity. What could be produced vs what was produced.
  • Schedule or Production Attainment: Measures the percentage of time the production plan is achieved. Completed work to Planned work. 
  • Machine Downtime: Measures the time a machine or workstation is not available for production. This includes scheduled downtime for maintenance, setups and unscheduled downtime and can include malfunctions, and breakdowns.
  • Overall Equipment Effectiveness (OEE): Is a measure of the performance of a single piece of equipment or an entire line. The measure is a multiplier of Availability x Performance x Quality.

Inventory: Measures your use of material inventory and how efficient you are.

  • WIP Inventory/Turns: Measures the efficient use of inventory materials by measuring the speed of work-in-progress through a production facility. It is calculated by dividing the cost of goods sold by the average inventory used to produce those goods.
  • Inventory Accuracy: Measures actual inventory on hand vs. what is recorded in the inventory system.

Human: Measures how well you are utilizing human capital. 

  • Employee Training/Certification: Is a measure of the total amount of money spent on training, by employee of by hour.
  • Employee Turnover: Measures employee satisfaction by looking at the number of job terminations in that period vs. the total number of employees. 
  • Employee Health and Safety Report: Measures the total number of recordable incidents or fatalities over a period of time. 

Compliance: Measures how well you meet statutory, regulatory, policy or standards requirements,

  • Reportable Health and Safety Incidents: The number of health and safety incidents that were reported to OSHA during a specified period of time. 
  • Reportable Health and Safety Rate: The number of work-related injuries per 100 employees during a specified period of time.
  • Reportable Environmental Incidents: The number of health and safety incidents that were reported to the EPA as occurring over a specified period of time.
  • Number of Non-Compliance Events / Year: The number of times a plant or facility operated outside of regulatory guidelines over a one-year period. 
  • Training/Certification: The percentage of employees that are fully trained and certified.
  • Compliance Audits – The number of compliance issues reported during an annual compliance audit.

Maintenance: Measures how well maintenance activities keep machines and equipment in an operational status.

  • Percentage Planned vs. Emergency Maintenance Work Orders: This is a ratio metric indicates how often scheduled maintenance takes place, versus more disruptive/un-planned maintenance.
  • Downtime in Proportion to Operating Time: This is the ratio of equipment downtime compared to equipment operating time. 
  • Machine Downtime: Includes all scheduled and unscheduled times that the machine is not in operation. 
  • Unscheduled Downtime: The amount of time a machine should be in operation but is not due to equipment failure.
  • Machine Set Up Time: The time it takes to set up a machine for a production run.
  • Maintenance Equipment Cost: The cost associated with maintaining and repairing equipment to ensure it is available for production.
  • Mean Time Between Failures: Measures the average time between equipment failures. 

Increasing Flexibility & Innovation: Measures how well an organization stays ahead of the competition.

  • Rate of New Product Introduction:  Measure how rapidly a new product can be introduced to the marketplace and includes design, development and manufacturing.

Reducing Costs & Increasing Profitability: Measures how well an organization meets costs and profitability goals.

  • Manufacturing Cost as a Percentage of Revenue: Is a measure of the total manufacturing costs to the overall revenues produced by an organization.
  • Productivity in Revenue per Employee: Is a measure of how much revenue is generated by a plant, business unit or company, divided by the number of employees.
  • Return on Assets/Return on Net Assets: Is a measure of financial performance calculated by dividing the net income from a plant by the value of fixed assets and working capital deployed.
  • Cash-to-Cash Cycle Time: The length of time between the purchase of a product, and the collection of payments.
Manufacturing Management Tools and Resources

Manufacturing Planning begins with the Systems Engineering Plan and SEP Outline. Manufacturing management Tools include:

Manufacturing Management Guidance and other Resources:

A Comprehensive List of Tools to Aid Manufacturing Personnel

This document identifies tools that could be used to help manage DoD acquisition technical, business, and management processes to include many manufacturing and quality activities. Most of these tools support systems engineering technical and technical management processes, but a few can be used to support business processes such as cost estimating, contract language, acquisition strategies, etc. Some tools could be used by a contractor, some by government personnel and some tools can bd used by many different people in many functional specialties. Most tools are available from multiple on-line sources, some tools may need to be purchased to use, many are free. 

Note: You can find more information on the tools listed below with a search of the web. 

Tools (listed in alphabetical order include):

  • 3Ps - Production Preparation Process
  • 5S’s (Sort, Straighten, Shine, Standardize, Sustain
  • 5-Whys
  • 7 - Basic Tools for Quality Improvement (includes the following which are also discussed separately):
    • Cause and Effect Diagram
    • Check Sheet
    • Control Chart
    • Histogram
    • Pareto Chart
    • Run Chart
    • Scatter Diagram
    • Stratification or Flowchart or 
  • 7 - Management and Planning Tools, or Advanced Tools for Quality Improvement (includes the following which are also discussed separately):
    • Affinity Diagram
    • Relations Diagram or Interrelationship Diagraph
    • Tree Diagram
    • Matrix Diagram
    • Matrix Data Analysis
    • Arrow Diagram
    • Process Decision Program Chart (PDPC)
  • 8D/PSP (Eight Disciplines/Problem Solving Process)
  • A3 Problem Solving Chart
  • Acceptable Quality Levels (AQL)
  • Acceptance Sampling 
  • Active Risk Manager (ARM)
  • Affinity Diagram
  • Advanced Product Quality Planning (APQP) Core Tools
  • Arrow Diagram (Chart)
  • AS6500 Manufacturing Management System (MMS)
  • AS9100 Advanced Quality Management Systems (QMS)
  • AS9103 Variation Management of Key Characteristics
  • AS9110 Maintenance
  • AS9120 Distributors
  • AS9102 FAI
  • AS9115 Software QA
  • AS9131 N/C Document
  • AS9132 Marking
  • AS9133 Supplier QA
  • AS 9137 AQAP Align
  • AS 9138 Statistical Process Control
  • AS9162 Self Verification
  • Axiomatic Design 
  • Balanced Scorecard
  • Baldrige Performance Excellence Criteria
  • Bekidou Rate
  • Benchmarking
  • Bill of Materials (BOM) 
  • Bone Diagram
  • Bottleneck Analysis
  • Box and Whisker Plot
  • Bubble Chart
  • Capacity Matrix
  • Capacity Analysis
  • Cause and Effect Diagram (Fishbone or IshIkawa)
  • Cause and Effect Matrix
  • CFMEA – Concept Failure, Mode and Effect Analysis
  • Check Sheet
  • Chokko Rate
  • Computer-Aided Design (CAD)
  • Computer-Aided Manufacturing (CAM)
  • Computer Aided Process Planning (CAPP)
  • Computer-Aided Three-Dimensional Interactive Application (CATIA)
  • Computer Integrated Manufacturing (CIM)
  • Consensogram
  • Contingency Planning
  • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart -U-chart for Attribute Data
    • Control Chart - NP-chart for Attribute Data
    • Control Chart - P-chart for Attribute Data
    • Control Chart - X-barR chart for Variable Data (measurable)
    • Control Chart - X-bar-S chart for Variable Data
    • Control Chart - X-MR/I-MR chart for Variable Data
  • Correlation Chart (Scatter Diagram)
  • Cost/Benefit Analysis
  • Cost of Quality Analysis
  • Cost of Quality (COQ)
  • Cost Modeling (Estimating)
  • Criteria Testing
  • Critical Chain Project Management 
  • Critical Design Review Checklist (DoD)
  • Critical Path/PERT
  • Critical to the Customer (CTC)
  • Critical to Quality (CTQ) Tree
  • Customer Contingency Table
  • DCOV – Define, Characterize, Optimize and Verify
  • Deming Cycle or Wheel (PDCA)
  • Departmental Purpose Analysis
  • Design of Experiments (DoE)
  • DFMEA - Design Failure Mode and Effects Analysis 
  • DFMA - Design for Manufacturing and Assembly 
  • Design to Cost (DTC)
  • DFSS - Design for Six Sigma 
  • DMAIC - Define, Measure, Analyze, Improve and Control
  • DMADV (see DCOV)
  • Domainal Mapping
    • Factory Modeling and Simulation
    • Producibility Analysis & Ergonomics
    • Process Planning 
    • Production Planning & Scheduling
    • Line Balancing & Bottleneck Analysis
    • Capacity Planning
    • Predictive Analytics & Optimization
    • Facility Planning, Layout and Design
    • Virtual Factory Mock-up
  • Failure Mode and Effects Analysis (FMEA)
  • Fault Tree Analysis 
  • First Article Inspection
  • First Article Testing 
  • Flow Chart or Process Flow Chart
  • Force Field Analysis
  • Gage R&R Studies
  • Gantt Chart
  • Histogram (Frequency or Bar Chart)
  • Hoshin Kanri (Quality Policy Deployment) 
  • Interrelationship Diagraph (also see Relations Diagram, or Network Diagram)
  • ISO 9001 Quality Management Systems (QMS) 
  • Kano Model
  • KJ model - Kawalota Jiro (see affinity diagram)
  • Lead Time Analysis
  • Learning Curve 
  • Learning Curve Analysis
  • Line of Balance (LOB)
  • Taguchi Loss Function
  • Lotus Diagram
  • Manufacturing Cost Estimating
  • Manufacturing Plan 
  • Manufacturing Readiness Assessment (MRA)
  • Manufacturing Readiness Level (MRL) Criteria
  • Matrix Diagram
  • Matrix Data Analysis Diagram
  • Measurement Systems Analysis (MSA) 
  • MIL-HDBK-896A Manufacturing and Quality Program
  • Multi-Vari Charts
  • Nominal Group Technique
  • One Piece Flow
  • Operations Process Chart
  • Overall Equipment Effectiveness (OEE)
  • Pareto Charts (Template)
  • Part-Family Analysis
  • Paynter Chart
  • P-Diagram or Parameter Design 
  • PERT Chart (Program Evaluation Routine Technique)
  • PFMEA – Process Failure, Mode and Effect Analysis
  • Pie Chart
  • Preliminary Design Review Checklist (DoD) 
  • Preliminary Hazards List (PHL)
  • Process Capability Studies (Cp and Cpk)
  • Process Performance Studies (Pp and Ppk)
  • Process Decision Program Chart (PDPC)
  • Producibility Analysis/Assessments
  • Producibility Assessment Worksheet
  • Producibility Engineering and Planning (PEP) Program
  • Production Part Approval Process
  • Production Readiness Review (PRR) Checklist
  • Programmatic Evaluation of ESOH (PESHE)
  • Pugh Matrix
  • Quadrant Chart
  • Quality Function Deployment (QFD)
  • Queuing Theory/Waiting Line Analysis
  • Radar Chart
  • Rational DOORS
  • Relation Diagram
  • Reliability Growth Analysis
  • Requirements Verification (Traceability) Matrix (RVM)
  • Risk Management Assessment Tool 
  • Route Sheet
  • Run Chart
  • Scatter Diagram (Mind Mapping)
  • SIPOC – System, Input, Process, Output and Customer
  • Six Sigma
  • SMART – Specific, Measurable, Attainable, Resources, Time
  • Spaghetti Diagram
  • Spider Diagram
  • Stratification
  • Statistical Process Control (SPC)
  • Supply Chain Management Risk Assessment
  • Swim Lane Chart (sometimes called a Deployment Flow Chart)
  • SWOT Model (Strength, Weaknesses, Opportunities and Threats
  • Systems Engineering Plan (SEP) 
  • Takt Time Analysis
  • Technical Risk Identification and Mitigation System (TRIMS)
  • Technology Readiness Level (TRL) Checklist
  • Theory of Constraints Analysis
  • Throughput Analysis Tool
  • Throughput Accounting
  • Tolerance Analysis 
  • Tolerance Design 
  • Total Productive Maintenance (TPM) 
  • Trade Studies/Analysis
  • Transition to Production (Willoughby Templates)
  • Tree Diagram
  • TRIZ Matrix 
  • Value Stream Mapping (VSM) 
  • Variability Reduction Program
  • Venn Diagram
  • Work Center
  • Work Measurement
  • X-Matrix
  • Yamazumi Chart

Note: You can find a lot of additional information by googling the tool, by visiting a number of different academic sites, or professional organizations, or by visiting various Communities of Practice (CoPs).

Facilities Management \ Tooling and Test Equipment 

Facilities management encompasses a variety of professional skills that focus on the design, construction, management, of an installation to include plant, equipment, and tooling. Facilities management includes all permanent and semi-permanent real property required to support a system throughout the systems life cycle. Facility management includes studies of facility requirements to include plant location, facility size and layout, production system or environment (job shop, batch processing, continuous flow, etc.), environmental, safety, and occupational health considerations, property management and control, environmental controls (HVAC), maintenance, security considerations, and budgeting of such property through final disposal or facility shutdown. Major facility concerns include:

  • Developing a Facility Strategy: Should include facility design, shop floor layout, physical and cybersecurity, plant safety, ESOH considerations, equipment (machine) design and layout, maintenance concepts, etc.
  • Conducting Facility risk assessments: Could include capacity analysis, bottleneck analysis, modeling and simulation, and flow analysis.
  • Monitoring and managing facility, facility goals and metrics
  • MIL-HDBK-896, Manufacturing Management Program Guide https://www.dodmrl.com/MIL-HDBK-896A%20Manufacturing%20Managment.pdf 

Tooling is designed and developed to aid in the manufacture of parts or components, or to support assembly operations. Tooling includes jigs, dies, fixtures, molds, patterns, taps, gauges, other equipment and manufacturing aids. Special tooling, special test and special inspection equipment are included under the broad definition of tooling. Production tools may be developed and used for a one-time or short production run or may need to be developed to withstand the robust environment of long-term rate production. 

Major tooling concerns include:

  • Developing a Tooling Strategy
  • Conducting Tooling risk assessments
  • Monitoring and managing tooling program to include managing tooling, tooling goals and metrics 

Facilities and Tooling Guidance and Resources:

  • DoDD 4275.5 Acquisition and Management of Industrial Resources 
Environment, Safety and Occupational Health (ESOH)
ESOH Definitions:
  • Environment. Air, water, land, living things, built infrastructure, cultural resources, and the interrelationships that exist among them. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.2.). Alternate definition: The aggregate of all external and internal conditions (such as temperature, humidity, radiation, magnetic and electrical fields, shock, vibration, etc.), whether natural, manmade, or self-induced, that influences the form, fit, or function of an item. (Reference MIL-HDBK-338B Electronic Reliability Design Handbook)
  • Safety. The programs, risk management activities, and organizational and cultural values dedicated to preventing injuries and accidental loss of human and material resources, and to protecting the environment from the damaging effects of DoD mishaps. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.14.)
  • Occupational Health. Activities directed toward anticipation, recognition, evaluation, and control of potential occupational and environmental health hazards; preventing injuries and illness of personnel during operations; and accomplishment of mission at acceptable levels of risk. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.12.)
  • ESOH Management. Sustaining the readiness of the U.S. Armed Forces by cost effectively maintaining all installation assets through promotion of safety, protection of human health, and protection and restoration of the environment. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.6.)

ESOH considerations need to be included in the Systems Engineering Plan (SEP), Programmatic ESOH Evaluation, and National Environmental Policy Act (NEPA)/Executive Order (EO) 12114 Compliance Schedule. 

  • NEPA and NEPA Compliance Schedule: Requires 
    • Environmental impacts to be considered during the planning process.
    • Agencies and public participation in the planning process (including the DoD).
    • Disclosure about the action, alternatives, environmental effects and mitigation of their actions.
  • Hazardous Material Management Program (DoDI 5000.02 and NAS 411): Requires the Program Manager to develop a Hazardous Material Management Implementation Plan that will include procedures for identifying, minimizing use of, tracking, storing, handling, packaging, transporting, and disposing of such material. In addition, the contractor shall conduct a HMMP tailored to the contract, that will eliminate, reduce, or control hazardous materials during the system life cycle while:
    • maintaining an appropriate balance with (specified) performance requirements and
    • the cost of HMMP is performed as part of the systems engineering process while protecting human health and the environment.
  • Pollution Prevention Program (DOD 5000.02 and DODI 4715.4): Tasks the PM to identify and evaluate environmental and occupational health hazards and establish a pollution prevention program.  The PM shall identify the impacts of the system on the environment during its life (including disposal), the types and amounts of pollution from all sources (air, water, noise, etc.) that will be released to the environment, actions needed to prevent or control the impacts, ESOH risks associated with using the new system, and other information needed to identify source reduction, alternative technologies, and recycling opportunities.  The pollution prevention program shall serve to minimize system impacts on the environment and human health, as well as environmental compliance impacts on program TOC.  A fundamental purpose of the pollution prevention program is to identify and quantify impacts, such as noise, as early as possible during system development, and to identify and implement actions needed to prevent or abate the impacts.
  • Programmatic Environmental Safety and Health Evaluation (PESHE): The PM for all programs, regardless of ACAT level, shall prepare a PESHE which incorporates the MIL-STD-882D process and includes the following:
    • identification of ESOH responsibilities
    • the strategy for integrating ESOH considerations into the systems engineering process
    • identification of ESOH risks and their status
    • a description of the method for tracking hazards throughout the life cycle of the system
    • identification of hazardous materials, wastes, and pollutants (discharges/emissions/ noise) associated with the system and plans for their minimization and/or safe disposal 
    • and a compliance schedule covering all system-related activities for the NEPA
  • System Safety and Health Program (MIL-STD-882E): The integration of system safety and health issues into the systems engineering process is a MIL-STD-882) requirement that goes from cradle to grave and includes:
    • System Safety
    • Software Safety
    • Explosives Safety
    • Laser Safety
    • Occupational Safety
    • Public Safety 

ESHO Guidance and other Resources:

ESOH Tools and Checklist:

  • ADDM PESHE Template, use internet search to find
  • AFLCMC ADDM PESHE Template, use internet search to find
  • ISO 14000 Operational Gap Analysis Tool, use internet search to find
  • DoD ESOH Management Evaluation Criteria, use internet search to find
DMSMS/Obsolescence 

Diminishing Manufacturing Sources and Material Shortages (DMSMS) is the loss of sources of items or material, surfaces when a source announces the actual or impending discontinuation of a product, or when procurements fail because of product unavailability. DMSMS may endanger the life-cycle support and viability of the weapon system or equipment.

DMSMS/Obsolescence requires the Program Manager, through the Product Support Manager, to develop, ensure funding, and execute a DMSMS management plan and conduct proactive risk-based DMSMS management per that plan to identify current DMSMS issues, forecast future DMSMS issues, program and budget for resolving DMSMS issues, and implement those resolutions IAW DODI 4245.15. Implementing DMSMS issue resolutions will take into account a parts management process that considers SCRM, supportability, loss of technological advantage, and obsolescence when selecting parts used in DMSMS resolutions. In addition, the PSM will use both current and forecasted DMSMS issues in developing product roadmaps for supportability.”

DMSMS typically follows a life cycle where new parts are introduced into the system and then the total number of parts grows in availability until the system(s) these parts are in start to decline. Then parts availability becomes a concern. Thus, design engineers need to take into account part availability and maturing when selecting parts for their product.

Image removed.

The SD-22 DMSMS Guidebook is another key resource and reference. It includes common practices developed by various DoD organizations to achieve these goals and includes examples of results for review and consideration as well. The primary objectives of the SD-22 are to:

  • Create awareness of the extent and impact of DMSMS issues on DoD systems
  • Provide best practices to PMs for implementing a robust, risk-based DMSMS management process, and building a cost-effective DMSMS management program
  • Encourage DMSMS resilience by using a modular, open system design approach along with other supportability-related design considerations in conjunction with part selection procedures that choose items with significant time left in their life cycle and with viable replacement options whenever possible in order to reduce the likelihood that a design will experience near-term DMSMS issues and increase the probability of a quick recovery when issues do occur
  • Define DMSMS support metrics to measure the effectiveness, efficiency, and return on investment (ROI) of a robust DMSMS management program
  • Promote affordable and efficient program office support through rapid and cost-effective DMSMS management best practices and resolutions that take into account equipment life cycles, technology changes, and planned obsolescence
  • Promote the exercise of best practices to address obsolescence risks throughout the life cycle. 

DAU Continuous Learning Modules and other training:

  • DMSMS: Best Practices for Contracting https://media.dau.edu/media/1_flz3e5jg 
  • CLL 201,“DMSMS Fundamentals” (continuous learning module
  • CLL 202,“DMSMS for Executives” (continuous learning module)
  • CLL 203,“DMSMS Essentials” (continuous learning module)
  • CLL 204,“DMSMS Case Studies” (continuous learning module)
  • CLL 205,“DMSMS for Technical Professionals” (continuous learning module).

Guidance and other Resources: 

Corrosion Control 

10 U.S.C. 2228 requires DoD to develop and implement a long-term strategy to address the corrosion of its equipment and infrastructure. A key element of this strategy is programmatic and technical guidance provided in this guidebook. 

“Corrosion is the deterioration of a material or its properties due to a reaction of that material with its chemical environment.” Corrosion is far more widespread and detrimental than merely rust of steel or iron. The acquisition program needs to consider additional materials, including other metals, polymers, composites, and ceramics affected by the operational environment. 

DoD Corrosion Prevention and Control Planning Guidebook for Military Systems and Equipment focuses on these keys to CPC success:

  • Integrate CPC planning and execution early and throughout the acquisition process. 
  • Resource the necessary funding and expertise. 
  • Manage CPC risks. 
  • Incorporate CPC language in procurement and contract documents. 
  • Monitor CPC planning and execution throughout the acquisition process so that the system design keeps corrosion prevention in mind.

DAU Continuous Learning Modules:

Guidance and other Resources:

Counterfeit Parts 

DoD Instruction 4140.67 establishes policy and assigns responsibilities to prevent the introduction of counterfeit materiel at all levels of the DoD supply chain. It applies to all life cycle phases of acquisitions and materiel management, from the time an operational requirement is identified to introduce an item or piece of equipment into the DoD supply chain, to the ultimate disposition, phase-out or retirement of that item or equipment.

Key DoD policy embodied in this issuance:

  • Employ a risk-based approach to reduce the frequency and impact of counterfeit material within DoD acquisition systems and life cycle sustainment processes
  • Require remediation for counterfeit materiel discovered after delivery of materiel
  • Direct application of authentication technologies
  • Report suspect and confirmed counterfeit materiel to GIDEP within 60 days

Guidance and other Resources:

Manufacturing Workforce 

Manufacturing Workforce addresses the contractor’s anticipated workforce needs (both in numbers and skills) should be evaluated as part of MRL assessments and PRRs. the manpower required for a given program should be compared with projected requirements for all programs in a given facility to determine if there are future constraints that must be addressed. To assess the ability to acquire skilled workers, consider the types of local industries, the competitiveness of the labor market, and the availability of technical and higher education.

Several Industry Associations (The Manufacturing Institute, Deloitte, National Association of Manufacturers, etc.) have conducted studies of the future of the U.S. manufacturing workforce. In addition, a 2019 DoD Study on the Factory of the Future, identified several manufacturing workforce findings: 

  • U.S. manufacturers face challenges in two high-level workforce areas that are increasingly limiting DoD’s ability to meet weapon system production needs: 
    • Skill, competency, and capability deficiencies
    • Growing labor demand-supply disconnect
  • The Department’s PQM acquisition career field has existed in a suboptimal state for nearly 25 years and will continue to impact readiness due to:
    • limited PQM billets allocated to programs and organizations across the DoD 
    • a resultant workforce that is insufficiently sized, trained, and seasoned; and 
    • an insufficient suite of manufacturing policies, practices and standards.
  • Manufacturing has not been promoted nationally as an attractive career choice, which is limiting workforce supply pipelines for defense manufacturers and DoD M&Q workers. 
  • The limited number of people, nationwide, who have the ability to work in manufacturing was stated as a primary concern for both government and industry manufacturing and sustainment operations and is not projected to improve.
  • Finding employees who can pass a security clearance check is increasingly challenging due, in part, to inconsistency between State drug laws and Federal employment requirements, in addition to the time it takes to obtain the clearance
  • Commercial industry and the DIB are increasingly competing against each other for talent from the same small talent pool; the lower level of benefits offered by DoD puts them at a disadvantage compared to industry. This, in turn, reduces the PQM accession pipeline.
  • The PQM career field billet structures in the military departments and agencies do not provide healthy and diverse pathways for career advancement. This is a static condition that does not appear to be improving.
  • Of the organic sites visited, several stated that insufficient funding for infrastructure is leading to facilities and equipment maintenance and obsolescence; and impacts ability to attract and retain talented workers
  • As the traditional DIB confronts workforce challenges (e.g., aging employees, retirement obligations), DoD is expected to struggle to generate needed capacity to support longstanding and emerging programs critical to national security 

Guidance and other Resources:

Manufacturing Risk Identification 

Manufacturing Risk Identification: Manufacturing assessments should be conducted early in the life cycle, and throughout the life of the acquisition program and include Feasibility Assessments, Manufacturing Readiness Level Assessments, and Production Readiness Reviews. (MIL-HDBK-896, para. 6.3)

Assessing Manufacturing Risk: A Best Practice

MIL-HDBK-896 requires:
  • Manufacturing Feasibility Assessments  
  • Manufacturing Readiness Level (MRL) Assessments
  • Production Readiness Reviews (PRR)
  • Independent Technical Risk Assessment (ITRA)
 
Manufacturing Feasibility Assessments are typically performed early in the life cycle when competing design concepts are being considered. The assessments are conducted to identify potential manufacturing constraints and risks and the capability of the contractor to execute the manufacturing efforts.
 
Manufacturing Readiness is the ability to harness the manufacturing, production, quality assurance, and industrial functions to achieve an operational capability that satisfies mission needs – in the quantity and quality needed by the warfighter. Public law requires the use of manufacturing readiness levels or other manufacturing readiness standards as a basis for measuring, assessing, reporting, and communicating manufacturing readiness and risk on major defense acquisition programs throughout the DoD. The use of MRLs to assess manufacturing readiness can foster better decision making, program planning and program execution through improved understanding and management of manufacturing risk. Often these assessments will take place during program reviews or technical reviews and audits. 
 
The Production Readiness Review (PRR) is a Systems Engineering Technical Review at the end of EMD that determines if a program is ready for production. MRL 8 is the target for Low-Rate Initial Production (LRIP) and MRL 9 is the target for Full Rate Production (FRP); these targets should be reflected in the acquisition program baseline. The PRR assesses whether the prime contractor and major subcontractors have completed adequate production planning and confirms that there are no unacceptable risks for schedule, performance, cost, or other established criteria. Generally, incremental PRRs are conducted at the prime and major subcontractors.
 
The Independent Technical Review Assessment (ITRA) is a requirement of law and policy.
  • NDAA 2017 - Section 807 requires an ITRA prior to MS A where critical technologies and manufacturing processes need to be matured, and MS B where there is a decision to enter LRIP or FRP
  • DoD Policy Memorandum for ITRAs for MDAP programs and DoDI 5000.88:
    • Conducted on all MDAPs prior to MS or Production decisions
    • Considers the full spectrum of Engineering, Technology, and Integration risks
    • ITRAs are independent of the program office
 
Guidance and other Resources:

DAU Continuous Learning Modules and other videos:

Note:  Go to http://dodmrl.com for the latest information on the MRL Deskbook, MRL Matrix, and MRL Users Guide

MRL Resources

MRL POCs: 

DoD Technical Reviews and Audits

Current list of DoD Technical Reviews and Audits, many of these are covered in the DoD Systems Engineering Guidebook:

  • Alternative Systems Review (ASR)
  • System Requirements Review (SRR)
  • System Functional Review (SFR)
  • Preliminary Design Review (PDR)
  • Critical Design Review (CDR)
  • Test Readiness Review (TRR)
  • System Verification Review (SVR)
  • Functional Configuration Audit (FCA)
  • Production Readiness Review (PRR)
  • Physical Configuration Audit (PCA)
  • In-Service Review (ISR)
  • Manufacturing Readiness Assessments (MRAs)
  • Technical Readiness Assessments (TRAs)
  • Independent Technical Risk Assessments (ITRAs)

DoD Checklist for Technical Reviews and Audits: 

DoD Systems Engineering Guidebook https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

 

Discussions / Manufacturing and Quality

No content available.

Events / Manufacturing and Quality

No content available.

Announcements / Manufacturing and Quality

Community Announcement / Manufacturing and Quality
DoD Manufacturing and Quality Body of Knowledge published
View Announcement

​The Undersecretary of Defense Research and Engineering has published the DoD Manufacturing and Quality Body of Knowledge (BoK). 

The Department of Defense (DoD) Manufacturing and Quality (M&Q) Body of Knowledge (BoK) is a compilation of best practices and lessons learned for completing M&Q activities across the DoD system acquisition life cycle.

Links to the six chapters can be found at the bottom of the page at https://ac.cto.mil/maq/.

Community Announcement / Manufacturing and Quality
DoD Producibility and Manufacturability Engineering Guide published
View Announcement

​The Undersecretary of Defense Research and Engineering has published the DoD Producibility and Manufacturability  Engineering Guide.  

This guide describes the elements of Department of Defense (DoD) producibility and manufacturability engineering over the system life cycle. Both producibility and manufacturability promote ease of manufacture of defense systems. Producibility focuses on design considerations while manufacturability focuses on improving manufacturing processes and factory floor operations. 

This guide is intended primarily to assist manufacturing and quality (M&Q) engineers to provide input to systems engineering activities starting with initial system concept and product design and continuing throughout the life cycle. The guide provides useful definitions, references, tools, and best practices. Although written primarily for M&Q practitioners, the guide includes information for engineering and technical management (ETM) and acquisition functional disciplines including design engineering, program management, quality assurance, contracting, logistics, and procurement.

DoD-Producibility-and-Manufacturability-2024.pdf (cto.mil) 

Community Announcement / Manufacturing and Quality
SAE AS6500 Manufacturing Management Program update
View Announcement

SAE has published an update to their Manufacturing Management Program Standard, AS6500. Please see the press release for additional information. SAE International Publishes Updated Aerospace Recommended Practice for Managing Manufacturing Operations​

Community Announcement / Manufacturing and Quality
New Manufacturing and QA metrics Data Item Description available
View Announcement

A new Data Item Description is available to let ​Program Offices collect Manufacturing and Quality Assurance metrics from contractors. The Manufacturing and Quality Assurance Status Report (MQASR), DI-QCIC-82323, provides the Government with the information needed to monitor the status of manufacturing development and production activities, supplier management, and quality performance. You can download a copy from the ASSIST database at  https://quicksearch.dla.mil/qsDocDetails.aspx?ident_number=283861

Resources / Manufacturing and Quality

Community Resource / Manufacturing and Quality
Systems Engineering Process Support

Systems Engineering is a disciplined approach for the specification, design, development, realization, technical management, operation, and retirement of a weapon system. SE is an interdisciplinary and collaborative effort requiring close interaction with many disciplines to include operations, maintenance, logistics, test, production, quality, etc. The practice of SE is composed of 16 processes: 8 technical processes and 8 technical management processes. These 16 processes provide a structured approach to increasing the technical maturity of a system, increasing the likelihood that the capability being developed balances mission performance with cost, schedule, risks, and design considerations. The DoD Systems Engineering model is located below, and M&Q personnel need to support these activities and processes.

 Systems Engineering Process | www.dau.edu

Source: DoD Systems Engineering Guidebook https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

Eight Technical Processes that may require M&Q participation:

  • Stakeholder Requirements Definition, Systems Engineering Guide, 4.2.1 
  • Requirements Analysis, Systems Engineering Guide, 4.2.2
  • Architecture Design, Systems Engineering Guide, 4.2.3
  • Implementation, Systems Engineering Guide, 4.2.4
  • Integration, Systems Engineering Guide, 4.2.5
  • Verification, Systems Engineering Guide, 4.2.6
  • Validation, Systems Engineering Guide, 4.2.7
  • Transition, Systems Engineering Guide, 4.2.8

Eight Technical Management Processes that may require M&Q participation:

  • Technical Planning, Systems Engineering Guidebook, 4.1.1  
  • Decision Analysis, Systems Engineering Guidebook,4.1.2
  • Technical Assessment, Systems Engineering Guidebook, 4.1.3  
  • Requirements Management, Systems Engineering Guidebook, 4.1.4   
  • Risk Management, Systems Engineering Guidebook, 4.1.5  
  • Configuration Management, Systems Engineering Guidebook, 4.1.6  
  • Technical Data Management, Systems Engineering Guidebook, 4.1.7  
  • Interface Management, Systems Engineering Guidebook, 4.1.8  

The industry standard for systems engineering is ISO/IEC/IEEE 15288, “Systems and Software Engineering–System Life Cycle Processes has a slightly different list of technical and technical management processes, which is described in the graphic below:

Technical Management Processes

  • Project Planning
  • Project Assessment and Control 
  • Decision Management 
  • Risk Management 
  • Configuration Management
  • Information Management
  • Measurement 
  • Quality Assurance

Technical Processes 

  • Business or Mission Analysis
  • Stakeholder Needs and Requirements Definition
  • System Requirements Definition
  • Architecture Definition
  • Design Definition
  • System Analysis
  • Implementation
  • Integration
  • Verification
  • Transition
  • Validation
  • Operation
  • Maintenance 
  • Disposal

Source: DoD Best Practices for Using Systems Engineering Standards ((ISO/IEC/IEEE 15288, IEEE 15288.1, and IEEE 15288.2) on Contracts for Department of Defense Acquisition Programs

This resource page will focus on the following Pinned Content:

  • Systems Engineering Resources and Guidance
  • Systems Engineering Tools and Checklist
  • Technical Reviews and Audits
  • Producibility Engineering 
  • Key Characteristics 
  • Producibility Best Practices 

DoD Systems Engineering Guidebook https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

DAU Continuous Learning Modules and other training: 

Systems Engineering Plan (SEP) 

The Systems Engineering Plan (SEP) is a planning and management tool that defines the methods used for implementing all system requirements having technical content, technical staffing, and technical management. The SEP is specific to a program and is an important tool for managing complex technology-based system developments. The SEP should be tailored to meet program needs and objectives.

The purpose of the Systems Engineering Plan (SEP) is to help Program Managers develop, communicate, and manage the overall systems engineering (SE) approach that guides all technical activities of the program. The SEP documents key technical risks, processes, resources, metrics, SE products, and completed and scheduled SE activities. The SEP is a living document that should be updated as needed to reflect the program’s evolving SE approach and/or plans and current status.

The SEP describes the integration of SE activities with other program management and control efforts, including the Acquisition Strategy, Integrated Master Plan (IMP), Work Breakdown Structure (WBS), Integrated Master Schedule (IMS), Risk Management Plan, Technical Performance Measures (TPMs) and other documentation fundamental to successful program execution. The SEP also describes the program’s technical requirements, engineering resources and management, and technical activities and products as well as the planning, timing, conduct, and success criteria of event-driven SE technical reviews throughout the acquisition life cycle.

The SEP is the program’s blueprint for the conduct, management, and control of all technical activities, the SEP captures decisions made during the technical planning process and communicates objectives and guidance to program personnel and other stakeholders. The SEP should define the “who, what, when, why, and how” of the SE approach and should include the following: 

  • The program organization with roles and responsibilities, authority, accountability, and staffing resources.
  • Key activities, resources, tools, and events that support execution of the SE technical processes and technical management processes.
  • The event-driven technical reviews to be conducted and the approach to technical reviews based on successful completion of key activities as opposed to calendar-based deadlines.
  • The approach for how requirements and technical performance trade-offs are balanced within the larger program scope to deliver operationally effective, suitable, and affordable systems.
  • The identification of key design considerations and criteria.
  • The use of and employment of modular design.
  • The identification of how manufacturing and quality planning will be incorporated into the SEP and systems engineering processes.
  • The identification of and management of Key Performance Parameters (KPPs) and Key System Attributes (KSAs) and development of a prototyping strategy to ensure system requirements can be met within cost and schedule constraints.
  • The program’s strategy for identifying, prioritizing, and selecting the set of TPMs and metrics (TPMM) should provide sufficient insight into the technical progress and program risks.

The Systems Engineering Plan Outline:

1 Introduction
2 Program Technical Definition
   2.1 Requirements Development
   2.2 Architectures and Interface Control
   2.3 Specialty Engineering
   2.4 Modeling Strategy
   2.5 Design Considerations
   2.6 Technical Certifications
3 Program Technical Management
   3.1 Technical Planning
         3.1.1 Technical Schedule
         3.1.2 Maturity Assessment Planning
         3.1.3 Technical Structure and Organization
   3.2 Technical Tracking
         3.2.1 Technical Risk, Issue, and Opportunity Management
         3.2.2 Technical Performance Measures
         3.2.3 Reliability and Maintainability Engineering
         3.2.4 Manufacturing and Quality Engineering
         3.2.5 Human Systems Engineering
         3.2.6 System Safety
         3.2.7 Corrosion Prevention and Control
         3.2.8 Software Engineering
         3.2.9 Technology Insertion and Refresh
         3.2.10 Configuration and Change Management
         3.2.11 Technical Data Management
         3.2.12 System Security Engineering
         3.2.13 Technical Reviews, Audits and Activities
Appendix A - Acronyms
Appendix B - Item Unique Identification Implementation Plan
Appendix C - Agile and Development Security and Operations Software Development Metrics
Appendix D - Concept of Operations Description
Appendix E - Digital Engineering Implementation Plan
References

SEP Guidance and other Resources: 

DAU Systems Engineering Plan training videos:

  • Several videos are available via a web search 
Systems Engineering Resources and Guidance
Systems Engineering Tools and Checklist 
A Comprehensive List of Tools to Aid the Engineering Community 

This document identifies tools that could be used to help manage DoD acquisition technical, business, and management processes to include many manufacturing and quality activities. Most of these tools support systems engineering technical and technical management processes, but a few can be used to support business processes such as cost estimating, contract language, acquisition strategies, etc. Some tools could be used by a contractor, some by government personnel and some tools can bd used by many different people in many functional specialties. Most tools are available from multiple on-line sources, some tools may need to be purchased to use, many are free. 

Note: You can find more information on the tools listed below with a search of the web. 

Tools (listed in alphabetical order include):

  • 3Ps - Production Preparation Process
  • 5S’s (Sort, Straighten, Shine, Standardize, Sustain
  • 5-Whys
  • 7 - Basic Tools for Quality Improvement (includes the following which are also discussed separately):
    • Cause and Effect Diagram
    • Check Sheet
    • Control Chart
    • Histogram
    • Pareto Chart
    • Run Chart
    • Scatter Diagram
    • Stratification or Flowchart or 
  • 7 - Management and Planning Tools, or Advanced Tools for Quality Improvement (includes the following which are also discussed separately):
    • Affinity Diagram
    • Relations Diagram or Interrelationship Diagraph
    • Tree Diagram
    • Matrix Diagram
    • Matrix Data Analysis
    • Arrow Diagram
    • Process Decision Program Chart (PDPC)
  • 8D/PSP (Eight Disciplines/Problem Solving Process)
  • A3 Problem Solving Chart
  • Acceptable Quality Levels (AQL)
  • Acceptance Sampling 
  • Active Risk Manager (ARM)
  • Affinity Diagram
  • Advanced Product Quality Planning (APQP) Core Tools
  • Arrow Diagram (Chart)
  • AS6500 Manufacturing Management System (MMS)
  • AS9100 Advanced Quality Management Systems (QMS)
  • AS9103 Variation Management of Key Characteristics
  • AS9110 Maintenance
  • AS9120 Distributors
  • AS9102 FAI
  • AS9115 Software QA
  • AS9131 N/C Document
  • AS9132 Marking
  • AS9133 Supplier QA
  • AS 9137 AQAP Align
  • AS 9138 Statistical Process Control
  • AS9162 Self Verification
  • Axiomatic Design 
  • Balanced Scorecard
  • Baldrige Performance Excellence Criteria
  • Bekidou Rate
  • Benchmarking
  • Bill of Materials (BOM) 
  • Bone Diagram
  • Bottleneck Analysis
  • Box and Whisker Plot
  • Bubble Chart
  • Capacity Matrix
  • Capacity Analysis
  • Cause and Effect Diagram (Fishbone or IshIkawa)
  • Cause and Effect Matrix
  • CFMEA – Concept Failure, Mode and Effect Analysis
  • Check Sheet
  • Chokko Rate
  • Computer-Aided Design (CAD)
  • Computer-Aided Manufacturing (CAM)
  • Computer Aided Process Planning (CAPP)
  • Computer-Aided Three-Dimensional Interactive Application (CATIA)
  • Computer Integrated Manufacturing (CIM)
  • Consensogram
  • Contingency Planning
  • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart -U-chart for Attribute Data
    • Control Chart - NP-chart for Attribute Data
    • Control Chart - P-chart for Attribute Data
    • Control Chart - X-barR chart for Variable Data (measurable)
    • Control Chart - X-bar-S chart for Variable Data
    • Control Chart - X-MR/I-MR chart for Variable Data
  • Correlation Chart (Scatter Diagram)
  • Cost/Benefit Analysis
  • Cost of Quality Analysis
  • Cost of Quality (COQ)
  • Cost Modeling (Estimating)
  • Criteria Testing
  • Critical Chain Project Management 
  • Critical Design Review Checklist (DoD)
  • Critical Path/PERT
  • Critical to the Customer (CTC)
  • Critical to Quality (CTQ) Tree
  • Customer Contingency Table
  • DCOV – Define, Characterize, Optimize and Verify
  • Deming Cycle or Wheel (PDCA)
  • Departmental Purpose Analysis
  • Design of Experiments (DoE)
  • DFMEA - Design Failure Mode and Effects Analysis 
  • DFMA - Design for Manufacturing and Assembly 
  • Design to Cost (DTC)
  • DFSS - Design for Six Sigma 
  • DMAIC - Define, Measure, Analyze, Improve and Control
  • DMADV (see DCOV)
  • Domainal Mapping
    • Factory Modeling and Simulation
    • Producibility Analysis & Ergonomics
    • Process Planning 
    • Production Planning & Scheduling
    • Line Balancing & Bottleneck Analysis
    • Capacity Planning
    • Predictive Analytics & Optimization
    • Facility Planning, Layout and Design
    • Virtual Factory Mock-up
  • Failure Mode and Effects Analysis (FMEA)
  • Fault Tree Analysis 
  • First Article Inspection
  • First Article Testing 
  • Flow Chart or Process Flow Chart
  • Force Field Analysis
  • Gage R&R Studies
  • Gantt Chart
  • Histogram (Frequency or Bar Chart)
  • Hoshin Kanri (Quality Policy Deployment) 
  • Interrelationship Diagraph (also see Relations Diagram, or Network Diagram)
  • ISO 9001 Quality Management Systems (QMS) 
  • Kano Model
  • KJ model - Kawalota Jiro (see affinity diagram)
  • Lead Time Analysis
  • Learning Curve 
  • Learning Curve Analysis
  • Line of Balance (LOB)
  • Taguchi Loss Function
  • Lotus Diagram
  • Manufacturing Cost Estimating
  • Manufacturing Plan 
  • Manufacturing Readiness Assessment (MRA)
  • Manufacturing Readiness Level (MRL) Criteria
  • Matrix Diagram
  • Matrix Data Analysis Diagram
  • Measurement Systems Analysis (MSA) 
  • MIL-HDBK-896A Manufacturing and Quality Program
  • Multi-Vari Charts
  • Nominal Group Technique
  • One Piece Flow
  • Operations Process Chart
  • Overall Equipment Effectiveness (OEE)
  • Pareto Charts (Template)
  • Part-Family Analysis
  • Paynter Chart
  • P-Diagram or Parameter Design 
  • PERT Chart (Program Evaluation Routine Technique)
  • PFMEA – Process Failure, Mode and Effect Analysis
  • Pie Chart
  • Preliminary Design Review Checklist (DoD) 
  • Preliminary Hazards List (PHL)
  • Process Capability Studies (Cp and Cpk)
  • Process Performance Studies (Pp and Ppk)
  • Process Decision Program Chart (PDPC)
  • Producibility Analysis/Assessments
  • Producibility Assessment Worksheet
  • Producibility Engineering and Planning (PEP) Program
  • Production Part Approval Process
  • Production Readiness Review (PRR) Checklist
  • Programmatic Evaluation of ESOH (PESHE)
  • Pugh Matrix
  • Quadrant Chart
  • Quality Function Deployment (QFD)
  • Queuing Theory/Waiting Line Analysis
  • Radar Chart
  • Rational DOORS
  • Relation Diagram
  • Reliability Growth Analysis
  • Requirements Verification (Traceability) Matrix (RVM)
  • Risk Management Assessment Tool 
  • Route Sheet
  • Run Chart
  • Scatter Diagram (Mind Mapping)
  • SIPOC – System, Input, Process, Output and Customer
  • Six Sigma
  • SMART – Specific, Measurable, Attainable, Resources, Time
  • Spaghetti Diagram
  • Spider Diagram
  • Stratification
  • Statistical Process Control (SPC)
  • Supply Chain Management Risk Assessment
  • Swim Lane Chart (sometimes called a Deployment Flow Chart)
  • SWOT Model (Strength, Weaknesses, Opportunities and Threats
  • Systems Engineering Plan (SEP) 
  • Takt Time Analysis
  • Technical Risk Identification and Mitigation System (TRIMS)
  • Technology Readiness Level (TRL) Checklist
  • Theory of Constraints Analysis
  • Throughput Analysis Tool
  • Throughput Accounting
  • Tolerance Analysis 
  • Tolerance Design 
  • Total Productive Maintenance (TPM) 
  • Trade Studies/Analysis
  • Transition to Production (Willoughby Templates)
  • Tree Diagram
  • TRIZ Matrix 
  • Value Stream Mapping (VSM) 
  • Variability Reduction Program
  • Venn Diagram
  • Work Center
  • Work Measurement
  • X-Matrix
  • Yamazumi Chart

Note: You can find a lot of additional information by googling the tool, by visiting a number of different academic sites, or professional organizations, or by visiting various Communities of Practice (CoPs).

Quality of Design / Quality by Design / Quality Engineering 

Quality By Design – Based on Juran Institute Processes

Quality by Design (Quality of Design or Concurrent Engineering) is the process of creating a design using multidisciplinary teams (IPTs) to conduct conceptual thinking, product design, and production planning all at one time. Quality by Design was developed by Dr. J. M. Juran who believed that quality should be built into a product from the beginning, and that most quality issues are related to the product's original design. 

Quality by Design differs from traditional sequential engineering practices and uses a five step process of Define, Discover, Design, Develop, and Deliver. Each process step has a focus and utilizes a variety of advanced quality and technical tools to meet customer requirements (satisfies the customer) often focusing on the eight dimensions of quality to create a design that is optimized for performance, cost and schedule. Quality by Design should require input from many different technical teams to include manufacturing and quality. It is the job of systems/design engineers to create the design, and it is the role of these technical personnel is to “influence the design” for producibility, manufacturability, reliability and maintainability, testability, and sustainability.

Define

Describe in general terms what the product is and what set of customers it is intended to serve. Establish the team, identify the customers and stakeholders, set goals, and create plans:

  • Operational Needs and Requirements
  • Sustainment Requirements
  • IOC Date
  • Weapon System Performance 
  • Cost (Procurement and Sustainment)
Discover

Discover the exact needs of the customer expressed in terms of the benefit to the customer. Collect and prioritize customer needs and translate those needs and create:

 

  • Work Breakdown Structure (WBS)
  • Functional Allocation 
  • Allocated Baseline
  • Measures of Effectiveness (MoE)
  • Key Performance Parameters (KPP) 
  • Measures of Performance (MoP)
  • Technical Performance Measures (TPM)
  • “Critical to Quality” measures. 

 

Develop planning worksheets for:

 

  • Customer requirements
  • Allocate Functions and Features
  • Allocate Measures and Goals (MoE, KPP, MoP, TPM and Critical to Quality
  • Detailed design features and goals 
  • Manufacturing Process features and goals 
  • Manufacturing Control features and goals

 
Design

Design a product that will meet those needs better than competitors’ and preceding products. Establish high-level product features and goals: (functional design), then develop detailed product features and goals (detailed design), optimize design features, set and approve final design.

 

  • Product Baseline
  • Functional
  • Allocated Design
  • Detail Design 
  • Design reviews and approvals
  • Identify and manage key and critical characteristics
  • Producibility Engineering
  • Trade Studies and Trade-off Analysis
  • Design Reviews
  • Above activities may require the use of the following advanced engineering techniques:
    • Design of Experiments 
    • Quality Function Deployment
    • Design for Manufacturing and Assembly (DFMA)
    • Design for X
    • Parameter Desing 

    • Tolerance Design 

       
Develop

Produce or manufacture the product by identifying, managing, controlling manufacturing processes and control.

 

  • Create manufacturing process planning (route sheets, flow diagrams, etc.)
  • Create assembly charts and operations process charts
  • Understand and control manufacturing processes:
    • Set goals for processes
    • Develop control plans
    • Establish audit procedures
    • Capture feedback and continuously improve
  • Manage capacity, identify bottlenecks:
    • Theory of Constraints
    • Optimize throughput
  • Optimize processes: 
    • Lean
    • Six Sigma
    • Standard Stable Processes
    • Identify and manage critical process parameters
    • Demonstrate process capability
    • Optimize process capability 
  • Manage Quality: 
    • Establish a Quality Management System
    • Identify critical quality attributes
    • Control variation
    • Statistical Process Controls
  • Manage Supply Chain 
Deliver

Plan for the transfer of the product to the customer, then measure and manage customer satisfaction.

 

  • Conforms to requirements
  • Performs as expected
  • Reliable product
  • On-time
  • Defect free
Technical Reviews and Audits

For DoD systems development, a properly tailored series of technical reviews and audits provide key points throughout the system development to evaluate significant achievements and assess technical maturity and risk. Technical reviews of program progress should be event driven and conducted when the system under development meets the review entrance criteria as documented in the SEP. An associated activity is to identify technical risks associated with achieving entrance criteria at each of these points. SE is an event-driven process based on successful completion of key events as opposed to arbitrary calendar dates. As such, the SEP should clarify the timing of events in relation to other SE and program events.

Technical reviews of program progress should be event driven and conducted when the system under development meets the review entrance criteria as documented in the program’s Systems Engineering Plan (SEP). An associated activity is to identify technical risks associated with achieving entrance criteria at each of these points (see the DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs). Systems Engineering (SE) is an event-driven process based on successful completion of key events as opposed to arbitrary calendar dates. As such, the SEP should clarify the timing of events in relation to other SE and program events. While the initial SEP and Integrated Master Schedule (IMS) have the expected occurrence in the time of various milestones (such as overall system Critical Design Review (CDR)), the plan should be updated to reflect changes to the actual timing of SE activities, reviews and decisions. Figure 3-1 of the SE Guidebook provides the end-to-end perspective and the integration of SE technical reviews and audits across all AAF pathways. Technical reviews should be tailored appropriately for other acquisition pathway.

The DoD Systems Engineering Guidebook provides guidance on support of the following technical reviews and audits https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

  • Alternative Systems Review (ASR): Is a technical review that assesses the preliminary materiel solutions that have been developed during the MSA Phase. The review ensures that one or more proposed materiel solution(s) have the best potential to be cost-effective, affordable, operationally effective, and suitable, and can be developed to provide a timely solution at an acceptable level of risk to satisfy the capabilities listed in an ICD.  The ASR helps the PM and Systems Engineer ensure that further engineering and technical analysis needed to draft the system performance specification is consistent with customer needs. https://acqnotes.com/acqnote/tasks/alternative-systems-review-2 
  • System Requirements Review (SRR): Is a formal review conducted to ensure that system requirements have been completely and properly identified and that a mutual understanding exists between the government and the contractor. It ensures that the system under review can proceed into initial systems development and that all system and performance requirements derived from the ICD or draft CDD are defined and testable, and are consistent with cost, schedule, risks, technology readiness, and other system constraints. https://acqnotes.com/acqnote/acquisitions/system-requirements-review-srr 
  • System Functional Review (SFR): Is a technical review to ensure that the system’s functional baseline is established and can satisfy the requirements of the ICD or draft CDD within the currently allocated budget and schedule. It also determines whether the system’s lower-level performance requirements are fully defined and consistent with the system concept and whether lower-level systems requirements trace to top-level system performance requirements. https://acqnotes.com/acqnote/acquisitions/system-functional-review 
  • Preliminary Design Review (PDR): Is a technical assessment that establishes the Allocated Baseline of a system to ensure a system is operationally effective.  A PDR is conducted before the start of detailed design and is the first opportunity for the Government to observe the Contractor’s hardware and software designs.  This review assesses the allocated design documented in subsystem product specifications for each configuration item in the system. It ensures that each function in the Functional Baseline has been allocated to one or more system configuration items. A PDR is required by statute for all Major Defense Acquisition Programs (MDAPs). https://acqnotes.com/acqnote/acquisitions/preliminary-design-review 
  • Critical Design Review (CDR): Is a multi-disciplined independent technical assessment to ensure that a system can proceed into fabrication, demonstration, and test and meet stated performance requirements within cost, schedule, and risk.  A successful CDR is predicated upon a determination that the detailed design satisfies the CDD.  Multiple CDRs may be held for key Configuration Items (CI) and/or at each subsystem level, culminating in a system-level CDR. https://acqnotes.com/acqnote/acquisitions/critical-design-review 
  • System Verification Review (SVR): Is a product and process assessment to ensure the system under review can proceed into LRIP and FRP within cost, schedule, risk, and other system constraints during the EMD Phase. It assesses the system functionality and determines if it meets the functional requirements in the CDD documented in the functional baseline. The SVR establishes and verifies final product performance and provides inputs to the CPD. https://acqnotes.com/acqnote/acquisitions/system-verification-review-svr#google_vignette 
  • Functional Configuration Audit (FCA): Examines the functional characteristics of the configured product. It verifies that the product has met the requirements specified in its Functional Baseline documentation approved at the PDR and CDR.  The FCA reviews the configuration item’s test and analysis data to validate that the intended function meets the system performance specification. The audit is more systems engineering-focused than program management official auditing. https://acqnotes.com/acqnote/tasks/functional-configuration-audit-2 
  • Production Readiness Review (PRR): Assesses a program to determine if the design is ready for production. It evaluates if the prime contractor and major subcontractors have accomplished adequate production planning without incurring unacceptable risks that will breach thresholds of schedule, performance, cost, or other established criteria. https://acqnotes.com/acqnote/acquisitions/production-readiness-review 
  • Physical Configuration Audit (PCA): Is a formal technical review that determines if the configuration of a system or item has met its documented requirements to establish a product baseline. The Milestone Decision Authority (MDA) receives proof from a successful PCA that the product design is stable, the capability satisfies end-user needs, and the production risks are tolerably low. https://acqnotes.com/acqnote/tasks/physical-configuration-auditaudit 
  • Independent Technical Risk Assessment (ITRA): is a formal review that is independent of the program office and is conducted in advance of milestone and production decisions in order to provide senior leaders with an independent review of a programs technical risks, including the maturity of critical technologies and manufacturing processes,

DAU Continuous Training Modules and other training:

Guidance and other resources:

Digital Engineering

Digital Engineering (DE), Digital Twin, Digital Threads, and Digital Models: 

Digital Engineering: Digital Engineering can be used to support all of the Systems Engineering functions. MIL-HDBK-539 Digital Engineering and Modeling Practices defines digital engineering as “an integrated, computation-based approach that uses authoritative sources of system data and models as a continuum across disciplines to support lifecycle activities.” DE is a cutting-edge approach that uses authoritative sources of system data and models throughout the development and life of a system. Digital engineering harnesses computational technology, modeling, analytics, and data sciences to update traditional systems engineering practices. In the face of increasing global challenges and dynamic threat environments, digital engineering is a necessary practice to support acquisition.

DoDI 5000.97-Digital Engineering calls for the use of digital engineering methodologies, technologies and practices across the life cycle of defense acquisition programs.  Further, the document:

  •  Mandates the incorporation of digital engineering for all new programs (exceptions can be granted by decision authority)
  •  Directs Components to use digital engineering practices in requirements, cost, business and sustainment.
  • Calls for the replacement of documents with the use of digital models as the primary means of communicating system information. 
  • During program planning and contracting, requires that appropriate data rights be obtained.
  • Singles out DAU as providing workforce training on digital engineering.

The diagram below, from the instruction, captures the digital Engineering framework:

The figure describes the DoD Digital Engineering Ecosystem

Digital Twin: Every product produced, and every process executed is unique. There can be thousands, of key input variables used to describe products, assets, entire lines, and processes.  A Digital Twin is a virtual replica of a physical object, manufacturing process, or plant that is designed to capture, map, and structure process variables into a continuously updated database. This database can be used by an organization to monitor, analyze, design, or optimize the process without having to go out into the field or do costly trials on the physical equipment. By making this data more readily available in a digital environment, teams can use data in other applications, models, or third-party programs to make meaningful discoveries.

Digital Thread: A digital thread is a framework that connects data about a product throughout its lifecycle, from design to disposal. It uses a variety of technologies, including computer-aided design (CAD), product lifecycle management (PLM), and Internet of Things (IoT) sensors, to collect and share data. A manufacturing digital thread is designed to expand upon a digital twin. Put simply, the digital thread captures digital twin data as they evolve. As manufacturers evolve their processes and their digital twins adapt to new settings or recipe changes, the digital thread encapsulates the link between these evolutions. 

The digital thread should seamlessly advance the controlled interplay of technical data, software, information, and knowledge in the digital engineering ecosystem. Digital threads are used to connect authoritative data and orchestrate digital models and information across a system’s life cycle. The digital thread informs decision makers throughout a system’s life cycle by providing the capability to access, integrate, and transform data into actionable information. The digital thread should support feedback over the life cycle.

Example Data Elements or Digital Artifacts

Engineering Design Data

Technical Product Data

Manufacturing and Quality Data

Enterprise Data
  • Producibility Analysis
  • CAD Data
  • Modeling and Simulation Data
  • Special Tooling Data
  • Special Inspection Equipment Data
  • Digital Drawings
  • Digital Models
  • Specifications
  • Standards
  • Critical Manufacturing Processes
  • Configuration Data
  • Interface Management Data
  • Analytical Data
  • Bill of Material
  • Manufacturing Floor Layout
  • Production Line Data
  • Pilot Line Data
  • LRIP/FRP Data
  • Industrial Engineering Data
  • Production Data 
  • Machining Instructions
  • Customer Demand Data
  • Rates and Quantities Data
  • Work Breakdown Structure Data
  • Supplier Data
  • FRACAS Data 
  • Test Plans
  • Schedules
  • Product Support Strategy

Digital Models Include:

  • Requirements Model
  • Structural Model
  • Functional Model
  • Architecture Model
  • Business Process Model 
  • Enterprise Model
  • Human Performance Model
  • Product Life Cycle Model 

DAU Continuous Learning Modules and other training: 

Guidance and other Resources:

Digital Engineering Body of Knowledge (DEBoK) 

The Digital Engineering Body of Knowledge (DEBoK) will serve as a reference for the DoD engineering community to use in implementing digital engineering practices starting with systems engineering and expanding to specific disciplines, engineering domains and specialty areas. The BoK will store collective data, information and knowledge on digital engineering. Members of the government, industry and academia working within this space will be able to contribute to the DEBoK and build their digital engineering solutions based on collective knowledge. Access the DoD DE BoK briefing at https://ndiastorage.blob.core.usgovcloudapi.net/ndia/2021/systems/Wed_23770_Zimmerman_Davidson_Salvatore.pdf 

As a best practice, when conducting early M&Q engineering analysis, the technical team should consider DE principles, methods, and tools. DE best practices and tools are defined in the DE Body of Knowledge (DEBoK). https://de-bok.org/ 

The DEBoK is also available to DoD Common Access Card users at the Defense Technical information Center (DTIC) website: 

https://www.dodtechipedia.mil/dodwiki/pages/viewpage.action?page Id=760447627 

Digital Engineering Strategy and other Resources 

DoD’s Digital Engineering Strategy provides guiding principles and promotes consistency in engineering processes through the use and reuse of digital tools, models, and curated data throughout the program’s life cycle. As a best practice, the technical team should consider M&Q digital data requirements (e.g. factory floor modeling and simulation, digital technical data packages and work instructions, digital data in supply chains) during early establishment and development of the digital thread.

Digital Engineering: An integrated digital approach that uses authoritative sources of systems’ data and models as a continuum across disciplines to support lifecycle activities from concept through disposal. Access to Digital Engineering Fundamentals can be found at the following urls:

https://www.cto.mil/wp-content/uploads/2023/06/Dig-Eng-Fundamentals-2022.pdf 

https://ac.cto.mil/wp-content/uploads/2019/06/DE-Fundamentals.pdf 

Digital Engineering Ecosystem: The interconnected infrastructure, environment, and methodology (process, methods, and tools) used to store, access, analyze, and visualize evolving systems’ data and models to address the needs of the stakeholders. End-to-end digital enterprise. 

Digital Artifact: An artifact produced within, or generated from, the digital engineering ecosystem. These artifacts provide data for alternative views to visualize, communicate, and deliver data, information, and knowledge to stakeholders.

Guidance and other Resources:

Modeling and Simulation (M&S) 

A model is a physical, mathematical, or logical representation of a system, entity, phenomenon, or process. Manufacturing models include plant diagrams, flow charts, 5Ms chart, 

A simulation is the implementation of a model over time, showing how the model works, and can be live, virtual, or constructive. Manufacturing simulations include 

The use of models and simulations in engineering is well recognized. Simulation technology is an essential tool for engineers in all application domains. A digital model represents an actual or conceptual system that involves physics, mathematics, or logical expressions. A simulation is a method for implementing a model over time. Together models and simulations allow the Department to vet potential requirements prior to the Request for Proposal release, assess engineering change orders or program upgrades, etc. M&S can be used to assess and optimize resource usage, examine process changes, support supply-chain management routing and inventory quantities, business decisions, etc.

Models and simulations are SE tools used by multiple functional area disciplines. Models, simulations, data, and other artifacts should be developed and used in a well-defined and controlled engineering ecosystem to support an effort’s reuse of the information across the life cycle of activities. Models, simulations, data, and artifacts should be integrated, managed, and controlled to ensure that the products maintain consistency with the system and external dependencies and provide a comprehensive view of the effort and increase efficiency and confidence throughout the project’s life span.

Systems Engineering process related tools:

Note: Each of these technical and technical management processes have commercial software tools that can be used to support these processes. A web search starting with Model-Based Systems Engineering (MBSE) tools should provide many links.

Systems Engineering Technical Processes  Tool Capabilities and Features
Stakeholder Requirements Definition
  • Assists in capturing and identifying stakeholder requirements
  • Assists in analyzing and maintaining stakeholder requirements
Requirements Analysis
  • Assists in requirements definition and decomposition 
  • Interfaces with architecting tools 
  • Supports requirements validation
Architecture Design
  • Assists in development of functional and physical architectures 
  • Provides traceability among system elements 
  • Supports multiple views
Implementation 
  • Assists in development of the system design, prototypes and alternate solutions 
  • Assists in realization of the system, system elements and enabling system elements
Integration 
  • Assists in integration-planning activities 
  • Assists in assembling lower-level system elements into successively higher-level system elements 
  • Provides analysis and simulation capability
Verification
  • Assists in determining the system and system elements performance as designed through demonstration, examination, analysis and test
Validation
  • Assists in determining, the effectiveness, suitability and survivability of the system in meeting end-user needs
Transition 
  • Assists in planning and executing delivery and deploying of the system to the end user for use in the operational environment
Systems Engineering Technical Management Processes  
Decision Analysis
  • Assists in trade-off analysis  Provides optimization and sensitivity analysis capability 
  • Assists in recording, tracking, evaluating, and reporting decision outcomes
Technical Planning 
  • Assists in planning and scheduling activities 
  • Assists in resource planning, tracking, and allocation  Facilitates cost estimation
Technical Assessment 
  • Assists in tracking, measuring, and assessing metrics 
  • Assists in metric collection
Requirements Management Provides requirements bi-directional traceability capability  Provides requirements flow-down capability  Tracks requirements changes
Risk Management
  • Assists in risk, issue, and opportunity planning, identification, analysis, mitigation/management and monitoring
Configuration Management
  • Assists in the identification of configuration items 
  • Assists in baseline/version control of all configuration items 
  • Assists in ensuring configuration baselines and changes are identified, recorded, evaluated, approved, incorporated and verified
Technical Data Management
  • Assists in identification of data requirements 
  • Assists in recording and managing data rights 
  • Assists in storage, maintenance, control, use and exchange of data including digital artifacts 
  • Assists in document preparation, update, and analysis
Interface Management 
  • Assists in capturing system internal and external interfaces and their requirement specifications 
  • Assists in assessing compliance of interfaces among system elements of the system or systems of systems 
  • Produces a view of interface connectivity
   

MBSE Software Tools:

  • Requirements Management software provides a single, centralized platform to store, organize, and manage requirements, which enables better collaboration and communication among team members and stakeholders. Traceability provides end-to-end traceability between requirements, system elements, and their associated models, which ensures consistency throughout the development process and simplifies change management. Examples include Visure Requirements Platform, Siemens Teamcenter, and Sparx Systems Enterprise Architect to name a few. Listing these tools here is not an endorsement of these tools.
  • Product Data Management (PDM) software manages design and engineering files such as CAD models and manufacturing instructions allowing teams to collaborate across concurrent design environments. PDM is mostly used by manufacturing companies to control product data from design to production. This type of software is beneficial for designers creating the initial specifications of a new product and production managers following manufacturing instructions. There are approximately 57 PDM software tool available to engineers. Examples include Autodesk Vault, Teamcenter, and Solidworks to name a few. Listing these tools here is not an endorsement of these tools.
  • Product Lifecycle Management (PLM) software manages all of the information and processes at every step of a product or service lifecycle across globalized supply chains. Today’s PLM software provides the foundation and intersection of critical, cradle-to-grave product lifecycle processes woven with real-time data from technologies, such as the Internet of Things (IoT), artificial intelligence (AI), and machine learning (ML). Global organizations are leveraging what emerged as a “digital thread” to change how they design, manufacture, and service products. There are dozens of PLM software tools available to engineers. Examples include SAP PLM, Oracle Agile, and Aras PLM to name a few. Listing these tools here is not an endorsement of these tools.
  • Ergonomic Design and Simulation software allows production engineers to design workstations and plant features while focusing on the man-machine interface in order to ensure safety of the worker, while providing for comfort, ease of use, productivity and performance. Examples include Delmia Ergonomic Workstations, Semins Human-centered design and simulation, ERGOMIX, and others. Listing these tools here is not an endorsement of these tools.
  • Producibility Analysis software allows design engineers, along with other technical personnel, to design product that promotes the ease of fabrication and assembly thus reducing production time, while increasing reliability. See DFMA for more information. Examples include Solidworks DFMXpress, DFMPro, DFMA software by Boothroyd Dewhurst, and others.  Listing these tools here is not an endorsement of these tools.
  • Validation and Verification Support software supports the validation and verification of requirements by linking them to test cases, test results, and other verification artifacts, ensuring that the system meets its intended purpose and satisfies stakeholder needs. Examples include Visure, ANSYS SCADE Suite, Simulink, and others. Listing these tools here is not an endorsement of these tools.
  • Change Management software provides efficient change management features such as version control, change tracking, and impact analysis, helping teams manage changes to requirements and their corresponding models effectively. Examples include Visure, Siemens Teamcenter, Topcased, and others. Listing these tools here is not an endorsement of these tools. 

    • Note: There are many software vendors that offer these tools that are mostly used by government contractors.  You can find more information on these software tools through a web search.

DAU Continuous Learning Modules and other training:

Guidance and other Resources:

M&S and DE Guidance and other Resources 
Producibility Engineering 

Producibility can be defined as “the measure of the relative ease of manufacturing.” That is, you can manufacture a part out of inexpensive material, using unskilled workers, simple tools, and manufacture it in a very short time. 

The terms "producibility" and "manufacturability" are often used interchangeably. The DoD Producibility and Manufacturability Engineering Guide distinguishes between producibility and manufacturability as distinct but complimentary and sometimes overlapping concepts.

Producibility is a "design" consideration to facilitate the ease of manufacture, that is, designing a product in a way so it is relatively easy to manufacture. That includes any technique that helps to improve the designs efficient and ability to be produced (see producibility tools below). Development teams should consider producibility during system development and design following detailed design guidelines and producibility principles. 

Manufacturability is a "factory floor or manufacturing operations," consideration used to enhance the ease of manufacture by developing and implementing efficient manufacturing processes. This includes and best practices like the use of Lean/Six Sigma, Theory of Constraints, Process Failure Modes and Effects Analysis, continuous process improvement, and others.

Note: Producibility Analysis is a requirement of MIL-HDBK-896, para. 6.2.1, and is addressed in the DoD Systems Engineering Guidebook. See Producibility Best Practices for more information on Producibility Engineering. 

Listed below are some things to consider during the design process: 

Producibility Resources

Producibility Tools (most will require a search of the web as there are no DoD links for these and there are dozens of links for each of the tools listed).

  • Design for Manufacturing (DFM)
  • Design for Assembly (DFA)
  • Design for Manufacturing and Assembly (DFMA) 
  • Process Failure Modes Effects Analysis (PFMEA)
  • Design for Ergonomics (DFE): 
  • Design for Reliability (DFR): 
  • Design for Maintainability (DFM): 
  • Design for Sustainability (DFS): 
  • Design for Quality (DFQ): 
  • Design for Supply Chain: 
  • Design for Safety (DFS): 
  • Producibility Assessment Worksheet 

The graphic below illustrated the improvements from producibility initiatives.

Note: You can find more information on these producibility tools through a web search.

DAU Continuous Learning Modules and training:

Guidance and other Resources:

Key Characteristics 

A key characteristic is a feature whose variation has the greatest impact on the fit, performance (function), or service life of the finished product from the perspective of the customer. In other words, it is a product or process characteristic that if you deviate from the target value, there is a high loss function the further from the target value you get, and it will cost you (see graphic below). Most characteristics have a low loss function and thus do not need the same management attention as does a Key or Critical characteristic. Thus, when you deviate for the target value on a characteristic that has a low loss function, there is not a significant impact to fit, function or service life.  However, Key Characteristics are the vital few characteristics that must be identified and managed in order to avoid this high loss function. 

Note: A KC can be either a product, process, or service KC. 

The management of KCs includes:

  • Identifying product characteristics of the design which most influence fit, performance or reliability
    • This will require the use of one or more of the approaches listed below
  • Supporting the mapping of product characteristics to production processes
  • Enabling the balancing of product design requirements with manufacturing process capabilities 
  • Enabling the development of the required process controls for production.

Engineers have used a wide variety of tools or approaches for identifying KCs with the identification process beginning in the design stage. There are objective and subjective approaches that may be used to help identify and manage KCs to include:

  • Quality Function Deployment (QFD)
  • Taguchi experimentation
  • Capability Analysis
  • Design Failure Mode and Effects Analysis (DFMEA)
  • Process Failure Mode and Effects Analysis (PFMEA)
  • Statistical Process Control (SPC)
  • Design Verification
  • Control Plans
  • Statistical analysis of yield 
  • Process Flow Charts
  • Advanced Product Quality Planning (APQP)
  • Production Part Approval Process (PPAP)
  • Field Reporting and Corrective Action System (FRACAS)
  • Reliability data from similar products.

Watch the Ford Batavia Transmission Quality study at the url listed below for an excellent example of a company that was facing a serious manufacturing problem with building transmissions that were reliable, and Ford used some of the tools listed above to identify and manage key characteristics and get control of their manufacturing problems that helped them to achieve cost, schedule and reliability goals. What Ford found was that out of thousands of measurement characteristics, only four were significant to achieving their manufacturing goals. This profound knowledge then allowed them to control their processes. Note that the film is quite old but one of the best examples available. 

The Batavia Movie https://www.youtube.com/watch?v=uAfUOfSY-S0 

NOTE: Technical personnel should also control the quality of parts designated as Critical Safety Items (CSIs) or Critical Application Items. In addition to managing KCs technical personnel need to manage special characteristics which AIAG divides into two categories critical characteristics and significant characteristics. You can find more information on these tools through a web search.

  • Special Characteristics: Product characteristics or manufacturing process parameters that can affect safety or compliance with regulations, fit, function, or performance.
  • Significant Characteristics: Characteristics that are important to the customer or final client, but do not have a direct impact on the safety, performance or functionality of the product, but can still affect customer satisfaction.
  • Critical Characteristics: Special characteristics that are crucial for the safety, performance, or functionality of the product. These characteristics have a direct impact on quality, reliability, and safety. If critical characteristics are not identified and managed, they could result in serious consequences. 
  • Source: IATF 16949:2016 clause 8.3.3.3 Special Characteristics 
  • These characteristics can be identified and managed using the tools and approaches listed above.

Below is a notional chart depicting the activities that should be accomplished during each planning and execution phase of development and production.

Key Characteristics Guidance can be found in: 

Producibility Best Practices 

Producibility analysis. Producibility should be considered as a part of design trade studies. The role of design trade studies in the manufacturing development process is to achieve a product design that effectively balances the system design with cost, schedule and performance elements to minimize total program risk. Institutionalizing producibility as part of the design trade study process is essential to an overall goal of affordable weapon system acquisition. Another excellent source for information on producibility programs is the Navy’s NAVSO P-3687, “Producibility System Guidelines.” This guide recommends a 5-step process: 1. establish a producibility infrastructure, 2. determine process capabilities, 3. address producibility during conceptual design, 4. address producibility during detailed design, and 5. measure producibility.

Producibility Best Practice tools include: the following and may require a web search to gather more information:

  • Quality Function Deployment (QFD): A structured process and set of tools that can be used to tools used to effectively define customer requirements and convert them into detailed engineering specifications and plans to produce the products that fulfill those requirements. QFD is used to translate customer requirements (or VOC) into measurable design targets and drive them from the assembly level down through the sub-assembly, component and production process levels.
  • Concurrent Engineering (CE): I more of an approach to engineering than a specific tool. Concurrent engineering can be used to reduce product development time while reducing costs and improving quality and reliability by concurrently and systematically product design along with associated manufacturing, quality and other processes.
  • Integrated Product and Process Development (IPPD): Is a DoD management technique that simultaneously integrates all essential acquisition activities through the use of Integrated Product Teams (IPTs) to optimize design, manufacturing, and supportability processes. IPPD facilitates meeting cost and performance objectives from product concept through production, including field support. It evolved in industry as an outgrowth of efforts such as Concurrent Engineering to improve customer satisfaction and competitiveness in a global economy.
  • Integrated Product Teams (IPT): An Integrated Product Team (IPT): Is a team composed of representatives from appropriate functional disciplines working together to build successful programs, identify and resolve issues, and make sound and timely recommendations to facilitate decision-making. IPTs are used in complex development programs/projects for review and decision-making. The emphasis of the IPT is on the involvement of all Stakeholders (users, customers, management, developers, contractors) in a collaborative forum.
  • Taguchi/Robust Design/Parameter Design: Is s a powerful statistical method to produce high quality product and optimize the process design problems in a cost-efficient way by reducing process variation through robust design of experiments. An experimental design is used to identify and exploit the interactions between control and noise factors. Once the significant factors have been identified and their control settings established the resultant product will be optimized by designing quality into the product and processes.
  • Taguchi Loss Function: Is a graphical technique to show how an increase in variation from the target value, on key characteristics, can have an exponential impact on cost, reliability and customer dissatisfaction. Traditional quality looks at product quality as either good or bad, that is it either meets the spec or does not. While this may be true for many characteristics, it is not true for key characteristics. 

  • Modeling and Simulation M&S): Manufacturing simulation is the use of computer modeling to virtually test manufacturing methods and procedures – including processes such as production, assembly, inventory, and transportation. Simulation software can be used to predict the performance of a planned manufacturing system and to compare solutions for any problems discovered in the system's design. This makes manufacturing simulation a significantly competitive capability - allowing manufacturers to test a range of scenarios before buying tooling, reserving capacity, or coordinating other expensive production resources. By using simulation software to determine exactly what is needed, the manufacturer can avoid problems during production while also reducing scrap and rework. Various types of Factory Modeling and Simulation tools currently available include, but are not limited to the following areas:

    •    Producibility Analysis and Ergonomics
    •    Process Planning
    •    Production Planning and Scheduling
    •    Line Balancing and Bottleneck Analysis
    •    Capacity Planning
    •    Predictive Analytics and Optimization
    •    Facility Planning, Layout and Design
    •    Virtual Factory Mock-up

  • Model Based Engineering (MBE): Uses annotated digital three-dimensional (3D) models of a product and relevant production equipment and processes as the authoritative information source for all activities in that product’s lifecycle including relevant production equipment and processes. MBE is an integral part of the technical baseline that evolves throughout the acquisition life cycle.
  • Model Based Systems Engineering (MBSE): Is a systems engineering methodology that focuses on creating and exploiting domain models as the primary means of information exchange between engineers, rather than on document-based information exchange. MBSE is generally defined as a formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases. MBSE uses models as an integral part of the technical baseline, which includes the requirements, analysis, design, implementation, and verification of a capability, system, and/or product throughout the acquisition life cycle
  • Computer Aided Design (CAD): Is the process of digitally creating design simulations of products in 2D or 3D, complete with scale, precision, and physics properties, to optimize and perfect the design – often in a collaborative manner – before manufacturing. The use of digital data allows various engineering functions to share, review, simulate, and edit technical data and allow organizations to introduce new product quickly.
  • Computer Aided Manufacturing (CAM): Involves the use of digital data, software and computer-controlled factory machinery to create products with a high quality by automating and optimizing manufacturing processes. CAM is used to either create new or improve upon existing manufacturing setups to boost efficiency and reduce wastage. It does so by expediting the manufacturing process and tooling and reducing energy requirements. The final results have a high degree of consistency, quality, and accuracy.
  • Note: You can find more information on these producibility tools through a web search.
CREATED:
BY:
Content Author No Reply
LAST MODIFIED:
BY:
Content Author No Reply
View Resource
Community Resource / Manufacturing and Quality
Quality Assurance and Control

Quality Assurance (QA) is a broad and organizational-wide system for managing and improving quality. Quality assurance includes the development and implementation of planned and systematic activities in a quality system to ensure that the quality requirements for a product or service are fulfilled. QA focuses on the entire quality system including suppliers and ultimate consumers of the product or service. It includes all activities designed to produce products and services of appropriate quality. QA begins before a product is made or before a project is even started.

Quality Control (QC) Is a subset of quality assurance and refers to the activities used during the production of a product that are designed to verify that the product meets the customer's requirement. QC focuses on the process of producing the product or service with the intent of eliminating problems that might result in defects. QC begins as the product is being produced.

This resource page will focus on the following topic areas: 

  • Quality Management Systems (QMS)
  • Product Quality Control
  • Process Capability and Control 
  • Supplier Quality Management 
  • Quality of Design / Quality Engineering 
  • Advanced Product Quality Planning
  • Measurement Systems Analysis (MSA)
  • Continuous Process Improvement (Leam/Six Sigma/TOC)
  • Quality Policy and Guidance
  • Quality Tools and Checklist
  • A Comprehensive List of Tools to Improve Manufacturing and Quality  

FAR 46.202 Types of Contract Quality Requirements: 

  • 46.202-1 Contracts for commercial items. The Government shall rely on contractors’ existing QA systems as a substitute for Government inspection and testing before tender for acceptance unless customary market practices for the commercial item includes in-process inspection.
  • 46.202-2 Government reliance on inspection by contractor. The Government shall rely on the contractor to accomplish all inspection and testing needed to ensure that supplies or services acquired conform to contract quality requirements before they are tendered to the Government. The Government shall not rely on contractor inspection if the Government has a need to test the supplies or services in advance of their tender for acceptance, or to pass judgment upon the adequacy of the contractor’s internal work processes.
  • 46.202-3 Standard inspection requirements. Requires the contractor to provide and maintain an inspection system that is acceptable to the Government;

     -  Give the Government the right to make inspections and tests while work is in process; and

     -  Require the contractor to keep complete, and make available to the Government, records of its inspection work.

  • 46.202-4 Higher-level contract quality requirements.

    (a) Requiring compliance with higher-level quality standards is appropriate in solicitations and contracts for complex or critical items or when the technical requirements of the contract require—

           (1) Control of such things as work operations, in-process controls, and inspection; or

           (2) Attention to such factors as organization, planning, work instructions, documentation control, and advanced metrology.

    (b) When the contracting officer finds it is in the Government’s interest to require that higher-level quality standards be maintained. The contracting officer shall indicate in the clause which higher-level quality standards will satisfy the Government’s requirement. Examples of higher-level quality standards are ISO 9001, 9002, or 9003; ANSI/ISO/ASQ Q9001-2000; ANSI/ASQC Q9001, Q9002, or Q9003; QS-9000; AS-9000; ANSI/ASQC E4; and ANSI/ASME NQA-1.

DFAR related Quality Clauses:

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

Note: DCMA has many quality related Continuous Learning Modules available in DAU's iCatalog

Quality Management System (QMS)

A Quality Management System (QMS) is about quality assurance. A QMS is a clearly defined set of processes and procedure that are formalized in documents that outline processes, procedures, and responsibilities for ensuring products or services consistently meet customer and regulatory requirements. Quality Assurance is about "how" a product is made and focuses on processes and building in quality and preventing defects before they happen. A Quality Management System is a requirement of ISO 9001 and AS9100.  ISO 9001 can be considered the baseline QMS while AS9100 is used for developing and advanced QMS. 

MIO-Q-9858 was the first standard that addressed an organizations QMS. This standard was 1st released in 1959 by the U.S. Department of Defense to cover U.S. defense contracts. This standard was later revised to become the NATO Allied Quality Assurance Publications (AQAP in 1969. Then in 1987 the Intenational Organization for Standards created the ISO 9001 series based on MIL-Q-9858 and NATO AQAP.

The ISO series of documents outlines the requirements for a quality management system:

  • ISO 900: Identifies the vocabulary and fundamentals for a QMS to include the seven quality management principles 
  • ISO 9001: Details the requirements organizations need to meet in order to meet the standard and become certified
  • ISO 9002: Provides for guidance for the implementation of ISO 9001
  • ISO 9004: Provides guidance for achieving sustained success through the application of a program of evaluation and continuous performance improvement 

ISO 9001 Quality Assurance Process Model:    

  • Requirements (Customers and Relevant Interested Parties)
  • Leadership
  • Planning
  • Support Processes
  • Operational Planning
  • Performance Evaluation 
  • Customer Satisfaction 

ISO 9001 QMS Principles:

  • Customer Focus   
  • Leadership
  • Engagement of People
  • Process Approach
  • Improvement
  • Evidence Based Decision Making
  • Relationship Management

The AS9100 series of documents outlines the requirements for a QMS within the aerospace, space, and defense industries. AS9100 constitutes and "advanced quality system" that goes well beyond ISO 9001 adding increased focus on:

  • A clear understanding of the Organizations Context
  • Risk-Based Management
  • Process Approach integrated with business processes
  • Emphasis on Change Management  
  • Introduction of Knowledge Management 
  • Explicit Performance Evaluation Requirements

AS9100 Process Model:

  • Organization and it's context
  • Customer Requirements
  • Interested Parties Needs and Expectations
  • Leadership
  • Planning
  • Support
  • Operations
  • Performance Evaluation 
  • Results
  • Customer Satisfaction 

AS9100 series of documents include:

  • AS 9100: Identifies the aerospace requirements for a QMS that demonstrates an organization's ability to provide products that meet statutory and regulatory requirements  
  • AS9101: Identifies the requirements for auditing aviation, space, and defense organizations against the 9100 family of standards 
  • AS9102: Identifies the requirements for First Article Inspection to ensure that a new product or part meet all requirements through production part verification 
  • AS9103: Identifies the requirement to plan for and manage Key Characteristics and any variation of those characteristics 
  • AS9110: Identifies the requirements for a QMS at aviation maintenance organizations and is based on ISO 9001, but includes additional requirements for aviation maintenance and airworthiness 
  • AS9120: Identifies the requirements for a QMS at stock distributors and is based on ISO 9001 but includes additional requirements. Stock distributors include organizations that resell, distribute, and warehouse parts for aerospace industries.
  • AS9131: Identifies requirements for the uniform identification, documentation, and management of nonconforming that requires formal decisions 
  • AS9134: Identifies the requirements for managing risk in the supply chain on both new and existing suppliers
  • AS9146: Identifies the requirements for the prevention of Foreign Object Damage for organizations involved in the design, development, delivery, and post-delivery provisions for maintenance, spares, and other materials
  • AS9015: Identifies the requirements for the delegation of product verification activities at an organization's suppliers

  • AS5553: Identifies the requirements for the management of electrical, electronic, and electromechanical parts to avoid the introduction of counterfeit parts into any aviation, space, and defense assemblies   

The QMS is used to:

  • Manage product and process quality 
  • Reduce the cost of poor quality (scrap, rework, repair, etc.)
  • Make better decisions based on statistical and other data
  • Engage in a process of continuous improvement  

ISO 9001 and AS9100 requirements include:

  • Leadership and Commitment
  • Risk Identification, Management and Opportunities 
  • Quality Planning 
  • Support Functions (Manpower, Facilities, etc.)
  • Operation and Production Control 
    • Design and Development
    • Release of Product
    • Control of Nonconforming Material 
    • Internal Audits and Management Review
  • Performance Evaluation (Monitoring and Measuring)
  • Quality Goals and Continuous Improvement

QMS documentation includes:

  • Policies
  • Procedures
  • Quality Manual
  • Training Materials
  • Work Instructions
  • Audit Forms
  • Process Maps
  • Control Plans

DAU Continuous Learning Modules (DCMA):

Guidance, Resources, and Tools: 

Product Quality Control

The American National Standards Institute (ANSI) and the American Society for Quality (ASQ) define quality as follows: "the totality of features and characteristics of a product or service that bears on its ability to satisfy given needs."

The American Society for Quality (ASQ) states that "quality can have two meanings: 1) the characteristics of a product or service that bear on its ability to satisfy stated or implied needs; 2) a product or service free of deficiencies." 

For many years American companies viewed quality as "conformance to requirements." This means that if a product is "in spec" it is deemed good, and if it is "out of spec" it is deemed defective. This remained true for many years until Japanese companies began taking away market share from many U.S. companies, and they did this by using techniques that they learned from American quality guru's (Deming, Juran, Crosby, Shewhart, et. al.). The Japanese view quality as "uniformity," or very little variation in the product. They achieved this by continuously improving product and processes leading to six sigma quality levels (see graphic below).

FAR 46.105 Contractor Responsibilities directs that the contractor is responsible under the contract for:

  • Controlling the quality of supplies and services 

  • Tendering to the Government supplies and services that conform to contract requirements

  • Ensuring the quality of their suppliers and vendors

  • Maintaining evidence that the supplies and services conform to contract requirements 

  • Providing to the Government that evidence

The Government has the right to perform contract quality assurance at such times and places as may be deemed necessary to determine that the supplies and services conform to contract requirements.

FAR Part 52.246 Contractor Inspection Requirements: The Contractor is responsible for performing or having performed all inspections and tests necessary to substantiate that the supplies or services furnished under this contract conform to contract requirements, including any applicable technical requirements for specified manufacturers’ parts. 

Product Quality requires an inspection and test system that is used to substantiate that the product meets the contract specifications. Dr. Juran would define product quality as "fitness for use, " while Phillip Crosby would say "conformance to requirements." The inspection system should include requirements for: 

  • Inspection System

  • Inspection and Test Status

  • Control of Nonconforming Material

  • Corrective Action

  • Document Controls

  • Control of Quality Records 

  • Process Controls

  • Inspection and Testing Criteria

  • Control of Measuring and Test Equipment 

Quality Control is a sub-set of quality assurance and is about the quality pf the product as determined by inspection and/or testing and is an aspect of quality management that consists of inspection, testing and quality measurements that verifies that the product deliverables conform to specification, is fit for purpose and meet stakeholder’s expectations. Quality control techniques are varied and driven by the nature of the product. Product inspections and tests that are done to check whether a product meets its specification is the most obvious form of QC. The inspection and test methods used depends on the technical nature of the product being developed. These methods could include product and process inspection, First Article Inspection/First Article Testing, Production Lot Testing, Production Part Approval, Qualification Testing, and Production Qualification Testing. Quality control begins at the supplier's, includes receiving inspection, in-process inspection, final inspection and testing, and customer satisfaction reporting.

First Article Inspection (FAI)/First Article Testing (FAR): FAI is a physical audit (see AS9100, AS9102 and DCMA Manual 2101-01). FAI conducted to ensure that the product meets contractual requirements and is a dimensional and qualitative inspection of a part or assembly to ensure the part or assembly fully conforms to technical drawings, specifications, customer envelope, and interface dimensions. A flow chart will be provided to validate the capability and stability of each process step. In addition, Key characteristics and critical characteristics shall be identified to ensure that they are validated during the FAI. First Article Inspections also verify production processes by examining work instructions, routing sheets, quality plans. FAIs also include reviews of in-process, acceptance testing procedures, and results. These tests may be witnessed by DCMA Quality Assurance Representatives (QAR) and/or government acquisition personnel.

Production Lot Testing: The purpose of production lot testing (PLT) is to validate quality conformance of products prior to lot acceptance. Product specialist will review the ESA testing requirements for completeness, accuracy, and applicability; coordinate any changes with the ESA; and enter the testing requirements in the material master. The test indicates that the manufacturer’s ability to create a consistent product within prescribed tolerances (quality). These tests may be witnessed by DCMA Quality Assurance Representatives (QAR) and/or government acquisition personnel.

Production Part Approval Process: Production Part Approval Process (PPAP) is used to accept an item from a manufacturing process. The purpose is to determine if engineering design records, functional, and specification requirements are understood and if the manufacturer's process has the capability to produce product consistently and continuously. The PPA provides the parts characteristics, part sample size, documentation, and requirements based on AF's needs for assessing the manufacturers' product. Varying degrees of requirements may be needed to demonstrate the manufacturing capability. These tests may be witnessed by DCMA Quality Assurance Representatives (QAR) and/or government acquisition personnel.

Qualification Testing: Qualification testing is a series of simulated operational, environmental, and endurance tests that prove the design "should" hold up and perform adequately in the field. There are no guarantees since the field can produce multiple environments and simultaneous stresses that are either impossible to produce in a test environment or too expensive to do in a test environment.

Production Qualification Testing: A technical test completed prior to the Full-Rate Production (FRP) decision to ensure the effectiveness of the manufacturing process, equipment, and procedures. This testing also provides data for the independent evaluation required for materiel release so the evaluator can address the materiel's adequacy with respect to the stated requirements. These tests are conducted on a number of random samples from the first production lot and are repeated if the process or design is changed significantly and when a second or alternative source is brought online.

Process Capability and Control 

Roles of Manufacturing:

  • Influence the design for producibility and manufacturability 
  • Prepare for production (planning)
  • Execute the production plan 
    • Reflect the design intent
    • Use repeatable processes
    • Focus on Continuous Process Improvement 

Goals of Manufacturing:

  • Deliver uniform, defect-free product 
  • Product provides consistent performance
  • Product is affordable 

One of the major goals of manufacturing is to provide the customer with “uniform, defect free product that has consistent performance and is affordable. M&Q personnel should support the assessment of manufacturing processes in order to determine if those processes are capability and in control. 

Process capability and control is a requirement of AS6500 Manufacturing Management Program standard, AS9100 quality standards, AS9003 Variation Management of Key Characteristics, and AS9138 Quality Management Systems Statistical Product Acceptance Requirements. These standards require a process control plan to describe activities that will demonstrate process capabilities. Process capability clarifies the inherent process variability of a given characteristic or process. A capability study is used to assess the ability of a process to meet a drawing/specification requirement. Typical measures include process capability (Cp/Cpk) and process performance (Pp/Ppk); X bar and R charts; control charts; and other statistical analysis tools. 

Process Capability and Control: Requires an analysis of the risks that the manufacturing processes are able to reflect the design intent (repeatability and affordability) of key characteristics.

Process Capability (Cp) is a statistical analysis or measurement of a process’s capacity or ability to produce product that meets specifications (within tolerance) and to manufacture parts repeatedly within technical specifications. Process capability is measured using capability indices (Cp, Cpk, Pp, and Ppk) which compare the processes variability (voice of the process) to the tolerance limits (voice of the customer). The data is used to calculate the standard deviation or sigma value, we can calculate Cp (Process Capability), Cpk (Process Capability Index), or Pp (Preliminary Process Capability), and Ppk (Preliminary Process Capability Index) to determine how our process is operating. Histograms are often used to analyze process capability to determine if the process is capable or not capable, and to the degree of capability often with a goal of 6 sigma for key and critical characteristics.  

Process control is a way to monitor and manage manufacturing processes using statistical process control to produce uniform, defect-free products. Statistical Process control and control charts were developed by Walter Shewhart at Bell Labs in the early 1920"s. Process control requires the management of process input variables, the identification and management of product deviations or variability from technical requirements, and the modification of process to ensure future production processes perform as expected. A process is in statistical control when all special causes of variation have been removed. Control charts are used to determine if a process is in control or not. 

An in-control process has many benefits:

  • Variability is known and thus scrap and rework can be estimated
  • Process settings can be identified and optimized during production
  • Tolerance design can be optimized to ensure drawings include appropriate tolerances 
  • Customers are better satisfied 

Process Capability and Control Guidance, Tools, and other Resources 

Supplier Quality Management 

Supply Chain: A supply chain includes the flow of material, information, and funds from raw material to the end customer and return if necessary. A supply chain includes all organizations and processes involved in the creation and delivery of products and services to the end customer. Creating faster flow results in:

  • Customer gets product faster (customer order cycle time)
  • Money moves faster (cash-to-cash cycle time)
  • Inventory is lower (cost are lower)

Supply chain quality management is the process of developing and executing a supplier quality program that ensures that products are delivered on-time, to the right place, in the right count and condition, at the agreed upon price, and in time to meet the customers’ requirements (production schedule).  A supply chain can be defined as a network of people, organizations, resources, activities and technology involved in the creation, production and delivery of a product. A supply chain includes the flow of material from a source to a destination. The Association for Operations Management (APICS) defines supply chain management (SCM) as the "design, planning, execution, control, and monitoring of supply chain activities with the objective of creating net value, building a competitive infrastructure, leveraging worldwide logistics, synchronizing supply with demand and measuring performance globally."

DoD contractors often invoke AS9120, Quality Management Systems - Requirements for Aviation, Space and Defense Distributors on their contracts. Detailed supplier requirements often are detailed under the Shewhart Cycle or Deming Cycle which includes Plan-Do-Check-Act (PDCA) processes. 

  • Plan 
    • Context of the Organization 
    • Leadership and Commitment
    • Planning 
    • Support 
  •  Do
    • Operational Planning
    • Product Requirements 
    • Production Control 
    • Control of Nonconforming Material
    • Release of Product 
  • Check 
    • Customer Satisfaction
    • Monitoring, Measuring, and Analysis
    • Internal Audits
  • Act
    • Nonconforming Material and Corrective Action 
    • Continuous Improvement

Supplier quality management begins early in product design and development and continues throughout the life cycle of the system or product. Supplier quality goes beyond lowest price to include identifying “best value” subcontractors and vendors that have a history of providing quality products and services, with low nonconformance rates and rapid response to problems. 

Quality Auditing 

Quality Auditing required by:

  • ISO 9001, Section 9.2 Internal Audit
  • ISO/OEC 17021
  • AS9100, Section 9.2 Audit
    • These documents require contractors and organizations to conduct internal audits
  • AS9104 Requirements for Aerospace Quality Management System Certification/Registrations Programs
  • AS9133 Qualification Procedure for Aerospace Standard Parts 
    • These documents require contractors to control the quality of their subcontractors/vendors 

Audit Activities:

  • Audit:  A planned, independent and documented assessment to determine whether agreed upon requirements are being met.
    • Compliance with written requirements
    • Effectiveness of written requirements in meeting basic management controls.
  • Compliance Audit:  Conformance to rules.
  • Management Audit:  Effectiveness of rules in achieving desired results.

Audit Requirements:

  • Identify program requirements
  • Identify audit criteria 
  • Plan, establish, implement and maintain an audit program 
  • Identify the frequency, methods, responsibilities, planning requirements and reporting
  • Identify consideration the results of previous audits
  • Define the audit criteria and scope for each audit
  • Identify the audit team and audit training 
  • Conduct fair and detailed audits to ensure objectivity and the impartiality 
  • Ensure that the results of the audits are reported to relevant management
  • Take correction and corrective actions 
  • Follow up to ensure closure of audit findings
  • Retain audit documentation 

Audit Types:

  • Process Audit: An audit conducted to verify that the processes are working within established limits and is producing conforming product. It evaluates an operation or method against predetermined instructions or standards to measure conformance to these standards and the effectiveness of the instructions. A process audit may:
    • Check conformance to defined requirements such as time, accuracy, temperature, pressure, composition, responsiveness, amperage, and component mixture.
    • Examine the resources (equipment, materials, people) applied to transform the inputs into outputs, the environment, the methods (procedures, instructions) followed, and the measures collected to determine process performance.
    • Check the adequacy and effectiveness of the process controls established by procedures, work instructions, flowcharts, and training and process specifications.
  • Product audit: An audit that examines a product to ensure it conforms to requirements (i.e., specifications, performance standards, and customer requirements).
  • System audit: An audit conducted on the quality management system. It can be described as a documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the system are appropriate and effective and have been developed, documented, and implemented in accordance and in conjunction with specified requirements.
    • A quality management system audit evaluates an existing quality management program to determine its conformance to company policies, contract commitments, and regulatory requirements.
    • Similarly, an environmental system audit examines an environmental management system, a food safety system audit examines a food safety management system, and safety system audits examine the safety management system
  • A first-party audit is performed within an organization to measure its strengths and weaknesses against its own procedures or methods and/or against external standards adopted by (voluntary) or imposed on (mandatory) the organization. A first-party audit is an internal audit conducted by auditors who are employed by the organization being audited but who have no vested interest in the audit results of the area being audited.
  • A second-party audit is an external audit performed on a supplier by a customer or by a contracted organization on behalf of a customer. A contract is in place, and the goods or services are being, or will be, delivered. Second-party audits are subject to the rules of contract law, as they are providing contractual direction from the customer to the supplier. Second-party audits tend to be more formal than first-party audits because audit results could influence the customer’s purchasing decisions.
  • A third-party audit is performed by an audit organization independent of the customer-supplier relationship and is free of any conflict of interest. Independence of the audit organization is a key component of a third-party audit. Third-party audits may result in certification, registration, recognition, an award, license approval, a citation, a fine, or a penalty issued by the third-party organization or an interested party.

Audit Report Components:

  • Executive Summary
  • Positive Practice Statements
  • Observations
  • Findings 

Quality Audit Objectives:

  • Ensure compliance to requirements
  • Improve performance, streamline processes and lower costs 
  • Meet and improve customer satisfaction 
  • Find and mitigate quality deviations 
  • Continuously improve 

DAU Continuous Learning Modules:

Quality of Design / Quality by Design / Quality Engineering 

Quality of design or quality by design is a deliberate and structured process for designing, developing and producing new products or improved products in a way to ensure customer satisfaction.  When an organization is designing a product the design team should be a multi-functional team utilizing a concurrent engineering approach that includes subject matter experts from design, materials, manufacturing, quality, reliability, sustainment, etc., in order to ensure that the product being developed achieves design, manufacturing, and sustainment efficiency while meeting customer requirements. 

Quality begins with the design:

  • Quality of design is the level of quality or features the producer is intending to deliver to the customer.
  • Requires the producer to understand the customer’s requirements, what is “Critical to Customer (CTC).”
  • Poor design contributes up to 40% of all quality problems, and between 60% and 80% of all costs are fixed at the design stage.
  • Lack of concentration on quality of design results in time consuming and costly reworking of products, processes, and design plans.

Quality By Design – Based on Juran Institute Processes

Quality by Design (Quality of Design or Concurrent Engineering) is the process of creating a design using multidisciplinary teams (IPTs) to conduct conceptual thinking, product design, and production planning all at one time. Quality by Design was developed by Dr. J. M. Juran who believed that quality should be built into a product from the beginning, and that most quality issues are related to the product's original design. 

Quality by Design differs from traditional sequential engineering practices and uses a five step process of Define, Discover, Design, Develop, and Deliver. Each process step has a focus and utilizes a variety of advanced quality and technical tools to meet customer requirements (satisfies the customer) often focusing on the eight dimensions of quality to create a design that is optimized for performance, cost and schedule. Quality by Design should require input from many different technical teams to include manufacturing and quality. It is the job of systems/design engineers to create the design, and it is the role of these technical personnel is to “influence the design” for producibility, manufacturability, reliability and maintainability, testability, and sustainability.

Define

Describe in general terms what the product is and what set of customers it is intended to serve. Establish the team, identify the customers and stakeholders, set goals, and create plans:

  • Operational Needs and Requirements
  • Sustainment Requirements
  • IOC Date
  • Weapon System Performance 
  • Cost (Procurement and Sustainment)
Discover

Discover the exact needs of the customer expressed in terms of the benefit to the customer. Collect and prioritize customer needs and translate those needs and create:

 

  • Work Breakdown Structure (WBS)
  • Functional Allocation 
  • Allocated Baseline
  • Measures of Effectiveness (MoE)
  • Key Performance Parameters (KPP) 
  • Measures of Performance (MoP)
  • Technical Performance Measures (TPM)
  • “Critical to Quality” measures. 

 

Develop planning worksheets for:

 

  • Customer requirements
  • Allocate Functions and Features
  • Allocate Measures and Goals (MoE, KPP, MoP, TPM and Critical to Quality
  • Detailed design features and goals 
  • Manufacturing Process features and goals 
  • Manufacturing Control features and goals

 
Design

Design a product that will meet those needs better than competitors’ and preceding products. Establish high-level product features and goals: (functional design), then develop detailed product features and goals (detailed design), optimize design features, set and approve final design.

 

  • Product Baseline
  • Functional
  • Allocated Design
  • Detail Design 
  • Design reviews and approvals
  • Identify and manage key and critical characteristics
  • Producibility Engineering
  • Trade Studies and Trade-off Analysis
  • Design Reviews
  • Above activities may require the use of the following advanced engineering techniques:
    • Design of Experiments 
    • Quality Function Deployment
    • Design for Manufacturing and Assembly (DFMA)
    • Design for X
    • Parameter Desing 

    • Tolerance Design 

       
Develop

Produce or manufacture the product by identifying, managing, controlling manufacturing processes and control.

 

  • Create manufacturing process planning (route sheets, flow diagrams, etc.)
  • Create assembly charts and operations process charts
  • Understand and control manufacturing processes:
    • Set goals for processes
    • Develop control plans
    • Establish audit procedures
    • Capture feedback and continuously improve
  • Manage capacity, identify bottlenecks:
    • Theory of Constraints
    • Optimize throughput
  • Optimize processes: 
    • Lean
    • Six Sigma
    • Standard Stable Processes
    • Identify and manage critical process parameters
    • Demonstrate process capability
    • Optimize process capability 
  • Manage Quality: 
    • Establish a Quality Management System
    • Identify critical quality attributes
    • Control variation
    • Statistical Process Controls
  • Manage Supply Chain 
Deliver

Plan for the transfer of the product to the customer, then measure and manage customer satisfaction.

 

  • Conforms to requirements
  • Performs as expected
  • Reliable product
  • On-time
  • Defect free

The Eight Dimensions of Quality 

Design Dimension Description Example 
Performance Will the product do the intended job?  What are it’ primary operating requirements or functions? GT automobile can go from 0-60 in under 4.5 seconds
Features Secondary performance characteristics, bells and whistles Automobile has heated and cooled seats
Reliability How it performs over time, mean time between failure Service at 5,000-mile intervals, expected life 200,000 miles. Reliability rating of 3.8 out of 5.0
Durability Products expected life Comes with a 60,000-mile warranty
Serviceability  How easy it is to maintain and repair, mean time to repair, total ownership cost 5-year cost for maintenance $4500, repairs $900
Conformance The degree to which a product conforms to the specifications Under 80 defect parts per million (DPM)
Aesthetics The appearance of the product, visual appeal, brand recognition  Looks fast while standing still
Reputation  Customer has a positive or negative experience J.D. Power's Award for Excellence

Quality by Design (Quality of Design) was first created by the quality guru Joseph M. Juran in his journal "Juran on Quality by Design." include prioritize a user-centered approach, that thoroughly understand user needs, implements a systematic design approach with iteration and feedback loops. 

Define
  • Identify the customers 
  • Identify product goals 
  • Establish the team
  • Construct high-level flow
  • Create plans 
Discover
  • Identify the needs of the customer 
  • Prioritize customer needs
Design
  • Identify product features (functional design)
Develop

Develop Process

  • Collect known baselines

Develop Process Controls 

  • Identify control subjects
Deliver
  • Plan for transfer to the customer

Quality by design may use some of the following tools:

  • Quality Function Deployment (QFD), this tool is probably the best tool for achieving Qualituy by Design
  • Design for Manufacturing and Assembly
  • Process Mapping 
  • Design Failure Modes and Effects Analysis (DFMEA)
  • Process Failure Modes and Effects Analysis (PFMEA)
  • Process Capability and Control

Note: You can find more information on these Quality of Design tools with a web search.

The role of manufacturing in this process is to "influence the design for producibility," which is the relative ease of fabrication and assembly. A better way to look at it is below:

Quality by Design is a component of the systems engineering process and an integral part of product and process verification and validation. It changes the way manufacturers approach all process design, process qualification, and process verification through the entire lifecycle of the product. 

Producibility is a design accomplishment resulting from a coordinated effort by design engineering and all the functional engineering specialties to create a functional hardware design that optimizes the ease and economy of fabrication, assembly, inspection, test, and acceptance of the hardware without sacrificing desired function, performance, or quality. Producibility is considered one of the most important determinants of product cost as producibility or lack thereof impacts both the product and the sustainment or life cycle cost.

Advanced Product Quality Planning (APQP) 

APQP is a structured approach to product and process design. This framework consists of a standardized ser of quality requirements (e.g. AS9145) that enable a supplier to design a product that satisfies the customer. The APQP approach consists of five steps or phases:

  • Plan and Define: Establishes the framework for the project and product and links customer requirements through product and resource planning focusing on the Voice of the Customer.
  • Product Design and Development: Part of the systems engineering process. Starts with the translation of requirements into the design and includes setting goals for design, reliability, manufacturing and quality. 
    • Identify Key Characteristics 
    • Identify Production Processes
    • Conduct Design Risk Analysis 
    • Conduct DFMEA
  • Process Design and Development: Part of the systems engineering process. Develops the manufacturing processes needed for production and identifies tools to aid in Producibility and Manufacturing Engineering:
    • Design Verification Plan and Report
    • Process Flow Charts
    • DFMEA/PFMEA
    • Product and Process Characteristics 
    • Pre-Launch Control Plans
  • Product and Process Validation: Demonstrates manufacturing and assembly processes to ensure that the given design and production processes can achieve customer requirements. 
    • Trial Runs
    • First Article Inspection 
    • Production Control Plans
    • Measurement System Analysis
  • Production: Conducts fabrication and assembly and fielding using process controls, production part approval, and uses feedback to continuously improve. 

There are five basic Core Tools detailed in separate guideline handbooks, including Advanced Product Quality Planning (APQP).  The other Core Tools are:

  • Failure Mode and Effects Analysis (FMEA)
  • Measurement Systems Analysis (MSA)
  • Statistical Process Control (SPC)
  • Production Part Approval Process (PPAP)

Resources and Guidance: 

Measurement Systems Analysis (MSA)

A measurement system has been described as a system of related measures that enables the quantification of particular characteristics. It can also include a collection of gages, fixtures, software and personnel required to validate a particular unit of measure or make an assessment of the feature or characteristic being measured. Variation often comes from a manufacturing process, but there are other sources of variation in a measurement process can include the following:

  • Process – test method, specification
  • Personnel – the operators, their skill level, training, etc.
  • Tools / Equipment – gages, fixtures, test equipment used and their associated calibration systems
  • Items to be measured – the part or material samples measured, the sampling plan, etc.
  • Environmental factors – temperature, humidity, etc.

Measurement System Analysis (MSA) is used to determine the suitability of a measurement system for use. It is crucial to have a well-functioning measurement system so that the data collected is accurate and precise. There are many factors to consider when conducting a measurement system analysis. This paper will discuss the importance of Measurement System Analysis and how to go about completing one.

MSA is a process used to evaluate the suitability of a measuring system for use. A measuring system can be any combination of a transducer, signal conditioner, display, recorder, or data acquisition system used to obtain a measurement. A measuring system is suitable if it meets the required technical performance specifications. MSA is used to identify and quantify the sources of variation in a measuring system.

Discussion on Metrology and Calibration:  

Government standards describe the requirement to provide a means for calibration and measurement traceability of all system, subsystem, and equipment. Standards define the requirement for establishing measurement traceability from actual system level measurements to the National Institute of Standards and Technology (NIST) or other approved measurement sources. 

Calibration defined: The set of operations that establish, under specified conditions, the relationship between values of quantities indicated by a measuring instrument or measuring system, or values represented by a material measure or a reference material, and the corresponding values realized by standards. 

Contractors shall establish and maintain a system for the calibration of all measurement and test equipment (M&TE) and measurement standards-used in fulfillment of contractual requirements. The calibration system shall be coordinated with the contractor’s inspection or quality control system and shall be designed to provide adequate accuracy in use of M&TE and measurement standards. All M&TE and measurement standards applicable to the contract, whether used in the contractor’s plant or at another source, shall be subject to such control as is necessary to assure conformance of supplies and services to contractual requirements. The calibration system shall provide for the prevention of inaccuracy by ready detection of deficiencies and timely positive action for their correction. The contractor shall make objective evidence of accuracy conformance readily available to the Government representative.

 Calibration requirements include:

  • Calibration system description 
  • Identification of acceptance criteria
  • Measurement traceability 
  • Environmental controls 
  • Storage and handling of measurement equipment
  • Intervals of calibration 
  • Calibration procedures documented 
  • Out-if-tolerance conditions and corrective action
  • Adequacy of the calibration system (accurate and precise)
  • Calibration sources/instruments
  • Calibration records 
  • Calibration status/labeling
  • Control of subcontractor calibration 

MIL-STD-1839D, DoD Standard Practice Calibration and Measurement System Requirements https://quicksearch.dla.mil/Transient/9050F2CC275C438888825B265D4FB4ED.pdf

MIL-HDBK-1839, Calibration and Measurement Requirements Handbook file:///C:/Users/ganoy/Downloads/MIL-HDBK-1839A.PDF

See Air Force Manual 21-113, Air Force Metrology and Calibration Program Management https://static.e-publishing.af.mil/production/1/af_a4/publication/afman21-113/dafman21-113.pdfDCMA CALIBRATION/CONTROL OF MEASURING & TEST EQUIPMENT CHECKLIST https://www.pdrep.csd.disa.mil/pdrep_files/documents/docs/QPR/04-QPR-Calibration-Control_MeasuringTestEquipment_July2019.docx

Discussion on Gage R&R Studies:

Gage Repeatability and Reproducibility (R&R) studies is a methodology for assessing the repeatability and reproducibility of the measurement system and the amount of variation in that system. These studies are a type of Measurement System Analysis. It is a study to understand how much of the process variation is caused by the measurement system itself as compared to the total variation. The study collects a series of measurements under consistent operating conditions over a set period of time with the intent of verifying that the output is the same as the input.

Measurement variation looks at two important factors. These are repeatability and reproducibility. 

  • Repeatability: Repeatability is the variation between successive measurements of the same part or trait by the same person using the same gage. In other words, how much variation do we see in measurements taken by the same person, on the same part, using the same tool?
  • Reproducibility: Reproducibility is the difference in the average of the measurements made by different people using the same instrument when measuring the identical characteristics on the same part. In other words, how much variation do we see in measurements taken by different people on the same part using the same tool?

There are many sources of measurement error:

  • Manpower: The ability of operators to achieve the same measurement. Sometimes different operators interpret measurement instructions differently or have different ability to measure and the result is a lack of reproducibility. 
  • Methods: The inspection/test methods and how the test fixtures are set up, used and the data recorded. Sometimes the measurement instructions are unclear or can be interpreted differently leading to measurement error.
  • Materials: The parts or specimens that are being measured may have inherent or natural variability that could lead to measurement error.
  • Measurement Instruments: Includes the instrument or gage that is being used to make measurements along with any fixtures, or other setup devices. Sometimes the equipment or instrument being used is not calibrated often enough, or there is wear and tear on the equipment, or the instrument itself lacks the precision needed for the measurements being taken, all can lead to measurement error.
  • Specification or Technical Requirement: Measurement values are compared to the specification or requirement. The tolerance of the specification is an important factor in assessing the measurement system.
  • Environmental Factors: Factory environmental settings can often vary during the day, or seasons. The effects of variation in temperature, humidity, cleanliness, lighting, etc., can lead to measurement error.
Continuous Process Improvement - Lean - Six Sigma - Theory of Constraints

CPI is an integrated system of improvement that focuses on doing the right things right. It is also an enterprise-wide "way of thinking" for achieving lower cost, shorter lead times, and higher quality. As a way of thinking, CPI is relevant to any process, regardless of complexity or relative importance. CPI provides an ongoing focus on enhancing the satisfaction of the Warfighter's needs. CPI can seek "incremental" improvement over time or "breakthrough" improvement all at once. Delivery (customer valued) processes are constantly evaluated and improved in the light of their efficiency, effectiveness and flexibility.

DoD CPI is a strategic approach for developing a culture of continuous improvement in the areas of reliability, process cycle times, costs in terms of less total resource consumption, quality, and productivity. In DoD, CPI comprises the application of a broad range of tools and methods, such as Lean, Six Sigma, and Theory of Constraints (TOC).

The role of the program manager (PM) is to direct the development, production, and initial deployment of a new defense system. This must be done within limits of cost, schedule, and performance, and as approved by the program manager's acquisition executive. The CPI tools outlined in this chapter can be used to support the achievement of these capabilities. A program manager should be able to:

  • Define quality and identify the various forms and structures associated with quality
  • Describe a few of the more significant quality initiatives
  • Identify several continuous process improvement tools
  • Describe the connection between quality and reliability/maintainability (R&M)
  • Describe how quality can be addressed in contract language

Major techniques used in a CPI program include:

  • Lean:  Focuses on eliminating waste or muda
  • Six Sigma: Focuses on variation and the reduction of variation on Key and Critical Characteristics
  • Theory of Constraints: Focuses on bottlenecks and the flow of material 
  • Note: Each of these techniques requires many pages of description, and there is a lot of material available on the web for each of them.

DAU Continuous Learning Modules:

Resources and other Guidance:

Lean, Six Sigma, and Theory of Constraints all have a different approach to quality. Each have a different focus, a different set of assumptions, and a different primary effect. These tools, while different can work together to improve overall product quality. See the graphic below that illustrates these differences. 

DAU Continuous Learning Modules and training:

Lean
The term “Lean” was coined by Womack, Jones and Roos in “The Machine that Changed the World” to describe the Toyota Production System (TPS) as developed by Shigeo Shingo. Toyota, using lean production, is able to reduce cost by eliminating waste, which reduces flow time and increase efficiency. They found that “Lean production” is different from craft and mass production and has the following characteristics:
  • Manpower is skilled and flexible, and are considered valuable and key resources
  • Machines, tools and methods are all flexible 
  • Suppliers are considered valuable and key and are treated as such by establishing good relationships
  • Quality is based on reducing waste and continuously improving 
  • Metrics are satisfied customers and high profits 
  • Inventories are kept low, often using a Just-in-Time delivery system
  • Flexible production, can produce a wide variety of products at either high or low volumes 
The Toyota Production System (TPS) focuses on five basic elements:
  • The use of standard/stable processes (work instructions) and a trained, multi-skilled workforce
  • The use of continuous improvement (Kaizen) focusing on perfection
  • The use of Just-in-Time (JIT) inventories to create continuous flow using a pull system based on a balanced workload 
  • Jidoka, also known as autonomation, which stops machines automatically when a defect is detected allowing for quicker responses to quality problems
  • A focus on customer satisfaction 
 
Lean Principles include:
  • Define Value: Form the customers perspective and express value in terms of specific product of service.
  • Map all of the steps:
    -  Value Added Steps
    -  Non-Value-Added Steps
  • Create Flow:  Processes must flow continuously from end-to-end.  Remove activities that impede flow.
  • Establish a Pull: Nothing moves until the customer (downstream) creates an actual demand that then drives production.
  • Continuously Improve: Until there is the complete elimination of all waste, and all remaining activities create value.
 
Lean does not call themselves Lean, they use the Toyota Production System (TPS) to achieve higher quality, lower cost, shorter cycle and throughput times, and higher customer satisfaction. Toyota has identified the seven forms of waste that cost time and money as:
  • Excess Transportation 
  • Excess Inventory
  • Excess Motion 
  • Excess Waiting 
  • Over-Production 
  • Over-Processing
  • Defects
 
DAU Continuous Learning Modules:
Guidance and other Resources:
Six Sigma

While Lean focuses on eliminating waste to improve, six sigma focuses on reducing waste by focusing on the reduction of variation. Six Sigma uses the DMAIC process for achieving quality goals. The DMAIC process is the disciplined methodology of defining, measuring, analyzing, improving and controlling the quality in every one of the Company’s products, processes and transactions-with the ultimate goal of virtually eliminating all defects. Often the goal of a quality program is to achieve six sigma outcomes where there are only 3.4 defects in a million opportunities on key and critical characteristics.  

Note: As you move from 3 sigma to 6 sigma your number of waste drops from 66,800 defects per million opportunities (DPMO) to only 3.4 DPMO. It cost a lot of money inspecting product to find those 66,800 defective units, and even with 100% inspection you will not find them all. With only 3.4 DPMO you should not inspect and that will reduce your cost of quality (COQ).

The main tool for achieving six sigma is DMAIC. 

  • Define what is important to the customer 
  • Measure how well you are currently doing 
  • Analyze what is wrong and where variation occurs
  • Improve by addressing root causes of problems 
  • Control to hold the gains made 

DMAIC utilizes many tools to help them achieve quality goals to include the following:

Note: You can find more information on Six Sigma and the DMAIC process with a web search.

DAU Continuous Learning Modules:
Theory of Constraints (TOC)

Theory of Constraints (TOC) is a systems thinking process that helps organizations improve by identifying and eliminating bottlenecks in their processes. All organizations and systems have constraints or bottlenecks that causes the organization to lose productivity and efficiency. The constraint or bottleneck controls and limits the capacity of a plant TOC uses a five-step problem solving methodology to identify and correct the bottleneck.

TOC is called a Thinking process asks three questions that are essential to improvement:

  • What needs to be changed
  • What should it be changed to
  • What action will cause the change
 
TOC Metrics identify three measures that are used to measure performance and guide management decisions:
  • Throughput
  • Investment 
  • Operating Expenses 
 
TOC operates on the following principles utilizing the Drum-Buffer-Rope process:
  • Bottlenecks govern both throughput and inventories
  • An hour lost at a bottleneck is an hour lost for the entire system
  • An hour saved at a non-bottleneck is an illusion of efficiency
  • It is ok to let a non-bottleneck to sit idle
  • It is never ok to let a bottleneck sit idle 
  • Balance flow not capacity 
TOC follows a five-step process called focusing steps:
  1. Identify the bottleneck
  2. Exploit the bottleneck
  3. Subordinate everything else to the bottleneck 
  4. Elevat the bottleneck 
  5. Repeat the above steps with the next bottleneck
Note: You can find more information on TOC with a search of the web. 
Quality Resources and Guidance

Note: All AS documents can be obtained at https://www.sae.org/standards/content/as9100/ 

Quality Tools and Checklist

The Tools and Checklist below relate to ISO 9001 and AS9100:

The Tools and Checklist below relate to the Seven Basic Quality Tools. Nine tools are presented as there is some disagreement on which are the seven. The first seven have been identified by ASQ as the 7 Quality Tools and these tools were developed at Toyota by Kaoru Ishikawa: All of these tools with definitions and templates can be found at https://asq.org/quality-resources/seven-basic-quality-tools 

  • Cause and Effect Diagram: Is a tool for identifying the potential cause of a related effect. Sometimes referred to as the Ishikawa or Fishbone diagram.
  • Check Sheet: Provides a count or tally of quantitative or qualitative data allowing you to analyze the data in order to identify defects in a process or product. 
  • Control Chart: Are used to determine whether a process will produce a product or service that has consistent and measurable properties. 
  • Histogram: A type of bar chart that depicts the frequency of data. This chart provides the easiest way to evaluate the distribution of data. 
  • Pareto Chart: A type of a histogram chart that is used to identify problems and then prioritize the problems to be solved. Prioritization is based on the 80/20 rule.
  • Scatter Diagram: Are used to study and identify the relationship between two different sets of variables.  
  • Stratification: Is a pictorial diagram that is used to sort data, objects, and people into distinct groups in order to determine patterns.
  • Flow Chart: Is a pictorial representation that captures the sequence of steps in a process, or system in a way that highlights the connections or relationships in that process.
  • Run Chart: Is a graph of data that is plotted over time. This chart allows you to identify patterns or trends in a process.

The Tools and Checklist below relate to the Seven Quality Management Tools: All of these tools with definitions and templates can be found at  https://asq.org/quality-resources/new-management-planning-tools 

  • Affinity Diagram:  Is an analytical tool that organizes many ideas into subgroups based on their "natural" relationships. This diagram is often developed using brainstorming techniques and is sometimes referred to as the KJ Method.
  • Interrelationship Diagram or Diagraph: Is a visual tool that identifies and depicts cause-and-effect relationships between factors and helps analyze the natural links between different aspects of a complex situation. 
  • Tree Diagram: Depicts the hierarchy of tasks and subtasks by breaking down tasks from broad categories into finer and finer levels of detail, helping to move step-by-step thinking from generalities to specifics.
  • Matrix Diagram: Provides a method for analyzing and displaying relationships between different data sets (two, three or more) and depicts the relationship between groups of information and can provide information about the relationship, such as its strength, the roles played by various individuals, or measurements.
  • Matrix Data Analysis: Is used to analyze and display relationships between data sets by classifying items by two major characteristics that are common to all items and then plotting each item on an x-y chart. 
  • Arrow Diagram: Is used to depict the order or sequence of tasks in a project or process and the interconnectivity of those tasks, the best schedule for the entire project, and potential scheduling and resource problems and their solutions.
  • Process Decision Program Chart: Provides a systematic means for identifying errors in a plan, and to identify what might go wrong in a plan under development.

Note: You can find more information on these tools with a web search. Also note, that there are literally hundreds of quality tools thar are available on the web to assist managers in managing projects and processes. 

A Comprehensive List of Tools to Assist Quality Managers 

This document identifies tools that could be used to help manage DoD acquisition technical, business, and management processes to include many manufacturing and quality activities. Most of these tools support systems engineering technical and technical management processes, but a few can be used to support business processes such as cost estimating, contract language, acquisition strategies, etc. Some tools could be used by a contractor, some by government personnel and some tools can bd used by many different people in many functional specialties. Most tools are available from multiple on-line sources, some tools may need to be purchased to use, many are free. 

Note: You can find more information on the tools listed below with a search of the web. 

Tools (listed in alphabetical order include):

  • 3Ps - Production Preparation Process
  • 5S’s (Sort, Straighten, Shine, Standardize, Sustain
  • 5-Whys
  • 7 - Basic Tools for Quality Improvement (includes the following which are also discussed separately):
    • Cause and Effect Diagram
    • Check Sheet
    • Control Chart
    • Histogram
    • Pareto Chart
    • Run Chart
    • Scatter Diagram
    • Stratification or Flowchart or 
  • 7 - Management and Planning Tools, or Advanced Tools for Quality Improvement (includes the following which are also discussed separately):
    • Affinity Diagram
    • Relations Diagram or Interrelationship Diagraph
    • Tree Diagram
    • Matrix Diagram
    • Matrix Data Analysis
    • Arrow Diagram
    • Process Decision Program Chart (PDPC)
  • 8D/PSP (Eight Disciplines/Problem Solving Process)
  • A3 Problem Solving Chart
  • Acceptable Quality Levels (AQL)
  • Acceptance Sampling 
  • Active Risk Manager (ARM)
  • Affinity Diagram
  • Advanced Product Quality Planning (APQP) Core Tools
  • Arrow Diagram (Chart)
  • AS6500 Manufacturing Management System (MMS)
  • AS9100 Advanced Quality Management Systems (QMS)
  • AS9103 Variation Management of Key Characteristics
  • AS9110 Maintenance
  • AS9120 Distributors
  • AS9102 FAI
  • AS9115 Software QA
  • AS9131 N/C Document
  • AS9132 Marking
  • AS9133 Supplier QA
  • AS 9137 AQAP Align
  • AS 9138 Statistical Process Control
  • AS9162 Self Verification
  • Axiomatic Design 
  • Balanced Scorecard
  • Baldrige Performance Excellence Criteria
  • Bekidou Rate
  • Benchmarking
  • Bill of Materials (BOM) 
  • Bone Diagram
  • Bottleneck Analysis
  • Box and Whisker Plot
  • Bubble Chart
  • Capacity Matrix
  • Capacity Analysis
  • Cause and Effect Diagram (Fishbone or IshIkawa)
  • Cause and Effect Matrix
  • CFMEA – Concept Failure, Mode and Effect Analysis
  • Check Sheet
  • Chokko Rate
  • Computer-Aided Design (CAD)
  • Computer-Aided Manufacturing (CAM)
  • Computer Aided Process Planning (CAPP)
  • Computer-Aided Three-Dimensional Interactive Application (CATIA)
  • Computer Integrated Manufacturing (CIM)
  • Consensogram
  • Contingency Planning
  • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart -U-chart for Attribute Data
    • Control Chart - NP-chart for Attribute Data
    • Control Chart - P-chart for Attribute Data
    • Control Chart - X-barR chart for Variable Data (measurable)
    • Control Chart - X-bar-S chart for Variable Data
    • Control Chart - X-MR/I-MR chart for Variable Data
  • Correlation Chart (Scatter Diagram)
  • Cost/Benefit Analysis
  • Cost of Quality Analysis
  • Cost of Quality (COQ)
  • Cost Modeling (Estimating)
  • Criteria Testing
  • Critical Chain Project Management 
  • Critical Design Review Checklist (DoD)
  • Critical Path/PERT
  • Critical to the Customer (CTC)
  • Critical to Quality (CTQ) Tree
  • Customer Contingency Table
  • DCOV – Define, Characterize, Optimize and Verify
  • Deming Cycle or Wheel (PDCA)
  • Departmental Purpose Analysis
  • Design of Experiments (DoE)
  • DFMEA - Design Failure Mode and Effects Analysis 
  • DFMA - Design for Manufacturing and Assembly 
  • Design to Cost (DTC)
  • DFSS - Design for Six Sigma 
  • DMAIC - Define, Measure, Analyze, Improve and Control
  • DMADV (see DCOV)
  • Domainal Mapping
    • Factory Modeling and Simulation
    • Producibility Analysis & Ergonomics
    • Process Planning 
    • Production Planning & Scheduling
    • Line Balancing & Bottleneck Analysis
    • Capacity Planning
    • Predictive Analytics & Optimization
    • Facility Planning, Layout and Design
    • Virtual Factory Mock-up
  • Failure Mode and Effects Analysis (FMEA)
  • Fault Tree Analysis 
  • First Article Inspection
  • First Article Testing 
  • Flow Chart or Process Flow Chart
  • Force Field Analysis
  • Gage R&R Studies
  • Gantt Chart
  • Histogram (Frequency or Bar Chart)
  • Hoshin Kanri (Quality Policy Deployment) 
  • Interrelationship Diagraph (also see Relations Diagram, or Network Diagram)
  • ISO 9001 Quality Management Systems (QMS) 
  • Kano Model
  • KJ model - Kawalota Jiro (see affinity diagram)
  • Lead Time Analysis
  • Learning Curve 
  • Learning Curve Analysis
  • Line of Balance (LOB)
  • Taguchi Loss Function
  • Lotus Diagram
  • Manufacturing Cost Estimating
  • Manufacturing Plan 
  • Manufacturing Readiness Assessment (MRA)
  • Manufacturing Readiness Level (MRL) Criteria
  • Matrix Diagram
  • Matrix Data Analysis Diagram
  • Measurement Systems Analysis (MSA) 
  • MIL-HDBK-896A Manufacturing and Quality Program
  • Multi-Vari Charts
  • Nominal Group Technique
  • One Piece Flow
  • Operations Process Chart
  • Overall Equipment Effectiveness (OEE)
  • Pareto Charts (Template)
  • Part-Family Analysis
  • Paynter Chart
  • P-Diagram or Parameter Design 
  • PERT Chart (Program Evaluation Routine Technique)
  • PFMEA – Process Failure, Mode and Effect Analysis
  • Pie Chart
  • Preliminary Design Review Checklist (DoD) 
  • Preliminary Hazards List (PHL)
  • Process Capability Studies (Cp and Cpk)
  • Process Performance Studies (Pp and Ppk)
  • Process Decision Program Chart (PDPC)
  • Producibility Analysis/Assessments
  • Producibility Assessment Worksheet
  • Producibility Engineering and Planning (PEP) Program
  • Production Part Approval Process
  • Production Readiness Review (PRR) Checklist
  • Programmatic Evaluation of ESOH (PESHE)
  • Pugh Matrix
  • Quadrant Chart
  • Quality Function Deployment (QFD)
  • Queuing Theory/Waiting Line Analysis
  • Radar Chart
  • Rational DOORS
  • Relation Diagram
  • Reliability Growth Analysis
  • Requirements Verification (Traceability) Matrix (RVM)
  • Risk Management Assessment Tool 
  • Route Sheet
  • Run Chart
  • Scatter Diagram (Mind Mapping)
  • SIPOC – System, Input, Process, Output and Customer
  • Six Sigma
  • SMART – Specific, Measurable, Attainable, Resources, Time
  • Spaghetti Diagram
  • Spider Diagram
  • Stratification
  • Statistical Process Control (SPC)
  • Supply Chain Management Risk Assessment
  • Swim Lane Chart (sometimes called a Deployment Flow Chart)
  • SWOT Model (Strength, Weaknesses, Opportunities and Threats
  • Systems Engineering Plan (SEP) 
  • Takt Time Analysis
  • Technical Risk Identification and Mitigation System (TRIMS)
  • Technology Readiness Level (TRL) Checklist
  • Theory of Constraints Analysis
  • Throughput Analysis Tool
  • Throughput Accounting
  • Tolerance Analysis 
  • Tolerance Design 
  • Total Productive Maintenance (TPM) 
  • Trade Studies/Analysis
  • Transition to Production (Willoughby Templates)
  • Tree Diagram
  • TRIZ Matrix 
  • Value Stream Mapping (VSM) 
  • Variability Reduction Program
  • Venn Diagram
  • Work Center
  • Work Measurement
  • X-Matrix
  • Yamazumi Chart

Note: You can find a lot of additional information by googling the tool, by visiting a number of different academic sites, or professional organizations, or by visiting various Communities of Practice (CoPs).

CREATED:
BY:
Noyes, George - Student
LAST MODIFIED:
BY:
Noyes, George - Student
View Resource
Community Resource / Manufacturing and Quality
Workforce Development, Training and Education

DoD workforce development, training, and education is concerned about DoD organizations having the right personnel that possess the necessary education, skills, and abilities to support acquisition and sustainment programs across a broad spectrum of industries and sectors, and technical, business, and academic competencies.

Department of Defense (DoD) Systems Engineering and Architecture (SE&A) partners with the Military Services and defense agencies to identify workforce challenges and champion initiatives to ensure DoD maintains its advantage in Warfighter readiness in a rapidly evolving technological environment. Cross-cutting workforce initiatives focus on building technical capability and capacity to support current and future leadership priorities in four main categories: Forecast Future Talent Needs, Strengthen Talent Pipeline, Advance Our Workforce Skills, and Close Capability Gaps.

Image removed.

The Defense Acquisition University (DAU) is one of the primary providers of acquisition training and you can access their services from their Homepage for access to your training, to apply for training, and access to web events:  https://www.dau.edu/?tour 

Apply for a Course https://www.dau.edu/training/apply-for-a-course 

This resource page will focus on the following Pinned Content:

  • Access to Specific DAU sites:
  • Access to Specific DAU courses:
  • Other Training and Education Opportunities 
  • Recommended Reading List 

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

DoD Workforce Development Resources 

DOD Workforce Development Resources can be found at https://www.cto.mil/sea/workforce/ 

  • DoD Advanced Technical Degree Guidebook 
  • DoD Engineers Make a Difference
  • DoD Civilian Careers: Engineering 
  • DoDI 1400.25, Vol. 250 DoD Civilian Strategic Human Capital Planning 
  • DoDI 5000.66. Defense Acquisition Workforce Education, Training, Experience, and Career Development Program 
  • Digital Engineering Workforce Plan
  • Acquisition Workforce Credentials
Access to specific DAU sites:
Access to specific DAU courses:

Engineering and Technical Management personnel can take any of the following courses:

Foundational: https://www.dau.edu/functional-areas/engineering-and-technical-management?field_level_id_value=2 

ACQ 1010 Fundamentals of Systems Acquisition Management https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12339 

ENG 101 Systems Engineering Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=2005 

ETM 1010 Leading Change Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12576 

ETM 1020 Mission and Systems Thinking Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12548 

ETM 1030 Requirements Definition and Analysis https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12577 

ETM 1040 Technical Management Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12573 

ETM 1050 Design Considerations Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12635 

ETM 1060 Product Realization Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12563 

ETM 1070 Digital Literacy Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12633 

ETM 1080 Software Literacy Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12631 

ETM 1090 Technical Perspectives on Defense Contracting Fundamentals https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=12634 

Practitioner: https://www.dau.edu/functional-areas/engineering-and-technical-management?field_level_id_value=2 

ETM 2010V Leading Change for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12570 

ETM 2020V Mission and Systems Thinking for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12568 

ETM 2030V Requirements Definition and Analysis for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12569 

ETM 2040V Technical Management for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12571 

ETM 2050V Desing Considerations for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12553 

ETM 2060V Product Realization for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12572 

ETM 2070V Digital Literacy for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12564 

ETM 2080M Software Literacy for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12584 

ETM 2090V Technical Perspectives on Defense Contracting for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12560 

Continuous Learning Modules at the Continuous Learning Center: https://www.dau.edu/continuous-learning-center 

CLE 001 Value Engineering https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=251 

CLE 003 Technical Reviews https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=274 

CLE 004 Introduction to Lean Enterprise Concepts https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=262 

CLE 007 Lean Six Sigma for Manufacturing https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=256 

CLE 008 Six Signa Concepts and Processes https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=309 

CLE 015 Continuous Process Improvement Familiarization https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=263 

CLE 017 Technical Planning https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=322 

CLE 019 Modular Open Systems Approach https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12258 

CLE 021 Technology Readiness Assessments https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=323 

CLE 026 Trade Studies https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=438 

CLE 028 Market Research for Engineering and Technical Personnel https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=67 

CLE 035 Introduction to Probability and Statistics https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=334 

CLE 036 Engineering Change Proposals for Engineers https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=470 

CLE 064 Standardization in the Acquisition Life Cycle https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1734 

CLE 065 Standardization Documents https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1785 

CLE 066 Systems Engineering for Systems of Systems https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1790 

CLE 068 Intellectual Property and Data Rights https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1911 

CLE 069 Technology Transfer https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2002 

CLE 070 Corrosion and Polymeric Coatings https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=1956 

CLE 074 Cybersecurity Throughout DoD Acquisition https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2048 

CLE 075 Introduction to DoD Cloud Computing https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2117 

CLE 076 Introduction to Agile Software Acquisition https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2147 

CLE 077 Defense Business Systems (DBS) Acquisition https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2152 

CLE 078 Software Acquisition for the Program Office Workforce https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12249 

CLE 079 Chemical, Biological, Radiological, and Nuclear (CBRN) Survivability https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12174 

CLE 080 SCRM for Information and Communications Technology https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12181 

CLE 084 Models, Simulations, and Digital Engineering https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12176 

CLE 085 Scientific Test and Analysis Techniques om T&E https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12241 

CLE 301 Reliability and Maintainability https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=275 

CLE 201 ISO 9000 https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=336 

CLL 032 Preventing Counterfeit Electronic Parts from Entering the DoD Supply System https://icatalog.dau.edu/mobile/CLModuleDetails.aspx?id=1729 

CLL 062 Counterfeit Prevention Awareness https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=1931 

 CMQ 100 Quality Assurance Basics https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=2021 

CMQ 230 Quality Control Graphics and Charting https://icatalog.dau.edu/mobile/CourseDetails.aspx?id=2047 

CLC 042 Predictive Analysis and Quality Assurance https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=338 

CMQ 101 Government Contract Quality Assurance (GCQA) https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2003 

CMQ 211 Quality Management System (QMS) Auditor https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12307 

ENG 0720 ISO 9000 https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12654 

CMQ 200 Statistical Sampling https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2024 

CMQ 1310 Data Collection and Analysis https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=13344 

CMQ 231 Data Collection and Analysis Application https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2054 

ETM 1060 Product Realization Fundamentals https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12563 

ETM 2060 Product Realization for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12572 

CME 103 Manufacturing and Delivery Surveillance https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12606 

CME 130 Surveillance Implications of Manufacturing and Subcontractor Management https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2077 

CME 230 Production Planning and Control (PP&C) https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=2076 

CLE 004 Introduction to Lean Enterprise Concepts https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=262 

CLE 007 Lean Six Sigma for Manufacturing https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=256 

CLE 008 Six Sigma: Concepts and Processes https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=309 

ETM 2090V Technical Perspective on Defense Contracting for Practitioners https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12560 

CLE 001 Value Engineering https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=251 

LOG 0390 Additive Manufacturing Overview https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12618 

LOG 0400 Additive Manufacturing Case Studies https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12885 

LOG 0640 DMSMS, What the PM Needs to Do and Why https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12513 

LOG 0650 DMSMS Fundamentals https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12509 

LOG 0660 DMSMS Executive Overview https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12498 

LOG 0670 DMSMS Research Essentials https://icatalog.dau.edu/onlinecatalog/courses.aspx?crs_id=12508 

Other Training and Education Opportunities:
  • Rapid eLearning Courses
  • LIFT IGNITE Program offers a 2-year program focusing on materials science and advanced manufacturing. You can find more information at https://lift.technology/wp-content/uploads/2020/08/Ignite-Brochure-9.25.18.pdf 
  • MIT EdX offers over 350 courses on manufacturing and supply chain. You can find out more at  https://www.edx.org/search?q=manufacturing+ 
  • NetFlex Flex Factor is associated with Manufacturing USA is an outreach, recruitment, and STEM education program designed to familiarize K-12 students with advanced manufacturing technology, entrepreneurship, and the education and career pathways that can lead to a STEM career. You can find out more at https://www.nextflex.us/ewd/flexfactor/ 
  • MxD (Manufacturing x Digital) advances economic prosperity and national security by strengthening U.S. manufacturing competitiveness through technology innovation, workforce development, and cybersecurity preparedness. MxD Learn is testing a three-year digital manufacturing curriculum and developing a separate four-week awareness program for high school students. https://www.mxdusa.org/focus-areas/workforce-development/ 
  • America Makes ACADEMI is a comprehensive set of immersive training experiences with advanced AM curriculum, delivered in an intense, hands-on environment, integrating skills from multiple disciplines employed in Design for Additive Manufacturing (DfAM) processes. https://www.americamakes.us/academi/ 
  • Udemy Manufacturing Operations, Planning, Management and Control https://www.udemy.com/topic/manufacturing/ 
    • 14 sections (Operations Management, Systems Desing and Capacity, Facility Layout, Forecasting Demand, Developing and Designing Product, Material Management, etc.) over 100 lectures 
  • Note: There are literally thousands of technical/trade schools, undergraduate and graduate programs that cover many M&Q skills, and higher level education in Industrial Engineering, Manufacturing Engineering, Quality Engineering, and Quality Management for M&Q personnel. 
Recommended Reading List

A Study of the Toyota Production System, Shingo, 1989

Becoming Lean: Inside Stories of US Manufacturers, by Liker, 1997

Conquering Complexity in Your Business, George, 2004

Creating a Level Pull, Smalley, 2004

Creating Continuous Flow, Rother and Harris

Creating Quality: Process Design for Results, Kolarik McGraw-Hill, 1999

Critical Chain, Eliyahu Goldratt, 1997

Factory Physics, Hopp and Spearman, 2001

Guide to Quality Control, Ishikawa, 1990

How Digital is Your Business?, Slywotzky, Adrian J. Crown Business, 2000.

Implementing Six Sigma, Breyfogle, 1999

It's Not Luck, Eliyahu Goldratt

Japanese Manufacturing Techniques, Richard Schonberger, 1982

Juran's Quality Handbook, 5th Edition, Juran, 1998

Lean Assembly, Baudin, 2004

Lean Logistics, Baudin, 2004

Lean Manufacturing: A Plant Flow Guide, Allen, Robinson and Steward, 2001

Lean Production Simplified, Dennis, 2002

Lean Six Sigma, George 2002

Lean Six Sigma for Service, George, 2003

Lean Thinking: Banish Waste and Create Wealth in Your Corporation, Womack, James P. Simon and Schuster, 2nd Edition, 2003.

Learning to See, Rother and Shook,

Let's Fix It: Overcoming the Crisis in Manufacturing, Richard Schonberger, 2001

Making Materials Flow, Harris and Wilson, 2003

Managing the Design Factory, Reinertsen, 1997

Manufacturing Planning and Control for Supply Chain Management, Jacobs, Berry, Whybark and Vollmann, Certification Edition, 

Manufacturing at Warp Speed, Schragenheim and Dettmer, 2000

Manufacturing Survival, Williams, 1995

Operations Management for Competitive Advantage, Chase, Jacob, and Aquilano, 11th Edition

Out of the Crisis, Deming 

Powered by Honda, Nelson, Moody and Mayo, 1998

Putting 5S to Work, Hirano, 1993

Quality Function Deployment: Integrating Customer Requirements Into Product Design, Akao 

Quality Function Deployment, Bossert

Quality, Productivity and Competitive Position,

Quality is Free, Crosby 

Quality, Productivity and Competitive Position, Deming

Quality Without Tears, Crosby 

Seeing the Whole, Jones and Womack

Six Sigma, Harry and Schroeder, 2000

The Certified Quality Engineer Handbook, Benbow, 2002

The Certified Quality Manager Handbook, Okes, 2001

The Complete Lean Enterprise: Value Stream Mapping for Administrative and Office Processes, Keyte and Locher

The Evolution of a Manufacturing System at Toyota, Fujimoto, 1999

The Goal, Goldratt and Cox, 1984

The Lean Design Guidebook, Mascitelli, 2004

The Lean Design Solution, Huthwaite, 2004

The Machine that Changed the World, Womack, Jones and Roos, 1990

The New Manufacturing Challenge, Kiyroshi Suzaki, 1987

The Purchasing Maching, Nelson, Moody and Stenger, 2001

The Six Sigma Handbook, Pyzdek, 2003

The Six Sigma Way, Pande, 2000

Toyota Production System:  Beyond Large Scale Production, Ohno, 1988 

The Toyota Way, Liker, 2004

Toyota Production System, Monden, 1993

Unleashing the Killer App, Downes and Mui. Harvard Business School Press, 1998. Hall Europe, 1998

Value Stream Management for the Lean Office, Tapping and Shuker, 2003

What is Lean Six Sigma, George, Rowlands and Kastle, 2004

CREATED:
BY:
Content Author No Reply
LAST MODIFIED:
BY:
Content Author No Reply
View Resource
Community Resource / Manufacturing and Quality
Manufacturing Management and Risk Assessment

Manufacturing Management (MM) is concerned with the conversion of raw materials and/or components into products or finished goods. This conversion is accomplished through a series of manufacturing procedures and processes. Manufacturing management includes such major functions as manufacturing planning, cost estimating and scheduling, engineering, fabrication and assembly, installation and checkout, demonstration and testing, product assurance, and shipment. Manufacturing considerations can begin as early as pre-MSA in which technical managers (system engineers, manufacturing, quality, etc.) assess the "manufacturing feasibility" associated with the current product or manufacturing approach.

Programs that require manufacturing will need to support manufacturing planning and control activities and may require that a manufacturing management system be put in place to support planned activities. The use of a comprehensive manufacturing management system will support the timely development, production, modification, fielding, and sustainment of affordable products by managing manufacturing risks and issues throughout the program life cycle. Meeting this objective is accomplished by including best practices and standards (i.e., AS6500, Manufacturing Management Program) in contracts with industry.

Per MIL-HNBK-896, states that "Manufacturing management system. SAE AS6500 requirements stipulate contractors should have an overall manufacturing management system that documents organizational responsibilities for each requirement in the standard. Refer to Section 6.4, Manufacturing planning for additional information on documented manufacturing plans."

Manufacturing Planning: The purpose of manufacturing planning is the identification of resources and integration into a structure that provides the capability to achieve production objectives. Manufacturing planning should include:

  • A Manufacturing Strategy
  • A Manufacturing Management Program (per AS6500 and MIL-HDBK-896)
  • A Manufacturing Plan
  • Material Management System (Material Requirements Planning)
  • Manufacturing Resource Planning, Scheduling, and Execution 
  • Workforce Plan 
  • Facilities and Tooling Planning, Scheduling, and Execution
  • Manufacturing requirements in contracts
  • Appropriate agreements with other agencies (e.g., DCMA)
  • Manufacturing assessments to support program decision points and major design reviews
  • Manufacturing cost estimating and cost analysis 
  • Manufacturing metrics and reviews at a frequency commensurate with manufacturing risks
  • Manufacturing System Verification
  • Manufacturing Surveillance 
  • Manufacturing risk assessment and management 
  • Smart Shutdown

Manufacturing Operations Management includes manufacturing planning and control systems. These planning and control systems are often managed using Manufacturing Resource Planning (MRP) software. These manufacturing planning and control systems have a three-tier planning hierarchy as listed below:

  • Front End: Sets the strategy and vision for the organization or program and creates the following long-range plans:
    • Demand Management Assessment and Planning
    • Capacity Planning 
      • Resource Planning 
      • Rough-cut Capacity Planning 
    • Material Planning 
      • Sales and Operations Planning
      • Master Production Scheduling
  • Engine: Takes the Front-End plans and develops more detailed plans that are aimed at mid-range planning timeframe
    • Detailed Capacity Planning
    • Detailed Material Planning
      • Material and Capacity Plans 
    • Distribution Requirements Planning (DRP)
  • Back End: This is the execution plans for production and supply chain operations and is based on the period of production associated with a particular product (hours, days, weeks, etc.).
    • Shop floor systems plan
      • Shop floor scheduling and control plans
    • Supplier systems plan
      • Vendor scheduling and control plans

Resource Pages will focus on the following Pinned Content:

  • MIL-HDBK-896 Manufacturing Management Program Guide
  • Production Part Approval Process (PPAP)
  • Manufacturing Key Performance Indicators (KPIs)
  • Manufacturing Management Tools and Resources 
  • Facilities Management / tooling and Test Equipment
  • Environment, Safety and Occupational Health (ESOH)
  • DMSMS/Obsolescence 
  • Corrosion Control 
  • Counterfeit Parts 
  • Manufacturing Workforce
  • Manufacturing Risk Identification 
  • MRL Resources 
  • DoD Technical Reviews and Audits 

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

Manufacturing Strategy and Plan

The program office shall develop and document the Manufacturing Plan/Strategy for the program. This can be in a standalone Manufacturing Plan or, as a minimum, documented in the Systems Engineering Plan (SEP), Life Cycle Sustainment Plan (LCSP), or Life Cycle Management Plan (LCMP). The program office may require the contractor to develop a Manufacturing Plan and contractor format is encouraged. If a contractor Manufacturing Plan is required, it should include the following elements:

  • Develop Manufacturing and Quality (M&Q) inputs to the Systems Engineering Plan (SEP)
    • Technical Schedule 
    • Technical Risk and Opportunity Management
    • Technical Program Office organization - Technical IPT
    • Engineering team identified 
    • Develop Technical Performance Measures and Metrics
  • Document the Quality Strategy in the Program Quality Plan 
  • Define manufacturing and quality assurance (M&Q) contractual requirements 
    • Specific M&Q requirements (AS6500 and AS9100
    • M&Q SOW inputs
    • Section L&M inputs
    • Develop M&Q Inputs to Award Fee/Incentive Fee plans
  • Develop IMP/IMS Entry and Exit requirements 
    • Anticipated manufacturing schedule
    • Identification of Program and Technical Reviews
    • M&Q reviews (MRL/PRR)
  • M&Q plans should include:
    • M&Q design considerations 
    • Producibility approach
    • Manufacturing process flow
    • Applicable elements of MIL-HDBK-896
    • Plans for KCs and Variability Reduction
    • Supplier management processes
    • Facility, tooling, test equipment, and manpower requirements
    • Plans for readiness assessments
    • Manufacturing process verification plans
    • Production rate analysis
    • Product acceptance strategy
    • First Article Inspections 
    • M&Q modeling and simulation models identified 
  • M&Q Supply Chain considerations 
  • Develop; DCMA Interface Agreements 

Guidance and other resources:

Manufacturing Planning and Control/Manufacturing Execution

Manufacturing Planning and Control (MPC) includes all of the systems and processes needed to design, develop, produce, and deliver a product to the customer, on time, with the right count, and right performance, all as specified in the contract. The system is concerned about planning and controlling all of the elements of manufacturing processes (manpower, machines, materials, methods, and measurements). Planning considerations include identifying and managing costs, availability, and process properties.  

Planning begins with the development of a Master Production Schedule (MPS) which drives the development of many lower-level detailed plans.  The MPS breaks down the production plan to show the quantity of each item to be made over a specific time period. The MPS takes into account the demand signal (contract for acquisition personnel) identifying what is needed (requirement), when it is needed, and where it is needed. The MPS also takes into account the Strategic or Operations Plan and the Resource Plan. The MPS is often managed by an integrated suite of software tools (MRP, MRP II, or ERP) that help users to manage their business and technical processes.

Material and Capacity Plans evolve from the MPS and detailed capacity plans and detailed material plans. Material planning begins with a Material Requirements Plan (MRP) that is used to help manage material requirements and inventory during the manufacturing process and provides a mechanism so that organizations can purchase raw materials and components in time to support production. Capacity plans are developed to determine if current production capabilities have the capacity to support the planned production rates and quantities based on labor, machines, facilities, etc. 

The Material Execution System (MES) is designed for managing, monitoring, and executing manufacturing processes in real time. The MES helps with:

  • Production Scheduling
  • Quality Management 
  • Inventory Management
  • Maintenance Management 
  • Work Order Management 

The MES works within the MRP/ERP systems used to manage major organizational functions, and the PLM system used to manage the lifecycle of the product from concept to production, sustainment, and to final disposal (see graphic below).

Manufacturing Execution Systems (MES) as a software suite that manages, tracks, and documents all manufacturing processes and the suite of tools is integrated to into a digital framework under Enterprise Resource Planning and Product Lifecycle Management. MES collects real-time manufacturing data from many sources and provides them to management of action. MES provides for the following:

  • Data Collection, Integration, Analysis, and Reporting:
  • Production Scheduling:
  • Work Order Management: 
  • Manpower and Machine Integration:
  • Material and Inventory Management:
  • Quality Management: 

Note: The DoD has not published many guides, references, resources for manufacturing planning and control as it is seen mainly as a contractor responsibility, with the exceptions of government production and maintenance facilities. The best references are college textbooks. see below for a couple of recommendations:

  • Operations Management, Chasse, Jacobs and Aquilano 
  • Factory Physics, Hopp and Spearman

Note: DAU offers a 3-day Supply Chain Management workshop several times a year and this workshop covers the following major manufacturing topics:

  • Manufacturing Production and Control
  • Forecasting Demand 
  • Demand Planning and Management 
  • Manufacturing and Material Planning 
  • Capacity Planning and Management 
  • Production and Control (Shop Floor Control)
  • Distribution

https://www.dau.edu/blogs/workshops-workshops-and-more-workshops 

DAU Continuous Learning Module:

Guidance and other resources:

Manufacturing Cost Estimating and Budgeting

Manufacturing Cost Estimating describes the nature and structure of manufacturing costs, the various techniques used to estimate cost those costs. Manufacturing and Quality (M&Q) personnel need to be able to support the following cost and budgeting activities:

  • Creation of cost estimates that are based on a cost estimating process and various cost estimating techniques:
    • Independent Cost Estimate (ICE) 
    • DoD Component Cost Estimate
    • Program Office Estimate (POE)
    • Dod Component Cost Position
    • Cost Capability Analysis (CCA) 
    • Independent Government Cost Estimate (IGCE)
    • Should Cost Estimate
  • DoD 5000.4-M, Cost Analysis Guidance and Procedures, identifies four major analytical methods or cost estimating techniques used to develop cost estimates for acquisition programs:
    • Analogy: The analogy method estimates the cost of a new item by starting with the cost of a similar existing item, then modifying this cost to take into account the differences between the old item and the new item.
    • Parametric (Statistical): The parametric (statistical) method uses regression analysis of a database of several similar systems to develop cost estimating relationships.
    • Engineering (Bottoms Up): The engineering method builds an estimate from the "bottom up" by analyzing the individual elements of the WBS.
    • Actual Costs: The actual cost method estimates the cost of future system units based on data from earlier/previous units, prototypes, or production lots of that same system (not a similar system).
  • Assessment of manufacturing and quality costs is a requirement of DoDI 5000.04, Cost and Software Data Reporting (CSDR) which requires program management offices (PMS’s) for “managing, overseeing, and executing funding (either appropriated funding or working capital funds) for developing, procuring (either initial procurement or procurement of spares or replacement parts), testing and evaluating, or sustaining a DoD acquisition program at any phase of the lifecycle.” Reporting those cost may require the following reports:
    • Integrated {Program Management Data Analysis Report (IPMDAR) 
    • Earned Value Management (EVM) analysis 
  • The identification manufacturing and quality cost risks and mitigation of cost growth on the program. The growth in cost was often matched by a slip in schedule and higher development risk. Cost growth is often linked to the lack of technical maturity, lack of design maturity, and lack of production maturity. 
  • Development of learning curves. The learning curve (cost improvement curve, or experience curve) is a well-known approach to modeling the effect of quantity on cost. Learning curve theorizes that people and organizations learn to do things more efficiently when performing repetitive tasks. The use of learning curves supports both the development of good cost estimates, but also the management of costs once the program is underway.

The cost to manufacture a weapon system or equipment results from a combination of the design, the physical facility, and the five M's (manpower, materials, methods, measurements, and machines) used to build the design and the management efficiency of the operation. As such, the manufacturing cost for a product should be viewed within the context of the factory in which the product will be built. Three other very significant cost factors will need to be identified to support the estimating activity, and these are rate, quantity and efficiency.

You will need to have a basic understanding of several accounting terms, especially as they relate to the manufacturing environment, if you are to understand manufacturing costs. These terms include:

  • Fixed Cost: A cost that does not chance with the production rate.
  • Variable Cost: Cost that do change with the production rate (materials).
  • Direct Cost: Cost that can be identified to a specific final cost objective>
  • Indirect Cost: Cost that cannot be identified to a specific cost element.
  • Recurring Cost: Cost that occur on a regular basis (labor, materials, etc.)
  • Nonrecurring Cost: Cost that occur only one time (travel, installation of equipment, etc.)

A classic division of manufacturing cost is between direct and indirect costs. Costs can also be described as fixed, or variable based on their behavior as production volume changes within broad limits. Finally, costs can be described as nonrecurring or recurring depending on when and how often costs are accumulated. Finally, costs can be described in multiple terms, thus materials could be both a direct and a variable cost.

DAU Cost Courses and other Videos:

Two important manufacturing cost tools are:

  • Learning Curve: Learning curve theorizes that people and organizations learn to do things more efficiently when performing repetitive tasks. The more often the task is performed or repeated, the more efficient the worker becomes and the less time it takes to perform those tasks. There is a usable pattern to the learning. And that pattern is different for different conditions. For that reason, a number of different learning curves have been developed. Learning curves are generally drawn showing that as the number of units produced doubles, the unit cost decreases in a predictable pattern. Learning that results in productivity and efficiency improvement can be attributed to:
    • Worker Learning 
    • Supervisor Learning
    • Applying 5S's to Workstations (Sort, Straighten, Shine, Standardize, and Sustain)
    • Tooling Improvements
    • Design/Producibility Improvements
    • Improved Work Methods
    • Improved Planning and Control
    • Increased Lot Sizes
    • Reduction in Waste (7 sources of waste in Lean are excess transportation, inventory, motion, waiting, over-production, over-processing, defects)
    • Improved Operation Sequencing and Synchronization 
    • Application of Lean and Six Sigma Activities 

Learning Curve Guidance and other Resources:

DAU Teaching Note: Application of Learning Curve Theory to Systems Acquisition

  • Work Measurement: Work Measurement is a labor standard based on a time and motion study that is used to measure and manage worker efficiency. Work measurement involves applying techniques to determine the time a qualified worker needs to complete a given task at a specific level of performance. The defined rate of working is the amount of product or work that can be produced by a qualified worker, working at a normal pace, in a normal space, using specific tools and processes.
    • Work Measurement Techniques:
      • Analytical Estimating
      • Historical Data 
      • Time Study
      • Predetermined Motion Time System
      • Work Sampling
      • Synthesis Method
    • Main Objectives of Work Measurement:
      • Identify and eliminate lost or non-productive time
      • Establish standard times for accomplishing a specific job
      • Measure performance against a defined standard
      • Establish operating goals and objectives
      • Improve performance against and productivity

Manufacturing Budgeting

Preparation of a manufacturing and quality budget requires Services and Agencies develop Program Objective Memorandums (POMs) to identify and request resources (money) to acquire capabilities and perform operations. The POM is part of the Programming Phase of the Program, Planning, Budget, and Execution (PPBE) process. 

In addition to developing POM inputs M&Q personnel need to focus on developing budgets that support various manufacturing and quality investments and operating expenses for the coming period and phase. Budgets should include an investment strategy that includes long lead funding for capital equipment, facilities, new processes, new materials, workforce development, sustainable manufacturing, supply chains, ManTech, continuous process improvements, and digital engineering efforts such as Industry 4.0 capabilities, etc.). 

Guidance and other Resources:

MIL-HDBK-896 Manufacturing Management Program Guide Requirements 

AS6500 and MIL-HDBK-896 requirements stipulate contractors should have an overall manufacturing management system that documents organizational responsibilities for each requirement in the standard. (MIL-HDBK-896 para 6.1)

AS6500 and MIL-HDBK-896A address many requirements including:

Manufacturing Planning (MIL-HDBK-896, PARA 6.4): Manufacturing plans should describe how their manufacturing management system meets the intent and requirements of the standard. The program office should require a deliverable manufacturing plan. Table III provides a list of topics to be addressed in the plan and includes the items listed below. 

  • Manufacturing System Verification
  • Facilities 
  • Tooling and Test Equipment
  • Manpower and Skills
  • Capacity Analysis
  • Capability Analysis 
  • Key Characteristics and Variability Reduction 
  • Process Capability and Control 
  • Supply Chain Management 
  • Modeling and Simulation 
  • Cost Estimating and Analysis 

Design Analysis (MIL-HDBK-896, PARA 6.2): Requires that producibility be considered as a part of design trade studies with manufacturing and quality engineers participating in the various systems engineering processes. The following is a list of important Design Analysis considerations:

  • Producibility Analysis: Requires that producibility be considered as a part of design studies. (MIL-HDBK-896, para. 6.2.1)
  • Key Characteristics (KCs) and processes: The identification of key product characteristics and key production process capabilities is a basic engineering task essential to successful manufacturing development. (MIL-HDBK-896, para. 6.2.2)
  • Design and Process Analysis: Requires the use of both Design Failure Modes and Effects Analysis (DFMEA) and Process Failure Modes and Effects Analysis (PFMEA) to identify and prevent failures early in the design and manufacturing processes. (MIL-HDBK-896, para. 6.2.3)

Manufacturing Risk Identification (MIL-HDBK-896, PARA 6.3): Manufacturing risk evaluations and assessments are performed as part of defense acquisition programs for oversight and risk assessment and come in a variety of forms (e.g. Production Readiness Reviews, Manufacturing Management/Production Capability Reviews, etc.). These evaluations and assessments are used to identify and manage risks as programs transition through the various acquisition phases and are often performed in support of program reviews and technical audits. These evaluations and assessments include:

  • Manufacturing Feasibility Assessment 
  • Manufacturing Readiness Level (MRL) Assessment
  • Production Readiness Review 

Manufacturing Operations Management (MIL-HDBK-896, PARA 6.5): Manufacturing Operations includes many functions and activities, to include:

  • Production Scheduling and Control
  • Process Planning and Control
  • Manufacturing Surveillance 
  • Manufacturing Data Analysis
  • Process Capability and Control 
  • Continuous Process Improvement 
  • Variability Reduction 
  • Measurement System Analysis 
  • Production Process Verification
  • First Article Inspection (FAI)/First Article Testing (FAT)
  • Supplier Management 
  • Cost Estimating and Cost Assessment

Integrated Master Plan (IMP) Entry Criteria (MIL-HDBK-896, PARA 6.6): The Integrated Master Plan (IMP) is an event-based, top-level plan consisting of a hierarchy of Program Events.  Each event is decomposed into specific accomplishments and each specific accomplishment is decomposed into specific Criteria.  The IMP is ultimately used to develop a time-based Integrated Master Schedule to show a networked, multi-layered schedule showing all the detailed tasks required to accomplish the work effort contained in the IMP. The IMP and IMS are related to the Work Breakdown Structure (WBS). The IMP provides a Program Manager (PM) with a systematic approach to planning, scheduling, and execution.  Includes manufacturing and quality entry criteria for many of the major life cycle milestones and design reviews:

  • Production Cost Estimates
  • Producibility (activities by phase)
  • Industrial Base Considerations 
  • Material Concerns (maturity, availability, etc.)
  • Technology Goals and ManTech studies
  • Supplier Goals
  • Production Demonstrations 
Production Part Approval Process (PPAP)

PPAP is structured process for new or revised parts, or parts produced from new or significantly revised production methods.

The Production Part Approval Process (PPAP) handbook is an industry standard that outlines the process to demonstrate engineering design and product specifications are met by the supplier’s manufacturing process. Through PPAP, suppliers and customers agree upon the requirements needed to obtain approval of supplier manufactured parts. Applicable to all parts and commodities, PPAP principles help reduce delays and non-conformances during part approval by providing a consistent approval process.

PPAP defines the approval process for new or revised parts, or parts produced from new or significantly revised production methods. The PPAP process consists of 18 elements that may be required for approval of production level parts. Not all of the elements are required for every submission. There are five generally accepted PPAP submission levels. The PPAP manual contains detailed information, guidelines and sample documents useful for completing the process requirements. The resulting PPAP submission provides the evidence that the supplier has met or exceeded the customer’s requirements, and the process is capable of consistently reproducing quality parts.

PPAPs 18 Elements:

  • Design Documentation (Records): Includes customer and supplier drawings and models. The documentation should include a copy of the purchase order.
  • Engineering Change Documents: Is required for a change to a part or product and provides a detailed description of changes of parts from previous revisions called Engineering Change Notice.
  • Customer Engineering Approval: Includes customer approval of sample production parts when required.
  • Design Failure Mode and Effects Analysis (DFMEA): Is an examination of the design risk by assessing the possible failure modes and their effects on the product or customer and the probability of occurrence. 
  • Process Flow Diagrams: Diagrams all the steps in manufacturing process from start to finish and includes components, measurement, and inspection.
  • Process Failure Mode and Effects Analysis (PFMEA): Identifies all of the possible failures within the manufacturing process for a specific product and includes a prediction of a potential process failure that could occur during production.
  • Control Plan: Identifies and details the preventive measures designed to mitigate possible PMFEAs and how quality will be implemented to ensure a stable, capable, and reliable process.
  • Measurement System Analysis (MSA): Documents the specifications and details of all equipment that will be used in the manufacturing process, and the conformance to ISO or TS standard. MSA usually includes Gage R&R studies on measurement equipment used to assess the impact on key and critical characteristics to control repeatability and reproducibility and confirmation that gages are calibrated to measure these characteristics to control measurement bias.
  • Dimensional Results: Are used to validate the measurements on drawing packages to ensure that they are correct. This includes a list of every dimension noted on the ballooned drawing or model with pass/fail assessment.
  • Design Verification Plan and Report (DVP&R): Provides Material / Performance Test Results which includes a summary of every test performed on the part, usually in the form of DVP&R (Design Verification Plan and Report).
  • Initial Process Studies: Documents all processes that will be used in the fabrication and assembly of a product and shows that critical processes are reliable. Includes SPC (statistical process control) charts.
  • Qualified Laboratory Documentation: Provides industry certifications for any lab that participated in validation testing or any offsite contracted test facilities that were used for validation or material testing.
  • Appearance Approval Report (AAR): Provides verification that the customer has approved the appearance of the product including color, texture, fit, and more.
  • Sample Production Parts: Are sample production parts that are sent to the customer for approval and can be stored at the customers site or supplier's site after the product development is complete.
  • Master Sample: Is the final sample of a part or product that has been signed off by customer and stored at the supplier.
  • Checking Aids: Isa detailed list of all tools used to inspect, test, and measure parts. This list should include the calibration schedule and frequency for the tool.
  • Records of Customer-Specific Requirements: Is a list of customer’s specific requirements for PPAP process.
  • Part Submission Warrant (PSW): Is a summary of entire PPAP submission.

Resources and Guidance:

Manufacturing Key Performance Indicators (KPIs)

What is a manufacturing Key Performance Indicator (KPI)?

A manufacturing KPI is a SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) metric that is used to track and improve the quality of production related activities. 

Many world-class manufacturing organizations measure their manufacturing performance in order to make sound business decisions and improve speed and quality. Digital technologies often help them capture this information and display that information on KRP dashboards. The following manufacturing KPIs contain both lagging and leading performance indicators and highlights that might be critical to your program. 

A key enabler for capturing this digital manufacturing information is the companies Enterprise Resource Program (ERP) that is a suite of software programs that support the automation of many business and manufacturing functions. Such software programs include SAP, Oracle, NetSuite, and others. This is NOT an endorsement of any of these tools. 

Manufacturing KPIs include: 

Customer: Measures how satisfied your customer is with your performance>

  • Customer Fill Rate: Measures the ratio of the number of orders delivered compared to the number of orders placed. 
  • On-Time delivery rate: Measures the ratio of product delivered on time compared to the total number of products delivered. 
  • Lead Time: The number of days it takes for a customer to receive an order, which includes order processing time + production time + delivery time. 
  • Customer Satisfaction Index: Is a percentage of the number customers who said that they were very or extremely satisfied with the order, divided by the number of customer surveys, and multiplied by 100.
  • Perfect Order Rate:  Is a percentage of times that customers receive the right order, at the right time, and to the right requirements.
  • On-Time Delivery to Commit:  Measures the percentage of time that the organization delivers the product on the schedule according to the contract or purchase order. 
  • Manufacturing Cycle Time: Measures the time it takes for an organization to produce a product from the time the order is released to production and completes production. 
  • Time to Make Changeovers: Measures the time it takes for an organization to switch a manufacturing line from making one product to making a different product.

Design: Measures how well the final design satisfies the customers performance and affordability requirements.

  • Drawing Release Rate: Measures the progress of designing the product.
  • Configuration Change Management: Measures the status and accounting of configuration changes, configuration verification, and configuration audits.
  • Hardware Qualification Testing: Measures hardware capability when meeting anticipated environmental and operational conditions.
  • Producibility: Is a measure of the relative ease of fabrication and assembly of the designed product.
  • Design Maturity: Is a measure of the design nearing completion counting the number of Class 1 and Class 2 changes vs. planned.

Quality: Measure important quality indicators of your ability to produce uniform, defect-free product.

  • 1st Pass Yield: Is a measure of the percentage of products that are manufactured without defects and to specifications the first time through the manufacturing process.
  • Cost of Quality (CoQ): The cost associated with appraisal and prevention, both internal and external. The cost of producing product that fails to meet requirements includes:
    • Scrap, Rework, and Repair
    • Waivers and Deviations
    • Failure Analysis
    • Planning Errors
    • Drawing Errors
    • Excess Inventory
  • Out of Station Work: Is a measure of work required on a product after it passes through a workstation. 
  • Quality Deficiency Reports: Is a measure of the number of quality problems requiring reporting. 
  • Material Review Board Actions: Is a measure of the number of actions going to the MRB, too many indicates quality problems. 
  • Defects per Million Opportunities (DPMO): Is a Six Sigma/DMAIC measure that identifies high performing quality targets.
  • Customer Rejects/Returns: Measures how many times customers rejects a product or request returns of products based on receipt a nonconforming product that is not conforming to requirements.
  • Supplier Quality Incoming: Measures the percentage of good product coming into receiving inspection from a given supplier.
  • Internal Audits: A systematic evaluation of an organizations Quality Management System to assess compliance, identify problems and implement corrective action. 
  • Warranty Claims and Costs:  Measures the costs associated with product problems after delivery. Cost can be 1.5-4% of sales.

Efficiency: Measures the use of plant, machinery, and equipment.

  • Throughput: Measures how much product is being produced on a machine, line, unit, or plant over a specified period of time (hour, day, week, month, etc.)
  • Capacity Utilization: Measures the utilization of a workstation given the relationship between actual output and the design capacity. What could be produced vs what was produced.
  • Schedule or Production Attainment: Measures the percentage of time the production plan is achieved. Completed work to Planned work. 
  • Machine Downtime: Measures the time a machine or workstation is not available for production. This includes scheduled downtime for maintenance, setups and unscheduled downtime and can include malfunctions, and breakdowns.
  • Overall Equipment Effectiveness (OEE): Is a measure of the performance of a single piece of equipment or an entire line. The measure is a multiplier of Availability x Performance x Quality.

Inventory: Measures your use of material inventory and how efficient you are.

  • WIP Inventory/Turns: Measures the efficient use of inventory materials by measuring the speed of work-in-progress through a production facility. It is calculated by dividing the cost of goods sold by the average inventory used to produce those goods.
  • Inventory Accuracy: Measures actual inventory on hand vs. what is recorded in the inventory system.

Human: Measures how well you are utilizing human capital. 

  • Employee Training/Certification: Is a measure of the total amount of money spent on training, by employee of by hour.
  • Employee Turnover: Measures employee satisfaction by looking at the number of job terminations in that period vs. the total number of employees. 
  • Employee Health and Safety Report: Measures the total number of recordable incidents or fatalities over a period of time. 

Compliance: Measures how well you meet statutory, regulatory, policy or standards requirements,

  • Reportable Health and Safety Incidents: The number of health and safety incidents that were reported to OSHA during a specified period of time. 
  • Reportable Health and Safety Rate: The number of work-related injuries per 100 employees during a specified period of time.
  • Reportable Environmental Incidents: The number of health and safety incidents that were reported to the EPA as occurring over a specified period of time.
  • Number of Non-Compliance Events / Year: The number of times a plant or facility operated outside of regulatory guidelines over a one-year period. 
  • Training/Certification: The percentage of employees that are fully trained and certified.
  • Compliance Audits – The number of compliance issues reported during an annual compliance audit.

Maintenance: Measures how well maintenance activities keep machines and equipment in an operational status.

  • Percentage Planned vs. Emergency Maintenance Work Orders: This is a ratio metric indicates how often scheduled maintenance takes place, versus more disruptive/un-planned maintenance.
  • Downtime in Proportion to Operating Time: This is the ratio of equipment downtime compared to equipment operating time. 
  • Machine Downtime: Includes all scheduled and unscheduled times that the machine is not in operation. 
  • Unscheduled Downtime: The amount of time a machine should be in operation but is not due to equipment failure.
  • Machine Set Up Time: The time it takes to set up a machine for a production run.
  • Maintenance Equipment Cost: The cost associated with maintaining and repairing equipment to ensure it is available for production.
  • Mean Time Between Failures: Measures the average time between equipment failures. 

Increasing Flexibility & Innovation: Measures how well an organization stays ahead of the competition.

  • Rate of New Product Introduction:  Measure how rapidly a new product can be introduced to the marketplace and includes design, development and manufacturing.

Reducing Costs & Increasing Profitability: Measures how well an organization meets costs and profitability goals.

  • Manufacturing Cost as a Percentage of Revenue: Is a measure of the total manufacturing costs to the overall revenues produced by an organization.
  • Productivity in Revenue per Employee: Is a measure of how much revenue is generated by a plant, business unit or company, divided by the number of employees.
  • Return on Assets/Return on Net Assets: Is a measure of financial performance calculated by dividing the net income from a plant by the value of fixed assets and working capital deployed.
  • Cash-to-Cash Cycle Time: The length of time between the purchase of a product, and the collection of payments.
Manufacturing Management Tools and Resources

Manufacturing Planning begins with the Systems Engineering Plan and SEP Outline. Manufacturing management Tools include:

Manufacturing Management Guidance and other Resources:

A Comprehensive List of Tools to Aid Manufacturing Personnel

This document identifies tools that could be used to help manage DoD acquisition technical, business, and management processes to include many manufacturing and quality activities. Most of these tools support systems engineering technical and technical management processes, but a few can be used to support business processes such as cost estimating, contract language, acquisition strategies, etc. Some tools could be used by a contractor, some by government personnel and some tools can bd used by many different people in many functional specialties. Most tools are available from multiple on-line sources, some tools may need to be purchased to use, many are free. 

Note: You can find more information on the tools listed below with a search of the web. 

Tools (listed in alphabetical order include):

  • 3Ps - Production Preparation Process
  • 5S’s (Sort, Straighten, Shine, Standardize, Sustain
  • 5-Whys
  • 7 - Basic Tools for Quality Improvement (includes the following which are also discussed separately):
    • Cause and Effect Diagram
    • Check Sheet
    • Control Chart
    • Histogram
    • Pareto Chart
    • Run Chart
    • Scatter Diagram
    • Stratification or Flowchart or 
  • 7 - Management and Planning Tools, or Advanced Tools for Quality Improvement (includes the following which are also discussed separately):
    • Affinity Diagram
    • Relations Diagram or Interrelationship Diagraph
    • Tree Diagram
    • Matrix Diagram
    • Matrix Data Analysis
    • Arrow Diagram
    • Process Decision Program Chart (PDPC)
  • 8D/PSP (Eight Disciplines/Problem Solving Process)
  • A3 Problem Solving Chart
  • Acceptable Quality Levels (AQL)
  • Acceptance Sampling 
  • Active Risk Manager (ARM)
  • Affinity Diagram
  • Advanced Product Quality Planning (APQP) Core Tools
  • Arrow Diagram (Chart)
  • AS6500 Manufacturing Management System (MMS)
  • AS9100 Advanced Quality Management Systems (QMS)
  • AS9103 Variation Management of Key Characteristics
  • AS9110 Maintenance
  • AS9120 Distributors
  • AS9102 FAI
  • AS9115 Software QA
  • AS9131 N/C Document
  • AS9132 Marking
  • AS9133 Supplier QA
  • AS 9137 AQAP Align
  • AS 9138 Statistical Process Control
  • AS9162 Self Verification
  • Axiomatic Design 
  • Balanced Scorecard
  • Baldrige Performance Excellence Criteria
  • Bekidou Rate
  • Benchmarking
  • Bill of Materials (BOM) 
  • Bone Diagram
  • Bottleneck Analysis
  • Box and Whisker Plot
  • Bubble Chart
  • Capacity Matrix
  • Capacity Analysis
  • Cause and Effect Diagram (Fishbone or IshIkawa)
  • Cause and Effect Matrix
  • CFMEA – Concept Failure, Mode and Effect Analysis
  • Check Sheet
  • Chokko Rate
  • Computer-Aided Design (CAD)
  • Computer-Aided Manufacturing (CAM)
  • Computer Aided Process Planning (CAPP)
  • Computer-Aided Three-Dimensional Interactive Application (CATIA)
  • Computer Integrated Manufacturing (CIM)
  • Consensogram
  • Contingency Planning
  • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart - C-chart for Attribute Data (Go/No Go, Good/Bad, etc.)
    • Control Chart -U-chart for Attribute Data
    • Control Chart - NP-chart for Attribute Data
    • Control Chart - P-chart for Attribute Data
    • Control Chart - X-barR chart for Variable Data (measurable)
    • Control Chart - X-bar-S chart for Variable Data
    • Control Chart - X-MR/I-MR chart for Variable Data
  • Correlation Chart (Scatter Diagram)
  • Cost/Benefit Analysis
  • Cost of Quality Analysis
  • Cost of Quality (COQ)
  • Cost Modeling (Estimating)
  • Criteria Testing
  • Critical Chain Project Management 
  • Critical Design Review Checklist (DoD)
  • Critical Path/PERT
  • Critical to the Customer (CTC)
  • Critical to Quality (CTQ) Tree
  • Customer Contingency Table
  • DCOV – Define, Characterize, Optimize and Verify
  • Deming Cycle or Wheel (PDCA)
  • Departmental Purpose Analysis
  • Design of Experiments (DoE)
  • DFMEA - Design Failure Mode and Effects Analysis 
  • DFMA - Design for Manufacturing and Assembly 
  • Design to Cost (DTC)
  • DFSS - Design for Six Sigma 
  • DMAIC - Define, Measure, Analyze, Improve and Control
  • DMADV (see DCOV)
  • Domainal Mapping
    • Factory Modeling and Simulation
    • Producibility Analysis & Ergonomics
    • Process Planning 
    • Production Planning & Scheduling
    • Line Balancing & Bottleneck Analysis
    • Capacity Planning
    • Predictive Analytics & Optimization
    • Facility Planning, Layout and Design
    • Virtual Factory Mock-up
  • Failure Mode and Effects Analysis (FMEA)
  • Fault Tree Analysis 
  • First Article Inspection
  • First Article Testing 
  • Flow Chart or Process Flow Chart
  • Force Field Analysis
  • Gage R&R Studies
  • Gantt Chart
  • Histogram (Frequency or Bar Chart)
  • Hoshin Kanri (Quality Policy Deployment) 
  • Interrelationship Diagraph (also see Relations Diagram, or Network Diagram)
  • ISO 9001 Quality Management Systems (QMS) 
  • Kano Model
  • KJ model - Kawalota Jiro (see affinity diagram)
  • Lead Time Analysis
  • Learning Curve 
  • Learning Curve Analysis
  • Line of Balance (LOB)
  • Taguchi Loss Function
  • Lotus Diagram
  • Manufacturing Cost Estimating
  • Manufacturing Plan 
  • Manufacturing Readiness Assessment (MRA)
  • Manufacturing Readiness Level (MRL) Criteria
  • Matrix Diagram
  • Matrix Data Analysis Diagram
  • Measurement Systems Analysis (MSA) 
  • MIL-HDBK-896A Manufacturing and Quality Program
  • Multi-Vari Charts
  • Nominal Group Technique
  • One Piece Flow
  • Operations Process Chart
  • Overall Equipment Effectiveness (OEE)
  • Pareto Charts (Template)
  • Part-Family Analysis
  • Paynter Chart
  • P-Diagram or Parameter Design 
  • PERT Chart (Program Evaluation Routine Technique)
  • PFMEA – Process Failure, Mode and Effect Analysis
  • Pie Chart
  • Preliminary Design Review Checklist (DoD) 
  • Preliminary Hazards List (PHL)
  • Process Capability Studies (Cp and Cpk)
  • Process Performance Studies (Pp and Ppk)
  • Process Decision Program Chart (PDPC)
  • Producibility Analysis/Assessments
  • Producibility Assessment Worksheet
  • Producibility Engineering and Planning (PEP) Program
  • Production Part Approval Process
  • Production Readiness Review (PRR) Checklist
  • Programmatic Evaluation of ESOH (PESHE)
  • Pugh Matrix
  • Quadrant Chart
  • Quality Function Deployment (QFD)
  • Queuing Theory/Waiting Line Analysis
  • Radar Chart
  • Rational DOORS
  • Relation Diagram
  • Reliability Growth Analysis
  • Requirements Verification (Traceability) Matrix (RVM)
  • Risk Management Assessment Tool 
  • Route Sheet
  • Run Chart
  • Scatter Diagram (Mind Mapping)
  • SIPOC – System, Input, Process, Output and Customer
  • Six Sigma
  • SMART – Specific, Measurable, Attainable, Resources, Time
  • Spaghetti Diagram
  • Spider Diagram
  • Stratification
  • Statistical Process Control (SPC)
  • Supply Chain Management Risk Assessment
  • Swim Lane Chart (sometimes called a Deployment Flow Chart)
  • SWOT Model (Strength, Weaknesses, Opportunities and Threats
  • Systems Engineering Plan (SEP) 
  • Takt Time Analysis
  • Technical Risk Identification and Mitigation System (TRIMS)
  • Technology Readiness Level (TRL) Checklist
  • Theory of Constraints Analysis
  • Throughput Analysis Tool
  • Throughput Accounting
  • Tolerance Analysis 
  • Tolerance Design 
  • Total Productive Maintenance (TPM) 
  • Trade Studies/Analysis
  • Transition to Production (Willoughby Templates)
  • Tree Diagram
  • TRIZ Matrix 
  • Value Stream Mapping (VSM) 
  • Variability Reduction Program
  • Venn Diagram
  • Work Center
  • Work Measurement
  • X-Matrix
  • Yamazumi Chart

Note: You can find a lot of additional information by googling the tool, by visiting a number of different academic sites, or professional organizations, or by visiting various Communities of Practice (CoPs).

Facilities Management \ Tooling and Test Equipment 

Facilities management encompasses a variety of professional skills that focus on the design, construction, management, of an installation to include plant, equipment, and tooling. Facilities management includes all permanent and semi-permanent real property required to support a system throughout the systems life cycle. Facility management includes studies of facility requirements to include plant location, facility size and layout, production system or environment (job shop, batch processing, continuous flow, etc.), environmental, safety, and occupational health considerations, property management and control, environmental controls (HVAC), maintenance, security considerations, and budgeting of such property through final disposal or facility shutdown. Major facility concerns include:

  • Developing a Facility Strategy: Should include facility design, shop floor layout, physical and cybersecurity, plant safety, ESOH considerations, equipment (machine) design and layout, maintenance concepts, etc.
  • Conducting Facility risk assessments: Could include capacity analysis, bottleneck analysis, modeling and simulation, and flow analysis.
  • Monitoring and managing facility, facility goals and metrics
  • MIL-HDBK-896, Manufacturing Management Program Guide https://www.dodmrl.com/MIL-HDBK-896A%20Manufacturing%20Managment.pdf 

Tooling is designed and developed to aid in the manufacture of parts or components, or to support assembly operations. Tooling includes jigs, dies, fixtures, molds, patterns, taps, gauges, other equipment and manufacturing aids. Special tooling, special test and special inspection equipment are included under the broad definition of tooling. Production tools may be developed and used for a one-time or short production run or may need to be developed to withstand the robust environment of long-term rate production. 

Major tooling concerns include:

  • Developing a Tooling Strategy
  • Conducting Tooling risk assessments
  • Monitoring and managing tooling program to include managing tooling, tooling goals and metrics 

Facilities and Tooling Guidance and Resources:

  • DoDD 4275.5 Acquisition and Management of Industrial Resources 
Environment, Safety and Occupational Health (ESOH)
ESOH Definitions:
  • Environment. Air, water, land, living things, built infrastructure, cultural resources, and the interrelationships that exist among them. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.2.). Alternate definition: The aggregate of all external and internal conditions (such as temperature, humidity, radiation, magnetic and electrical fields, shock, vibration, etc.), whether natural, manmade, or self-induced, that influences the form, fit, or function of an item. (Reference MIL-HDBK-338B Electronic Reliability Design Handbook)
  • Safety. The programs, risk management activities, and organizational and cultural values dedicated to preventing injuries and accidental loss of human and material resources, and to protecting the environment from the damaging effects of DoD mishaps. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.14.)
  • Occupational Health. Activities directed toward anticipation, recognition, evaluation, and control of potential occupational and environmental health hazards; preventing injuries and illness of personnel during operations; and accomplishment of mission at acceptable levels of risk. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.12.)
  • ESOH Management. Sustaining the readiness of the U.S. Armed Forces by cost effectively maintaining all installation assets through promotion of safety, protection of human health, and protection and restoration of the environment. (Reference: DoDD 4715.1E Environment, Safety, and Occupational Health (ESOH), Para E1.1.6.)

ESOH considerations need to be included in the Systems Engineering Plan (SEP), Programmatic ESOH Evaluation, and National Environmental Policy Act (NEPA)/Executive Order (EO) 12114 Compliance Schedule. 

  • NEPA and NEPA Compliance Schedule: Requires 
    • Environmental impacts to be considered during the planning process.
    • Agencies and public participation in the planning process (including the DoD).
    • Disclosure about the action, alternatives, environmental effects and mitigation of their actions.
  • Hazardous Material Management Program (DoDI 5000.02 and NAS 411): Requires the Program Manager to develop a Hazardous Material Management Implementation Plan that will include procedures for identifying, minimizing use of, tracking, storing, handling, packaging, transporting, and disposing of such material. In addition, the contractor shall conduct a HMMP tailored to the contract, that will eliminate, reduce, or control hazardous materials during the system life cycle while:
    • maintaining an appropriate balance with (specified) performance requirements and
    • the cost of HMMP is performed as part of the systems engineering process while protecting human health and the environment.
  • Pollution Prevention Program (DOD 5000.02 and DODI 4715.4): Tasks the PM to identify and evaluate environmental and occupational health hazards and establish a pollution prevention program.  The PM shall identify the impacts of the system on the environment during its life (including disposal), the types and amounts of pollution from all sources (air, water, noise, etc.) that will be released to the environment, actions needed to prevent or control the impacts, ESOH risks associated with using the new system, and other information needed to identify source reduction, alternative technologies, and recycling opportunities.  The pollution prevention program shall serve to minimize system impacts on the environment and human health, as well as environmental compliance impacts on program TOC.  A fundamental purpose of the pollution prevention program is to identify and quantify impacts, such as noise, as early as possible during system development, and to identify and implement actions needed to prevent or abate the impacts.
  • Programmatic Environmental Safety and Health Evaluation (PESHE): The PM for all programs, regardless of ACAT level, shall prepare a PESHE which incorporates the MIL-STD-882D process and includes the following:
    • identification of ESOH responsibilities
    • the strategy for integrating ESOH considerations into the systems engineering process
    • identification of ESOH risks and their status
    • a description of the method for tracking hazards throughout the life cycle of the system
    • identification of hazardous materials, wastes, and pollutants (discharges/emissions/ noise) associated with the system and plans for their minimization and/or safe disposal 
    • and a compliance schedule covering all system-related activities for the NEPA
  • System Safety and Health Program (MIL-STD-882E): The integration of system safety and health issues into the systems engineering process is a MIL-STD-882) requirement that goes from cradle to grave and includes:
    • System Safety
    • Software Safety
    • Explosives Safety
    • Laser Safety
    • Occupational Safety
    • Public Safety 

ESHO Guidance and other Resources:

ESOH Tools and Checklist:

  • ADDM PESHE Template, use internet search to find
  • AFLCMC ADDM PESHE Template, use internet search to find
  • ISO 14000 Operational Gap Analysis Tool, use internet search to find
  • DoD ESOH Management Evaluation Criteria, use internet search to find
DMSMS/Obsolescence 

Diminishing Manufacturing Sources and Material Shortages (DMSMS) is the loss of sources of items or material, surfaces when a source announces the actual or impending discontinuation of a product, or when procurements fail because of product unavailability. DMSMS may endanger the life-cycle support and viability of the weapon system or equipment.

DMSMS/Obsolescence requires the Program Manager, through the Product Support Manager, to develop, ensure funding, and execute a DMSMS management plan and conduct proactive risk-based DMSMS management per that plan to identify current DMSMS issues, forecast future DMSMS issues, program and budget for resolving DMSMS issues, and implement those resolutions IAW DODI 4245.15. Implementing DMSMS issue resolutions will take into account a parts management process that considers SCRM, supportability, loss of technological advantage, and obsolescence when selecting parts used in DMSMS resolutions. In addition, the PSM will use both current and forecasted DMSMS issues in developing product roadmaps for supportability.”

DMSMS typically follows a life cycle where new parts are introduced into the system and then the total number of parts grows in availability until the system(s) these parts are in start to decline. Then parts availability becomes a concern. Thus, design engineers need to take into account part availability and maturing when selecting parts for their product.

Image removed.

The SD-22 DMSMS Guidebook is another key resource and reference. It includes common practices developed by various DoD organizations to achieve these goals and includes examples of results for review and consideration as well. The primary objectives of the SD-22 are to:

  • Create awareness of the extent and impact of DMSMS issues on DoD systems
  • Provide best practices to PMs for implementing a robust, risk-based DMSMS management process, and building a cost-effective DMSMS management program
  • Encourage DMSMS resilience by using a modular, open system design approach along with other supportability-related design considerations in conjunction with part selection procedures that choose items with significant time left in their life cycle and with viable replacement options whenever possible in order to reduce the likelihood that a design will experience near-term DMSMS issues and increase the probability of a quick recovery when issues do occur
  • Define DMSMS support metrics to measure the effectiveness, efficiency, and return on investment (ROI) of a robust DMSMS management program
  • Promote affordable and efficient program office support through rapid and cost-effective DMSMS management best practices and resolutions that take into account equipment life cycles, technology changes, and planned obsolescence
  • Promote the exercise of best practices to address obsolescence risks throughout the life cycle. 

DAU Continuous Learning Modules and other training:

  • DMSMS: Best Practices for Contracting https://media.dau.edu/media/1_flz3e5jg 
  • CLL 201,“DMSMS Fundamentals” (continuous learning module
  • CLL 202,“DMSMS for Executives” (continuous learning module)
  • CLL 203,“DMSMS Essentials” (continuous learning module)
  • CLL 204,“DMSMS Case Studies” (continuous learning module)
  • CLL 205,“DMSMS for Technical Professionals” (continuous learning module).

Guidance and other Resources: 

Corrosion Control 

10 U.S.C. 2228 requires DoD to develop and implement a long-term strategy to address the corrosion of its equipment and infrastructure. A key element of this strategy is programmatic and technical guidance provided in this guidebook. 

“Corrosion is the deterioration of a material or its properties due to a reaction of that material with its chemical environment.” Corrosion is far more widespread and detrimental than merely rust of steel or iron. The acquisition program needs to consider additional materials, including other metals, polymers, composites, and ceramics affected by the operational environment. 

DoD Corrosion Prevention and Control Planning Guidebook for Military Systems and Equipment focuses on these keys to CPC success:

  • Integrate CPC planning and execution early and throughout the acquisition process. 
  • Resource the necessary funding and expertise. 
  • Manage CPC risks. 
  • Incorporate CPC language in procurement and contract documents. 
  • Monitor CPC planning and execution throughout the acquisition process so that the system design keeps corrosion prevention in mind.

DAU Continuous Learning Modules:

Guidance and other Resources:

Counterfeit Parts 

DoD Instruction 4140.67 establishes policy and assigns responsibilities to prevent the introduction of counterfeit materiel at all levels of the DoD supply chain. It applies to all life cycle phases of acquisitions and materiel management, from the time an operational requirement is identified to introduce an item or piece of equipment into the DoD supply chain, to the ultimate disposition, phase-out or retirement of that item or equipment.

Key DoD policy embodied in this issuance:

  • Employ a risk-based approach to reduce the frequency and impact of counterfeit material within DoD acquisition systems and life cycle sustainment processes
  • Require remediation for counterfeit materiel discovered after delivery of materiel
  • Direct application of authentication technologies
  • Report suspect and confirmed counterfeit materiel to GIDEP within 60 days

Guidance and other Resources:

Manufacturing Workforce 

Manufacturing Workforce addresses the contractor’s anticipated workforce needs (both in numbers and skills) should be evaluated as part of MRL assessments and PRRs. the manpower required for a given program should be compared with projected requirements for all programs in a given facility to determine if there are future constraints that must be addressed. To assess the ability to acquire skilled workers, consider the types of local industries, the competitiveness of the labor market, and the availability of technical and higher education.

Several Industry Associations (The Manufacturing Institute, Deloitte, National Association of Manufacturers, etc.) have conducted studies of the future of the U.S. manufacturing workforce. In addition, a 2019 DoD Study on the Factory of the Future, identified several manufacturing workforce findings: 

  • U.S. manufacturers face challenges in two high-level workforce areas that are increasingly limiting DoD’s ability to meet weapon system production needs: 
    • Skill, competency, and capability deficiencies
    • Growing labor demand-supply disconnect
  • The Department’s PQM acquisition career field has existed in a suboptimal state for nearly 25 years and will continue to impact readiness due to:
    • limited PQM billets allocated to programs and organizations across the DoD 
    • a resultant workforce that is insufficiently sized, trained, and seasoned; and 
    • an insufficient suite of manufacturing policies, practices and standards.
  • Manufacturing has not been promoted nationally as an attractive career choice, which is limiting workforce supply pipelines for defense manufacturers and DoD M&Q workers. 
  • The limited number of people, nationwide, who have the ability to work in manufacturing was stated as a primary concern for both government and industry manufacturing and sustainment operations and is not projected to improve.
  • Finding employees who can pass a security clearance check is increasingly challenging due, in part, to inconsistency between State drug laws and Federal employment requirements, in addition to the time it takes to obtain the clearance
  • Commercial industry and the DIB are increasingly competing against each other for talent from the same small talent pool; the lower level of benefits offered by DoD puts them at a disadvantage compared to industry. This, in turn, reduces the PQM accession pipeline.
  • The PQM career field billet structures in the military departments and agencies do not provide healthy and diverse pathways for career advancement. This is a static condition that does not appear to be improving.
  • Of the organic sites visited, several stated that insufficient funding for infrastructure is leading to facilities and equipment maintenance and obsolescence; and impacts ability to attract and retain talented workers
  • As the traditional DIB confronts workforce challenges (e.g., aging employees, retirement obligations), DoD is expected to struggle to generate needed capacity to support longstanding and emerging programs critical to national security 

Guidance and other Resources:

Manufacturing Risk Identification 

Manufacturing Risk Identification: Manufacturing assessments should be conducted early in the life cycle, and throughout the life of the acquisition program and include Feasibility Assessments, Manufacturing Readiness Level Assessments, and Production Readiness Reviews. (MIL-HDBK-896, para. 6.3)

Assessing Manufacturing Risk: A Best Practice

MIL-HDBK-896 requires:
  • Manufacturing Feasibility Assessments  
  • Manufacturing Readiness Level (MRL) Assessments
  • Production Readiness Reviews (PRR)
  • Independent Technical Risk Assessment (ITRA)
 
Manufacturing Feasibility Assessments are typically performed early in the life cycle when competing design concepts are being considered. The assessments are conducted to identify potential manufacturing constraints and risks and the capability of the contractor to execute the manufacturing efforts.
 
Manufacturing Readiness is the ability to harness the manufacturing, production, quality assurance, and industrial functions to achieve an operational capability that satisfies mission needs – in the quantity and quality needed by the warfighter. Public law requires the use of manufacturing readiness levels or other manufacturing readiness standards as a basis for measuring, assessing, reporting, and communicating manufacturing readiness and risk on major defense acquisition programs throughout the DoD. The use of MRLs to assess manufacturing readiness can foster better decision making, program planning and program execution through improved understanding and management of manufacturing risk. Often these assessments will take place during program reviews or technical reviews and audits. 
 
The Production Readiness Review (PRR) is a Systems Engineering Technical Review at the end of EMD that determines if a program is ready for production. MRL 8 is the target for Low-Rate Initial Production (LRIP) and MRL 9 is the target for Full Rate Production (FRP); these targets should be reflected in the acquisition program baseline. The PRR assesses whether the prime contractor and major subcontractors have completed adequate production planning and confirms that there are no unacceptable risks for schedule, performance, cost, or other established criteria. Generally, incremental PRRs are conducted at the prime and major subcontractors.
 
The Independent Technical Review Assessment (ITRA) is a requirement of law and policy.
  • NDAA 2017 - Section 807 requires an ITRA prior to MS A where critical technologies and manufacturing processes need to be matured, and MS B where there is a decision to enter LRIP or FRP
  • DoD Policy Memorandum for ITRAs for MDAP programs and DoDI 5000.88:
    • Conducted on all MDAPs prior to MS or Production decisions
    • Considers the full spectrum of Engineering, Technology, and Integration risks
    • ITRAs are independent of the program office
 
Guidance and other Resources:

DAU Continuous Learning Modules and other videos:

Note:  Go to http://dodmrl.com for the latest information on the MRL Deskbook, MRL Matrix, and MRL Users Guide

MRL Resources

MRL POCs: 

DoD Technical Reviews and Audits

Current list of DoD Technical Reviews and Audits, many of these are covered in the DoD Systems Engineering Guidebook:

  • Alternative Systems Review (ASR)
  • System Requirements Review (SRR)
  • System Functional Review (SFR)
  • Preliminary Design Review (PDR)
  • Critical Design Review (CDR)
  • Test Readiness Review (TRR)
  • System Verification Review (SVR)
  • Functional Configuration Audit (FCA)
  • Production Readiness Review (PRR)
  • Physical Configuration Audit (PCA)
  • In-Service Review (ISR)
  • Manufacturing Readiness Assessments (MRAs)
  • Technical Readiness Assessments (TRAs)
  • Independent Technical Risk Assessments (ITRAs)

DoD Checklist for Technical Reviews and Audits: 

DoD Systems Engineering Guidebook https://ac.cto.mil/wp-content/uploads/2022/02/Systems-Eng-Guidebook_Feb2022-Cleared-slp.pdf 

 

CREATED:
BY:
Content Author No Reply
LAST MODIFIED:
BY:
Content Author No Reply
View Resource
Community Resource / Manufacturing and Quality
Contracting for Manufacturing and Quality

Contracting for Manufacturing and Quality 

DoD acquisition and contracting requirements and activities are required by various statutory and regulatory requirements to include the FAR/DFAR and by many DoD, Service and agency regulations, policies, and guidance documents. The graphic below outlines the DoD contracting process and the three major contracting phases.

The contract is the vehicle used to establish the formal relationship between the government and a prime contractor. Government business processes include the business strategy or acquisition strategy, contracting approach, contracting strategies, contract language, and financial strategies. M&Q personnel may be asked to support the above contract processes to ensure that M&Q considerations and risks are addressed in contracting documents and activities. The Service Acquisition Process Framework outlines the following activities:

  • Forming the Team
  • Develop/Review the Contract Strategy
  • Conduct Market Research
  • Conduct Requirements Analysis:
    • Draft/Update Requirements Roadmap 
    • Performance Work Statement or SOO
    • QASP
    • Develop a Government Cost Estimate 
  • Develop/Review the Source Selection Plan
  • Develop/Review the Request for Proposal and then Issue
  • M&Q Inputs to the Contract (Section C, E, L and M) (refer to MIL-HDBK-245E)
  • Manage the Contract from Contract Evaluation, Negotiation, Award, and entire Lifecycle 

This resource page will focus on the following Pinned Content:

  • Pre-Solicitation Activities
  • Solicitation and Award Activities 
  • Contracting Resources and Guidance 
  • Manufacturing and Quality Data Item Descriptions (DIDs)
  • Contract Post-award Evaluations 
  • Work Breakdown Structure (WBS)

DAU Continuous Learning Modules and other training:

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

Pre-Solicitation Activities 

The pre-solicitation phase begins with a valid acquisition need that requires a material solution that drives initial planning and team formation. This phase is performed entirely by the Government before the issuance of the solicitation and includes input from industry as a result of market research efforts from the Government. Major activities that occur during this phase include:

  • the Government’s need through market research, development of the acquisition strategy and/or acquisition plan, preparation of a random order of magnitude (ROM) or detailed Independent Government Cost Estimate (IGCE), and budget commitment
  • the Government’s requirements, including preparation of the statement of work (SOW), performance work statement (PWS), or statement of objectives (SOO)
  • the extent of competition based on available competition pools and consideration of the use of set-asides to one or more of the Federal contracting assistance programs managed by SBA
  • the factors, criteria, and method of procurement to be used in the source selection process
  • the provisions and clauses for selection and use in the solicitation.

Initial Planning:

Initial planning begins with the formation of the acquisition team and the acquisition plan. M&Q personnel as members of a source selection team (SST) may be asked to support:

  • Form the Team
  • Market Research/Analysis
  • Requirements Analysis 
  • Acquisition Strategy and Source Selection Plan (including the Systems Engineering Plan)
  • Risk Assessment 
  • Independent Management Reviews 

Market Research:

Market Research (FAR Part 10) is conducted to determine the availability of commercial products and services and to identify and evaluate market capabilities for satisfying a requirement prior to developing new requirements documents, or before issuing solicitations. 

M&Q personnel as members of a source selection team (SST) may be asked to support market research can help define requirements, identify alternatives, and monitor the industry for any new developments that may affect DoD. 

The main resource is SD-5 Market Research https://quicksearch.dla.mil/qsDocDetails.aspx?ident_number=106786 

DAU Continuous Learning Module: 

Contract Strategy: 

Contracting strategy plays a critical role in the overall acquisition strategy for programs as contracts facilitate the progression of a program through the defense acquisition life cycle while encouraging competition and affordability. The development of the contract strategy starts at the early stages of the materiel solution analysis phase and continues through to the operations and support phase. Effectively and efficiently managing contracts is critical in maintaining a programmatic schedule, as the contracting timeline can take years. Contracts should produce measurable performance outcomes that cumulatively contribute to the system Key Performance Parameters (KPPs)/Key System Attributes (KSAs), to their threshold or objective levels. To motivate the contractor to achieve the desired behavior, appropriate contract incentives (including award fee, incentive fee, award term, and cost sharing) need to be developed to promote and facilitate contractor performance. Teh development of a contracting strategy includes:

  • Identification of current initiatives/contracts
  • Review and documentation of current contractor performance
  • Identification of program risks 
  • Documentation of current processes 
  • Determining status of Government furnished property, materials, and facilities 
  • Identification of stakeholder requirements
  • Review of statutory requirements
  • Identify and define desired results 

Statement of Work: The Statement of Work (SOW) defines (either directly or by reference to other documents) all (non-specification) performance requirements for contractor effort.  The SOW should specify in clear, understandable terms the work to be done in developing the goods or services to be provided by a contractor. MIL-HDBK-245 https://quicksearch.dla.mil/Transient/0FA4E2D4706D4B578F7124FEB1D09EBE.pdf 

DAU Continuous Learning Module:

DAU Acquipedia article on SOW/SOO https://www.dau.edu/acquipedia-article/statement-work-performance-work-statement-statement-objectives 

Statement of Objectives: The SOO is a Government prepared document incorporated into the solicitation that states the overall performance objectives. It is used in solicitations when the Government intends to provide the maximum flexibility to each offeror to propose an innovative approach. That portion of a contract that establishes a broad description of the government’s required performance objectives.

Source Selection Plan:

FAR 15.101, “Best Value” section, states that an agency can obtain best value in negotiated acquisitions by using any one or a combination of source selection approaches. The Source Selection Plan (SSP) is a key document which specifies how the source selection activities will be organized, initiated, and conducted. 

The Source Selection Plan (SSP) is a key document that specifies how the source selection activities will be organized, initiated, and conducted. It serves as the guide for conducting the evaluation and analysis of proposals and the selection of source(s) for the acquisition. SSP must clearly and succinctly express the Government’s minimum needs (evaluation factors) and their relative order of importance. The SSP should include one or more of the following evaluation areas:

DAU Continuous Learning Modules and other training:

Acquisition Strategy:

An acquisition strategy is a high-level business and technical management approach designed to achieve program objectives within specified resource constraints. It is the framework for planning, organizing, staffing, controlling, and leading a program. It provides a master schedule for research, development, test, production, fielding and other activities essential for program success, and for formulating functional strategies and plans. M&Q personnel as members of a source selection team (SST) may be asked to support the development of an Acquisition Strategy which should include a Systems Engineering Plan.

The pre-solicitation phase ends with the development of the acquisition strategy that is documented in an acquisition plan. 

DAU Continuous Learning Modules and other training:

Solicitation-Award Activities

The Contract Formation and Source Selection (Award) phase begins at the point when Acquisition Planning is completed, and the method of procurement has been established. The major tasks in this phase include:

  • Solicitation which includes the development of a synopsis, the solicitation preparation, pre-award inquiries, and any amendments to the solicitation.
  • Proposal evaluation includes an administrative review, a technical review, a price/cost evaluation, the setting of a competitive range and negotiations. 
  • Contract award includes the preparation of the award, issue notice, debriefing, and handling of protest.

Solicitation:

A solicitation is any request to submit offers or quotations to the Government. Solicitations under sealed bid procedures are called "invitation for bids." Solicitations under negotiated procedures are called "requests for proposal." Solicitations under simplified acquisition procedures may require submission of either a quotation or an offer. The development of a solicitation often includes industry collaboration. 

Request for Proposal: A Request for Proposal (RFP) is a solicitation used in negotiated acquisition to communicate government requirements to prospective contractors and to solicit proposals. The RFP describes the work to be done, how it will be judged, and the terms and conditions of the proposed agreement. It also tells the supplier how to send their proposal and what forms or paperwork should be included. The RFP process is to let the person or group who asked for the proposals choose the one that best fits their needs and budget.

M&Q inputs to any Request for Proposal should include requirements for:

  • Manufacturing Management Program (AS6500 and MIL-HDBK-896)
  • Quality Management Systems Requirements (AS9100)
  • Manufacturing Risk Assessments 
  • Production Readiness Review
  • Manufacturing and Quality Metrics
  • Key Characteristics and Variation Reduction 
  • First Article Inspection/First Article Testing 
  • Advanced Product Quality Planning (APQP) and Production Part Approval Process (PPAP)
  • Other Considerations (Configuration Management, Parts Management and Control, DMSMS/Obsolescence, Counterfeit Parts, Corrosion Control, GIDEP, ESOH, etc.)

FAR 15.2 Solicitation and Receipt of Proposals and Information https://www.acquisition.gov/far/part-15 

Two important sections of the RFP that M&Q personnel need to support are sections L and M:

  • Section L - Instructions, Conditions, and Other Statements of Offeror's
  • Section M - Evaluation Factors for Award 

Sample M&Q RFP Inputs https://ac.cto.mil/wp-content/uploads/2022/12/MQBOK-AppD-Sample-RFP-Input-11Nov2022.pdf 

Contract / Proposal Evaluation:

The FAR 15.305 Proposal Evaluation requires Government contracting officials to obtain a price that is fair and reasonable to both the contractor and the government. The evaluation phase begins when the government contracting office (CO) receives the offerors’ proposals to the solicitation. In order to determine which proposal will provide the government with the best quality product or service at a fair and reasonable price/cost, CO reviews both the offerors’ technical and business proposals. Determinations are based on a full and fair assessment of each proposal. As the technical experts, the Program Office reviews and evaluates the technical proposal. Then, along with CO’s guidance and assistance, the Program Office reviews the business proposals.

Proposals shall be evaluated based on the factors/subfactors below, to determine if the offeror provides a sound, compliant approach that meets the requirements of the Statement of Work (SOW) and Data Item Descriptions (DID) and demonstrates a thorough knowledge and understanding of requirements and associated risks. The technical and management proposals must address subfactors in sufficient detail. The Government evaluates proposals in order to select the proposal providing the best value to the Government. DCMA Manual 2501-01 covers Contract Receipt and Review procedures. https://www.dcma.mil/Portals/31/Documents/Policy/MAN_2501-01r_(C1)_(20220201)_V508C_03012023.pdf

Typical Evaluation factors can include:

  • Cost/Price (Reasonableness, Realism, and Affordability)

  • Technical (Management Approach, Technical Capability, Transition Plan, and Small Business Utilization) 

  • Past Performance (Past Contracts, Relevance of past contract to this effort, and Performance Confidence)

The following evaluations tools are available the Program Office:

Pre-Award Surveys are thorough evaluations of a contractor's capability, capacity, and financial stability to meet the proposed contract requirements. 

Contract Negotiation:

 The Government has a vested interest in the long-term success and survival of government contractors. Win/win negotiations enhance competition by encouraging more firms to do business with the Government, thus increasing competition and quality while reducing contract costs. Numerous DCMA Instructions provide guidance on negotiation procedures. M&Q personnel may be asked to support the activities listed below.

Fact Finding:

  • Obtain information on contractor position
  • Identify assumptions
  • Clarify matters affecting costs
  • Resolve inconsistencies

Negotiation Preparation:

  • Be prepared. Know the strengths and weaknesses of both sides; there is no substitute for good preparation
  • Organize the negotiation team
  • Identify issues and objectives
  • Research contractor’s probable approach
  • Assess each side's bargaining power
  • Establish priorities and positions
  • Establish a price range
  • Develop the negotiation approach
  • Develop the overall plan

Negotiation Characteristics of Win/Win Outcomes:

  • Resolve conflicts
  • Obtain short-term and long-term satisfaction
  • Establish cordial relations
  • Combine sincere efforts to satisfy the other side and solve problems
  • Display win/win attitudes
  • Avoid deception
  • Be persuasive
  • Be respectful and polite
  • Win agreements instead of arguments
  • Use common sense and be open to negotiate all issues

Bargaining Techniques:

  • Aim high to produce better outcomes
  • Give yourself room to compromise. Concessions are always necessary. Open with a low (but justifiable) offer; you can always raise it later
  • Do not volunteer weaknesses
  • Satisfy the other side’s non-price issues. Price is never the only issue
  • Use concessions wisely. When giving, ask for something in return. Concede slowly, and in small amounts
  • Put pressure on the other side. Believe in the unknown pressures facing the other side. Resist artificial pressures, fancy offices, and credentials
  • Use the power of patience
  • Be willing to walk away from or back to negotiations, deadlocks cannot always be avoided
  • Say it right. Be cordial and business-like. State things in a win/win manner

Contract Award:

Occurs when the contracting officer (CO) has signed and distributed the contract to the contractor.

Post-Award Activities - Contract Administration / Closeout

Contract Administration involves those activities performed by government officials after a contract has been awarded to determine how well the government and the contractor performed to meet the requirements of the contract. It encompasses all dealings between the government and the contractor from the time the contract is awarded until the work has been completed and accepted or the contract terminated, payment has been made, and disputes have been resolved. As such, contract administration constitutes that primary part of the procurement process that assures the government gets what it paid for. 

Primary functions of Contract Administration include:

  • Assignment of contract administration functions (often DCMA)
  • Development of a contract administration plan
  • Post-award orientation
  • Monitoring of contractor performance (cost, schedule, and technical)
  • Accept product
  • Contractor payments 

Guidance and other resources:

DAU Continuous Learning Modules and other training:

Contract Surveillance (monitoring performance):

The purpose of contract administration is to ensure that the contractor performs in accordance with the terms and conditions of the contractual agreement (surveillance). DoD contractor surveillance requirements and activities are required by the FAR/DFAR and by many DoD, Service, and agency regulations, policies, and guidance documents. FAR 42.3 Identifies Contract Administration Office functions, and FAR 42.11 identifies requirements for Production Surveillance and Reporting. DCMA INST 2303-01 provides guidance on surveillance activities which include the need for detailed planning, execution, and reporting. Surveillance activities may be performed by program office or DCMA personnel.  M&Q personnel from the Program Office or DCMA may be involved in the following surveillance activities:

  • Perform Contract Administration Service (CAS) Functions
  • DCMA Surveillance Support at Industry Sites (may require a MOA or LOD)
  • Monitor and Track Risks
  • Participate in Program Reviews

DAU Continuous Learning Modules and other training:

Deliverables:

Contract deliverables establish and track both contractual and noncontractual commitments that must be honored as part of negotiations and contractual agreements between businesses and suppliers or customers based on contract intent. Contract Deliverables includes Hardware Software, Services, as required by the contract. Deliverables also includes Contract Data Requirements List (CDRL) and Data Item Descriptions (DID). The Defense Acquisition University (DAU) has resources on contract deliverables. The CDRL is a standard format for identifying data requirements in a contract and solicitation. A DID is a document that defines the data a contractor is required to provide. See M&Q DIDs for typical contract deliverables. 

Contract Payments:

M&Q personnel may be asked to support review of progress payments and work performed under the contract.

Progress Payments: Payments made to a prime contractor during the life of a fixed-price type contract on the basis of a percentage of incurred total costs or total direct labor and material costs. Payments are made on the basis of costs incurred by the contractor as work progresses under the contract. Under this arrangement, the contractor is typically paid between 7 and 30 days of submitting an approved request to the Contracting Officer. This form of contract financing does not include:

  • Payments based on the percentage or stage of completion accomplished
  • Payments for partial deliveries accepted by the Government
  • Partial payments for a contract termination proposal
  • Performance-based payments. (FAR 32.102(b)) https://www.dau.edu/acquipedia-article/progress-payments 

Performance-based Payments: Method of providing financing to contractors performing under fixed-price contracts in which payments are based on achievement of specific events or accomplishments that are defined and valued in advance by the parties to the contract. Performance-based payments (PBPs) are a customary method of contract financing that may be available under fixed-price contracts, except for contracts awarded using Sealed Bidding procedures. PBPs differ from the more traditional progress payments based on costs because these contract financing payments are made on the basis of the contractor’s achievement of objective, quantifiably measurable events, results or accomplishments that are defined and valued in the contract prior to performance.

Guidance and other Resources:

Contract Closeout:

Once the procuring or administrative contracting officer confirms that all receivables have been delivered/completed (often with the assistance of a contracting officer representative), the contracting officer begins the closeout process. Per FAR 4.804-4, a contract is physically complete when:

  • The contractor has completed the required deliveries, and the Government has inspected and accepted the supplies
  • The contractor has performed all services, and the Government has accepted these services
  • All option provisions, if any, have expired
  • The Government has given the contractor a notice of complete contract termination
  • M&Q personnel may be asked to support the evaluation that all deliverables were inspected and accepted. 

DAU Continuous Learning Modules and other training:

Guidance and other Resources:

M&Q Contract Administration Functions

42.302 -- Contract Administration Functions

PFI 242 - Contract Administration Functions

Contract Administration includes all actions required to be performed at a contractor's plant for the benefit of the government, which are necessary to managing and providing oversight of the performance of a contract or in support of the buying offices, system/project managers, and other organizations, including quality assurance (QA), engineering support, production surveillance, pre-award surveys, mobilization planning, contract administration, property administration, industrial security, and safety.

The following is a list of CAS Functions that M&Q personnel may be required to accomplish or support:

(3) Conduct post-award orientation conferences.

(4) Review and evaluate contractors’ proposals when negotiation will be accomplished by the contracting officer, furnish comments and recommendations to that officer.

(5) Negotiate forward pricing rate agreements.

 (13) Review and approve or disapprove the contractor’s requests for payments under the progress payments or performance-based payments clauses.

 (16) Ensure timely notification by the contractor of any anticipated overrun or underrun of the estimated cost under cost-reimbursement contracts.

(17) Monitor the contractor’s financial condition and advise the contracting officer when it jeopardizes contract performance.

(18) Analyze quarterly limitation on payments statements and take action to recover overpayments from the contractor.

(21) For classified contracts, administer those portions of the applicable industrial security program delegated to the CAO.

(22) Issue work requests under maintenance, overhaul, and modification contracts.

(23) Negotiate prices and execute supplemental agreements for spare parts and other items selected through provisioning procedures when prescribed by agency acquisition regulations.

(24) Negotiate and execute contractual documents for settlement of partial and complete contract terminations for convenience, except as otherwise prescribed.

(25) Negotiate and execute contractual documents settling cancellation charges under multiyear contracts.

(27) Perform property administration.

(28) Perform necessary screening, redistribution, and disposal of contractor inventory.

(29) Issue contract modifications requiring the contractor to provide packing, crating, and handling services on excess Government property. When the ACO determines it to be in the Government’s interests, the services may be secured from a contractor other than the contractor in possession of the property.

(30) When contractors request Government property—

(i) Evaluate the contractor’s requests for Government property and for changes to existing Government property and provide appropriate recommendations to the contracting officer;

(ii) Ensure required screening of Government property before acquisition by the contractor;

(iii) Evaluate the use of Government property on a non-interference basis, Use and Charges;

(iv) Ensure payment by the contractor of any rental due; and

(v) Modify contracts to reflect the addition of Government-furnished property and ensure appropriate consideration.

(31) Perform production support, surveillance, and status reporting, including timely reporting of potential and actual slippages in contract delivery schedules.

(32) Perform pre-award surveys.

(33) Advise and assist contractors regarding their priorities and allocations responsibilities and assist contracting offices in processing requests for special assistance and for priority ratings for privately owned capital equipment.

(34) Monitor contractor industrial labor relations matters under the contract; apprise the contracting officer and, if designated by the agency, the cognizant labor relations advisor, of actual or potential labor disputes; and coordinate the removal of urgently required material from the strikebound contractor’s plant upon instruction from, and authorization of, the contracting officer.

(35) Perform traffic management services, including issuance and control of Government bills of lading and other transportation documents.

(36) Review the adequacy of the contractor’s traffic operations.

(37) Review and evaluate preservation, packaging, and packing.

(38) Ensure contractor compliance with contractual quality assurance requirements.

(39) Ensure contractor compliance with contractual safety requirements.

(40) Perform engineering surveillance to assess compliance with contractual terms for schedule, cost, and technical performance in the areas of design, development, and production.

(41) Evaluate for adequacy and perform surveillance of contractor engineering efforts and management systems that relate to design, development, production, engineering changes, subcontractors, tests, management of engineering resources, reliability and maintainability, data control systems, configuration management, and independent research and development.

(42) Review and evaluate for technical adequacy the contractor’s logistics support, maintenance, and modification programs.

(43) Report to the contracting office any inadequacies noted in specifications.

(44) Perform engineering analyses of contractor cost proposals.

(45) Review and analyze contractor-proposed engineering and design studies and submit comments and recommendations to the contracting office, as required.

(46) Review engineering change proposals for proper classification, and when required, for need, technical adequacy of design, producibility, and impact on quality, reliability, schedule, and cost; submit comments to the contracting office.

(47) Assist in evaluating and make recommendations for acceptance or rejection of waivers and deviations.

(48) Evaluate and monitor the contractor’s procedures for complying with procedures regarding restrictive markings on data.

(49) Monitor the contractor’s value engineering program.

(50) Review, approve or disapprove, and maintain surveillance of the contractor’s purchasing system.

 (56) Maintain surveillance of flight operations. (Note: Government Flight ops is often a QA function.)

 (58) Ensure timely submission of required reports.

(60) Cause release of shipments from contractor’s plants according to the shipping instructions. When applicable, the order of assigned priority shall be followed; shipments within the same priority shall be determined by date of the instruction.

(61) Obtain contractor proposals for any contract price adjustments resulting from amended shipping instructions. Review all amended shipping instructions on a periodic, consolidated basis to ensure that adjustments are timely made. Except when the ACO has settlement authority, the ACO shall forward the proposal to the contracting officer for contract modification. The ACO shall not delay shipments pending completion and formalization of negotiations of revised shipping instructions.

(62) Negotiate and/or execute supplemental agreements, as required, making changes in packaging subcontractors or contract shipping points.

(64) Negotiate and execute one-time supplemental agreements providing for the extension of contract delivery schedules up to 90 days on contracts with an assigned Criticality Designator of C. Notification that the contract delivery schedule is being extended shall be provided to the contracting office. Subsequent extensions on any individual contract shall be authorized only upon concurrence of the contracting office.

(65) Accomplish administrative closeout procedures.

 (67) Support the program, product, and project offices regarding program reviews, program status, program performance and actual or anticipated program problems.

(68) Monitor the contractor’s environmental practices for adverse impact on contract performance or contract cost, and for compliance with environmental requirements specified in the contract. ACO responsibilities include--

(i) Requesting environmental technical assistance, if needed;

(ii) Monitoring contractor compliance with specifications requiring the delivery or use of environmentally preferable products, energy-efficient products, products containing recovered materials, and biobased products. This must occur as part of the quality assurance procedures; and

(iii) As required in the contract, ensuring that the contractor complies with the reporting requirements relating to recovered material content utilized in contract performance.

Contracting Resources and Guidance 

The following is a list of Policy, Guidance, Regulations and other documents that support the contracting process. This list is not all inclusive. 

Manufacturing and Quality and other Data Item Descriptions (DIDs)
Contract Pre-award Evaluations/Surveys  

A Contract Pre-award Survey is a study of financial, organizational, and operational status made prior to contract award to determine a prospective contractor's responsibility and eligibility for government procurement. The purpose of a pre-award survey is to obtain information not available to the contracting officer to make a responsibility determination prior to contract award. 

Defense Contract Management Agency (DCMA) conducts nearly all pre-award surveys required by government buying activities. The process begins with a buying activity's request for a survey and concludes with a procuring contracting officer's (PCO) decision based on a recommendation by a DCMA Contract Management Office (CMO) survey team. 

A pre-award survey can focus on virtually every facet of the contractor's business operations from technical capability to financial stability, from quality assurance to plant safety. In a sense, the survey process is the contractor’s opportunity to provide evidence (i.e., Plan of Performance) that they can successfully fulfill the terms of the contract. Listed below are some of the factors that are often the focus of pre-award surveys. 

  • Technical Capability
  • Production Capability
  • Quality Assurance
  • Financial Stability
  • Accounting Practices
  • Government Property Control
  • Transportation and Packaging
  • Security (Plant and Cyber)
  • Plant Safety
  • Environmental/Energy Compliance
  • Flight Operations/Safety
Guidance and other Resources:

 The following DCMA pre-award evaluations are available the Program Office:

Work Breakdown Structure (WBS) 

The Work Breakdown Structure (WBS) provides a consistent and visible framework for defense materiel items and contracts which include Memorandums of Agreement (MOA), Military Interdepartmental Purchase Requests (MIPR) and other government and industry agreements completed to accomplish a program. MIL-STD-881 offers uniformity in definition and consistency of approach for developing all levels of the WBS. Generating and applying uniform work breakdown structures improves communication in the acquisition process. It also provides direction to industry and supporting government entities in extending their contract (industry or government entity executing an agreement equivalent to a contract) and Program (government) work breakdown structures.

The WBS is a product-oriented family tree of hardware, software services and data which results from systems engineering efforts during development and production of an item. How far down into the product structure you go depends on how far you need to go for planning, execution, and tracking. The WBS displays and defines the product(s) and relates the elements of work to each other and the end product, and completely defines the program. 

There are two types of work breakdown structures (WBSs):

  • PROGRAM WBS encompasses entire program and consists of at least three levels of the program:
    • Prepared by the Program Office
    • This is a “Government Document”
    • Developed IAW MIL-STD-881
    • Defines the Product and Relates Elements of Work to Each Other
    • Completely Defines the Program
    • Used by the government activity to define the contract WBS
    • Used by contractors to develop and extend a contract WBS requirements
  • CONTRACT WBS is for a specific contract. It is developed by the contractor in accordance with the statement of work (SOW). It is the final government approved WBS for reporting purposes: 
    • Begins at Level 3 of the Program WBS
    • Includes all the elements for the products under contract
    • Contract work statement should provide reporting requirements
    • Cost Reporting
    • Schedule Reporting

Sample WBS shown below:

The WBS is used by many functional activities and plays a key role in developing and tracking cost and schedule elements:

 

  • Program Management would look at Cost & Schedule Management down to the Work Package level
  • Engineering would look at Design and Design to Cost estimating and tie into performance metrics
  • Manufacturing/QA would look Manufacturing/QA for planning, estimating and tracking
  • Test & Evaluation would look at each work package for testing opportunities, scheduling and other information
  • Finance/Budgeting would look at each work package for cost
  • Logistics would look at each work package for sustainment considerations, cost and schedule
The WBS supports the integration of many activities:
  • Defining the Work
  • Scheduling the Work
  • Allocating Budget to the Work Tasks
     
Thus, if you can define the work (WBS) you can:
  • Measure the work
  • Estimate how long it will take to do the work
  • Schedule the work
  • Estimate the cost of the work
  • Track the progress of the work:
    • Cost
    • Schedule
 
Guidance and other Resources:
CREATED:
BY:
Content Author No Reply
LAST MODIFIED:
BY:
Content Author No Reply
View Resource
Community Resource / Manufacturing and Quality
Advanced Manufacturing / Additive Manufacturing

Manufacturing.Gov defines Advanced Manufacturing as "the use of innovative technologies to create existing products and the creation of new products. Advanced manufacturing can include production activities that depend on information, automation, computation, software, sensing, and networking." 

Advanced Manufacturing (AM) is a broad term that includes the use of advanced and innovative technologies and methods to improve and enhance competitiveness within the manufacturing sector. The DoD is adopting AM in order to incorporate cutting-edge advancements, such as artificial intelligence and composite materials, in order to optimize every aspect of the DoD supply chain.

Advanced manufacturing technologies are divided into the three areas listed below:

  • Efficient production involves design, simulation, physical and computer modelling, advanced production technologies, and control techniques. The emphasis is on simultaneous rather than sequential engineering. Relevant production technologies include rapid prototyping, near net shape manufacturing, and precision casting, machining and joining techniques.
  • Intelligent production involves the use of ICT in manufacturing and related logistics systems. As well as production orientated intelligent machines, cells and production lines, the philosophy involves implementing systems for the extended life and optimal use of production facilities through efficient monitoring, maintenance and repair strategies.
  • Effective organization involves the efficient co-ordination and exploitation of manufacturing resources. This encompasses both physical resources and knowledge. Relevant topics include virtual tendering and enterprises, shared facilities and resources, novel organizations, incubation units, knowledge management and trading, and electronic commerce. Emphasis in this area is on the use of technology to enhance the involvement and capability of SMEs as well as large organizations.

Advanced Manufacturing is a concept that encompasses the areas listed below, and there is not a consensus on what Advanced Manufacturing is:

  • Additive Manufacturing: 3D printing (et. al.), is a process used to fabricate a physical object from a three-dimensional (3D) digital model, typically by laying down and bonding a large number of successive thin layers of materials.
  • Advanced Materials: Any new or significantly improved material that provides a distinct advantage in (physical or functional) performance when compared to conventional materials.
  • Advanced Composite Materials: Materials that are generally characterized by unusually high strength fibers with unusually high stiffness, or modulus of elasticity characteristics, compared to other materials, while bound together by weaker matrices.
  • Robotics and Automation: A robot is a programmable machine capable of carrying out tasks autonomously, in the case of Robotnik, or semi-autonomously. On the other hand, automation refers to a broader concept that involves using technology to perform tasks automatically, without direct human intervention. Automation is the process of using technology to perform tasks without direct human intervention, while robotics is the design and use of robots to perform tasks. Automation is a broader concept that includes robotics, but the two are often used together to improve productivity, quality, and safety
  • Laser, Machining and Welding:  Laser technologies that enable manufacturers to create products with high accuracy and efficiency for cutting, engraving, marking, welding, or other industrial applications. Advanced machining processes are those machining processes that are different from current conventional machining processes. Advanced welding is the use of new techniques include plasma welding, which has four times the energy density of TIG welding, and electron beam welding, which uses an electron gun to melt and bond workpieces together. Other advanced welding processes include friction stir welding, explosive welding, ultrasonic welding, and laser welding
  • Nonotechnology:  The study and manipulation of matter at the nanoscale, which is a size range of about 1 to 100 nanometers. At this scale, matter exhibits unique physical, chemical, and biological properties.
  • Network and IT Integration:  The process of connecting different systems, machines, and departments within a manufacturing facility through a network, allowing them to seamlessly communicate and exchange data, thereby optimizing operations and improving efficiency across the entire production process; essentially, it's about linking different parts of a manufacturing operation to work together cohesively, often utilizing technologies like industrial automation, IoT devices, and enterprise resource planning (ERP) systems to achieve manufacturing efficiency.
Advanced Manufacturing (Industry 4.o, IIoT, and FoF)

Advanced Manufacturing (AM):  AM is a broad term that refers to the use of innovative technologies and processes to improve the efficiency and productivity of manufacturing. It can include the creation of new products, refinement of existing products, and production activities that improve the quality and process of manufacturing. Advanced Manufacturing (AM) is defined as the innovation of improved manufacturing methods for manufacturing existing products, and the production of new products enabled by advanced technologies. 

Recent advances in areas such as automation, data science, artificial intelligence, machine learning, biotechnology, and materials science, combined with urgent technical challenges in economy-wide decarbonization, healthcare, and national security are creating new opportunities for advanced manufacturing. In order to compete globally, the United States must leverage and protect its technology leadership through rapid development and implementation of innovative manufacturing technologies. The National Strategy for Advanced Manufacturing has the following objectives: 

  • Enable Clean and Sustainable Manufacturing to Support Decarbonization
  • Accelerate Manufacturing Innovation for Microelectronics and Semiconductors
  • Implement Advanced Manufacturing in Support of the Bioeconomy
  • Develop Innovative Materials and Processing Technologies
  • Lead the Future of Smart Manufacturing

Source: National Strategy for AM, National Science and Technology Council https://www.manufacturingusa.com/reports/national-strategy-advanced-manufacturing 

Advanced Manufacturing is a concept that encompasses the areas listed below:

  • Smart Manufacturing, Industry 4.0, Factories of the Future, and the Industrial Internet of Things (IIoT)
  • Digital Engineering and Digital Threads
  • Additive Manufacturing
  • Advanced Composite Materials 
  • Robotics and Automation 
  • Laser and Machining and Welding 
  • Nonotechnology 
  • Network and IT Integration 

The Advanced Manufacturing Enterprise (AME) Subpanel of the Joint Defense Manufacturing Technology Panel (JDMTP) is to address the technologies and practices to fully realize government and industry-wide use of manufacturing readiness tools and processes, including design for producibility and sustainability across the Army, Navy, Air Force, Defense Logistics Agency, Missile Defense Agency, Office of Secretary of Defense, industry, and academia. 

The AME Subpanel encompasses the technologies, processes, and practices that foster rapid, superior execution of manufacturing enterprises across the life cycle. This includes:

  • Model-based tools and approaches that optimize producibility during early design and support standard data environments for life cycle support
  • Network centric manufacturing capabilities to facilitate resilient and adaptable supply chains
  • Intelligent manufacturing planning and factory execution
  • Artificial Intelligence (AI), robotics, machine learning and automation
  • Modeling and simulation capabilities advancing the above business practice

Guidance and other Resources:

Smart Manufacturing/Industry 4.0: Modern or Smart Factories are utilizing automation and cognitive computing seamlessly connect to smart factories, supply chains are entering into a fourth industrial revolution known as Industry 4.0. This transformation, through advanced digital technologies across engineering and manufacturing, is set to bring the U.S. manufacturing ecosystem to the forefront of modernization — and with it, a demand for a sustained pipeline of talent and strong domestic manufacturing centers. Industry 4.0 allows manufacturers to identify and address issues before they become major problems, improving product quality and reducing the need for costly recalls.

Industrial Internet of Things (IIoT): The Industrial Internet of Things (IIoT) provides a way for increased data collection and boosts in efficiency and overall production effectiveness. IIoT are devices with dedicated purpose, microprocessors, and Internet Protocol capabilities; one such example being smart sensors. IIoT  coined the term in 2012 and IIoT is a key component of Industry 4.0.

Industrial IoT enables organizations to get a wealth of actionable data from their operations. When properly aggregated and analyzed, the data helps them better control operations, with the potential to:

  • Improve worker safety
  • Improved efficiency by identifying bottlenecks and other production problems so as to increase productivity
  • Reduced costs by increase production uptime and improved maintenance with predictive maintenance of machinery
  • Improved product quality and customer satisfaction
  • Improved supply chain management with better communication up and down the supply chain and greater visibility and control over supply chain activities
  • Improved regulatory compliance
  • Reduced time to market and accelerated response times with real-time collection and processing of operational data

IIoT connects machines and devices in industries such as manufacturing, transportation, oil and gas, power generation and transmission, mines, and ports. Commercial, enterprise, or consumer IoT—also simply known as IoT—is used to describe connected devices within homes and office spaces, such as cameras, badge readers, and HVAC control systems.

Failure of IIoT can have catastrophic consequences, creating high risk and potentially life-threatening situations. Downtime of other IoT devices may result in inconveniences, but it does not usually cause emergency situations.

Note: You can find many IIoT guides and references by doing a web search.

Factory of the Future: Factories of the Future is primarily a European Union Public-Private Partnership (PPP) started in 2017 to support the transformation of manufacturing in Europe through pre-competitive research, development and innovation projects. The PPP’s overall objective is to increase the European Union’s industrial competitiveness and sustainability through research, development and innovation, with the development of new knowledge-based production technologies and systems across multiple sectors. This objective should be achieved through a comprehensive approach which addresses the factory shop floor, the value-network, and the eco-system. The overall scope of the FoF PPP can be described as follows:

  • Industrial automation, mechanical and electrical machinery and robotics 
  • Industrial software for plant design, instantiation and management. 
  • Competitive and sustainable production technologies. 
  • Factories eco-systems including material flows and logistics.

 Key priorities of PPP include:

  • Agile value networks: Lot-size one and distributed manufacturing 
  • Excellence in manufacturing: Advanced manufacturing processes and services for zero-defect processes and products
  • Human factor: Human competencies in synergy with technological assets
  • Sustainable value networks: Manufacturing in a circular economy
  • Interoperable digital manufacturing platforms: Connecting manufacturing services, 

Guidance and other Resources:

See the FoF PPP Report at https://effra.eu/wp-content/uploads/2023/12/fof_cppp_progress_monitoring_report_for_2017_online-1.pdf

Manufacturing USA / other Government Advanced Manufacturing Organizations 

Description automatically generated" width="171" height="113" class="align-right" loading="lazy">Manufacturing USA (MFG USA) is a network of research institutes that focuses on developing manufacturing technologies through public-private partnerships among U.S. industry, universities, and federal government agencies. Manufacturing USA has a network of 16 institutes focusing on Advanced Manufacturing Technology Leadership. Manufacturing USA has several key initiatives to include projects to improve domestic manufacturing processes to make them more innovative, competitive and resilient. Another initiative is manufacturing workforce development. this imitative focuses on identifying and providing access to emerging skills and training needed to meet future requirements.  Finally, Manufacturing USA is working to foster the development of energy-efficient and clean-energy technologies that will reduce manufacturing cost and the introduction of pollutants into our ecosystem.

Institute

Technology

Location
Advanced Functional Fabrics of America (AFFOA)  Textiles Cambridge, Mass.
Advanced Regenerative Manufacturing Institute (ARMI/BioFabUSA) Regenerative medicine, tissue engineering Manchester, NH
Advanced Robotics for Manufacturing (ARM) Robotics Pittsburg, PA
American Institute for Manufacturing Integrated Photonics (AIM Photonics)

Photonic integrated circuits

 Rochester, NY 
Bioindustrial Manufacturing and Design Ecosystem (BioMADE) Biomanufacturing St. Paul, MN
Clean Energy Smart Manufacturing Innovation Institute (CESMII) Smart Manufacturing Los Angeles, CA
Cybersecurity Manufacturing Innovation Institute (CyManII) Cybersecurity Sam Antonio, TX
Flexible Hybrid Electronics Manufacturing Innovation Institute (NextFlex) Flexible Electronics San Jose, CA
Institute for Advanced Composites Manufacturing Innovation (IACMI) Advanced Composite Materials Knoxville, TN
Lightweight Innovations for Tomorrow (LIFT) Lightweight Materials  Detroit, MI
Manufacturing times Digital (MxD) Digital Manufacturing  Chicago, IL
National Additive Manufacturing Innovation Institute (AmericaMakes) 3D Printing, Additive Manufacturing  Youngstown, OH
National Institute for Innovation in Manufacturing Biopharmaceuticals (NIIMBL) Biopharmaceuticsls Newark, DE
Next Generation Power Electronics Institute (PowerAmerica) Wide-bandgap Semiconductors Raleigh, NC
Rapid Advancement in Process Intensification Deployment (RAPID) Process Engineering and Modularization New York, NY
Reducing EMbodied-energy And Decreasing Emissions (REMADE) Remanufacturing  Rochester, NY

Manufacturing USA homepage https://www.manufacturingusa.com/ 

Other Government Advanced Manufacturing Organizations:

Note: You can contact each of these organizations to learn more about their efforts and how you can participate or benefit from their activities.

Digital Engineering 

Digital Engineering (DE), Digital Twin, Digital Threads, and Digital Models: 

Digital Engineering: Digital Engineering can be used to support all of the Systems Engineering functions. MIL-HDBK-539 Digital Engineering and Modeling Practices defines digital engineering as “an integrated, computation-based approach that uses authoritative sources of system data and models as a continuum across disciplines to support lifecycle activities.” DE is a cutting-edge approach that uses authoritative sources of system data and models throughout the development and life of a system. Digital engineering harnesses computational technology, modeling, analytics, and data sciences to update traditional systems engineering practices. In the face of increasing global challenges and dynamic threat environments, digital engineering is a necessary practice to support acquisition.

DoDI 5000.97-Digital Engineering calls for the use of digital engineering methodologies, technologies and practices across the life cycle of defense acquisition programs.  Further, the document:

  •  Mandates the incorporation of digital engineering for all new programs (exceptions can be granted by decision authority)
  •  Directs Components to use digital engineering practices in requirements, cost, business and sustainment.
  • Calls for the replacement of documents with the use of digital models as the primary means of communicating system information. 
  • During program planning and contracting, requires that appropriate data rights be obtained.
  • Singles out DAU as providing workforce training on digital engineering.

The diagram below, from the instruction, captures the digital Engineering framework:

The figure describes the DoD Digital Engineering Ecosystem

Digital Twin: Every product produced, and every process executed is unique. There can be thousands, of key input variables used to describe products, assets, entire lines, and processes.  A Digital Twin is a virtual replica of a physical object, manufacturing process, or plant that is designed to capture, map, and structure process variables into a continuously updated database. This database can be used by an organization to monitor, analyze, design, or optimize the process without having to go out into the field or do costly trials on the physical equipment. By making this data more readily available in a digital environment, teams can use data in other applications, models, or third-party programs to make meaningful discoveries.

Digital Thread: A digital thread is a framework that connects data about a product throughout its lifecycle, from design to disposal. It uses a variety of technologies, including computer-aided design (CAD), product lifecycle management (PLM), and Internet of Things (IoT) sensors, to collect and share data. A manufacturing digital thread is designed to expand upon a digital twin. Put simply, the digital thread captures digital twin data as they evolve. As manufacturers evolve their processes and their digital twins adapt to new settings or recipe changes, the digital thread encapsulates the link between these evolutions. 

The digital thread should seamlessly advance the controlled interplay of technical data, software, information, and knowledge in the digital engineering ecosystem. Digital threads are used to connect authoritative data and orchestrate digital models and information across a system’s life cycle. The digital thread informs decision makers throughout a system’s life cycle by providing the capability to access, integrate, and transform data into actionable information. The digital thread should support feedback over the life cycle.

Example Data Elements or Digital Artifacts

Engineering Design Data

Technical Product Data

Manufacturing and Quality Data

Enterprise Data
  • Producibility Analysis
  • CAD Data
  • Modeling and Simulation Data
  • Special Tooling Data
  • Special Inspection Equipment Data
  • Digital Drawings
  • Digital Models
  • Specifications
  • Standards
  • Critical Manufacturing Processes
  • Configuration Data
  • Interface Management Data
  • Analytical Data
  • Bill of Material
  • Manufacturing Floor Layout
  • Production Line Data
  • Pilot Line Data
  • LRIP/FRP Data
  • Industrial Engineering Data
  • Production Data 
  • Machining Instructions
  • Customer Demand Data
  • Rates and Quantities Data
  • Work Breakdown Structure Data
  • Supplier Data
  • FRACAS Data 
  • Test Plans
  • Schedules
  • Product Support Strategy

Digital Models Include:

  • Requirements Model
  • Structural Model
  • Functional Model
  • Architecture Model
  • Business Process Model 
  • Enterprise Model
  • Human Performance Model
  • Product Life Cycle Model 

DAU Continuous Learning Modules and other training: 

Guidance and other Resources:

Digital Engineering Body of Knowledge (DEBoK)

The Digital Engineering Body of Knowledge (DEBoK) will serve as a reference for the DoD engineering community to use in implementing digital engineering practices starting with systems engineering and expanding to specific disciplines, engineering domains and specialty areas. The BoK will store collective data, information and knowledge on digital engineering. Members of the government, industry and academia working within this space will be able to contribute to the DEBoK and build their digital engineering solutions based on collective knowledge. Access the DoD DE BoK briefing at https://ndiastorage.blob.core.usgovcloudapi.net/ndia/2021/systems/Wed_23770_Zimmerman_Davidson_Salvatore.pdf 

As a best practice, when conducting early M&Q engineering analysis, the technical team should consider DE principles, methods, and tools. DE best practices and tools are defined in the DE Body of Knowledge (DEBoK). https://de-bok.org/ 

The DEBoK is also available to DoD Common Access Card users at the Defense Technical information Center (DTIC) website: 

https://www.dodtechipedia.mil/dodwiki/pages/viewpage.action?page Id=760447627 

Digital Engineering Strategy and other Resources

DoD’s Digital Engineering Strategy provides guiding principles and promotes consistency in engineering processes through the use and reuse of digital tools, models, and curated data throughout the program’s life cycle. As a best practice, the technical team should consider M&Q digital data requirements (e.g. factory floor modeling and simulation, digital technical data packages and work instructions, digital data in supply chains) during early establishment and development of the digital thread.

Image removed.Digital Engineering: An integrated digital approach that uses authoritative sources of systems’ data and models as a continuum across disciplines to support lifecycle activities from concept through disposal. Access to Digital Engineering Fundamentals can be found at the following urls:

https://www.cto.mil/wp-content/uploads/2023/06/Dig-Eng-Fundamentals-2022.pdf 

https://ac.cto.mil/wp-content/uploads/2019/06/DE-Fundamentals.pdf 

Digital Engineering Ecosystem: The interconnected infrastructure, environment, and methodology (process, methods, and tools) used to store, access, analyze, and visualize evolving systems’ data and models to address the needs of the stakeholders. End-to-end digital enterprise. 

Digital Artifact: An artifact produced within, or generated from, the digital engineering ecosystem. These artifacts provide data for alternative views to visualize, communicate, and deliver data, information, and knowledge to stakeholders.

Guidance and other Resources:

Modeling and Simulation for Manufacturing 

A model is a physical, mathematical, or logical representation of a system, entity, phenomenon, or process. Manufacturing models include plant diagrams, flow charts, 5Ms chart, 

A simulation is the implementation of a model over time, showing how the model works, and can be live, virtual, or constructive. Manufacturing simulations include 

The use of models and simulations in engineering is well recognized. Simulation technology is an essential tool for engineers in all application domains. A digital model represents an actual or conceptual system that involves physics, mathematics, or logical expressions. A simulation is a method for implementing a model over time. Together models and simulations allow the Department to vet potential requirements prior to the Request for Proposal release, assess engineering change orders or program upgrades, etc. M&S can be used to assess and optimize resource usage, examine process changes, support supply-chain management routing and inventory quantities, business decisions, etc.

M&Q managers can be involved with many aspects of M&S as they apply to factory floor environments. Some of these environments are listed below.

  • Manufacturing M&S can be conducted at many levels:
    • Enterprise: Demand Forecasting and Management, Material Planning, Production Planning 
    • Plant: Facility Design, Capacity Analysis, Material and Supply Chain, Scheduling, Flow Analysis and Optimization
    • Line: Line Design, Flow Analysis and Optimization, Capacity Analysis and Management, Production Control, Throughput, Line Balancing, Work-in-Progress, Yields, Quality, Costs 
    • Workstation/Cell: Process Planning, Workstation Optimization, and Capacity Analysis
    • Individual Operation: Work Measurement, Cycle Time, Yield, Quality, Routing and Sequencing, Process Analysis, Tool and Fixture Design
    • Product Concept: Design for Manufacturing and Assembly,  
  • M&S software can be used to support the following M&Q activities:

    • Facility/Plant Design: Allows plant engineers to design and assess different plant models before laying any brick or concrete. These models can create the entire plant model to include electrical, mechanical, HVAC, machines, etc.  

    • Process Planning:  Allows production engineers to layout initial factory floor designs, modify and simulate them in order to achieve more efficient flow, analyze capacity and constraints. There are four basic layout patterns to include Product Layout, Process Layout, Fixed Position, and Combination. 

    • Production Planning and Control:  Allows production engineers to plan capacity requirements and availability, scheduling of material and work-centers, utilization of equipment and manpower, manage and minimize inventory, and schedule discrete production activities and monitor work as it progresses. 

    • Shop Floor Execution: Allows production managers to create, manage, and simulate real-time shop floor controls within the manufacturing execution system (MES).

    • Robotics: Allows production engineers to automate complex fabrication and assembly processes providing for faster and more efficient operations while improving quality, safety, and profit.

    • Supply Chain Management: Allows supply chain managers to create a realistic representation of the supply chain that can help businesses make intelligent, data-backed decisions to improve operations and improve many supply chain functions:

      • Demand Planning

      • Procurement

      • Vendor Management

    • Ergonomic Design and Simulation: Allows production engineers to design workstations and plant features while focusing on the man-machine interface in order to ensure safety of the worker, while providing for comfort, ease of use, productivity and performance. 

    • Producibility Analysis: Allows design engineers, along with other technical personnel, to design product that promotes the ease of fabrication and assembly thus reducing production time, while increasing reliability. See DFMA for more information. 

    • Note: There are many software vendors that offer these tools that are mostly used by government contractors.  You can find more information on these software tools through a web search.

Common Manufacturing Simulation applications:

M&S Application

Type of M&S

Features and Goals
Assembly Line Balancing DES Design and balancing of assembly lines with a goal of ensuring smooth product flow by assigning an equal amount of work to all workstations.
Capacity Planning DES, SD, Monte Carlo, Petri-net Supports the determination of production capacity needed to meet future demand requirements based on labor, machines, processes, and facilities. 
Cellular Manufacturing Simulation  Virtual Simulation Analysis of manufacturing cells to determine work cell design to be self-contained, capable of performing necessary operations in an efficient manner.
Transportation Management DES, ABS, Petri-net Identifies manufacturing transportation cost drivers (bottlenecks, etc.) in order to optimize processes and workflow.
Facility Design, Layout and Location Hybrid Techniques Evaluate various factory floor designs in order to select the most efficient and cost-effective layout. Used to assess the introduction of new machines and manufacturing processes prior to purchasing the equipment.
Demand Forecasting DES, SD, Monte Carlo, Petri-net Supports the creation and comparing different forecast scenarios to evaluate the effectiveness of different forecasting models.
Material Planning DES, SD, Monte Carlo, Petri-net Supports the ability of production organizations to forecast, plan, schedule, and manage inventory.
Inventory Management DES, Monte Carlo Supports the assessment of various inventory management techniques (push, pull, JIT, etc.) to evaluate the cost of holding, inventory levels, replenishment, and determining batch sizes.
Just-in-Time DES Design of Kanban systems
Process Engineering DES, SD, ABS, Monte Carlo, Petri-net, Hybrid Process improvement, start-up problems, equipment problems, design of new facility, performance measurement
Production Planning and Control DES, ABS, Distributed, Hybrid Safety stock, batch size, bottlenecks, forecasting, and scheduling rules
Resource Allocation  DES Allocating equipment to improve process flows, raw materials to plants, resource selection
Scheduling  DES Throughput, reliability of delivery, job sequencing, production scheduling, minimize idle time, demand, order release
Supply Chain Management DES, SD, ABS, Simulation gaming, Petri-net, Distributed Instability in supply chain, inventory/distribution systems
Quality Management  DES, SD Quality assurance and quality control, supplier quality, continuous improvement, total quality management, lean approach
  • DES: Discrete Event Simulation
  • SD: System Dynamics
  • ABM: Agent-Based Modeling 
  • Perti-net: Place/Transition Net
  • Monte Carlo 

DAU Continuous Learning Modules and other training:

Guidance and other Resources:

M&S and DE Guidance and other Resources
Additive Manufacturing 

Additive Manufacturing (AM):  AM is a broad term used to describe the process of creating an object by building it one layer at a time. It is the opposite of subtractive manufacturing, in which an object is created by cutting away at a solid block of material until the final product is complete. Additive manufacturing started out with 3D printing but now encompasses seven types of additive technologies. The seven types of AM technologies (manufacturing processes) come with various strengths and weaknesses. These strengths and weaknesses range from speed, precision, investment costs, etc. The key is matching the right AM technology to the business requirement.

Additive Manufacturing in the U.S. is led by the National Additive Manufacturing Innovation Institute (NAMII), also known as AmericaMakes is led by the National Center for Defense Manufacturing and Machining and based in Youngstown, OH. The consortium's members include 40 companies, nine research universities, five community colleges and 11 nonprofit organizations. In addition, all of the services and Agencies have Additive Manufacturing organizations that are conduction research in additive manufacturing focused on organizational priorities. NAMII is working to accelerate the adoption of additive manufacturing by convening, coordinating, and catalyzing the AM industry in order to advance manufacturing competitiveness and national security,

The Joint Additive Manufacturing Working Group (JAMWG) is a cross-cutting DoD community focused on communication and coordination among the Services and Defense Agencies to maximize the application of additive manufacturing in support of the warfighter and sustainers.  Formalized in July 2017 and led by the OSD ManTech Program, this group consists of leaders from Military Services and Defense Agencies that engage across the Department and other Federal agencies, industry, and academia to reduce barriers to adoption of innovative AM technologies that benefit DoD and the warfighter.

JAMWG engages with several DoD sponsored public-private partnerships to engage with industry, academia and other partners to gather input, share information and advance state of the art developments for AM. 
  • America Makes, the National Additive Manufacturing Innovation Institute
  • Additive Manufacturing for Maintenance Operations
  • Additive Manufacturing Standards Collaborative
  • MxD National Center for Cybersecurity in Manufacturing 

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

Additive Manufacturing Technologies

Additive Manufacturing A process of joining materials to make parts from three-dimensional (3D) model data, usually layer by layer, also known as three-dimensional (3D) printing. The American Society for Testing Materials (ASTM) defines seven fundamental processes in the realm of additive manufacturing. Each of these processes utilize different materials which require different 3D printers.

Additive Process

Technology

 Materials Used Typical Technologies Description 
Binder Jetting Metals, Polymers and Ceramics  3D Printing Uses a powder-based material and a binder to produce parts. The binder acts as an adhesive between powder layers. The liquid binder bonds the parts. A roller spreads powder over the build platform then the print head moves across the platform and sprays the binder agent onto the powder layer. 
Material Extrusion Polymers Fused Deposition Modeling (FDM) Is the least expensive and used plastic polymer as the base material. The polymer is drawn through a nozzle, where it is heated and is then deposited layer by layer. It is often used for prototyping.
Powder Bed Fusion Metal, Polymers, Composites, and Ceramics Selective Laser Sintering (SLS), Selective Laser Melting (SLM), and Electron Beam Melting (EBM) Uses energy to melt or sinter powdered material. Then a laser or electron beam is directed across the powder which bonds the powder. 
Vat Polymerization Polymers and Ceramics Stereolithography (SLA), Digital Light Processing (DLP) Uses a vat of liquid photopolymer resin, the model is constructed layer by layer. An ultraviolet (UV) light is used to cure or harden the resin. 
Direct Energy Deposition Metals and Hybrid Metals Laser Metal Deposition (LMD)  Pushes a wire or powder through a nozzle and is melted using a laser or other focused energy source. 
Material Jetting  Polymers and Composites 3D inkjet Printing  Melts photoreactive resins into a liquid photopolymer. The print head moves across a bed depositing material which is then hardened using UV light. 
Sheet Lamination Hybrids, Metals, and Ceramics Ultrasonic Consolidation (UC) Uses sheets or ribbons of material, usually metal, which are bound together using ultrasonic welding. Often used for rapid prototyping of parts.

Guidance and other Resources:

Additive Manufacturing Guidance and Resources 
CREATED:
BY:
Noyes, George - Student
LAST MODIFIED:
BY:
Noyes, George - Student
View Resource
Community Resource / Manufacturing and Quality
Industrial Base / Supply Chain Management

National Technology and Industrial Base (NTIB):

The National Defense Authorization Act for FY2017 defines the NTIB as consisting of the people and organizations engaged in research and development (R&D), production, maintenance, and related activities within the United States, Canada, the United Kingdom, Canada, Australia, and Northern Ireland. Thid includes the organic base comprised of government-owned, government operated, and contractor operated facilities. 10 USC 2440 requires the Secretary of Defense to consider the National Technology Industrial Base (NTIB) in the development and implementation of acquisition plans for each MDAP. Acquisition planning and plans shall include considerations of the NTIB for all MDAPs.

The U.S. Industrial Base includes:

  • Defense Contractors (large and small) providing military weapon systems, subsystems, and components or parts 
  • Any company or organization with a manufacturing capability
  • Any organization with the capability to conduct research and development
  • Any organization that can design, produce, deliver, sustain, and dispose of military hardware
  • Companies providing 
  • Federally Funded Research and Development Centers
  • Colleges and Universities
  • Some foreign entities listed above that can provide weapon systems, subsystems, and components or parts, or can design, produce, deliver, sustain, and dispose of military hardware

Note: Additional information, guidance, tools, and other resources, by acquisition phase, may be found in the M&Q Body of Knowledge at https://www.cto.mil/sea/mq/ 

Industrial Base considerations should include:

  • The ability to support development and production (rates and quantities)
  • The identification of IB risks in the supply chain
  • The identification of single points of failure in the supply chain (sole source, foreign source, etc.)
  • Support for a resilient supply base for critical defense capabilities
  • Support for procurement surges and contractions
  • Small Business considerations
  • Competition

Common Industrial Base risks include:

  • Loss of competition due to mergers and acquisitions
  • Supply Chain shortages
  • Industrial cybersecurity
  • Obsolete items (DMSMS/Obsolescence) 
  • Foreign dependence
  • Financial stability and visibility
  • Sole Source/Single Source suppliers
  • Limited production capacity and capability 
  • Disasters (natural and man-made)
  • Loss of skills 

This resource page will focus on the following Pinned Content:

  • Industrial Base Guidance and Other Resources
  • Industrial Base Tools and Checklist
  • Manufacturing Technology (ManTech)
  • Industrial Cybersecurity 
  • Industrial Cybersecurity Tools and Checklist 
  • Supply Chain Management
  • Supply Chain Metrics
  • SCM Guidance and Other Resources 
  • SCM Tools and Checklist
Industrial Base Guidance and other Resources 
Industrial Base Tools and Checklist 
Manufacturing Technology (ManTech)

The objective of the ManTech program is to improve performance while reducing acquisition cost by identifying, developing, maturing, and transitioning advanced manufacturing technologies. Manufacturing feasibility assessment should identify high-risk manufacturing process areas that are technology voids or gaps and may require investments in ManTech or other programs. ManTech program investments should be directed toward areas of greatest need and potential benefit. These investments must be identified early so that these manufacturing capabilities will be matured in time to support production.

The OSD ManTech program focuses on cross-cutting defense manufacturing needs - those that are beyond the ability of a single service to address = and stimulates the early development of manufacturing processes and enterprise business practices concurrent with science and technology development to achieve the largest cost-effective impact and to facilitate the developments enabling capabilities to our warfighters. 

The Accelerating the flow of technology to the warfighter is one of the top priorities of DoD, services, and agencies. The ManTech program focuses on advancing state-of-the-art manufacturing technologies and processes from the research and development environment (laboratory) to the production and shop floor environment. ManTech addresses Critical Technology Elements (CTEs) that are often immature and have process limitations that need to be assessed, and plans made to mature the CTE.

DoD ManTech Offices: 

Guidance and other Resources: 

Tools and Checklist:

Industrial Cybersecurity 

Industrial cybersecurity is concerned with the ability of organizations to securely create, manage, control, and share information digitally while the management and exchange of information is critical, it is equally important to do so in a safe and secure environment. Industrial cybersecurity is concerned with the transfer of digital data via Operational Technologies (OT) inside a facility and through the cloud to other organizations and facilities. Current digital environments are complex and made up of many systems with digital threads that connect government program offices to industry, prime contractor to subcontractors, laboratories to program offices, within an organization, etc. This digital thread includes design data in the form of model-based designs, model-based systems engineering, shop floor machines that use the design data to manufacture product, the cloud to share data with suppliers, retailers, and other service organizations. There are several documents that provide guidance and support.

The integration of Information Technology (IT) and Operational Technologies (OT) is helping manufacturing organizations to improve productivity and efficiency. However, it has also provided malicious actors (nation states, criminals, insider threats, etc.) the ability to exploit cybersecurity vulnerabilities. Once malicious actors gain access, they can harm an organization by compromising data or system integrity, hold industrial control systems (ICS) and/or OT systems ransom, damage ICS machinery, or cause physical injury to workers.

Operational technologies and Industrial Control Systems can include:

  • Enterprise resource planning (ERP) system supports functional management resources within an enterprise, and control process performance.
  • Product lifecycle management (PLM) systems for creating and managing the design process. 
  • Manufacturing execution system (MES) support the planning, execution, and synchronization of manufacturing processes across multiple functions, distributed plants, and suppliers.
  • Programmable Logic Controllers (PLCs)
  • Supervisory Control and Data Acquisition (SCADA) Systems 
  • Distributed Control Systems (DCS)

DoD Service and Agency Cybersecurity homepages: 

DAU Continuous Learning Modules and other training:

Industrial Cybersecurity Guidance and other Resources 
Industrial Cybersecurity Tools and Checklist 
Supply Chain Management

According to the Association for Supply Chain Management a supply chain is a "system of organizations, people, technologies, activities, information and resources involved in moving materials, products and services all the way through the manufacturing process, from the original supplier of materials supplier to the end customer.” 

Much of the program’s components and subsystems comes from the supply chain. Supply Chain Management (SCM) is a pivotal task. Often program problems originate in the supply chain, but do not manifest themselves until the component is integrated into the system. Program offices and contractors often make efforts to identify and manage problems at the first tier, but do not do well below that level. M&Q managers need to routinely review and assess contractors supply chain and procurement activities and efforts. 

Essential features of a supply chain include:

  • Visibility is about knowing where your product is in the supply chain so that you can track it.
  • Velocity is about knowing that your product will be delivered on-time because the cycle time is known.  And more importantly the cycle time is “short.”
  • Variability is about bad things that can happen and cause variation.  Variation is the enemy of efficiency and effectiveness.  Variation may cause the loss of Visibility and Velocity.
  • Visibility and Velocity is good.
  • Variability is bad. 

Supply chain management objectives include:

  • Cost: Reduce total supply chain costs through effective marketing, production and distribution strategies.
  • Responsiveness: Ensure responsiveness to consumer demand, changes, and flexibility
  • Lead Times: Minimize the lead time between the production of an item and its sale to the final customer to the practical minimum
  • Cash Flow: Maximize cash flow by reducing inventory and improving payment terms
  • Competitive Advantage: Ensure Competitive advantage in the timely introduction of new products and services
  • Customer Service: Improve customer service

The DoD Supply Chain is based on five distinct process building blocks (Plan, Source, Make, Deliver, and Return) as depicted below using the Supply-Chain Operations Reference-model (SCOR) Model.

Extensive additional Supply Chain Management (SCM) resources are also available on the DAU Acquisition Community Connection (ACC) Life Cycle Logistics site.  https://www.dau.edu/cop/log 

Note: The inclusion of related websites is provided as an initial list of sources of potentially valuable information.  These links are not inclusive and are not intended to endorse any individual website. 

Note: DAU offers a 3-day Supply Chain Management workshop several times a year and this workshop covers the following major manufacturing topics:

  • Supply Chain Management Definitions and Examples 
  • Supply Chain Management and the Supply Chain Operational Reference (SCOR) Model
  • Manufacturing Production and Control
  • Forecasting Demand 
  • Demand Planning and Management 
  • Manufacturing and Material Planning 
  • Capacity Planning and Management 
  • Production and Control (Shop Floor Control)
  • Distribution Management
  • Supply Chain Metrics and Assessments 
  • Supply Chain Challenges and Concerns 

https://www.dau.edu/blogs/workshops-workshops-and-more-workshops 

DAU Continuous Learning Modules and other training:

Supply Chain Risks and Challenges 

DoD supply chain risk management is a systematic process for managing supply chain risk by identifying susceptibilities, vulnerabilities and threats throughout DoD's “supply chain” and developing mitigation strategies to combat those threats whether presented by the supplier, the supplied product and its subcomponents, or the supply chain (e.g., initial production, packaging, handling, storage, transport, mission operation, and disposal).

DoD Supply Chain Risks and Challenges:  

  • Increasing Complexity of DoD weapon systems
    • Diverse Product Lines
    • Short Product Life Cycles
  • Heightened Warfighter Expectations 
    • I want it now (speed to warfighter)
    • I want is customized (agility)
    • I want the newest technology and features (updates)
  • Globalization of Supply Chains
    • Security of supply chains (cybersecurity, etc.)
    • International Trade Initiatives and Restrictions (ITAR)
    • Business rules and compliance
    • Different measurement systems and currency
  • Lack of Insight into Prime's Supply Chain
    • Lack of visibility into sub-tier suppliers
    • SCM relationships limit transparency
    • Communication up and down the chain 
  • Material Availability (Single/Sole or Foreign Sources)
  • Demand Forecasting (Lead times and Delays)
  • Inventory Strategies and Management
  • Digital Transformation (Industry 4.0)
  • Data Analytics (Lack of Data and Analytics)
  • Cybersecurity
  • Human Capital Shortages (Talent)
Resources and other Guidance:
 
DAU Continuous Learning Modules and other training:
Supply Chain Resilience

Supply chain resilience is the ability of a supply chain to quickly adapt to and recover quickly from unexpected events, such as economic shocks, disruptions, or other challenges. Covid-19 was perhaps the single most impactful supply chain event since the great depression and World War I and II. The impacts of Covid-19 was felt in all countries and in all industry sectors. They also affect a wide spectrum of products, from expensive goods, such as cars and electronics, to necessities, such as food, medicines, oil and gas – all of which has an impact on the cost of living. The significant and compounding effects of supply chain bottlenecks and rising freight costs on prices, particularly for producers and manufacturers that import products like fertilizer and construction materials, have serious economic implications. Economies in the Asia Pacific, many of which rely heavily on imported goods, are vulnerable to the consequences of supply chain disruption for production and consumption.

Supply chain resilience can be achieved through better data on the structure of supply chains, investments in redundancy, greater ability to substitute between inputs, and improved communication across the supply chain to include increased visibility down to third tier suppliers.

DoD Supply Chain Resilience Recommendations: 

  • Build domestic production capacity: For those supply chains that are critical for national defense, the U.S. is committed to ensuring reliable production access within the defense industrial base, both domestic and allied. 
  • Engage with partners and allies: The U.S. is collaborating with its international partners and allies to develop policies and arrangements that strengthen our defense industrial bases and improve supply chain resilience. 
  • Mitigate Foreign Ownership, Control, or Influence (FOCI) and safeguard markets: The Department is committed to protecting its supply chains and the defense industrial base from adversarial FOCI by scaling efforts to identify and mitigate FOCI concerns. 
  • Conduct data analysis: DoD will continue to build on previous efforts to expand its visibility into supply chains by collecting and organizing key data. 
  • Aggregate demand: The Department will signal to industry what the likely total demand is across multiple programs, so industry can better anticipate number of orders from year to year. 
  • Develop common standards: To leverage commercial sector innovations, and to embed modernizing technologies in weapon systems, the DoD will work, where possible, to limit its use of military-unique requirements when developing performance requirements. 
  • Update acquisition policies: DoD should engage in efforts to develop a whole-of-government strategy and implementation plan to engage with industry and Congress to determine which policy and regulatory changes would encourage expansion of capabilities

Guidance and other Resource:

Supply Chain Metrics

Supply chain metrics should be SMART (specific, measurable, achievable, relevant, and timely). The Gartner Hierarchy of SCM Metrics, which is considered the industry standard for supply chain performance measurement, identifies 3 key performance indicators (KPIs); demand forecasting, perfect order, and supply chain management cost.  Basically, a highly accurate forecasting model (Demand Forecast) coupled with a healthy balance of customer service (Perfect Order) and cost control (SCM Cost) indicates a healthy supply chain operation. Any issues with these indicators can be further diagnosed by a mid evel tier of metrics that focus on your cashflow.  Root cause analysis metrics are located at the ground level, where targeted intervention can be carried out towards improving supply chain effectiveness.

The DoD Supply Chain Metrics Guide provides an analytical framework for presenting the metrics, the Guide links each metric to one of the following desired attributes for DoD supply chain management: 

  • Materiel readiness—the ability of the supply chain to support weapon systems in undertaking and sustaining their assigned missions at planned peacetime and wartime utilization rates. Supporting materiel readiness is the mission imperative of the end-to-end DoD supply chain. 
  • Responsiveness—the ability of the DoD supply chain to respond to customer materiel requests by providing the right support when and where it is needed. For DoD, responsiveness is the speed at which the DoD supply chain fulfills warfighter needs. This attribute is most representative of the customer's perspective of the DoD supply chain. 
  • Reliability—the dependability and consistency of the supply chain providers to deliver required materiel support at a time and place specified by the customer. Reliability is key to DoD customer confidence in the DoD supply chain. This attribute focuses on how well the supply chain processes are being executed. 
  • Cost—the price paid for the supply chain resources required to deliver a specific performance outcome. Cost effectiveness is key to right-sizing the DoD inventory investment and controlling supply chain costs. This attribute is an implied constraint on supply chain operations; it evaluates the DoD investment in the supply chain and assesses financial effects on supply chain customers. 
  • Planning and precision—the ability of the supply chain to accurately anticipate customer requirements and plan, coordinate, and execute accordingly. Planning and precision are key to DoD supply chain management. Their effectiveness affects all other attributes.
SCM Guidance and other Resources 
SCM Tools and Checklist
  • AS9133, Supplier Audit Checklist, conduct an internet search 
  • MRA Supplier Component Supplier Risk Tool, conduct an internet search 
  • Supplier Performance Risk System https://www.sprs.csd.disa.mil/ 
  • Interactive MRL Users Guide (Checklist), Supply Chain Management sub-thread https://www.dodmrl.com/ 
CREATED:
BY:
Content Author No Reply
LAST MODIFIED:
BY:
Content Author No Reply
View Resource
Community Resource / Manufacturing and Quality
Counterfeit Parts

In Mar 2012, DUSD (AT&L) published a memorandum called "Overarching DoD Counterfeit Prevention Guidance", identifying counterfeit items as serious threats to safety and operational effectiveness of DoD systems, particularly items such as mission critical components, critical safety items, electronic parts and load-bearing mechanical parts affecting “…system performance or operations, the preservation of life, or safety of operating personnel.”

Counterfeit parts guidance includes:

Note: Both of these documents are industry standards and cost to download.

CREATED:
BY:
Content Author No Reply
LAST MODIFIED:
BY:
Content Author No Reply
View Resource

Documents / Manufacturing and Quality

Engineering of Defense Systems Guidebook
Modified: Changed by Arthur, Robert on
Summary

This link will take you to the webpage for access to the DoD Engineering of Defense Systems Guide. 

http://www.cto.mil/wp-content/uploads/2024/10/Eng-Def-Sys-Change2-7October2024-…
Requirements Traceability Matrix Guide
Modified: Changed by Zearley, Jennifer on
Summary

This link will take you to a website for access to Requirements Traceability Matrix Guide.

https://www.dau.edu/cop/ace/documents/sample-services-requirements-traceability…
NIST SP 800-82, Guide to Industrial Control Systems (ICS) Security
Modified: Changed by Noyes, George - Student on
Summary

This document provides guidance for establishing secure industrial control systems (ICS). These ICS, which include supervisory control and data acquisition (SCADA) systems, distributed control systems (DCS), and other control system configurations such as Programmable Logic Controllers (PLC) are often found in the industrial control sectors. 

https://csrc.nist.gov/pubs/sp/800/82/r2/final
Critical Manufacturing Sector Security Guide, CISA 
Modified: Changed by Noyes, George - Student on
Summary

The Critical Manufacturing Sector comprises processes and products that are crucial to the economic prosperity and continuity of the United States. Among myriad roles and responsibilities, manufacturers in the sector process raw materials and primary metals; produce engines, turbines, and power transmission equipment; produce electrical equipment and components; and manufacture cars, trucks, commercial ships, aircraft, rail cars, and their supporting components. The Critical Manufacturing Sector produces highly specialized parts and equipment that are essential to primary operations in several U.S. industries— particularly transportation, defense, electricity, and major construction. Central to the sector’s operations is the global transport of raw materials and finished products along large, complex supply chains.

https://www.cisa.gov/resources-tools/resources/critical-manufacturing-sector-se…
DoDI 8500.01 Cybersecurity
Modified: Changed by Noyes, George - Student on
Summary

DoD will implement a multi-tiered cybersecurity risk management process to protect U.S. interests, DoD operational capabilities, and DoD individuals, organizations, and assets from the DoD Information Enterprise level, through the DoD Component level, down to the IS level as described in National Institute of Standards and Technology (NIST) Special Publication (SP) 800-39 (Reference (o)) and Committee on National Security Systems (CNSS) Policy (CNSSP) 22 (Reference (p)).

https://www.esd.whs.mil/portals/54/documents/dd/issuances/dodi/850001_2014.pdf
DoD Instruction 5000.90, Cybersecurity for Acquisition Decision Authorities and Program Managers
Modified: Changed by Noyes, George - Student on
Summary

This link will take you to a website for access to DoD Instruction 5000.90, Cybersecurity for Acquisition Decision Authorities and Program Managers.

 

https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500090p.PDF
Supplier Performance Risk System
Modified: Changed by Noyes, George - Student on
Summary

This link will take you to a website for access to Supplier Performance Risk System. 

https://www.sprs.csd.disa.mil/
SAE EIA 649B-2011, Configuration Management Standard
Modified: Changed by Noyes, George - Student on
Summary

This link will take you to a website for access to SAE EIA 649B-2011, Configuration Management Standard.

Note: This is a commercial standard and cost to download.

https://webstore.ansi.org/Standards/SAE/SAEEIA649B2011EIA649B?gclid=EAIaIQobChM…
To participate in this community, you must be registered. Join Now