Configuration Management Assessment
A frequent question we hear from aerospace and defense industry executives is how can they perform a Configuration Management Assessment of the performance of their organizations, programs, and products with regard to the practice of configuration management.
CMstat has learned over our several decades that the underlying concern of managers, which triggers this question, is not their professional interest in or curiosity about configuration management, even though we might wish that was true!
Instead, it’s more often like “how much exposure and risk – technical, financial, regulatory, legal, and reputation – do we have if something goes wrong or fails?”
Those of us with long business careers know that Risk of Failure (ROF) is as powerful a motivator as Return on Investment (ROI), even if financial analysts have not come up with a metric to quantify the former as they have the latter.
Nevertheless, hearing the question reveals management, at some level, understands that oversight of the CM function is not a domain to be relegated to a few specialists locked away in the back office. Instead, it is an enterprise quality management function in which program and product executives have a responsibility to provide oversight and governance.
We discussed the relationship of configuration management to quality management in the CMsights post Why Configuration Management is Quality Management.
Resources for Configuration Management Assessments
In working with our customers we have learned to respond to their configuration management assessment question in several ways. In this CMsights post we will summarize a few of the approaches used and will conclude with a checklist of specific discovery questions useful in performing a configuration management assessment.
For Program and Project Managers in aerospace and defense we have published a “Primer on Configuration Management which may be downloaded at HERE.
One of the most important items covered in the primer has to do with the importance of CM Plans. More specifically, having a plan to use PDM software or CM tools is not the same as having a CM Plan! To help managers understand the importance of a CM Plan, and what should go into one, we published this article Configuration Management Plan Review.
Another topic that often confuses managers is the difference between change control and configuration management. We recommend this discussion on How Configuration Management is Different from Change Control.
Still, some may erroneously think that if their organization has change management under control, then they must be performing configuration management and all is well. For these managers we suggest that they ask their organization ten questions as referenced in this blog on how to Determine if Configuration Management and Change Control is Under Management Control.
Starting at the End, Not at the Beginning
A CMsights reader recently commented that when trying to engage their managers about the important role of CM they first had to help them see the CM problems that exist before they could educate them further. They found greater success in first exposing the problems, challenges, and pains as symptoms of inadequate CM. That is, starting at the end with “what happens without CM” and working backwards instead of starting at the beginning with “what is CM” and working forward. They asked if CMstat had a “configuration management assessment checklist” of our own.
Years ago, we would have responded that a generic checklist was not possible because everyone is dead set on thinking their industry, business, products, processes, and customers are all very different from even those with whom they compete in the same industry for the same contracts. Accordingly, they conclude incorrectly that their pains, problems, needs, and requirements must all be quite unique. Yet, when it comes to deficiencies in CM this is rarely the case. Everyone is indeed very special, but not everyone is very different!
The value of Configuration Management within various industries for different applications is driven by many of the same problems and pains. Yes, how a firm responds to (or ignores) these challenges, and their consequences, is very different which then gives them the impression that they are all very different. But that does not mean their underlying deficiencies and problems are different.
A Configuration Management Checklist
To illustrate this further, let’s look at a short version of CMstat’s Configuration Management Assessment Checklist that we use in our CM consulting practice.
Which of these problems does your organization experience, or it would have without an effective CM implementation?
The sheer volume, complexity, and variations in configurations of component parts, design requirements, product deployments, and complete systems overwhelms legacy methodologies and tools.
The actual as-deployed or as-maintained configuration of deployed products and systems has evolved to be very different from what was as-shipped or as-installed.
Time and effort required to find and verify accurate configuration information eats into the project schedule.
Lack of confidence in producing upon demand the “real-time live configuration” of deployed assets by specific lot number or serial number.
Discovery of where-used, and analysis of change-impact on as-deployed configurations, is unwieldy or untrustworthy, especially for systems operating far downstream of the OEM’s engineering release.
Management of processes that create, use, or revise configuration data is absent, ad hoc, or broken; much less automated or optimized.
Losing track of the change process due to changes being made without approval or enterprise awareness, or change history information unavailable, inaccurate, or obsolete.
Losing configuration control prior to delivery with products produced that are not conforming to requirements, or data and document deliverables not conforming to what was released.
Losing configuration control after delivery from not being able to track what components are installed on serialized end items.
What’s actually bid, or delivered, by the contractor is not what was specified or bought by the customer.
Failing the Product Configuration Audit mandated by the contract or program office.
Lack of Baseline Configuration Management along with inability to identify and track Configuration Items over their individual lifecycles.
Processes and procedures for mass customizations and variant management are themselves one-off customizations that are difficult to replicate or scale.
Difficulty in keeping all the different or downstream BOMs accurate and synchronized with engineering changes across all product variants or lot and serial numbers.
Multiple copies of documents stored on servers and customer archives that are not the same despite having the same document ID and revision level.
Disconnects between the as-designed and as-built parts BOM due to company and program approved part substitutions not being captured in the PLM system.
Program management and project budget suffers due to a lack of visibility into CM problems and challenges before it is too late.
Inability to find the acceptance data package or COC specific to a delivered serial number when delivering more than one item.
Installing unauthorized parts, purchasing obsolete parts, or work stoppage due to lack of approved parts.
Parts produced for the wrong configuration without visibility to alternate or substitute parts that are authorized due to lack of proper communication and documentation.
Software versioning and releases not compatible with hardware releases.
Manufacturing inventory, tooling, and processes are misaligned with product configurations.
Reject, rework, or scrap rates are too high due to old or wrong specifications.
Installation instructions, work orders, test procedures, training manuals and service bulletins are often obsolete because of configuration inaccuracies.
Integration and complexity of CM for the supply or service chain that has design and change authority.
Configuration of digital twin models for service is hard to create or maintain, much less synchronize with the actual physical asset.
Service contractors not having access to the OEM’s product configuration data, or being overly-constrained to use only that product data.
Completeness, accuracy, and security of data retention and document repository is undependable.
Extended lifecycles of assets that are in-service for decades creates extraordinary LOTAR burden on CM data.
Sustainment groups are unable to locate documentation or other historical artifacts.
Failure to maintain, document, and track ITAR Compliance.
Certification and other documentation records not matching the true status of the deployed configuration.
Disconnect between the requirements that the systems engineering function is working to versus what is actually specified on the contract.
Without a CM Plan the requirements for CM processes and tools are constantly in flux leaving different product organizations to do their own thing.
Program management and project budget suffers due to a lack of visibility into CM problems and challenges before it is too late.
If you checked off more than a few of the above items, your organization may be underperforming in its CM functional capability, workflow capacity, or skills maturity. Or if there are problems you have seen over your career that we have missed please share them by leaving a comment below.
To schedule an independent assessment of your organization’s CM capacity, processes, and tools so you can begin to improve upon them contact the CM specialists at CMstat with a phone call to (877) 537-1959 or email to information@cmstat.com.
Receive CMsights
Subscribe to CMsights News for the latest updates from CMstat on Configuration Management, Data Management, EPOCH CM, and EPOCH DM.
Request a Demo
See how EPOCH CM and EPOCH DM support industry standards and best practices in Configuration Management and Data Management.
