Configuration Management Assessment FAQ
In this month’s CMsights we speak with Lisa Fenwick, Vice President of CMstat, to ask her the most important questions about performing Configuration Management Assessments. Lisa is a CMPIC Certified CM Assessor with 15 years’ worth of successful CM Assessments from over 25 years of CM experience. She holds CMPIC Master’s and SME Certifications as well as CMII-C from IpX. Lisa serves as the lead instructor for CMPIC’s Course 7 – CM Assessor Certification.
Lisa, first of all, what exactly is a Configuration Management Assessment?
A Configuration Management Assessment (CMA) is an independent review of CM-related processes, metrics, data handling, and supporting software to determine what is working, what needs improvement and why, and where performance gaps may exist between the current state and a desired benchmark standard such as EIA-649C.
The CM Assessment looks at all of these elements, and more, with special attention to how they interrelate over the lifecycle requirements of a product, complex system, product line portfolio, deployed operational asset, new A&D program, or entire enterprise along with its supply chain.
The result of a CMA is an analysis with recommended strategy and roadmap forward to achieve the next level of process maturity. This typically leads to a set of updated processes that will enable the organization to realize the full potential of performing world-class, standards-based, proactive, data-driven management of their products.
We often hear those who are not CM experts confuse assessments with audits. So how is a CM assessment different from a CM audit?
The main way a CM Assessment is different is the outcome you are trying to achieve. A product audit or quality audit will look for deficiencies and make a list of actions to solve the immediate problems. CM Assessments are holistic in nature and result in evolving your process maturity, taking the good that is being done and building on that to lead your team to operating the way industry leaders perform excellent CM.
It’s also important to note that the CM Assessments we are talking about are not audits like Functional Configuration Audits (FCA) or Physical Configuration Audits (PCA). These types of audits are often specified in defense contracts as part of the System Verification Review process prior to delivery of a product or system. However, the failure of a FCA or PCA can often reveal problems that could have been identified and remediated by the results of a CMA performed before the audit. You can read more about these types of audits in a previous CMsights article we have on our website.
When and why should an organization consider performing a CMA?
Well, it seems all too often a customer will come to CMstat to perform an assessment, or to CMPIC for training to learn how, when it is rather late and problems have become both endemic and entrenched. I often feel that it would be a rare privilege to perform an assessment in the early stages of a new organization, product development, or program before bad practices set in and there is resistance to change. The truth is, we’re often called in once there is a crisis, or worse, when there is no time left in the schedule to change midstream.
Some may ask how is this possible when there are few processes, tools, and data sets yet in production use to review. In this case, the focus of the CMA is on ensuring that the planning is comprehensive, and that processes and systems will work in an integrated fashion to provide efficiency and traceability. This is a must-have regardless of where organizations find themselves on the path to a digital environment. It is all too tempting for a new product line or project team to simply adopt legacy expectations, legacy processes, and legacy software without adequate discernment of what is actually best for going forward.
Other times, I’ve been asked to perform assessments even when the organization had one completed before. In this case, the assessment would be providing greater insights and refining processes, or perhaps even taking advantage of a new ability to improve that did not exist during the first assessment.
There are a number of additional warning signs as to when a CM Assessment is in order that are referenced in the 25 most common CM problems cited in Challenges Implementing Configuration Management and also in a CM deficiencies checklist found in this earlier post about Configuration Management Assessments.
Is it only the CM experts and their teams who should care about assessments?
No, there are many standards that recognize the benefit of evaluating your processes every now and then to make sure they are still delivering value to the entire organization, not just the CM group.
There is a related question I hear that asks if a CM Assessment is just another business process exercise. Also no, as the difference is that CM Assessments, when implemented correctly by qualified and certified professionals, will provide not only the direction but also a business case for improvements that management would have a hard time saying no to. And more importantly, CM itself is not a one and done set of processes. This is a misconception. The benefits of CM can only continue to be achieved when processes are in a continuous improvement mode and a roadmap for refinements is being followed with regular reviews built-in. And many of those processes occur outside the CM function.
So then do CM Assessments have to be performed across the whole organization?
Not at all. I have been involved with CM assessments that just looked at a single product along with its configurable permutations. This is typical of a manufacturer’s division that focuses design and production on one product family. But I have also been asked to look at a whole new A&D program that will span across multiple supply chain and sustainment service partners.
On a couple of occasions, I’ve been asked to assess a specific CM process such as change processing to identify and resolve bottlenecks. Obviously, the scope of work, budget, and time required can vary by an order of magnitude. Other times, I’ve been asked to review and update an existing assessment in a way that provides greater insights and clarity on the order of implementing specific processes or software tools, especially in accordance with changing standards or regulatory rules.
There are many business and technical objectives an organization may have — some not always compatible — that can be addressed by a CM Assessment. The width of effort is based on these stated goals, including reconciling any conflicts between them.
Who should perform a CMA?
I generally find that an independent outside consultant can produce a more comprehensive, accurate, and honest assessment because they look at the situation from afresh, without biases or loyalties to any one legacy process, tool, or group. This is not to say that a senior level employee cannot also lead the CMA effort, but they must be seen as having executive-level support and organizational independence for their work.
Regardless of who does it, it is imperative that the individual be someone who has gone thru certification process like that provided by CM Assessor training. Several times a year I teach CMPIC’s CM Assessor training both in-person and virtually. While I believe CMPIC’s course content sufficiently educates the students to go back and lead a CMA, it has been my experience that the interaction and learning between participants from different companies and industries is just as valuable.
What has surprised you the most when performing a CMA or training others to do so?
It still amazes me that after so many years of training and certifying practitioners in CM, that there is so much need for CM fundamentals education. The CM function is often not adequately staffed or funded leading to major gaps in existing processes largely because those with the budget authority may not have an understanding of CM nor have seen the benefits of effective CM applied to their products or programs. We write about the need to educate managers often such as these example CMsights articles on Getting Managers to Care About Configuration Management. and The Role of Configuration Management Champions.
As a result, some will enter the CMA process expecting that an assessment means we spend all our time with the CM and DM experts deep inside engineering. Yet, for most companies the role and value of CM when implemented correctly spans across numerous departments and stakeholders. That means a comprehensive review must also consider the needs of all the different creators, users, revisers, and beneficiaries of configuration-related processes and data.
This list of stakeholders can be long and may include: program and product management, contracts, procurement, manufacturing, logistics, test, quality, compliance, tech pubs, maintenance, internal and external customers, service, and sustainment. That often surprises the managers at some sites who thought that CM was mostly a change control engineering function; a big red flag warning in itself.
Here’s my favorite question of consultants: is there a question we should have asked that is often overlooked?
Yes, and it has to do with what an organization should do to prepare for a CM Assessment, regardless of who performs it. I often joke that everyone is the same, but all for different reasons! The level of preparation varies for so many reasons due to the industry they are in, the internal culture of the company, business drivers, organizational competencies (and incompetencies), management support, and of course their history of not just CM but related systems such as PDM and PLM.
The first step of that preparation is to engage an experienced, certified CM Assessor and discuss what you are hoping to achieve and the why. I find the “why” is more insightful than the what. The best assessors will challenge you to share performance problems, operational pains, and organizational issues that you might at first be reluctant to disclose to an outside consultant at the onset to size the SOW of a CMA.
More often than not, those skillfully-moderated discussions will flush out underlying factors that may not be getting the visibility they deserve with all stakeholders. And an external assessor can do that in a way that is not about assigning blame for the past but charting out solutions for the future.
While a CMA is a rather technical process, it must be conducted within the context of the business needs. If not, then the findings and recommendations of a CMA could lead the organization away from realizing the full benefits of CM. Understanding that business context and getting alignment across all stakeholders becomes the real work of preparation for both the assessor and their client.
One final question, Lisa, where can one go to find more information on CMA?
I’m excited to share information about our own CMstat consulting work where there is much information to be found about CM topics, including assessments, on our website at https://cmstat.com. Contact me at lfenwick@cmstat.com or 877-234-2156 for a discussion and proposal about performing a CMA at your site. I also recommend checking out CMPIC where you can inquire about CM Assessor training classes that I teach several times a year.
Receive CMsights
Subscribe to CMsights News for the latest updates from CMstat on Configuration Management, Data Management, EPOCH CM, and EPOCH DM.
Request a Demo
See how EPOCH CM and EPOCH DM support industry standards and best practices in Configuration Management and Data Management.
