By Noel E. Donlin
I read with interest the article in the February 1997 issue of Military & Aerospace Electronics entitled "Survey: PEMs are more reliable than mil-spec parts." It states, "Hakim, an outspoken proponent of COTS, conducted a survey of 16 defense and commercial integrators and subsystem manufacturers last May, and to his surprise found that none of them reported any advantage in using mil-approved parts." I know many people are being mislead by the published results of this survey.
I have read several prominent publications referencing the results of Mr. Hakim`s survey. A copy of this survey form appears later in these comments. The survey results also surprised Mr. Hakim. He is quoted as saying, "It was totally unexpected that all of the respondents did not see any advantage in using mil-approved parts; in fact, they say there is a disadvantage." The survey sheet is structured so there is no opportunity or incentive to provide real numerical reliability data. This is a skewed survey that is structured to provide programmed answers (yes, unknown, no) to "PEM only" queries.
Two examples reflecting false information published pertinent to this survey follow:
1. The Department of Defense report, "Overcoming Barriers to the Use of Commercial Integrated Circuit Technology in Defense Systems," Oct. 1996, page 3-4 states, "Likewise, all of the respondents to a survey of more than 60 military equipment and subsystem suppliers indicated that commercial ICs were equivalent or superior to Mil-Spec ICs in terms of performance, cost, reliability, size, weight, and availability." No respondent answered any of the survey item 9 inquires with the terms equivalent or superior. This report was signed by Dr. Kaminski.
2. The December 1996 issue of M&AE also references Mr. Hakim`s survey and states, "The report also cites a survey in which more than 60 subsystem suppliers said that commercial ICs were equivalent or to mil-spec ICs in terms of performance, cost, reliability, size, weight, and availability. This survey is from the Display Science and Reliability Branch of the U.S. Army Research Laboratory at Fort Monmouth, N.J." Again, no respondent answered any of the survey item No. 9 inquires with the terms equivalent or superior. The only place I found the words equivalent and superior used was on the U.S. Army Research Laboratory graphical analysis of the survey results.
Mr. Hakim transmitted his survey to the potential respondents under a cover letter dated April 18, 1996. Two sentences from the cover letter stand out and intone the intent of the survey. "It is vital that you or other responsible individuals respond to the enclosed survey for each equipment produced by your company which uses PEMs. Results will be used to support Secretary of Defense William Perry`s efforts to incorporate commercial technology or present to OSD showing the harm being inflicted.`` This specific wording presents the notion that the survey is not interested in gathering factual technical data for hermetic parts whether they are classified as Mil-Spec or commercial. There also are systems that may have a mixture of hermetic and PEMs, but the survey did not request this type of comparative data.
Mr. Hakim submitted this survey to over 60 military equipment and subsystem suppliers. This is verified by a June 17, 1996 letter that Mr. Hakim wrote to The Honorable Paul G. Kaminski, Undersecretary of Defense for Acquisition and Technology. The opening sentence states, "The enclosed letter and survey was sent to over 60 military equipment and subsystem suppliers." The first essential in conducting an endorsable survey is knowing the total number of survey forms transmitted to potential respondents. The article states that Mr. Hakim conducted a survey of 16 defense and commercial integrators and subsystem manufacturers last May. The fact is that he conducted a survey of "over 60 suppliers" and received responses from 16 suppliers.
The cover letter to Under Secretary Kaminski also provides the insight for Mr. Hakim`s motivation for conducting the survey by stating, "It is hoped the results of this survey can be used to eliminate programs funding Mil-approved parts and to mandate once and for all that all commercial parts, purchased correctly, should be used in all systems. The exception could result from radiation-level requirements and classified design applications." The letter also goes on to say, "The survey addressed not only the savings due to the PEMs alone, but, often not discussed, the overall system savings resulting from reduced size, weight, reject rates, and improved reliability.
My analysis shows that there was no respondent majority consensus that the use of PEM in their system improved reliability. It is impossible to use the results of this survey to prove that the use of PEMs improves reliability.
Dr. Kaminski`s response to Mr. Hakim`s letter was provided on July 10, 1996 by Walter B. Bergmann II, director, acquisition practices, Department of Defense. Several sentences are quoted as follows, "I am responding to your letter to him [Dr. Kaminski] where you suggested eliminating program funding for military-approved microcircuits. We do not agree with the suggestion and will not mandate the use of commercial parts in all systems. Secretary Perry`s direction is to state requirements in performance terms, thus allowing the use of whatever military or commercial parts meet those requirements. To direct program offices and their contractors to use only commercial parts would undermine the program manager`s responsibility and authority, and transfer risk from the contractor to the government."
A retyped copy of the Hakim survey sheet follows.
Survey PEMs and Military Systems Using PEMs
1. System nomenclature
2. How many PEMs per system
3. DIPS/SMT or both
4. How many systems built to date
5. Total to be built
6. Average cost of PEMs
7. What is test cost adder
8. Average cost of MIL equivalent (either estimated or from earlier use)
9. Advantages of PEMs At device level (yes) (unknown) (no)
Cost
Cycle time (order to incoming)
Availability
Incoming reject rate
Module reject rate
Other
At system level (yes) (unknown) (no)
Cost
Weight
Through-put
Rework
Performance
Reliability
Other
10. Module/system reliability testing reject rate due to PEMs
11. System field reliability
12. Nominal environment Temp (degrees Celsius), RH(%)
13. When production units first fielded
14. Other
My analysis for the respondents survey sheets is limited to survey item No. 9.
There is no nomenclature where all 25 hardware items were rated yes. Cost, size, weight, and availability were each classified within the 88 percent to 92 percent "yes" range. These four categories are widely proclaimed PEM proponent advantages. It is surprising that each of these categories did not receive a 100 percent "yes" response.
Performance is a key system design and reliability factor. Only 10 of the 25 survey sheets (40 percent) stated that PEMs provide a performance improvement. The overwhelming degree of performance improvement proclaimed by Mr. Hakim is not validated by this survey. The degree of PEM performance improvement cannot be determined from this survey.
Eight of the 25 survey sheets replied that PEMs do provide improved reliability while eight listed the reliability as better. The total response for a reliability improvement is 64 percent. This is not a very impressive percentage to support the proclaimed reliability improvement. Reliability is variable and is IC and system dependent. The results of this survey indicate that there was no effort to compare the degree of PEM reliability versus hermetic reliability for each system.
Rework is also an item of deep interest. The survey is deficient because it only solicits information for rework at the system level. I was provided the audit results of a government quality survey team that recently visited a survey respondent listing no rework on the survey sheet at the system level. The survey team noted that rework was commonly performed by this supplier respondent at the PC board level, subsystem, and system levels for systems on the survey sheet. These systems also exhibited IC failures at the piece part level. It appears that this respondent`s survey sheets did not provide honest responses for the rework query. Incidents of this type flaw the survey.
Nine of the 25 people surveyed replied that their systems will not field production units until 1997 or later. The survey information for these systems is then very speculative. Design changes can occur, the PEM parts used to generate the survey information may not be available, etc. The speculative system information should be separated from the production fielded system information.
Mr. Hakim is quoted as saying, "At the system level the systems are smaller, lighter, and from the numbers we are seeing, the reliability is better." A review of the supplier response categories shows that no reliability data was provided. The PEM survey reliability responses leave "reliability improvement" as a question mark. "Increased reliability of PEMs results from sheer numbers," Hakim says. "The commercial parts are coming off lines generating millions of parts a month to demanding customers." The fact that millions of ICs are coming off lines, even on a daily basis, does not improve their reliability. Increased reliability does not come from sheer numbers as Mr. Hakim states.
The word reliability is indiscriminately used in the Hakim article and in everyday general usage. The word must be properly defined to establish its importance relative to the PEM versus hermetic IC issues. Reliability is the probability that an item will perform its intended function for a specified period under stated conditions. Reliability is a design characteristic and is a system-to-system variable. Reliability must be designed into the total IC.
Mr. Hakim incorrectly states that the Texas Instruments Javelin antitank missile is using PEMS. The Command Launch Unit (CLU), which is a piece of ground equipment for the Javelin system, uses PEMs but the Javelin missile is presently not using PEMs.
The criteria established for developing military hardware is prioritized as follows: (1) safety, (2) reliability, (3) producibility, and (4) economic analysis (lifecycle cost). I am still mired at the PEM safety and reliability issues. There presently is no pertinent reliability data available for surface-mount PEM; therefore, I am not as confident as Mr. Hakim that the life and safety of the user is not in jeopardy when PEMs are used in military hardware that will be subjected to years of long-term dormant storage in harsh environments.
Dr. Noel E. Donlin is assigned to the Army Missile Command (MICOM), Research Development and Engineering Center, Product Assurance Directorate, Advanced Technology and Concepts Branch, at the U.S. Army Redstone Arsenal in Huntsville, Ala. A registered professional engineer, Donlin is a senior member of the Institute of Environmental Sciences, and a member of the Institute of Electrical and Electronic Engineers. He is the team lead engineer for the MICOM Parts Technology Group and is responsible for assessing the quality and reliability of active and passive piece parts used in MICOM missile systems. He is assigned as the parts technical advisor to all Missile Program Management Offices. He has 46 years of experience in electronic hardware design, test equipment design, parts testing, parts procurement and specifications, failure analysis, parts test laboratory management, part testing programs, and the areas of quality, reliability, and maintainability. He has performed IC supplier audits for 15 years and recommends MICOM IC supplier approval. He was an instructor for seven years with the Georgia Tech Continuing Education Program conducting courses pertinent to IC quality and reliability.
