Case Study 6

Health Drink Manufacturer Experiences Failures of Variable Frequency Drives & Industrial Power Supplies since Opening

An industrial plant in the Western United States was a researcher & manufacturer of health drinks in the forms of liquid & powder. The plant was designed specifically to provide custom products to major fast-food customers that sold them to the public. This plant was designed by the manufacturer who was the original occupant.

The plant was about 200,000 square feet on two levels. The building was a concrete re-enforced building with a rubber roof sitting on a concrete slab. The plant was fed by four 2,500-kVA utility service transformers—two located on the east end of the plant & the other two located on the west end. The transformers were fed at a primary voltage of 25-kV & delivered a secondary voltage of 480 volts to each of the four 4,000-amp switchgear stations installed inside the plant on the two outside walls on the east & west ends. None of the switchgear stations were fully loaded & could accept additional load if the plant were to expand or upgrade its production lines.

Because of the number of health drinks (liquids & powders) the plant produces for its customers, the plant houses about 36 different production lines. Some of the plant also produced some key ingredients used in the manufacture of several of the drinks. Each production line was a specific process designed specifically for each product. Each production line used many power / control cabinets that powered & controlled the process from beginning to end on each line. Industrial power supplies (IPSs), programmable logic controllers (PLCs) & variable frequency drives (VFDs) were among the major pieces of electronic equipment used in each cabinet. The cabinets used anywhere from 8 to 35 VFDs per cabinet. All of the VFDs were from a single manufacturer & installed at the same time. The range of horsepower of the VFDs was from five to 15. Some of the cabinets were powered at three-phase 120/208 volts with a four-wire circuit. Some of the cabinets were powered at three-phase 277/480-volt systems with the same four-wire approach.

Since the plant was commissioned, VFDs have been failing. The failures were sudden failures with some visible evidence of arcing on the printed circuit board that contained the switching power electronic (IGBTs) components. No other components seemed to be failing. VFD failures continued for several years. The plant reported that when any of the VFDs failed, the production line that relied on those VFDs was down for several hours & sometimes for an entire eight-hour shift. Thus, the failures did impact the plant’s ability to keep on schedule.

The plant’s inhouse electrician started looking to see if he could determine why the VFDs had been failing. He checked to see if he could find anything wrong in any of the power / control cabinets. He measured the line voltage with a digital voltmeter & found that the voltage was within acceptable levels. He checked the connections in these cabinets & didn’t find any problems. He was unable to determine why the VFDs had been failing. Yet, they continued to fail at a rate of about 30 per month with the failures being spread across random cabinets & production lines.

One day the plant experienced a few IPS failures. The new plant manager decided that the occurrence of IPS failures combined with the continued VFD failures was enough to start trying to find someone who could determine why these failures were occurring. The plant manager told his electrician to get on the Internet & find an expert who they could talk to about the failures & what to do next.

The electrician found a company called PBE Engineers. He contacted them & told them their story. The electrician asked the PBE engineer if the incoming voltage from the utility company could be causing the problem. Based on the history of failures, PBE said it was possible but not likely. PBE asked the electrician to send them the single one-line electrical drawing of the plant’s electrical system, a few of the failed VFDs & a few of the failed IPSs. After receipt of the drawings & these devices, PBE reviewed & examined them. After PBE examined the devices, they contacted the electrician & plant manager to schedule a date & time convenient to them for a conference call.

During the call, the PBE engineer discussed what they had found when reviewing the drawings & examining the failures & asked if it would be ok if PBE designed a plan to determine why the failures were occurring, how to solve the problem & how to prevent future failures. The customer agreed that would be the best approach given the history of the problems & their desire to find a solution.

In addition to the questions the plant manager, maintenance manager & plant electrician asked PBE, the electrician told PBE that although he thought the plant’s electrical system was compliant with the latest National Electrical Code (NEC), he believed there were some internal grounding problems that were contributing to or causing the device failures. The PBE engineer explained that the NEC was designed to help protect against electrical shock & fire but was not designed to help ensure the performance of the plant’s electrical system & electronic loads & not designed to protect the plant’s electrical system & electronic loads from power quality problems. This made sense to the customer team.

PBE Engineers assembled a proposal & contacted the customer again to schedule a date & time to present the plan. PBE explained to the customer that power quality monitoring should be done along with an evaluation of the switchgear stations & each downstream electrical system that powers the power / control cabinets controlling the production lines. PBE also explained that monitoring starts at the beginning of the evaluation & continues through the evaluation as well as after it’s completed to ensure ample time is given to capture disturbances that could be contributing to or causing the device failures. The customer liked the plan & decided to have PBE visit the plant to start the evaluation to determine why the devices had been failing.

Upon PBE’s arrival at the plant, they installed four temporary power quality monitors, so they could collect monitoring data remotely. (Remote monitoring is used to ensure PBE engineers receive the monitoring data during the monitoring period while & after the evaluation is completed.) PBE installs its monitors with the power ON, so no interruption to power is needed. (PBE provides & wears their own full set of PPE for arc flash protection.) After PBE installed the monitors, they started evaluating each of the four switchgear stations, working their way downstream to the branch circuits feeding the power / control cabinets as well as the cabinets themselves. The electrical system in the cabinets is the last opportunity for the IPSs, PLCs & VFDs to “see” an acceptable low-impedance ground & acceptable voltage quality of the 480-volt supply. If the electronic devices in these cabinets cannot “see” a low-impedance ground & a ground that is stable, their risk of failure is high.

PBE’s evaluation of the switchgear stations & downstream electrical systems revealed several wiring & grounding problems in the switchgear stations as well as several problems in downstream electrical panels & dry type transformers. PBE also identified several problems in the power / control cabinets that increased the risk of the IPSs, PLCs & VFDs failing in the plant. All of the characteristics of these problems were documented in a detailed manner using digital photography. Additional specialized power quality measurements were made to characterize the quality of the grounding system at each point from the switchgear stations to the power / control cabinets. The evaluation had to be conducted with the plant running, so the PBE engineers could assess the performance of the wiring & grounding system while the plant is operating. No changes to any part of the plant’s electrical system were made (by plant staff) during the evaluation to ensure none of the electrical conditions were changed until the evaluation was completed.

PBE also examined the monitoring data (collected so far) that had been collected during the evaluation & determined that no incoming voltage quality problems were occurring at any of the four switchgear stations during this short period. (As mentioned above, the monitors were left in place after the evaluation was completed for a period of at least one month.) PBE did determine that some neutral-to-ground voltage transients that could damage the IPSs & VFDs were occurring at the monitoring points & at various locations across the plant’s electrical system. However, none of these transients lined up with any specific loads switching ON or OFF in the plant. The transients that were recorded occurred at random dates & times but did not coincide with any IPS or VFD failures. PBE explained to the customer that damage to some of the IPSs & VFDs had occurred, but failures didn’t occur until at least one electronic component inside the device had failed. In other words, the IPSs & VFDs failures that occurred were latent failures. PBE also explained that latent failures sometimes occur in these types of cases.

The monitors were left in place to collect data for about a one-month period. While the monitors were continuing to collect data, PBE was assembling the technical report of their findings when the plant’s wiring & grounding system was evaluated. PBE also included the results of the monitoring at all four switchgear stations as well as monitoring conducted at selected power / control cabinets where some of the VFDs & IPSs failed.

PBE scheduled a date & time convenient with the customer to present the confidential & proprietary final report. The report contained the problems found in the wiring & grounding system, the performance of the grounding system & the types of repairs & upgrades that needed to be made to the system to help avoid future failures of their electronic loads. PBE explained to the customer that if the repairs & upgrades were made, the risk of other devices failing would be significantly lower. They also explained that conducting the repairs & upgrades to the plant’s wiring & grounding system would not guarantee that no failures would ever occur. Failures of parts of the electrical system & the electronic loads it supports will always occur but occur in a reasonable & expected time period when system components & devices are expected to fail due to them reaching the end of their life. PBE also explained that if a catastrophic failure of some sort were to occur in the plant’s electrical system, system components & electronic devices could fail.

PBE also explained to the customer that PBE could conduct the repairs & upgrades to their wiring & grounding systems. The customer asked PBE to explain some of the benefits in having that done. PBE explained that the best benefit was to ensure that the repairs & upgrades were done correctly & that PBE could verify the improved performance for the customer.

Lastly, PBE reminded the customer why power quality is important to the life of the plant’s electrical system & the electronic loads it powers & that best power quality engineering practices should be practiced throughout the life of the plant to manage risks & help ensure the system could reduce the risk of failure to its components & its downstream loads.