Establishing an In-house Predictive Maintenance Program

By Brent Miley John Shinn Jr. Jack R. Nicholas Jr.

While many companies are perfectly satisfied with the performance of their outsourced predictive maintenance (PdM) programs, some do not get the desired results for the time and money expended. Others recognize that they have a pool of in-house talent that could do the job given enough time and training, or are facing budget cuts and pressures that force limiting, reducing or defunding PdM contracts.

Whether your organization has an established contractor-provided PdM program that you want to bring in-house or is starting a program from scratch, it should be done only after careful analysis of the expected benefits and problems that are likely to be encountered. For example, it takes a considerable amount of time for PdM technicians to become proficient in some technologies, particularly vibration analysis and infrared thermography.

Having a competent PdM contractor already engaged with your assets will make the transition to an in-house program easier and quicker if the contractor is willing and able to train your personnel in the technologies already being applied. Most attempts to bring a PdM program in-house fail not because of the contractor but because of internal factors such as personnel turnover and failure to select workers suited to the practice of predictive maintenance. Related issues include but are not limited to:

• Lack of computer literacy among those who would like to be PdM candidates;
• Inability to learn complex PdM technologies;
• Lack of appreciation by managers, supervisors, team candidates and co-workers for the difficulty of achieving competency in a complex PdM technology;
• Failure of managers and co-workers to appreciate that a PdM technician’s work is every bit as demanding as that of maintenance crew personnel;
• Failure to create and maintain a current PdM program master plan;
• Failure to establish and defend an adequate budget for all aspects of a PdM program;
• Failure to educate management, supervisors and co-workers on the benefits of a PdM program;
• Failure to continuously calculate the return on investment and document other benefits of a PdM program;
• Failure to provide for retention of PdM technicians after they become competent in assigned technologies; and
• Failure to establish a succession scheme for PdM team personnel who retire or move on to jobs with greater responsibility.

The last item above is particularly important. When ideal candidates are selected for a PdM team, management must expect that sooner or later at least some of them will move to better-paying, higher-level positions. Preferably, this will be within the current organization where they should become “champions of PdM.” The worst-case scenario is that they leave with only a two-week notice and go elsewhere. It then may take months to identify novice candidates or to hire partially experienced or even certified replacements and get them up to speed. In the meantime, monitoring languishes, reliability decreases and the PdM program may even be abandoned if its true worth hasn’t been documented. This can result in a decline in availability while the maintenance strategy reverts to more costly approaches such as reactive maintenance. To mitigate this risk, it is a good idea to continue a relationship with the PdM contractor to support the program when required.

Another pitfall to avoid is deciding that a PdM program can be done cheaply. With this strategy, simpler, easy-to-learn PdM tools are acquired, but more sophisticated suites are never applied, even with contractor support. In these cases, the diagnostic and long-term analysis capabilities of advanced PdM software and the full potential of a comprehensive PdM program are never realized.

The opposite situation can also create problems. Sophisticated and costly PdM tools are purchased and put into the hands of maintenance crew personnel who are given training they either don’t apply soon after the classes or have too few opportunities to learn how to apply them on the job.

Benefits of a Two-tiered PdM Program

A two-tiered PdM program that applies multiple technologies can be beneficial for many organizations, especially those with plants dispersed over a wide geographic area and/or where semi-autonomous maintenance crews are employed. These benefits are over and above those of a traditional team-only or decentralized, non-integrated approach.

Under this two-tiered arrangement, a skilled PdM team with sophisticated data-collection hardware and analysis software supports and collaborates with maintenance crews equipped with simpler but sufficiently sensitive tools for detecting and confirming problems in their plants. With this type of program, local maintenance crew personnel are empowered to declare an asset ready for return to operational service.

The PdM capability provides maintenance personnel with their own quality-assurance tools. This is done by having the local maintenance crews responsible for post-maintenance testing, which is performed to determine if a restorative or mitigating task has been successful in fixing a reported problem, as well as to ensure that new problems weren’t introduced in the course of maintenance. In the event post-maintenance testing reveals an abnormal condition, help from the PdM team can be requested to define what may have gone wrong during maintenance so it can be remedied.

There is also a division of labor and responsibilities in the employment of predictive technologies between the PdM specialists on the PdM team and the PdM practitioners on the maintenance crews. This can pay dividends in three ways:

1. The maintenance crews, which are equipped with simple but effective tools, can perform post-maintenance testing and make the “go/no-go” calls with a reasonable level of assurance. This boosts their confidence and self-esteem.
2. It enables staff through involvement in PdM technology to determine if they are interested in becoming future PdM team member candidates.
3. The PdM team is relieved of having to perform post-maintenance testing under time constraints and pressures to return the asset to service as soon as possible.

Another advantage of a two-tiered PdM program is that it offers added opportunities for cooperation between the PdM team and local maintenance crew members. For example, when baseline testing by PdM team members is being conducted, maintenance crew PdM practitioners should be present with their PdM tools to take readings at the same time. This allows the maintenance crew members to compare readings from their simpler tools to those taken by the PdM team members on their more sophisticated equipment. This will educate both groups on each other’s capabilities and limitations, and enhance future cooperation.

Case Study: The Metropolitan Sewer District of Greater Cincinnati

The Metropolitan Sewer District of Greater Cincinnati (MSDGC) serves 800,000 customers and has approximately 600 employees who work at facilities located throughout Ohio’s Hamilton County. The MSD Wastewater Treatment Division operates and maintains seven major treatment plants and more than 100 smaller treatment facilities that process an average of 180 million gallons of raw sewage per day. Most of the major treatment facilities were built in the 1950s and contain more than 16,000 total discrete assets that are critical to meeting MSD’s mission of protecting public health and the environment through water reclamation and watershed management.

When the Wastewater Treatment Division decided to establish an in-house PdM program, one of its strategic goals was to build internal PdM core competencies aligned with the mission and vision of the organization. This required alleviating problems associated with data collection and correlation, communications (particularly feedback and follow-up on PdM-initiated work orders), and planning and scheduling. For several years, the organization had been using local maintenance personnel to perform ultrasonic analysis as well as online and offline motor testing to help complement its contractor-managed vibration and infrared (IR) thermography program.

MSD’s contractor had no access to its in-house technology or computerized maintenance management system (CMMS) databases due to strict firewall maintenance. This made correlating data very difficult or impossible for those outside the firewalls. All in-house technology databases were managed by MSD planners who issued work orders based on the results of the in-house testing but without much internal analyzing. The planners also wrote work orders from the quarterly PdM report provided by the contractor. Opportunities for data correlation were often overlooked or too difficult for those not trained or dedicated to doing so.

The Metropolitan Sewer District Wastewater Treatment Division operates seven major treatment plants and more than 100 smaller treatment facilities that process an average of 180 million gallons of raw sewage each day.

The previous program operated under a basic plan. On a quarterly basis, the contractor performed vibration analysis and IR thermography on assets deemed critical to the wastewater treatment process. An asset list was sent out the week before the contractor’s quarterly visits. The week of the visit, the contractor would collect data on all operating equipment during the first day or two. On the second or third day, the operations department was required to switch “missed” equipment so it would be running during the last days of the data-collection week.

While this may seem to be an adequate plan, there were inherent flaws that took years to correct and required work-arounds to be devised. Not having in-house PdM technicians skilled in even basic vibration analysis and infrared thermography became problematic almost immediately after the start of the contractor-based PdM program. If critical equipment could not be switched by operations (for any number of reasons), that piece of equipment was missed until the next quarterly contractor visit (or longer).

Delays in post-maintenance testing (PMT) were another issue. When the contractor identified a problem, the in-house maintenance crew would make the repair but had no way to tell if the reported defect had been corrected. Since MSD could not allow a machine that wasn’t repaired properly to operate (or remain idle) for a full quarter before PdM readings were taken, the organization decided to call in the contractor to perform PMT. If satisfactory conditions were found, new baseline readings would be obtained before being turned over to operations. However, getting the contractor to the asset to be tested and having the results analyzed and reported might take several days.

In developing this process, MSD discovered that it was calling in the contractor several times a month for PMT on only one item, which was expensive. A work-around was designed to minimize cost. The contractor would only be called in when more than one asset was ready for PMT. Unfortunately, this put the organization back in the same sinking boat; now they just had to use a bigger bucket to bail.

After several years of trial-and-error and countless meetings to improve the program, MSD decided it needed an effective in-house PdM program. In late 2011, through a series of workshops with operations, maintenance and engineering, MSD refreshed its shared vision, mission and core values. To support the mission and accomplish its vision, MSD developed a strategic maintenance and reliability plan that had six goals and objectives. One of the goals was to reduce downtime through a centralized and decentralized predictive maintenance program. An expert with more than 40 years of PdM experience was chosen to guide the organization in the program development. A five-year plan was created outlining tools, training, software, workflows and budgets. The result was a two-tiered system of low-end and high-end PdM technology tools and users serving all seven plants.

High-level tools such as vibration analysis suites, IR thermography analysis software, ultrasonic analysis devices, and online and offline motor-circuit analysis suites were placed in the hands of PdM team members selected from the in-house maintenance crews based on their past experience and aptitudes. Previously, motor-circuit testing had been carried out by maintenance crews but with highly variable results across the division. This was due to personnel turnover and the need for substantial training and field experience to become proficient in the use of the equipment and in performing proper data interpretation after testing.

Following some research by crew leaders, vibration meters, simple infrared “guns,” laser alignment tools and ultrasonic detection devices were purchased based on their suitability for wastewater treatment assets and the maintenance personnel who would be assigned as PdM practitioners.

The PdM contractor supported a training, mentoring and certification program with appropriate milestones, number of hours and courses needed for the team to become capable in vibration analysis and infrared thermography. Quarterly contractor visits for monitoring became “shadowing” sessions where the PdM team would observe the data-collection process and learn the basics of the technologies. The contractor also conducted training and certification examinations in visual inspection for both the PdM team and maintenance crew personnel.

After one or two quarters, the PdM team started collecting the data. Reaching this point was a huge milestone in the program. Instead of missing equipment each quarter, the team began collecting data on equipment throughout the quarter and sending it to the contractor for analysis. As proficiency was gained in analysis, team members progressively assumed this responsibility also. The PdM team now initiates work orders in the CMMS for crews to repair and conduct PMT. This more complete approach to data collection has been well-received by operations.

Ultrasonic sensor-aided grease guns, which had been acquired earlier and used by local maintenance crews for some time, would continue to be utilized by them. Sophisticated ultrasonic analysis devices that had previously been issued to the maintenance crews but not used much because of their complexity and crew member turnover were transferred to the PdM team. Maintenance crew PdM practitioners who received and were trained on the simple-to-learn tools began using them to conduct PMT.

An existing laboratory-supported lubricant and wear particle analysis program remained in the capable hands of local maintenance crews and their planners. Another key was having personnel both within the PdM team and the local maintenance crews with the desire and drive to make the PdM program work.

Annual alignment PMs and workflows were developed to obtain baseline readings on newly installed equipment and rebuilds or major repairs. Once alignment is completed, a task is created for the PdM team to acquire new baseline readings.

After the newly formed team was in place, a predictive maintenance management software (PDMMS) contract was established to allow personnel to manage and correlate PdM data from all the efforts of the team, the outside contractor and the maintenance crews. The PDMMS is Web-based and operates outside of network firewalls, which requires some manual transfer of data (such as work-order numbers, etc.) between the CMMS and PDMMS.

Classroom training by the contractor and technology vendor instructors along with in-the-field training and mentoring has proven invaluable. Once hours of on-the-job practice are met, certification in the assigned technologies is required. This is overseen by a PdM specialist from the outside contractor, who provides vibration analysis and infrared thermography services until the PdM team takes over. After two years, all PdM team members are expected to be certified in at least two technologies.

The most important part of the two-tiered PdM system is the involvement of the local maintenance crews. Instead of a PdM team coming out and telling them what to fix and whether they did their job correctly, it’s more of a unified approach.

The hard-working staff at MSD along with the senior and executive leadership have been essential in promoting a culture that is striving toward world-class maintenance reliability. While their simple approach of fostering a proactive culture based on proven best practices works, it requires time and energy. A commitment to follow-through at all levels is delivering results, and efforts to continuously improve are being rewarded by a noticeable culture change and success stories.

The vast improvement in the overall program is evident by the increase in proactive labor from 46 percent in 2011 to 70 percent today. The monthly emergency failure rate has also decreased by 55 percent, and documented cost avoidance has reached more than $650,000 due to proactive maintenance activities. Thus, assets are more reliable, and downtime has decreased as a result. Less than halfway into its five-year PdM plan, MSD is excited to see what the future holds.