Podcast: Firefighting is not a maintenance strategy

In this episode of Great Question: A Manufacturing Podcast, sponsored by Hydro Inc., Thomas Wilk speaks with Ares Panagoulias and Robert McCowan about building effective condition monitoring programs that reduce maintenance firefighting and improve asset reliability. The discussion explores the foundational steps organizations should take before deploying sensors, including defining goals, assigning responsibilities, and focusing on critical assets. They also examine the role of AI and machine learning in condition monitoring, and finish with practical advice on managing alerts, driving action from insights, and maximizing ROI from predictive maintenance investments.

Below is an excerpt from the podcast:

PS: Hi everyone, and welcome to a new episode of Great Question! A Manufacturing Podcast. I'm Tom Wilk, the Chief Editor of Plant Services. In today's episode, we're going to be talking about the building blocks of a successful condition monitoring program, especially in the age of artificial intelligence, and we're going to be looking at the kind of program that also helps teams cut down on the level of firefighting within the maintenance function.

This episode is sponsored by Hydro Inc. and today we're talking with Ares Panagoulias, the Director of Condition Monitoring and Asset Reliability in Hydro’s Centaur Business Unit; and Robert McCowan, Director of Sales and Commercial Strategy for Hydro’s Centaur Business Unit. Guys, welcome to the podcast.

Robert McCowan: Tom, thanks for having us. 

Ares Panagoulias: Yeah, Tom, nice to be here.

PS: I love this topic because at the maintenance conferences that I attend throughout the year, you inevitably get the teams that are not sure how to get these programs started. They know there's a need for them. They know there's a need to cut down on firefighting, but the range of options and first steps can be overwhelming. 

So let me start with a more general question. And Robert, if you could take this one first. Let's say you're building a condition monitoring program from scratch. What are the foundational building blocks that you have to get right before you install a single sensor for that program?

RM: Yeah, Tom, that's a great question, and it's something that I frequently review with a lot of our customers before they take on a condition monitoring program from the start. Part of that reason is because the way I say it is if you're going to start looking for more problems, you're going to start finding more problems. 

So having a plan of action, delegating responsibilities, creating a hierarchy of responsibility, knowing who's going to be in charge of what really helps these plans go well. Without that teamwork, without having some type of organization, knowing how to elevate conditions from a small problem to a larger problem, having that teamwork cohesion, you're going to be left with a bunch of problems and you're not going to know where to start. And the last thing we want is for you to not do anything and to feel overwhelmed. 

So before you put any sensors on, before you deploy them, create a critical asset list. Probably take a phased approach so the issues you find are more manageable up front, and then allow that process to be repeated until all of your critical equipment is being monitored and you're able to handle it. Not that those problems don't exist to begin with, but you start discovering them, you can kind of get that information overload, that anxiety overload. Unless you're prepared to handle that and you create the right work order plan and know responsibilities, you really shouldn't be looking at putting sensors out there if you're not going to do anything with the data they provide. The full circle on condition monitoring is that if we don't take that data and turn it into an actionable item and actually make a change, then the whole thing is for naught.

AP: Yeah, I would just add to what Robert said, which was excellent. But we need to define a little bit what the goals are before we even begin to say what sensors we need or how many of them or what areas of the plant are we going to focus on? I think too often people are coming at it from a reactive standpoint, that they're not defining that upfront, and that's why we found some success with what we do in supporting clients, looking at a targeted approach at the beginning. Let's take a bad actor and let's define, hey, are we trying to improve mechanical sealing? Are we trying to improve downtime? And then you can take those concepts, those goals, and apply them on a plant-wide basis. 

PS: Let's move to AI and machine learning, since that's kind of in the air. And maybe Ari you can take this one on first. We hear a lot about AI doing some of the heavy lifting when it comes to data analysis. But as Robert said, there's a human element that has to be in place for these programs to be successful. In your opinion, why is the human element still the critical missing link in interpreting that data?

AP: I'm starting to think that the human is irrelevant based on all we hear about these days. It's just AI, AI, AI. It seems like the only companies getting VC funding have AI in the title, in the name, in the core product. It's remarkable. 

No, we're big believers in the human element. And at the end of the day, the machine learning, the capabilities in terms of pattern recognition, they are making leaps and bounds. There's no doubt about that. And the intelligence of the sensors and the software that are involved in remote condition monitoring, for example, those have made significant progress, specifically in the last three to five years, and they'll continue to do that. But what do they know about the context of the machine? And what will they do in terms of the maintenance action to execute what the recommended work orders are? 

There's still a need, a dire need for (1) translation of the data, and (2) subject matter expertise, and that's the key area that humans still play a role in. We think that the best approach to this involves both the aspects of machine learning and AI, but also that subject matter expertise, also that machinery context and understanding. You can't divorce the two. I know we're going to get down to this in a little bit, but when we talk about the best ways to execute these types of programs, they involve all of that, and I see too often people sort of going to the allure of, “oh, it's all AI, AI is going to tell you exactly what to do and what the maintenance actions are.” We're just not there yet. Will we get there one day? I don't know, probably. But we're not there yet. We're not going to be there in the short term. And so when we talk about implementing these programs, it's taking the best of all of those facets and marrying them together. That's the challenge.

RM: I'd add to that, we're talking about critical equipment. It can be nuclear plants, it can be power plants, chemical operations. Are you really going to subject your critical equipment to evaluation by an AI system? And if you are and you pull a machine out for repair, which the AI got wrong, because often it can, now you're behind on timeline of repair, trying to fix problems that weren't problems to begin with. So we really don't want to induce problems from a system trying to just believe that it's an end-all, be-all solution without having that verified by a real human who really understands your rotating equipment.

And furthermore, the system can't make judgment on what it's not privy to see. If it doesn't have adequate data fed into it – process data, environmental conditions, whatever they may be, and it's only relational to maybe pressure flow or vibrational data – it's still missing key elements to how that machine's operating under certain conditions. So you're taking a limited data set and you're trying to apply a comprehensive AI program to that, which is just not able to do without that human feeding and verifying the data.

PS: One of the outputs of these programs are dashboards, often highly automated dashboards, which can help process the data and show them in real time to the technicians who need to see them. Robert, in your experience, is there a risk that these dashboards can make technicians less likely to walk the floor, sort of gathering the operational context, as you were saying? Maybe even newer technicians and digital natives who might be gravitating towards the dashboards.

RM: Yeah, that's a great question, and the way I see it is a lot of the old school methodology or traditional methodology was, do your routes, walk by your equipment. People understand that they sound differently. If you're around rotating equipment or your rotating equipment long enough, you know how they should sound, how they should feel, even the floor vibrates a certain way. You get a sense of that, and that's all well and good, but you're still left doing this manual route, this walk-by, it takes a lot of time and it's inefficient. 

Where the dashboards, it could make you lazy, sure. But they could also give you a laser focus, which allows you to see the problems picked up by live monitoring in real time and say, something's not right here, now let me go to that asset and go do a deeper dive. So I think like anything, even with AI, we were just talking about there are those who will use AI and become lazy because of it, and then there are those who will use AI and become stronger because it's a tool that helps them, and that's the same situation that I think the dashboards are going to create. If you use it appropriately, it's going to make you more efficient and give you better visibility into issues that maybe you didn't pick up because they didn't sound different.

AP: It's funny, I think 10 years ago, we were convincing older technicians and folks that have been doing things a certain way, that the way they've done it should be changed, and utilize this new technology to bring some of that digital efficiency into their day-to-day. I wonder if in 10 years we're going to be trying to convince digital natives that they should go out to the plant and actually get a sense for what the equipment feels like, smells like, all those other different sensory ways that we can evaluate the health of rotating equipment. You get a signal, a vibration signal, that says, hey, there's looseness in this pump or this motor, and you see that on the data side and you're wondering what to do. You could just walk the plant and say, oh, look, that bolt, I can visually see that it's loose. So it's kind of interesting the way that it's maybe almost flipping.

RM: It's a pendulum, right? Back and forth.

AP: It's about like anything in life, right? It's about finding the right balance, the right efficiency, the right way that we can utilize a lot of data to manage a lot of assets with less and less labor. But I also think there's no replacing the amount of information you get from being next to something.

PS: Ari, is there a rule of thumb that you use to balance screen time, so to speak, with actual machine time? Or do you sort of help people understand what the operational context is to set that balance?

AP: It looks different for everybody. You know, if you're working with a midstream company that's got assets up and down, let's say, the US or Canada, that might look a little different because the screen time might be a great way to reduce driving time. But if you're working with a refinery, a chemical process plant that's right there next to the equipment, maybe you can get out to it a little bit more. 

So I think you've got to tailor that approach to the type of company you're working with, what type of resources they have, how many people they have, what the skill set of those people are. It looks different for everybody. And I think that that's really what the technology enables, is a little bit of right-sizing. You know, do we need to support on the analytical side so that they've got a little bit more muscle in terms of making sense of the data? Or do we need to provide a little bit more onsite support so that we can execute what the data is telling us to do? Everyone's a little bit different.

PS: Let me ask you a question about alerts, Ari. I've seen this come up time and again when people start setting up their condition monitoring program, they set their baselines for when the system alerts them that action might be required, and suddenly they are flooded with alert after alert after alert. And instead of being people who are tempted to pencil whip actual pen and paper, they're smashing buttons on their phone to get rid of the alerts. Given that modern sensors can generate so much data, terabytes of data, what is your process for filtering out all that noise, identifying the signals that warrant opening a work order, and minimizing for want of a better phrase, “alert fatigue”?

AP: If you’ve got an answer, I'd love to hear it! This is one of those things that I think is coming up, not just in industrial life, but in real personal life too. I wear an Apple Watch and I wear it to sleep and I get the sleep data. I've got three young kids. My sleep is not great. And I get a notification, “hey, you had 10 interruptions last night, you really need to work on your sleep score.” Do I need that alert to know that that's the case? Not really, but here we are anyway. 

I think for industrial sensors, there's often a break-in period when you first install those, and you do need to tailor what those baselines are. I think we're coming out of an age where people are relying on standards for what “good” looks like, and they're tailoring those alarms to the baseline conditions, and they're looking for deltas from that. I think that's a first-process way that you can get rid of a flood of alarms that aren't going to lead to work orders, or are going to lead to folks turning off the value of these types of systems. 

The other one I would say is filtering techniques that do utilize AI can be helpful in reducing the amount of alarms. And so we've seen, as we've iterated our software, including more of that on the front end before we, let's say, release those alarms directly to clients, that's helped tremendously. I'd say it's driven down alarm volume by about 80%, nuisance alarms. So I think there's a couple of technology approaches and then a couple of process approaches that can really reduce it. But it's a great question because when we started this, going down our path eight years ago, we would turn on these systems at plants and it would just be a deluge. And some folks would just say, “you know what, I can't even deal with this, it's too much.” And I get it.

RM: I think that goes back to my earlier point about having a workflow hierarchy role responsibility. That way one guy isn't getting every alert for every piece of equipment. If he's got a team that he can kind of filter through, then maybe he's got four guys underneath him and then they each get 1/4 of the work, that can definitely help. There's going to be some type of ramp-up period when you start to learn to understand how the machine works, tailoring those alarms and those situations and making sure that the data we're collecting is more accurate to how it likes to run rather than just a general approach. So there is a ramp-up to that and there is some fatigue overload. But once it settles in, it settles in nicely and then things, they work pretty well.

AP: I think another piece of this is closing the feedback loop. And I think it gets underrated, but the back and forth between the user of the system and the system, or the folks that are also involved in analyzing the data – the more tightly you can close that feedback loop, the more likely it is that you're going to manage those alarm settings and get them to a point where they are actionable. 

I think sometimes when people don't invest that time on the front end to really have a tight feedback loop, you see just a whole bevy of alarms get generated with no action, and then you start to lose faith and confidence in the system. So if you're willing to work up front and put a little bit of time in to developing the system, I think that goes a long, long way towards its efficacy down the road. And ultimately, it gets to what we're all after with these types of systems, which is making an investment and seeing a return, seeing a reduction in downtime at the plant, seeing a reduction in maintenance spend on a certain asset. That's what we're after. 

I would say that an underrated piece of it, too, is reducing the amount of headaches and calls in the middle of the night. But that's something that's a little bit harder to put a dollar figure to it in all these cases.

PS: I think you're both putting your finger on something behaviorally important to plant teams that take on these condition monitoring programs. You don't want to get used to just pushing the button “no,” “no,” “no,” “no…” when you get the alerts. You need some way to filter out the noise because the last thing you want is to be conditioned to hit “no” when they actually have to hit “yes.”

AP: And the behavior piece is interesting because I think the tech, the tech has sort of been able to achieve what we wanted. I think back 20 years ago, the tech was behind where maybe our aspirations were. We were trying to get the tech to work to fit into a paradigm that we had in terms of doing maintenance. Now I think the tech from the AI side to the sensor side to what can be monitored and how quickly it can be scaled, I think that's ahead of maybe where our theories, paradigms, and behaviors are. So I think that's really where the work is today.

PS: We'll get you out of here on this one, guys. When a plant invests heavily in a predictive maintenance program and it fails to deliver the expected ROI, either overall or the time window is too farther out than people want it, in your opinion, guys, what is usually the root cause? What's the first place to look to tweak that program or fix it?

AP: I think right now I would look at the process. I don't doubt the value of the data, but it goes back to first principles things that we talked about early on, which is what are the goals of the program? What are the goals related to the specific asset? What are the goals of rolling out the condition monitoring program to begin with? When there is an ill-defined expectation, that is just going to manifest as an ROI that either can't be calculated or isn't what we're looking for. But if we define that clearly upfront, it sets us up well to achieve that. Robert, maybe you want to help clarify what I'm trying to get at here.

RM: I like using analogies for a lot of things and my wife gets tired of them, but I'm going to use one here. It's like going to the doctor and you'd get an x-ray done and he says, “you've got a broken leg.” And if you button up and go home and don't put a cast on it, don't do anything, maybe no surgery, you’d expect that to get better? So our sensors are going to give you the data, and they're the X-ray. They're going to tell you what needs to be done. But if you don't take the steps to fix that equipment, there is no ROI. So we have to employ the continuous improvement cycle, which is closing the last loop, closing the loop within the last step, it’s turning the data, the recommendations into an action, the immediacy issue. And you then rinse and repeat that cycle. That's how you're going to get the most out of the system.

AP: Even in that you are assuming that we're even making maintenance recommendations. How many systems are out there that just raise their hand and say, oh, there's a high temperature event. Oh, there's a high vibration event, you know, without even an assessment done on potential root causes or a probability matrix of what might be going on or including the machinery context in that support/refute matrix.

RM: How many times have we offered recommendations and we come back months and months later and nothing's been done and the customer's taken no action? I think it comes down to that philosophy: the condition monitoring technology won't turn the wrench, it's not going to fix it for you. It's going to show you where to go turn the wrench, and with what we do, with the expertise we have and the skills we have, we're able to help advise you on exactly what needs to be done, but you’ve still got to take the action.

PS: Oh, well, I got to be honest, guys. This conversation is prompting me to take action on some dental x-rays that I had done and actually commit to a course of action. Now I’ll call up my provider today and get the next appointment set. So now thanks for the conversation on condition monitoring programs. These are challenges which every plant faces, and I appreciate your insights into how to solve start solving them.

RM: Thanks, Tom. Appreciate the question today.

AP: Happy to chat. Thanks, Tom.

About the Podcast
Great Question: A Manufacturing Podcast offers news and information for the people who make, store and move things and those who manage and maintain the facilities where that work gets done. Manufacturers from chemical producers to automakers to machine shops can listen for critical insights into the technologies, economic conditions and best practices that can influence how to best run facilities to reach operational excellence.

Listen to another episode and subscribe on your favorite podcast app