“Dive! Dive!” That two-word exclamation, punctuated with the familiar “ah-OOG-ah” alarm, was the signal throughout the ship that we were about to embark on another adventure under the sea. For many sailors, this quickly became routine, but there is nothing routine about submerging an 18,000-ton, 560-foot vessel like the USS Nevada (SSBN 733) with 165 people hundreds of feet beneath the ocean’s surface.
One small defect or fault in any of dozens of systems, or one valve out of position, and our crew would find itself on a one-way trip to the bottom of the ocean. Remarkably, since the U.S. Navy’s nuclear submarine program began, only two subs have been lost at sea, the last being USS Scorpion (SSN 589), which sank in 1968, more than 50 years ago. This kind of uptime record is worth investigating. Let’s see what a nuclear submarine can teach today’s manufacturers in their quest to avoid costly downtime.
In the mid-1990s, I served as an officer on board the Ohio-class nuclear submarine USS Nevada, before I began my career in manufacturing. In recent years, preventing downtime has created abundant interest in utilizing technology to make better maintenance decisions. At the heart of these technologies are the fundamental principles of predictive maintenance: gathering data, analyzing it, predicting failures, and taking proactive measures to prevent downtime. These fundamental principles are unchanged from how we operated the Nevada 20 years ago.
A crew of 165 people living in an underwater vessel for months at a time amplifies the meaning of hazardous conditions. Consider for a moment a transmission bearing failure causing a loss of propulsion, or a reactor coolant pump malfunction causing a meltdown, or a valve failure flooding the vessel with seawater. Nuclear submarine downtime costs lives, not just profits.
The U.S. Navy has never had a death on board a U.S. submarine due to a radiation accident. With all the complexities of multiple systems crammed into a tiny space, operating in harsh environments, how has it maintained such an amazing safety record?
The two submarine tragedies in the 1960s resulted in the Navy’s SUBSAFE (submarine safety) program, which covers all systems exposed to sea pressure or that are critical to flooding recovery. (The program was so successful that it was implemented throughout NASA after the Space Shuttle Columbia disaster.) SUBSAFE and the Navy’s nuclear power program are based on a foundation of quality in design, material, fabrication, and testing. They span the submarine’s life from initial design and construction through ongoing maintenance and updates. Strict adherence to these programs has ensured safe and successful missions for half a century. But for the most part, sailors don’t think about all of the factors that went into a submarine’s design and construction. We were far more focused on keeping it operating safely and efficiently. Navy submarines are always in a hostile environment. Beyond facing the potential threat of attack from another nation, the ship is also completely surrounded by saltwater, and storms at sea can be extremely hazardous. (This is not to mention the explosive ordnance being carried onboard.) When you are dependent on every system running smoothly to harvest your oxygen and freshwater and generally keep water out of the “people tank,” downtime must be hunted down and eliminated.
We used a variety of maintenance practices to ensure everything was working at peak performance. These included weekly and monthly preventive maintenance (PM) inspections and check-off sheets for routine items such as oil filter changes or motor-generator brush repair. The principle here is to do the maintenance at a specific interval, whether or not it really needed to be done. When set up properly, this can be an effective, though conservative (and potentially wasteful), approach to doing maintenance.
However, there is no guarantee that just because you do an established weekly or monthly PM, nothing will go wrong between the cycles that results in a system failure. So we added another feature into our daily operations – ongoing data collection and trend analysis.
This is essentially how predictive maintenance works in industry today – continuously evaluating how your equipment is working so you know when and how to maintain it optimally. We had a team of 10 trained personnel who were responsible for operating the reactor plant and electrical and propulsion systems 24 hours a day all the time we were underway. This team worked in six-hour shifts (watches). When an issue or complex process arose, this team was there to address it, often with the help of another team of 10 people who were “off duty.”
This team would record the readings of all of the hundreds of gauges, dials, and indicators throughout the engine room at least every hour, and in some cases, every 15 minutes. The watch team supervisors were expected to review these log readings every three hours, and the division officer (DO) and engineering officer (ENG) would review them daily. At first, this vast collection of numbers was overwhelming, but with experience, the team gained a sense of what was normal and could quickly identify readings that signaled something was amiss.
A key example of this is the bearing temperatures and oil-flow bubblers of the main reduction gears. The reduction gears take the power generated by steam turbines turning at thousands of RPM and use it to drive the propulsion shaft at much slower speeds, which turns the screw and propels the submarine through the water.
These gears are machined to extremely precise measurements and are carrying a lot of power. They are massive and can be replaced only in an extended dry-dock period, requiring cutting open the hull of the submarine. They are lubricated and cooled in a continuous flow of purified oil. If something were to happen to contaminate or stop the flow of oil, the gears could break or seize together, effectively shutting down the submarine’s propulsion system and rendering us “dead in the water.” There are dozens of bubbler sight glasses on the reduction gears that visually show both the flow and condition of the oil. These are recorded every hour, as are bearing temperatures, to help ensure that the reduction gears are properly lubricated at all times. Using trend analysis to find a problem in this system before it turns into downtime could determine whether or not we make it back home.
What was the point of recording and reviewing all of these numbers? There are multiple benefits that come about doing this:
- The operator recording them would have to look at the gauges every hour (or every 15 minutes). This ensures that someone is paying attention, and if the numbers are way out of their expected position, the operator will see that something is wrong before a long period of time passes, allowing for a quick response.
- Supervisors are looking to ensure that this work is being done, but of more importance, they are looking at trends. What is changing over the course of the six-hour watch period or the past day? Is there a slowly rising bearing temperature? Is something changing with the electrical system? Is pressure in the reactor coolant system changing? Is the temperature in the condenser rising? Do these changes match with what is expected based on changing plant conditions?
- The DO and ENG are looking at longer trends to see that the plant is operating normally. Additionally, regular analysis is done on the chemistry of the reactor coolant and radiation readings throughout the ship to verify that nothing unusual is happening in the reactor vessel.
- We used this data collection as a training tool, as well. Occasionally, for a drill or a test, the ENG would replace normal log sets with artificial ones that had anomalies or that gave other indications that something was happening, and the watch team would be evaluated on their ability to detect the problem and respond in the correct way.
While this was a very manual process for collecting and analyzing data, it created a rigorous and detailed system for evaluating what was going on in the entire engine room. Some elements of this labor-intensive version of predictive maintenance are being replaced thanks to today’s digital technology, but the core principles remain.
Bryan Van Itallie is COO of Grace Engineered Products and has spent time behind the wheel of an Ohio-class nuclear submarine. After obtaining a bachelor’s degree in aerospace engineering from the University of Colorado, Bryan joined the U.S. Navy, where he served aboard submarines. Leaving the Navy as a Lt. Commander, he launched into a career in the industrial sector. Van Itallie also holds an MBA from Duke University.
Clearly there is a case for predictive maintenance on life-critical systems like those on a nuclear submarine. However, modern manufacturing facilities also need this capability. For decades, companies have worked to introduced process improvements such as Lean Six Sigma and just-in-time manufacturing to streamline their operations, improve quality, and minimize expenses, thus making the organization more efficient and profitable. But maintenance has always lagged behind operations in receiving improvements. Many major manufacturers still rely on scheduled maintenance or run-to-failure approaches to managing maintenance. This strategy can result in wasting maintenance resources doing unnecessary repairs, or significant costly downtime when unexpected failures occur.
The log-taking process remains fundamentally the same, with watchstanders manually recording the data throughout the engine room on PDA’s which makes it instantly available for review by the supervisors. Some critical reactor plant readings are collected automatically and can be graphed to better identify trends.
Only recently have companies started to consider smarter ways of doing predictive maintenance. The strongest trend in this area is to use data to develop indications and predictions of failure before they occur, which is exactly what we were doing with the data logs on the submarine. Today this is done with sensors that are linked to an analytics platform which often resides in the cloud or local servers. This data can then be evaluated and used to identify changes or anomalies that could be precursors to failures. This linking of data and machines is what is being referred to as the industrial internet of things (IIoT) or Industry 4.0.
Technology has now reached a point where remote condition monitoring and analysis can be done affordably and reliably. Rather than having an army of personnel manually recording gauge readings every hour and taking the time to study the data looking for trends, now this can be done with sensors, machine learning, and analytics. More-advanced sensors now exist that enable detection of changes in vibration that indicate early symptoms of a future problem. These devices can also now communicate their findings directly to anyone who needs to know and take appropriate actions. Computing power continues to grow by leaps and bounds, enabling better and faster analysis and decision-making.
It’s been more than 20 years since I deployed on a nuclear submarine, but taking advantage of new technologies and the rich data they generate can help a variety of asset-intensive industrial organizations and facilities make smarter, timelier decisions in maintaining and operating their equipment.
Predictive maintenance will bring huge dividends for everyone, even those who don’t live on a submarine.