Infrastructure options for industrial communication networks are diverse and often complex. When they don’t deliver the necessary degree of reliability, the process becomes jeopardized and unplanned downtime, or worse, can result. Most of the bad practices seen today relate to the components selected, installation practices or design. But with a solid architecture plan, the system will perform as intended.
The challenge is that industrial networks aren’t typically built from the ground up, but rather develop incrementally over the life of a plant. Plant expansions, relocations, technology updates and remediation are among the primary reasons for altering a network infrastructure. Or the goal might be strategic: bridging the manufacturing and corporate networks, strengthening reliability and security, or adopting predictive maintenance practices that require remote monitoring of asset conditions. Less common is the construction of greenfield plants, where an entire network is built from the ground up.
Hot topics: Ethernet and wireless
Open industrial network protocols like ControlNet and Modbus have been losing favor to Ethernet, which is extremely stable and has better speed, bandwidth, flexibility and communication management than the earlier protocols. Assembly lines and warehouse picking lines are often among the first areas to be automated with Ethernet. “Some areas of a network will remain on a ControlNet-like infrastructure, with its islands of information and slower data rates, when the equipment doesn’t allow for Ethernet connectivity or it’s not cost-effective to change,” says Mara White, marketing manger at Fluke Networks.
Likewise, networks primarily remain wired where high speed or safety is critical, but other hardwired applications are giving way to wireless alternatives using Ethernet bridges, Wi-Fi or cellular networks. Process I/O used in alarming, alerting and condition monitoring is increasingly moving over to wireless, for example.
Transitioning from wired to wireless is a selective process. “As the installation cost to wire becomes prohibitive or the capacity to install additional fiber or cables is insufficient, wireless becomes a viable alternative for these functions,” explains Michael Martinez from Invensys Operations Management’s Critical Infrastructure and Security Practice. “Once a wireless infrastructure is in place, incremental wireless solutions often follow,” adds Hesh Kagan, director of technology innovation for Invensys. “For instance, real-time location systems are very inexpensive to add on and deliver a large return on investment.”
- Do you want to enable your field workers with wireless handheld devices to access data and interact with various servers in the facility?
- Can your operations survive without the information brought wirelessly?
- Do you want to manage just one wireless security system or many?
- How long do you want your wireless devices to be self-powered?
- How many devices do you want your network to handle now and in the future?
- How many application interfaces will need the wireless data?
A more ubiquitous wireless option might be needed to track assets that move among multiple locations, contractors and suppliers over large geographical areas. For those applications, cellular networks, such as AT&T Mobility’s manufacturing enterprise business solutions, fill the niche. Although the capacity of cellular networks is far lower than the alternatives, it’s suitable for many low-volume wireless asset management applications. Additionally, using a carrier-owned and -operated cellular network frees the end user from network maintenance.
Planning process: The logical view
When correctly planned, a network infrastructure will support its application and be productive. The first step is a detailed assessment of the network’s present state, from the logical, physical and security perspectives. What are its strengths and weaknesses? How does the network compare to industry standards?
“Most of the problems we see have to do with the network being built without a plan. It might have the wrong architecture or incorrect components or switches that don’t segregate traffic properly, and the network doesn’t work anymore,” explains Pete Morrell, global manager of network and security services for Rockwell Automation. In the planning phase, several considerations must be taken into account.
Application: The type of application determines the type of product used. What will it tie into? How far back into the corporate infrastructure? Users get frustrated quickly if the technology doesn’t fit the application. For example, an in-plant crane requires very different technology than a remote monitor at an isolated site.
Reliability: Wired networks are necessary when you need relentless speed and reliability. Wireless is suited to applications that don’t change very fast, like tank level monitoring. “In a 10,000-gallon storage tank, level doesn't change very fast, so any intermittent data loss because of interference doesn’t cause an upset in the process,” says Scott Killian, director of solutions connectivity for Sixnet. “In comparison, a high-speed machine application that requires lots of continuous data would not be the first location to try wireless.”
Practicality: Wireless provides the added benefit of convenience and, in some cases, can improve both uptime and reliability, such as those where cable is particularly subject to degradation. “Customers deploying wireless for the first time should ask, ‘Does it work for me?’ If they can’t get sufficient productivity and system performance for the application, the answer is no,” reasons Jim Weikert, strategic product manager for wireless at ProSoft Technology.
[pullquote]
Environment: Basic cabling is copper, but if there is higher noise, fiberoptic cable is better. Sealed connectivity and abrasion protection are needed in extreme temperature, vibration, chemical and washdown environments. Noise and interference can be challenges for wireless in industrial environments, but application engineers now have many creative solutions. If you’re considering cellular wireless, check the signal strength. “As long as there is some type of 2G or 3G data coverage on a phone, then you know a data modem will work,” says Killian.
{pb}
Bandwidth: In automation today, more data is flowing seamlessly in real time and consuming more bandwidth. Parts of a wired network might be so antiquated that they only support incremental throughput, and when new applications are added, the network might get out of balance. “The next-generation IEEE 802.11n wireless communication standard is more reliable and supports greater bandwidth and speed,” says Bill Wotruba, director of networking and connectivity products for Belden. “It helps to minimize common connectivity and throughput problems, such as when an antenna is partially obstructed by moving cranes or fork lifts.”
Timeliness: Latency defines whether data is fresh or stale. Some applications are more sensitive to network latency than to network bandwidth. “Profinet and DeviceNet are simple serial networks with tight controls on timing, allowing data to cross the network in milliseconds,” says Dan Payerle, business unit manager for the DataComm Test division at Ideal Industries. “Ethernet isn’t designed to be particularly time-sensitive. Wireless has even more chance of delay, because of low signal strength or dropped packets.”
“High-bandwidth networks don’t always guarantee low latency,” advises Al King, director of applications engineering for the Wireless Network Group at CalAmp. “Wireless networks should be optimized to meet the requirements of the application.”
Flexibility: The ability to expand an industrial network at will requires scalability, versatility and interoperability with common management tools. Ideally, the hardware should support a variety of topologies and allow versatility in how individual segments are designed.
Cost: Even when wired networks are feasible, wireless can be less expensive to install and maintain. “There are significant costs associated with running fiber,” says Kagan. “As plants expand, the demand for wireless increases. Today’s wireless allows a robust architecture that is cost-competitive.”
Security: A comprehensive risk assessment and security policy defined at the outset ensures a secured industrial network. It’s safer, easier and more cost-effective than adding security at the back end. Protecting intellectual property and keeping the plant running means avoiding intrusions, malware and denial-of-service attacks from the outside, limiting access internally and preventing accidents such as unintended switching or bumping loose a critical connection. “Some people are extraordinarily fearful of wireless as a weak link, but its security is excellent when approached in a thoughtful, engineered fashion,” says Martinez. IEEE 802.11 has new security enhancements, such as AES encryption and WEP authentication, which alleviates some concerns.
A single wireless system can support multiple devices and protocols with common security. (Honeywell)
“A robust, defense-in-depth approach incorporates multiple layers of security, from cameras or Webcams to firewalls, data encryption, VPN access controls, and user and patch management,” says Dan McGrath, industrial automation solutions manager for Panduit. “It includes blocking out unused ports and locking in critical connections, as well as setting up PCs dedicated for outside users so that guests don’t plug their devices into your system,”
Methodology: Many details about planning and implementing an industrial network can be found in “Guidelines for Industrial Ethernet Infrastructure Implementation,” developed jointly by Rockwell Automation and Cisco Systems. The guide logically organizes how to arrange the zones and structure the network, with the goal of architecting the system to achieve the intended performance, security and remote access objectives.
Network design: The physical view
Network designers tend to overlook the physical infrastructure, such as where and how to run and terminate the cables. “About 70% of network problems are tied to the physical infrastructure, and the rest originate on the logical side, like errors in bandwidth calculations,” says Morrell. To avoid problems, you need to design and install for the network’s lifecycle, taking a structured, standards-based approach.
{pb}
One of the biggest concerns in industrial network cabling is how to mitigate electromagnetic interference (EMI) problems in the design and install. “Multi-mode glass fiber is used in normal local area networks (LANs), but the newer option, plastic optical fiber, is more robust, durable, easy to terminate without special skills or tools and has no EMI or electrical noise issues,” says Payerle. Assessing the factory’s grounding and bonding system helps to avoid high ground potential differences that can lead to noise coupling, according to McGrath.
The connectors also must suit the application. “Regular RJ45 plugs aren’t suitable when exposed to oil, water, dust or chemicals, but an encapsulated RJ45 has a threaded cap that makes it watertight and resistant to adverse conditions,” explains Payerle. “M12 four-pin threaded connectors are watertight, dust-tight and ruggedized for industrial applications, but have impedance faults not found in RJ45 that can interfere with the timing of machine transmission and re-transmission.” A high-performance M12 eight-pin connector that will support Gigabyte Ethernet and eliminate impedance problems is currently being tested.
Each wireless technology has its own advantages. “IEEE 802.11 frequently is used in the plants because of the large amount of data exchanged between systems. Frequency-hopping often is used for SCADA systems because of its long-range capability. Cellular is popular with machine builders to enable remote troubleshooting at the customer’s location,” says Weikert.
A drawing with lines and boxes isn’t a network/security design. (Rockwell Automation)
A site survey is needed to understand the physical environment for wireless, because the access points and where they are located are key to a network’s success. “Access points should only overlap in certain areas, and obstructions should be avoided to ensure coverage throughout the plant,” says Morrell. There are many tools available to evaluate an environment for a wireless installation, but Weikert believes the radios themselves frequently provide the best tools to do the evaluation. “In addition to providing insight into how they’re performing, they can provide information on interference, what other networks are nearby, what channels are being used and their signal strengths.”
For remote applications needing high-bandwidth wireless Internet protocol (IP) communications, low-power spread-spectrum radios have been the norm. An alternative is a long-range, high-power IP radio such as CalAmp’s Viper. It uses the licensed radio frequency (RF) spectrum, giving it extensive coverage and acceptable bandwidth that isn’t shared with thousands of unlicensed radios.
Normally, OPC servers are found in operations centers to allow applications such as historians and alarm systems to share data. Now, in distributed operations such as utilities and oil and gas, OPC servers such as Kepware’s KepServer are moving out on network. The KepServer works with CalAmp’s narrow-band radio systems to push intelligence to the edge of distributed communications systems. “The OPC unified architecture (OPC-UA) format allows multiple applications to monitor the network and investigate traffic issues proactively, allowing network issues to be addressed before there’s a communication failure,” says King.
Companies integrating networks from many vendors might benefit from a data transfer engine like ProSoft Technology’s Message Manager, which allows Schneider Electric, Rockwell Automation and Siemens devices to talk together in industrial networks.
“While the office environment is standards-based space, the industrial environment is more like the Wild West, with a plug-and-play approach that inadvertently sets up real risks and lifecycle costs,” says McGrath. To simplify the network design process, Panduit offers a free “Physical Infrastructure Reference Architecture Guide” with physical layer product advice on how to translate the logical view to the physical plant — from the machines and control panels connected via zoned network distribution to the control room or data center closet.
Implementation: Go industrial-strength
The install is the next big area that can fall apart, especially when certified installers aren’t involved. “IT generally is unfamiliar with industrial device issues so it makes a great deal of sense to have an integrator or supplier do the initial commissioning,” explains Wotruba. “They have industrial and IT knowledge and can facilitate the conversation with the automation team, ensuring a network implementation that meets the needs of both.”
For example, pins on RJ45 connectors are numbered differently than M12, and the installer must be aware of the conversion to avoid errors in the wiring. For those companies doing their own installation, the LAN Tester from Ideal has both an RJ45 jack and an M12 jack, which gives the tester more flexibility and eliminates manual conversions, making the install, terminate and hook-up more logical.
{pb}
“People underestimate the degree of sophistication needed to implement from a construction point of view,” says Kagan. “The techniques to install radios are not obvious and involve power, grounding, cabling, antennae and enclosure issues. Customers who do their own mechanical install might create problems, such as discrepancies in the physical construction that leave coverage lacking.”
Audit and maintain
Once the industrial network is live, companies should regularly monitor its health and periodically check performance characteristics as new devices are added. An alternative is to outsource the network infrastructure and have it managed remotely. “In an unmanaged network, performance often degrades as more and more devices that require faster speeds and greater bandwidth are added,” says Martinez.
A Proctor and Gamble plant’s wired infrastructure for rotating packaging equipment was causing reliability problems because of flexing and twisting Ethernet cables. When attempts to fix it with heavy-duty cables and creative cable securing failed, the plant implemented a wireless solution from Rockwell Automation and ProSoft Technology, resulting in fewer dropped packets and no more downtime due to communication errors.
William Grant and Sons, a distiller in Scotland, chose a private network on a licensed radio band to control and monitor its groundwater extraction over extended distances wirelessly. The solution provided by Invensys Operations Management is highly secure and expandable and keeps interference from existing wireless networks to a minimum. It operates over a wireless mesh with multiple communication paths, so one failed transmitter won’t interrupt the signal transfer. A remote telemetry system was later added to the wireless mesh to control the distillery’s effluent flow.
Qenos, a three-plant plastics and rubber company in Australia, replaced its old wireless and fixed-line product transfer metering system with the OneWireless universal wireless mesh network. With the leased line eliminated, the engineers now directly access daily totals from flow computers using a reliable, lower-cost infrastructure that is designed to accommodate future wireless projects.
Mahindra Vehicle Manufacturers’ greenfield plant in Pune, India, includes a wireless input/output (I/O) control system for its electrified monorail system (EMS). The EMS conveyor supports lean-manufacturing principles by moving vehicles efficiently between assembly workstations. Rockwell Automation’s EtherNet/IP industrial Ethernet protocol and ProSoft Technology’s wireless communication systems provide the necessary real-time control and synchronization capabilities.
Regulated environments especially must audit their applications and networks. When auditing against industry standards, either you pass or fail. “We help customers adhere to standards, such as TIA for Ethernet. For troubleshooting, determining where a standard is violated typically solves the problem,” explains White.
The major industrial network suppliers have software, technology and IT tools specific to industrial automation. For example, Belden’s Industrial HiVision is a diagnostic network visualization tool that displays the network’s complete hierarchical structure and topology and provides device-level connectivity diagnostics. Among Cisco’s tools are the Output Interpreter for network troubleshooting and the Dynamic Configuration Tool.
Network management systems help identify devices that are rebooting themselves automatically or sending lots of traffic on the network or switches with outdated operating software. Without it, you might not see a problem until the system loses information or it brings down the facility.
Specialized tools can help certify and guarantee the physical infrastructure. They perform telecommunications and Ethernet testing on the enterprise side, assist with new site designs for fully automated plants, and troubleshoot and verify changes to manufacturing processes. Fluke’s EtherScope, for instance, provides visibility into devices on a TCP/IP network.
For cable continuity testing, Ideal Industries offers IENet PRO. The company’s Lantek solution tests installed cable performance, sweeping all frequency ranges to ensure it meets requirements and isolating termination problems. An adapter is available to test M12 cabling for high-frequency performance.
For security, there are tools to disable automatically any handheld devices that leave the sphere of influence of the network. AirDefense by Motorola prevents hacks into wireless networks. On the wired side, tools help to manage overloads or attacks.
Do it yourself?
Plant personnel shouldn’t have to be experts in networking, but ultimately they will own the industrial network. If there’s contention between IT and the plant floor, you might have difficulty developing an industrial network. Outside service personnel are trained to bridge the gap, ask the right questions and help the two sides overcome their language barrier.
Increasingly, the technology itself is bringing the organizations closer together despite different vested interests. “In the past, IT was a department that the plant engineers didn’t want involved in the process,” explains Killian. “Now, with the tremendous growth in Ethernet, the industrial networks benefit from the knowledge that IT brings to the process when a company looks at security, network optimization and tools to monitor the network that IT has compared to the process engineers.” As a result, in-house expertise has grown.
“During the past year or two, we began hearing that IT experts have been hired in the automation side, and less external support is needed,” says White.
“Many wireless systems can be handled well directly by plant personnel. Many vendors provide assistance for plant engineers to understand the effect of the wireless network on system performance,” says Weikert. “However, if the customer is performing a network overhaul, an integrator often can be of assistance.”
The case for outside assistance remains compelling. New ways to attack networks are increasing exponentially and it’s difficult for a shop to keep up without outside help. Trying to self-troubleshoot can do more harm than good. The available reference architecture, design and installation guides take time to read, digest and implement. There are numerous stand-alone design, maintenance and security tools available for wired and wireless infrastructures. And the technologies and standards keep evolving. “Proposed standards for the installation (IEC 61918) and safety (IEC 61784-5-n) of industrial communication networks are up for votes this year, as is the North American version, TIA-1005-A,” says Payerle.
“How can a small shop get its arms around all of it?” asks Kagan. “We help to integrate the pieces into something easy to use, because if the network and security aren’t managed properly, vulnerabilities are exacerbated and faults occur.”
Ethernet has become ubiquitous internationally, thanks to its standards-based open technology and huge bandwidth. Wireless is continuing to grow in the industrial space as the newer technologies are more capable and secure. Fortunately, there are numerous guides, standards and services available to explain the myriad options to optimize the industrial network for precise business needs. As companies get more sophisticated about networking from a complete wired-wireless-total security perspective, they’ll build reliable enterprise communication networks that they can grow with.