Root cause analysis in the video age
Condition monitoring has eluded food and beverage manufacturing for the most part, but the video component of that technology is carving out a troubleshooting niche.
Just as engineers can’t control what they can’t measure, maintenance professionals can’t fix what they can’t see. As fillers, packaging machines and process equipment ramp up in speed, seeing when and where a disruption occurred becomes more difficult for the human eye. That’s creating an opportunity for video recording systems that allow frame-by-frame review for root cause analysis.
Cracks in pump shafts and turbine blades can have catastrophic consequences for a jet engine or a power plant, and a number of suppliers have devised systems that analyze signal data and detect faults in rotating parts. Among them is Monitoring Technology Corp., Fairfax, VA. Sophisticated software to process digital machine signals lies at the heart of the company’s condition-monitoring system. Cameras also are a component in many set-ups. Six years ago, the company packaged camera technology with custom software and introduced 20/20 Hindsight, a troubleshooting tool for process industries. Second-generation systems now are being installed at food and beverage facilities where high-speed lines can mesmerize technicians who try to eyeball process upsets.
Helping to train plant maintenance specialists on the system is Product Manager Russell D. “Rusty” Leonard. Leonard joined the firm in 1999 after working with Hughes Network Systems, the US Navy, British Aerospace, NASA and the European Space Agency, primarily as a consultant on satellite systems. He earned a BS in computer science with a minor in mathematics from James Madison University before attending the University of South Carolina, where he was awarded an MBA.
FE: Compare the video unit with a condition-monitoring system.
Leonard: Video is used with both systems. Some of the monitoring systems we’ve installed in paper and pulp plants have a big (computer) rack and up to 20 cameras wired into it. They can detect a break in a sheet at any point and tell the operators what to do to correct the problem. With 20/20, we don’t tell operators and maintenance personnel to do anything. We simply provide them with an ultra slow-motion video so they can see what happened and determine the corrective action. When people watch a line with the naked eye, the mind condenses the scene to see what it expects to see and misses the event.
In a food plant, the camera usually is enclosed in a dust- or water-proof enclosure. It’s attached to a stainless steel cart by a 25 ft.-long cable. The cart houses a video monitor, keyboard and PC with a 1 terabyte (1,000 gigabytes) hard drive. Originally the system could only buffer 10 to 60 seconds of video; now there’s enough memory to store over 50 hours of color video at 60 frames per second. There’s no on/off switch: The camera is continuously recording. When storage space is filled, the system begins recording over the stored images, beginning with the oldest. It continues to record while the user reviews stored video.
FE: What type of camera is used?
Leonard: Any camera that outputs a composite video signal can be used. Some companies use a PTZ (pan, tilt, zoom) camera, which can be manipulated by the PC’s mouse. A variety of lenses, even microscopes, can be used. There’s a lot of interest in using lipstick cameras, which are extremely small and can be placed in tight areas. We’ve designed a custom-made, high-speed camera that has a 1/128,000th second shutter speed and can shoot 3,000 frames a second. Additional lighting is needed at high speeds, but resolution at typical shutter speeds is adequate with available light. Maintenance technicians often start at 120 frames a second. If they can’t capture the event at that speed, they’ll go to higher speeds and add lighting as necessary.
Motion pictures are shot at 16 frames a second because that’s all the human eye can process. The advantage of high speed is to enable the user to review what actually occurred in an event.
FE: Couldn’t an electrical engineer put his own monitoring system together?
Leonard: Users have to justify system cost, and they ask us what’s the difference between buying a camera for $2,000 at Circuit City and installing it with a VCR. But this is a completely digital industrial appliance that lets you pick any millisecond during the day and view what happened. And the software is more than half the magic. You don’t even have to be in the facility to conduct a root cause analysis: You can take corrective action based on a remote video feed.
FE: What system improvements have been made over the years?
Leonard: The first generation could only shoot 60 frames a second, which got us a lot of attention but wasn’t enough for some users. Cigarette machines output 14,000 cigarettes a minute, for example.
Higher frame speeds were the direct result of feedback from users. There are three or four software engineers constantly adding features to make the system easier to use or removing features that no one is interested in. Based on the feedback, we’ll be increasing frame rates and boosting storage capacity again next year. Some users hate the 25-ft. camera cable and are pleading for a wireless connection.
FE: How do users determine if they’re viewing a symptom or a cause of disruption?
Leonard: There can be some trial and error in setting up the camera. One plant was having problems with cartons flipping over as they were feeding into a case packer. Until the camera angle was altered and a frame-by-frame analysis was done, no one realized a broken link in a conveyor 20 ft. away was causing the problem.
FE: Are cameras being deployed primarily in packaging, or are process problems also being addressed?
Leonard: A yogurt manufacturer used a camera to analyze a filler during start-up. They found one machine required 30 to 40 more strokes to attain acceptable viscosity. By correcting the problem, the plant saved $10,000 a week in lost product.
FE: Some packaging equipment manufacturers have developed video monitoring systems. How do they compare?
Leonard: Typically those systems are PLC-driven, with a controller signaling the camera to record when a fault is detected. The disadvantage is that the equipment has to be hardwired to the controls, and if you want to troubleshoot multiple areas, you need multiple cameras. Our system has an interface for a PLC, but that would defeat the purpose of a mobile system that can be wheeled anywhere in the plant.
One maintenance supervisor did a side-by-side comparison of a packaging firm’s video monitor and ours. A dozen operators and mechanics evaluated them. All voted for this system.
FE: Didn’t Kodak offer a motion analysis system years ago?
Leonard: I ran across one of those the other day. It worked great for ballistics tests and other situations where you knew when the incident was going to occur. But line disruptions tend to occur intermittently, and having a camera that’s always on is a big advantage. The old Kodak system was very expensive and only gave the operator four seconds of recording. It was difficult to set up, too, so people avoided using it.
Russell “Rusty” Leonard, 20/20 Hindsight product manager, Monitoring Technology Corp., Fairfax, VA.
Cracks in pump shafts and turbine blades can have catastrophic consequences for a jet engine or a power plant, and a number of suppliers have devised systems that analyze signal data and detect faults in rotating parts. Among them is Monitoring Technology Corp., Fairfax, VA. Sophisticated software to process digital machine signals lies at the heart of the company’s condition-monitoring system. Cameras also are a component in many set-ups. Six years ago, the company packaged camera technology with custom software and introduced 20/20 Hindsight, a troubleshooting tool for process industries. Second-generation systems now are being installed at food and beverage facilities where high-speed lines can mesmerize technicians who try to eyeball process upsets.
Helping to train plant maintenance specialists on the system is Product Manager Russell D. “Rusty” Leonard. Leonard joined the firm in 1999 after working with Hughes Network Systems, the US Navy, British Aerospace, NASA and the European Space Agency, primarily as a consultant on satellite systems. He earned a BS in computer science with a minor in mathematics from James Madison University before attending the University of South Carolina, where he was awarded an MBA.
A maintenance engineer reviews slow-motion video of a beer-can sealer as a fault occurs, as shown in the left foreground. Source: Monitoring Technology Inc.
Leonard: Video is used with both systems. Some of the monitoring systems we’ve installed in paper and pulp plants have a big (computer) rack and up to 20 cameras wired into it. They can detect a break in a sheet at any point and tell the operators what to do to correct the problem. With 20/20, we don’t tell operators and maintenance personnel to do anything. We simply provide them with an ultra slow-motion video so they can see what happened and determine the corrective action. When people watch a line with the naked eye, the mind condenses the scene to see what it expects to see and misses the event.
In a food plant, the camera usually is enclosed in a dust- or water-proof enclosure. It’s attached to a stainless steel cart by a 25 ft.-long cable. The cart houses a video monitor, keyboard and PC with a 1 terabyte (1,000 gigabytes) hard drive. Originally the system could only buffer 10 to 60 seconds of video; now there’s enough memory to store over 50 hours of color video at 60 frames per second. There’s no on/off switch: The camera is continuously recording. When storage space is filled, the system begins recording over the stored images, beginning with the oldest. It continues to record while the user reviews stored video.
FE: What type of camera is used?
Leonard: Any camera that outputs a composite video signal can be used. Some companies use a PTZ (pan, tilt, zoom) camera, which can be manipulated by the PC’s mouse. A variety of lenses, even microscopes, can be used. There’s a lot of interest in using lipstick cameras, which are extremely small and can be placed in tight areas. We’ve designed a custom-made, high-speed camera that has a 1/128,000th second shutter speed and can shoot 3,000 frames a second. Additional lighting is needed at high speeds, but resolution at typical shutter speeds is adequate with available light. Maintenance technicians often start at 120 frames a second. If they can’t capture the event at that speed, they’ll go to higher speeds and add lighting as necessary.
Motion pictures are shot at 16 frames a second because that’s all the human eye can process. The advantage of high speed is to enable the user to review what actually occurred in an event.
FE: Couldn’t an electrical engineer put his own monitoring system together?
Leonard: Users have to justify system cost, and they ask us what’s the difference between buying a camera for $2,000 at Circuit City and installing it with a VCR. But this is a completely digital industrial appliance that lets you pick any millisecond during the day and view what happened. And the software is more than half the magic. You don’t even have to be in the facility to conduct a root cause analysis: You can take corrective action based on a remote video feed.
FE: What system improvements have been made over the years?
Leonard: The first generation could only shoot 60 frames a second, which got us a lot of attention but wasn’t enough for some users. Cigarette machines output 14,000 cigarettes a minute, for example.
Higher frame speeds were the direct result of feedback from users. There are three or four software engineers constantly adding features to make the system easier to use or removing features that no one is interested in. Based on the feedback, we’ll be increasing frame rates and boosting storage capacity again next year. Some users hate the 25-ft. camera cable and are pleading for a wireless connection.
FE: How do users determine if they’re viewing a symptom or a cause of disruption?
Leonard: There can be some trial and error in setting up the camera. One plant was having problems with cartons flipping over as they were feeding into a case packer. Until the camera angle was altered and a frame-by-frame analysis was done, no one realized a broken link in a conveyor 20 ft. away was causing the problem.
FE: Are cameras being deployed primarily in packaging, or are process problems also being addressed?
Leonard: A yogurt manufacturer used a camera to analyze a filler during start-up. They found one machine required 30 to 40 more strokes to attain acceptable viscosity. By correcting the problem, the plant saved $10,000 a week in lost product.
FE: Some packaging equipment manufacturers have developed video monitoring systems. How do they compare?
Leonard: Typically those systems are PLC-driven, with a controller signaling the camera to record when a fault is detected. The disadvantage is that the equipment has to be hardwired to the controls, and if you want to troubleshoot multiple areas, you need multiple cameras. Our system has an interface for a PLC, but that would defeat the purpose of a mobile system that can be wheeled anywhere in the plant.
One maintenance supervisor did a side-by-side comparison of a packaging firm’s video monitor and ours. A dozen operators and mechanics evaluated them. All voted for this system.
FE: Didn’t Kodak offer a motion analysis system years ago?
Leonard: I ran across one of those the other day. It worked great for ballistics tests and other situations where you knew when the incident was going to occur. But line disruptions tend to occur intermittently, and having a camera that’s always on is a big advantage. The old Kodak system was very expensive and only gave the operator four seconds of recording. It was difficult to set up, too, so people avoided using it.
Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!
