Manufacturing Business Technology
Onsite-Offline Testing Helps Link Production Lines With Emerging Technologies
March 2017
The pharmaceutical industry offers an informative look at the challenges — and opportunities — that will be faced by almost all manufacturers as they integrate their packaging lines with corporate information technology systems.
Manufacturers worldwide are weighing the benefits and risks of integrating production lines with corporate data systems to apply new technologies such as artificial intelligence, augmented and virtual reality. Their objective: to improve quality and embrace innovation.
In many, if not most instances, that exposes production lines to the larger universe of corporate IT problems for the first time. Once isolated from the routine hiccups that can occur in corporate information systems, production lines are now vulnerable to issues that can interrupt operations enough to affect a line’s throughput. For example, a database error generating serial numbers.
The risk of costly interruption is especially high when companies are implementing new technologies onto production lines. Experienced engineers can anticipate and remedy many production line issues. Most, however, have never been exposed to a database glitch or an application error on the corporate network that can shut down a production line.
An implementation model that’s gaining favor in pharmaceutical serialization projects can help manufacturers avoid the pitfalls that go with broader integration. Based on established but often-overlooked best practices, it is built on creating onsite, but offline, testing environments that enable engineers to validate new solutions without risking extended production downtime. Typically, after the Factory Acceptance Test, equipment is shipped to the site, the line is taken out of production and installation, testing, and validation take place.
Onsite but offline
“Onsite but offline testing” gives engineers and quality assurance personnel greater flexibility to address problems with new production line technology implementations. Properly planned and included from the beginning of the project, offline testing can eliminate weeks of potential downtime from IT problems that reach the plant floor.
The foundation of onsite but offline testing is setting up the new solution on a model line, ideally behind the same firewall as the target production line. The model test line would consist of equipment identical to what’s existing on the floor — such as spares or older versions that are kept in reserve — or temporarily idle production lines. Manufacturers without spares or idle lines should consider the purchase of used equipment, as the savings from shorter production line downtime can often justify the cost.
This “simulated” production line enables startup and validation teams to do what they do best. In Pharma Manufacturing, this is a proscribed process with phases of Site Acceptance Testing (SAT), Installation Qualification (IQ) and parts of Operational Qualification (OQ) amenable to this process all while the targeted production continues uninterrupted.
This testing process also helps to overcome a major shortcoming of vendor-provided Factory Acceptance Tests (FATs). Those tests are designed to validate the performance of individual components, but not their performance in an integrated environment.
Perhaps the most important benefit of onsite offline testing is that it gives start-up and validation teams a risk-free environment for testing and ensuring connections with corporate IT systems.
Testing failure, rejection and recovery modes — as well as loading software to ensure connectivity with serial number management systems — helps minimize problems that could keep a production line down for days, if not weeks.
The Pharma Example
The Pharmaceutical serialization mandate illustrates how this onsite-offline approach works in practice. The Drug Supply Chain Security Act (DSCSA) says that manufacturers must begin serializing all prescription drug products for the U.S. market at the saleable unit (bottle, carton, tray) and homogenized case level starting in November 2017.
To the uninitiated, serialization appears to be the straightforward process of printing an identifying number on products and shipping cases — similar to the lot and date codes that are currently applied to each product. Yet, it requires adding printers and cameras to the production lines, testing failure, rejection and recovery modes, and loading software. That’s routine for an experienced engineer. The real pitfalls can occur with the necessary integration between the production line technology and the corporate databases that generate and store the product serial numbers.
Exposure to the corporate network introduces potential problems. A database error far from the production line, for example, can slow or stop the production line if it interrupts generating serial numbers. The same could occur from a faulty connection between the production line and serial number management systems.
There are also several standard operating procedures that require modifications to maintain the integrity of the serialization scheme. For instance, when performing quality checks on the target line, QA can no longer simply remove an item from anywhere on the line for testing. The onsite-offline testing period is the ideal time to perform training on these additional/modified SOPs, as well as train on the equipment functionality itself.
While this example is specific to the pharmaceutical industry, the implications are clear for other industries looking to link production with corporate information systems. Onsite-offline gives validation teams a realistic environment for troubleshooting without the added pressure of worrying about extended downtime caused by failures during testing. Companies need that time cushion so they can adapt to new technologies and market needs without sacrificing productivity and profitability.
John Jordon, P.E., is Vice President of Business Strategies and Vito Pirrera is Executive Vice President of Operations at Vantage Consulting Group.