Tips for New Process Automation Folks

IEEE Tech Talk – control system device cyber security is missing in government and engineering societies

2021-12-16T05:35:00Z

IEEE (www.IEEE.org) is one of the most influential professional engineering societies in the world. It is well-regarded and has played an important role in the development of many control system standards.

On December 14, 2021, I was honored to give an IEEE Tech Talk to the Seattle Chapter of the IEEE Power and Energy Society on control system cyber security. The link to the recording is at https://www.youtube.com/watch?v=eZqkCC6wqcE

The theme of the presentation was:

- Focus of cyber security is IT malware and ransomware

Not addressing physical damage and injuries

- All physical infrastructures (“critical” or not) are monitored and controlled using instrumentation and control systems

- For physical infrastructure cyber security,

Measurements are the input, but instrumentation is ignored
Control systems are used to control physics, but physics is ignored
Cyber forensics and attribution do not exist for control system devices
Cyber security training generally not available for control system engineers
Control system cyber security is about 5-10 years behind IT cyber security
Culture is broken between Engineering and IT/OT

In the QA session following my presentation, it was mentioned that IEEE-USA issued a position paper on cyber security and control systems were not addressed. According to the representative from IEEE-USA, this a major hole that needs to be addressed (around the 1:15:00 time frame on the tape). Addressing this gap will require collaboration between the International Society of Automation (ISA), IEEE, and other industry organizations. It will also require CISA (including TSA), DOE, FERC, NRC, and other government organizations to address the control systems gap. The continuing singular focus on networks is making the US very vulnerable to extended outages, equipment damage, and deaths.

Gaps in Standards

Even the most sophisticated engineering associations can suffer from blind spots when it comes to the complex interactions of multi-level systems. This is especially true when situations like cyber security bring together organizations with different goals, nomenclature, etc. In this case, networking and engineering attempting to secure engineering networks have not addressed the Level 0,1 engineering devices (process sensors, actuators, drives, etc.). Examples where gaps occur include:

- Electric – NERC CIP excludes Level 0,1 devices

- Water/wastewater – AWWA doesn’t address Level 0,1 devices

- IEC TC57 – Doesn’t address Level 0,1 devices

- Food – FSMA doesn’t address control system cyber security

- TSA Pipelines – Doesn’t address control system pipeline issues

- TSA Transportation – Doesn’t address control system issues

- ISA 62443 – Doesn’t address process sensor integrity

- NIST 800-53, 800-82, 160 doesn’t address Level 0,1

- NIST Cyber Security Framework – Can’t “detect”

- IEC TC65 - Functional safety communication protocols do not address cyber security

Why Level 0,1 matters

It is not possible to cyber secure, or assure safety, of the physical infrastructures when the Level 0,1 devices have no cyber security, authentication, or cyber logging. Yet, cyber security of Level 0,1 devices continues to be ignored by the IT and OT networking communities. The following examples from multiple sectors illustrate why we are at such high risk from insecure process sensors:

- The Oak Ridge National Laboratory (ORNL), Pacific Northwest National Laboratory PNNL), and National Renewable Energy Laboratory (NREL) issued a report on sensor issues in Buildings. A typical situation could include sensor data being modified by hackers and sent to the control loops, resulting in extreme control actions. To the best of the authors’ knowledge, no such study has examined this challenge.

- “We have several temperature, pressure and flow sensors on a new medical-device cleaning skid that we are developing. These instruments are connected to a PLC as 4-20 mA inputs, and there is also a 4-20 mA output used to control a pump motor speed. A recent failure of a flow sensor brought the process skid instrumentation to my company’s quality manager’s attention. He asked how do we know that the temperatures, pressures, and flow are accurate, and how do we know that we are cleaning properly?” FDA has not addressed these issues.

- Process sensors not reliable or safe in refinery operation. Almost 50% of the nuisance alarms were reranged sensors (that is, the sensor settings have been changed so they have lost their ability to initiate safety systems when needed). Because this was from insiders, it was assumed the reranged sensors were unintentional.

- One sensor failure in combined cycle plant in Florida caused a 200MW load swing at the plant that rippled across the Eastern Interconnect causing a 50MW load swing in New England.

- Russia, China, and Iran are aware of the gap in cyber security of process sensors, and, in some cases, are already exploiting this gap.

Moreover, monitoring the electrical characteristics of the process sensors (sensor health monitoring) provides benefits beyond cyber security. As the process sensors are the “eyes of the process”, sensor health monitoring provides a predictive maintenance capability, improved performance and productivity, and improved safety. Additionally, sensor monitoring becomes a check of the network monitoring systems. If the sensor health monitoring does not directly match the network monitoring, the network monitoring needs to be examined.

The presentation included actual control system cyber incidents including pipeline ruptures that are not addressed by the TSA pipeline cyber security directive and train crashes not covered by the TSA rail cyber security directive. What makes critical infrastructures different than retail and other IT-centric organizations are the control system devices. It is also what makes them dangerous. So, why is TSA ignoring the control system devices?

The lack of addressing control system issues can be seen by the government and industry’s response to the Log4j (Apache open-source software) vulnerability disclosed December 10, 2021. It is similar to the government and industry’s response to SolarWinds with the entire focus on the networks to the exclusion of control systems.

The 2004 ICS Cyber Security Conference in Idaho Falls was held in conjunction with the ribbon cutting for the INL SCADA Testbed. As part of the Conference, INL did a cyberattack demonstration that exploited a zero-day (it wasn’t called zero-day in 2004) buffer overflow in Apache open-source software. The attack sent exploited code from the Sandia National Laboratory (SNL) business network to the INL business network to the INL SCADA Testbed network. The firewalls did not block the compromised scripts. The attack demonstration

- Remotely opened and closed a relay (Aurora)

- Remotely opened and closed multiple relays (2015 Ukrainian cyberattack)

- Remotely opened a relay but without indication the relay was open (2003 NE outage)

- Relay not changed, but the status indication was changed (Stuxnet)

The Apache Log4j vulnerability could potentially cause the same issues, yet control systems are being ignored in the government and industry guidance currently issued. Off-line monitoring of the sensors would not be affected by ransomware or the Log4j types of vulnerabilities.

Recommendations

Each Society has to put together a section call out the other just like what was done for the NEC (National Electric Code – Power) The NEC added a whole section 800 on communications a few years back. IEEE has to at least recognize IT Networking in their IEEE SCADA/Control Systems and ISA has to recognize the existing of power/Physics IEEE. The same can be said for ASME, AICHE, ASCE, SAE, INCOSE, and other industry Societies. The IT networking societies should not be making recommendations for securing control systems without assuring that recommendations for IT will not cause harm to control systems as has happened in the past.

As an aside, Nadine Miller and Rob Stephens from JDS Energy and Mining and myself will have a paper in the January issue of IEEE Computer magazine titled: “Control System Cyber Incidents Are Real—and Current Prevention and Mitigation Strategies Are Not Working”.

Joe Weiss

June 8th and 9th virtual keynotes to cyber security conferences – gaps between networking and engineering

2021-06-03T17:54:00Z

Given the virtual world we live in, I am able to support two important cyber security conferences – June 8th is the Cyber Observatory IOT and ICS Conference and June 8th and 9th is the New York State Cyber Security Conference. As control system-unique cyber issues are still misunderstood by many in the mainstream cyber security community, my presentations will be an engineer’s view of control system cyber security with a focus on actual impacts.

There have been almost 12 million control system cyber incidents. Yet there has been an alarming reticence for government and industry to identify control system cyber incidents as being “cyber”. Examples include the 2003 Davis Besse Slammer worm incident where NRC wouldn’t use the word “cyber” or the more than 350 control system cyber incidents in the North American electric system that NERC wouldn’t identify as “cyber”. Even the most recent NERC Lessons Learned refuses to call a power plant control system incident that affected the entire Eastern Interconnect a cyber incident. This event started with 200MW swings because a sensor and control system problem at a power plant in Florida and ended up with 50 MW swings in New England! (https://www.controlglobal.com/blogs/unfettered/process-sensor-issues-continue-to-be-ignored-and-are-placing-the-country-at-extreme-risk). Consequently, it should be evident that government initiatives that require identification of control system cyber incidents aren’t being met. This should be of concern given the increasing cyber oversight by insurance and credit rating agencies.

To date, the government guidance provided following control system cyber incidents has been generic such as don’t connect IT and OT networks or do good cyber hygiene but does not address the root cause of the incidents. The lack of providing guidance for the root cause has two ramifications: a false sense of security by only doing the basics and not addressing the root cause leaves the facilities open for the incidents to recur. Moreover, most of the root causes were not unique to just one facility.

Control system devices such as process sensors, actuators, and drives have no cyber security, authentication, or cyber logging and so it takes more than just network security to address them. Additionally, these devices are not capable of meeting the requirements in the Cybersecurity Executive Order (EO) 14028 or the TSA pipeline cyber security requirements. Understanding control system cyber security is critical as Russia, China, and Iran are aware of these deficiencies and some of these gaps are currently being exploited.

June 8th, I will be giving a keynote at the Cyber Observatory IOT and ICS conference (https://www.cyberinnovationsummits.com/industrial-cybersecurity-iiot-event/). I also will be participating in an executive roundtable – “The critical infrastructure supply chain: how can this massive operational and cyber security challenge be addressed?” The Chinese hardware backdoors in large electric transformers bring up hardware challenges that do not appear to be addressed in the ongoing supply chain initiatives.

June 8th, I will also be participating in a panel session at the New York State (NYS) Cyber Security Conference at 11AM Eastern with Matt Nielsen of GE R&RD and Sanjay Goel from SUNY Albany. The panel will address: Threats to the Energy Infrastructure of the United States”. The panel will discuss some of the recent cyberattacks on our power grid and what if should we be doing to mitigate the threat to our power infrastructure.

June 9th I will be giving a keynote at the 2021 NYS Cyber Security Conference (2021 NYS Cyber Security Conference) is held in conjunction with the Annual Symposium on information Assurance (http://www.albany.edu/iasymposium). My presentation will provide an engineer’s complement to Kevin Mandia’s Tuesday June 9th keynote on the state of cyber security.

The presentations will address some of the most significant recent control system cyber security incidents: SolarWinds and its impact on control systems, the Chinese hardware backdoors in large electric transformers, Chinese hidden control system networks in a pharma facility, the Colonial Pipeline hack, the Oldsmar water hack, counterfeit process sensors, and building hacks. It will identify some of the gaps between real incidents and EO 14028 and the TSA pipeline requirements. The presentation will provide recommendations to improve control system cyber security.

Joe Weiss

Residing on Residence Time

2020-04-21T00:55:00Z

The post, Residing on Residence Time, first appeared on ControlGlobal.com's Control Talk blog.

The time spent residing on this column is time well spent if you want to become famous for improving process performance with the side benefit of becoming best buds with the process engineer. The implications are enormous in terms of process efficiency and capacity from the straightforward concept of how much time a fluid resides in process equipment.

The residence time is simply the equipment volume divided by the fluid volumetric flow rate. The fluid can be back mixed (swirling in opposite direction of flow) in the volume due to agitation, recirculation or boiling. A lot of back mixing makes nearly all of the residence time a process time constant. If there is hardly any back mixing, we have plug flow and nearly all of the residence time becomes deadtime (transportation delay).

Deadtime is always bad. The ultimate limit to the peak and integrate errors for a load disturbance is proportional to the deadtime and deadtime squared, respectively.

A particular process time constant can be good or bad. If the process time constant in question is the largest time constant in the loop, it slows down disturbances on the process input and enables a larger PID gain. The process variability can be dramatically reduced for a process time constant much larger than the total loop deadtime. The slower time to reach setpoint can be speeded up by the higher PID gain provided there is proportional action on error and not just on PV in the PID structure.

If the process time constant in question is smaller than another process time constant possibly due volumes in series or heat transfer lags, a portion of the smaller time constants become effectively deadtime in a first order approximation. Thus, heat transfer lags and volumes between the manipulated variable and controlled variable create detrimental time constants. A time constant due to transmitter damping or signal filtering will add effective deadtime and should be just large enough to keep fluctuations in the PID output due to noise from exceeding the valve or variable speed driver deadband and resolution, whichever is largest.

At low production rates, the residence time gets larger, which is helpful if the volume is back mixed, but the process gain increases dramatically for temperature and composition control. If the volume is plug flow, we are in dire straits because the larger residence time creates a larger transportation delay resulting in a double whammy of high process gain and high deadtime causing oscillations as explained in the Control Talk Blog “Hidden factor in Our Most Important Loops”. For gas volumes (e.g., catalytic reactors), the residence time is usually very small (e.g., few seconds) and the effect is mitigated.

If you want more information on opportunities to learn what is really important, please join the ISA Mentor Program and ask the questions whose answers can be shared via Mentor Q&A Posts.

You can also get a comprehensive resource focused on what you really need to know for a successful automaton project including nearly a thousand best practices in the 98% new McGraw-Hill 2019 Process/Industrial Instruments and Controls Handbook Sixth Edition capturing the expertise of 50 leaders in industry.

What Factors Affect Dead Time Identification for a PID Loop?

2020-03-18T11:42:25Z

The post What Factors Affect Dead Time Identification for a PID Loop? first appeared on the ISA Interchange blog site.

The following technical discussion is part of an occasional series showcasing the ISA Mentor Program, authored by Greg McMillan, industry consultant, author of numerous process control books, 2010 ISA Life Achievement Award recipient and retired Senior Fellow from Solutia Inc. (now Eastman Chemical). Greg will be posting questions and responses from the ISA Mentor Program, with contributions from program participants.

In the ISA Mentor Program, I am providing guidance for extremely talented individuals from Argentina, Brazil, Malaysia, Mexico, Saudi Arabia, and the USA. This question comes from Adrian Taylor.

Adrian Taylor is an industrial control systems engineer at Phillips 66.

Adrian Taylor’s Question

One would expect dead time to be the easiest parameter to estimate, yet when using software tools that identify the process model in closed loop I find identification of dead time is inconsistent. Furthermore when using software identification tools on simulated processes where the exact dead time is actually known, I find on occasions the estimate of dead time is very inaccurate. What factors affect dead time identification for a PID loop?

Russ Rhinehart’s Answer

When we identify dead time associated with tuning a PID loop, it is normally part of a model such as First Order Plus Dead Time (FOPDT), or a slightly more complicated Second Order model (SOPDT). Normally, and traditionally, we generate the data by step-testing (start at steady state, SS, and make a step-and-hold in the manipulated variable, MV, (the controller output), then observe the process-controlled variable, CV, response. We pretend that there were no uncontrolled disturbances, and make the simple linear model best fit the data. This procedure has served us well for 80 years, or so, that we’ve used models for tuning PID feedback controllers or setting up feedforward devices; but there are many issues that would lead to inconsistent or unexpected results.

One of the issues is that these models do not exactly match the process behavior. The process may be of higher order than the model. Consider a simple flow rate response. If the i/p device driving the valve has a first-order response, and the valve has a first-order response, and there is a noise filter on the measurement, then the flow rate measurement has a third-order response to the controller output. The distributed nature of heat exchangers and thermowells, and the multiple trays in distillation all lead to high order responses.

So, the FOPDT model will not exactly represent the process, the optimization algorithm, the modeling approach, seeks the simple model that best fits overall the process response. In a zero dead time, high-order process, the best model will delay the modeled response so that the subsequent first-order part of the model can best fit the remaining data. The best model will report a dead time even if there is none. The model does not report the process dead time, but provides a pseudo-delay that makes the rest of the model best fit the process response. The model dead time is not the time point where one can first observe the CV change.

A second issue is that processes are usually nonlinear, and the linear FOPDT model cannot match the process. Accordingly, steps up or down from a nominal MV value, or testing at alternate operating conditions will experience different process gains and dynamics, which will lead to linear models of different pseudo-dead time values.

A third issue is that the best fit might be in a least squares sense over all the response data, or it might be on a two-point fit of mid-response data. The classic hand-calculated “reaction curve” models use the point of highest slope of the response to get the delay and time-constant by extrapolating the slope from that point to where it intersects the initial and final CV values. A “parametric” method might use the points when the CV rose one-quarter and three-quarters of the way from the initial to the final steady state values and estimate delay and time-constant from those two points. By contrast, a least squares approach would seek to make the model best fit all the response data not just a few points. The two-point methods will be more sensitive to noise or uncontrolled disturbances. My preference is to use regression to best fit the model over all the data to minimize the confounding aspects of process noise.

A fourth issue is that the step testing might not have started at steady-state, SS, nor ended at SS. If the process was initially changing because of its response to prior adjustments, then the step test CV response might initially be moving up or down. This will confound estimating the pseudo-delay and time-constant of any modeling approach. If the process does not settle to a SS, but is continuing to slowly rise, then the gain will be in error, and if gain is used in the estimation procedure for the pseudo-delay, it will also include that error. If replicate trials have a different background beginning, a different residual trend, then the models will be inconsistent.

A fifth issue relates to the assumption of no disturbances. If a disturbance is affecting the process then, similar the case of not starting at SS, the model will be affected by the disturbance, not just the MV.

Here is a sixth. Delay is nonlinear, and it is an integer. If the best value for the pseudo-delay was 8.7 seconds, but the sample interval was on a 1-sec interval, the delay would either be rounded or truncated. It might be reported as 8 or as 9 sec. This is a bit inconsistent. Further, even if the model is linear in differential equation terminology, the search for an optimum pseudo-delay is nonlinear. Most optimizers end up in a local minimum, which depends on the initialization values. In my explorations, the 8.7-sec ideal value might be reported within a 0- to 10-sec range on any one particular optimization trial. Optimizers need to be run from many initial values to find the global.

So, there are many reasons for the inconsistent and inaccurate results.

You might sense that I don’t particularly like the classic single step response approach. But I have to admit that it is fully functional. Even if a control action is only 70 percent right because the model was in error, the next controller correction will reduce the 30 percent error by 70 percent. And, after several control actions, the feedback aspect will get the controller on track.

Although fully functional, I think that the classic step-and-hold modeling approach can be improved. I used to recommend 4 MV steps – up-down-down-up. This keeps the CV in the vicinity of the nominal value, and the 4 steps temper the effect of noise, nonlinearity, disturbances, and a not-at-SS beginning. However, it takes time to complete 4 steps, production usually gets upset with the extended CV deviations, and it requires an operator monitoring to determine when to start each new test.

My preference now is to use a “skyline” MV sequence, which is patterned after the MV sequence used to develop models for model-predictive control, MPC, also termed advanced process control, APC. In the skyline testing, the MV makes steps to random values within a desired range, at random time intervals ranging from about ½ to 2 time-constants. In this way, in the same time interval for the 4-step up-down-down-up response, the skyline generates about 10 responses, can be automated, and does not push the process as far or for an extended period from the nominal value as traditional step testing. The large number of responses does a better job of tempering noise and disturbances, while requiring less attention and causing smaller process upsets.

Because the skyline input sequence does not create step-and-hold responses from one SS to another, the two-point methods for reaction curve modeling cannot be used. But regression certainly can be used. What is needed is an approach to nonlinear regression (to find the global minimum in the presence of local optima), and a nonlinear optimizer that can handle the integer aspects of the delay. I offer open-code software on my web site in Visual Basic for Applications, free to any visitor. Visit r3eda.com and use the menu item “Regression” then the sub-item “FOPDT Modeling.”

You can enter your data in the Excel spread sheet and press the run button to let the optimizer find the best model. The model includes both the reference values for the MV and CV (FOPDT models are deviations from a reference) and initial values (in the case the data does not start at an ideal SS). The optimizer is Leapfrogging, one of the newer multiplayer direct search algorithms that can cope with multi-optima, nonlinearity, and discontinuities. It seeks to minimize the sum of squared deviations, SSD, over all the data. The optimizer is reinitialized as many times as you wish to ensure that the global is found, and the software reports the cumulative distribution of SSD values to reveal confidence that the global best has been found.

ISA Mentor Program Posts & Webinars

Did you find this information of value? Want more? Click this link to view other ISA Mentor Program blog posts, technical discussions and educational webinars.

Adrian Taylor’s Reply to Russ Rhinehart

Many thanks for your very detailed response. I look for to having a play with your skyline test + regression method. I have previously set up a spreadsheet to carry out the various two point methods using points at 25 percent/75 percent, 35.3 percent/85.3 percent and 28.3 percent/63.2 percent. As your recursive method minimizes the errors and should always give the best fit, it will also be interesting to compare to the various two point methods to see which of these methods most closely match your recursive best fit method for various different type of process dynamics. For the example code given in your guidance notes: I presume r is a random number between 0 and 1? I note the open settling time is required. Is the procedure to still carry out an open loop step test initially to establish open loop setting time, and then in turn use this to generate the skyline test?

Russ Rhinehart’s Reply to Adrian Taylor

Yes, RND is a uniform distributed random number on the 0-1 interval. It is not necessary to have an exact number for the settling time. In a nonlinear process, it changes with operating conditions; and the choice of where the process settles is dependent on the user’s interpretation of a noisy or slowly changing signal. An intuitive estimate from past experience is fully adequate. If you have any problems with the software, let me know.

Michel Ruel’s Answer

See the ISA Mentor Program webinar Loop Tuning and Optimization for tips. Usually the dead time is easily identified in closed loop techniques but in open loop you can miss a chunk of it. Most modern tools analyze the process response in the frequency domain and in this case, dead time corresponds to high frequencies. Tests using a series of pulses (or double pulses) are rich in high frequencies and in this case dead time is well identified (if we use a first or second order + dead time, remember that dead time represents real dead time plus small time constants).

Adrian Taylor’s Reply to Michel Ruel

Many thanks for your response. While I have experienced the problem with various different identification tools, the Dead time estimate when using a relay test based identification tool seemed to be particularly inconsistent at identifying Dead time. My understanding now is that while the relay test method is very good at identifying ultimate gain/ultimate period, attempts to convert to an FOPDT model can be more problematic for this method.

Mark Darby’s Answer

Model identification results can be poor due to the quality of the test/data as well as the capabilities of the model identification technology/software. Insufficient step sizes can lead to poor results. For example, not making big enough test moves relative to valve limitations (dead band and stick/slip) and noise level of the measurements you want to model. Also, to get good results, multiple steps may needed to minimize the impact of unmeasured disturbances.

Another factor is the identification algorithm itself and capabilities of the software. Not all are equivalent and there is a wide range of approaches used, including how dead time is estimated. One needs to know if the identification approach works with closed-loop data. Not all do. Some include provisions for pre-filtering the data to minimize the impact of unmeasured disturbances by removing slow trends. This is known as high pass filtering, in contrast to low pass filtering which removes higher frequency disturbances.

If sufficient number of steps is done, most identification approaches will obtain good model estimates, including dead time. Dead time estimates can usually be improved by making higher frequency moves (e.g., fractions of the estimated state-state response time).

As indicated in my response to the question by Vilson, the user will often need to specify whether the process is integrating. Estimates of process model parameters can be used to check or constrain the identification. As mentioned, may be able to obtain model estimates from historical data – either by eye ball or using selected historical data in the model identification, and thereby avoid a process test.

Greg McMillan’s Answer

Digital devices including historians create a dead time that is one-half the scan time or execution rate plus latency. If the devices are executing in series and the test signal is introduced as a change in controller output, then you can simply add up the dead times. Often test setups do not have the same latency or same order of execution or process change injection point as in the actual field application. If the arrival time is different within a digital device execution, the dead time can vary by as much as the scan time or execution rate.

If there is compression, backlash or stiction, there is also a dead time equal to the dead band or resolution limit divided by the rate of change of the signal assuming the signal change is larger than dead band or resolution limit. If there is noise or disturbances, the dead time estimate could be smaller or larger depending upon the whether the induced change is in the same or opposite direction, respectively.

Some systems have a slow execution or large latency compared to the process dead time. Identification is particularly problematic for fast systems (e.g., flow, pressure) and any loop where the largest sources of dead time are in the automation system resulting in errors of several hundred percent. Electrode and thermowell lags can be incredibly large varying with velocity, direction of change, and fouling of sensor. Proven fast software directly connected to the signals designed to identify the open loop response (e.g., Entech Toolkit) and multiple tests with different size perturbations and direction and at different operating conditions (e.g., production rates, setpoints and degrees of fouling) is best.

I created a simple module in Mimic that offers a rough fast estimate of dead time and ramp rate and the integrating process gain for near-integrating and true integrating processes within 6 dead times that is accurate to about 20 percent if the process dead time is much larger than software execution rate. While the relay method is not able to identify the open loop gain and time constant, it can identify the dead time. I have done this in the Mimic “Rough n Ready” tuner I developed. Some auto tuning software may be too slow or take a conservative approach using the largest observed delay between a PV change and a MV change plus a maximum assumed update rate and possibly use a deranged algorithm thinking larger is better.

For Additional Reference:

McMillan, Gregory K., Good Tuning: A Pocket Guide.

Additional Mentor Program Resources

See the ISA book 101 Tips for a Successful Automation Career that grew out of this Mentor Program to gain concise and practical advice. See the InTech magazine feature article Enabling new automation engineers for candid comments from some of the original program participants. See the Control Talk column How to effectively get engineering knowledge with the ISA Mentor Program protégée Keneisha Williams on the challenges faced by young engineers today, and the column How to succeed at career and project migration with protégé Bill Thomas on how to make the most out of yourself and your project. Providing discussion and answers besides Greg McMillan and co-founder of the program Hunter Vegas (project engineering manager at Wunderlich-Malec) are resources Mark Darby (principal consultant at CMiD Solutions), Brian Hrankowsky (consultant engineer at a major pharmaceutical company), Michel Ruel (executive director, engineering practice at BBA Inc.), Leah Ruder (director of global project engineering at the Midwest Engineering Center of Emerson Automation Solutions), Nick Sands (ISA Fellow and Manufacturing Technology Fellow at DuPont), Bart Propst (process control leader for the Ascend Performance Materials Chocolate Bayou plant), Angela Valdes (automation manager of the Toronto office for SNC-Lavalin), and Daniel Warren (senior instrumentation/electrical specialist at D.M.W. Instrumentation Consulting Services, Ltd.).

About the Author
Gregory K. McMillan, CAP, is a retired Senior Fellow from Solutia/Monsanto where he worked in engineering technology on process control improvement. Greg was also an affiliate professor for Washington University in Saint Louis. Greg is an ISA Fellow and received the ISA Kermit Fischer Environmental Award for pH control in 1991, the Control magazine Engineer of the Year award for the process industry in 1994, was inducted into the Control magazine Process Automation Hall of Fame in 2001, was honored by InTech magazine in 2003 as one of the most influential innovators in automation, and received the ISA Life Achievement Award in 2010. Greg is the author of numerous books on process control, including Advances in Reactor Measurement and Control and Essentials of Modern Measurements and Final Elements in the Process Industry. Greg has been the monthly "Control Talk" columnist for Control magazine since 2002. Presently, Greg is a part time modeling and control consultant in Technology for Process Simulation for Emerson Automation Solutions specializing in the use of the virtual plant for exploring new opportunities. He spends most of his time writing, teaching and leading the ISA Mentor Program he founded in 2011.

Connect with Greg

Variability Appearance and Disappearance

2020-03-07T22:13:00Z

The post, Variability Appearance and Disappearance originally appeared on the ControlGlobal.com Control Talk blog.

Particularly confusing is variability that seems to come out of nowhere and then disappear. There are not presently good tools to track down the sources because it turns out most of them are self-inflicted involving automation system deficiencies, dynamics and transfer of variability. You need to understand the possible causes to be able to identify and correct the problem. Here we provide fundamental knowledge needed on the major sources of these particularly confusing sources of variability.

One of the most prevalent and confusing problems are oscillations that break out and disappear in cascade control loops and loops manipulating large valves and variable frequency drives (VFD). The key thing to look for is if these oscillations only start for large changes in controller output. If you have a slow secondary loop, valve or VFD, for small changes in controller output over a period of several controller executions for small setpoint changes or small disturbances, the secondary loop, valve or VFD can keep up with the requested changes. For large changes, oscillations break out. The amplitude and settling time increase as the degree of mismatch between requested rate of change and the rate of change capability of the secondary loop, valve or VFD. The best solution is of course to make the capabilities of what is being manipulated faster. You can make a secondary loop faster by decreasing the secondary loop’s dead time and lag times (faster sensors, filters, damping, and update rates) and making the secondary loop tuning faster. You can make the control valve faster by a higher gain and no integral action in positioner, putting a volume booster on the positioner output(s) with booster bypass valve slightly open to provide booster stability and increasing the size of air supply lines and if necessary, the actuator air connections. You can make VFDs faster by making sure there is no speed rate limiting in the drive setup, keeping fast speed control with VFD in the equipment room (not putting it into a much slower control system controller) and increasing the motor rating and size as needed. If the problem persists, turning on external-reset feedback with fast accurate readback of the process variable of the manipulated secondary loop, actual position of valve and speed of VFD can stop the oscillations.

Another confusing trigger for oscillations is a low production rate. The process gain and dead time both increase at low production rates causing oscillations as explained in the Control Talk Blog “Hidden factor in Our Most Important Loops”. Also, stiction is much greater as the valve operating point approaches the closed position due to higher friction from sealing and seating surfaces. Valve actuators may also be undersized for operating with the higher pressure drops near closure. Stiction oscillations size and persistence increase with valves designed to reduce leakage. Most valve suppliers do not want to do valve response testing below 20% output because it makes the valve dead band and resolution worse. The installed flow characteristic of linear trim distorts to quick opening for a valve drop to system pressure drop ratio at maximum flow is less than 0.25. The amplification of oscillations from backlash and stiction and the instability from the steep slope (high valve gain) from the quick opening installed characteristic cause the oscillations to be larger. Even more insidious is the not commonly recognized reality that a VFD has an installed flow characteristic that becomes quick opening if the static head to system pressure drop ratio is greater than 0.25 triggering the same sort of problems. Signal characterization can help linearize the loop but you still need adaptation of the controller tuning settings for the increase in the process gain from the hidden factor and the increase in dead time from transportation delays and you are still stuck with stiction. Besides the increase in the VFD gain multiplying effect on the open loop gain, there is an amplification of oscillations from the 0.35% resolution limit of traditional of VFD I/O card and the dead band introduced in VFD setup in a misguided attempt to reduce reaction to noise. Then there are oscillations from erratic signals and noise from measurement rangeability problems discussed in last month’s Control Talk Blog “Lowdown on Turndown”. Low production rates can also cause operation near the split range point and crisscrossing of the split range point causing consequential persistent oscillations from the severe nonlinearities and discontinuities besides greater stiction. Again external-reset feedback can help but the better solution is control strategies and configurations to eliminate the unnecessary crossings of the split range point as discussed in the Control Talk Column “Ways to improve split range control”.

If you want more information on opportunities to learn what is really important, please join the ISA Mentor Program and ask the questions whose answers can be shared via Mentor Q&A Posts.

You can also get a comprehensive resource focused on what you really need to know for a successful automaton project including nearly a thousand best practices in the 98% new McGraw-Hill 2019 Process/Industrial Instruments and Controls Handbook Sixth Edition capturing the expertise of 50 leaders in industry.

Alarm Management and DCS versus PLC/SCADA Systems

2020-02-19T12:42:56Z

The post Alarm Management and DCS versus PLC/SCADA Systems first appeared on the ISA Interchange blog site.

In the ISA Mentor Program, I am providing guidance for extremely talented individuals from countries such as Argentina, Brazil, Malaysia, Mexico, Saudi Arabia, and the USA. This question comes from Aaron Doxtator.

Aaron Doxtator is a process control EIT with XPS | Expert Process Solutions. He has experience providing process engineering and controls engineering services, primarily for clients in the mining and mineral processing sector.

Aaron Doxtator’s First Question

I am working on a project that I believe many other sites may wish to undertake, and I was looking for best practice information.

Using ISA 18.2, we are performing an alarm audit using the rationalization and documentation process for all plant alarms. This has been working well, but an examination of the plant’s “bad actors” has slowed down the process. These bad actors are a significant portion of the annunciated alarms, but many of them are considered redundant in certain scenarios and could only be mitigated with state-based alarming.

While the rationalization and documentation process is useful for examining many of the nuisance process alarms, it quickly becomes much more complicated when state-based alarming is to be considered.

The high-level process to implement these changes is as follows:

Identify the need for a state-based alarm;
Perform a risk review/MOC process with all stakeholders;
Implement the changes; and
Document the state-based alarms.

Is there a recommended best practice that one could follow in order to document the state-based alarm changes?

Nick Sands’ Answer

ISA has a series of technical reports that give guidance on implementation of the standard. TR4 is on Advanced and Enhance alarming, including state-based alarming.

There is guidance that includes:

Use redundant indications of states to minimize state-based logic failures.
Be cautious about suppression of alarms that can indicate the transfer of material or energy to an undesired location (example is high temperature alarms on columns, high levels on tanks…)

ISA Mentor Program Posts & Webinars

Did you find this information of value? Want more? Click this link to view other ISA Mentor Program blog posts, technical discussions and educational webinars.

Darwin Logerot’s Answer

In my experience, there are very few if any chemical or refinery units that would not benefit from state-based alarming (SBA). The basic problem is that most alarm systems are configured for only a single process condition (usually running at steady state), but real processes must operate through a variety of states: starting up, shutting down, product transitions, regeneration, partial bypass, etc. In these situations, alarm systems can and will produce multiple alarms that are meaningless to the operator (nuisance alarms). Alarm floods are the natural result. Alarm floods can be problematic in that they tend to distract the operator from the more important task at hand, can be misleading, and can hide important information.

Now, for a look at actually answering your question: ISA-18.2 TR4 as Nick cited is a good starting point for information on SBA. I will add some pointers and caveats as well:

SBA at its core is a relatively simple concept – determine the current operating state of the process or system, then apply appropriate alarm attribute modifications. But, as in many situations, “the devil is in the details”. So, best advice is to consult with a knowledgeable practitioner before embarking on a SBA project.
Apply SBA only to a well-rationalized alarm system, or in parallel with a thorough and principles-based rationalization.
Apply state transition techniques to prevent nuisance alarms when the process transitions into a running state.
Utilize commercially available software for SBA, rather than trying to develop custom logic and coding in the control system.
Another available resource is the Alarm Management chapter in the Instrument and Controls Handbook recently published.

One final observation, I note that you referred to a bad actors review as slowing down the process. My normal approach is to not concentrate on the bad actors, but to conduct a comprehensive rationalization (adding SBA in the process) that includes all tags in the control system. The bad actors will be covered in this process.

Greg McMillan’s Answer

I suggest you check out the Control Talk columns with Nick Sands Alarm management is more than just rationalization and Darwin Logerot The dynamic world of alarms.

Aaron Doxtator’s Second Question

While most of my experience has involved using PLC/DCS (or just DCS) for plant control, some clients have expressed interest in shifting away from using a DCS altogether and utilizing exclusively PLC/SCADA. Aside from client preference, are there recommendations for when one solution (or one combination of solutions) may be preferred over the other?

Hunter Vegas’ Answer

I wanted to address your question about DCS versus PLC/SCADA. Historically DCS and PLC systems were very different solutions. A DCS was generally very expensive, had slow processing/scan times, and was specifically designed to control large, continuous processes (IE refineries, petrochemical plants) with minimal downtime. PLCs boasted very high speed processing, were designed for digital IO and sequencing, and typically utilized for machine control and smaller processes. Over the years both DCS and PLC manufacturers have modified their products to expand into that “middle ground” between the two systems. DCSs were made more scalable (to make them competitive in small applications) and added extensive sequencing logic to make them better suited for digital control. At the same time PLCs added much more analog logic, the ability to program in function blocks and other languages, and began incorporating a graphical layer to make them look and feel more like a DCS.

While the two systems are undeniably much more similar then they were in the past, there are definitely some significant differences between the two technologies that makes one better suited than the other in a variety of situations. Generally we try to remain “vendor agnostic” in our answers so I won’t specifically name names but I will say that the offerings of the DCS and PLC vendors vary widely and some systems have much more capabilities than the other. That being said I’ll try to keep my answer fairly generic.

DCS systems were specifically designed to allow online changes because they were designed for plants that can run years without a shutdown. In such a plant the ability to make programming changes, add cards and racks of IO, and even upgrade software while continuing to run is paramount. PLCs generally have some ability to make online changes but there can be extensive limitations to what can be changed while running. Unfortunately many PLC vendors will say “there are virtually no limits to the changes you can make while running” – and you typically find out the hard way that this is not true even when running redundant processors. If you are looking at installing a control system on a process that must run continuously for long periods, spend a lot of time talking with users (not salespeople) to understand what you truly can (and cannot) do while running. Sometimes the solution can be as simple as creating dummy racks and IO while you are down so you can add racks later.

DCS systems typically have much slower processing speeds/scan times than PLCs. While some very recent DCS processors boast high speeds, most controllers can only process a limited number of modules at high speeds and even that speed (50ms or so) is slow compared to a PLC. If the process is extremely fast, a PLC will likely outperform a DCS.

DCS systems are usually much better at handling various networks and fieldbuses though PLCs have been improving in this regard and several third party manufacturers are now selling PLC compatible network cards. If you have existing bus systems (ASI, Foundation Fieldbus, Profibus PA, Devicenet, Bacnet, Modbus, etc) look at the system carefully and make sure it can communicate with your network. Fortunately the IOT buzz has driven both DCS and PLC manufacturers to communicate over an increasingly large array of networks so most systems are getting better in this regard.

Batch capabilities vary significantly by manufacturer so that capability is hard to define on a PLC vs DCS level. I can name DCS manufacturers who have great batch functionality and others that have minimal capability and are very difficult to program. Similarly some PLCs have good batch capabilities and others have virtually none. If you have batch and are looking at a new control system take the time to dig deep and talk extensively with people who program those systems regularly. The better systems offer extensive aliasing capabilities, have few limits in executing logic in phases, have a good batch operator interface, have an integrated tag database, and allow changes to phases/operations even as a recipe is running. Weaker systems have limited ability to alias (you must create a copy of every phase even if they are identical other than tag names), have limitations in what logic can run in the phases, have poor interfaces, and limit online changes.

Probably the last major point is cost and what are you trying to do with the data. Historically DCSs have had a much better capability to handle classical analog control, advanced control algorithms, and batch processing. Because of that they typically utilize a tag count based pricing model. This pricing strategy can become very expensive if the system is mostly being utilized to bring in reams of data for display and historization but not using it specifically for control. If the process has very large tag counts but doesn’t require extensive control capability, a PLC/SCADA system can be a cheaper alternative.

I hope this helps. If you have any questions about a specific vendor, ask me directly and I can share my experience.

Greg McMillan’s Answer

Most of the PID capability I find valuable in terms of advanced features most notably external-reset feedback and enhancements to deal with large wireless update time and analyzer cycle times are not available in PLCs. The preferred PID Standard (Ideal) Form is less common and multiple PID structures by setpoint weights for integral and derivative modes and the ability to bumplessly write to the gain setting may not exist as well. Some PLCs use the Parallel or Independent Form that negates conventional tuning practices. Even worse, computation of the PID modes in a few PLCs uses signals is in engineering units rather than percent of scale leading to bizarre tuning requirements.

A pneumatically actuated control valve in the loop is much slower than a DCS that can execute every 100 milliseconds. If the loop manipulates a variable frequency drive speed without deadband and rate limiting or speed to torque cascade control, the process deadtime is less than 100 milliseconds, and the sum of time constants from signal filtering and transmitter damping is less than 200 milliseconds, the DCS may not be fast enough but this is a lot of “Ifs” rarely seen in the process industry where fluids are flowing through a pipeline. It is a different story in parts and silicon wafer manufacturing.

For Additional Reference:

Bill R. Hollifield and Eddie Habibi, Alarm Management: A Comprehensive Guide

Nicholas Sands, P.E., CAP and Ian Verhappen, P.Eng., CAP., A Guide to the Automation Body of Knowledge. To read a brief Q&A with the authors, plus download a free 116-page excerpt from the book, click this link.

Additional Mentor Program Resources

Connect with Greg

Lowdown on Turndown

2020-02-03T21:36:00Z

The post, Lowdown on Turndown, appeared first on the ControlGlobal.com Control Talk blog.

There are a lot of misconceptions on what is the turndown capability of measurements and final control elements (e.g., control valves and variable frequency drives). Here is a very frank concise discussion of what really determines turndown and things to watch for in terms of limiting factors. For flow measurements and final control elements, the term rangeability is often used.

The turndown of vortex meters and magmeters is determined by a minimum velocity. The actual turndown experienced is typically a lot less than stated in publications because the maximum velocity for the meter size is usually greater than the maximum velocity for the process. Much larger than needed meters are often chosen because of conservative factors built into the stated requirements by process and piping engineers and the desire to minimize pressure drops across the meters. Less than optimum straight runs of upstream and downstream piping can also reduce rangeability for vortex meters and particularly differential herd meters (flow being the square root of pressure drop) because of the introduction of flow sensor noise that becomes large relative to size of sensor signal at low flows. Also, physical properties of the fluid most notably fluid kinematic viscosity for vortex meters and fluid conductivity for magmeters can significantly reduce turndown capability.

The turndown for valves more commonly stated as rangeability is severely limited by backlash and stiction that is often a factor of two or more greater near the seat than at the mid stroke range where response testing is normally done. Also, valve actuator sizes should provide at least 150% of the maximum torque or thrust requirement to deal with less than ideal conditions and tightening of stem packing. Valve rangeability is also greatly limited by the installed flow characteristic particularly if the valve to system pressure drop ratio at max flow is less than 0.25 in a misguided attempt to reduce pressure drop and provide more flow capacity than what is actually needed.

The literature does not alert users to the fact that variable frequency drives can have a very nonlinear installed flow characteristic and poor turndown. To maximize rangeability of variable frequency drives, use a pulse width modulated inverter with slip control, speed to torque cascade control in the field (not control room), a pump head that is at least 4 times the maximum static head, totally enclosed fan cooled inverter rated motor, high resolution signal card, and minimal dead band setting in drive setup.

Transmitters and some sensors have an error that is expressed as a percent of span that reduces turndown. Transmitters selected that have a range narrowed to be closer to the actual maximum consequently improve turndown. The use of thermocouples (TCs) and resistance temperature detectors (RTDs) input cards instead of transmitters introduce a huge error and resolution limit and reduction in real rangeability due to the large spans. The use of TCs instead of RTDs severely reduces rangeability due to larger sensitivity errors and drift provided the temperature is in the recommended range for RTDs.

You can achieve greater rangeability by putting small and large flow meters and control valves in parallel. The process control loop manipulates the smaller valve using the smaller flow meter for cascade control for more precise control. A valve position controller (VPC) manipulates the large valve to keep the small valve in a good throttle range. External-reset feedback is used to reduce interactions and provide a fast correction if the small valve is moving toward the lower or upper best part of its installed flow characteristic. Feedforward or flow ratio control can be used to provide quicker correction. See the Control article “Don’t Over Look PID in APC” for much more on the many uses of VPC. Note that the use of split range control is not as good because you are normally manipulating the large valve and using the large flow meter with error and resolution limitations that are large due to being a percent of span.

Going for more flow capacity, lower pressure drop or a cheaper installation generally hurts turndown. Remember bigger is not better and cheaper is not really cheaper in the long run.

If you want more information on opportunities to learn what is really important, please join the ISA Mentor Program and ask the questions whose answers can be shared via Mentor Q&A Posts.

You can also get a comprehensive resource focused on what you really need to know for a successful automaton project including nearly a thousand best practices in the 98% new McGraw-Hill Process/Industrial Instruments and Controls Handbook Sixth Edition capturing the expertise of 50 leaders in industry.

How to Manage Control Valve Response Issues in the Field

2020-01-15T12:42:38Z

The post How to Manage Control Valve Response Issues in the Field first appeared on the ISA Interchange blog site.

Mohd Zhafran A. Hamid is a senior instrument engineer from Malaysia working in an EPC company, Toyo Engineering Corporation. He has worked in the field of control and instrumentation for about 10 years mostly in both engineering design and involvement at site/field.

Mohd Zhafran’s First Question

If you have selected a control valve whose installed flow characteristics significantly deviates from linear (either by mistake or forced to select due to certain circumstances), what is a practical way in the field after installation to linearize the installed flow characteristic?

Greg McMillan’s Answer

You need a sensitive flow measurement to identify the installed flow characteristic online. If you have a flow measurement and make changes in the manual controller output 5 times larger than dead band or resolution limit spaced out by a time interval greater than the response time, the slope of the installed flow characteristic is the change in per cent flow divided by the change in per cent signal. You need at least 20 points identified on the installed flow characteristic.

A signal characterizer is then inserted on the controller output to convert the flow in percent of scale to percent signal to get a piecewise linear fit that would linearize the characteristic so far as the controller is concerned. The controller output and linearized signal to the valve should be displayed. This linearization can be done in a positioner, but I prefer it being done in the DCS or PLC for better visibility and maintainability. For much more on signal characterizers see my Control Talk blog Unexpected benefits of signal characterizers.

ISA Mentor Program Posts & Webinars

Did you find this information of value? Want more? Click this link to view other ISA Mentor Program blog posts, technical discussions and educational webinars.

Mohd Zhafran’s Second Question

I recently read the addendum “Valve Response Truth or Consequences” in Greg’s article How to specify valves and positioners that do not compromise control. I am curious for fast loop whereby the control valve is used with volume booster but without positioner, how come you can move the stem/shaft by hand only even though the valve size is big. Would you mind sharing the overall schematic? Also, would you also share the schematic of using positioner with booster and booster bypass?

Greg McMillan’s Answer

Positive feedback from a very sensitive booster outlet port is greatly assisting attempts to move the shaft either manually or due to fluid forces on a butterfly disk as described in item 5 of my Control Talk blog Missed opportunities in process control – Part 6. There is a schematic of the proper installation in slide 18 of the ISA Mentor Program webinar How to Get the Most out of Control Valves. I don’t have a schematic of the wrong thing to do where the volume booster input is connected to current to pneumatic transducer (I/P) output.

For new high pressure diaphragm actuators or boosters with lower outlet port sensitivity, this may not happen since diaphragm flexure and consequential change in pressure from change in actuator volume may be less than booster outlet port sensitivity but it is not worth the risk in my book. The rule positioners should not be used on fast loops is mostly bogus as explained in my point 4 in the same Control Talk blog. If you need a response time faster than 0.5 seconds, you should use a variable frequency drive with a pulse width modulated inverter.

Mohd Zhafran’s Third Question

Greg highlighted the importance to specify valve gain requirement. Is there any publicly available modeling software that we design engineer can utilize to perform valve gain analysis? So far, I have encountered only one valve manufacturer that provides control valve sizing software (publicly available) with feature of valve gain graph. This manufacturer calculates process model based on the principle that the pressure losses in a piping system are approximately equal to flow squared.

Greg McMillan’s Answer

The Control Talk column Why and how to establish installed flow characteristic describes how one practitioner uses Excel to compute the installed flow characteristic. The analysis of all the friction losses in a piping system can be quite complicated because of the effect of process fluid properties and fouling determined by process conditions and operating history and the piping system including fittings, elbows, inline equipment (e.g., heat exchangers and filters), and valves.

A dynamic model in a Digital Twin that includes system pressure drops and the effect of fouling and the ability to enter the inherent flow characteristic perhaps by a piecewise linear fit can show how the valve gain changes for more complex and realistic scenarios. Ideally, there would be flow and pressure measurements to show key pressure drops particularly where fouling is a concern so that resistance coefficients can be back calculated.

The fouling of heat transfer surfaces can be detected by an increase in the difference needed between the process and utility temperature to compensate for the decrease in heat transfer coefficient. A slow ramp of the valve signal followed by a slow ramp in a flow measurement could reveal the installed flow characteristic by a plot of flow ramp versus the signal ramp assuming there are no pressure disturbances and flow measurement has sufficient signal to noise ratio and rangeability.

Additional Mentor Program Resources

Connect with Greg

How to Use Industrial Simulation to Increase Learning and Innovation

2019-12-18T14:00:01Z

The post How to Use Industrial Simulation to Increase Learning and Innovation first appeared on the ISA Interchange blog site.

Damien Hurley is a control and instrumentation (C&I) engineer for Fluor in the UK. He is currently involved in the detailed design phase of a project to build a new energy plant in an existing refinery in Scotland. His chief responsibility is C&I interface coordinator with construction, the existing site C&I contractor and the client.

Damien Hurley’s Question

How can I begin implementing process simulations in my learning? My background is in drone control where all learning has a significant emphasis on simulation and testing, usually via programs such as MATLAB. Upon starting in the oil and gas engineering, procurement and construction (EPC) industry, I began getting to grips with the wide array of final elements and my knowledge of Process simulation has suffered as a result.

I’m also not exposed to simulations on a daily basis, as I was previously in the unmanned aerial vehicle (UAV) industry. How can I get started with simulation again? Specifically is the simulation of processes relevant to our industry? Can you point me in the direction of a good resource to begin getting to grips with this worthwhile subject?

Greg McMillan’s Answer

Dynamic simulation is the key to most of deep learning and significant innovation in my 50-year career. Simulation has played a big role in industrial processes, especially in refining and energy plants. There are a lot of basic and advanced modeling objects for the unit operations in these plants. You can learn a lot about what process inputs and parameters are important in the building of first principle models. Even if the simulations are built for you, the practice of changing process inputs and seeing the effect on process outputs is a great learning experience. You are free to experiment and see results where your desire to learn is the main limit.

You can also learn a lot about what affects process control. Here it is critical to include all of the automation system dynamics often ignored in the literature despite most often being the biggest source of control loop dead time with also a significant contributing effect to the open loop gain and nonlinearity by way of the installed flow characteristic of control valves and variable frequency drives (VFDs).

You need to add variable filter times to simulate sensors particularly thermowell and electrode lags, transmitter damping, and signal filters. You need to add variable dead time blocks to simulate transportation delays associated with injection of manipulated fluids into the unit operation and to the sensor for measurement of the controlled variables. The variable deadtime block is also needed for simulating the effect of positioners with poor sensitivity where the response time increases by two orders of magnitude for changes in signal less than 0.25 percent. You need backlash-stiction blocks to simulate the deadband and resolution limits of control valves as detailed in the Control article How to specify control valves and positioners that don’t compromise control.

VFDs can have a surprisingly large deadband introduced in the setup in a misguided attempt to reduce reaction to noise and a resolution limit caused by an 8-bit signal input card. You also need to add rate of change limits to model slewing rates for large control valves and introduced in the VFD setup in a misguided attempt to reduce motor overload instead of properly sizing the motor. You need software that will provide PID tuning settings with proper identification of total loop dead time. Finally, a performance metrics block to identify the integrated and peak error for load disturbances and the rise time, overshoot, undershoot, and settling time for disturbances is a way of judging how well you are doing.

A couple of years ago I helped develop a dynamic simulation of the control system and the many headers, boilers, and users at a large plant to optimize the cogeneration and minimize the disruption to the steam system from large changes in the steam use and generation in all the headers for the whole plant. ISA Mentor Program resource James Beall and protégé Syed Misbahuddin were part of the team. Over 30 feedforward and decouple signals were developed and thoroughly tested by dynamic simulation resulting in a smooth implementation of much more efficient and safe system. I learned via the simulation in one case that the feedforward I thought was needed for a boiler caused more harm than good due to changes in header pressure preceding the supposedly proactive feedforward to a header letdown valve to compensate for the effect of a change in firing rate demand.

First principle process models material and energy balances of volumes in series can model the many unanticipated changes. I recently was alerted to the fact that the use of a bypass valve around a heat exchanger provides first a fast response from a change in flow bypassing and going through the exchanger but is followed by a delayed response in the opposite direction caused by the same utility flow rate heating or cooling a different flow rate through the exchanger. Unless a feedforward changes the utility flow, the tuning of the PID for temperature of the blended stream must not overreact to the initial temperature change.

Often there are leads besides lags in the temperature response associated with inline temperature control loops for jackets. For heat exchangers in a recirculation line for a volume, the self-regulating response of the exchanger outlet temperature controller is followed by a slow integrating response from recirculation of the changes in the volume temperature. Also, feedforward signals that arrive too soon can create an inverse response or that arrive too late create a second disturbance that makes control worse than the original feedback control. Getting the dynamics right by inclusion of automation besides process dynamic is critical.

ISA Mentor Program Posts & Webinars

Did you find this information of value? Want more? Click this link to view other ISA Mentor Program blog posts, technical discussions and educational webinars.

We learn the most by our mistakes. To avoid the price of making them in field, we can use dynamic simulation as a safe way of hands-on learning for exploration and prototyping of existing and new systems finding good and bad effects that offers much more flexibility and is non-intrusive to the process. Dynamic models using the digital twin enables a deeper process understanding to be gained and used to make much more intelligent automation. See the Control Talk blog Simulation breeds innovation for an insightful history and future of opportunities for a safe sandbox allowing creativity by synergy of process and automation system knowledge.

Often simulation fidelity is simply stated as low, medium or high. I prefer defining at least five levels as seen below in the chapter Tip #98: How to Achieve Process Simulation Fidelity in the ISA book 101 Tips for a Successful Automation Career. Note that the term “virtual plant” I have been using for decades should be replaced with the term “digital twin” in my books and articles prior to 2018 to be in tune with the terminology for digitalization and digital transformation.

Fidelity Level 1: measurements can match setpoints and respond in the proper direction to loop outputs; for operator training.
Fidelity Level 2: measurements can match setpoints and respond in the proper direction when control and block valves open and close and prime movers (e.g., pumps, fans, and compressors) start and stop; for operator training.
Fidelity Level 3: loop dynamics (e.g., process gain, time constant, and deadtime) are sufficiently accurate to tune loops, prototype process control improvements, and see process interactions; for basic process control demonstrations.
Fidelity Level 4: measurement dynamics (e.g., response to valves, prime movers, and disturbances) are sufficiently accurate to track down and analyze process variability and quantitatively assess control system capability and improvement opportunities; for rating control system capability, and conducting control system research and development.
Fidelity Level 5: process relationships and metrics (e.g., yield, raw material costs, energy costs, product quality, production rate, production revenue) and process optimums are sufficiently accurately modeled for the design and implementation of advanced control, such as model predictive control (MPC) and real time optimization (RTO), and in some cases virtual experimentation.

A lot of learning is possible by using Fidelity Levels 3 models. Fidelity Level 4 and 5 simulations with advanced modeling objects are generally needed for complex unit operations where components are being separated or formed, such as biological and chemical reactors and distillation columns, or to match the dynamic response of trajectories to detail advanced process control including PID control that involves feedforwards, decouplers, and state based control. Developing and testing inferential measurements, data analytics, performance metrics, and MPC and RTO applications, generally requires Level 5.

In all cases I recommend a digital twin that has the blocks addressing nearly every type of automation system dynamics and metrics often neglected in dynamic simulation packages. The digital twin should have the same PID Form, Structure and options used in the process industry and a tool like the Mimic Rough-n-Ready tuner to get started with reasonable PID tuning settings.

Many software packages that were not developed by automation professionals may unfortunately seriously mess you up by not having the many sources of dead time, lags, and nonlinearities, and by employing a PID with a Parallel (Independent) Form working in engineering units instead of percent signals. A fellow protégé also in the UK who is now an automation engineer at Phillips 66 can relate his experiences in using Mimic software. If you pursue this dynamic simulation opportunity, we can do articles and Control Talk blogs together to share the understanding gained to help advance our profession.

For Additional Reference:

McMillan, Gregory K., and Vegas, Hunter, 101 Tips for a Successful Automation Career.

Additional Mentor Program Resources

Connect with Greg

Biggest Valve Sizing Mistake

2019-12-05T01:24:00Z

The post, Biggest Valve Sizing Mistake, appeared first on ControlGlobal.com's Control Talk blog.

There is a common mistake made in the sizing of most control valves. The intentions that lead to this mistake may be good but the results are insidiously bad. While you would think that the proliferation of improvements in technology and communications would lead to better awareness, the problem appears to getting worse because of pervasive persistent misconceptions fostered by missing fields on valve specification forms.

Presently, valve specification forms have fields for maximum flow, available pressure drop and leakage. Most people filling out the form would think that a valve that can easily handle a greater flow with a lower pressure drop and less leakage would be better. This leads often to rotary valves with tight shutoff seals. These valves are cheaper than sliding stem valves and the actuators often included are designed for a rotary stem and can handle greater shutoff pressures. The resulting ball and butterfly valves have piston actuators designed more for on-off action. These valves are usually already in the piping specification extensively for automated sequential actions and shutdown. While rangeability may not be on the valve specification, it is thought to be extraordinarily great for these valves due to a prevalent definition of rangeability being a maximum flow divided by a minimum flow whose Cv is within the specified inherent flow characteristic. Gosh, you don’t even need to be concerned with piping reducers. What seals the deal is the very attractive price. Not understood is that these on-off valves posing as throttling valves are a disaster. A low valve pressure drop to system pressure drop ratio causes distortion of the installed flow characteristic making the characteristic much more nonlinear. The backlash of the actuator linkages and the keylock actuator shaft to stem to ball or disk connections is excessive, the stiction from the ball or disk seal and shaft packing is terrible, and the resolution of the piston actuator poor. The result is limit cycles and a real rangeability that is lousy to the point of being a disaster for any loop where control better than within 5% of setpoint is desired. Often the oscillations are blamed on other sources due to lack of understanding. The real rangeability is drastically reduced to perhaps 10% of the stated rangeability due to distortion of the nonlinear installed flow characteristic and the backlash and stiction that gets worse near the closed position. Operating valve positions are much less than expected due to conservative factors built into pump sizing and maximum flow specified. Most valve suppliers will not do response testing and if requested, the testing will not be done below 10% valve position because of the deterioration in response. The user is setup for a terrible scenario of limit cycling.

So what can we do? Please, add in a backlash and resolution (e.g., 0.5%) and response time (e.g., 2 sec) and installed flow characteristic valve gain (e.g., 0.5 to 2.0 % flow per % signal) requirement at 10%, 50% and 90% positions for step changes of 0.25%, 0.5%, 1%, and 2% for all valves plus 50% step change for surge valves and gas pressure valves. To achieve these specification requirements, use splined shaft connections, integrally cast stem and ball or disk, v-notch balls and contoured disks, and low friction seals for rotary valves and a valve pressure drop that is at least 25% of the system pressure drop for maximum flow, low friction packing (e.g., ultra low friction (ULF) packing), sensitive diaphragm actuators (now available for much higher actuator pressures) and digital positioners tuned with maximum gain and no integral action for all valves. The installed flow characteristic should be plotted with the help of process and piping engineers for the worst case. When asked why the valve cost is higher, tell them the cheaper valve will cause sloppy control putting the plant safety seriously at risk.

Remember bigger is not better and cheaper is not really cheaper in the long run.

For much more than you have ever wanted on valve response checkout the Control article “How to specify valves and positioners that don’t compromise control" and the associated white paper “Valve Response - Truth or Consequences”.

For much more on valve specification see the ISA Mentor Program Q&A post “Basic Guidelines for Control Valve Selection and Sizing”

If you want more information on opportunities to learn what is really important, please join the ISA Mentor Program and ask the questions whose answers can be shared via Mentor Q&A Posts.

You can also get a comprehensive resource focused on what you really need to know for a successful automaton project including nearly a thousand best practices in the 98% new 2019 Process/Industrial Instruments and Controls Handbook Sixth Edition capturing the expertise of 50 leaders in industry.

How to Avoid Using Multivariable Flow Transmitters

2019-11-18T14:00:56Z

The post How to Avoid Using Multivariable Flow Transmitters first appeared on the ISA Interchange blog site.

Jeff Downen is an I&C commissioning engineer with cross-training in DCS and high voltage electrical testing. His expertise is in start-up and commissioning of natural gas, combined cycle, power plants.

Jeff Downen’s Question

Our multivariable flow transmitters on new construction sites fail a lot. If the transmitter loses the RTD, the whole 4-20 loop goes bad quality along with the HART variables. I like the three devices being separate and their signals joined in the DCS logic much more. I understand that it is more expensive. I want to see if there was any other reasoning behind it on the engineering side and how I can help get a better up front design.

How can we avoid the increasing use of multivariable flow transmitters as an industry standard despite a significant loss in reliability, accuracy, and diagnostic and computational capability from not having individual separate pressure, temperature and flow sensors and transmitters?

Greg Brietzke’s Answer

I like Jeff’s question on multivariable flow transmitters, as it would be relevant to control engineers, maintenance/reliability engineers, as well as maintenance personnel. What is the application? What are the accuracy requirements? Can you bring the individual variables back to the DCS/PLC through additional variable assignment? Would the increased cost of infrastructure justify the increased expense of a true mass flowmeter? This could be addressed from so many different viewpoints it could be a great discussion topic.

ISA Mentor Program Posts & Webinars

Did you find this information of value? Want more? Click this link to view other ISA Mentor Program blog posts, technical discussions and educational webinars.

Greg McMillan’s Answer

I suggest you explain to plant and project personnel the advantages of separate measurements and true mass flowmeters. Separate flow, temperature and pressure measurements offer better diagnostics, reliability, sensors, and installation location that is particularly important for temperature (e.g., RTD in tapered thermowell with tip centered in pipe with good velocity profile). They can provide faster and perhaps more accurate and maintainable measurements that could be used for personalized performance monitoring calculations and safety instrumented systems.

Coriolis meters provide the only true mass flow measurements offering an incredibly accurate density measurement as well. Most people don’t realize that pressure and temperature compensation of volumetric flow meters to get a mass flow measurement only works if the concentration is constant and known. The Coriolis mass flow is not affected by component concentrations or physical properties in the same phase. Density can provide an inferential measurement of concentration for a two component process fluid. The Coriolis meter accuracy and rangeability is the best by far as noted in the Control Talk column Knowing the best is the best.

David De Sousa’s Answer

Using dedicated and separated measurements also allows for the use of hybrid virtual flowmeters in complex process applications where, for example, the technology for inline multiphase flow metering is not yet mature enough, or where physical units will greatly increase the cost of the associated facilities.

With the digital transformation initiatives associated with Industry 4.0, the use of distributed instrumentation, data-driven learning algorithms, and physical flow models, are being tested and explored more and more in the process industries, especially in upstream oil & gas wellsite applications.

Additional Mentor Program Resources

Connect with Greg

How to Avoid Multi-Variable Flow Transmitters

2019-11-18T14:00:56Z

The post How to Avoid Multi-Variable Flow Transmitters first appeared on the ISA Interchange blog site.

Jeff Downen’s Question

Our multi-variable flow transmitters on new construction sites fail a lot. If the transmitter loses the RTD, the whole 4-20 loop goes bad quality along with the HART variables. I like the 3 devices being separate and their signals joined in the DCS logic much more. I understand that it is more expensive. I want to see if there was any other reasoning behind it on the engineering side and how I can help get a better up front design.

How can we avoid the increasing use of multi-variable flow transmitters as an industry standard despite a significant loss in reliability, accuracy, and diagnostic and computational capability from not having individual separate pressure, temperature and flow sensors and transmitters?

Greg Brietzke’s Answer

I like Jeff’s question on multi-variable flow transmitters, as it would be relevant to control engineers, maintenance/reliability engineers, as well as maintenance personnel. What is the application? What are the accuracy requirements? Can you bring the individual variables back to the DCS/PLC through additional variable assignment? Would the increased cost of infrastructure justify the increased expense of a true mass flowmeter? This could be addressed from so many different viewpoints it could be a great discussion topic.

ISA Mentor Program Posts & Webinars

Did you find this information of value? Want more? Click this link to view other ISA Mentor Program blog posts, technical discussions and educational webinars.

Greg McMillan’s Answer

David De Sousa’s Answer

Using dedicated and separated measurements, also allows for the use of hybrid virtual flowmeters in complex process applications where, for example, the technology for inline multiphase flow metering is not yet mature enough, or where physical units will greatly increase the cost of the associated facilities.

Additional Mentor Program Resources

See the ISA book 101 Tips for a Successful Automation Career that grew out of this Mentor Program to gain concise and practical advice. See the InTech magazine feature article Enabling new automation engineers for candid comments from some of the original program participants. See the Control Talk column How to effectively get engineering knowledge with the ISA Mentor Program protégée Keneisha Williams on the challenges faced by young engineers today, and the column How to succeed at career and project migration with protégé Bill Thomas on how to make the most out of yourself and your project. Providing discussion and answers besides Greg McMillan and co-founder of the program Hunter Vegas (project engineering manager at Wunderlich-Malec) are resources Mark Darby (principal consultant at CMiD Solutions), Brian Hrankowsky (consultant engineer at a major pharmaceutical company), Michel Ruel (executive director, engineering practice at BBA Inc.), Leah Ruder (director of global project engineering at the Midwest Engineering Center of Emerson Automation Solutions), Nick Sands (ISA Fellow and Manufacturing Technology Fellow at DuPont), Bart Propst (process control leader for the Ascend Performance Materials Chocolate Bayou plant), Angela Valdes (automation manager of the Toronto office for SNC-Lavalin), and Daniel Warren (senior instrumentation/electrical specialist at D.M.W. Instrumentation Consulting Services, Ltd.).

Connect with Greg

Webinar Recording: Lessons Learned During the Migration to a New DCS

2019-10-02T13:00:14Z

The post Webinar Recording: Lessons Learned During the Migration to a New DCS first appeared on the ISA Interchange blog site.

This educational ISA webinar was presented by Greg McMillan. Greg is an industry consultant, author of numerous process control books, 2010 ISA Life Achievement Award recipient and retired Senior Fellow from Solutia Inc. (now Eastman Chemical).

This ISA webinar on control valves is introduced by Greg McMillan and presented by Hector Torres, in conjunction with the ISA Mentor Program. Hector is a recipient of ISA’s John McCarney Award for the article on opportunities and challenges for enabling new automation engineers. Hector has been a member of the ISA Mentor Program since its inception. In this webinar, he provides a detailed view of how to use key PID controller features that can greatly expand what you can achieve. The setting of anti-reset windup ARW limits, dynamic reset limit, eight different structures, integral dead band, and set-point filter. Feedforward and rate limiting are covered with some innovative application examples

Principal ISA Mentor Program mentee Hector Torres shares his extensive knowledge gained after migrating a plant from an 1980s vintage DCS to a state-of-the art, new DCS. The following important topics are covered: the proper setting of tuning parameters, controller output scales, anti-reset windup limits, and the many grounding, wiring, and configuration practices found to be essential in a migration project that exceeded expectations.

ISA Mentor Program Posts & Webinars

Did you find this information of value? Want more? Click this link to view other ISA Mentor Program blog posts, technical discussions and educational webinars.

Connect with Greg

Missed Opportunities in Process Control - Part 6

2019-08-01T01:01:00Z

The post, Missed Opportunities in Process Control - Part 6, first appeared on the ControlGlobal.com Control Talk blog.

Here is the Sixth part of a point blank decisive comprehensive list of what we really need to know in a detailed attempt to reduce the disparity between theory and practice. Please read, think and take to heart the opportunities to increase the performance and recognized value of our profession. The list is necessarily concise in detail. If you want more information on these opportunities, please join the ISA Mentor Program and ask the questions whose answers can be shared via Mentor Q&A Posts.

You can also get a comprehensive resource focused on what you really need to know for a successful automaton project including nearly a thousand best practices in the 98% new 2019 Process/Industrial Instruments and Controls Handbook Sixth Edition capturing the expertise of 50 leaders in industry.

Add small amounts of dissolved carbon dioxide (DCO2) and conjugate salts to make computed titration curves match laboratory titration curves. The great disparity between theoretical and actual titration curves is due to conjugate salts and incredibly small amounts of DCO2 from simple exposure to air and a corresponding amount of carbonic acid created. Instead of titration curve slopes and thus process gains increasing by 6 orders of magnitude as you go from 0 to 7 pH for strong acids and strong bases, in reality the slope increases by 2 orders of magnitude, still a lot but 4 orders of magnitude off. Thus, control system analysis and supposed linearization by translation of the controlled variable from pH to hydrogen ion concentration by the use of theoretical equations for a strong acid and strong base is off by 4 orders of magnitude. I made this mistake early in my career (about 40 years ago) but learned at the start of the 1980s that DCO2 was the deal breaker. I have seen the theoretical linearization published by others about 20 years ago and most recently just last year. For all pH systems, the slope between 4 and 7 pH is greatly moderated due to the carbonic acid pKa = 6.35 at 25 degrees Centigrade. The titration curve is also flattened within two pH of logarithmic acid dissociation constant (pKa) of an acid or base that has a conjugate salt. To match computer generated titration curves to laboratory titration curves, add small amounts DCO2 and conjugate salts as detailed in the Chemical Processing feature article “Improve pH control”.
Realize there is a multiplicative effect for biological process kinetics that creates restrictions on experimental methods to analyze or predict cell growth or product formation. While the incentive is greater for high value biologic products, there are challenges with models of biological processes due to multiplicative effects (neural networks and data analytic models assume additive effects). Almost every first principle model (FPM) has specific growth rate and product formation the result of a multiplication of factors each between 0 and 1 to detail the effect of temperature, pH, dissolved oxygen, glucose, amino acid (e.g., glutamine), and inhibitors (e.g., lactic acid). Thus, each factor changes the effect of every other factor. You can understand this by realizing that if the temperature is too high, cells are not going to grow and may in fact die. It does not matter if there is enough oxygen or glucose. Similarly if there is not enough oxygen, it does not matter if the all the other conditions are fine. One way to address this problem is to make all factors as close to 1 and as constant as possible except for the one of interest. It has been shown data analytics can be used to identify the limitation and/or inhibition FPM parameter for one condition, such as the effect of glucose concentration via the Michaelis-Menten equation if all other factors are constant and nearly 1.
Take advantage of the great general applicability and ease of parameter adjustment in Michaelis-Menten equations for the effect of concentrations and Convenient Cardinal equations for the effect of temperature and pH on biological processes. The Mimic bioreactor model in a digital twin takes advantage of these breakthroughs in first principle modeling. For temperature and pH, Convenient Cardinal equations are used where the optimum temperature for growth and production phases is simply the temperature or pH setpoint including any shifts for batch phases. The minimum and maximum temperatures complete the parameter settings. This is a tremendous advancement to traditional uses of Arrhenius equations for temperature and Villadsen-Nielsen equations for pH that required parameters not readily available set with a precision to the sixth or seventh decimal place. Generalized Michaelis-Menten equations shown to be useful for modeling intracellular dynamics can model the extracellular limitation and inhibition effects of concentrations. The equations provide a link between macroscopic and microscopic kinetic pathways. If the limiting or inhibition effect is negligible or needs to be temporarily removed, the limitation and inhibition parameter is simply set to 0 g/L and 100 g/L, respectively. The biological significance and ease of setting parameters is particularly important since most kinetics are not completely known and what is defined can be quite subject to restrictions on operating conditions. These revolutionary equations enable the same generalized kinetic model to be used for all types of cells. Previously, yeast cells (e.g., ethanol production), fungal cells (e.g., antibiotic production), bacterial cells (e.g., simple proteins), and mammalian cells (e.g., complex proteins) had specialized equations developed that did not generally carry over to different cells and products.
Always use smart sensitive valve positioners with good feedback of actual position tuned with a high gain and no integral action on true throttling valves (please read the following despite its length since misunderstandings are pervasive and increasing). A very big and potentially dangerous mistake persists today from a decades old rule that positioners should not be used on fast loops. The omission of a sensitive and tuned valve positioner can increase limit cycle period and amplitude by an order of magnitude and severely jeopardize rangeability and controllability. Without a positioner, some valves may require a 25% change in signal to open meaning that controlling below 25% signal is unrealistic. As a young lead I&E engineer for the world’s largest acrylonitrile plant back in the mid-1970s, I used the rule about fast loops. The results were disastrous. I had to hurriedly install positioners on all the loops during startup. A properly designed, installed and tuned positioner should have a response time of less than 0.5 seconds. A positioner gain greater than 10 is sought with rate action added when offered. Positioners developed in the 1960s were proportional only with a gain of 50 or more and a high sensitivity (0.1%). Since then spool positioners with extremely poor sensitivity (2%) have been offered and integral action included and even recommended in some misguided documents by major suppliers. Do not use integral action in the positioner despite default settings to the contrary. A volume booster can be added to the positioner output to make the response time faster. Using a volume booster instead of a positioner is dangerous as explained in next point. If you cannot achieve the 0.5% response time, you have something wrong with type of valve, packing, positioner, installation and/or tuning of the positioner and is not a reason for saying that you should not use positioners on fast loops. An increasing threat ever since the 1970s has been on-off valves posing as throttling valves. They are much less expensive, in the piping spec and have much tighter shutoff. The actuator shaft feedback may change even though the actual ball or disk has not changed for a signal change of 8% or more. In this case, even the best positioner is of little help since it is being lied to as to actual position. Valve specifications have an entry for leakage but typically have nothing on valve backlash, stiction, and response time and actuator sensitivity. I can’t seem to get even a discussion started as to how to get this changed and how rangeability and controllability are so adversely affected. If you need a faster response or are stuck with an on-off valve, then you need to consider a variable frequency drive with a pulse width modulated inverter (see point 35 in part 4 of this series). Also be aware that theoretical studies based solely on process dynamics are seriously flawed since most fast loops, sensors, transmitters, signal filter, scan and execution times are a larger source of time constants and dead time than the actual process making the loop much slower than what is shown in studies based on process response as noted in point 9 in part 1. For much more on how to deal with this increasing threat read the Control articles “Is your control valve an imposter?” and “How to specify valves and positioners that don’t compromise control”.
Put a volume booster on output of positioner with a bypass valve opened just enough to make the valve stroking time much faster recognizing that replacing a positioner with a booster poses a major safety risk. Another decades old rule said to replace a positioner with a booster on fast loops. For piston actuators, a positioner is required to work at all. For diaphragm actuators, a volume booster instead of a positioner creates a positive feedback (flexure of diaphragm changes volume and consequently pressure seen by booster high outlet port sensitivity) causing a fail open butterfly valve to slam shut from fluid forces on the disk. This has happened to me on a huge compressor and was subsequently avoided on the next project when I showed I could position 24 inch fail open butterfly valves for furnace pressure control by simply grabbing the shaft due to positive feedback from booster and diaphragm actuator combination. Since properly sized diaphragm actuators generally have an order of magnitude better sensitivity than piston actuators and the operating pressure of newer diaphragm actuators has been increased, diaphragm actuators are increasingly the preferred solution.
Understand and address the reality that there are processes that have either a dead time dominant and balanced self-regulating, near-integrating, true integrating, or runaway response. Most of the literature studies balanced self-regulating processes where the process time constant is about the same size as the process dead time. Some studies address dead time dominant processes where the dead time is much greater than the process time constant. Dead time dominant processes are less frequent and mostly occur when there is a large dead time from process transportation delay (e.g., plug flow volumes or conveyors) or analyzer sample and cycle time (see points 1 and 9 in part 3 on how to address these applications). The more important loops tend to be near-integrating where the process time constant is more than 4 times larger than the process dead time, true integrating where the process will continually ramp when the controller is in manual and runaway where the process deviation will accelerate when the controller is in manual. Continuous temperature and composition loops on volumes with some degree of mixing due to reflux, recycle or agitation have a near-integrating response. Batch composition and temperature have a true integrating response. The runaway response occurs in highly exothermic typically polymerization reactors but is never actually observed because it is too dangerous to put the controller in manual during a reaction long enough to see much acceleration. Most gas pressure loops and of course, nearly all level loops have an integrating response. It is critical to tune PID controllers on near-integrating, true integrating and runaway processes with maximum gain, minimum reset action and maximum rate action so that the PID can provide the negative feedback action missing in these processes. As a practical matter, near-integrating, true integrating, and runaway processes are tuned with integrating process tuning rules where the initial ramp rate is used to estimate an integrating process gain.
Maximize synergy between chemical engineering, biochemical engineering, electrical engineering, mechanical engineering and computer science. All of these degrees bring something to the party for a successful automation system implementation. The following simplification provides some perspective: chemical and biochemical engineers offer process knowledge, electrical engineers offer control, instrumentation and electrical system knowledge, mechanical engineers offer equipment and piping knowledge, and computer scientists offer data historian and industrial internet knowledge. All of these people plus operators should be involved in process control improvements and whatever is expected from the next big thing (e.g., Industrial Internet of Things, Digitalization, Big Data and Industry 4.0). The major technical societies especially AIChE, IEE, ISA, and ASME should see the synergy of exchange of knowledge rather than the current view of other societies being competition.
Identify and document justifications to develop new skills, explore new opportunities and innovate. Increasing emphasis on reducing project costs is overloading practitioners to the point they don’t have time to attend short courses, symposiums or even online presentations. Contributing factors are loss of expertise from retirements, fear of making any changes, and present day executives who have no industry experience and are focused on financials, such as reducing project expenditures and shortening project schedules. At this point practitioners must be proactive and do investigation on their own time of opportunities and process metrics. Developing skills with the digital twin can be way of defining and showing associates and management the type and value of improvements as noted in all points in part 5. The digital twin with demonstrated key performance indicators (KPI) showing value of the increases in process capacity or efficiency, plus data analytics and Industry 4.0 can lead to people teaching people, eliminating silos, spurring creativity and deeper involvement, nurturing a sense of community and common objectives, and connecting the layers of automation and expertise so everybody knows everybody. To advance our profession, practitioners should seek to publish what is learned, which can be done generically without disclosing proprietary data.
Use inferential measurements periodically corrected by at-line analyzers to provide fast analytical measurements of key process compositions. First principle models or experimental models identified by model predictive control or data analytics software can be used to provide immediate composition measurements with no delay associated with process sample system and analyzer cycle time. The inferential measurement result is synchronized with an at-line analyzer result by the insertion of a dead time equal to sample transportation delay plus 1.5 times the analyzer cycle time. A fraction, usually less than 0.5 of the difference between the inferential measurement and at-line analyzer result after elimination of outliers, is added to correct the inferential measurement whenever there is an updated at-line analyzer result.
Use inline analyzers and at-line analyzers whose sensors are in the process or near the process, respectively. There are many inline sensors available today (e.g., conductivity, chlorine, density, dissolved carbon dioxide, dielectric spectroscopy, dissolved oxygen, focused beam reflectance, laser based measurements, pH, turbidity, and viscosity). The next best alternative is an at-line analyzer located as close as possible to the process connection to minimize sample transportation delay. The practice of locating all analyzers in one building creates horrendous dead time. An example of an innovative fast at-line analyzer capable of extensive sensitive measurements of components plus cell concentration and size for biological processes is the Nova Bioprofile Flex. Chromatographs, near infrared, mass spectrometers, nuclear magnetic resonance, and MLT gas analyzers using a combination of non-dispersive infrared, ultraviolet and visible spectroscopy with electrochemical and paramagnetic sensors have increased functionality and maintainability.

How Often Do Measurements Need to Be Calibrated?

2019-07-17T12:00:52Z

The post How Often Do Measurements Need to Be Calibrated? first appeared on the ISA Interchange blog site.

Greg Breitzke is an E&I reliability specialist – instrumentation/electrical for Stepan. Greg has focused his career on project construction and commissioning as a technician, supervisor, or field engineer. This is his first in-house role, and he is tasked with reviewing and updating plant maintenance procedures for I&E equipment.

Greg Breitzke’s Question

I am working through an issue that can be beneficial to other Mentor Program Participants. NFPA 70B provides a detailed description for the prescribed maintenance and frequency based on equipment type, making the electrical portion fairly straight forward. The instrumentation is another matter. We are working to consolidate an abundance of current procedures based on make/model, to a reduced list based on technology. The strategy is to “right size” frequencies for calibration and functional testing; decreasing non-value maintenance to have the ability to increase value added activities, within the existing head count.

My current plan for the instrumentation consists of:

Sort through the historical paper files with calibration records to determine how long a device has remained in tolerance before a correction was applied,
Compare data against any work orders written against the asset that may reduce the frequency,
Apply safety factors relative to the device impact on safety, regulatory compliance, quality, custody transfer, basic control, or indication only.

I am trying to provide a reference baseline for review of these frequencies, but having little luck in the industry standards I have access to. Is there a standard or RAGAGEP for calibration and functional testing frequency min/max by technology, that I can reference for a baseline?

Nick Sands’ Answer

The ISA recommended practice is not on the process of calibration but on a calibration management system: ISA-RP105.00.01-2017, Management of a Calibration Program for Industrial Automation and Control Systems. While I contributed, Leo Staples would be a good person for more explanation.

For SIS, there is a requirement to perform calibration, which is comparison against a standard device, within a documented frequency and with documented limits, and correction when outside of limits. This is also required by OSHA for critical equipment under the PSM regulation. EPA has similar requirements under PSRM of course. Correction when out of limits is considered a failed proof test of the instrument in some cases, potentially affecting the reliability of the safety function. Paul Gruhn would be a good person for more explanation.

Paul Gruhn’s Answer

The ISA/IEC 61511 is performance based and does not mandate specific frequencies. Devices must be tested as some interval to make sure they perform as intended. The frequency required will be based on many different factors (e.g., SIL (performance) target, failure rate of the device in that service, diagnostic coverage, redundancy used (if any), etc.).

Leo Staples’ Answer

Section 5.6 ISA of the ISA technical report ISA-RP105.00.01-2017 addresses in detail calibration verification intervals or frequencies. Users should establish calibration intervals for a loop/component based on the following:

criticality of the loop/component
the performance history of the loop/component
the ruggedness/stability of the component(s)
the operating environment.

Exceptions include SIS related devices where calibration intervals are established to meet SIL requirements. Other factors that can drive calibration intervals include contracts regulatory requirements.

The idea for the technical report came about after years of frustration dealing with ambiguous gas measurement contracts and government regulations. In many cases these simply stated users should follow good industry practices when addressing all aspects of calibrations.

Calibration intervals alone do not address the other major factors that affect measurement accuracy. These include the accuracy of the calibration equipment, knowledge of the calibration personnel, adherence to defined calibration procedures, and knowledge of the personnel responsible for the calibration program. I have lots of war stories if anyone is interested.

One of the last things that I did at my company before I retired was develop a Calibration Program Standard Operating Procedure (SOP) based on ISA-RP105.00.01-2017. The SOP was designed for use in the Generation, Transmission & Distribution, and other Division of the Company. Some of you may find this funny, but it was even used to determine the calibration frequency for NERC CIP physical security entry control point devices. Initially personnel from the Physical Security Department were testing these devices monthly only because that was what they had always done. While this was before the SOP was established my team used the concepts in establishing the calibration intervals for these devices. This work was well received by the auditors. As a side note, the review of monthly calibration intervals for these devices found the practices caused more problems than it prevented.

The ISA recommended practice is not on the process of calibration, but on a calibration management system: ISA-RP105.00.01-2017, Management of a Calibration Program for Industrial Automation and Control Systems.

ISA Mentor Program

The ISA Mentor Program enables young professionals to access the wisdom and expertise of seasoned ISA members, and offers veteran ISA professionals the chance to share their wisdom and make a difference in someone’s career. Click this link to learn more about the ISA Mentor Program.

Greg McMillan’s Answer

The measurement drift can provide considerable guidance in that when the number of months between calibrations multiplied by drift per month approaches the allowable error, it is time for a calibration check. Most transmitters today have a low drift rate but thermocouples and most electrodes have a drift rate much larger than the transmitter. The past records of calibration results will provide an update on actual drift for an application. Also, fouling of sensors, particularly electrodes, is an issue revealed in 86% response time during calibration tests (often overlooked). The sensing element is the most vulnerable component in nearly all measurements. Calibration checks should be made more frequently at the beginning to establish a drift rate and near the end of the sensor life when drift and failure rates accelerate. Sensor life for pH electrodes can decrease from a year to a few weeks due to high temperature, solids, strong acids and bases (e.g., caustic) and poisonous ions (e.g., cyanide). For every 25 oC increase in temperature, the electrode life is cut in half unless a high temperature glass is used.

Accuracy is particularly important for primary loops (e.g., composition, pH, and temperature) to ensure you are at the right operating point. For secondary loops whose setpoint is corrected by a primary loop, accuracy is less of an issue. For all loops, the 5 Rs (reliability, resolution, repeatability, rangeability and response time) are important for measurements and valves.

Drift in a primary loop sensor shows up as a different average controller output for a given production rate assuming no changes in raw materials, utilities, or equipment. Fouling of a sensor shows up as an increase in dead time and oscillation loop period.

Middle signal selection using 3 separate sensors provides an incredible amount of additional intelligence and reliability reducing unnecessary maintenance. Drift shows up as a sensor with a consistently increasing average deviation from the middle value. The resulting offset is obvious. Coating shows up as a sensor lagging changes in the middle value. A decrease in span shows up as a sensor falling short of middle value for a change in setpoint.

The installed accuracy greatly depends upon installation details and process fluid particularly taking into account sensor location in terms of seeing a representative indication of the process with minimal measurement noise. Changes in phase can be problematic for nearly all sensors. Impulse lines and capillary systems are a major source of poor measurement performance as detailed in the Control Talk columns Prevent pressure transmitter problems and Your DP problems could be a result of improper use of purges, fills, capillaries and seals.

At the end of this post, I give a lot more details on how to minimize drift and maximize accuracy and repeatability by better temperature and pH sensors and through middle signal selection.

Free Calibration Essentials eBook

For an additional educational resource, download Calibration Essentials, an informative eBook produced by ISA and Beamex. The free e-book provides vital information about calibrating process instruments today. To download the eBook, click this link.

Hunter Vegas’ Answer

There is no easy answer to this very complicated question. Unfortunately the answer is ‘it depends’ but I’ll do my best to cover the main points in this short reply.

1) Yes there are some instrument technologies that have a tendency to drift more than others. A partial list of ‘drifters’ might include:

- pH (drifts for all kinds of reasons – aging of probe, temperature, caustic/acid concentration, fouling, etc. etc.)
- Thermocouples (tend to drift more than RTDs especially at high temperature or in hydrogen service)
- Turbine meters in something other than very clean, lubricating service will tend to age and wear out so they will read low as they age. However cavitation can make them intermittently read high.
- Vortex meters with piezo crystals can age over time and their low flow cut out increases.
- Any flow/pressure transmitter with a diaphragm seal can drift due to process temperature and/or ambient temperature.
- Most analyzers (oxygen, CO, chromatographs, LEL)
- This list could go on and on.

2) Some instrument technologies don’t drift as much. I’ve had good success with Coriolis and radar. (Radar doesn’t usually drift as much as it just cuts out. Coriolis usually works or it doesn’t. Obviously there are situations where either can drift but they are better than most.) DP in clean service with no diaphragm seals is usually pretty trouble free, especially the newer transmitters that are much more stable.

3) The criticality of the service obviously impacts how often one needs to calibrate. Any of these issues could dramatically impact the frequency:

- Is it a SIS instrument? The proof testing frequency will be decided by the SIS calculations.
- Is it an environmental instrument? The state/feds may require calibrations on a particular frequency.
- Is it a custody transfer meter? If you are selling millions of pounds of X a year you certainly want to make sure the meter is accurate or you could be giving away a lot of product!
- Is it a critical control instrument that directly affects product quality or throughput?

4) Obviously if a frequency is dictated by the service then that is the end of that. Once those are out of the way one can usually look at the service and come up with at least a reasonable calibration frequency as a starting point. Start calibrating at that frequency and then monitor history. If you are checking a meter every six months and have checked a meter 4 times in the last two years and the drift has remained less than 50% of the tolerance, then dropping back to a 12 month calibration cycle make perfect sense. Similarly if you calibrate every 6 months and find the meter drift is > 50% every calibration then you probably need to calibrate more often. However if the meter is older it may be cheaper to replace the meter with a new transmitter which is more stable.

5) The last comment I’ll make is to make sure you are actually calibrating something that matters. I could go on for pages about companies who are diligently calibrating their instrumentation but aren’t actually calibrating their instrumentation. In other words they go through the motions, fill out the paperwork, and can point to reams of calibration logs yet they aren’t adequately testing the instrument loop and it could still be completely wrong. (For instance, shooting a temperature transmitter loop but not actually checking the RTD or thermocouple that feeds it, using a simulator to shoot a 4-20mA signal into the DCS to check the DCS reading but not actually testing the instrument itself, etc. They often check one small part of the loop and after a successful test, consider the whole loop ‘calibrated’.

Greg McMillan’s Answer

The Process/Industrial Instruments and Controls Handbook Sixth Edition 2019 edited by me and Hunter Vegas provide insight on how to maximize accuracy and minimize drift for most types of measurements. The following excerpt written by me is for temperature:

Temperature

The repeatability, accuracy and signal strength are two orders of magnitude better for an RTD compared to a TC. The drift for a RTD below 400 ^oC is also two orders of magnitude less than a TC. The 1 to 20^oC drift per year of a TC is of particular concern for biological and chemical reactor and distillation control because of the profound effect on product quality from control at the wrong operating point. The already exceptional accuracy for a Class A RTD of 0.1^oC can be improved to 0.02 ^oC by “sensor matching” where the four constants of a Callendar-Van-Dusen (CVD) equation provided by the supplier for the sensor are entered into the transmitter. The main limit to accuracy of an RTD is the wiring.

The use of three extension lead wires between the sensor and transmitter or input card can enable the measurement to be compensated for changes in resistance in the lead wires due to temperature assuming the change is exactly the same for both lead wires. The use of four extension lead wires enables total compensation that accounts for the inevitable uncertainty in resistance of lead wires. Standard lead wires have a tolerance of 10% in resistance. For 500 feet of 20 gauge lead wire, the error could be as large as 26^oC for a 2-wire RTD and 2.6^oC a 3-wire RTD. The “best practice” is to use a 4 wire RTD unless the transmitter is located close to the sensor, preferably on the sensor. The transmitter accuracy is about 0.1^oC.

A handheld signal generator of resistance and voltage can be used to simulate the sensor to check or change a transmitter calibration. The sensor connected to the transmitter with linearization needs to be inserted in a dry block simulator. A bath can be used for low temperatures to test thermowell response time but a dry block is better for calibration. The accuracy of the reference temperature sensor in the block or bath should be 4 times more accurate than the sensor being tested. The block or bath readout resolution must be better than the best possible precision of the sensor. The block or bath calibration system should have accuracy traceable to the National Metrology Institute of the user country (NIST in USA).

The accuracy at the normal setpoint to ensure the proper process operating point must be confirmed by a temperature test with a block. For factory assembled and calibrated sensor and thermowell with integral temperature transmitter, a single point temperature test in a dry block is usually sufficient with minimal zero or offset adjustment needed. For an RTD with “sensor matching,” adjustment is often not needed. For field calibration, the temperature of the block must be varied to cover the calibration range to set the linearization, span and zero adjustments. For field assembly, it would be wise to check the 63% response time in a bath.

Middle Signal Selection

The best solution in terms of increasing reliability, maintainability, and accuracy for all sensors with different durations of process service is automatic selection of the middle value for the loop process variable (PV). A very large chemical intermediates plant extended middle signal selection to all measurements that in combination with triple redundant controller essentially eliminated the one or more spurious trips per year. Middle signal selection was a requirement for all pH loops in Monsanto and Solutia.

The return on investment for the additional electrodes from improved process performance and reduced life cycle costs is typically more than enough to justify the additional capital costs for biological and chemical processes if the electrode life expectancy has been proven to be acceptable in lab tests for harsh conditions. The use of the middle signal inherently ignores a single failure of any type including the most insidious failure that gives a pH value equal to the set point. The middle value reduces noise without the introduction of the lag from damping adjustment or signal filter and facilitates monitoring the relative speed of the response and drift, which are indicative of measurement and reference electrode coatings, respectively. The middle value used as the loop PV for well-tuned loops will reside near the set point regardless of drift.

A drift in one of the other electrodes is indicative of a plugging or poisoning of its reference. If both of the other electrodes are drifting in the same direction, the middle value electrode probably has a reference problem. If the change in pH for a set point change is slower or smaller for one of the other electrodes, it indicates a coating or loss in efficiency, respectively for the subject glass electrode. Loss of pH glass electrode efficiency results from deterioration of glass surface due to chemical attack, dehydration, non-aqueous solvents, and aging accelerated by high process temperatures. Decreases in glass electrode shunt resistance caused by exposure of O-rings and seals to a harsh or hot process can also cause a loss in electrode efficiency.

pH Electrodes

Here is some detailed guidance on pH electrode calibration from the ISA book Essentials of Modern Measurements and Final Control Elements.

Buffer Calibrations

Buffer calibrations use two buffer solutions, usually at least 3 pH units apart, which allow the pH analyzer to calculate a new slope and zero value, corresponding to the particular characteristics of the sensor to more accurately derive pH from the milliVolt and temperature signals.

The slope and zero value derived from a buffer calibration provide an indication of the condition of the glass electrode from the magnitude of its slope, while the zero value gives an indication of reference poisoning or asymmetry potential, which is an offset within the pH electrode itself.
The slope of pH electrode tend to decrease from an initial value relatively close to the theoretical value of 59.16 mV/pH, largely due in many cases to the development of a high impedance short within the sensor, which forms a shunt of the electrode potential.
Zero offset values will generally lie within + 15 mV due to liquid junction potential, larger deviations are indications of poisoning.
Buffer solutions have a stated pH value at 25°C, but the stated value changes with temperature especially for stated values that are 7 pH or above. The buffer value at the calibration temperature should be used or errors will result.
The values of a buffer at temperatures other than 25°C are usually listed on the bottle, or better, the temperature behavior of the buffer can be loaded into the pH transmitter allowing it to use the correct buffer value at calibration.
Calibration errors can also be caused by buffer calibrations done in haste, which may not allow the pH sensor to fully respond to the buffer solution.
This will cause errors, especially in the case of a warm pH sensor not being given enough time to cool down to the temperature of the buffer solution.
pH transmitters employ a stabilization feature, which prevents the analyzer from accepting a buffer pH reading that has not reached a prescribed level of stabilization, in terms of pH change per time.

pH Standardization

Standardization is a simple zero adjustment of a pH analyzer to match the reading of a sample of the process solution made using a laboratory or portable pH analyzer. Standardization eliminates the removal and handling of electrodes and the upset to the equilibrium of the reference electrode junction. Standardization also takes into account the liquid junction potential from high ionic strength solutions and non-aqueous solvents in chemical reactions that would not be seen in buffer solutions. For greatest accuracy, samples should be immediately measured at the sample point with a portable pH meter.

If a lab sample measurement value is used, it must be time stamped and the lab value compare to a historical online value for a calibration adjustment. The middle signal selected value from three electrodes of different ages can be used instead of a sample pH provided that a dynamic response to load disturbances or setpoint changes of at least two electrodes is confirmed. If more than one electrode is severely coated, aged, broken or poisoned, the middle signal is no longer representative of actual process pH.

Standardization is most useful for zeroing out a liquid junction potential, but some caution should be used when using the zero adjustment.
A simple standardization does not demonstrate that the pH sensor is responding to pH, as does a buffer calibration, and in some cases, a broken pH electrode can result in a believable pH reading, which may be standardized to a grab sample value.
A sample can be prone to contamination from the sample container or even exposure to air; high purity water is a prime example, a referee measurement must be exposed to a flowing sample using a flowing reference electrode.
A reaction occurring in the sample may not have reached completion when the sample was taken, but will have completed by the time it reaches the lab.
Discrepancies between the laboratory measurement and an on-line measurement at an elevated temperature may be due to the solution pH being temperature dependent. Adjusting the analyzer’s solution temperature compensation (not a simple zero adjustment) is the proper course of action.
It must be remembered that the laboratory or portable analyzer used to adjust the on-line measurement is not a primary pH standard, as is a buffer solution, and while it is almost always assumed that the laboratory is right, this is not always the case.

The calibration of pH electrodes for non-aqueous solutions is even more challenging as discussed in the Control Talk column The wild side of pH measurement.

Additional Mentor Program Resources

See the ISA book 101 Tips for a Successful Automation Career that grew out of this Mentor Program to gain concise and practical advice. See the InTech magazine feature article Enabling new automation engineers for candid comments from some of the original program participants. See the Control Talk column How to effectively get engineering knowledge with the ISA Mentor Program protégée Keneisha Williams on the challenges faced by young engineers today, and the column How to succeed at career and project migration with protégé Bill Thomas on how to make the most out of yourself and your project. Providing discussion and answers besides Greg McMillan and co-founder of the program Hunter Vegas (project engineering manager at Wunderlich-Malec) are resources Mark Darby (principal consultant at CMiD Solutions), Brian Hrankowsky (consultant engineer at a major pharmaceutical company), Michel Ruel (executive director, engineering practice at BBA Inc.), Leah Ruder (director of global project engineering at the Midwest Engineering Center of Emerson Automation Solutions), Nick Sands (ISA Fellow and Manufacturing Technology Fellow at DuPont), Bart Propst (process control leader for the Ascend Performance Materials Chocolate Bayou plant), Angela Valdes (automation manager of the Toronto office for SNC-Lavalin), and Daniel Warren (senior instrumentation/electrical specialist at D.M.W. Instrumentation Consulting Services, Ltd.).

Connect with Greg