Emerson Exchange 365

MX Controller Cross Communications

Hi all,

I wanted to ask about what to check, look out for and what the limiting factors are when implementing control strategies in separate controllers when the control strategies have a dependence on each other.

Looking in Diagnostics, both controllers are MX and each have unsolicited parameters sent of 440 and 750, unsolicited parameters received of 26 and 52. One has a free processor time of 26 and the other 27.

Given these values should I be considering doing the configuration in one controller alone, are there other considerations to take into account?

Thanks,   

David Billington

• 5 Aug 2016 1:29 PM
• Cancel

3 Replies

• Andre Dicaire

  

  • 9 Aug 2016 6:33 PM

  The DeltaV Communication layer is built to facilitate cross controller communication.
  
  Given the values above, there is no indication that you need to change anything. I can't comment on whether you've got anything to be concerned about based on exactly how your configuration is designed. What version are you one? An MX controller at 26 % Free time is getting close to full, with recommended minimum Free time at 20%. Having a remote reference is fully supported and expected. The added communication update time must be considered, and usually we done doe closed loop control with the input on one controller and the output on an other, though nothing but loop performance limits this.
  
  Here are a few fun facts to know.
  
  - How many parameters can be communicated between two DeltaV nodes?
  There are no limits to the number of parameters you can reference between controllers. But, there are limits to how many values are received and sent per second via unsolicited exception reporting.
  The Unsolicited Send parameters in diagnostics indicate the number of values being sent per second from that controller to all nodes in the system that have requested data from it. Unsolicited data is sent by exception, which significantly reduces the number of values sent from the number of parameters mapped. Much of the parameters requested by consoles are static, like Engineering Unit Scaling parameters, Alarm limits, Tuning variables, etc. So although the consoles receive way more parameters, the values per second is a fraction of the total number of parameters.
  
  In addition to reducing the number of values sent by exception reporting, each controller has a maximum number of values per second it can send (SX/MX = 4000, SQ/MQ/MDPLUS= 2000, as documented in system Limits in BOL) To prevent one client node from monopolizing this pipe, there is also a limit of 750 values per second to each requesting node. Finally, the communication layer prunes parameters that are no longer used. This pruning primarily impacts the data serving Operator Graphics or Control Studio, further optimizing data transfer to the consoles. As you can see in your example, the normal unsolicited is 440 and 750 for all data being exchanged. This is typical.
  
  - What happens if you reach the 750 values per second to one node, or the 4000 values per second for the controller?
  Data up dates will slow such that the average per second does not exceed the limit. So if 1500 values needed to be sent to one node, it would take 2 seconds for all the data. IF the total number of values sent exceeds 4000, then all parameters will be throttled to keep the rate at 4000. Burst load conditions are typically due to Operator graphic navigation, where a user cycles through multiple displays and calls up detail parameter pages. although this activity may add many parameters to the communication layer, exception reporting will still keep the number of values sent relatively low. Other traffic to Controllers, Historian and OPC connections will generally be stable.
  
  - How does DeltaV manage peer to peer communication between controllers?
  There are two mechanisms for sharing data. The most common one is unsolicited communication. The second is the Asynchronous Write mechanism.
  
  A module running in one controller requires a value in a different controller. This is configured as an External reference, and defined by the Host Module name. So if MODULEA wants to read parameter called PV in MODULEB, the external reference is defined in MODULEA as //MODULEB/PV.CV. When MODULEA is downloaded it determines the need for a remote connection and sets up what is called a PROXY CLIENT for MODULEB, and adds parameter PV to the list of parameters. The PROXY CLIENT locates the controller hosting MODULEA and that controller sets up the PROXY SERVER for MODULEA, and retrieves the parameter PV. All this happens automatically at runtime. To monitor the health of this remote parameter, the Connection Status (.CST) field provides the destination with the ability to check the communication status of the parameter. (CST=0 means good). The update rate for this parameter is based on the execution speed of MODULEB, the requesting module. The PROXY SERVER checks the MODULEA/PV parameter at this frequency for a change. Of course, the parameter can only change when the MODULEA updates. If there is no change, the parameter is not sent. If it is flagged as changed, the value is sent. So the update rate of remote parameters is the slower of the two modules involved.
  
  The second mechanism is the Asynchronous Write mechanism. This mechanism uses another status field called AWST, for Asynchronous Write Status. When a module writes to a parameter located in a different controller, the remote reference is established as an unsolicited read, and the proxy objects are created. So if MODULEB wrote to a parameter in MODULEA, say a parameter called SP, the same proxy Client/Server objects would be used, and Parameter SP would be added. When MODULEB executes and writes to MODULEA/SP, the SP.AWST is set to a pending status, and the value is handed off to the AW task. The write will be executed within 200 ms, and the result of the write will be reported to .AWST. If the write is successful, the .AWST goes to 0. It may fail if the value was out of range, or the block is in the wrong mode or the value does not match. (writing a string to FLOAT or setting a Named Set to a value not defined.). This allows the writing module to execute the write and move on. The AWST provides a mechanism to verify if the value was changed. By having the proxy client report the current value, logic can check if a write is required, if CST is good, and after the write can confirm if the write was successful.
  
  Every parameter has the .ST, .CST and .AWST fields. .CST and .AWST will always be 0 when read by a module in the same controller. So if you write logic to check if CST or AWST are good (=0), they will always be good for local parameters. So the same logic will work for local and remote parameters. The .ST field will show BAD if CST is also bad. IF CST is GOOD, you know that .ST reflects the status of the remote parameter.
  
  So if you add an action to write to a remote parameter, the controller will first setup the proxy to read that parameter and make it available for the logic to check. It is recommended that writes be used sparingly. They should always be done by exception. It is always recommended to read a value rather than write. Here's the difference. The unsolicited reads are handled in the communication layer and the data is made available to all modules in the destination controller. If a parameter is read by one or more modules in a controller, a single proxy client is created there and all modules can read that value. It is sent once based on the fastest requesting module and the source module execution rates. When you write a parameter, the AW task must confirm the success of the write and must check the state of the parameter for limits, data type and block mode. This action is done at a medium priority and takes CPU time. Too many writes into a controller can take away CPU from medium and low priority control tasks. This can result in module slippage. Writes should be done by exception by checking if the value needs to be changed. SFC Action statements are the mode common use of writes and by their nature are typically one time events. Avoid a module with a wired connection to an External reference to a remote parameter. This creates a write action every scan. Emerson recommends less than 20 writes per second. Note this is from the point of the controller being written two. Writes can come from any controller, so if you have 20 controllers and all of them write 2 parameters per second to a 21st controller, that will be 40 writes per second.
  
  So read rather than write, wherever possible.
  
  -What is the limit on Unsolicited receives per controller?
  This is in BOL as well, with SX/MX at 500 and SQ/MQ at 250. This is not a hard limit, like with Sends, where data is throttled. However, exceeding this limit can result in a reduced capacity to send data. There is a limited amount of CPU allocated for unsolicited communication, ensuring CPU is allocated for things like Control. From a controller perspective, the unsolicited receives reflect external references to remote parameters. In most cases those values are process values that change every scan. So although there is no limit on the parameters mapped, respecting the limit of Unsolicited receive values means you likely have no more than 500 or 250 parameters to any single controller. If you map static values like EU range or a SP that is infrequently changed, these parameters will not have changing values and will not count in this limit.
  
  Note that once a parameter is mapped between controllers, it is never pruned. Pruning of parameters is only done by consoles, when the value is no longer used by any client application. For controllers, the expectation is that once a parameter is mapped, it will be required and should always be available. The controllers there don't benefit from pruning.
  
  - How many proxy server/clients are created?
  DeltaV uses Uni-Cast Ethernet messaging to send data between nodes. Each Proxy Server serves data to exactly one Proxy Client. So if a console needs a value every second, and the historian needs the same value every 5 seconds, no problem. The Proxy client at the console sets up the 1 second update Proxy Server, while the Historian sets up a 5 second Proxy Server. On a controller, a module will have multiple Proxy Servers, one for every node that is requesting at least one parameter from that module. In each console or controller reading data, there will be a Proxy Client that is managing the unsolicited data received. All applications or control modules running in that node will get parameter data from the same proxy client.
  
  In this way, data is managed to each destination with a dedicated Proxy Server, each dealing with one Proxy Client. A module sent to 20 consoles will have 20 Proxy Servers in the controller and one proxy client in each console.
  
  - How loaded can the DeltaV network get?
  As you can see, this mechanism limits the bandwidth usage of each controller, and prevents the system from becoming overloaded, even at 100 MB network speeds. A heavily loaded MX controller reaches about 1 % of the network bandwidth. One would have to try very hard to do everything wrong to create a DeltaV network that would be overloaded, but I know there are some clever people out there so I'm sure someone might be able to.
  
  - What happens on a controller switch over?
  This is where .CST and other configuration options come in to play. in version prior to v11 and v10, a controller switchover would cause existing Proxy Servers to "disappear", and the Proxy Client Connection Status would go to BAD briefly. The recommendation was to evaluate for .CST and to use the feature "ABORT ON READ ERROR" in Calc and CND blocks. This would allow an expression to stop executing when the connection was BAD, and to explicitly define how the control should respond to a BAD connection status.
  
  In v12, enhancements were made to better handle the switchover condition, where parameter and status are held on a switchover. A loss of connection due to controller failure or network outage will still result in the loss of connection status. The appropriate response to a loss of connection must still be considered, but the controller switchover is now more robust as a result of these enhancements.
  
  So as a rule,
  - control strategies that are tightly coupled should be in the same controller.
  - Remote parameter values should be read whenever possible.
  - Parameter writes should only be done by exception to avoid continuous writes to a controller.
  - And Status handling should be considered to properly respond to loss of communications.

  Andre Dicaire

  • Cancel
• ReyK

  

  • 10 Aug 2016 7:38 AM

  In reply to Andre Dicaire:

  Andre Dicaire,
  thanks a lot for very helpful and detailed description.
  
  I haven´t read Books Online for this issue recently, but recall the content wasn´t written as well as you did.
  I recommend the Books Online should be updated with many of the valuable information given by you.
  
  Reynir Kristbjörnsson

  • Cancel
• Andre Dicaire

  

  • 10 Aug 2016 4:17 PM

  In reply to ReyK:

  BOL has some very good information, but it is not written to solve problems. most of this information is in BOL.
  Here is a bit more detail on .CST and .AWST values, from a table in BOL under "Dynamic Reference Parameter"
  
  CST:
  -3 means 'external reference not resolved'
  -2 means 'parameter not configured'
  -1 means 'module not configured'
  0 means 'good'
  1 means 'not communicating'
  
  AWST:
  -4 means 'write rejected'
  -3 means 'external reference not resolved'
  -2 means 'parameter not configured'
  -1 means 'module not configured'
  0 means 'success'
  1 means 'not communicating'
  2 means 'write pending'
  
  AWST = 4 results from an invalid parameter state or out of range. It does not tell you why it failed, but a value of 4 tells you the write got all the way to the parameter before it was rejected, so it's the value and current state of the destination module at issue.

  Andre Dicaire

  • Cancel