• Not Answered

New Controllers to Lighten the Load

Hi,

Hoping you guys can help me out here!

I'm looking at adding a number of new SX controllers and re-assigning a portion of the I/O to reduce the loading. The thing is, is that there was a sound logical approach taken with the node assignment in the first place. There might be two units that have a reasonable amount of interaction, i.e. there's module references, cross unit aliasing and dynamic binding between unit phases, all currently assigned to the same node, but they need to be split to address the loading issue.

What are the key things to consider before re-assigning the I/O and modules? I'm going through the references and aliases. What else do I need to consider. What's going to catch us out when we come to get things up and running again?

Any help would be welcome!

Thanks

Ajay

9 Replies

  • Are you sing CHARM IO? I'm thinking you are.

    This somewhat depends on what revision of DeltaV you are using. If you are v13.3.1, you have very few concerns. The Redundancy Analyzer identifies very few items that once would show up as items to verify. For instance, if you were reading a remote parameter, you were cautioned to use the ALGO_OPTS setting for the expression. and use the .CST status to explicitly know communication.

    In v13.3.1, the behavior is handled automatically so that on a switchover, remote reads are handled more smoothly, without false trips due to errors in communication.

    The one thing you would look at is avoiding cross controller writes. Writes execute with higher CPU priority and excessive writes can impact control performance. Writes are fine when executed as a sequence, but you want to avoid writing a remote setpoint continuously.

    The analysis tool I think still can identify remote references. You want to make sure these are all Read references for continuous exchange.

    Andre Dicaire

  • In reply to Andre Dicaire:

    Thanks for the reply Andre,

    It's a combination of Foundation Fieldbus, DeviceNet and Ethernet (VIMs) and we're on 13.3.1.

    I have considered the cross controller writes and we don't have anything that's going to come near to the 40 writes per second constraint for the SX controllers, and we're analysing the interaction at a module level, but I'm concerned that commands written in actions at phase level via aliasing, and waiting on confirms across controllers are going to cause issues when batches are running. Are there any rules for changing the syntax in SFC actions if we are reading across controllers?

    Thanks Again!
    Ajay
  • In reply to Ajay:

    When a write is performed locally, it completes immediately into the memory of the controller. when remote, the write goes to the asynchronous write tasks, which passes the value to destination controller within 200 or 250 ms. The write goes into the memory of that parameter and a confirmation is sent to indicate if the write was successful.

    The .AWST field reflects the status of this remote write, indicating the write is pending and then completes either successfully, or fails. A value of 0 is successful.

    Every parameter has the AWST, even if they are local. if local it is always 0. So the syntax for validating successful writes can be the same (complete when xxxx.AWST = 0).

    When you write to a remote parameter for the first time, the controller needs to "find" the location of that parameter's module and set up the proxy server/client objects. The controller knows it is not a local module, so it requests the node of the module from the directory services, contacts the host node and sets up the communications. The host (remote) controller will then start sending the value of the parameter to your controller as unsolicited communications.

    If you've already read or written to this remote module from this controller, the Proxy client is already there. Now you are writing to a new parameter for the first time, the parameter is added to the Proxy client and server that already exist.

    So now the write goes via the Asynchronous Write route, returning the AWST (status) you can use in your Confirm statement.

    You can also read the value to confirm it reaches the desired state. However, since the write can fail due to invalid mode or range, checking the AWST tells you if the write completed, but maybe there is conflicting logic that reset the value. So you still might want to check for the value to be correct, but you know the write completed. If the write failed, that is different than the value not remaining at what you wrote.

    SInce the target path is added to the proxy client and is updated, you can verify this value as a precheck to determine if you need to write to it in the first place.

    Now the next time you write, all that proxy stuff is already setup. The controllers maintain their proxy clients and do not prune them. However, on a switch over of the controller , the new active does not have those proxy clients and they have to be rebuilt.

    Note that it does not matter what module causes a proxy client to be created, all the modules in that same controller use the same proxy client for all data from that remote module. The Proxy Server updates the Proxy client periodically. So having a module that monitors the status of a remote module continuously will cause that module to be immediately mapped into the comm layer on download or following a switchover. Later, when a sequence needs to read or write to that module, the infrastructure will be ready.

    I don't know if this makes a huge difference overall to pre-load the proxy clients, but it keeps things consistent whether it is first time or not you are writing to the parameter.

    Andre Dicaire

  • In reply to Ajay:

    Have you considered changing scan rates of modules?
    - For example if everything is executing at 1 sec you could reduce the loading by decreasing some scan rates and sometimes I've seen this having a significant decrease to loading.

    Some examples:
    Temp AIs to 5 sec (mostly Slow reacting)
    Level/Weight AIs to 5 sec (mostly Slow reacting)
    Discrete Valves to 2 sec (Valve Transition time > 2?)
    etc

    You can also verify that you don't have configuration that is included in modules that is not being used which could be removed to reduce the loading as well.

    I would suggest evaluating these first before adding new hardware.
  • In reply to Andre Dicaire:

    Hi Andre,

    If memory servers, a change was made to how/when the AWST changed when a write was requested in version 9. The change was that the AWST did not change to pending immediately when the command to execute the write was processed. Our confirmation expressions used to just look at the .AWST, but had to be updated to look at the .AWST and that the step time was greater then 1 (maybe 0 - I don't remember), to ensure the .AWST updated before the confirmation expression executed to check it.

    Your description above seemed to imply to me that this isn't necessary anymore. Can you help me understand that part?

    Does the proxy use the same buffer as graphic updates?

    Also the 40 writes per second mentioned above - that is just these writes requiring a proxy - right? not the ones due to external or dynamic references?

    Thanks,
    Brian
  • In reply to Brian Hrankowsky:

    Brian, you are testing my memory. I thought that there was an issue where the Pending status could be missed and this was fixed in or around v9. I would have to do some testing/digging to confirm it. Maybe the issue was introduced and later resolved. I would hope so as the purpose of the Pending status is to simplify the confirm expressions, not complicate them. Your work around makes sense.

    All DeltaV Communications result in the set up of the unsolicited communications of the referenced parameter. Whether the write is in an SFC action or is an external reference. The Write action is handled by the asynchronous write task which allows the module to continue its execution rather than wait for a response, hence Asynchronous write. All writes to remote modules are executed at the priority of the module doing the write, but at the destination, the write action happens at an elevated priority so that the response can be returned. This is why it is recommended to limit such writes to less than 40 per second. the more of these you have per second, the more CPU you consume in the control task. If unchecked, you could starve control CPU from modules. You have to really try hard, but if you have heavily loaded controllers, you don't want to waste that resource.

    Note that the Async write diagnostic tells you how many writes a controller is doing. But it doesn't tell you how many are targeted to any particular controller. If you had 10 controllers each writing 10 values to each other, you would be fine, but if all 10 controllers write 10 values per second to an 11th controller, it is receiving 100 values per second and you are exceeding the limit. Unfortunately there is no "Writes Received" diagnostic. You have to pay attention to this in your configuration.
    Rule 1: Do not wire to an External/Dynamic reference bound to a remote module parameter. Always set up a READ external reference to pull continuous data.
    Rule 2: Check that a value needs to be written before writing so that you only due writes by exception. This applies a lot more to CALC blocks or Actions Blocks that are executing continuously. In an SFC, where the Action will only fire once, get confirmed and move on, the SFC limits the write frequency inherently.

    DeltaV communications is point to point, or unicast. Each node that requires data will have a dedicated Proxy SErver for each Proxy Client. This allows the update rate to each client to be set accordingly. So if you have a module sending data to 5 consoles, that module will have 5 Proxy servers in the controller, and one proxy client in each console, and each Proxy server will send only the data required by the console. OVer time, the consoles will "Prune" parameters and even modules (Proxy Client per module) as data is no longer needed. The device connection stays, serving diagnostics and monitoring Communication status.
    If this module is written to by a module in another controller, a Proxy server will be setup and that parameter will be added and sent to that controller so the value can be available for verification.

    So yes, a proxy server is set up to handle communication to a client node.

    Now I'm curious and will have to check out the Pending status of the AWST....

    Andre Dicaire

  • In reply to Brian Hrankowsky:

    I had a conversation with Product Engineering 6 months ago or more on this very subject, the person in PE indicated that they thought this was also the case as well but I didn't recall this ever being an issue and I was challenging this behavior documented in KBA AP-0900-0113. We have always just used AWST = 0 with no added delays and have never seen a problem with this implementation for many years prior to v9 and since using the same method. PE did a lot of testing and couldn't confirm this behavior was the case (1/2 billion writes with a few recommended limits being exceeded WITHOUT a single write failure). I have never seen this delay required but maybe this was an issue for a small period of time and hotfixed? You don't have to have any added delay to the confirm of cross controller parameter writes using AWST = 0 and PE has updated the KBA AP-0900-0113 to remove the delay requirement.
  • In reply to Matt Stoner:

    Thank you very much for the info. I looked at the new version of AP-0900-0113, but its not clear what was changed (I don't have an old copy). DO you know what it said previously? I can't seem to find any documentation that explains how and when the AWST is updated. Can you point me to something? I will be passing this thread along to a couple people in my area.
  • In reply to Brian Hrankowsky:

    "When using an AWST (Asynchronous Write Status) in a Pulse Action confirm expression or a transition expression, the PARAMETER.AWST status value will initially be “0” (Success) before the pulse write becomes active. A time delay of 1 second is required between writing to the PARAMETER.CV and checking the PARAMETER.AWST; the .AWST status should cycle from “0” (Success) to “2” (Write Pending) and back to “0” (Success) in approximately 0.5 second for a successful write."

    This 'documentation' has been removed from the KBA but unsure if the above was exactly the same wording in the KBA or not but it was very similar to it.