• Not Answered

PCSD DC_CTRL Block Throwing "SP Write Error : REQ SP Failure" Error

Hello all, apologies for the direct technical support ask here, but I've dug into this one pretty deep and am totally at a loss.

About a month ago, we had a valve (PCSD V5.2) generate a "WA_ALM" with the Alarm message "SP Write Error : REQ SP Failure". This caused the parent EM to receive the failure, and therefore hold the phase and ultimately vent our tank.

Now, digging into this, the perplexing part is that in order to get this specific alarm, DC_CTRL/SP1/OUT2 would need to have a value of 1.

Not to get too lost in the details (though I'm happy to if anybody wants more information), OUT2 is an internal parameter to this calc block, so it cannot be overwritten from external logic. And within the calc expression, there are 8 lines of code that could possibly set OUT2 to a value of 1. But meticulously going through the execution line-by-line, there is always some condition that would explicitly prevent those lines from executing in the case we observed. Genuinely, it appears impossible that we could have seen this alarm, and yet - we did.

Has anybody seen anything like this, whether related to to this specific error, or otherwise a situation where a parameter was set to a transient, seemingly impossible value?

A couple pictures below. Again, apologies for the long-winded message but I'm running out of leads.

(If anyone from Emerson sees this, GC-2505-6946)

4 Replies

  • First OUT2 being an internal variable can't be updated is not entirely correct, at the beginning of the execution of the CALC block it copies the current values of INs and OUTs parameter to the internal variables. So this means another block could update the OUTx parameter and the next execution of that calc block the internal variable will be update to what value. Now I don't think another block is doing this...

    REQ SP Failure is generated when logic has written to the REQ_SP parameter and it's not a valid SP which are (note: this is from memory of the logic and there might be more or slightly different than I recall):

    1. Different than current SP and Target Mode is <> CAS
    2. Interlocked and REQ_SP different than current position 

    I tried to look up the call but didn't find that call number, is there a typo?

  • In reply to Matt Stoner:

    Hey Matt, thanks for the response.

    You're correct - the variables are loaded in, but the first line of the calc block logic sets

    OUT2 := 0;

    So we always know we're starting from a known value every scan.

    For interlocks and permissives, we have no permissives configured on this particular block. There are two interlock conditions, neither of which was active at the time of the event.

    Additionally, the valve mode remained in CAS throughout the duration of the event - which is important, as we're using Force SPs heavily here to manipulate this block valve based on a PID controller output. And on that note we were able to replicate a Force SP error, but that is distinct from the "REQ SP Failure" we saw here.

    There are a number of other expressions that set OUT2 in the DC_CTRL block. A handful can only set the value to multiples of 2, or 5: never 1.

    And then there are a few which rely on a parameter called 'REQ_SP_TRK', which in our case as far as I can tell is set statically to 'TRUE', and causes the remaining lines to skip execution.

    In the GC ticket there is a PDF I prepared that goes into more detail - and it appears to still be open when viewed in Guradian (screenshot below).

  • In reply to Michael Guynn:

    Ok I found the call....How do you know that the interlock condition(s) weren't true?

    What are the ERROR_OPT of the interlock condition blocks set to?

    I'm pretty sure that the system had the interlock evaluated as true for some reason (actual condition, condition read error, switchover, etc) and SP was giving to another state causing this issue.

    Was there anything in the event log around this time like network errors, switchover, etc?
  • In reply to Matt Stoner:

    With respect to the interlocks, it is possible there was a transient condition - my evidence for no interlocks active is two things:
    1. Manually checked the interlock conditions (pressure PV vs limit at the time, burst disc status, etc.) and no condition appeared to be present.
    2. We have a second alarm which triggers when parameter 'INTERLOCK/INTERLOCK' is true, and this alarm was not observed at the same time as the WA_ALM.

    INTERLOCK/INTERLOCK (output) is wired to DC_CTRL/INTERLOCK (input) - this is then one of the variables initialized in DC_CTRL. These blocks' execution orders are #2 and #5, respectively, so I'm struggling to imagine a scenario where DC_CTRL gets the interlock signal for a scan without triggering the module-level interlock alarm.

    ERROR_OPT on the interlock Condition blocks is set to "UseLast".


    As far as event logs, for at least the immediate time before the event, I'm not finding anything suspicious. Typical things like logins, prompt responses, unrelated module alarms. The only system or network-related alarm I see is a "BAD INTEGRITY" node status on the controller running this valve, but immediately before that there is a MODBAD event for a temperature probe, so I suspect that's what caused it. Other than that, no unusual conditions.