Intermittent drops in Controller Expansion and Medium Priority Modules on Time


Over the past fews week we have been getting a substantial amount of heartbeat alarms across our multiple serial card interfacing. But on the DeltaV side there is one that is more worrying than the others. We are noticing a massive drop in Medium Priority modules on time, caused from a simultaneous drop in CPU performance, seen in the control expansion parameter. This is happening on a controller that has all of the modules that come from that interface, the serial card is under the same controller. My theory is this interface is being lost and the sudden burst of IO Input Failures (more alarms happen in 1 minute than the entire previous day/week), which would be happening on almost every control module in this controller, is causing this drop in Controller performance, not that the drop in controller performance is causing the loss in communication. 

This one controller is substantially more heavily loaded than the other cards, not necessarily from a module or Control expansion perspective, but because its interface is multiple times larger than the other 3. Has anyone seen these drop in performance from intermittent comms losses on Serial, or even VIM cards? We have done a redundancy switchover and seen the same behavior on the backup controller, I find it unlikely both cards are bad in the exact same way.

Ultimately, I don't believe it's a true DeltaV problem, but I would like to know why the DeltaV side problems are happening.