Emerson Exchange 365

PHV Trends Very Slowly

Anyone else having issues with trending behaving very slowly (1-4 minutes to open a trend, go back in time, expand the time, etc.)? I have three plants (2 13.3.1, 1 14.3.1) and they all experience this issue. Seems to have started on the 13.3.1 plants after a hotfix (not sure which one). Unchecking View/Display Events in PHV usually speeds it back up (trends pull up in a second or two, same for adjustments). Restarting the ProPlus (Event Chronicle) also resolves the issue temporarily (a few days). 

I've also notice that when the issue occurs, opening the Event Chronicle Administration tool takes several minutes rather than the usual few seconds. Once open I don't see anything unusual. DeltaV Diagnostics also doesn't indicate any issues.

Have Hotix WS_28_CSS installed on the 13.3.1 plants and WS_08_CSS on 14.3.1.

JoshC

• 1 Jul 2020 1:04 PM
• Cancel

22 Replies

• Cedric Dawnhawk

  

  • 1 Jul 2020 1:50 PM

  Do you have a third (LAN) network ?

  • Cancel
• JoshC

  

  • 1 Jul 2020 3:07 PM

  In reply to Cedric Dawnhawk:

  Yes, and and on the 14.3.1 plant a fourth.

  • Cancel
• Lun.Raznik

  

  • 1 Jul 2020 3:38 PM

  In reply to JoshC:

  This can ip/name resolution issue and/or binding/metric priority.
  
  Can you share the output of the following:
  
  ping -a 10.4.0.6
  
  ping %userdomain%
  
  netsh interface ipv4 show interfaces

  • Cancel
• JoshC

  

  • 1 Jul 2020 4:03 PM

  In reply to Lun.Raznik:

  All pings came back with response times of less than 1 ms. This issue is also experienced on the ProPlus which I wouldn't think would have any issues resolving or communicating to the Event Chronicle since it is the Event Chronicler.
  
  %userdomain% resolved to 10.8.0.42 which is the Application Station and also a domain controller.
  
  Idx Met MTU State Name
  --- ---------- ---------- ------------ ---------------------------
  1 75 4294967295 connected Loopback Pseudo-Interface 1
  4 5 1500 connected REMOTE
  8 3 1500 disconnected Plant LAN
  10 4 1500 connected DeltaV Primary
  5 2 1500 connected DeltaV Secondary

  • Cancel
• Cedric Dawnhawk

  

  • 1 Jul 2020 4:04 PM

  In reply to JoshC:

  This sounds exactly like a problem I had with a 13.3.1 installation. I contacted Global Support and looked through routing, binding order, name resolution etc with no joy. In the end Global Support suggested installing a hosts file on each workstation which I was a bit dubious about. However it worked like a dream.
  
  The issue seems to be that PHV has trouble resolving the name of the events server which is why it affects charts with events. The hosts file contains the name and IP address of each workstation that is an event server and needs to be copied to C:\Windows\System32\drivers\etc\hosts The contents of the hosts file for our simple system is as follows, where WS1 and WS4 are the workstations that are event servers.
  
  # Trial host file to eliminate events delay
  10.8.0.6 WS1
  10.8.0.58 WS4
  10.4.0.6 WS1
  10.4.0.58 WS4

  • Cancel
• Lun.Raznik

  

  • 1 Jul 2020 4:22 PM

  In reply to Cedric Dawnhawk:

  Note that default DeltaV installation does not resolve names on Plant/3rd NIC. So, investigate if your DNS configuration. I would engage with local IT who is familiar with DNS.
  
  I would also configure reverse DNS - this is covered in a KBA to help with resolution.
  
  Also, check for stale records. Check that the IP/name are correct in the DNS entries.
  
  The hosts file help becuase it bypasses the multiple mechanisms that Windows tries to located the node. In the example above, if 10.8.0.0 network goes down, then PHV will not use the other network. And this affects not juse PHV, any application that uses TCP/IP will stop working when 10.8.0.0 goes down versus it looking for alternative route.

  • Cancel
• Lun.Raznik

  

  • 1 Jul 2020 4:25 PM

  In reply to JoshC:

  The network metric assignment looks OK to me. I would check for stale records in DNS and also try to define reverse DNS. I don't have the KBA handy at this point.
  
  Can you try changing PHV so it connects to Continuous Historian using IP?

  • Cancel
• JoshC

  

  • 1 Jul 2020 4:30 PM

  In reply to Cedric Dawnhawk:

  Thanks Cedric. I'll give it a try and will follow-up in a couple of weeks to let you know if you it seemed to work or did not work.

  • Cancel
• Lun.Raznik

  

  • 1 Jul 2020 4:37 PM

  In reply to JoshC:

  Here is the KBA - NK-2000-0327. I would also check with GSC for assistance.

  • Cancel
• JoshC

  

  • 1 Jul 2020 4:40 PM

  In reply to Lun.Raznik:

  The DeltaV Domain Servers are only hosting/assigning on the DeltaV Primary and Secondary. Devices on the Plant and Remote LAN are excluded and have static IP addresses. I don't have any stale records. I'll be surprised if the Hosts file works since restarting the ProPlus makes it work for days before it breaks again. Makes me think it has something to do with the Event Chronicle itself and not the network.

  • Cancel
• kathrynpate

  

  • 1 Jul 2020 4:41 PM

  In reply to Lun.Raznik:

  We had a similar problem when we migrated to 14.3.1 (also installed new workstations/servers). Setting up the reverse DNS corrected the issue for us. Thank you for sharing the KBA:)

  Kathy Pate

  Toray Carbon Fibers America

  Decatur, AL 35601

  kathy.pate@toraycma.com

  • Cancel
• JoshC

  

  • 1 Jul 2020 4:44 PM

  In reply to Lun.Raznik:

  I'll wait until it acts up again and try an IP address. I guess it would make sense to fix it for the Event Chronicle since that seems to be causing the issue.

  • Cancel
• JoshC

  

  • 1 Jul 2020 4:53 PM

  In reply to Lun.Raznik:

  Thanks. I'll try the reverse DNS instructions.

  • Cancel
• JoshC

  

  • 16 Jul 2020 6:58 PM

  In reply to JoshC:

  I implemented the reverse DNS and the Host file on one of the 13.3.1 systems and the slow trending returned today. Restarting the ProPlus resolved it but I expect it to return again. I'll open up a ticket with GSC and let you know if I discover anything.

  • Cancel
• JoshC

  

  • 1 Mar 2021 5:06 PM

  In reply to JoshC:

  On the 14.LTS system, installing Hotfix WS_14 resolved this issue. Not yet sure about the 13.3.1 systems.

  • Cancel