Emerson Exchange 365

• Not Answered

DeltaV Live Database Issues

For starters this is a brand new site installation using DV 14.3.1 and Live.  We first noticed issues while trying to access and make changed using DV Live Administration->Workstation Management from an operator workstation.  Specifically when logged in using an administrator account that is different than what was used for the original VM creation we receive an error message stating “You do not have permission to access Workstation Management.  Please contact the system administrator.”.  Looking at the workstation list in DV Live Admin->Workstation Management we noticed that only the initially created, DVS VM Template, workstations are present when opened from an operator station (ProPlus and 5 operator stations created during the initial installation).  This same issue was noted at the ProPlus, however after re-running workstation configuration on the ProPlus all 18 workstations now appear in the Workstation management list and multiple “admin” user accounts work on the ProPlus without the permission error stated above.  Similarly running workstation configuration on the operator workstations did not correct the issues as it did for the ProPlus.

 Digging into this further we noted the SQL Server (“WORKSTATION NAME”\DELTAV_INSTANCE) security->logins doesn’t contain any domain groups, only the typical service accounts, and the original admin account used to create the workstation.  The ProPlus SQL Server, after re-running workstation configuration, has both the original admin account, and several domain groups.  After additional evaluation it was noted that the other 13 workstations that are not visible in Workstation Management (other than from the ProPlus) also do not have an “ADDBRuntime” despite being setup from the ProPlus as DV Live enabled, and having the correct licensing.  There appears to be an issue preventing the database from being created however in DeltaV Live Diagnostics the overall integrity is displayed as “Good” for all workstations.  The 13 workstations not showing up were created using Emerson templates, with some created by us and some created by our local impact partner, all exhibiting the same issue.

Another thing we noted is there appears to be some inconsistencies in the SQL server name.  If you look within the SQL securities->logins->[admin account]->securables it references the the server as "1431SV16\DELTAV_INSTANCE" on the ProPlus rather than by "WORKSTATION NAME"\DELTAV_INSTANCE.  Similarly it on the operator stations it shows the Windows 10 template name vs. the workstation name format.  We thought this could affect the synchronization, however our local SQL database admin doesn't believe this is a problem since Emerson is using SQL Server Express which apparently doesn't have native replication functionality.  This leads us to believe the synchronization / replication is an Emerson devised approach, but we are unclear how the database synchronization actually works.

 I should note we have an open ticket with Guardian Support, but we are looking for some outside help as we haven’t reached a solution yet.  We are getting down to crunch time for rolling out our new system and really hope one of the resident experts on Emerson Exchange can assist.

-Jesse

Jesse Delanoy

• 26 Aug 2021 12:54 AM
• Cancel

3 Replies

• Jesse Delanoy

  

  • 11 Sep 2021 6:01 PM

  Update to this issue. We are still working with Guardian Support and have not resolved the problem. There is definitely a database sync issue. As mentioned above all of our new nodes do not have a SQL ADDBRuntime database (no database in SQL and no db or other files installed in the d:\DeltaV\ProgramData\Emerson\Databases\ADDBRuntime\ folder). We found that if we re-run workstation configuration on one of the original 5 nodes that had an ADDBRuntime database and changed a registry setting as directed by Emerson to force rebuilding the ADDBRuntime database that this results in the database being deleted and not rebuilt. Basically it ended up the same as creating a brand new node where there is no ADDBRuntime database. We also found that if we shut off the ADDB services on any given node it will connect to the ProPlus database and allow us to open DeltaV Live and DV Live utilities.
  
  As a separate test we created a temporary graphic from one of the 4 remaining nodes that has an ADDBRuntime database, and published the graphic. The Graphic was written to the ProPlus database, but not replicated to any of the four nodes with an ADDBRuntime db. The only way to see the new graphic on any node is to shut off the ADDB services, forcing the node to look at the ProPlus database. Which further confirms we have a db sync issue.
  
  We even when as far as removing our antivirus software from the ProPlus and several of our engineering workstations with the thought that maybe it was causing an issue, even though we are using the Emerson recommended configuration. This did not resolve any of our issues.
  
  Any thoughts or ideas on what we can test to help identify the root cause would be appreciated. We are at the point we will most likely rebuild our ProPlus from the ground up to see if it resolves the issue. Given that all nodes are not syncing it seems likely the root cause is related to the ProPlus source. I should note we are using DV 14.3.1, with hotfix WS_21 installed.

  • Cancel
• Andre Dicaire

  

  • 13 Sep 2021 2:56 PM

  In reply to Jesse Delanoy:

  Troubleshooting the new Live architecture is especially challenging as there is little information available on the structure of this software and the normal functionality of the various components. For instance you explained that you are able to add a new display from one of these workstations and it appears in the Pro Plus, but not the other stations. Adding or editing a display is done via the Graphics Studio client that connects to the Pro Plus and the database resides in the Proplus DB configuration database.
  
  To view the display it has to be published, at which point the HTML version of the display is made available. This mechanism flags the appropriate workstations that a new version of the display is available and the workstations pull this from the pro plus. This is different than Auto Update which would push files to workstations. If the displays are on the Pro Plus and can be published and viewed on the pro plus, and can be opened and edited on other workstations with Graphics Studio, the issue is with the workstations pulling them to update their Live session.
  
  On my system, my physical workstation with Live has the AddbRuntime Database, which I have to assume is a SQL database that is managing the runtime HTML display data that is published by the Pro Plus. The Pro Plus has multiple databases including the AddbRuntime, but also the ConfigDB, AddbMaster and AddbConfig databases.
  
  The first question I would be asking is why do the workstations not have their AddbRuntime database? Since all were created form VM Templates, it is not surprising they all have the same issue. But why?
  
  I would suggest doing an uninstall and reinstall of DeltaV on one of these machines. After uninstalling, have the computer leave the Domain and move to a workgroup. During the DeltaV Workstation Config, it will rejoin the Domain. I had some issues with one App Station when installing Hotfix WS_19 or WS20, where the DeltaV services did not start. We resolved by rerunning Workstation Configuration, after first leaving the Domain. We also had to run Regall/ register after for some reason. Point is, redo the install on one station to remove the install created by the VM Template. If this resolves the issue on this station, you know it's not the Pro Plus, and you have a path forward. If it does not resolve the issue, it may just be the pro plus. Knowing the issue is Works station install related or Proplus based would be a good datapoint.

  Andre Dicaire

  • Cancel
• Jesse Delanoy

  

  • 15 Sep 2021 3:50 PM

  In reply to Andre Dicaire:

  Andre,
  
  Thanks for the recommendations. We have two physical workstations that were not build from a template that are exhibiting the same issues as our VM templates, so we didn't go down the path of taking a VM back to the base operating system and reinstalling DeltaV. We did try your suggestion to remove a VM from the domain and re-run workstation configuration, along with running the regall /register, unfortunately this did not correct the database replication issue.
  
  In our conversations with Guardian Support we learned that replication issues with a single node can impact the entire system. In Live administration we disabled Live on all nodes expect one engineering workstation, and the ProPlus (which can't be disabled anyway). The function of the Live Admin disable is a little unclear as to whether or not it actually disables replication or just changes the registry and flexlock to prevent launching live from the workstation. Not fully understanding the Live Admin disable, we went and shutdown the ADDB config services on all of the nodes we wanted to disable Live on. This didn't correct our replication issues. We had hoped that if this worked we could, by process of elimination, identify a specific workstation that was causing the replication issues.
  
  Interestingly one of the request from Guardian Support was creating a new Live database and switching it to active. We tested this late yesterday and it appears to have corrected the replication issues. We only have Live enabled on a few engineering workstations at the moment, and haven't validated this addressed the issue for all workstations. That said it is promising as we are seeing newly published graphics on the two workstations with Live enabled. We are also seeing the complete node list in Live Workstation Management on the engineering stations as well, where previously only the ProPlus could see all nodes. We are still working with Guardian to understand why the original database stopped replicating and why creating a new db and setting it to active re-started replication. If we learn any additional details I will follow up with another reply to pass the information on.
  
  - Jesse

  • Cancel