Dear all i am having an issue regarding deltav power up directory. It says that the standby controller power directory files are not present.The standby sx controller cannot be powered up and show that the cmmunication is bad. How to reset my power up directory so that it may have the old standby controller files.
Regards
ASIM
Andre Dicaire
In reply to Andre Dicaire:
I was actually having bad on the redundant controller communication.
I checked the carriers and checked the network cables. Everything was working fine.
Later on i decommission the controller swapped it with the redundant and recommissioned again. I think this is where i made the mistake.
I started getting these errors and both controllers started showing up as separate entities on the diagnostics.More or less it became the same situation as what u have described earlier .
Is there a way out of this, or probably i have to recreate the whole database put the controllers back to their positions and recommission them.
Thanks and regards
SO today i decommissioned and detached the redundant controller from the network and Connected it with new test work station.
The controller gave me a bad whenever i tried to sense it. When i checked the IPs on the node address. They were fluctuating.
On Primary connection i was getting
10.4.0.30
10.5.0.30
and on secondary connection i was getting
10.4.0.26
10.5.0.26
Whenever i created a workstation it showed me
10.4.0.6 for the primary and
10.8.0.6 for the secondary
so i think due to these fluctuations i can not ping my controller, i think that's why its not communicating and giving me a bad.Can u suggest a way that the controller can be put to 10.4 or 10.8 network permanently.
Thanks for the awesome support .
Thanks and Regards
In reply to M_ASIM_B:
ASIM, You need to recheck your post or your installation. The secondary network must be 10.8.x.x and 10.9.x.x. This shows you have two primary nodes. all addresses on the controller need to have the same last two octets, either 0.30 or 0.26.
I suggest you first try establishing the two controllers as two simplex decommissioned controllers. To do this, make sure the controller is decommissioned in the database prior to disconnecting the controllers physically.
Then disconnect the two 2-wide carriers to create two simplex carriers. Remove both controllers and reinstall them to force them through a reset. Make sure all network connections are correct and that you have two separate switches for primary and secondary. You should see two decommissioned controllers and you can flash the lights to confirm the MAC address of each controller.
At this point, I would commission one of the simplex controllers to confirm that it is assigned an IP address and that you can download it.You will than be able to confirm its assigned IP addresses on both primary and secondary, and they should match (10.4.x.y and 10.8.x.y). The controller will come up with a yellow triangle if its IP addresses were successfully assigned, and you will be able to ping these addresses. If this does not work, troubleshoot this simplex node first and find your problem.
Then decommission this controller and commission the other simplex to the same place holder. You should see all the same behaviors and the same IP addresses assigned to this other controller. IF this works, you can move on to a redundant node.
Assuming both controllers commissioned successfully, make sure both nodes are decommissioned in the database. This is important. If the database node has not been decommissioned, the Pro Plus will try to commission that physical controller to the database node whenever it sees it on the network. We want both these simplex controllers to be unassociated with any database nodes. Both simplex controllers should therefore be flashing their LED lights in the decommissioned manner.
Now, remove power from one of these controllers and slide the two carriers together. Once you see this de-energized controller disappear from the decommission list, you can power it back up. You should now see a redundant controller in the decommissioned list, and if you look at its properties, see the same two MAC addresses for the two controllers. You should also see all four IP addresses with the same relative address (10.4.x.y, 10.8.x.y, 10.5.x.y and 10.9.x.y). Now you can commission this redundant node.
Some DeltaV controller commissioning facts:
When a controller is connected to DeltaV it initializes and sends a broadcast to announce itself on the network. This broadcast tells the Pro Plus the controllers MAC address. If the Pro Plus recognizes that address as a commissioned node, it automatically sends the controller its IP address and node name information, and the controller will reset and come up as a commissioned controller, waiting for download. If this happens, the controller will not show up in the Decommissioned list, and its place holder in the database will have a yellow triangle.
If the controller was configured with Cold Restart, it may download itself, either from its local NVM memory or from the Pro plus. In this case, the yellow triangle will disappear from Explorer.
If the MAC address is not known to the Pro Plus, then the node will be listed in the decommissioined node list. it will have a default node name that includes its last three octets of the MAC address.
If the DetlaV networks are crossed, the broadcast of the decommissioned node will still be seen by the Pro Plus, but once the node is commissioned and assigned an IP address, the crossed networks will prevent the node for communicating with the pro plus or any other DeltaV node. This will result in the Red X on DeltaV Explorer. Note that if you commission a new node and don't download the pro plus so that it is aware of the new node, the Red X will also appear in Explorer, so make sure the pro plus setup data is downloaded.
If you commission a simplex controller, the database associates the MAC address of that physical controller with the Node in the database. If you physically remove that controller, the MAC address remains associated with that node and if you re-install that controller in another location, the database will automatically commission it to the configuration of its original place holder. You must first decommission the controller in the database to disassociate the physical controller from the database node. Then you can reuse the controller in another location.
For redundant nodes, the active controller will inform the Pro Plus if its standby nodes has been removed. When you remove the standby, the Pro Plus removes its MAC address from the database, allowing the physical controller to be used elsewhere in the system without decommissioning any node. If you remove both controllers on a redundant pair without decommissioning the database, the database will automatically commission those nodes when they are next seen on the network. So if you are removing physical controllers from a node, be sure to decommission the node to free up the MAC addresses for use else where. If you are only removing a standby, the system will take care of this for you.
If you connect a controller and it does not show up in the decommissioned list, check that the node did not automatically get commissioned. If it did, find that node in the database and decommission it. Then the controller will show up in the decommission list and can be commissioned to the right node in the database. If the controller LED's indicate that it is decommissioned but does not show up in the decommissioned list, check the network connections as this would be the only explanation for not showing up in the pro Plus. Note that the issue could be at the Pro Plus if no other nodes are communicating.
The controller has two network LED's that blink when they see network traffic. If you disconnect power to the secondary switch, you should see the primary network yellow LED continue to flash, but the secondary stay off. If on a redundant node you don't see the same behavior on both the primary and secondary controller, your networks are crossed on one controller, or both if the primary LED goes out when you disconnect power to the secondary switch.
When you power up a standby controller for the first time, the controller goes through reset, but before it broadcasts its MAC address, it checks for the redundancy connection. If it sees it has a redundant pair, it waits for the active controller to tell the Pro Plus about its new partner. The Pro Plus then sends the initialization message with the standby IP addresses to the standby and it resets to come up with its newly assigned IP Addresses.
The Standby and active controllers have the same name, but communication to these nodes is with DeltaV communications and this does not use a name resolution. The two controllers are seen as one entity in DeltaV.
Hop this information is useful.
Thanks alot Andre for the information , facts and guidelines. I tried to work with ur solution but it seems that i am stuck with the step one.
I decommissioned both of the controllers and tried to commission only the one with the fault on a seperate two wide carrier with a variable PS.
The controller couldn't communicate with the database. The node showed in commsioned nodes in my explorere but with a cross.
When i opened the diagnostics, the controller still gave me two fluctuating primary node IP addresses, I don't know why it couldn't pick up
the primary node IP i.e 10.4.x.y . It still showed me a fluctuating IP between 10.4.x.y and 10.5.x.y.
It looks like i have to to send this one for maintenance to Emerson.
Now for redundancy , there is an extra application that comes up with the DeltaV application suite that checks the redundancy of the controllers and generate
reports for the diagnostics.Would that be of any help in this case?. I might give it a try.
Asim from what I understand from your conversation that you arn't in running process so could you try to make rerun for work station config. as it seem that problem in the problus also for hardware part the interface between 2 two wide carrier is lose or not connecting well so make sure from this point.
Regarding the MAC address just I need to add one more point in decommission mode problus communicate with MAC address of the controller butin commsion it start to give IP in some cases the problus will not remove the MAC address of redundant controller from the MAC table and you can see it from cntrl properties and when you try to download it fails so to clear this issue remove old redundant controller and restart proplus then check properties and you will find that MAC address disappered then you can install the new reduant controller.
In reply to Mostafa Mohammed Lakosha:
Dear all,
Good news is that i have a working standby controller with me. I have tested the controller on a blank database and the redundancy is working fine. However when i decommission and commission the controller on a working database i.e to which my all IO modules have been assigned to, i got a bad redundancy.When i see the diagnostics i have BAD IO integrity, but that shouldn't matter because i have no IO modules attached. Moreover even when these are attached , as per my understanding the IO status shouldn't effect the controller redundancy. The controller should keep a redundant connection even with the BAD IO integrity.
Kindly help me in solving this issue.
Asim, The standby controller does communicate with the IO, or attempts to. If it is unable to communicate to any configured IO card, it must assume its ability to do is compromised. If there are no cards configured, then there is no IO communication attempted.
During the IO Scan, the Active controller passes control of the IO bus to the standby and it polls one card to confirm it can talk to the card. On the next pass, it will poll the next card and so on. So if you have 64 cards, the standby will poll each card every 64 scans. This allows the standby to confirm it is able to communicate to the IO and it also obtains a status for all the IO cards it is supposed to communicate with.
One condition that can trigger a switch over is if the Active controller loses communication with the IO cards while the Standby can communicate with the IO, the Standby will force a switch over and assume control. However, if the Standby is unable to talk to any IO card, it will not initiate the switch over based on this criteria. Make sense?
From the information provided, it seems your IO hardware does not match the configuration and the BAD IO message is not unexpected.
To check for redundancy health, verify that you can see Standby is available PAVAIL = True. Also check the Redundancy container in diagnostic and verify you can see the standby controller revision, free memory etc. If you can see all these parameters and PAVAIL is true, the standby is configured and working. BAD IO is the result of missing IO cards.
Speaking of revisions, make sure all the controllers and workstations are flashed to the same version before going any further. If you have different versions of software on the controllers, this could affect how the controllers behave in a redundant pair (i.e. don't behave as expected).
Andre, my main control panel has been shipped and i am checking only the redundancy on a remote work station, so as a matter of fact i don't have any IO hardware attached to my controller.So what if i don't un assign my IO hardware and do the redundancy check.I think my redundancy should keep on working because let's suppose if any working card is taken out of a working redundant IO configuration , the controller redundancy should work (correct me if i am wrong ). For redundancy health i am having the following statuses
PAVAIL=FALSE
SOFTWARE REV=YES
STATUS=FAILED COMPONENT
LastDnldStat=Normal Completion
Before download , Auto sense IO error .
Software and hardware revisions, memory etc and communication are all showing up and good .
I have tested the controllers individually and on redundant configuration on a separate empty IO configuration data base and they are working fine .
Asim, Under normal operation, loss of an IO card does not affect redundancy. Your issue is that during commissioning of the controller, there is no IO present, which means the controller is unable to perform its control functions. Maybe the issue is that the standby cannot go available until it confirms that it is able to communicate with the IO subsystem, which means at least talk to one of the configured cards.
During the AutoSense, what error did you see? I would expect the reconcile dialog would show the database cards configured and that the physical IO slots are empty. Did you see this dialog or was there a different Error message.
If you disable redundancy on this node, can you commission the controllers individually as simplex nodes? Slide the carriers apart to create two decommissioned nodes and try commissioning each to this node. Do they both behave in the same way? Any unexpected errors (other than IO configuration does not match)?
If both controllers work as simplex, and both give the same responses, it'd say they are both good. Tray swapping the 2 wide back planes so that you are using the other redundancy connection. (something to try).
As a redundant Pair, the controller shows up as a redundant node in Decommissioned controllers?
Commission this pair to a test node with some control modules defined, but no IO cards configured?
- Don't auto sense as there are no IO to sense
- Does this controller pair come up with PAVAIL True?
If so, configure a couple of IO cards, enable some channels but don't reference them in modules yet? Download. Does PAVAIL remain TRUE? Check for BAD IO, expecting that it is now true.
Configure references in the modules to the new IO and download. Does PAVAIL remain TRUE?
If so, decommission, and recommission to the same test node. Does the problem observed on your configured node now happen on this test node? What are the differences.
When you check diagnostics, FREMEM is a number, so provide the number you see. Following the initial download, the standby will then be downloaded and the controllers must synchronize. The STANDBY Green LED should be on solid. If it is flashing, the download has not completed or is being rejected for some reason.
Again, contacting the GSC would be prudent to get confirmation the behavior you are seeing is normal or that an issue needs to be investigated by them.