Forge on multiple nodes

If I’m correct with SDK 3.0 it’s not possible to enable forging on multiple nodes for a single delegate, you must disable forging on all node and then enable it on a single one.

My question is in case a node goes down (power failure, network partition, app crash, hardware issue), how can we disable the forging to enable it on another node ?

Enabling forging on multiple nodes is possible, but it would violate the protocol by producing two identical blocks in the same slot (double forging). If your forging node goes down, it can’t forge (forging is disabled on restart by default). So you only have to enable forging on a different node to continue forging. You would have to have a script that does it for you.

@przemer are you sure you don’t need to inform the network your node is no longer forging ? I’ve read somewhere you must disable on all node before enabling on a new one. Therefore my question.

You don’t have to inform the network. Disabling forging on all nodes before enabling on a new one is a good practice to avoid double forging.

Exactly what @przemer said.

It is to prevent double forging or as seen in the link to prevent case 3 from happening to your delegate.

Alright so false alarm, no issue here. Thank you guys :slight_smile:

Obviously failover scripts are still necessary. In my solution I first always disabled forging on delayed node before enabling it on another. Is there now any period that should be considered as safe value between enabling forging on different server? Is status that forging has been disables enough to prevent protocol violation?

1 Like

The full technical details can be found in LIP0014:

1 Like

It is not enough that the other node does not forge any more. Assume you want to change from node1 to node2. Then you need to basically do the following:

  1. Deactivate forging on node1.
  2. Get last block A forged by node1.
  3. Use last block A to initialize the forging on node2 correctly.

The last step ensures that node2 is on the correct chain and does not cast contradicting consensus votes that could lead to a protocol violation. See the links posted by @michielmulders for details.

Note that Lisk SDK 3.0 uses a new improved consensus algorithm which avoids that nodes do not forge because of low Broadhash consensus (actually there is no Broadhash any more). Therefore, it should not be necessary to have multiple forging servers to guarantee high availability.

If you are worried about DoS attacks, then a sentry architecture good be a good solution:

  • 1 server which forges block, but which keeps it IP address private. This can be achieved by only connecting to trusted fixed peers and not advertise your IP, see LIP 0004 for details.
  • Multiple servers that do not forge and run default p2p configuration with whitelisting the forging server. This means that the IP address of these servers is broadcast as part of the normal P2P discovery in lisk-p2p. The private forging server only connects these trusted public nodes.

This setup ensures that the IP of the forging server is never known to anyone in the Lisk P2P network except for the trusted public nodes.