You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Apart from the mention solution which is the best solution, to have updated information regarding the whole network the killed node should update his peer list by copying the list of the first node he reconnects to. Therefore, there is no need to ask the bootstrap nodes.
Go-ethereum uses levelDB to store data of nodes in the node's memory. The database uses the system-defined ID of a node, IP of the node, and some other properties of a node as a key and the last ping and the last pong as a value. We can use the same structure for peer list management. The attributes of a node are address and port which can be a key and the last ping could be a good value option.
@sgkim126 Could you review this design?
There are some points we can discuss.
Where to save peer information: DB or a file.
I agree with @MSNTCS. Saving the information in the DB is simple.
Whether to save ping response time or not.
I like the idea of saving the ping response time. It seems that we don't have a ping request in our p2p code. Should we add a PingExtension? Adding the PingExtension could be included in a minor update.
@majecty I think we can do it with the current implementation by storing additional data in Established state. But, in the case of failure of one node, there is no way to update the DB. Which one do you prefer?
@MSNTCS CodeChain is removing the Routing table entry when the TCP connection is closed. What do you think about using the "TCP connection closed" event?
When a TCP connection is closed, deregister_stream function is called. You can see the function in the link below: CodeChain-io/foundry#11
@majecty I think using the current structure is more efficient than adding a new function which costly leads to change in other parts. If we can update DB as soon as the event happens, this is a good idea to use deregister_stream. No need for pingExtension.
Using db to store the peer list seems reasonable to me too.
We don't need ping/pong, because we use TCP; TCP will eventually time out and close.
Storing the last message time(whether sending or receiving) would be useful, but I'm not sure about it.
The most common convention of the p2p network is to prefer the oldest connection that is not dead because there are many short-lived nodes in the p2p world. So most protocols assume that the node who lived longer would live longer. I think this assumption is better than an assumption that a recently used node will live longer.
The idea is to save the peer list in the client database by passing the client database (client.get_kvdb()) to RoutingTable structure as another field.
Then, in order to maintain the database (INSERT and DELETE), first, we consider another column in db.rs as COL_PEER.
Afterward, we need to INSERT in the database whenever two nodes get connected that happens when handler.connect() is called. This eventually calls routing_table.local_public(SocketAddr). Therefore, we should insert in the database in routing_table.local_public(SocketAddr).
For DELETE, we need to watch out for TCP connection close, which eventually calls routing_table.remove(&SocketAddr).
Problem: If we want to pass COL_PEER we would have a dependency loop. For now, I pass COL_PEER as a constant variable from run_node.rs to routing_table.rs.
@sgkim126COL_STATE, COL_HEADERS, etc. are defined in the core crate. To use DB in the network module, we should import the core crate. It makes a circular dependency. What do you think about moving the COL_XXs in the types crate?
@majecty
I'm not sure whether it's appropriate to move or not.
How about changing the network crate to provide a trait for storing addresses and making the core crate implementing it?
To reduce the burden of bootstrap nodes, codecahin needs to save the received
peer list in the disk. In order to save the peer list of a node in the disk, a
procedure is designed a database to store nodes that are connected to the node.
The solution introduces PeerDb to update disk per 10 times to reduce disk usage in the boot time. It also makes the db store all peers in the same field. The constant "peer-list" is used
for the key. In addition, there is no dedicated column for peerDb. The
addresses will be stored in the extra column.
Activity
[-]Save peer list in the disk[/-][+]Save the peer list on the disk[/+]MSNTCS commentedon Dec 16, 2019
Apart from the mention solution which is the best solution, to have updated information regarding the whole network the killed node should update his peer list by copying the list of the first node he reconnects to. Therefore, there is no need to ask the bootstrap nodes.
MSNTCS commentedon Dec 27, 2019
Go-ethereum uses levelDB to store data of nodes in the node's memory. The database uses the system-defined ID of a node, IP of the node, and some other properties of a node as a key and the last ping and the last pong as a value. We can use the same structure for peer list management. The attributes of a node are address and port which can be a key and the last ping could be a good value option.
majecty commentedon Dec 27, 2019
@sgkim126 Could you review this design?
There are some points we can discuss.
Where to save peer information: DB or a file.
I agree with @MSNTCS. Saving the information in the DB is simple.
Whether to save ping response time or not.
I like the idea of saving the ping response time. It seems that we don't have a ping request in our p2p code. Should we add a
PingExtension? Adding thePingExtensioncould be included in a minor update.MSNTCS commentedon Dec 27, 2019
@majecty I think we can do it with the current implementation by storing additional data in Established state. But, in the case of failure of one node, there is no way to update the DB. Which one do you prefer?
majecty commentedon Dec 27, 2019
@MSNTCS CodeChain is removing the Routing table entry when the TCP connection is closed. What do you think about using the "TCP connection closed" event?
When a TCP connection is closed,
deregister_streamfunction is called. You can see the function in the link below:CodeChain-io/foundry#11
MSNTCS commentedon Dec 27, 2019
@majecty I think using the current structure is more efficient than adding a new function which costly leads to change in other parts. If we can update DB as soon as the event happens, this is a good idea to use
deregister_stream. No need forpingExtension.majecty commentedon Dec 27, 2019
@sgkim126 ping
sgkim126 commentedon Dec 27, 2019
Using db to store the peer list seems reasonable to me too.
We don't need ping/pong, because we use TCP; TCP will eventually time out and close.
Storing the last message time(whether sending or receiving) would be useful, but I'm not sure about it.
The most common convention of the p2p network is to prefer the oldest connection that is not dead because there are many short-lived nodes in the p2p world. So most protocols assume that the node who lived longer would live longer. I think this assumption is better than an assumption that a recently used node will live longer.
MSNTCS commentedon Dec 30, 2019
Status up until this point:
The idea is to save the peer list in the client database by passing the client database (
client.get_kvdb()) toRoutingTablestructure as another field.Then, in order to maintain the database (INSERT and DELETE), first, we consider another column in
db.rsasCOL_PEER.Afterward, we need to INSERT in the database whenever two nodes get connected that happens when
handler.connect()is called. This eventually callsrouting_table.local_public(SocketAddr). Therefore, we should insert in the database inrouting_table.local_public(SocketAddr).For DELETE, we need to watch out for TCP connection close, which eventually calls
routing_table.remove(&SocketAddr).Problem: If we want to pass
COL_PEERwe would have a dependency loop. For now, I passCOL_PEERas a constant variable fromrun_node.rstorouting_table.rs.majecty commentedon Dec 30, 2019
@sgkim126
COL_STATE,COL_HEADERS, etc. are defined in the core crate. To use DB in thenetworkmodule, we should import thecorecrate. It makes a circular dependency. What do you think about moving theCOL_XXs in thetypescrate?sgkim126 commentedon Dec 30, 2019
@majecty
I'm not sure whether it's appropriate to move or not.
How about changing the
networkcrate to provide a trait for storing addresses and making thecorecrate implementing it?MSNTCS commentedon Feb 6, 2020
To reduce the burden of bootstrap nodes, codecahin needs to save the received
peer list in the disk. In order to save the peer list of a node in the disk, a
procedure is designed a database to store nodes that are connected to the node.
The solution introduces
PeerDbto update disk per 10 times to reduce disk usage in the boot time. It also makes the db store all peers in the same field. The constant "peer-list" is usedfor the key. In addition, there is no dedicated column for
peerDb. Theaddresses will be stored in the
extra column.