invbloom - New command for reduction of bandwidth #1404

Reason

After the version/verack handshake, Bitmessage sends a list of all inventory vectors to the other side. This causes a huge amount of bandwidth wastage, in particular for nodes that accept inbound connections. The proposal is to add a new command invbloom which will reduce this step in synchronisation.

This issue will be updated with more specifics as time goes on.

Data

After the command, there will be some metadata, and data. The bloom filter will use Murmurhash v3, just like Bitcoin's bloom filters. It's implemented in pybloomfiltermmap and it's very fast.

When to use

The support for this command will be indicated by a new bitfield in the version command. If both sides support this bitfield, instead of sending the list of all inventory vectors (referred to as "biginv" in PyBitmessage source code), the two nodes will send an invbloom.

Since a bloom filter allows for false positives, the invbloom command should be issued periodically to reduce the probability that an object goes missing. The delay should use a Poisson probability and pick a random outbound connection (similarly to how Dandelion works).

How to handle

Upon receiving invbloom, the node can send a list of inventory vectors that aren't in the bloom filter. This allows to reuse the rest of the protocol with no change.

Further development

Additional improvements can be added in the future with further protocol changes.

Anonymity

A different bloom filter should be used for each connection to prevent tracking of a node. I did tests with pybloomfilter and it looks like it isn't deterministic, so it probably isn't necessary to add random data to it.