add `addpermissionflags` RPC (rpc, p2p)

https://github.com/bitcoin/bitcoin/pull/26441

Host: brunoerg  -  PR author: brunoerg

The PR branch HEAD was 153e86e at the time of this review club meeting.

Notes

  • By default, a Bitcoin Core node allows up to 125 connections (8 of which are outbound) to other peers.

  • There are different kinds of connections, such as block-relay-only, inbound, outbound-full-relay, manual, feeler, block-relay and addr-fetch.

  • You can use the getpeerinfo RPC to get data about each connected network peer such as connection type, permissions and other information.

  • -whitelist is a startup option that allows to add permission flags to the peers connecting from the given IP address or CIDR-notated network. It uses the same permissions as -whitebind: bloomfilter, noban, forcerelay, relay, mempool, download, addr).

  • This PR builds on the work done in #17167. #17167 proposed to change -whitelist to allow whitelisting outgoing connections, and this PR adds the addpermissionflags RPC to it.

Questions

  1. Did you review the PR? Concept ACK, approach ACK, tested ACK, or NACK? What was your review approach?

  2. What does this PR do?

  3. -whitelist only allows to add permission flags to inbound peers. Why only for inbound ones? Does it make sense to extend the permissions to outbound peers? Why?

  4. Considering we already have the -whitelist startup option, why would an RPC be useful? What do we want to avoid?

  5. This PR adds a ConnectionDirection parameter in TryParsePermissionFlags to control whether it will apply the permissions to inbound or outbound connections. In netbase.h,ConnectionDirection has 2 operators overloading. Could you explain how Operator Overloading in C++ works and how it has been used in ConnectionDirection?

  6. ConnectionDirection can be Both, In, Out or None. What happens in TryParsePermissionFlags if it is None? In which scenarios can this happen?

  7. In the addpermissionflags RPC we receive an array of permission flags and the IP (or network). However, we convert it to a string of the following format: “<[permissions@]IP address or network>”. Why?

  8. (Bonus) How could this PR avoid the “problem” presented in #26970?

Meeting Log

  117:00 <brunoerg> #startmeeting
  217:00 <LarryRuane> hi!
  317:00 <brunoerg> hi, everyone!
  417:00 <brunoerg> feel free to say hi! :)
  517:00 <codo> hi
  617:00 <d33r_gee> hello
  717:00 <svav> Hi
  817:00 <hernanmarino> Hi Bruno and everyone
  917:00 <brunoerg> anyone here for the first time?
 1017:01 <effexzi> Hi every1
 1117:01 <pablomartin> hello!
 1217:01 <brunoerg> Today we’re gonna see: https://bitcoincore.reviews/26441
 1317:02 <brunoerg> We will discuss the questions but feel free to ask/discuss anything else during it.
 1417:02 <glozow> hi
 1517:02 <dzxzg> hi
 1617:03 <brunoerg> hi, @glozow and @dzxzg
 1717:03 <brunoerg> so, let's begin!
 1817:03 <brunoerg> Did you review the PR? Concept ACK, approach ACK, tested ACK, or NACK? What was your review approach?
 1917:04 <LarryRuane> I just started reviewing, did not get to the point of understanding even basic concepts
 2017:04 <svav> I read the notes
 2117:04 <brunoerg> Nice!
 2217:04 <brunoerg> So, what does this PR do?
 2317:05 <hernanmarino> read the code, approach ACK . I have a couple of doubts regarding the questions, that I'm hoping to clear on this chat :)
 2417:05 <svav> It adds whitelisting for outbound connections
 2517:05 <hernanmarino> It implements a new RPC command to whitelist connections
 2617:05 <brunoerg> feel free to ask anything, hernanmarino!
 2717:06 <hernanmarino> brunoerg: will do later when we get there :)
 2817:06 <brunoerg> svav and hernanmarino you're right, but I'd say this PR does both!
 2917:06 <roze_paul> hi
 3017:06 <hernanmarino> yes
 3117:07 <brunoerg> ok, so this PR adds whitelisting for outbound connections as well as implements a new RPC command for it
 3217:07 <brunoerg> let's go to the question n3 to understand some motivations
 3317:07 <brunoerg> -whitelist only allows to add permission flags to inbound peers. Why only for inbound ones? Does it make sense to extend the permissions to outbound peers? Why?
 3417:08 <hernanmarino> for the first question, i believe it's because you have more risk of being victim of an attack
 3517:09 <LarryRuane> could someone first summarize for me, if not too difficult, what the concept of whitelisting is all about?
 3617:09 <dzxzg> https://github.com/bitcoin/bitcoin/pull/17167
 3717:09 <dzxzg> Vasild offers one example of a use case: "This would allow us to use transient (aka one-time / random / disposable) I2P addresses when making outgoing connections to all nodes, except to whitelisted ones for which we would use our persistent I2P address."
 3817:10 <brunoerg> what kind of attack, hernanmarino? and why?
 3917:10 <svav> whitelisting is just saying connections from or to a given IP address is ok
 4017:10 <svav> a whitelist is a list of IP addresses usually
 4117:11 <brunoerg> LarryRuane: giving some permissions for peers
 4217:11 <LarryRuane> sorry, but why would a connection to or from a given IP *not* be okay?
 4317:12 <glozow> I imagine the answer to the first question is that the expected use case = manually connecting one node to another, so you `addnode` on one (which initiates a manual) and `whitelist` on the other (which receives an inbound).
 4417:12 <hernanmarino> perhaps I'm thinking out loud , and there are other reasons, but the simplest form of attack I can think of are eclipse attacks, where the attacker chooses to isolate you
 4517:13 <brunoerg> you can whitelist a peer to immune it to DoS banning, for example.
 4617:13 <brunoerg> or all transactions they broadcast are always relays
 4717:14 <brunoerg> glozow: it makes sense
 4817:14 <LarryRuane> this helps me a lot, https://github.com/bitcoin/bitcoin/blob/master/src/net_permissions.h#L18
 4917:16 <brunoerg> about the other part of the question: does it make sense to extend the permissions to outbound peers?
 5017:16 <glozow> is this including manual connections? or only automatic outbound?
 5117:17 <roze_paul> @brunoerg Well an inbound peer could always overwrite an outboundpeers permissions
 5217:17 <brunoerg> glozow: I think it includes manual as well
 5317:18 <LarryRuane> brunoerg: the answer must be yes, but I'm not sure why... our outbound peers typically are chosen randomly from addrman, which are gossiped to us, so could be anything .. so might make sense to treat certain addresses in a special way? (but I'm not sure)
 5417:19 <hernanmarino> brunoerg : this was one of the question I had doubts with, but i guess the PR author can tell us a little bit about it :)
 5517:20 <LarryRuane> I see this comment https://github.com/bitcoin/bitcoin/pull/10594#issue-236018335 but it doesn't explain why
 5617:21 <brunoerg> Well, we have different permissions: bloomfilter, relay, forcerelay, download, noban, mempool and addr
 5717:22 <brunoerg> I could use noban with outgoing peers to speed up relay
 5817:23 <roze_paul> also thinking outloud here: an outbound whitelist -mempool between two miners in a pool could ensure they had the same mempool, which is something they could want, if only to ensure equal state with one another
 5917:24 <hernanmarino> yes, noban is the first use case that came to my mind for outgoing whitelisting
 6017:25 <LarryRuane> so a node operator might give noban permission to an address that it knows is a "good" peer? Like, maybe this node operator also runs it?
 6117:25 <brunoerg> LarryRuane: yes!
 6217:28 <brunoerg> I think we can go to the next question but we can continue discussing that one
 6317:28 <lightlike> would this make sense for automatic outbound peers? with thousands of possible peers in the network it seems unlikely that we'll connect to any given peer again anytime soon, so would permanent permissions make sense?
 6417:28 <pablomartin> +1 roze_paul
 6517:28 <brunoerg> Considering we already have the -whitelist startup option, why would an RPC be useful? What do we want to avoid?
 6617:28 <brunoerg> > would this make sense for automatic outbound peers?
 6717:29 <brunoerg> I don't believe permanent permissions make sense, perhaps this is one of the motivations for the RPC
 6817:30 <brunoerg> thinking about it, could make sense add a "remove" option in the RPC
 6917:31 <roze_paul> Right now to change the whitelist Core must reboot. With an RPC, the whitelist can update 'onthefly'
 7017:31 <pablomartin> brunoberg: yeah, and an update?
 7117:31 <brunoerg> roze_paul: perfect!
 7217:31 <LarryRuane> the permissions set by this new RPC are not persisted to disk, right? (they're lost if a restarts happens?)
 7317:32 <brunoerg> LarryRuane: you're right
 7417:32 <LarryRuane> (pretty sure, just wanted to double-check)
 7517:32 <LarryRuane> could this new RPC possibly also be useful for testing?
 7617:32 <roze_paul> ..but they can be added to the config.file?
 7717:32 <brunoerg> LarryRuane: yes, we're gonna see it in the last question
 7817:33 <LarryRuane> brunoerg: thanks, sorry, i'm under-prepared today!
 7917:33 <brunoerg> LarryRuane: don't worry!
 8017:34 <brunoerg> but yes, the motivation is basically to be able to manage the permissions without having to restart our node.
 8117:35 <brunoerg> any other question?
 8217:36 <brunoerg> so, let's go the next one!
 8317:36 <brunoerg> This PR adds a ConnectionDirection parameter in TryParsePermissionFlags to control whether it will apply the permissions to inbound or outbound connections. In netbase.h,ConnectionDirection has 2 operators overloading. Could you explain how Operator Overloading in C++ works and how it has been used in ConnectionDirection?
 8417:39 <brunoerg> Anyone wants to explain Operator Overloading?
 8517:39 <roze_paul> Operator overloading is the usage of custom-operator definitions for custom classes. IIUC a custom class will not be able to, for instance, add two values together, even if they appear to be integers, unless that effect is defined in the class setup...
 8617:39 <roze_paul> for the second part, i believe the connectiondirection[class?] uses standard bitmasking
 8717:39 <hernanmarino> A quick read (as of now) suggests & and | operations for easy configuration of combinations of permissions
 8817:39 <roze_paul> ie around line 32. 1U <<0 1U << 1 etc is just bitmasking operations
 8917:39 <roze_paul> oh am i way off the mark?
 9017:40 <brunoerg> roze_paul: you're right
 9117:40 <LarryRuane> if we have a `ConnectionDirection` variable, we can use a natural syntax to "add" another connection direction to it ... even though this is a `class enum`
 9217:40 <hernanmarino> sorry not permissions, directions
 9317:40 <brunoerg> LarryRuane: nice!
 9417:41 <LarryRuane> if you use just plain `enum` rather than `class enum`, you can treat variables and expressions as just normal integers... but using `class enum` has advantages and is the more "modern" way to do enumerations
 9517:41 <LarryRuane> "plain" enums are the same as C (inherited into c++ from c)
 9617:42 <brunoerg> Great explanation
 9717:43 <LarryRuane> you could also have defined `+` or maybe `+` instead of `|` (or `|=`) ... did y ou consider that?
 9817:43 <brunoerg> no, but i'm gonna take a look at it, can make sense
 9917:43 <LarryRuane> i guess the "or" semantic is better though, because if the variable already has a direction, and you "add" the same one to it, it shouldn't change
10017:44 <brunoerg> and then we avoid an unnecessary change
10117:44 <LarryRuane> i.e. if variable has `In` and you want to make sure it has `In` then i guess "or" is better
10217:45 <hernanmarino> I like boolean operators more
10317:45 <LarryRuane> hernanmarino: +1 as i think about it more, i agree
10417:45 <brunoerg> about question n6: ConnectionDirection can be Both, In, Out or None. What happens in TryParsePermissionFlags if it is None? In which scenarios can this happen?
10517:49 <brunoerg> Basically, if we don't specify "in" or "out", the permissions will be applied for inbound or outbound peers? or both?
10617:50 <LarryRuane> looks like only In
10717:50 <LarryRuane> (to be backward compatible?)
10817:50 <brunoerg> perfect!
10917:50 <brunoerg> if we don't specify the direction, it will apply only for In
11017:52 <brunoerg> to keep it backward compatible
11117:52 <brunoerg> ok, let's jump in next question
11217:52 <brunoerg> In the addpermissionflags RPC we receive an array of permission flags and the IP (or network). However, we convert it to a string of the following format: “<[permissions@]IP address or network>”. Why?
11317:53 <codo> So it can be parsed by the same function that parses for -whitelist?
11417:53 <brunoerg> codo: yes!
11517:53 <hernanmarino> cool
11617:54 <brunoerg> it's simpler to convert it to a string of that format than changing the whole function that makes the validation.
11717:55 <brunoerg> `TryParsePermissionFlags` is built to handle with strings like [permissions]@ip
11817:55 <brunoerg> because this is how -whitelist works
11917:56 <brunoerg> -whitelist=[permissions]@ip
12017:56 <brunoerg> right?
12117:57 <brunoerg> ok, let's discuss the last question
12217:57 <brunoerg> How could this PR avoid the “problem” presented in #26970?
12317:59 <hernanmarino> By allowing for easier testing ?
12417:59 <brunoerg> Yes, it could speed up our tests
12517:59 <LarryRuane> we could whitelist 127.0.0.1 in the outbound direction also?
12617:59 <brunoerg> See that 'In the functional test wallet_groups.py we whitelist peers on all nodes (-whitelist=noban@127.0.0.1) to enable immediate tx relay for fast mempool synchronization'
12717:59 <svav> It's something to do with making outbound connections faster
12818:00 <brunoerg> but it doesn't work because -whitelist only works for inbound peers
12918:01 <LarryRuane> brunoerg: that's cool, just wondering, have you tried it (undo the 26970 fix)?
13018:01 <brunoerg> not yet, it's on my plan
13118:01 <brunoerg> but other reviewers pointed that it could be a fix/improvement
13218:01 <brunoerg> #endmeeting>>>>