Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add limit configuration documentation #1421

Merged
merged 2 commits into from
Oct 11, 2022
Merged

Conversation

achingbrain
Copy link
Member

@achingbrain achingbrain commented Oct 10, 2022

To help people better configure their libp2p nodes to defend against malicious peers, document all the limit settings and protections in one place.

To help people better configure their libp2p nodes to defend against
malicious peers, document all the limit settings in one place.
@achingbrain
Copy link
Member Author

Instead of duplicating advice in https://github.com/libp2p/docs/blob/master/content/reference/dos-mitigation.md I thought I'd link out from there to the sections being added here - what do you think @BigLep ?

@achingbrain achingbrain changed the title docs: add limit configuartion documentation docs: add limit configuration documentation Oct 11, 2022
@achingbrain achingbrain marked this pull request as ready for review October 11, 2022 11:15
@achingbrain
Copy link
Member Author

I'm going to merge this to move the dos docs forward, there may be a follow up PR to this one with any required changes.

@achingbrain achingbrain merged commit 25d935e into master Oct 11, 2022
@achingbrain achingbrain deleted the docs/add-limit-docs branch October 11, 2022 11:16
Copy link
Contributor

@BigLep BigLep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together. It's helpful to see the end customer experience. Also definitely agreed of linking to this from https://docs.libp2p.io/reference/dos-mitigation/ . No copy and paste!

A few things:

  1. How do we handle transient connections? Is there a way to limit them? It looks like yes?

    maxIncomingPendingConnections?: number
    I didn't see this documented.

  2. Sometimes I see notes about how there are default values and other times there aren't defaults. What happens in the cases where these values aren't set? Can we link to the authoritative place for defaults (e.g.,

    const defaultOptions: Partial<ConnectionManagerInit> = {
    ) ?

  3. Is it possible to control the max amount of data buffered across all js-libp2p connections in total? If not, is at least possible to set a max buffer limit at the connection level? I'm assuming not. It looks like (maxInboundStreams + maxOutboundStreams) * maxStreamBufferSize is the upper bound that a single connection could consume (if all streams appraoched the maxStreamBufferSize). I wonder if it would be easier for a user to reason about if we had a limit like maxConnectionBuufferSize or maxMuxerBufferSize. Anyways, assuming this is a no go, maybe documenting this a bit more?

  4. LIMITS.md answers specifics about js-libp2p. I think it would be good to link out to https://docs.libp2p.io/reference/dos-mitigation/, which give a wholistic take on DoS mitigation (potentially providing useful context).

I'll do my best to respond to feedback, but please don't block on me. Opening and merging a PR like this works for me.

doc/LIMITS.md Show resolved Hide resolved
doc/LIMITS.md Show resolved Hide resolved
* A remote peer may attempt to open up to this many connections per second,
* any more than that will be automatically rejected
*/
inboundConnectionThreshold: 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we're not going to change now, but given this is a "rate", I think it would be ideal to get the word "rate" into the name.

doc/LIMITS.md Show resolved Hide resolved
- [TCP](#tcp)
- [Allow/deny lists](#allowdeny-lists)

## Connection limits
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to specify different limits for inbound vs. outbound?

I can imagine applications taking the posture of trusting their outbound connections (and thus no limits) but being defensive on inbound connections (and thus having limits).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not at the moment, but I can see why this is something you might want to control

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@achingbrain : I think this is worth backlogging. Then maybe lets add a note here like:

"There currently isn't a way to specify different limits for incoming vs. outgoing. Connection limits are applied across both incoming and outgoing connections combined. There is a backlog item for this here."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this as the first paragraph:

It's possible to limit the total amount of connections a node is able to make (combining incoming and outgoing). When this limit is reached and an attempt to open a new connection is made, existing connections may be closed to make room for the new connection (see [Closing connections][#closing-connections]).

  • Note: there currently isn't a way to specify different limits for incoming vs. outgoing. Connection limits are applied across both incoming and outgoing connections combined. There is a backlog item for this here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opened #1508

doc/LIMITS.md Show resolved Hide resolved

/**
* If the number of open connections goes below this number, the node
* will try to connect to nearby peers from the peer store
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put "nearby" in quotes?

Where is "nearby" defined?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, it'll actually sort-of-randomly select some peers from the peer store (preferring ones we have the public key for or protocol list they support), will update.

It should really choose peers that have been turned up by the periodic "findClosestPeers" query we run. "close" here is defined by the peer routing implementation, so KAD-DHT would choose KAD-close peers, etc.

doc/LIMITS.md Show resolved Hide resolved
doc/LIMITS.md Show resolved Hide resolved
doc/LIMITS.md Show resolved Hide resolved
@BigLep
Copy link
Contributor

BigLep commented Oct 12, 2022

Also, not required, but it wouldn't hurt to flag for @MarcoPolo in case he wants to look given his work on DoS docs and familiarity with multiple of the libp2p implementations.

Copy link
Collaborator

@MarcoPolo MarcoPolo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @BigLep that we should use the word "rate" instead of threshold when we're talking about rate limits.

Also a couple other comments.

doc/LIMITS.md Show resolved Hide resolved

When registering listeners for custom protocols, the maximum number of simultaneously open inbound and outbound streams per-connection can be specified. If not specified these will default to 32 inbound streams and 64 outbound streams.

If more than this number of streams for the given protocol are opened on a single connection, subsequent new streams for that protocol will be immediately reset.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a mechanism to coalesce connections? What happens if peer B opens 10 connections to peer A. Does peer A see that there already is a connection to peer B and close the older ones?

I'm wondering because this limit is per connection, and I'm curious why it isn't per peer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet, all we can do is just close duplicate connections I think, at least until the stream migration protocol gets a bit more fleshed out.

doc/LIMITS.md Show resolved Hide resolved
allow: [
'/ip4/43.123.5.23/tcp/3984',
'/ip4/234.243.64.2',
'/ip4/52.55',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a valid multiaddr, isn't that confusing? I think we should prefer to use the standard IP cidr format (I know this is just the docs, so this will involve impl changes) e.g. "/ip4/52.55" should be "/ip4/52.55.0.0/ipcidr/16".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are treated as strings prefixes not full multiaddrs. It'd probably be more correct to make them multiaddrs and less error prone but I don't think the @multiformats/multiaddr module will let us match address ranges.

A good improvement for the future.

@p-shahi p-shahi linked an issue Oct 13, 2022 that may be closed by this pull request
3 tasks
achingbrain added a commit that referenced this pull request Oct 19, 2022
Incorporate feedback from #1421
achingbrain added a commit that referenced this pull request Dec 6, 2022
Incorporate feedback from #1421

Co-authored-by: Steve Loeppky <biglep@protocol.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add documentation on mechanisms for node monitoring and protection
3 participants