Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: clarify async iterator leak #28997

Closed
wants to merge 1 commit into from

Conversation

ronag
Copy link
Member

@ronag ronag commented Aug 6, 2019

Clarifies that creating multiple async iterators from the same stream can lead to event listener leak.

Checklist

@nodejs-github-bot nodejs-github-bot added doc Issues and PRs related to the documentations. stream Issues and PRs related to the stream subsystem. labels Aug 6, 2019
@ronag ronag force-pushed the doc-async-iterator branch 4 times, most recently from 8a6b51d to 5fb5109 Compare August 6, 2019 09:47
@ronag
Copy link
Member Author

ronag commented Aug 6, 2019

@Trott flaky V8 compile, please restart?

@nodejs-github-bot
Copy link
Collaborator

@@ -2499,6 +2499,14 @@ and async iterators are provided below.
})();
```

Async iterators register a permanent error handler. If the iterator is not
fully consumed, any unhandled stream error will be swallowed. When consuming
streams using async iterators any errors emitted after `'end'` or `'close'`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
streams using async iterators any errors emitted after `'end'` or `'close'`
streams using async iterators, any errors emitted after `'end'` or `'close'`

Trott
Trott previously approved these changes Aug 7, 2019
@Trott
Copy link
Member

Trott commented Aug 7, 2019

@nodejs/streams

@Trott Trott dismissed their stale review August 7, 2019 05:03

going to defer to stream folks on this

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should land this change.

When a stream is async iterated it will always exit the iteration in a destroyed state, i.e. break or  throw will call destroy() on the stream. As a result, we cannot create multiple streams one after the other.
Moreover, creating multiple parallel AsyncIterators out of the same stream is problematic and should not be done: the behavior is going to be very unpredictable (which of the two iterators will get the data? only one will).

What should be documented is that, because it's left in a destroyed state, there will be a  'error' event handler attached to prevent further exceptions to crash the process.

@ronag
Copy link
Member Author

ronag commented Aug 7, 2019

Isn’t it possible to create multiple iterators? i.e when using the iterator API directly and not through a for loop?

Should we maybe throw if a second iterative is created?

@mcollina
Copy link
Member

mcollina commented Aug 7, 2019

Isn’t it possible to create multiple iterators? i.e when using the iterator API directly and not through a for loop?

The semantics of that are currently not what somebody is going to expect. The two iterators are going to compete for the chunks (as they use 'readable' / read() internally). An iterator does not use .pipe(), so there is no multiple destination logic in there.

@ronag
Copy link
Member Author

ronag commented Aug 7, 2019

The semantics of that are currently not what somebody is going to expect. The two iterators are going to compete for the chunks (as they use 'readable' / read() internally). An iterator does not use .pipe(), so there is no multiple destination logic in there.

Hence, should we throw if a secondary iterator is created? Also possibly add a note in the docs?

@ronag
Copy link
Member Author

ronag commented Aug 7, 2019

Updated description in accordance with @mcollina's previous suggestion.

@jasnell
Copy link
Member

jasnell commented Aug 7, 2019

Throwing if there's an attempt to create a second iterator would make sense to me. What do you think @mcollina?

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mcollina
Copy link
Member

mcollina commented Aug 9, 2019

Throwing if there's an attempt to create a second iterator would make sense to me. What do you think @mcollina?

I'm definitely +1 on that. @ronag, would you like to send a PR for that?

@ronag ronag mentioned this pull request Aug 9, 2019
4 tasks
@ronag
Copy link
Member Author

ronag commented Aug 12, 2019

@Trott: This fails because I didn't rebase master. However, since it's a simple doc change I don't think it's worth a rebase?

@Trott
Copy link
Member

Trott commented Aug 12, 2019

@@ -2502,6 +2502,9 @@ and async iterators are provided below.
})();
```

Async iterators register a permanent error handler on the stream to prevent any
unhandled post-destroy errors.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that what I"m about to suggest is a good idea, so I'll defer to everyone else's judgment:

Is it worth re-wording to make it clear that this means the destroy() method?

Suggested change
unhandled post-destroy errors.
unhandled errors after `destroy()` executes.

I'm not sure that's an improvement to be honest, but maybe?

@Trott
Copy link
Member

Trott commented Aug 12, 2019

Landed in d7a4ace

@Trott Trott closed this Aug 12, 2019
Trott pushed a commit to Trott/io.js that referenced this pull request Aug 12, 2019
Clarifies that creating multiple async iterators from the same stream
can lead to event listener leak.

PR-URL: nodejs#28997
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
targos pushed a commit that referenced this pull request Aug 19, 2019
Clarifies that creating multiple async iterators from the same stream
can lead to event listener leak.

PR-URL: #28997
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc Issues and PRs related to the documentations. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants