-
Notifications
You must be signed in to change notification settings - Fork 6.8k
Add a tutorial for control flow operators. #12340
Conversation
09ceac8
to
cdfacfc
Compare
@aaronmarkham could you review this tutorial, please? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice tutorial. I have a few suggestions though.
MXNet currently provides three control flow operators: `cond`, `foreach` and `while_loop`. Like other MXNet operators, they all have a version for NDArray and a version for Symbol. These two versions have exactly the same semantics. We can take advantage of this and use them in Gluon to hybridize models. | ||
|
||
In this tutorial, we use a few examples to demonstrate the use of control flow operators in Gluon and show how a model that requires control flow is hybridized. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are there any prerequisites worth mentioning?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think so.
foreach(body, data, init_states, name) => (outputs, states) | ||
``` | ||
|
||
It iterates over the first dimension of the input data (it can be an array or a list of arrays) and run the Python function defined in `body` for every slice from the input arrays. The signature of the `body` function is defined as follows: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
runs
|
||
The inputs of the `body` function have two parts: `data` is a slice of an array (if there is only one input array in `foreach`) or a list of slices (if there are a list of input arrays); `states` are the arrays from the previous iteration. The outputs of the `body` function also have two parts: `outputs` is an array or a list of arrays; `states` is the computation states of the current iteration. `outputs` from all iterations are concatenated as the outputs of `foreach`. | ||
|
||
The pseudocode below illustrates the execution of `foreach`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use "the following" instead of below (different consumption modes means the content isn't always below)
``` | ||
|
||
### Example 1: foreach works like map | ||
`foreach` can work like a map function of a functional language. In this case, the states of foreach can be an empty list, which means the computation doesn't carry computation states across iterations. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tick each use of foreach
### Example 1: foreach works like map | ||
`foreach` can work like a map function of a functional language. In this case, the states of foreach can be an empty list, which means the computation doesn't carry computation states across iterations. | ||
|
||
In this example, we use `foreach` to add each element in an array by one. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use foreach with an array to increase each element's value by one
``` | ||
|
||
|
||
```python |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why two different code blocks? Is there supposed to be some commentary here?
from mxnet.gluon import HybridBlock | ||
``` | ||
|
||
# foreach |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd go to two hashes for your subsections and only use the single hash for the document title.
res, states = lstm(rnn_data, [x for x in init_states], valid_length) | ||
``` | ||
|
||
# while_loop |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
two hashes
<NDArray 1 @cpu(0)>] | ||
|
||
|
||
# cond |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
two hashes
|
||
|
||
# cond | ||
`cond` is defined with the following signature: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice to talk about what it is and what it does before showing the signature. The same goes for the other sections/functions.
@@ -0,0 +1,390 @@ | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will this be added to http://mxnet.incubator.apache.org/tutorials/index.html ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how to do that?
@zheng-da @eric-haibin-lin @aaronmarkham Can you check if all the feebdack was addressed and is this PR ready to merge now? |
my comments are addressed |
@Roshrini I have addressed all of @aaronmarkham's comments. |
* the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix.
* Add a tutorial for control flow operators. (#12340) * the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix. * Update ONNX API docs references (#12317) * update onnx API references * update descriptions * [MXAPPS-581] Disable an additional long test in the SD nightly (#12343) * Disable an additional test in the SD nightly that also takes over the timeout. * Documentation update related to sparse support (#12367) * Update sparse.md * Update sparse.md * Update csr.md * Update row_sparse.md * Update train.md
* the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix.
Description
This is to provide a tutorial for control flow operators.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.
Changes
Comments