Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement excluding a build-step via --exclude #48105

Merged
merged 5 commits into from
Feb 15, 2018

Conversation

Mark-Simulacrum
Copy link
Member

First step to fixing #47911. This doesn't change any CI configuration, but implements what I believe necessary to make that feasible in rustbuild.

In theory this should be sufficient to allow someone to open a PR against .travis.yml and appveyor.yml which splits the Windows 32-bit tests and maybe the OS X tests into multiple builders (depending on what our cost-concerns are) to reduce runtimes.

r? @alexcrichton
cc @kennytm

@shepmaster shepmaster added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 9, 2018
@Mark-Simulacrum
Copy link
Member Author

This will need a refactor of how we make multi-path default steps work. Waiting-on-author...

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 9, 2018
Previously, a Step would be able to tell on its own when it was invoked
"by-default" (that is, `./x.py test` was called instead of `./x.py test
some/path`). This commit replaces that functionality, invoking each Step
with each of the paths it has specified as "should be invoked by."

For example, if a step calls `path("src/tools/cargo")` and
`path("src/doc/cargo")` then it's make_run will be called twice, with
"src/tools/cargo" and "src/doc/cargo." This makes it so that default
handling logic is in builder, instead of spread across various Steps.

However, this meant that some Step specifications needed to be updated,
since for example `rustdoc` can be built by `./x.py build
src/librustdoc` or `./x.py build src/tools/rustdoc`. A `PathSet`
abstraction is added that handles this: now, each Step can not only list
`path(...)` but also `paths(&[a, b, ...])` which will make it so that we
don't invoke it with each of the individual paths, instead invoking it
with the first path in the list (though this shouldn't be depended on).

Future work likely consists of implementing a better/easier way for a
given Step to work with "any" crate in-tree, especially those that want
to run tests, build, or check crates in the std, test, or rustc crate
trees. Currently this is rather painful to do as most of the logic is
duplicated across should_run and make_run. It seems likely this can be
abstracted away into builder somehow.
@alexcrichton
Copy link
Member

@Mark-Simulacrum was this ready to go or did you want to queue up some more changes? (it looked ready!)

@Mark-Simulacrum
Copy link
Member Author

I think with the latest commits this is ready for review. I have another branch with more work but that's separate and not quite ready.

@alexcrichton
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Feb 12, 2018

📌 Commit f104b12 has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 12, 2018
@aidanhs
Copy link
Member

aidanhs commented Feb 13, 2018

@bors p=1
We're getting the three hour timeouts on non-windows builds (#48192) and our queue is growing - would be good to get this in soon.

@bors
Copy link
Contributor

bors commented Feb 13, 2018

⌛ Testing commit f104b12 with merge b8134c7b4df103f484d790d40be51404fe7d1491...

@bors
Copy link
Contributor

bors commented Feb 13, 2018

💔 Test failed - status-travis

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 13, 2018
@aidanhs
Copy link
Member

aidanhs commented Feb 14, 2018

Many failures, e.g.

[00:03:15] + python2.7 ../x.py dist --host armv7-unknown-linux-gnueabihf --target armv7-unknown-linux-gnueabihf
[00:03:16]     Finished dev [unoptimized] target(s) in 0.0 secs
[00:03:17] thread 'main' panicked at '"dist::Mingw" should have at least one pathset', bootstrap/builder.rs:193:13
[00:03:17] note: Run with `RUST_BACKTRACE=1` for a backtrace.
failed to run: /checkout/obj/build/bootstrap/debug/bootstrap dist --host armv7-unknown-linux-gnueabihf --target armv7-unknown-linux-gnueabihf
[00:03:17] Build completed unsuccessfully in 0:00:01

Some Steps are by-default run but don't have any paths associated with
them. We need to have at least one PathSet per each Step, though, so we
add an empty one on calls to `never()`.
@Mark-Simulacrum
Copy link
Member Author

@bors r=alexcrichton

@bors
Copy link
Contributor

bors commented Feb 14, 2018

📌 Commit a64575c has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 14, 2018
@bors
Copy link
Contributor

bors commented Feb 14, 2018

⌛ Testing commit a64575c with merge 4928171c911aca5e7538a49352f8815c9ce8d96f...

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Feb 14, 2018
@bors
Copy link
Contributor

bors commented Feb 14, 2018

⌛ Testing commit a64575c with merge 3e13f0e5cb7f97c681bb1e2a18d1b998cb5c3aac...

@bors
Copy link
Contributor

bors commented Feb 14, 2018

💔 Test failed - status-appveyor

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 14, 2018
@kennytm
Copy link
Member

kennytm commented Feb 14, 2018

@bors retry #48116

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 14, 2018
@bors
Copy link
Contributor

bors commented Feb 14, 2018

⌛ Testing commit a64575c with merge fb623fa...

bors added a commit that referenced this pull request Feb 14, 2018
Implement excluding a build-step via --exclude

First step to fixing #47911. This doesn't change any CI configuration, but implements what I believe necessary to make that feasible in rustbuild.

In theory this should be sufficient to allow someone to open a PR against .travis.yml and appveyor.yml which splits the Windows 32-bit tests and maybe the OS X tests into multiple builders (depending on what our cost-concerns are) to reduce runtimes.

r? @alexcrichton
cc @kennytm
@bors
Copy link
Contributor

bors commented Feb 14, 2018

💔 Test failed - status-appveyor

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 14, 2018
@kennytm
Copy link
Member

kennytm commented Feb 14, 2018

@bors retry #46903

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 14, 2018
@bors
Copy link
Contributor

bors commented Feb 15, 2018

⌛ Testing commit a64575c with merge c83fa5d...

bors added a commit that referenced this pull request Feb 15, 2018
Implement excluding a build-step via --exclude

First step to fixing #47911. This doesn't change any CI configuration, but implements what I believe necessary to make that feasible in rustbuild.

In theory this should be sufficient to allow someone to open a PR against .travis.yml and appveyor.yml which splits the Windows 32-bit tests and maybe the OS X tests into multiple builders (depending on what our cost-concerns are) to reduce runtimes.

r? @alexcrichton
cc @kennytm
@bors
Copy link
Contributor

bors commented Feb 15, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing c83fa5d to master...

@bors bors merged commit a64575c into rust-lang:master Feb 15, 2018
@bors bors mentioned this pull request Feb 15, 2018
kennytm added a commit to kennytm/rust that referenced this pull request Feb 15, 2018
…, r=alexcrichton"

This reverts commit c83fa5d, reversing
changes made to 90759be.
bors added a commit that referenced this pull request Mar 17, 2018
…ulacrum

re-enable testing librustdoc

This was originally put in in #44274, but #48105 accidentally hid it. This change puts librustdoc unit/doc tests back in the main test listing.

fixes #44237 (again)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants