Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nested closures result in exponential compilation time increase #72408

Closed
VFLashM opened this issue May 21, 2020 · 2 comments · Fixed by #72412
Closed

Nested closures result in exponential compilation time increase #72408

VFLashM opened this issue May 21, 2020 · 2 comments · Fixed by #72412
Labels
A-closures Area: Closures (`|…| { … }`) C-bug Category: This is a bug. I-compiletime Issue: Problems and improvements with respect to compile times. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@VFLashM
Copy link
Contributor

VFLashM commented May 21, 2020

Closures include captured types twice in a type tree.

Wrapping one closure with another leads to doubling the amount of types in the type tree.

With nested closures compile time increases exponentially and it's extremely easy to break the compiler. Obviously it's even easier if each level captures more than one closure.

I think I have a fix for this one ready, about to push it.

I tried this code:

fn dup<F: Fn(i32) -> i32>(f: F) -> impl Fn(i32) -> i32 {
    move |a| f(2*a)
}

fn main() {
    let f = |a| a;

    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);

    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);

    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);

    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);

    // Compiler dies around here if it tries
    // to walk the tree exhaustively.

    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);

    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);
    let f = dup(f);

    println!("Hello {}", f(1));
}

I expected to see this happen: no error.

Instead, this happened:

error: reached the type-length limit while instantiating `dup::<[closure@src/main.rs:2:5: ...rs:6:13: 6:18]]]]]]]]]]]]]]]]]]>`
 --> src/main.rs:1:1
  |
1 | / fn dup<F: Fn(i32) -> i32>(f: F) -> impl Fn(i32) -> i32 {
2 | |     move |a| f(2*a)
3 | | }
  | |_^
  |
  = note: consider adding a `#![type_length_limit="1835001"]` attribute to your crate

Meta

I tried it both on stable and nightly rust:

$ rustc --version --verbose
rustc 1.45.0-nightly (7ebd87a7a 2020-05-08)
binary: rustc
commit-hash: 7ebd87a7a1e0e21767422e115c9455ef6e6d4bee
commit-date: 2020-05-08
host: x86_64-pc-windows-msvc
release: 1.45.0-nightly
LLVM version: 9.0
$ rustc --version --verbose
rustc 1.43.0 (4fb7144ed 2020-04-20)
binary: rustc
commit-hash: 4fb7144ed159f94491249e86d5bbd033b5d60550
commit-date: 2020-04-20
host: x86_64-pc-windows-msvc
release: 1.43.0
LLVM version: 9.0
@VFLashM VFLashM added the C-bug Category: This is a bug. label May 21, 2020
@panstromek
Copy link
Contributor

Just for the record, similar issue has been fixed recently #68061

It's similar but without any captures.

@VFLashM
Copy link
Contributor Author

VFLashM commented May 21, 2020

There is already a src/test/ui/closures/deeply-nested_closures.rs, so I think it's a separate issue.

@VFLashM VFLashM closed this as completed May 21, 2020
@VFLashM VFLashM reopened this May 21, 2020
@jonas-schievink jonas-schievink added A-closures Area: Closures (`|…| { … }`) I-compiletime Issue: Problems and improvements with respect to compile times. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 21, 2020
VFLashM added a commit to VFLashM/rust that referenced this issue May 22, 2020
This fixes rust-lang#72408.

Nested closures were resulting in exponential compilation time.
VFLashM added a commit to VFLashM/rust that referenced this issue Sep 15, 2020
This fixes rust-lang#72408.

Nested closures were resulting in exponential compilation time.

As a performance optimization this change introduces MiniSet,
which is a simple small storage optimized set.
bors added a commit to rust-lang-ci/rust that referenced this issue Sep 18, 2020
…xponential, r=tmandry

Issue 72408 nested closures exponential

This fixes rust-lang#72408.

Nested closures were resulting in exponential compilation time.

This PR is enhancing asymptotic complexity, but also increasing the constant, so I would love to see perf run results.
@bors bors closed this as completed in 2f32961 Sep 18, 2020
Mark-Simulacrum pushed a commit to Mark-Simulacrum/rust that referenced this issue Oct 3, 2020
This fixes rust-lang#72408.

Nested closures were resulting in exponential compilation time.

As a performance optimization this change introduces MiniSet,
which is a simple small storage optimized set.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-closures Area: Closures (`|…| { … }`) C-bug Category: This is a bug. I-compiletime Issue: Problems and improvements with respect to compile times. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants