From 44b0a00635679d28b895620c912ba6ff5ef55a92 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Thu, 1 Feb 2024 14:51:56 -0800 Subject: [PATCH 01/24] Non-Escapable Types proposal Briefly, non-escapable types are types that cannot "escape" the local execution context. These types can also participate in extended lifetime dependency constraints. This enables compile-time lifetime enforcement for values that contain pointers into other values. This makes it possible to implement containers that require no runtime lifetime management of their iterators or slice objects. In particular, this is a foundational technology for BufferView. --- proposals/NNNN-non-escapable.md | 387 ++++++++++++++++++++++++++++++++ 1 file changed, 387 insertions(+) create mode 100644 proposals/NNNN-non-escapable.md diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md new file mode 100644 index 0000000000..9020686f4b --- /dev/null +++ b/proposals/NNNN-non-escapable.md @@ -0,0 +1,387 @@ +# Non-Escapable Types + +* Proposal: [SE-NNNN](NNNN-filename.md) +* Authors: [Andrew Trick](https://github.com/atrick), [Tim Kientzle](https://github.com/tbkka) +* Review Manager: TBD +* Status: **Awaiting implementation** +* Roadmap: [BufferView Language Requirements](https://forums.swift.org/t/roadmap-language-support-for-bufferview) +* Implementation: **Pending** +* Upcoming Feature Flag: `NonescapableTypes` +* Review: ([pitch](https://forums.swift.org/...)) + +## Introduction + +We propose adding a new type constraint `~Escapable` for types that can be locally copied but cannot be assigned or transferred outside of the immediate context. +This complements the `~Copyable` types added with SE-0390 by introducing another set of compile-time-enforced lifetime controls that can be used for safe, highly-performant APIs. + +In addition, these types will support lifetime-dependency constraints (being tracked in a separate proposal), that allow them to safely hold pointers referring to data stored in other types. + +This feature is a key requirement for the proposed `BufferView` type. + +**See Also** + +* [SE-0390: Noncopyable structs and enums](https://github.com/apple/swift-evolution/blob/main/proposals/0390-noncopyable-structs-and-enums.md) +* [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) +* [Roadmap for improving Swift performance predictability: ARC improvements and ownership control](https://forums.swift.org/t/a-roadmap-for-improving-swift-performance-predictability-arc-improvements-and-ownership-control/54206) +* [Ownership Manifesto](https://forums.swift.org/t/manifesto-ownership/5212) +* ### TODO: Link to BufferView proposal +* ### TODO: Link to lifetime dependency annotations proposal + +## Motivation + +Swift's current notion of an "iterator" has several weaknesses that become apparent when you try to use it in extremely performance-constrained environments. +These weaknesses arise from the desire to ensure safety while simultaneously allowing iterator values to be arbitrarily copied to support multi-iterator algorithms. + +For example, the standard library iterator for Array logically creates a copy of the Array when it is constructed; this ensures that changes to the array cannot affect the iteration. +This is implemented by having the iterator store a reference-counted pointer to the array storage in order to ensure that the storage cannot be freed while the iterator is active. +These safety checks all incur runtime overhead. + +In addition, the use of reference counting to ensure correctness at runtime makes types of this sort unusable in highly constrained embedded environments. + +## Proposed solution + +Currently, the notion of "escapability" appears in the Swift language as a feature of closures. +Closures that are declared as `@nonescapable` can use a very efficient stack-based representation; +closures that are `@escapable` store their state on the heap. + +By allowing Swift developers to mark various types as non-escapable, we provide a mechanism for them to opt into a specific set of usage limitations that: + +* Can be automatically verified by the compiler. In fact, the Swift compiler internally already makes heavy use of escapability as a concept. +* Are strict enough to permit high performance. The compiler uses this concept precisely because values that do not escape can be managed much more efficiently. +* Do not interfere with common uses of these types. + +For example, if an iterator type were marked as non-escapable, the compiler would produce an error message whenever the user of that type tried to copy or store the value in a way that might limit efficient operation. +These checks would not significantly reduce the usefulness of iterators which are almost always created, used, and destroyed in a single local context. +These checks would also still allow local copies for multi-iterator uses, with the same constraints applied to those copies as well. + +A separate proposal will show how we can further improve safety by allowing library authors to impose additional constraints that bind the lifetime of the iterator to the object that produced it. +These "lifetime dependency" constraints can also be verified at compile time to ensure that the source of the iterator is not modified and that the iterator specifically does not outlive its source. + +**Note**: We are using iterators here to illustrate the issues we are considering. +We are not at this time proposing any changes to Swift's current `Iterator` protocol. + +## Detailed design + +#### New Escapable Concept + +We add a new type constraint `Escapable` to the standard library and implicitly apply it to all current Swift types (with the sole exception of `@nonescapable` closures). +`Escapable` types can be assigned to global variables, passed into arbitrary functions, or returned from the current function or closure. +This matches the existing semantics of all Swift types prior to this proposal. + +Specifically, we will add this declaration to the standard library: + +``` +// An Escapable type may or may not be Copyable +protocol Escapable: ~Copyable {} +``` + +#### In concrete contexts, `~Escapable` indicates non-escapability + +Using the same approach as used for `~Copyable` and `Copyable`, we use `~Escapable` to indicate the lack of the `Escapable` attribute on a type. + +``` +// Example: A type that is not escapable +struct NotEscapable: ~Escapable { } + +// Example: Basic limits on ~Escapable types +func f() -> NotEscapable { + let ne = NotEscapable() + borrowingFunc(ne) // OK to pass to borrowing function + let another = ne // OK to make local copies + globalVar = ne // 🛑 Cannot assign ~Escapable type to a global var + return ne // 🛑 Cannot return ~Escapable type +} +``` + +Without a `~Escapable` marker, the default for any type is to be escapable. Since `~Escapable` indicates the lack of a capability, you cannot put this in an extension. + +``` +// Example: Escapable by default +struct Ordinary { } +extension Ordinary: ~Escapable // 🛑 Extensions cannot remove a capability +``` + +Classes cannot be declared `~Escapable`. + +#### In generic contexts, `~Escapable` marks the lack of an Escapable requirement + +When used in a generic context, `~Escapable` allows you to define functions or types that can work with values that might or might not be escapable. +That is, `~Escapable` indicates the lack of an escapable requirement. +Since the values might not be escapable, the compiler must conservatively prevent the values from escaping: + +``` +// Example: In generic contexts, ~Escapable is +// the lack of an Escapable requirement. +func f(_ value: MaybeEscapable) { + // `value` might or might not be Escapable + globalVar = value // 🛑 Cannot assign possibly-non-escapable type to a global var +} +f(NotEscapable()) // Ok to call with non-escapable argument +f(7) // Ok to call with escapable argument +``` + +This also permits the definition of types whose escapability varies depending on their generic arguments. +As with other conditional behaviors, this is expressed by using an extension to conditionally add a new capability to the type: + +``` +// Example: Conditionally Escapable generic type +// By default, Box is itself non-escapable +struct Box: ~Escapable { + var t: T +} + +// Box gains the ability to escape whenever its +// generic argument is Escapable +extension Box: Escapable when T: Escapable { } +``` + +**Note:** There is no relationship between `Copyable` and `Escapable`. +Copyable or non-copyable types can be escapable or non-escapable. + +#### Constraints on non-escapable local variables + +A non-escapable value can be freely copied and passed into other functions as long as the usage can guarantee that the value does not persist beyond the current scope: + +``` +// Example: Local variable with non-escapable type +func borrowingFunc(_: borrowing NotEscapable) { ... } +func consumingFunc(_: consuming NotEscapable) { ... } +func inoutFunc(_: inout NotEscapable) { ... } + +func f() { + var value: NotEscapable + let copy = value // OK to copy as long as copy does not escape + globalVar = value // 🛑 May not assign to global + SomeType.staticVar = value // 🛑 May not assign to static var + borrowingFunc(value) // OK to pass borrowing + inoutFunc(&value) // OK to pass inout + consumingFunc(value) // OK to pass consuming + // `value` was consumed above, but NotEscapable + // is Copyable, so the compiler can insert + // a copy to satisfy the following usage: + borrowingFunc(value) // OK +} +``` + +#### Constraints on non-escapable arguments + +A value of non-escapable type received as an argument is subject to the same constraints as any other local variable. +In particular, a `consuming` argument (and all direct copies thereof) must actually be destroyed during the execution of the function. +This is in contrast to an escaping `consuming` argument which can be disposed of by being stored in a global or static variable. + +#### Values that contain non-escapable values must be non-escapable + +Stored struct properties and enum payloads can have non-escapable types if the surrounding type is itself non-escapable. +(Equivalently, an escapable struct or enum can only contain escapable values.) +Non-escapable values cannot be stored as class properties, since classes are always inherently escaping. + +``` +// Example +struct OuterEscapable { + // 🛑 Escapable struct cannot have non-escapable stored property + var nonesc: NonEscapable +} + +enum EscapableEnum { + // 🛑 Escapable enum cannot have a non-escapable payload + case nonesc(NonEscapable) +} + +struct OuterNonEscapable: ~Escapable { + var nonesc: NonEscapable // OK +} + +enum NonEscapableEnum: ~Escapable { + case nonesc(NonEscapable) // OK +} +``` + +#### Returned non-escapable values require lifetime dependency + +A simple return of a non-escapable value is not permitted. + +``` +func f() -> NotEscapable { // 🛑 Cannot return a non-escapable type + var value: NotEscapable + return value // 🛑 Cannot return a non-escapable type +} +``` + +A separate proposal describes “lifetime dependency annotations” that can relax this requirement by tying the lifetime of the returned value to the lifetime of some other object, either an argument to the function or `self` in the case of a method or computed property returning a non-escapable type. +In particular, struct and enum initializers (which build a new value and return it to the caller) cannot be written without some mechanism similar to that outlined in our companion proposal. + +#### Globals and static variables cannot be non-escapable + +Non-escapable values must be constrained to some specific local execution context. +This implies that they cannot be stored in global or static variables. + +#### Closures and non-escapable values + +Escaping closures cannot capture non-escapable values. +Non-escaping closures can capture non-escapable values subject only to the usual exclusivity restrictions. + +Returning a non-escapable value from a closure requires explicit lifetime dependency annotations, as covered in the companion proposal. + +#### Non-escapable values and concurrency + +All of the requirements on use of non-escapable values as function arguments and return values also apply to async functions, including those invoked via `async let`. + +The closures used in `Task.init` or `Task.detached` are escaping closures and therefore cannot capture non-escapable values. + +## Source compatibility + +The compiler will treat any type without an explicit `~Escapable` marker as escapable. +This matches the current behavior of the language. + +Only when new types are marked as `~Escapable` does this have any impact. + +Adding `~Escapable` to an existing concrete type is generally source-breaking because existing source code may rely on being able to escape values of this type. +Removing `~Escapable` from an existing concrete type is not generally source-breaking since it effectively adds a new capability, similar to adding a new protocol conformance. + +## ABI compatibility + +As above, existing code is unaffected by this change. +Adding or removing a `~Escapable` constraint on an existing type is an ABI-breaking change. + +## Implications on adoption + +Manglings and interface files will only record the lack of escapability. +This means that existing interfaces consumed by a newer compiler will treat all types as escapable. +Similarly, an old compiler reading a new interface will have no problems as long as the new interface does not contain any `~Escapable` types. + +These same considerations ensure that escapable types can be shared between previously-compiled code and newly-compiled code. + +Retrofitting existing generic types so they can support both escapable and non-escapable type arguments is possible with care. + +## Future directions + +#### `BufferView` type + +This proposal is being driven in large part by the needs of the `BufferView` type that has been discussed elsewhere. +Briefly, this type would provide an efficient universal “view” of array-like data stored in contiguous memory. +Since values of this type do not own any data but only refer to data stored elsewhere, their lifetime must be limited to not exceed that of the owning storage. +We expect to publish a sample implementation and proposal for that type very soon. + +#### Lifetime dependency annotations + +Non-escapable types have a set of inherent restrictions on how they can be passed as arguments, stored in variables, or returned from functions. +A companion proposal builds on this by supporting more detailed annotations that link the lifetimes of different objects. +This would allow, for example, a container to vend an iterator value that held a direct unmanaged pointer to the container's contents. +The lifetime dependency would ensure that such an iterator could not outlive the container to whose contents it referred. + +``` +// Example: Non-escaping iterator +struct NEIterator { + // `borrow(container)` indicates that the constructed value + // cannot outlive the `container` argument. + init(over container: MyContainer) -> borrow(container) Self { + ... initialize an iterator suitable for `MyContainer` ... + } +} +``` + +#### Expanding standard library types + +We expect that many standard library types will need to be updated to support possibly-non-escapable types, including `Optional`, `Array`, `Set`, `Dictionary`, and the `Unsafe*Pointer` family of types. + +Some of these types will require first exploring whether it is possible for the `Collection`, `Iterator`, `Sequence`, and related protocols to adopt these concepts directly or whether we will need to introduce new protocols to complement the existing ones. + +The more basic protocols such as `Equatable`, `Comparable`, and `Hashable` should be easier to update. + +#### Refining `with*` closure-taking APIs + +The `~Escapable` types can be used to refine common `with*` closure-taking APIs by ensuring that the closures cannot save or hold their arguments beyond their own lifetime. +For example, this can greatly improve the safety of locking APIs that expect to unlock resources upon completion of the closure. + +#### Non-escapable classes + +We’ve explicitly excluded class types from being non-escapable. In the future, we could allow class types to be declared non-escapable as a way to avoid most reference-counting operations on class objects. + +#### Concurrency + +Structured concurrency implies lifetime constraints similar to those outlined in this proposal. It may be appropriate to incorporate `~Escapable` into the structured concurrency primitives. + +We expect a future proposal will deal with the relationship to `TaskGroup` and other concurrency constructs. + +#### Global non-escapable types with immortal lifetimes + +This proposal currently prohibits putting values with non-escapable types into global or static variables. +We expect to eventually allow this by explicitly annotating a “static” or “immortal” lifetime. + +## Alternatives considered + +#### Require `Escapable` to indicate escapable types without using `~Escapable` + +We could avoid using `~Escapable` to mark types that lack the `Escapable` property by requiring `Escapable` on all escapable types. +However, it is infeasible to require updating all existing types in all existing Swift code with a new capability marker. + +Apart from that, we expect almost all types to continue to be escapable in the future, so the negative marker reduces the overall burden. +It is also consistent with progressive disclosure: +Most new Swift programmers should not need to know details of how escapable types work, since that is the common behavior of most data types in most programming languages. +When developers use existing non-escapable types, specific compiler error messages should guide them to correct usage without needing to have a detailed understanding of the underlying concepts. +With our current proposal, the only developers who will need detailed understanding of these concepts are library authors who want to publish non-escapable types. + +#### `NonEscapable` as a marker protocol + +We considered introducing `NonEscapable` as a marker protocol indicating that the values of this type required additional compiler checks. +With that approach, you would define a conditionally-escapable type such as `Box` above in this fashion: + +``` +// Box does not normally require additional escapability checks +struct Box { + var t: T +} + +// But if T requires additional checks, so does Box +extension Box: NonEscapable when T: NonEscapable { } +``` + +However, we felt it best to stick with the precedent set by `~Copyable`. + +#### Rely on `~Copyable` + +As part of the `BufferView` design, we considered whether it would suffice to use `~Copyable` instead of introducing a new type concept. +Andrew Trick's analysis in [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) concluded that making `BufferView` be non-copyable would not suffice to provide the full semantics we want for that type. +Further, introducing `BufferView` as `~Copyable` would actually preclude us from later expanding it to be `~Escapable`. + +The iterator example in the beginning of this document provides another motivation: +Iterators are routinely copied in order to record a particular point in a collection. +Thus we concluded that non-copyable was not the correct lifetime restriction for types of this sort, and it was worthwhile to introduce a new lifetime concept to the language. + +#### Returns and initializers + +This proposal does not by itself provide any way to initialize a non-escapable value, requiring the additional proposed lifetime dependency annotations to support that mechanism. +Since those annotations require that the lifetime of the returned value be bound to that of one of the arguments, this implies that our current proposal does not permit non-escapable types to have trivial initializers: + +``` +struct NE: ~Escapable { + init() {} // 🛑 Initializer return must depend on an argument +} +``` + +We considered introducing an annotation that would specifically allow this and related uses: + +``` +struct NE: ~Escapable { + @unsafeNonescapableReturn + init() {} // OK because of annotation +} +``` + +We omitted this annotation from our proposal because there is more than one possible interpretation of such a marker. And we did not see a compelling reason for preferring one particular interpretation because we have yet to find a use case that actually requires this. + +In particular, the use cases we’ve so far considered have all been resolvable by adding an argument specifically for the purpose of anchoring a lifetime dependency: + +``` +struct NE: ~Escapable { + // Proposed lifetime dependency notation; + // see separate proposal for details. + init(from: SomeType) -> borrow(from) Self {} +} +``` + +We expect that future experience with non-escapable types will clarify whether additional lifetime modifiers of this sort are justified. + +## Acknowledgements + +Many people discussed this proposal and gave important feedback, including: Kavon Farvardin, Meghana Gupta, John McCall, Slava Pestov, Joe Groff, and Guillaume Lessard. From 0106f6bdee58e7f3551638867a7c3bd14af5e1f0 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Thu, 1 Feb 2024 14:57:57 -0800 Subject: [PATCH 02/24] Formatting fix --- proposals/NNNN-non-escapable.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 9020686f4b..e0e8772a76 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -24,8 +24,8 @@ This feature is a key requirement for the proposed `BufferView` type. * [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) * [Roadmap for improving Swift performance predictability: ARC improvements and ownership control](https://forums.swift.org/t/a-roadmap-for-improving-swift-performance-predictability-arc-improvements-and-ownership-control/54206) * [Ownership Manifesto](https://forums.swift.org/t/manifesto-ownership/5212) -* ### TODO: Link to BufferView proposal -* ### TODO: Link to lifetime dependency annotations proposal +* **TODO: Link to BufferView proposal** +* **TODO: Link to lifetime dependency annotations proposal** ## Motivation From 78f858da0ac1153a93c16bf582a6463eb7c6027d Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Tue, 6 Feb 2024 10:07:05 -0800 Subject: [PATCH 03/24] Link to forum thread with pitch discussion --- proposals/NNNN-non-escapable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index e0e8772a76..cd6d9e9662 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -7,7 +7,7 @@ * Roadmap: [BufferView Language Requirements](https://forums.swift.org/t/roadmap-language-support-for-bufferview) * Implementation: **Pending** * Upcoming Feature Flag: `NonescapableTypes` -* Review: ([pitch](https://forums.swift.org/...)) +* Review: ([pitch](https://forums.swift.org/t/pitch-non-escapable-types-and-lifetime-dependency/69865)) ## Introduction From e06bc454b983cd53d91a02f7c5ec3df43b9814c9 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Wed, 7 Feb 2024 17:38:41 -0800 Subject: [PATCH 04/24] Expand concurrency explanation a bit --- proposals/NNNN-non-escapable.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index cd6d9e9662..8bc675a0d8 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -226,7 +226,7 @@ Returning a non-escapable value from a closure requires explicit lifetime depend All of the requirements on use of non-escapable values as function arguments and return values also apply to async functions, including those invoked via `async let`. -The closures used in `Task.init` or `Task.detached` are escaping closures and therefore cannot capture non-escapable values. +The closures used in `Task.init`, `Task.detached`, or `TaskGroup.addTask` are escaping closures and therefore cannot capture non-escapable values. ## Source compatibility @@ -299,9 +299,11 @@ We’ve explicitly excluded class types from being non-escapable. In the future #### Concurrency -Structured concurrency implies lifetime constraints similar to those outlined in this proposal. It may be appropriate to incorporate `~Escapable` into the structured concurrency primitives. +Structured concurrency implies lifetime constraints similar to those outlined in this proposal. +It may be appropriate to incorporate `~Escapable` into the structured concurrency primitives. -We expect a future proposal will deal with the relationship to `TaskGroup` and other concurrency constructs. +For example, the current `TaskGroup` type is supposed to never be escaped from the local context; +making it `~Escapable` would prevent this type of abuse and possibly enable other optimizations. #### Global non-escapable types with immortal lifetimes From 72e3c5065987cc9c8fdd9d2c1915bc1351d32c2d Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Wed, 7 Feb 2024 17:39:06 -0800 Subject: [PATCH 05/24] Clarify one of the Alternatives Considered --- proposals/NNNN-non-escapable.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 8bc675a0d8..d43c48cbb1 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -338,7 +338,10 @@ struct Box { extension Box: NonEscapable when T: NonEscapable { } ``` -However, we felt it best to stick with the precedent set by `~Copyable`. +However, this would imply that any `NonEscapable` type was a +subtype of `Any` and could therefore be placed within an `Any` existential box. +An `Any` existential box is both `Copyable` and `Escapable`, +so it cannot be allowed to contain a non-escapable value. #### Rely on `~Copyable` From e57f1e05b3644ec0075d67f42294eede3a778289 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Wed, 7 Feb 2024 17:40:14 -0800 Subject: [PATCH 06/24] Acknowledge Franz Busch --- proposals/NNNN-non-escapable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index d43c48cbb1..800a2ec390 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -389,4 +389,4 @@ We expect that future experience with non-escapable types will clarify whether a ## Acknowledgements -Many people discussed this proposal and gave important feedback, including: Kavon Farvardin, Meghana Gupta, John McCall, Slava Pestov, Joe Groff, and Guillaume Lessard. +Many people discussed this proposal and gave important feedback, including: Kavon Farvardin, Meghana Gupta, John McCall, Slava Pestov, Joe Groff, Guillaume Lessard, and Franz Busch. From 12b07c2ee8ef0310551dd9e50b449d2e50afdb4a Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Wed, 20 Mar 2024 12:37:11 -0700 Subject: [PATCH 07/24] Incorporate editorial feedback from Andrew Trick --- proposals/NNNN-non-escapable.md | 176 +++++++++++++++++--------------- 1 file changed, 93 insertions(+), 83 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 800a2ec390..2446f75558 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -1,4 +1,4 @@ -# Non-Escapable Types +# Nonescapable Types * Proposal: [SE-NNNN](NNNN-filename.md) * Authors: [Andrew Trick](https://github.com/atrick), [Tim Kientzle](https://github.com/tbkka) @@ -16,7 +16,7 @@ This complements the `~Copyable` types added with SE-0390 by introducing another In addition, these types will support lifetime-dependency constraints (being tracked in a separate proposal), that allow them to safely hold pointers referring to data stored in other types. -This feature is a key requirement for the proposed `BufferView` type. +This feature is a key requirement for the proposed `StorageView` type. **See Also** @@ -24,8 +24,8 @@ This feature is a key requirement for the proposed `BufferView` type. * [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) * [Roadmap for improving Swift performance predictability: ARC improvements and ownership control](https://forums.swift.org/t/a-roadmap-for-improving-swift-performance-predictability-arc-improvements-and-ownership-control/54206) * [Ownership Manifesto](https://forums.swift.org/t/manifesto-ownership/5212) -* **TODO: Link to BufferView proposal** -* **TODO: Link to lifetime dependency annotations proposal** +* [Draft StorageView Proposal](https://github.com/apple/swift-evolution/pull/2307) +* [Draft Lifetime Dependency Annotations Proposal](https://github.com/apple/swift-evolution/pull/2305) ## Motivation @@ -44,13 +44,13 @@ Currently, the notion of "escapability" appears in the Swift language as a featu Closures that are declared as `@nonescapable` can use a very efficient stack-based representation; closures that are `@escapable` store their state on the heap. -By allowing Swift developers to mark various types as non-escapable, we provide a mechanism for them to opt into a specific set of usage limitations that: +By allowing Swift developers to mark various types as nonescapable, we provide a mechanism for them to opt into a specific set of usage limitations that: * Can be automatically verified by the compiler. In fact, the Swift compiler internally already makes heavy use of escapability as a concept. * Are strict enough to permit high performance. The compiler uses this concept precisely because values that do not escape can be managed much more efficiently. * Do not interfere with common uses of these types. -For example, if an iterator type were marked as non-escapable, the compiler would produce an error message whenever the user of that type tried to copy or store the value in a way that might limit efficient operation. +For example, if an iterator type were marked as nonescapable, the compiler would produce an error message whenever the user of that type tried to copy or store the value in a way that might limit efficient operation. These checks would not significantly reduce the usefulness of iterators which are almost always created, used, and destroyed in a single local context. These checks would also still allow local copies for multi-iterator uses, with the same constraints applied to those copies as well. @@ -64,25 +64,30 @@ We are not at this time proposing any changes to Swift's current `Iterator` prot #### New Escapable Concept -We add a new type constraint `Escapable` to the standard library and implicitly apply it to all current Swift types (with the sole exception of `@nonescapable` closures). +We add a new suppressible type constraint `Escapable` to the standard library and implicitly apply it to all current Swift types (with the sole exception of `@nonescapable` closures). `Escapable` types can be assigned to global variables, passed into arbitrary functions, or returned from the current function or closure. This matches the existing semantics of all Swift types prior to this proposal. Specifically, we will add this declaration to the standard library: -``` +```swift // An Escapable type may or may not be Copyable protocol Escapable: ~Copyable {} ``` -#### In concrete contexts, `~Escapable` indicates non-escapability +#### In concrete contexts, `~Escapable` indicates nonescapability Using the same approach as used for `~Copyable` and `Copyable`, we use `~Escapable` to indicate the lack of the `Escapable` attribute on a type. -``` +```swift // Example: A type that is not escapable -struct NotEscapable: ~Escapable { } +struct NotEscapable: ~Escapable { + ... +} +``` +A nonescapable type is not allowed to escape the local context: +```swift // Example: Basic limits on ~Escapable types func f() -> NotEscapable { let ne = NotEscapable() @@ -93,9 +98,12 @@ func f() -> NotEscapable { } ``` +**Note**: The inability to return a nonescapable type has implications for how initializers must be written. +The section "Returned nonescapable values require lifetime dependency" has more details. + Without a `~Escapable` marker, the default for any type is to be escapable. Since `~Escapable` indicates the lack of a capability, you cannot put this in an extension. -``` +```swift // Example: Escapable by default struct Ordinary { } extension Ordinary: ~Escapable // 🛑 Extensions cannot remove a capability @@ -109,23 +117,23 @@ When used in a generic context, `~Escapable` allows you to define functions or t That is, `~Escapable` indicates the lack of an escapable requirement. Since the values might not be escapable, the compiler must conservatively prevent the values from escaping: -``` +```swift // Example: In generic contexts, ~Escapable is // the lack of an Escapable requirement. func f(_ value: MaybeEscapable) { // `value` might or might not be Escapable - globalVar = value // 🛑 Cannot assign possibly-non-escapable type to a global var + globalVar = value // 🛑 Cannot assign possibly-nonescapable type to a global var } -f(NotEscapable()) // Ok to call with non-escapable argument +f(NotEscapable()) // Ok to call with nonescapable argument f(7) // Ok to call with escapable argument ``` This also permits the definition of types whose escapability varies depending on their generic arguments. As with other conditional behaviors, this is expressed by using an extension to conditionally add a new capability to the type: -``` +```swift // Example: Conditionally Escapable generic type -// By default, Box is itself non-escapable +// By default, Box is itself nonescapable struct Box: ~Escapable { var t: T } @@ -135,15 +143,18 @@ struct Box: ~Escapable { extension Box: Escapable when T: Escapable { } ``` +[SE-0427 Noncopyable Generics](https://github.com/apple/swift-evolution/blob/main/proposals/0427-noncopyable-generics.md) provides more detail on +how suppressible protocols such as `Escapable` are handled in the generic type system. + **Note:** There is no relationship between `Copyable` and `Escapable`. -Copyable or non-copyable types can be escapable or non-escapable. +Copyable or noncopyable types can be escapable or nonescapable. -#### Constraints on non-escapable local variables +#### Constraints on nonescapable local variables -A non-escapable value can be freely copied and passed into other functions as long as the usage can guarantee that the value does not persist beyond the current scope: +A nonescapable value can be freely copied and passed into other functions as long as the usage can guarantee that the value does not persist beyond the current scope: -``` -// Example: Local variable with non-escapable type +```swift +// Example: Local variable with nonescapable type func borrowingFunc(_: borrowing NotEscapable) { ... } func consumingFunc(_: consuming NotEscapable) { ... } func inoutFunc(_: inout NotEscapable) { ... } @@ -163,70 +174,69 @@ func f() { } ``` -#### Constraints on non-escapable arguments +#### Constraints on nonescapable arguments -A value of non-escapable type received as an argument is subject to the same constraints as any other local variable. -In particular, a `consuming` argument (and all direct copies thereof) must actually be destroyed during the execution of the function. -This is in contrast to an escaping `consuming` argument which can be disposed of by being stored in a global or static variable. +A value of nonescapable type received as an argument is subject to the same constraints as any other local variable. +In particular, a nonescapable `consuming` argument (and all direct copies thereof) must actually be destroyed during the execution of the function. +This is in contrast to an _escapable_ `consuming` argument which can be disposed of by being returned or stored to an instance property or global variable. -#### Values that contain non-escapable values must be non-escapable +#### Values that contain nonescapable values must be nonescapable -Stored struct properties and enum payloads can have non-escapable types if the surrounding type is itself non-escapable. +Stored struct properties and enum payloads can have nonescapable types if the surrounding type is itself nonescapable. (Equivalently, an escapable struct or enum can only contain escapable values.) -Non-escapable values cannot be stored as class properties, since classes are always inherently escaping. +Nonescapable values cannot be stored as class properties, since classes are always inherently escaping. -``` +```swift // Example struct OuterEscapable { - // 🛑 Escapable struct cannot have non-escapable stored property - var nonesc: NonEscapable + // 🛑 Escapable struct cannot have nonescapable stored property + var nonesc: Nonescapable } enum EscapableEnum { - // 🛑 Escapable enum cannot have a non-escapable payload - case nonesc(NonEscapable) + // 🛑 Escapable enum cannot have a nonescapable payload + case nonesc(Nonescapable) } -struct OuterNonEscapable: ~Escapable { - var nonesc: NonEscapable // OK +struct OuterNonescapable: ~Escapable { + var nonesc: Nonescapable // OK } -enum NonEscapableEnum: ~Escapable { - case nonesc(NonEscapable) // OK +enum NonescapableEnum: ~Escapable { + case nonesc(Nonescapable) // OK } ``` -#### Returned non-escapable values require lifetime dependency - -A simple return of a non-escapable value is not permitted. +#### Returned nonescapable values require lifetime dependency -``` -func f() -> NotEscapable { // 🛑 Cannot return a non-escapable type +As mentioned earlier, a simple return of a nonescapable value is not permitted: +```swift +func f() -> NotEscapable { // 🛑 Cannot return a nonescapable type var value: NotEscapable - return value // 🛑 Cannot return a non-escapable type + return value // 🛑 Cannot return a nonescapable type } ``` -A separate proposal describes “lifetime dependency annotations” that can relax this requirement by tying the lifetime of the returned value to the lifetime of some other object, either an argument to the function or `self` in the case of a method or computed property returning a non-escapable type. +A separate proposal describes “lifetime dependency annotations” that can relax this requirement by tying the lifetime of the returned value to the lifetime of some other object, either an argument to the function or `self` in the case of a method or computed property returning a nonescapable type. In particular, struct and enum initializers (which build a new value and return it to the caller) cannot be written without some mechanism similar to that outlined in our companion proposal. -#### Globals and static variables cannot be non-escapable +#### Globals and static variables cannot be nonescapable -Non-escapable values must be constrained to some specific local execution context. +Nonescapable values must be constrained to some specific local execution context. This implies that they cannot be stored in global or static variables. -#### Closures and non-escapable values +#### Closures and nonescapable values -Escaping closures cannot capture non-escapable values. -Non-escaping closures can capture non-escapable values subject only to the usual exclusivity restrictions. +Escaping closures cannot capture nonescapable values. +Nonescaping closures can capture nonescapable values subject only to the usual exclusivity restrictions. -Returning a non-escapable value from a closure requires explicit lifetime dependency annotations, as covered in the companion proposal. +Returning a nonescapable value from a closure requires explicit lifetime dependency annotations, as covered in the companion proposal. -#### Non-escapable values and concurrency +#### Nonescapable values and concurrency -All of the requirements on use of non-escapable values as function arguments and return values also apply to async functions, including those invoked via `async let`. +All of the requirements on use of nonescapable values as function arguments and return values also apply to async functions, including those invoked via `async let`. -The closures used in `Task.init`, `Task.detached`, or `TaskGroup.addTask` are escaping closures and therefore cannot capture non-escapable values. +The closures used in `Task.init`, `Task.detached`, or `TaskGroup.addTask` are escaping closures and therefore cannot capture nonescapable values. ## Source compatibility @@ -251,26 +261,26 @@ Similarly, an old compiler reading a new interface will have no problems as long These same considerations ensure that escapable types can be shared between previously-compiled code and newly-compiled code. -Retrofitting existing generic types so they can support both escapable and non-escapable type arguments is possible with care. +Retrofitting existing generic types so they can support both escapable and nonescapable type arguments is possible with care. ## Future directions -#### `BufferView` type +#### `StorageView` type -This proposal is being driven in large part by the needs of the `BufferView` type that has been discussed elsewhere. +This proposal is being driven in large part by the needs of the `StorageView` type that has been discussed elsewhere. Briefly, this type would provide an efficient universal “view” of array-like data stored in contiguous memory. Since values of this type do not own any data but only refer to data stored elsewhere, their lifetime must be limited to not exceed that of the owning storage. We expect to publish a sample implementation and proposal for that type very soon. #### Lifetime dependency annotations -Non-escapable types have a set of inherent restrictions on how they can be passed as arguments, stored in variables, or returned from functions. +Nonescapable types have a set of inherent restrictions on how they can be passed as arguments, stored in variables, or returned from functions. A companion proposal builds on this by supporting more detailed annotations that link the lifetimes of different objects. This would allow, for example, a container to vend an iterator value that held a direct unmanaged pointer to the container's contents. The lifetime dependency would ensure that such an iterator could not outlive the container to whose contents it referred. -``` -// Example: Non-escaping iterator +```swift +// Example: Nonescaping iterator struct NEIterator { // `borrow(container)` indicates that the constructed value // cannot outlive the `container` argument. @@ -282,7 +292,7 @@ struct NEIterator { #### Expanding standard library types -We expect that many standard library types will need to be updated to support possibly-non-escapable types, including `Optional`, `Array`, `Set`, `Dictionary`, and the `Unsafe*Pointer` family of types. +We expect that many standard library types will need to be updated to support possibly-nonescapable types, including `Optional`, `Array`, `Set`, `Dictionary`, and the `Unsafe*Pointer` family of types. Some of these types will require first exploring whether it is possible for the `Collection`, `Iterator`, `Sequence`, and related protocols to adopt these concepts directly or whether we will need to introduce new protocols to complement the existing ones. @@ -293,9 +303,9 @@ The more basic protocols such as `Equatable`, `Comparable`, and `Hashable` shoul The `~Escapable` types can be used to refine common `with*` closure-taking APIs by ensuring that the closures cannot save or hold their arguments beyond their own lifetime. For example, this can greatly improve the safety of locking APIs that expect to unlock resources upon completion of the closure. -#### Non-escapable classes +#### Nonescapable classes -We’ve explicitly excluded class types from being non-escapable. In the future, we could allow class types to be declared non-escapable as a way to avoid most reference-counting operations on class objects. +We’ve explicitly excluded class types from being nonescapable. In the future, we could allow class types to be declared nonescapable as a way to avoid most reference-counting operations on class objects. #### Concurrency @@ -305,9 +315,9 @@ It may be appropriate to incorporate `~Escapable` into the structured concurrenc For example, the current `TaskGroup` type is supposed to never be escaped from the local context; making it `~Escapable` would prevent this type of abuse and possibly enable other optimizations. -#### Global non-escapable types with immortal lifetimes +#### Global nonescapable types with immortal lifetimes -This proposal currently prohibits putting values with non-escapable types into global or static variables. +This proposal currently prohibits putting values with nonescapable types into global or static variables. We expect to eventually allow this by explicitly annotating a “static” or “immortal” lifetime. ## Alternatives considered @@ -320,34 +330,34 @@ However, it is infeasible to require updating all existing types in all existing Apart from that, we expect almost all types to continue to be escapable in the future, so the negative marker reduces the overall burden. It is also consistent with progressive disclosure: Most new Swift programmers should not need to know details of how escapable types work, since that is the common behavior of most data types in most programming languages. -When developers use existing non-escapable types, specific compiler error messages should guide them to correct usage without needing to have a detailed understanding of the underlying concepts. -With our current proposal, the only developers who will need detailed understanding of these concepts are library authors who want to publish non-escapable types. +When developers use existing nonescapable types, specific compiler error messages should guide them to correct usage without needing to have a detailed understanding of the underlying concepts. +With our current proposal, the only developers who will need detailed understanding of these concepts are library authors who want to publish nonescapable types. -#### `NonEscapable` as a marker protocol +#### `Nonescapable` as a marker protocol -We considered introducing `NonEscapable` as a marker protocol indicating that the values of this type required additional compiler checks. +We considered introducing `Nonescapable` as a marker protocol indicating that the values of this type required additional compiler checks. With that approach, you would define a conditionally-escapable type such as `Box` above in this fashion: -``` +```swift // Box does not normally require additional escapability checks struct Box { var t: T } // But if T requires additional checks, so does Box -extension Box: NonEscapable when T: NonEscapable { } +extension Box: Nonescapable when T: Nonescapable { } ``` -However, this would imply that any `NonEscapable` type was a +However, this would imply that any `Nonescapable` type was a subtype of `Any` and could therefore be placed within an `Any` existential box. An `Any` existential box is both `Copyable` and `Escapable`, -so it cannot be allowed to contain a non-escapable value. +so it cannot be allowed to contain a nonescapable value. #### Rely on `~Copyable` -As part of the `BufferView` design, we considered whether it would suffice to use `~Copyable` instead of introducing a new type concept. -Andrew Trick's analysis in [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) concluded that making `BufferView` be non-copyable would not suffice to provide the full semantics we want for that type. -Further, introducing `BufferView` as `~Copyable` would actually preclude us from later expanding it to be `~Escapable`. +As part of the `StorageView` design, we considered whether it would suffice to use `~Copyable` instead of introducing a new type concept. +Andrew Trick's analysis in [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) concluded that making `StorageView` be non-copyable would not suffice to provide the full semantics we want for that type. +Further, introducing `StorageView` as `~Copyable` would actually preclude us from later expanding it to be `~Escapable`. The iterator example in the beginning of this document provides another motivation: Iterators are routinely copied in order to record a particular point in a collection. @@ -355,10 +365,10 @@ Thus we concluded that non-copyable was not the correct lifetime restriction for #### Returns and initializers -This proposal does not by itself provide any way to initialize a non-escapable value, requiring the additional proposed lifetime dependency annotations to support that mechanism. -Since those annotations require that the lifetime of the returned value be bound to that of one of the arguments, this implies that our current proposal does not permit non-escapable types to have trivial initializers: +This proposal does not by itself provide any way to initialize a nonescapable value, requiring the additional proposed lifetime dependency annotations to support that mechanism. +Since those annotations require that the lifetime of the returned value be bound to that of one of the arguments, this implies that our current proposal does not permit nonescapable types to have trivial initializers: -``` +```swift struct NE: ~Escapable { init() {} // 🛑 Initializer return must depend on an argument } @@ -366,9 +376,9 @@ struct NE: ~Escapable { We considered introducing an annotation that would specifically allow this and related uses: -``` +```swift struct NE: ~Escapable { - @unsafeNonescapableReturn + @_unsafeNonescapableResult init() {} // OK because of annotation } ``` @@ -377,7 +387,7 @@ We omitted this annotation from our proposal because there is more than one poss In particular, the use cases we’ve so far considered have all been resolvable by adding an argument specifically for the purpose of anchoring a lifetime dependency: -``` +```swift struct NE: ~Escapable { // Proposed lifetime dependency notation; // see separate proposal for details. @@ -385,7 +395,7 @@ struct NE: ~Escapable { } ``` -We expect that future experience with non-escapable types will clarify whether additional lifetime modifiers of this sort are justified. +We expect that future experience with nonescapable types will clarify whether additional lifetime modifiers of this sort are justified. ## Acknowledgements From 54999f0e164cbb61ad29e4725a071f8cea8dcd91 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Thu, 21 Mar 2024 12:37:55 -0700 Subject: [PATCH 08/24] New subsection on conditional escapability; many small edits --- proposals/NNNN-non-escapable.md | 96 +++++++++++++++++++++++---------- 1 file changed, 67 insertions(+), 29 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 2446f75558..87a6a6639f 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -41,8 +41,8 @@ In addition, the use of reference counting to ensure correctness at runtime make ## Proposed solution Currently, the notion of "escapability" appears in the Swift language as a feature of closures. -Closures that are declared as `@nonescapable` can use a very efficient stack-based representation; -closures that are `@escapable` store their state on the heap. +Nonescapable closures can use a very efficient stack-based representation; +closures that are `@escapable` store their captures on the heap. By allowing Swift developers to mark various types as nonescapable, we provide a mechanism for them to opt into a specific set of usage limitations that: @@ -64,7 +64,7 @@ We are not at this time proposing any changes to Swift's current `Iterator` prot #### New Escapable Concept -We add a new suppressible type constraint `Escapable` to the standard library and implicitly apply it to all current Swift types (with the sole exception of `@nonescapable` closures). +We add a new suppressible protocol `Escapable` to the standard library and implicitly apply it to all current Swift types (with the sole exception of nonescapable closures). `Escapable` types can be assigned to global variables, passed into arbitrary functions, or returned from the current function or closure. This matches the existing semantics of all Swift types prior to this proposal. @@ -77,7 +77,7 @@ protocol Escapable: ~Copyable {} #### In concrete contexts, `~Escapable` indicates nonescapability -Using the same approach as used for `~Copyable` and `Copyable`, we use `~Escapable` to indicate the lack of the `Escapable` attribute on a type. +Using the same approach as used for `~Copyable` and `Copyable`, we use `~Escapable` to suppress the `Escapable` conformance on a type. ```swift // Example: A type that is not escapable @@ -86,7 +86,7 @@ struct NotEscapable: ~Escapable { } ``` -A nonescapable type is not allowed to escape the local context: +A nonescapable value is not allowed to escape the local context: ```swift // Example: Basic limits on ~Escapable types func f() -> NotEscapable { @@ -98,10 +98,10 @@ func f() -> NotEscapable { } ``` -**Note**: The inability to return a nonescapable type has implications for how initializers must be written. -The section "Returned nonescapable values require lifetime dependency" has more details. +**Note**: +The section "Returned nonescapable values require lifetime dependency" explains the implications for how you must write initializers. -Without a `~Escapable` marker, the default for any type is to be escapable. Since `~Escapable` indicates the lack of a capability, you cannot put this in an extension. +Without `~Escapable`, the default for any type is to be escapable. Since `~Escapable` suppresses a capability, you cannot put this in an extension. ```swift // Example: Escapable by default @@ -111,15 +111,13 @@ extension Ordinary: ~Escapable // 🛑 Extensions cannot remove a capability Classes cannot be declared `~Escapable`. -#### In generic contexts, `~Escapable` marks the lack of an Escapable requirement +#### In generic contexts, `~Escapable` suppresses the default Escapable requirement When used in a generic context, `~Escapable` allows you to define functions or types that can work with values that might or might not be escapable. -That is, `~Escapable` indicates the lack of an escapable requirement. +That is, `~Escapable` indicates the default escapable requirement has been suppressed. Since the values might not be escapable, the compiler must conservatively prevent the values from escaping: ```swift -// Example: In generic contexts, ~Escapable is -// the lack of an Escapable requirement. func f(_ value: MaybeEscapable) { // `value` might or might not be Escapable globalVar = value // 🛑 Cannot assign possibly-nonescapable type to a global var @@ -128,21 +126,6 @@ f(NotEscapable()) // Ok to call with nonescapable argument f(7) // Ok to call with escapable argument ``` -This also permits the definition of types whose escapability varies depending on their generic arguments. -As with other conditional behaviors, this is expressed by using an extension to conditionally add a new capability to the type: - -```swift -// Example: Conditionally Escapable generic type -// By default, Box is itself nonescapable -struct Box: ~Escapable { - var t: T -} - -// Box gains the ability to escape whenever its -// generic argument is Escapable -extension Box: Escapable when T: Escapable { } -``` - [SE-0427 Noncopyable Generics](https://github.com/apple/swift-evolution/blob/main/proposals/0427-noncopyable-generics.md) provides more detail on how suppressible protocols such as `Escapable` are handled in the generic type system. @@ -238,9 +221,64 @@ All of the requirements on use of nonescapable values as function arguments and The closures used in `Task.init`, `Task.detached`, or `TaskGroup.addTask` are escaping closures and therefore cannot capture nonescapable values. +#### Conditionally `Escapable` types + +You can define types whose escapability varies depending on their generic arguments. +As with other conditional behaviors, this is expressed by using an extension to conditionally add a new capability to the type: + +```swift +// Example: Conditionally Escapable generic type +// By default, Box is itself nonescapable +struct Box: ~Escapable { + var t: T +} + +// Box gains the ability to escape whenever its +// generic argument is Escapable +extension Box: Escapable when T: Escapable { } +``` + +This can be used in conjunction with other suppressible protocols. +For example, many general library types will need to be copyable and/or escapable following their contents. +Here's a compact way to declare such a type: +```swift +struct Wrapper { ... } +extension Wrapper: Copyable where T: ~Escapable {} +extension Wrapper: Escapable where T: ~Copyable {} +``` + +The above declarations all in a single source file will result in a type `Wrapper` that is `Escapable` exactly when `T` is `Escapable` and `Copyable` exactly when `T` is `Copyable`. +To see why, first note that the explicit `extension Wrapper: Escapable` in the same source file implies that the original `struct Wrapper` must be `~Escapable` and similarly for `Copyable`, exactly as if the first line had been +```swift +struct Wrapper: ~Copyable & ~Escapable {} +``` + +Now recall from SE-427 that suppressible protocols must be explicitly suppressed on type parameters in extensions. +This means that +```swift +extension Wrapper: Copyable where T: ~Escapable {} +``` +is exactly the same as +```swift +extension Wrapper: Copyable where T: Copyable & ~Escapable {} +``` +which implies that `Wrapper` becomes `Copyable` when `T` is `Copyable`. +Finally, remember that `~Escapable` means that `Escapable` is not required, so +this condition on `Copyable` applies regardless of whether `T` is `Escapable` or not. + +Similarly, +```swift +extension Wrapper: Escapable where T: ~Copyable {} +``` +is exactly the same as +```swift +extension Wrapper: Escapable where T: Escapable & ~Copyable {} +``` +which means that `Wrapper` is `Escapable` whenever `T` is `Escapable` regardless of whether `T` is `Copyable` or not. + ## Source compatibility -The compiler will treat any type without an explicit `~Escapable` marker as escapable. +The compiler will treat any type without explicit `~Escapable` as escapable. This matches the current behavior of the language. Only when new types are marked as `~Escapable` does this have any impact. @@ -325,7 +363,7 @@ We expect to eventually allow this by explicitly annotating a “static” or #### Require `Escapable` to indicate escapable types without using `~Escapable` We could avoid using `~Escapable` to mark types that lack the `Escapable` property by requiring `Escapable` on all escapable types. -However, it is infeasible to require updating all existing types in all existing Swift code with a new capability marker. +However, it is infeasible to require updating all existing types in all existing Swift code with a new explicit capability. Apart from that, we expect almost all types to continue to be escapable in the future, so the negative marker reduces the overall burden. It is also consistent with progressive disclosure: From cfb2dedacb0432d26b2e5c920f27b4dae2c8b15c Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Thu, 21 Mar 2024 13:32:53 -0700 Subject: [PATCH 09/24] Mention async/throwing --- proposals/NNNN-non-escapable.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 87a6a6639f..d096b94d4d 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -134,19 +134,21 @@ Copyable or noncopyable types can be escapable or nonescapable. #### Constraints on nonescapable local variables -A nonescapable value can be freely copied and passed into other functions as long as the usage can guarantee that the value does not persist beyond the current scope: +A nonescapable value can be freely copied and passed into other functions, including async and throwing functions, as long as the usage can guarantee that the value does not persist beyond the current scope: ```swift // Example: Local variable with nonescapable type func borrowingFunc(_: borrowing NotEscapable) { ... } func consumingFunc(_: consuming NotEscapable) { ... } func inoutFunc(_: inout NotEscapable) { ... } +func asyncBorrowingFunc(_: borrowing NotEscapable) async -> ResultType { ... } func f() { var value: NotEscapable let copy = value // OK to copy as long as copy does not escape globalVar = value // 🛑 May not assign to global SomeType.staticVar = value // 🛑 May not assign to static var + async let r = asyncBorrowingFunc(value) // OK to pass borrowing borrowingFunc(value) // OK to pass borrowing inoutFunc(&value) // OK to pass inout consumingFunc(value) // OK to pass consuming From 0c6825b46951f29794261384edd35f63fe0b4c15 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Tue, 18 Jun 2024 16:31:08 -0700 Subject: [PATCH 10/24] Editorial comments --- proposals/NNNN-non-escapable.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index d096b94d4d..7f7110f3b0 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -237,7 +237,7 @@ struct Box: ~Escapable { // Box gains the ability to escape whenever its // generic argument is Escapable -extension Box: Escapable when T: Escapable { } +extension Box: Escapable where T: Escapable { } ``` This can be used in conjunction with other suppressible protocols. @@ -322,9 +322,9 @@ The lifetime dependency would ensure that such an iterator could not outlive the ```swift // Example: Nonescaping iterator struct NEIterator { - // `borrow(container)` indicates that the constructed value + // `dependsOn(container)` indicates that the constructed value // cannot outlive the `container` argument. - init(over container: MyContainer) -> borrow(container) Self { + init(over container: MyContainer) -> dependsOn(container) Self { ... initialize an iterator suitable for `MyContainer` ... } } @@ -385,7 +385,7 @@ struct Box { } // But if T requires additional checks, so does Box -extension Box: Nonescapable when T: Nonescapable { } +extension Box: Nonescapable where T: Nonescapable { } ``` However, this would imply that any `Nonescapable` type was a @@ -431,7 +431,7 @@ In particular, the use cases we’ve so far considered have all been resolvable struct NE: ~Escapable { // Proposed lifetime dependency notation; // see separate proposal for details. - init(from: SomeType) -> borrow(from) Self {} + init(from: SomeType) -> dependsOn(from) Self {} } ``` From 88329b13e4e68bf2e007f4dc5ed663efad0a19a5 Mon Sep 17 00:00:00 2001 From: Guillaume Lessard Date: Thu, 22 Aug 2024 08:52:47 -0700 Subject: [PATCH 11/24] StorageView -> Span --- proposals/NNNN-non-escapable.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 7f7110f3b0..9e2530da6b 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -16,7 +16,7 @@ This complements the `~Copyable` types added with SE-0390 by introducing another In addition, these types will support lifetime-dependency constraints (being tracked in a separate proposal), that allow them to safely hold pointers referring to data stored in other types. -This feature is a key requirement for the proposed `StorageView` type. +This feature is a key requirement for the proposed `Span` type. **See Also** @@ -24,7 +24,7 @@ This feature is a key requirement for the proposed `StorageView` type. * [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) * [Roadmap for improving Swift performance predictability: ARC improvements and ownership control](https://forums.swift.org/t/a-roadmap-for-improving-swift-performance-predictability-arc-improvements-and-ownership-control/54206) * [Ownership Manifesto](https://forums.swift.org/t/manifesto-ownership/5212) -* [Draft StorageView Proposal](https://github.com/apple/swift-evolution/pull/2307) +* [Draft Span Proposal](https://github.com/apple/swift-evolution/pull/2307) * [Draft Lifetime Dependency Annotations Proposal](https://github.com/apple/swift-evolution/pull/2305) ## Motivation @@ -305,9 +305,9 @@ Retrofitting existing generic types so they can support both escapable and nones ## Future directions -#### `StorageView` type +#### `Span` type -This proposal is being driven in large part by the needs of the `StorageView` type that has been discussed elsewhere. +This proposal is being driven in large part by the needs of the `Span` type that has been discussed elsewhere. Briefly, this type would provide an efficient universal “view” of array-like data stored in contiguous memory. Since values of this type do not own any data but only refer to data stored elsewhere, their lifetime must be limited to not exceed that of the owning storage. We expect to publish a sample implementation and proposal for that type very soon. @@ -395,9 +395,9 @@ so it cannot be allowed to contain a nonescapable value. #### Rely on `~Copyable` -As part of the `StorageView` design, we considered whether it would suffice to use `~Copyable` instead of introducing a new type concept. -Andrew Trick's analysis in [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) concluded that making `StorageView` be non-copyable would not suffice to provide the full semantics we want for that type. -Further, introducing `StorageView` as `~Copyable` would actually preclude us from later expanding it to be `~Escapable`. +As part of the `Span` design, we considered whether it would suffice to use `~Copyable` instead of introducing a new type concept. +Andrew Trick's analysis in [Language Support for Bufferview](https://forums.swift.org/t/roadmap-language-support-for-bufferview/66211) concluded that making `Span` be non-copyable would not suffice to provide the full semantics we want for that type. +Further, introducing `Span` as `~Copyable` would actually preclude us from later expanding it to be `~Escapable`. The iterator example in the beginning of this document provides another motivation: Iterators are routinely copied in order to record a particular point in a collection. From 376201974d0143ec9f84c7a41ceab57e3c5bc141 Mon Sep 17 00:00:00 2001 From: Guillaume Lessard Date: Thu, 22 Aug 2024 13:58:00 -0700 Subject: [PATCH 12/24] edits in motivation --- proposals/NNNN-non-escapable.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 9e2530da6b..8219ef49b4 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -30,9 +30,9 @@ This feature is a key requirement for the proposed `Span` type. ## Motivation Swift's current notion of an "iterator" has several weaknesses that become apparent when you try to use it in extremely performance-constrained environments. -These weaknesses arise from the desire to ensure safety while simultaneously allowing iterator values to be arbitrarily copied to support multi-iterator algorithms. +These weaknesses arise from the desire to ensure safety while simultaneously allowing iterator values to be arbitrarily copied in support of multi-iterator algorithms. -For example, the standard library iterator for Array logically creates a copy of the Array when it is constructed; this ensures that changes to the array cannot affect the iteration. +For example, the standard library iterator for Array logically creates a copy of the Array when it is initialized; this ensures that changes to the array cannot affect the iteration. This is implemented by having the iterator store a reference-counted pointer to the array storage in order to ensure that the storage cannot be freed while the iterator is active. These safety checks all incur runtime overhead. From 9853602f634573b5bdf68171ea6ded725d4cd89b Mon Sep 17 00:00:00 2001 From: Guillaume Lessard Date: Thu, 22 Aug 2024 16:19:06 -0700 Subject: [PATCH 13/24] =?UTF-8?q?don=E2=80=99t=20rely=20on=20pseudo=20code?= =?UTF-8?q?=20as=20exposition?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- proposals/NNNN-non-escapable.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 8219ef49b4..93bdd49bc3 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -87,6 +87,10 @@ struct NotEscapable: ~Escapable { ``` A nonescapable value is not allowed to escape the local context: + +- It cannot be assigned to a binding in a larger scope +- It cannot be returned from the current scope + ```swift // Example: Basic limits on ~Escapable types func f() -> NotEscapable { @@ -99,7 +103,6 @@ func f() -> NotEscapable { ``` **Note**: -The section "Returned nonescapable values require lifetime dependency" explains the implications for how you must write initializers. Without `~Escapable`, the default for any type is to be escapable. Since `~Escapable` suppresses a capability, you cannot put this in an extension. From c058e570ffaece42a589f073b064552a16dcc4b5 Mon Sep 17 00:00:00 2001 From: Guillaume Lessard Date: Thu, 22 Aug 2024 16:19:38 -0700 Subject: [PATCH 14/24] =?UTF-8?q?add=20a=20link=20to=20the=20=E2=80=9Cretu?= =?UTF-8?q?rned=20values=E2=80=9D=20subsection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- proposals/NNNN-non-escapable.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 93bdd49bc3..a729ac511a 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -103,8 +103,9 @@ func f() -> NotEscapable { ``` **Note**: +The section ["Returned nonescapable values require lifetime dependency"](#Returns) explains the implications for how you must write initializers. -Without `~Escapable`, the default for any type is to be escapable. Since `~Escapable` suppresses a capability, you cannot put this in an extension. +Without `~Escapable`, the default for any type is to be escapable. Since `~Escapable` suppresses a capability, you cannot declare it with an extension. ```swift // Example: Escapable by default @@ -195,7 +196,7 @@ enum NonescapableEnum: ~Escapable { } ``` -#### Returned nonescapable values require lifetime dependency +#### Returned nonescapable values require lifetime dependency As mentioned earlier, a simple return of a nonescapable value is not permitted: ```swift From 4b55adbf1dbad0ce7ac0196066630295c85fc742 Mon Sep 17 00:00:00 2001 From: Guillaume Lessard Date: Thu, 22 Aug 2024 16:20:30 -0700 Subject: [PATCH 15/24] make example names more consistent --- proposals/NNNN-non-escapable.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index a729ac511a..9bfb842514 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -177,7 +177,7 @@ Nonescapable values cannot be stored as class properties, since classes are alwa ```swift // Example -struct OuterEscapable { +struct EscapableStruct { // 🛑 Escapable struct cannot have nonescapable stored property var nonesc: Nonescapable } @@ -187,7 +187,7 @@ enum EscapableEnum { case nonesc(Nonescapable) } -struct OuterNonescapable: ~Escapable { +struct NonescapableStruct: ~Escapable { var nonesc: Nonescapable // OK } From f2faa8964d2498969e184e6a3a768175543855e5 Mon Sep 17 00:00:00 2001 From: Guillaume Lessard Date: Thu, 22 Aug 2024 16:25:12 -0700 Subject: [PATCH 16/24] small tweaks --- proposals/NNNN-non-escapable.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 9bfb842514..b0f48d1e80 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -172,7 +172,7 @@ This is in contrast to an _escapable_ `consuming` argument which can be disposed #### Values that contain nonescapable values must be nonescapable Stored struct properties and enum payloads can have nonescapable types if the surrounding type is itself nonescapable. -(Equivalently, an escapable struct or enum can only contain escapable values.) +Equivalently, an escapable struct or enum can only contain escapable values. Nonescapable values cannot be stored as class properties, since classes are always inherently escaping. ```swift @@ -217,7 +217,7 @@ This implies that they cannot be stored in global or static variables. #### Closures and nonescapable values Escaping closures cannot capture nonescapable values. -Nonescaping closures can capture nonescapable values subject only to the usual exclusivity restrictions. +Nonescaping closures can capture nonescapable values subject to the usual exclusivity restrictions. Returning a nonescapable value from a closure requires explicit lifetime dependency annotations, as covered in the companion proposal. @@ -245,7 +245,7 @@ extension Box: Escapable where T: Escapable { } ``` This can be used in conjunction with other suppressible protocols. -For example, many general library types will need to be copyable and/or escapable following their contents. +For example, many general library container types will need to be copyable and/or escapable according to their contents. Here's a compact way to declare such a type: ```swift struct Wrapper { ... } From 3656dcc8d7b215e2081739de8367d75bf5c856a5 Mon Sep 17 00:00:00 2001 From: Guillaume Lessard Date: Thu, 22 Aug 2024 16:27:05 -0700 Subject: [PATCH 17/24] clean up argument vs. parameter, value vs. type, value vs. binding - I may be wrong on some of these, but this needs to be consistent with Swift documentation. --- proposals/NNNN-non-escapable.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index b0f48d1e80..0e68da97e9 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -163,13 +163,13 @@ func f() { } ``` -#### Constraints on nonescapable arguments +#### Constraints on nonescapable parameters -A value of nonescapable type received as an argument is subject to the same constraints as any other local variable. -In particular, a nonescapable `consuming` argument (and all direct copies thereof) must actually be destroyed during the execution of the function. -This is in contrast to an _escapable_ `consuming` argument which can be disposed of by being returned or stored to an instance property or global variable. +A value of nonescapable type received as an parameter is subject to the same constraints as any other local variable. +In particular, a nonescapable `consuming` parameter (and all direct copies thereof) must actually be destroyed during the execution of the function. +This is in contrast to an _escapable_ `consuming` parameter which can be disposed of by being returned or stored to an instance property or global variable. -#### Values that contain nonescapable values must be nonescapable +#### Types that contain nonescapable values must be nonescapable Stored struct properties and enum payloads can have nonescapable types if the surrounding type is itself nonescapable. Equivalently, an escapable struct or enum can only contain escapable values. @@ -206,7 +206,7 @@ func f() -> NotEscapable { // 🛑 Cannot return a nonescapable type } ``` -A separate proposal describes “lifetime dependency annotations” that can relax this requirement by tying the lifetime of the returned value to the lifetime of some other object, either an argument to the function or `self` in the case of a method or computed property returning a nonescapable type. +A separate proposal describes “lifetime dependency annotations” that can relax this requirement by tying the lifetime of the returned value to the lifetime of another binding. The other binding can be a parameter of a function returning a vaule of a nonescapable type, or con be `self` for a method or computed property returning a value of a nonescapable type. In particular, struct and enum initializers (which build a new value and return it to the caller) cannot be written without some mechanism similar to that outlined in our companion proposal. #### Globals and static variables cannot be nonescapable @@ -223,7 +223,7 @@ Returning a nonescapable value from a closure requires explicit lifetime depende #### Nonescapable values and concurrency -All of the requirements on use of nonescapable values as function arguments and return values also apply to async functions, including those invoked via `async let`. +All of the requirements on use of nonescapable values as function parameters and return values also apply to async functions, including those invoked via `async let`. The closures used in `Task.init`, `Task.detached`, or `TaskGroup.addTask` are escaping closures and therefore cannot capture nonescapable values. From 738fafbec17a332a14c26385ebb2481da9142bde Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Fri, 23 Aug 2024 15:48:49 -0700 Subject: [PATCH 18/24] Revisit lifetime dependency discussions --- proposals/NNNN-non-escapable.md | 120 ++++++++++---------------------- 1 file changed, 35 insertions(+), 85 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 0e68da97e9..c317893f1d 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -14,7 +14,7 @@ We propose adding a new type constraint `~Escapable` for types that can be locally copied but cannot be assigned or transferred outside of the immediate context. This complements the `~Copyable` types added with SE-0390 by introducing another set of compile-time-enforced lifetime controls that can be used for safe, highly-performant APIs. -In addition, these types will support lifetime-dependency constraints (being tracked in a separate proposal), that allow them to safely hold pointers referring to data stored in other types. +In addition, these types will support lifetime-dependency constraints (being tracked in a future proposal), that allow them to safely hold pointers referring to data stored in other types. This feature is a key requirement for the proposed `Span` type. @@ -165,7 +165,7 @@ func f() { #### Constraints on nonescapable parameters -A value of nonescapable type received as an parameter is subject to the same constraints as any other local variable. +A value of nonescapable type received as a parameter is subject to the same constraints as any other local variable. In particular, a nonescapable `consuming` parameter (and all direct copies thereof) must actually be destroyed during the execution of the function. This is in contrast to an _escapable_ `consuming` parameter which can be disposed of by being returned or stored to an instance property or global variable. @@ -206,8 +206,8 @@ func f() -> NotEscapable { // 🛑 Cannot return a nonescapable type } ``` -A separate proposal describes “lifetime dependency annotations” that can relax this requirement by tying the lifetime of the returned value to the lifetime of another binding. The other binding can be a parameter of a function returning a vaule of a nonescapable type, or con be `self` for a method or computed property returning a value of a nonescapable type. -In particular, struct and enum initializers (which build a new value and return it to the caller) cannot be written without some mechanism similar to that outlined in our companion proposal. +A future proposal will describe “lifetime dependency annotations” that can relax this requirement by tying the lifetime of the returned value to the lifetime of another binding. +In particular, struct and enum initializers (which build a new value and return it to the caller) cannot be written without some such mechanism. #### Globals and static variables cannot be nonescapable @@ -219,7 +219,7 @@ This implies that they cannot be stored in global or static variables. Escaping closures cannot capture nonescapable values. Nonescaping closures can capture nonescapable values subject to the usual exclusivity restrictions. -Returning a nonescapable value from a closure requires explicit lifetime dependency annotations, as covered in the companion proposal. +Returning a nonescapable value from a closure will only be possible with explicit lifetime dependency annotations, to be covered in a future proposal. #### Nonescapable values and concurrency @@ -245,43 +245,14 @@ extension Box: Escapable where T: Escapable { } ``` This can be used in conjunction with other suppressible protocols. -For example, many general library container types will need to be copyable and/or escapable according to their contents. +For example, many general library container types will need to be copyable and/or escapable depending on their contents. Here's a compact way to declare such a type: ```swift -struct Wrapper { ... } -extension Wrapper: Copyable where T: ~Escapable {} -extension Wrapper: Escapable where T: ~Copyable {} +struct Wrapper: ~Copyable, ~Escapable { ... } +extension Wrapper: Copyable where T: Copyable {} +extension Wrapper: Escapable where T: Escapable {} ``` -The above declarations all in a single source file will result in a type `Wrapper` that is `Escapable` exactly when `T` is `Escapable` and `Copyable` exactly when `T` is `Copyable`. -To see why, first note that the explicit `extension Wrapper: Escapable` in the same source file implies that the original `struct Wrapper` must be `~Escapable` and similarly for `Copyable`, exactly as if the first line had been -```swift -struct Wrapper: ~Copyable & ~Escapable {} -``` - -Now recall from SE-427 that suppressible protocols must be explicitly suppressed on type parameters in extensions. -This means that -```swift -extension Wrapper: Copyable where T: ~Escapable {} -``` -is exactly the same as -```swift -extension Wrapper: Copyable where T: Copyable & ~Escapable {} -``` -which implies that `Wrapper` becomes `Copyable` when `T` is `Copyable`. -Finally, remember that `~Escapable` means that `Escapable` is not required, so -this condition on `Copyable` applies regardless of whether `T` is `Escapable` or not. - -Similarly, -```swift -extension Wrapper: Escapable where T: ~Copyable {} -``` -is exactly the same as -```swift -extension Wrapper: Escapable where T: Escapable & ~Copyable {} -``` -which means that `Wrapper` is `Escapable` whenever `T` is `Escapable` regardless of whether `T` is `Copyable` or not. - ## Source compatibility The compiler will treat any type without explicit `~Escapable` as escapable. @@ -316,24 +287,36 @@ Briefly, this type would provide an efficient universal “view” of array-like Since values of this type do not own any data but only refer to data stored elsewhere, their lifetime must be limited to not exceed that of the owning storage. We expect to publish a sample implementation and proposal for that type very soon. -#### Lifetime dependency annotations +#### Initializers and Lifetime Dependencies -Nonescapable types have a set of inherent restrictions on how they can be passed as arguments, stored in variables, or returned from functions. -A companion proposal builds on this by supporting more detailed annotations that link the lifetimes of different objects. -This would allow, for example, a container to vend an iterator value that held a direct unmanaged pointer to the container's contents. -The lifetime dependency would ensure that such an iterator could not outlive the container to whose contents it referred. +All values come into existence within the body of some initializer and are returned to the caller of that initializer. +Since nonescapable types cannot be returned, +it follows that nonescapable types cannot have initializers without some additional language affordance. +A subsequent proposal will provide such an affordance. +This will allow values to be returned subject to the requirement that they not outlive some other specific value. +For example, a nonescapable iterator might be initialized so as to not outlive the container that created it: ```swift -// Example: Nonescaping iterator -struct NEIterator { - // `dependsOn(container)` indicates that the constructed value - // cannot outlive the `container` argument. - init(over container: MyContainer) -> dependsOn(container) Self { - ... initialize an iterator suitable for `MyContainer` ... - } +struct Iterator: ~Escapable { + // ⚠️️ Returned Iterator will not be allowed to outlive `container` + // Details in a future proposal: This may involve + // additional syntax or default inference rules. + init(container: borrowing Container) { ... } } + +let iterator: Iterator +do { + let container = Container(...) + let buffer = container.buffer + iterator = Iterator(buffer) + // `container` lifetime ends here +} +use(iterator) // 🛑 'iterator' outlives `container` ``` +These lifetime dependencies will be enforced entirely at compile time without any runtime overhead. +Invalid uses such as the one above will produce compiler errors. + #### Expanding standard library types We expect that many standard library types will need to be updated to support possibly-nonescapable types, including `Optional`, `Array`, `Set`, `Dictionary`, and the `Unsafe*Pointer` family of types. @@ -349,7 +332,8 @@ For example, this can greatly improve the safety of locking APIs that expect to #### Nonescapable classes -We’ve explicitly excluded class types from being nonescapable. In the future, we could allow class types to be declared nonescapable as a way to avoid most reference-counting operations on class objects. +We’ve explicitly excluded class types from being nonescapable. +In the future, we could allow class types to be declared nonescapable as a way to avoid most reference-counting operations on class objects. #### Concurrency @@ -407,40 +391,6 @@ The iterator example in the beginning of this document provides another motivati Iterators are routinely copied in order to record a particular point in a collection. Thus we concluded that non-copyable was not the correct lifetime restriction for types of this sort, and it was worthwhile to introduce a new lifetime concept to the language. -#### Returns and initializers - -This proposal does not by itself provide any way to initialize a nonescapable value, requiring the additional proposed lifetime dependency annotations to support that mechanism. -Since those annotations require that the lifetime of the returned value be bound to that of one of the arguments, this implies that our current proposal does not permit nonescapable types to have trivial initializers: - -```swift -struct NE: ~Escapable { - init() {} // 🛑 Initializer return must depend on an argument -} -``` - -We considered introducing an annotation that would specifically allow this and related uses: - -```swift -struct NE: ~Escapable { - @_unsafeNonescapableResult - init() {} // OK because of annotation -} -``` - -We omitted this annotation from our proposal because there is more than one possible interpretation of such a marker. And we did not see a compelling reason for preferring one particular interpretation because we have yet to find a use case that actually requires this. - -In particular, the use cases we’ve so far considered have all been resolvable by adding an argument specifically for the purpose of anchoring a lifetime dependency: - -```swift -struct NE: ~Escapable { - // Proposed lifetime dependency notation; - // see separate proposal for details. - init(from: SomeType) -> dependsOn(from) Self {} -} -``` - -We expect that future experience with nonescapable types will clarify whether additional lifetime modifiers of this sort are justified. - ## Acknowledgements Many people discussed this proposal and gave important feedback, including: Kavon Farvardin, Meghana Gupta, John McCall, Slava Pestov, Joe Groff, Guillaume Lessard, and Franz Busch. From fe0579788eb90a81d8e83563f8a9439ab209e402 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Fri, 23 Aug 2024 15:51:03 -0700 Subject: [PATCH 19/24] s/Iterator/IteratorProtocol/ Co-authored-by: Karoy Lorentey --- proposals/NNNN-non-escapable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index c317893f1d..212280cd48 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -58,7 +58,7 @@ A separate proposal will show how we can further improve safety by allowing libr These "lifetime dependency" constraints can also be verified at compile time to ensure that the source of the iterator is not modified and that the iterator specifically does not outlive its source. **Note**: We are using iterators here to illustrate the issues we are considering. -We are not at this time proposing any changes to Swift's current `Iterator` protocol. +We are not at this time proposing any changes to Swift's current `IteratorProtocol` construct. ## Detailed design From 0c71b72d5951fab139ba8f76dd54acf4fffc5959 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Fri, 23 Aug 2024 15:54:45 -0700 Subject: [PATCH 20/24] Review feedback from lorentey --- proposals/NNNN-non-escapable.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 212280cd48..d2e0af2253 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -16,7 +16,7 @@ This complements the `~Copyable` types added with SE-0390 by introducing another In addition, these types will support lifetime-dependency constraints (being tracked in a future proposal), that allow them to safely hold pointers referring to data stored in other types. -This feature is a key requirement for the proposed `Span` type. +This feature is a key requirement for the proposed `Span` family of types. **See Also** @@ -58,7 +58,7 @@ A separate proposal will show how we can further improve safety by allowing libr These "lifetime dependency" constraints can also be verified at compile time to ensure that the source of the iterator is not modified and that the iterator specifically does not outlive its source. **Note**: We are using iterators here to illustrate the issues we are considering. -We are not at this time proposing any changes to Swift's current `IteratorProtocol` construct. +We are not at this time proposing any changes to Swift's current `IteratorProtocol` protocol. ## Detailed design @@ -280,9 +280,9 @@ Retrofitting existing generic types so they can support both escapable and nones ## Future directions -#### `Span` type +#### `Span` family of types -This proposal is being driven in large part by the needs of the `Span` type that has been discussed elsewhere. +This proposal is being driven in large part by the needs of the `Span` types that have been discussed elsewhere. Briefly, this type would provide an efficient universal “view” of array-like data stored in contiguous memory. Since values of this type do not own any data but only refer to data stored elsewhere, their lifetime must be limited to not exceed that of the owning storage. We expect to publish a sample implementation and proposal for that type very soon. From 3c1e92f925938b462f19c70ee649c2a77370ee62 Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Fri, 23 Aug 2024 16:31:21 -0700 Subject: [PATCH 21/24] Atrick's corrections --- proposals/NNNN-non-escapable.md | 39 +++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index d2e0af2253..5c00d38b44 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -289,33 +289,44 @@ We expect to publish a sample implementation and proposal for that type very soo #### Initializers and Lifetime Dependencies -All values come into existence within the body of some initializer and are returned to the caller of that initializer. -Since nonescapable types cannot be returned, -it follows that nonescapable types cannot have initializers without some additional language affordance. - -A subsequent proposal will provide such an affordance. -This will allow values to be returned subject to the requirement that they not outlive some other specific value. -For example, a nonescapable iterator might be initialized so as to not outlive the container that created it: +Nonescapable function parameters may not outlive the function scope. +Consequently, nonescapable values can never be returned from a function. +Nonescapable values come into existence within the body of the initializer. +Naturally, the initializer must return its value, and this creates an exception to the rule. +Without further language support, implementing a nonescapable initializer requires an unsafe construct. +That unsafe handling is not covered by this proposal because we don't want to surface unnecessary unsafety in the language. +Instead, a subsequent proposal will support safe initialization by allowing the initializer to specify lifetime dependencies on its parameters. +The parameters to the initializer typically indicate a lifetime that the nonescapable value cannot outlive. +An initializer may, for example, create a nonescapable value that depends on a container variable that is bound to an object with its own lifetime: ```swift struct Iterator: ~Escapable { - // ⚠️️ Returned Iterator will not be allowed to outlive `container` - // Details in a future proposal: This may involve - // additional syntax or default inference rules. init(container: borrowing Container) { ... } } +let container = ... +let iterator = Iterator(container) +consume container +use(iterator) // 🛑 'iterator' outlives the source of its dependency +``` + +Lifetime dependencies will make this use of iterator a compile-time error. +Or, as part of implementing data type internals, a nonescapable initializer may depend on a variable that is bound to a value that is only valid within that variable's local scope. +Subsequent uses of the initialized nonescapable object are exactly as safe or unsafe as it would be to use the variable that the initializer depends at the same point: + +```swift let iterator: Iterator do { let container = Container(...) let buffer = container.buffer iterator = Iterator(buffer) - // `container` lifetime ends here + // `iterator` is safe as long as `buffer` is safe to use. } -use(iterator) // 🛑 'iterator' outlives `container` +use(iterator) // 🛑 'iterator' outlives the source of its dependency ``` -These lifetime dependencies will be enforced entirely at compile time without any runtime overhead. -Invalid uses such as the one above will produce compiler errors. +Again, lifetime dependencies will make this use of iterator a compile-time error. +Typically, a pointer is valid for the duration of its variable binding. +So, in practice, nonescapable value that depends on the pointer to be available within the same scope. #### Expanding standard library types From 001c2723ee2932505676a83d121eb6116077711a Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Fri, 23 Aug 2024 16:46:27 -0700 Subject: [PATCH 22/24] More feedback from atrick --- proposals/NNNN-non-escapable.md | 25 +++++-------------------- 1 file changed, 5 insertions(+), 20 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index 5c00d38b44..eb0849e9b4 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -305,28 +305,13 @@ struct Iterator: ~Escapable { let container = ... let iterator = Iterator(container) -consume container -use(iterator) // 🛑 'iterator' outlives the source of its dependency +consume container // `container` lifetime ends here +use(iterator) // 🛑 'iterator' outlives `container` ``` -Lifetime dependencies will make this use of iterator a compile-time error. -Or, as part of implementing data type internals, a nonescapable initializer may depend on a variable that is bound to a value that is only valid within that variable's local scope. -Subsequent uses of the initialized nonescapable object are exactly as safe or unsafe as it would be to use the variable that the initializer depends at the same point: - -```swift -let iterator: Iterator -do { - let container = Container(...) - let buffer = container.buffer - iterator = Iterator(buffer) - // `iterator` is safe as long as `buffer` is safe to use. -} -use(iterator) // 🛑 'iterator' outlives the source of its dependency -``` - -Again, lifetime dependencies will make this use of iterator a compile-time error. -Typically, a pointer is valid for the duration of its variable binding. -So, in practice, nonescapable value that depends on the pointer to be available within the same scope. +Lifetime dependencies will make this and similar misuses into compile-time errors. +This will allow developers to safely define and use values that contain pointers into +other values, ensuring that the pointers never outlive the underlying storage. #### Expanding standard library types From 135caf73c25b278c4a96c86029d2b9585615698c Mon Sep 17 00:00:00 2001 From: Tim Kientzle Date: Fri, 30 Aug 2024 15:24:27 -0700 Subject: [PATCH 23/24] Require all suppressible for conditional conformance (LSG feedback) --- proposals/NNNN-non-escapable.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index eb0849e9b4..a6d3729ff7 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -249,8 +249,8 @@ For example, many general library container types will need to be copyable and/o Here's a compact way to declare such a type: ```swift struct Wrapper: ~Copyable, ~Escapable { ... } -extension Wrapper: Copyable where T: Copyable {} -extension Wrapper: Escapable where T: Escapable {} +extension Wrapper: Copyable where T: Copyable, T: ~Escapable {} +extension Wrapper: Escapable where T: Escapable, T: ~Copyable {} ``` ## Source compatibility From 96c1f19c917a7b600b7b43d051f8dc04731167f2 Mon Sep 17 00:00:00 2001 From: Andrew Trick Date: Sun, 1 Sep 2024 12:03:25 -0700 Subject: [PATCH 24/24] Update Future direction: Initializers and Lifetime Dependencies Clearly tie initialization of nonescapable values to the experimental lifetime dependencies feature. --- proposals/NNNN-non-escapable.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/proposals/NNNN-non-escapable.md b/proposals/NNNN-non-escapable.md index a6d3729ff7..1f461d20c9 100644 --- a/proposals/NNNN-non-escapable.md +++ b/proposals/NNNN-non-escapable.md @@ -293,9 +293,6 @@ Nonescapable function parameters may not outlive the function scope. Consequently, nonescapable values can never be returned from a function. Nonescapable values come into existence within the body of the initializer. Naturally, the initializer must return its value, and this creates an exception to the rule. -Without further language support, implementing a nonescapable initializer requires an unsafe construct. -That unsafe handling is not covered by this proposal because we don't want to surface unnecessary unsafety in the language. -Instead, a subsequent proposal will support safe initialization by allowing the initializer to specify lifetime dependencies on its parameters. The parameters to the initializer typically indicate a lifetime that the nonescapable value cannot outlive. An initializer may, for example, create a nonescapable value that depends on a container variable that is bound to an object with its own lifetime: ```swift @@ -309,9 +306,10 @@ consume container // `container` lifetime ends here use(iterator) // 🛑 'iterator' outlives `container` ``` -Lifetime dependencies will make this and similar misuses into compile-time errors. -This will allow developers to safely define and use values that contain pointers into -other values, ensuring that the pointers never outlive the underlying storage. +Specifying a dependency from a function parameter to its nonescapable result currently requires an experimental lifetime dependency feature. +With lifetime dependencies, initialization of nonescapable types is safe: misuses similar to the one shown above are compile-time errors. +Adopting new syntax for lifetime dependencies merits a separate, focussed review. +Until then, initialization of nonescapable values remains experimental. #### Expanding standard library types