Issue #9076

lachlan-roberts · 2023-02-02T03:11:01Z

Fix parsing bug in multipart (shared backing array for buffers from ServletMultiPartFormData#parse).
Add cleanup for temporary parts (save Parts in request attribute for delayed handler and cleanup).
Use MAX_FORM_KEYS_KEY and MAX_FORM_CONTENT_SIZE_KEY parameters to configure limits for multipart.
Allow all parts to be written to file, multiple writes will instead do a file move, and Part.delete() will delete any file storage for that part.
Can use newContentSource() method to get a new Content.Source every time from the part.

other fixes and cleanups

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

…9066-MultiPart-getParameters

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

…9066-MultiPart-getParameters

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

…9066-MultiPart-getParameters

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

…multipartCleanups

jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/MultiPartByteRanges.java

jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/MultiPartFormData.java

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/content/ChunksContentSource.java

jetty-core/jetty-server/src/main/java/org/eclipse/jetty/server/Request.java

...-ee10/jetty-ee10-servlet/src/main/java/org/eclipse/jetty/ee10/servlet/ServletApiRequest.java

...e9/jetty-ee9-nested/src/main/java/org/eclipse/jetty/ee9/nested/MultiPartFormInputStream.java

jetty-ee9/jetty-ee9-nested/src/main/java/org/eclipse/jetty/ee9/nested/Request.java

jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/MultiPartFormData.java

jetty-core/jetty-server/src/main/java/org/eclipse/jetty/server/HttpStream.java

sbordet · 2023-02-06T15:27:15Z

jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/MultiPart.java

+            List<Content.Chunk> newChunks = content.stream()
+                .map(chunk -> Content.Chunk.from(chunk.getByteBuffer().slice(), chunk.isLast()))
+                .toList();
+            return new ChunksContentSource(newChunks);


Discussing this with @lorban, I think it's best if we store these ChunksContentSource in a List, and we fail them all when this part is closed.

The reason is that in this particular case, if a ChunksContentSource is read after completing the Handler callback, it can read content from buffers that were returned to the pool and may be used for other requests.
By failing them, we guarantee that if they are read after Handler completion, an error is returned.

What if someone calls newContentSource() after the part has been closed, do we need to keep an atomic state and if this has been closed return them a failed instance of Content.Source when they call newContentSource()?

Why can't we just say if this have been closed it is invalid to use it. Just like if you return a RetainableByteBuffer to the pool it is invalid to continue using it.

opened #9336

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

sbordet

Please fix test failures, and then LGTM.

…multipartCleanups

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

…x-documentation-operations-logging * upstream/jetty-12.0.x: (35 commits) Fixes jetty#9326 - Rename DecryptedEndPoint to SslEndPoint. Jetty 10 Upgrade to Hazelcast 5 and totally disable auto join multicast etc.. (fix build on CI) (jetty#9331) jetty#9328 - changes from review jetty#9287 - catch error in ee9 maxRequestSize MultiPart test Jetty 12.0.x 9301 fix ee10 jstl jpms (jetty#9321) Issue jetty#9301 Fix dependencies for ee10-glassfish-jstl module (jetty#9303) Jetty 12 Hazelcast 5.x and disable auto detection/multicast" (jetty#9332) jetty#9287 - fix further test failures Fixed imports. Issue jetty#7650 - Fix race condition when stopping QueuedThreadPool (jetty#9325) jetty#9287 - remove unpaired release of Content.Chunk Issue jetty#8991 - rename websocket isDemanding() method to isAutoDemanding() Issue jetty#9287 - fix failing tests changes f rom review add todo to revert to normal pool after fix for jetty#9311 Issue jetty#9309 - Introducing test for requestlog format with spaces use non-pooling RetainableByteBufferPool to work around performance bug consumeAvailable should use number of reads instead of bytes fix for retainable merge changes from review ...

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

gregw · 2024-08-30T02:49:33Z

jetty-core/jetty-server/src/main/java/org/eclipse/jetty/server/internal/HttpConnection.java

+        if (_parser.isTerminated())
+            throw new RuntimeIOException("Parser is terminated");


@lachlan-roberts @sbordet I think this is causing #12212
Can you recall why this was necessary?

@gregw this was added in d4682f4, with the commit description "Fix testMaxRequest size test in MultiPartServletTest".

I re-run those tests, and the changes #12213 pass those tests, so I think either was not necessary (maybe just a zealous sanity check), or it is not anymore.

lachlan-roberts added 18 commits January 11, 2023 14:39

Issue #9066 - parse multipart form data when getParameters is called

da39f93

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

add test to reproduce some issue with multipart

0432817

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

add more tests for multipart

1ff251c

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

fixes to pass new multipart test cases

02f9cc0

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

fix multipart test cases, add more api to the core MultiPart class

3816e4b

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

improvements to multipart

1e20abc

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

improvements to multipart

5bcfeaa

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

Merge remote-tracking branch 'origin/jetty-12.0.x' into jetty-12.0.x-…

539113d

…9066-MultiPart-getParameters

Fix testMaxRequest size test in MultiPartServletTest

d4682f4

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

fix some more tests in MultiPartServletTest

2dfe7e6

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

Merge remote-tracking branch 'origin/jetty-12.0.x' into jetty-12.0.x-…

36e7f71

…9066-MultiPart-getParameters

changes from review

7387124

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

changes from review

719c60d

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

changes from review

8133dd6

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

changes from review

88c6f20

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

Merge remote-tracking branch 'origin/jetty-12.0.x' into jetty-12.0.x-…

4ffad09

…9066-MultiPart-getParameters

fix for retainable merge

02823ec

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

consumeAvailable should use number of reads instead of bytes

d66e36b

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

lachlan-roberts requested review from gregw, joakime and sbordet February 2, 2023 03:11

lachlan-roberts added 3 commits February 4, 2023 00:49

use non-pooling RetainableByteBufferPool to work around performance bug

cd47a07

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

add todo to revert to normal pool after fix for #9311

1610cd3

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

Merge remote-tracking branch 'origin/jetty-12.0.x' into jetty-12.0.x-…

c7e7fd8

…multipartCleanups

sbordet requested changes Feb 6, 2023

View reviewed changes

changes f rom review

d005e97

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

sbordet approved these changes Feb 7, 2023

View reviewed changes

lachlan-roberts added 3 commits February 8, 2023 11:55

Merge remote-tracking branch 'origin/jetty-12.0.x' into jetty-12.0.x-…

6950d73

…multipartCleanups

Issue #9287 - fix failing tests

cb511be

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

#9287 - remove unpaired release of Content.Chunk

1e92759

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

lachlan-roberts added 2 commits February 9, 2023 08:37

#9287 - fix further test failures

341ac15

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

#9287 - catch error in ee9 maxRequestSize MultiPart test

820fbb7

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

lachlan-roberts merged commit 3259a55 into jetty-12.0.x Feb 9, 2023

lachlan-roberts deleted the jetty-12.0.x-multipartCleanups branch February 9, 2023 10:25

lachlan-roberts added a commit that referenced this pull request Feb 13, 2023

bring back some multipart improvements from #9287

a5344d7

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>

This was referenced Feb 13, 2023

Cleanup Multipart handling #9344

Merged

multipart cleanups jetty 9 #9345

Merged

Clean up multipart #9076

Closed

TCK multipart not set as request parameter #9066

Closed

gregw reviewed Aug 30, 2024

View reviewed changes

gregw mentioned this pull request Aug 30, 2024

ShutdownOutput for non-persistent HTTP/1 connections #12212

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue #9076 - Cleanups and fixes for multipart in Jetty 12 #9287

Issue #9076 - Cleanups and fixes for multipart in Jetty 12 #9287

lachlan-roberts commented Feb 2, 2023

sbordet Feb 6, 2023

lachlan-roberts Feb 7, 2023

lachlan-roberts Feb 9, 2023

sbordet left a comment

gregw Aug 30, 2024

sbordet Aug 30, 2024

		if (_parser.isTerminated())
		throw new RuntimeIOException("Parser is terminated");

Issue #9076 - Cleanups and fixes for multipart in Jetty 12 #9287

Issue #9076 - Cleanups and fixes for multipart in Jetty 12 #9287

Conversation

lachlan-roberts commented Feb 2, 2023

Issue #9076

sbordet Feb 6, 2023

Choose a reason for hiding this comment

lachlan-roberts Feb 7, 2023

Choose a reason for hiding this comment

lachlan-roberts Feb 9, 2023

Choose a reason for hiding this comment

sbordet left a comment

Choose a reason for hiding this comment

gregw Aug 30, 2024

Choose a reason for hiding this comment

sbordet Aug 30, 2024

Choose a reason for hiding this comment