Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dispatch conversion jobs after database transactions have been committed #3687

Merged
merged 1 commit into from
Aug 20, 2024

Conversation

chrispage1
Copy link
Contributor

@chrispage1 chrispage1 commented Aug 5, 2024

I have been running into an issue (#3677) and it has been raised previously (#3674) that media conversions sometimes immediately fail.

Having done some debugging I've found the cause to be media that is added within a database transaction. In my case this was apparent in Laravel Nova when creating a post with large media.

This means when the queue worker tries to generate the conversions, they immediately fail because the record has not yet been persisted to the database.

The fix is to ensure that these jobs are only dispatched after the data has been committed to the database. This can easily be enforced by calling ->afterCommit() on the dispatched job. If there is no active transaction the job will be dispatched in the usual way.

@royduin
Copy link
Contributor

royduin commented Aug 9, 2024

I can confirm this fixed the issue 🚀

@francoism90
Copy link
Contributor

@chrispage1
Copy link
Contributor Author

You can set this globally: https://laravel.com/docs/11.x/queues#jobs-and-database-transactions

I did see that this was possible and mentioned it in #3674 but this PR would address the issue for anyone running this package regardless of whether the config variable is set or not. A worthy addition I think 👍🏻

@chrispage1
Copy link
Contributor Author

Hey @freekmurze - reckon this is something you could merge in? Any particular concerns? Thanks!

@freekmurze freekmurze merged commit 4530541 into spatie:main Aug 20, 2024
9 checks passed
@freekmurze
Copy link
Member

Thanks!

@chrispage1 chrispage1 deleted the feature/dispatch-after-commit branch August 20, 2024 09:49
@chrispage1
Copy link
Contributor Author

Thanks @freekmurze ! Have a great day!

@Tahiaji
Copy link

Tahiaji commented Aug 21, 2024

This fix is ​​not very obvious, but it crashed tests on my project. Tests are wrapped in a transaction. Access to conversions is no longer available. For now, I rolled back to 11.8.2

@chrispage1
Copy link
Contributor Author

chrispage1 commented Aug 21, 2024

This fix is ​​not very obvious, but it crashed tests on my project. Tests are wrapped in a transaction. Access to conversions is no longer available. For now, I rolled back to 11.8.2

Any specific error you're getting conversions just aren't being shown?

What version of Laravel are you running, 10 or 11?

And how many transaction layers deep are you?

@chrispage1
Copy link
Contributor Author

This fix is ​​not very obvious, but it crashed tests on my project. Tests are wrapped in a transaction. Access to conversions is no longer available. For now, I rolled back to 11.8.2

Ah I see, your tests are wrapped in a transaction. In that case conversions wont be generated until after the transaction has completed. Any particular reason for your tests being within a transaction?

@royduin
Copy link
Contributor

royduin commented Aug 22, 2024

I think it's default behaviour due the RefreshDatabase trait: https://laravel.com/docs/master/database-testing#resetting-the-database-after-each-test

@Tahiaji
Copy link

Tahiaji commented Aug 22, 2024

Exactly. We use trait RefreshDatabase to significantly speed up test execution.

The change seems minor, but it breaks backwards compatibility to some extent.

@chrispage1
Copy link
Contributor Author

chrispage1 commented Aug 22, 2024

Created a PR #3698 - if accepted, this will allow you to optionally change the default behaviour which would be particularly useful within your tests.

I did consider just disabling this behaviour when running unit tests, but I think that could cause additional undesirable behaviour. At least this way, it'll be easy to simply turn the behaviour on/off.

@Tahiaji
Copy link

Tahiaji commented Aug 22, 2024

Created a PR #3698 - if accepted, this will allow you to optionally change the default behaviour which would be particularly useful within your tests.

I did consider just disabling this behaviour when running unit tests, but I think that could cause additional undesirable behaviour. At least this way, it'll be easy to simply turn the behaviour on/off.

Thank you very much. The ability to disable this specifically for testing purposes would be extremely useful.

@chrispage1
Copy link
Contributor Author

Created a PR #3698 - if accepted, this will allow you to optionally change the default behaviour which would be particularly useful within your tests.
I did consider just disabling this behaviour when running unit tests, but I think that could cause additional undesirable behaviour. At least this way, it'll be easy to simply turn the behaviour on/off.

Thank you very much. The ability to disable this specifically for testing purposes would be extremely useful.

It looks like Freek was on the case and this has been merged!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants