Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source & target storage support #4842

Merged
merged 62 commits into from
Sep 13, 2022
Merged

Source & target storage support #4842

merged 62 commits into from
Sep 13, 2022

Conversation

Marishka17
Copy link
Contributor

@Marishka17 Marishka17 commented Aug 25, 2022

Motivation and context

UI

  • Added ability to specify source & target storage on project and task creating pages, import & export modals.
  • Modified annotations import
  • Modified backups import and export

Server

  • small enhancement: moved file downloading process to job

How has this been tested?

Manually
Small demo showing how it works.

Documentation and tests for this functionality will be in other PRs

Checklist

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@Marishka17
Copy link
Contributor Author

/check

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2022

❌ Some checks failed
📄 See logs here

@Marishka17 Marishka17 changed the title [WIP][UI] Source/target storage Source & target storage support Sep 5, 2022
@Marishka17 Marishka17 added enhancement New feature or request ui labels Sep 5, 2022
@Marishka17
Copy link
Contributor Author

/check

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2022

✔️ All checks completed successfully
📄 See logs here

@BonVant
Copy link

BonVant commented Nov 14, 2022

Hi @Marishka17 ! Thanks for the contribution. I was wondering whether all the data is still downloaded from source to container before being exported to target storage ?
S3 source manifest + files -> Local container volume such as /home/django/data -> S3 target manifest + job _annotations.zip

Would be interesting albeit maybe complex to avoid storing locally large chunks of data and directly interact with the S3 target bucket ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants