Open resource links from any organization/sandbox #5892

klakhov · 2023-03-21T08:35:23Z

Motivation and context

Server part:

Update swagger docs
For object requests: define iam_context according to the obj.organization_id
For list and create requests: iam_context is defined according to the query parameters org_id or org (slug)
Fix tests

UI part:

Change organization when link leads to resource in different organization
Fix tests

How has this been tested?

Checklist

I submit my changes into the develop branch
I have added a description of my changes into the CHANGELOG file
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues (see GitHub docs)
I have increased versions of npm packages if it is necessary
(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

…e and list rest api methods

nmanovic · 2023-05-23T17:17:03Z

I got the error below when I requested a task.

Could not receive the requested project from the server
Error: Request failed with status code 403. {"detail":"You do not have permission to perform this action."}.

nmanovic · 2023-05-23T17:31:33Z

@kirill-sizov , could you please look at /api/organizations request. The number of queries are growing together with number of organizations. Indeed I believe the number of requests should be less than 10. Please try to minimize.

nmanovic · 2023-05-23T17:37:00Z

@kirill-sizov , /api/jobs request doesn't work in debug at least. Could you please look?

…ng extra db query

…pencv/cvat into kl/remove-org-from-id-requests

sizov-kirill · 2023-05-25T06:52:26Z

@nmanovic I fixed the performance problem and the problem with getting jobs. But confused about this issue:

I got the error below when I requested a task.

Could not receive the requested project from the server
Error: Request failed with status code 403. {"detail":"You do not have permission to perform this action."}.

I have checked accessing tasks with different parameters of privileges and organization roles (this is also covered by our test) and couldn't reproduce this problem. Could please clarify how did you get this error?

Did you get this error in the development environment or in docker?
Does this task belong to the organization or project?
Which privilege does the requester have?
Which organization role does the requester have?
Does the requester assign to the task or corresponding project?

…pencv/cvat into kl/remove-org-from-id-requests

zhiltsov-max · 2023-05-26T20:37:19Z

cvat/apps/iam/filters.py

+            return []
+
+        return [
+            {


Shouldn't these values be aligned with the ORGANIZATION_OPEN_API_PARAMETERS somehow? The code can easily diverge.

zhiltsov-max · 2023-05-26T20:38:06Z

cvat/apps/engine/schema.py

@@ -228,4 +229,27 @@ class CloudStorageReadSerializerExtension(_CloudStorageSerializerExtension):
 class CloudStorageWriteSerializerExtension(_CloudStorageSerializerExtension):
    target_class = 'cvat.apps.engine.serializers.CloudStorageWriteSerializer'

+ORGANIZATION_OPEN_API_PARAMETERS = [


When this should be used? Shouldn't it be documented and declared in IAM or in the organizations apps?

We use this in "schema" decorators for views like here. But yes I agree probably it would be more logical to move it in iam

zhiltsov-max · 2023-05-26T20:41:21Z

cvat/apps/iam/filters.py

+        # Filter works only for "list" requests and allows to return
+        # only non-organization objects if org isn't specified


Probably, these parameters should be documented somehow, especially the (past?) logic about org=''/org=None etc.

zhiltsov-max · 2023-05-26T20:44:01Z

cvat/apps/iam/schema.py

@@ -107,35 +78,6 @@ def get_security_definition(self, auto_schema):
        return [sessionid_schema, csrftoken_schema]

 class CustomAutoSchema(AutoSchema):


This class can be moved to engine if there are no IAM specifics added. The only reason it is here is that we can't define it in the engine and import and inherit here.

Moving this class in apps/engine/schema.py still leads to error due to this line

zhiltsov-max · 2023-05-26T21:16:56Z

cvat/apps/events/handlers.py

-        if oid is None:
-            return instance.get_organization_id()
-        return oid
+        return getattr(instance, "organization_id", None)


Should this attr be enforced on the objects somehow? It's easy to miss it and it won't be noticed.

Currently objects that have foreign key to Organization has organization_id attribute, for those models that don't have explicit relation with Organization has get_organization_id attribute. This logic is used in few places of code: events, webhooks, permissions. Currently I don't know how this can be done without defining organization_id in each model

zhiltsov-max

Do you know if we can do anything about places like this https://github.com/opencv/cvat/blob/develop/cvat/apps/engine/views.py#L1832 ?

zhiltsov-max

Probably, such tests should be updated too https://github.com/opencv/cvat/blob/develop/tests/python/rest_api/test_labels.py#L249-L253

sizov-kirill · 2023-05-29T08:31:02Z

Do you know if we can do anything about places like this https://github.com/opencv/cvat/blob/develop/cvat/apps/engine/views.py#L1832 ?

Yes I agree, case with Labels is unusual. Currently the least laborious solution I can suggest is to change iam_organization_field(str) to iam_organizationd_fields(tuple), which would allow multiple organization fields for objects like Label. But I would suggest to postpone this idea until we have other such objects besides labels.

zhiltsov-max · 2023-05-29T09:27:39Z

But I would suggest to postpone this idea until we have other such objects besides labels.

There is a couple more in the Honey Pot feature endpoints. Probably, an option to provide a list or even a mapping would be useful, similarly to what is done for other list filters.

### Motivation and context Applied comments: - [x] Aligning ORGANIZATION_OPEN_API_PARAMETERS #5892 (comment) - [x] Moving ORGANIZATION_OPEN_API_PARAMETERS #5892 (comment) ~~- [ ] Moving CustomerAutoSchema #5892 (comment) this uncritical comment that cannot be done easily, [see answer](#5892 (comment)) - [x] Raise error if cannot get `organization_id` for objects #5892 (comment) - [x] Multiply fields for `iam_organization_field` #5892 (comment) Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>

## \[2.4.5] - 2023-06-02 ### Added - Integrated support for sharepoint and cloud storage files, along with directories to be omitted during task creation (server) (<#6074>) - Enabled task creation with directories from cloud storage or sharepoint (<#6074>) - Enhanced task creation to support any data type supported by the server by default, from cloud storage without the necessity for the `use_cache` option (<#6074>) - Added capability for task creation with data from cloud storage without the `use_cache` option (<#6074>) ### Changed - User can now access resource links from any organization or sandbox, granted it's available to them (<#5892>) - Cloud storage manifest files have been made optional (<#6074>) - Updated Django to the 4.2.x version (<#6122>) - Renamed certain Nuclio functions to adhere to a common naming convention. For instance, `onnx-yolov7` -> `onnx-wongkinyiu-yolov7`, `ultralytics-yolov5` -> `pth-ultralytics-yolov5` (<#6140>) ### Deprecated - Deprecated the endpoint `/cloudstorages/{id}/content` (<#6074>) ### Fixed - Fixed the issue of skeletons dumping on created tasks/projects (<#6157>) - Resolved an issue related to saving annotations for skeleton tracks (<#6075>) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru> Co-authored-by: Roman Donchenko <roman@cvat.ai> Co-authored-by: Maria Khrustaleva <maya17grd@gmail.com> Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com> Co-authored-by: Nikita Manovich <nikita@cvat.ai> Co-authored-by: Anastasia Yasakova <yasakova_anastasiya@mail.ru> Co-authored-by: Snyk bot <snyk-bot@snyk.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai> Co-authored-by: Paweł Kotiuk <kotiuk@zohomail.eu> Co-authored-by: SK <450723+senthilkumarkj@users.noreply.github.com> Co-authored-by: Kirill Lakhov <kirill.9992@gmail.com>

added organization watcher

317de1c

klakhov requested a review from bsekachev as a code owner March 21, 2023 08:35

klakhov and others added 11 commits March 21, 2023 11:36

Merge branch 'develop' into kl/remove-org-from-id-requests

e1cf2fd

rename 'get_organization_id' -> 'organization_id' (property)

30d8a30

rename 'get_organization_id' -> 'organization_id' (property)

cc29bba

remove iam_context middleware (initial commit)

141707a

Merge branch 'develop' into kl/remove-org-from-id-requests

b6e53dd

add 'organization_id' property for models

1319d61

use 'organization_id' property in events handler

6c95b64

fix 'get_organization' function

e34b0cf

use 'request.organization' instead of 'request.iam_context' for creat…

90122a2

…e and list rest api methods

fix tests

3268f0e

remove 'IsMemberInOrganization' permission class

1250820

sizov-kirill requested review from azhavoro and Marishka17 as code owners April 10, 2023 10:46

sizov-kirill changed the title ~~[WIP] Open resource links from any organization/sandbox~~ Open resource links from any organization/sandbox Apr 10, 2023

fix tests

f274001

sizov-kirill changed the title ~~Open resource links from any organization/sandbox~~ [WIP] Open resource links from any organization/sandbox Apr 10, 2023

sizov-kirill added 5 commits April 11, 2023 11:40

init iam_context once

4444e58

Merge branch 'develop' into kl/remove-org-from-id-requests

5a23f2d

code style

8933412

remove unecessary changes

273769c

fix tests

95df580

sizov-kirill changed the title ~~[WIP] Open resource links from any organization/sandbox~~ Open resource links from any organization/sandbox Apr 11, 2023

sizov-kirill added 5 commits April 12, 2023 11:26

add comment

14ccc46

return back 'X-Organization' header

f1d9435

fix membership deleting

afc99f4

fix sdk test

f8a9a52

fix e2e tests

c5f2565

bsekachev approved these changes Apr 14, 2023

View reviewed changes

sizov-kirill added 5 commits May 24, 2023 11:31

fix problem with getting jobs in debug mode

704aafc

remove unused function, rid of unnecessary db request

22ed652

move getting privilege to middleware step in order to rid of performi…

a218546

…ng extra db query

optimise fetching objects from db

2c670f6

Merge branch 'kl/remove-org-from-id-requests' of https://github.com/o…

f0cd235

…pencv/cvat into kl/remove-org-from-id-requests

nmanovic and others added 3 commits May 26, 2023 17:57

Merge branch 'develop' into kl/remove-org-from-id-requests

e55dd78

fix ui error message

972f758

Merge branch 'kl/remove-org-from-id-requests' of https://github.com/o…

16a9135

…pencv/cvat into kl/remove-org-from-id-requests

nmanovic approved these changes May 26, 2023

View reviewed changes

nmanovic merged commit ed3dbe8 into develop May 26, 2023

nmanovic deleted the kl/remove-org-from-id-requests branch May 26, 2023 19:01

zhiltsov-max reviewed May 26, 2023

View reviewed changes

sizov-kirill mentioned this pull request May 29, 2023

Fix comments for PR 5892 #6206

Merged

11 tasks

azhavoro mentioned this pull request Jun 2, 2023

Release v2.4.5 #6233

Merged

mikhail-treskin pushed a commit to retailnext/cvat that referenced this pull request Jul 1, 2023

Open resource links from any organization/sandbox (cvat-ai#5892)

e0d3741

AnnaPetrovicheva mentioned this pull request Oct 27, 2023

[Snyk] Security upgrade axios from 0.27.2 to 1.6.0 #7077

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Open resource links from any organization/sandbox #5892

Open resource links from any organization/sandbox #5892

klakhov commented Mar 21, 2023 •

edited by sizov-kirill

Loading

nmanovic commented May 23, 2023

nmanovic commented May 23, 2023

nmanovic commented May 23, 2023

sizov-kirill commented May 25, 2023 •

edited

Loading

zhiltsov-max May 26, 2023

zhiltsov-max May 26, 2023 •

edited

Loading

sizov-kirill May 29, 2023

zhiltsov-max May 26, 2023

zhiltsov-max May 26, 2023 •

edited

Loading

sizov-kirill May 29, 2023

zhiltsov-max May 26, 2023

sizov-kirill May 29, 2023

zhiltsov-max left a comment

zhiltsov-max left a comment

sizov-kirill commented May 29, 2023

zhiltsov-max commented May 29, 2023

		# Filter works only for "list" requests and allows to return
		# only non-organization objects if org isn't specified

		@@ -107,35 +78,6 @@ def get_security_definition(self, auto_schema):
		return [sessionid_schema, csrftoken_schema]

		class CustomAutoSchema(AutoSchema):

Open resource links from any organization/sandbox #5892

Open resource links from any organization/sandbox #5892

Conversation

klakhov commented Mar 21, 2023 • edited by sizov-kirill Loading

Motivation and context

How has this been tested?

Checklist

License

nmanovic commented May 23, 2023

nmanovic commented May 23, 2023

nmanovic commented May 23, 2023

sizov-kirill commented May 25, 2023 • edited Loading

zhiltsov-max May 26, 2023

Choose a reason for hiding this comment

zhiltsov-max May 26, 2023 • edited Loading

Choose a reason for hiding this comment

sizov-kirill May 29, 2023

Choose a reason for hiding this comment

zhiltsov-max May 26, 2023

Choose a reason for hiding this comment

zhiltsov-max May 26, 2023 • edited Loading

Choose a reason for hiding this comment

sizov-kirill May 29, 2023

Choose a reason for hiding this comment

zhiltsov-max May 26, 2023

Choose a reason for hiding this comment

sizov-kirill May 29, 2023

Choose a reason for hiding this comment

zhiltsov-max left a comment

Choose a reason for hiding this comment

zhiltsov-max left a comment

Choose a reason for hiding this comment

sizov-kirill commented May 29, 2023

zhiltsov-max commented May 29, 2023

klakhov commented Mar 21, 2023 •

edited by sizov-kirill

Loading

sizov-kirill commented May 25, 2023 •

edited

Loading

zhiltsov-max May 26, 2023 •

edited

Loading

zhiltsov-max May 26, 2023 •

edited

Loading