Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix export job dataset #5052

Merged
merged 14 commits into from
Oct 15, 2022
Merged

Fix export job dataset #5052

merged 14 commits into from
Oct 15, 2022

Conversation

Marishka17
Copy link
Contributor

@Marishka17 Marishka17 commented Oct 6, 2022

Motivation and context

In the previous implementation, the annotation file for the job contained information related to the task. Zip archive with the dataset contained all images of the task. This PR fixed this issue.

How has this been tested?

Manually

Checklist

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 9, 2022

❌ Some checks failed
📄 See logs here

@github-actions
Copy link
Contributor

github-actions bot commented Oct 10, 2022

❌ Some checks failed
📄 See logs here

@Marishka17 Marishka17 marked this pull request as ready for review October 10, 2022 08:24
@Marishka17 Marishka17 changed the title [WIP] Fix export job dataset Fix export job dataset Oct 10, 2022
@zhiltsov-max
Copy link
Contributor

zhiltsov-max commented Oct 11, 2022

When I tried to export the 2nd job in a task with 6 images and 3 jobs (2 img each) as Segmentation mask 1.1, I've got the following error:

Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/rq/worker.py", line 936, in perform_job
    rv = job.perform()
  File "/venv/lib/python3.9/site-packages/rq/job.py", line 684, in perform
    self._result = self._execute()
  File "/venv/lib/python3.9/site-packages/rq/job.py", line 690, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/cvat/apps/dataset_manager/views.py", line 108, in export_job_annotations
    return export(dst_format,job_id=job_id, server_url=server_url, save_images=False)
  File "/cvat/apps/dataset_manager/views.py", line 81, in export
    export_fn(db_instance.id, temp_file, dst_format,
  File "/cvat/apps/dataset_manager/task.py", line 716, in export_job
    job.export(f, exporter, host=server_url, save_images=save_images)
  File "/cvat/apps/dataset_manager/task.py", line 561, in export
    exporter(dst_file, job_data, **options)
  File "/cvat/apps/dataset_manager/formats/registry.py", line 37, in __call__
    f_or_cls(*args, **kwargs)
  File "/cvat/apps/dataset_manager/formats/mask.py", line 28, in _export
    apply_colormap=True, label_map=make_colormap(instance_data))
  File "/cvat/apps/dataset_manager/formats/utils.py", line 53, in make_colormap
    labels = [label for _, label in instance_data.meta[instance_name]['labels']]
KeyError: 'task'

This also applies to other segmentation formats (CamVid, Cityscapes). CVAT for images and COCO seem to work.

Also, don't you think the old button for task export in the menu of the annotation page should be restored? Users may expect it to be there, and it was quite useful there.
image

Also, consider adding a job menu button here:
image

@github-actions
Copy link
Contributor

github-actions bot commented Oct 12, 2022

❌ Some checks failed
📄 See logs here

@Marishka17
Copy link
Contributor Author

Marishka17 commented Oct 12, 2022

When I tried to export the 2nd job in a task with 6 images and 3 jobs (2 img each) as Segmentation mask 1.1, I've got the following error:

Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/rq/worker.py", line 936, in perform_job
    rv = job.perform()
  File "/venv/lib/python3.9/site-packages/rq/job.py", line 684, in perform
    self._result = self._execute()
  File "/venv/lib/python3.9/site-packages/rq/job.py", line 690, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/cvat/apps/dataset_manager/views.py", line 108, in export_job_annotations
    return export(dst_format,job_id=job_id, server_url=server_url, save_images=False)
  File "/cvat/apps/dataset_manager/views.py", line 81, in export
    export_fn(db_instance.id, temp_file, dst_format,
  File "/cvat/apps/dataset_manager/task.py", line 716, in export_job
    job.export(f, exporter, host=server_url, save_images=save_images)
  File "/cvat/apps/dataset_manager/task.py", line 561, in export
    exporter(dst_file, job_data, **options)
  File "/cvat/apps/dataset_manager/formats/registry.py", line 37, in __call__
    f_or_cls(*args, **kwargs)
  File "/cvat/apps/dataset_manager/formats/mask.py", line 28, in _export
    apply_colormap=True, label_map=make_colormap(instance_data))
  File "/cvat/apps/dataset_manager/formats/utils.py", line 53, in make_colormap
    labels = [label for _, label in instance_data.meta[instance_name]['labels']]
KeyError: 'task'

Thanks, this issue has been fixed

This also applies to other segmentation formats (CamVid, Cityscapes). CVAT for images and COCO seem to work.

Also, don't you think the old button for task export in the menu of the annotation page should be restored? Users may expect it to be there, and it was quite useful there. image

I guess that button to export task should be on the task detail page or on the tasks page. Let's wait for customer requests.

Also, consider adding a job menu button here: image

We have a button to export jobs in the annotation view and on jobs page. Let's wait for customer requests.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 12, 2022

✔️ All checks completed successfully
📄 See logs here

cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved
cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved
cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved
cvat/apps/dataset_manager/bindings.py Outdated Show resolved Hide resolved
cvat/apps/dataset_manager/formats/cvat.py Outdated Show resolved Hide resolved
@Marishka17
Copy link
Contributor Author

/check

@github-actions
Copy link
Contributor

github-actions bot commented Oct 14, 2022

✔️ All checks completed successfully
📄 See logs here

@nmanovic nmanovic merged commit 15c3a63 into develop Oct 15, 2022
@nmanovic nmanovic deleted the mk/fix_bug_with_export branch October 15, 2022 16:40
@nmanovic nmanovic mentioned this pull request Dec 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants