function.yaml for automatic annotation of tags.

[tommy-hayashida]

My actions before raising this issue

• Read/searched the docs
• Searched past issues

I have written main.py based on the following issue and pull request. #3358 #3896

Can you tell me how to write a function.yaml to auto-annotate tags?
I don't know what should be in annotations.type.

Steps to Reproduce (for bugs)

I filled in that field with tag and deployed it.
However, when I looked at the cvat model list, the deployed function did not appear.

metadata:
  name: exapmle
  namespace: cvat
  annotations:
    name: exapmle
    type: tag
    framework: onnx
    spec: |
      [
        { "id": 0, "name": "zero" },
        { "id": 1, "name": "one" },
        { "id": 2, "name": "two" }
      ]
...

23.06.01 09:52:21.313                     nuctl (I) Function deploy complete ...

Your Environment

• CVAT version
  
• Docker version docker version (e.g. Docker 17.0.05): Docker 24.0.1
• Operating System and version (e.g. Linux, Windows, MacOS): server Ubuntu18.04, client Windows10

[bsekachev]

type can be only detector, interactor, tracker, reid.
In your case the correct value is probably detector.

You do not need special function.yaml for tags.
Just from main.py::handler you need to return list of structures where "type": "tag".

Like this:

results.append({
                "confidence": confidence_value,
                "label": "zero",
                "type": "tag",
            })

[tommy-hayashida]

Thanks!
After replacing 'tag' with 'detector' in function.yaml, the model now appears in the CVAT model list.

However, when I run the auto-annotation, I get an error message 'Could not create annotations Error: Object shape must be one of: ["rectangle","polygon","polyline","points","ellipse","cuboid","skeleton","mask"]'.

main.py is written as follows. Is this correct?

    ...
    results = [{'type': 'tag', 'label': infered_label}]
    return context.Response(body=json.dumps(results), headers={},
        content_type='application/json', status_code=200)

[tommy-hayashida]

I found this issue. #4212
Does this mean there are other codes to add?

[bsekachev]

1. You can try running the detector on task (offline mode)
2. In case with online mode, probably need to modify client code a little bit, because before we have not ever had any detectors returnings tags.

This file: cvat-ui/src/components/annotation-page/standard-workspace/controls-side-bar/tools-control.tsx.

In the simplest case to add something like this:

                                if (data.type === ObjectType.TAG) {
                                    return new core.classes.ObjectState({
                                        label: jobLabel,
                                        frame,
                                        objectType: ObjectType.TAG,
                                        source: 'auto',
                                    });
                                }

[tommy-hayashida]

I modifed tools-control.tsx.

Then I received another error message on CVAT.

Detection error occurred
Error: Request failed with status code 503. "HTTPConnectionPool(host='host.docker.internal', port=32769): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.co
nnection.HTTPConnection object at 0x7f390045f190>: Failed to establish a new connection: [Errno 111] Connection refused'))".

[bsekachev]

Try to remove the function and deploy it again.