ToolBox/ Scripts to extract annotations from CVAT XML files

[pnambiar]

Looks like the CVAT XML converter (to Pascal VOC) script works only on data collected in the annotation mode and not on interpolation mode. It will be very useful to have a matlab or python scripts/ toolbox available to extract bounding box coordinates files takes in interpolation mode (with track id). This will help us convert BB to whatever format we want (Eg.YOLO format)

[nmanovic]

Sample scripts can be found here: https://github.com/opencv/cvat/tree/develop/utils
In context of the task it will be necessary:

• Prepare a python library which can parse CVAT XML annotations into some python structures (you can call it cvat/utils/cvat/parser or something like that). The library should support both interpolation and annotation CVAT XML formats.
• Improve documentation for CVAT XML format if it is necessary (https://github.com/opencv/cvat/blob/develop/cvat/apps/documentation/xml_format.md)
• Reuse the library for all existing converters inside cvat/utils directory (to eluminate code duplication)
• Pascal VOC should support "interpolation mode" as well
• Support YOLO format
• Support TF Record format

All converters should have some tests to be sure that we don't break them in the future.

[jrjbertram]

https://gist.github.com/jrjbertram/7cb44b58c590eefe48f44274a1dc3fee

a quick hack of converter.py that handles interpolation. this could be a stepping stone towards the PR solution.

note the hack has some rough edges. it breaks support for non-interpolated annotation, so that would need fixed in the full version. it also has hard coded video frame sizes which would also need fixed.

it wasn't clear to me if interpolated and non-interpolated annotation can be used in the same video file?

one other issue is that when CVAT is used to annotate a video and you want to generate a Pascal VOC dataset, you will have to manually extract each frame into a directory. I used ffmpeg to do this.

hopefully this helps someone!

• Josh

[dbarrejon]

Hello everyone! I see that the our goal is pretty much the same at using this annotation tool @jrjbertram, at least sorta! I want to obtain a data set of annotated video, and using your contributoin of converting from CVAT to Pascal VOC will surely help me.
But again I still need to extract the corresponding annotated frames from the video. You have used ffmpeg to do so, right? Which command is it that you have used? Did you have to specify manually each single frame you wanted to extract, could you pass a .txt file or anything to specify it, or did you even use some python script to do so?

I would be really greatful if you could help me in this :)

Thank you very much!

Daniel

[jrjbertram]

@100330734 I think that we've determined on the cvat chat channel that cvat uses -vsync 0, so the following should work. I'll be testing it on my dataset sometime in the next couple of days hopefully.

ffmpeg -i -vsync 0 /frame_%08d.jpg

[nmanovic]

I'm going to close the issue. Now CVAT supports downloading annotations in Pascal VOC directly from UI. Soon (in a week) it will support YOLO and TF records.

[tanmayGIT]

Hi,
I am also facing a similar problem. I have annotations on the image level. I have labeled some images as invalid during the annotation.

But when I export the "Pascal VOC XML" format, I don't get the information, related to these image-level labels i.e. "unvalid" or "invalid"

Any idea?

[nmanovic]

@tanmayGIT , how should information about image-level labels be present in Pascal VOC XML format?

[tanmayGIT]

Thanks for your response. Then how could I obtain the image level label information from the exported annotations? Should I use some other annotation formation to get such information ?
I really appreciate any help you can provide.

[tanmayGIT]

Hello again,
In fact, I got the answer. If we export in "CVAT for Images 1.1" format then we will have the information at the image level as well as the annotation level. Then, we just need to parse the XML and convert it into the desired format. For example in my case, I needed it in the YOLO format.

[tanmayGIT]

Here is the code, I wrote to parse the CVAT formatted XML file, it may help others :
Note the image level tag is obtained as: image_tag_meta = elem.findall("tag"). Then, I filter out the

unvalid i.e. invalid tags.

    def parse_cvat_xml(xml_file):
        """
            Function to read the XML file and extract the information in dictionary format:
            Args:
                xml_file: The path of the XML file
            Returns:
        """
        root = EleTree.parse(xml_file).getroot()

        keep_all_valid_imag_info_together = []
        get_meta_data = root.findall("image")

        # Parse the XML Tree
        for elem in get_meta_data:

            image_whole_info = {}

            image_id = int(elem.attrib["id"])
            image_full_path = elem.attrib["name"]
            image_width = int(elem.attrib["width"])
            image_height = int(elem.attrib["height"])

            image_whole_info["image_id"] = image_id
            image_whole_info["full_path"] = image_full_path
            image_whole_info["image_width"] = image_width
            image_whole_info["image_height"] = image_height

            imag_id_valid_flag = AnalyzeAnnotationFiles.verify_annotation_image_ids_range(image_id)
            if not imag_id_valid_flag:  # if the image id flag is not valid i.e. the image is out of required range
                continue

            image_tag_meta = elem.findall("tag")

            unvalid_flag = False
            if any(True for _ in image_tag_meta):

                for get_tag in image_tag_meta:
                    obtain_tag = get_tag.attrib["label"]
                    if obtain_tag == "unvalid":
                        unvalid_flag = True
                        break

            if unvalid_flag:
                continue

            image_level_all_bbox_yolo_coords = {'yolo_coords': []}
            image_level_all_bbox_attrib = {'boox_attrib': []}

            object_metas = elem.findall("box")
            for bbox in object_metas:

                box_label = bbox.attrib["label"]
                xtl = float(bbox.attrib["xtl"])
                ytl = float(bbox.attrib["ytl"])
                xbr = float(bbox.attrib["xbr"])
                ybr = float(bbox.attrib["ybr"])

                # CVAT to yolo
                yolo_x = round(((xtl + xbr) / 2) / image_width, 6)
                yolo_y = round(((ytl + ybr) / 2) / image_height, 6)
                yolo_w = round((xbr - xtl) / image_width, 6)
                yolo_h = round((ybr - ytl) / image_height, 6)

                # Keeping all the yolo coords
                image_level_all_bbox_yolo_coords['yolo_coords'].append([box_label, yolo_x, yolo_y, yolo_w, yolo_h])

                bbox_metas = bbox.findall("attribute")

                keep_each_attribute = {}
                for attr in bbox_metas:
                    attribute_name = attr.attrib["name"]
                    attribute_value = attr.text

                    keep_each_attribute[attribute_name] = attribute_value

                image_level_all_bbox_attrib['boox_attrib'].append(keep_each_attribute)

            # keeping all the bbox yolo coordinates info of the image
            image_whole_info["all_bbox_yolo_coords"] = image_level_all_bbox_yolo_coords

            # keeping all the bbox attribute info of the image
            image_whole_info["all_bbox_attributes_info"] = image_level_all_bbox_attrib

            keep_all_valid_imag_info_together.append(image_whole_info)

        return keep_all_valid_imag_info_together

See the final dictionary will look like this for all the images in the dataset (e.g. 2107 images here)

Then let's say the information of the 7th image looks like this; where I keep the information of image id, image path, width, and height. I also keep the information, related to all the bounding box coordinates in the images in the dictionary, called "all_bbox_yolo_coords" and the attribute information of each bounding box is kept in the dictionary, called "all_box_attributes_info"

The "all_bbox_yolo_coords" dictionary looks like this, where the information, related to all the bounding boxes is kept :

The dictionary of all the attributes, related to each bounding box is kept like this :