Improve documentation for overlap parameter

[kodonnell]

My understanding is that overlap just specifies how many frames overlap when splitting a video into segments. If that's correct, what's the purpose? Does it actually do anything for the user? (E.g. if I do the tracks on the first segment, are they copied across to the next segment in the overlapping region? This doesn't appear to be the case.) Put another way- should I just set overlap=0 in my video tagging examples, to avoid having to manually resolve different taggings from each segment in the overlap?

Sorry if I've misunderstood something obvious.

PS - great tool!

[nmanovic]

Hi @kodonnell ,

Thanks for your question and feedback. Your understanding is correct. There are several ways to use the parameter:

• For an interpolation task (video sequence) if an object exists on overlapped segments it will be automatically merged into one track (with overlap=0 in annotation file you will have several tracks, one for each segment, which correspond to the object): https://github.com/opencv/cvat/blob/master/cvat/apps/engine/annotation.py#L824

• For an annotation task (separate images) if an object exists on overlapped segments bounding boxes will be automatically merged into one (if it is possible): https://github.com/opencv/cvat/blob/master/cvat/apps/engine/annotation.py#L897

• Use overlap=n to estimate quality of the annotations (it isn't implemented)

Does it answer your question?

[kodonnell]

Thanks for the thorough response @nmanovic

For an interpolation task (video sequence) if an object exists on overlapped segments it will be automatically merged into one track ...

Hmmm - I thought I tested this by creating a task with multiple segments and overlaps. When I added a track to the first segment (I just left one box in the same place the whole time), then closed it and opened the next segment, I didn't see the track in the overlap. Is there a step I missed? (I can also try to reproduce - I'm going by memory, so might not have done it correctly.)

For an annotation task (separate images) ...

Just to clarify for others reading this: I take it by 'separate images' you just mean single-frame annotations on the video? (Given CVAT is only for video.) Otherwise I'm not sure how separate images can overlap?

Use overlap=n to estimate quality of the annotations (it isn't implemented)

OK, I'm getting a little confused - I thought overlap=n meant there were n frames overlapping between segments?

Just thinking - maybe a video illustrating this would be good? I found the annotation video really useful, so maybe something quick demonstrating how segments/overlap work (and what happens in edge case you refer to e.g. "automatically merged into one (if it is possible)").

[nmanovic]

I didn't see the track in the overlap. Is there a step I missed?

You have to annotate it explicitly on all segments. After that merge will work. It will not propagate a bounding box from one segment to another automatically because in common case it isn't correct. Thus you annotate an object on first segment. You annotate the same object on second segment and if you do it right you will have one track inside your annotation file. If annotations on different segments (on overlapped frames) are very different or match is bad or overlap is 0 you will have 2 tracks for the same object.

I take it by 'separate images' you just mean single-frame annotations on the video? (Given CVAT is only for video.)

CVAT can annotate video or images (you can upload a set of images or even an archive with images). For example, you upload 100 images, segment size is 50 and overlap is 10. In this case you should have (if I remember correctly) 3 segments: 0 - 49, 40 - 89, 80 - 99. Thus two adjacent segments have 10 images overlap. In general "overlapped" images can be annotated differently. Merge works in such cases.

[kodonnell]

Thanks for the clarification.

It will not propagate a bounding box from one segment to another automatically because in common case it isn't correct.

In our case it would be correct to do this - we'd prefer to label something only once. I.e. I add tracks in segment 1, and then when I go to segment 2, it carries across all tracks from e.g. the last 10 frames of segment 1 (for overlap = 10) to the first 10 frames of segment 2. There won't be any need to merge the tracks, as it's obvious that if you've copied track 1, then it's still track 1. However - maybe ours is a special use case (and we have no experience with big tagging tasks).

Anyway, thank @nmanovic - feel free to close the task, or leave it open as a reminder to update the docs etc.