Merge remote-tracking branch 'origin/develop' into az/rename_git_app

CHANGELOG.md

@@ -19,13 +19,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 It supports regular navigation, searching a frame according to annotations
 filters and searching the nearest frame without any annotations (<https://github.com/openvinotoolkit/cvat/pull/2221>)
 - MacOS users notes in CONTRIBUTING.md
+- Ability to prepare meta information manually (<https://github.com/openvinotoolkit/cvat/pull/2217>)
+- Ability to upload prepared meta information along with a video when creating a task (<https://github.com/openvinotoolkit/cvat/pull/2217>)
+- Optional chaining plugin for cvat-canvas and cvat-ui (<https://github.com/openvinotoolkit/cvat/pull/2249>)
 
 ### Changed
 - UI models (like DEXTR) were redesigned to be more interactive (<https://github.com/opencv/cvat/pull/2054>)
 - Used Ubuntu:20.04 as a base image for CVAT Dockerfile (<https://github.com/opencv/cvat/pull/2101>)
 - Right colors of label tags in label mapping when a user runs automatic detection (<https://github.com/openvinotoolkit/cvat/pull/2162>)
 - Nuclio became an optional component of CVAT (<https://github.com/openvinotoolkit/cvat/pull/2192>)
 - A key to remove a point from a polyshape [Ctrl => Alt] (<https://github.com/openvinotoolkit/cvat/pull/2204>)
+- Updated `docker-compose` file version from `2.3` to `3.3`(<https://github.com/openvinotoolkit/cvat/pull/2235>)
 
 ### Deprecated
 -
@@ -37,12 +41,14 @@ filters and searching the nearest frame without any annotations (<https://github
 - Fixed multiple errors which arises when polygon is of length 5 or less (<https://github.com/opencv/cvat/pull/2100>)
 - Fixed task creation from PDF (<https://github.com/opencv/cvat/pull/2141>)
 - Fixed CVAT format import for frame stepped tasks (<https://github.com/openvinotoolkit/cvat/pull/2151>)
+- Fixed the reading problem with large PDFs (<https://github.com/openvinotoolkit/cvat/pull/2154>)
 - Fixed unnecessary pyhash dependency (<https://github.com/openvinotoolkit/cvat/pull/2170>)
 - Fixed Data is not getting cleared, even after deleting the Task from Django Admin App(<https://github.com/openvinotoolkit/cvat/issues/1925>)
 - Fixed blinking message: "Some tasks have not been showed because they do not have any data" (<https://github.com/openvinotoolkit/cvat/pull/2200>)
 - Fixed case when a task with 0 jobs is shown as "Completed" in UI (<https://github.com/openvinotoolkit/cvat/pull/2200>)
 - Fixed use case when UI throws exception: Cannot read property 'objectType' of undefined #2053 (<https://github.com/openvinotoolkit/cvat/pull/2203>)
 - Fixed use case when logs could be saved twice or more times #2202 (<https://github.com/openvinotoolkit/cvat/pull/2203>)
+- Fixed issues from #2112 (<https://github.com/openvinotoolkit/cvat/pull/2217>)
 - Git application name (renamed to dataset_repo) (<https://github.com/openvinotoolkit/cvat/pull/2243>)
 
 ### Security

components/analytics/docker-compose.analytics.yml

@@ -1,4 +1,4 @@
-version: '2.3'
+version: '3.3'
 services:
   cvat_elasticsearch:
     container_name: cvat_elasticsearch

components/serverless/docker-compose.serverless.yml

@@ -1,4 +1,4 @@
-version: '2.3'
+version: '3.3'
 services:
   serverless:
     container_name: nuclio

cvat-canvas/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cvat-canvas",
-  "version": "2.1.1",
+  "version": "2.1.2",
   "description": "Part of Computer Vision Annotation Tool which presents its canvas library",
   "main": "src/canvas.ts",
   "scripts": {
@@ -20,6 +20,7 @@
     "@babel/cli": "^7.5.5",
     "@babel/core": "^7.5.5",
     "@babel/plugin-proposal-class-properties": "^7.8.3",
+    "@babel/plugin-proposal-optional-chaining": "^7.11.0",
     "@babel/preset-env": "^7.5.5",
     "@babel/preset-typescript": "^7.3.3",
     "@types/node": "^12.6.8",

cvat-canvas/webpack.config.js

@@ -28,7 +28,10 @@ const nodeConfig = {
             use: {
                 loader: 'babel-loader',
                 options: {
-                    plugins: ['@babel/plugin-proposal-class-properties'],
+                    plugins: [
+                        '@babel/plugin-proposal-class-properties',
+                        '@babel/plugin-proposal-optional-chaining'
+                    ],
                     presets: [
                         ['@babel/preset-env'],
                         ['@babel/typescript'],

cvat-ui/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cvat-ui",
-  "version": "1.9.8",
+  "version": "1.9.9",
   "description": "CVAT single-page application",
   "main": "src/index.tsx",
   "scripts": {
@@ -16,6 +16,7 @@
   "devDependencies": {
     "@babel/core": "^7.6.0",
     "@babel/plugin-proposal-class-properties": "^7.5.5",
+    "@babel/plugin-proposal-optional-chaining": "^7.11.0",
     "@babel/preset-env": "^7.6.0",
     "@babel/preset-react": "^7.0.0",
     "@babel/preset-typescript": "^7.6.0",

cvat-ui/webpack.config.js

@@ -39,9 +39,13 @@ module.exports = {
             use: {
                 loader: 'babel-loader',
                 options: {
-                    plugins: ['@babel/plugin-proposal-class-properties', ['import', {
-                        'libraryName': 'antd',
-                    }]],
+                    plugins: [
+                        '@babel/plugin-proposal-class-properties',
+                        '@babel/plugin-proposal-optional-chaining',
+                        ['import', {
+                            'libraryName': 'antd',
+                        }]
+                    ],
                     presets: [
                         ['@babel/preset-env', {
                             targets: '> 2.5%', // https://github.com/browserslist/browserslist

cvat/apps/documentation/data_on_fly.md

@@ -0,0 +1,35 @@
+# Data preparation on the fly
+
+## Description
+Data on the fly processing is a way of working with data, the main idea of which is as follows:
+Minimum necessary meta information is collected, when task is created.
+This meta information allows in the future to create a necessary chunks when receiving a request from a client.
+
+Generated chunks are stored in a cache of limited size with a policy of evicting less popular items.
+
+When a request received from a client, the required chunk is searched for in the cache.
+If the chunk does not exist yet, it is created using a prepared meta information and then put into the cache.
+
+This method of working with data  allows:
+- reduce the task creation time.
+- store data in a cache of limited size with a policy of evicting less popular items.
+
+## Prepare meta information
+Different meta information is collected for different types of uploaded data.
+### Video
+For video, this is a valid mapping of key frame numbers and their timestamps. This information is saved to `meta_info.txt`.
+
+Unfortunately, this method will not work for all videos with valid meta information.
+If there are not enough keyframes in the video for smooth video decoding, the task will be created in the old way.
+
+#### Uploading meta information along with data
+
+When creating a task, you can upload a file with meta information along with the video,
+which will further reduce the time for creating a task.
+You can see how to prepare meta information [here](/utils/prepare_meta_information/README.md).
+
+It is worth noting that the generated file also contains information about the number of frames in the video at the end.
+
+### Images
+Mapping of chunk number and paths to images that should enter the chunk
+is saved at the time of creating a task in a files `dummy_{chunk_number}.txt`