-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modernize OpenVINO-based Nuclio functions and allow them to run on Kubernetes #6129
Changes from all commits
f06ee38
fe31d61
1687e75
0228ce6
9057d8a
b61aa7a
8230d9b
fc3cca7
ccad38c
6adb68f
244e997
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
This file was deleted.
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,30 @@ | ||
#!/bin/bash | ||
# Sample commands to deploy nuclio functions on CPU | ||
|
||
set -eu | ||
|
||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
FUNCTIONS_DIR=${1:-$SCRIPT_DIR} | ||
|
||
nuctl create project cvat | ||
export DOCKER_BUILDKIT=1 | ||
|
||
docker build -t cvat.openvino.base "$SCRIPT_DIR/openvino/base" | ||
|
||
nuctl create project cvat --platform local | ||
|
||
shopt -s globstar | ||
|
||
for func_config in "$FUNCTIONS_DIR"/**/function.yaml | ||
do | ||
func_root=$(dirname "$func_config") | ||
echo Deploying $(dirname "$func_root") function... | ||
nuctl deploy --project-name cvat --path "$func_root" \ | ||
--volume "$SCRIPT_DIR/common:/opt/nuclio/common" \ | ||
--platform local | ||
done | ||
func_root="$(dirname "$func_config")" | ||
func_rel_path="$(realpath --relative-to="$SCRIPT_DIR" "$(dirname "$func_root")")" | ||
Comment on lines
+19
to
+20
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should not we updated deploy_gpu.sh the same way? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right, thanks for the reminder. I ported relevant changes to |
||
|
||
nuctl get function | ||
if [ -f "$func_root/Dockerfile" ]; then | ||
docker build -t "cvat.${func_rel_path//\//.}.base" "$func_root" | ||
fi | ||
|
||
echo "Deploying $func_rel_path function..." | ||
nuctl deploy --project-name cvat --path "$func_root" --platform local | ||
done | ||
|
||
nuctl get function --platform local |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,23 @@ | ||
#!/bin/bash | ||
# Sample commands to deploy nuclio functions on GPU | ||
|
||
set -eu | ||
|
||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
FUNCTIONS_DIR=${1:-$SCRIPT_DIR} | ||
|
||
nuctl create project cvat | ||
nuctl create project cvat --platform local | ||
|
||
shopt -s globstar | ||
|
||
for func_config in "$FUNCTIONS_DIR"/**/function-gpu.yaml | ||
do | ||
func_root=$(dirname "$func_config") | ||
echo "Deploying $(dirname "$func_root") function..." | ||
func_root="$(dirname "$func_config")" | ||
func_rel_path="$(realpath --relative-to="$SCRIPT_DIR" "$(dirname "$func_root")")" | ||
|
||
echo "Deploying $func_rel_path function..." | ||
nuctl deploy --project-name cvat --path "$func_root" \ | ||
--volume "$SCRIPT_DIR/common:/opt/nuclio/common" \ | ||
--file "$func_config" --platform local | ||
done | ||
|
||
nuctl get function | ||
nuctl get function --platform local |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
FROM openvino/ubuntu20_runtime:2022.3.0 | ||
|
||
USER root | ||
|
||
RUN apt-get update \ | ||
&& apt-get -y --no-install-recommends install python-is-python3 \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
RUN pip install --no-cache-dir opencv-python-headless pillow pyyaml | ||
|
||
COPY model_loader.py shared.py /opt/nuclio/common/openvino/ | ||
|
||
ENV PYTHONPATH=/opt/nuclio/common/openvino:$PYTHONPATH | ||
|
||
USER openvino |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
FROM openvino/ubuntu20_dev:2022.3.0 AS build | ||
|
||
USER root | ||
|
||
RUN apt-get update \ | ||
&& apt-get -y --no-install-recommends install patch \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
WORKDIR /root | ||
|
||
ARG DEXTR_COMMIT=352ccc76067156ebcf7267b07e0a5e43d32e83d5 | ||
|
||
# TODO: use `ADD --checksum` when that feature becomes stable | ||
ADD https://data.vision.ee.ethz.ch/csergi/share/DEXTR/dextr_pascal-sbd.pth ./ | ||
|
||
ADD https://github.com/scaelles/DEXTR-PyTorch/archive/$DEXTR_COMMIT.zip dextr.zip | ||
|
||
RUN python3 -m zipfile -e dextr.zip . | ||
|
||
WORKDIR /root/DEXTR-PyTorch-$DEXTR_COMMIT | ||
|
||
ADD export.py adaptive-pool.patch . | ||
|
||
RUN patch -p1 -i adaptive-pool.patch | ||
|
||
RUN python3 export.py /root/dextr_pascal-sbd.pth /root/dextr.onnx | ||
|
||
RUN mo --input_model=/root/dextr.onnx --model_name=dextr --output_dir=/root | ||
|
||
FROM cvat.openvino.base | ||
|
||
COPY --from=build --chown=root:root /root/dextr.xml /root/dextr.bin /opt/nuclio/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
This is a hack to work around the the lack of support for AdaptiveAvgPool2d | ||
in PyTorch's ONNX exporter (<https://github.com/pytorch/pytorch/issues/42653>). | ||
|
||
It might become unnecessary in the future, since OpenVINO 2023 is to add support | ||
for AdaptiveAvgPool2d exported with operator_export_type=ONNX_ATEN_FALLBACK | ||
(<https://github.com/openvinotoolkit/openvino/pull/14682>). | ||
|
||
diff --git a/networks/deeplab_resnet.py b/networks/deeplab_resnet.py | ||
index ecfa084..e8ff297 100644 | ||
--- a/networks/deeplab_resnet.py | ||
+++ b/networks/deeplab_resnet.py | ||
@@ -99,7 +99,14 @@ class PSPModule(nn.Module): | ||
self.final = nn.Conv2d(out_features, n_classes, kernel_size=1) | ||
|
||
def _make_stage_1(self, in_features, size): | ||
- prior = nn.AdaptiveAvgPool2d(output_size=(size, size)) | ||
+ kernel_size, stride = { | ||
+ 1: (64, 64), | ||
+ 2: (32, 32), | ||
+ 3: (22, 21), | ||
+ 6: (11, 9), | ||
+ }[size] | ||
+ | ||
+ prior = nn.AvgPool2d(kernel_size=kernel_size, stride=stride) | ||
conv = nn.Conv2d(in_features, in_features//4, kernel_size=1, bias=False) | ||
bn = nn.BatchNorm2d(in_features//4, affine=affine_par) | ||
relu = nn.ReLU(inplace=True) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
#!/usr/bin/env python3 | ||
|
||
import sys | ||
|
||
import torch | ||
import torch.nn | ||
import torch.onnx | ||
|
||
import networks.deeplab_resnet as resnet | ||
|
||
net = resnet.resnet101(1, nInputChannels=4, classifier='psp') | ||
|
||
state_dict_checkpoint = torch.load(sys.argv[1], map_location=torch.device('cpu')) | ||
|
||
net.load_state_dict(state_dict_checkpoint) | ||
|
||
full_net = torch.nn.Sequential( | ||
net, | ||
torch.nn.Upsample((512, 512), mode='bilinear', align_corners=True), | ||
torch.nn.Sigmoid(), | ||
) | ||
full_net.eval() | ||
|
||
input_tensor = torch.randn((1, 4, 512, 512)) | ||
|
||
torch.onnx.export(full_net, input_tensor, sys.argv[2]) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
FROM openvino/ubuntu20_dev:2022.3.0 AS build | ||
|
||
USER root | ||
|
||
RUN omz_downloader \ | ||
--name face-detection-0205,emotions-recognition-retail-0003,age-gender-recognition-retail-0013 \ | ||
--precisions FP32 \ | ||
-o /opt/nuclio/open_model_zoo | ||
|
||
FROM cvat.openvino.base | ||
|
||
COPY --from=build --chown=root:root /opt/nuclio/open_model_zoo /opt/nuclio/open_model_zoo |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
FROM openvino/ubuntu20_dev:2022.3.0 AS build | ||
|
||
USER root | ||
|
||
RUN omz_downloader \ | ||
--name person-reidentification-retail-0277 \ | ||
--precisions FP32 \ | ||
-o /opt/nuclio/open_model_zoo | ||
|
||
FROM cvat.openvino.base | ||
|
||
USER root | ||
|
||
RUN pip install --no-cache-dir scipy | ||
|
||
COPY --from=build /opt/nuclio/open_model_zoo /opt/nuclio/open_model_zoo | ||
|
||
USER openvino |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In our documentation we have some lines to build nuclio functions without
deploy_cpu.sh
script.https://opencv.github.io/cvat/docs/contributing/setup-additional-components/
Don't they work anymore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I updated the docs. One of the
nuctl deploy
commands uses a TF function, which is unaffected by these changes, so I kept it as-is. The rest I replaced with calls todeploy_cpu.sh
.