DeepStream을 통한 low precision YOLOv3 실행

DeepStream을 통한 low precision YOLOv3 실행

---

소스코드 다운로드 방법

공식 홈페이지에서 다운

• DeepStream SDK on Jetson Downloads

Github 에서 다운은 최신이긴 하나 여러 platform 빌드가 섞여있어서 compile이 쉽지는 않다.

• https://github.com/vat-nvidia/deepstream-plugins#note

trt-yolo는 github에 저장됨

DeepStream SDK 3.0 EA on Jetson AGX Xavier

• DeepStream SDK 3.0 EA on Jetson AGX Xavier
  Whatʼs new in this release Support for multi-stream applications: DeepStream 3.0 on Jetson AGX Xavier supports a new set of plugins and functionality for natively managing and batching multiple video streams in a single application for efficient memory management.

• Extending with data analytics: DeepStream application capabilities can be extended with stream and batch processing framework for processing of the metadata to gain rich insights. The SDK’s new plugins and reference application framework recipe can be used to seamlessly integrate these capabilities.

• An end-to-end deep learning solution for IVA with support for Transfer Learning Toolkit: Accelerate deep learning training by deploying high-performance pre-trained models optimized for NVIDIA Volta GPUs on AGX Xavier with DeepStream 3.0.
  Quick Start Instructions
  The DeepStream SDK requires:

NVIDIA® JetPack 4.1.1 or NVIDIA® Linux for Tegra (L4T) R31.1 on Ubuntu 18.04 or 16.04

• NVIDIA® Jetson AGX Xavier™ Developer Kit
• The DeepStream SDK package, including:
• binaries (deepstream_sdk_on_jetson.tbz2)
• sources (apps/ select plugins and libraries)
• samples (config files, models, video streams)

Additional components be installed on the target:

• CUDA 10.0
• cuDNN 7.3
• TensorRT 5.0
• OpenCV 3.3.1
• VisionWorks 1.6
• librdkafka (for messaging with the Cloud)

DeepStream을 이용해서 TensorRT로 최적화된 YOLOv3인 trt-yolo 실행하기

deepstream plugin github 코드를 다운 받음
yolo model weights를 다운 받음
data/yolo쪽으로 저장한다.

Repo의 root 디렉터리에서 아래와 같이 설정 후 실행
Makefile.config

CUDA_VER:=10.0
PLATFORM:=Tegra

cd source/apps/trt-yolo
make && sudo make install이후에 bin 디렉터리에 실행파일이 이동 된다. 따라서 trt-yolo-app명령어가 등록되어 진다.

내부적으로 보면 config/yolov3.txt를 수정하면 테스트 이미지를 볼 수 있다.

data/yolo에 weight값과 cfg파일 값 모두를 저장한다.

test_image.txt는 경로를 의미한다.

trt-yolo-app --flagfile=./config/yolov3.txt

Low-Precision 종류들

• kFLOAT (32 floating pointing)
• kHALF (16 floating pointing)
• kINT8 (integer pointing)

TRT-YOLOv3: kFLOAT

Using previously generated plan file located at data/yolo/yolov3-kFLOAT-batch1.engine
Loading TRT Engine...
Loading Complete!
Total number of images used for inference : 1
[======================================================================] 100 %
Network Type : yolov3Precision : kFLOAT Batch Size : 1 Inference time per image : 221.405 ms

TRT-YOLOv3: kHALF

Using previously generated plan file located at data/yolo/yolov3-kHALF-batch1.engine
Loading TRT Engine...
Loading Complete!
Total number of images used for inference : 1
[======================================================================] 100 %
Network Type : yolov3Precision : kHALF Batch Size : 1 Inference time per image : 83.944 ms

TRT-YOLOv3: kINT8, 실패

File does not exist : data/yolo/yolov3-kINT8-batch1.engine
Unable to find cached TensorRT engine for network : yolov3 precision : kINT8 and batch size :1
Building the TensorRT Engine...
New calibration table will be created to build the engine
Unable to open image : /path/to/image1.jpg
trt-yolo-app: ds_image.cpp:54: DsImage::DsImage(const string&, const int&, const int&): Assertion 0 failed.
Aborted (core dumped)

Xavier에 기본 YOLOv3

`./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg`
Loading weights from yolov3.weights...Done!
data/dog.jpg: Predicted in  `0.164729` seconds.

Geforce-1080에 기본 YOLOv3

Loading weights from yolov3.weights...Done!
data/dog.jpg: Predicted in 0.051647 seconds.