Stage 3: Optimizing TensorRT Inference - [x] [TensorRT Loading YoloV8 Example](https://freedium.cfd/https://alimustoofaa.medium.com/how-to-load-model-yolov8-tensorrt-6c03de059622) - [ ] [Old Nvidia Guide for Inference](https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/) - [ ] [New Nvidia Guide for Inference](https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorrt-updated/) - [x] [ONNX Example](https://docs.nvidia.com/deeplearning/tensorrt/quick-start-guide/index.html#ex-deploy-onnx) - [ ] Experiment with FP16 Precision mode, compare and report results - [ ] Experiment with INT8 Precision mode, (research process for calibrating as well), compare and report results - [ ] [Quantization Basics](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/quantization-basics-2b9b1a49882d) - [ ] [Quantization Blog](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/quantization-yolov8-qat-x2-speed-up-on-your-jetson-orin-nano-2-how-to-achieve-the-best-qat-c6069fb83ab7) - [ ] [Quantization Blog 2](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/quantization-achieve-accuracy-drop-to-near-zero-yolov8-qat-x2-speed-up-on-your-jetson-orin-2b99819775e4) - [ ] [Quantization Blog 3](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/quantization-yolov8-qat-x2-speed-up-on-your-jetson-orin-nano-1-why-quantization-e052a72c506d) - [ ] [Quantization Blog 4](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/quantization-go-faster-with-relu-yolov8-qat-x2-speed-up-on-your-jetson-orin-nano-3-4d4733c9e435) - [ ] [Knowledge Distillation Blog](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/yolov9-model-optimization-knowledge-distillation-1-why-knowledge-distillation-for-object-08d420499141) - [ ] [Knowledge Distillation Blog 2](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/yolov9-model-optimization-knowledge-distillation-2-how-to-implement-feature-based-kd-5bdae730094d) - [ ] [Batched NMS Blog](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/batched-nms-1-yolov8-model-modification-without-modeler-using-onnx-graphsurgeon-d876b75478af) - [ ] [Batched NMS Blog 2](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/batched-nms-2-running-yolov8-on-deepstream-8229096bc85b) - [ ] [Batched NMS Blog 3](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/batched-nms-3-improving-speed-by-insreting-efficientnms-plugin-38d8e388e6f3) - [ ] [Batched NMS Blog 4](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/yolov8-batch-inference-implementation-using-tensorrt-3-batch-inference-using-tensorrt-python-cf30ae10920c) - [ ] [Batched NMS Blog 5](https://freedium.cfd/https://medium.com/@DeeperAndCheaper/yolov8-batch-inference-implementation-using-tensorrt-4-nms-post-processing-implementation-daecfef41b78)
Stage 3: Optimizing TensorRT Inference