NVIDIA DeepStream on Jetson Orin NX | Low-Latency Vision AI with the AstrOptix Orin Super Carrier Board and Sony FCB-EV9520L
Introduction
As Vision AI applications continue to move from research labs into production environments, developers face an increasingly common challenge: delivering high inference throughput while maintaining low latency on resource-constrained edge devices.
Applications such as industrial inspection, intelligent surveillance, robotics, autonomous systems, and traffic analytics require AI pipelines that can process live camera streams continuously without sacrificing accuracy or responsiveness.
This is where NVIDIA DeepStream SDK becomes one of the most powerful software frameworks available for edge AI deployment.
DeepStream is a GPU-accelerated streaming analytics SDK built on top of the GStreamer multimedia framework. Rather than treating video processing and AI inference as separate stages, DeepStream combines video decoding, preprocessing, TensorRT inference, object tracking, visualization, metadata generation, encoding, and cloud connectivity into a single optimized pipeline. This architecture minimizes unnecessary memory copies, maximizes GPU utilization, and significantly simplifies production deployment. DeepStream further expands these capabilities with JetPack support, an open-source GitHub monorepo, Python and C++ development workflows, Service Maker, and new multi-camera analytics capabilities.
Watch the Full Demo Video
See this exact DeepStream pipeline running live on the Jetson Orin NX with the AstrOptix Orin Super Carrier Board and the Sony FCB-EV9520L camera.
Our Development Environment
For our implementation, we deployed DeepStream on the latest NVIDIA Jetson software stack.
Hardware Stack
- NVIDIA Jetson Orin NX
- Sony FCB EV9520L Camera
- AstrOptix Orin Super Carrier Board (with built-in LVDS to MIPI CSI-2 bridge)
- Jetson configured in MAXN SUPER (Uncapped Maximum Performance Mode)
Software Stack
| Component | Version |
|---|---|
| JetPack | 7.2 |
| CUDA | 13.2 |
| TensorRT | 10.16.2.10 |
| GStreamer | 1.24.2 |
| DeepStream SDK | 9.1 |
This latest software stack provides improved compatibility between CUDA, TensorRT and DeepStream while offering access to the newest optimizations available for NVIDIA edge platforms. DeepStream 9.1 officially supports JetPack 7.2 on Jetson Orin platforms.
AstrOptix Orin Super Carrier Board
The AstrOptix Orin Super Carrier Board, from OPPILA, is the hardware foundation that makes this entire pipeline possible. It is a compact carrier board built specifically for Jetson Orin Nano and Orin NX modules, and it solves a very practical integration problem: professional broadcast and industrial cameras such as the Sony FCB-EV9520L output video over LVDS, while Jetson platforms natively expect a MIPI CSI-2 camera input.
Instead of requiring an external converter box or a custom cabling solution, the AstrOptix Orin Super integrates an on-board LVDS to MIPI CSI-2 bridge directly into the carrier board. This gives a direct, low-latency path from the camera sensor straight into the Jetson's camera capture pipeline, which is what allows DeepStream to begin processing frames with minimal added delay.
Key features of the board
- Compatible with both Jetson Orin Nano and Jetson Orin NX modules
- Camera support for Sony, Tamron, and KT&C sensors
- Ultra-compact form factor, suited for space-constrained enclosures
- MAXN-Super ready, allowing the Jetson module to run in uncapped maximum performance mode
- Built-in LVDS to MIPI CSI-2 conversion, removing the need for external bridging hardware
Product link: AstrOptix Orin Super Carrier Board
AI Vision Dataflow Architecture
The diagram below traces the full path a video frame takes through this system, from the Sony camera sensor to the final AI-annotated output. The Sony FCB-EV9520L outputs raw video over LVDS into the AstrOptix Orin Super Carrier Board, where the on-board bridge converts it to MIPI CSI-2. This is handed off to the NVIDIA Jetson Orin NX, where DeepStream takes over: decoding, batching, running TensorRT inference, tracking objects, and producing the real-time object detection overlay shown on screen.
Understanding the DeepStream Pipeline
A typical DeepStream application follows a sequence of optimized processing stages:
- Camera Source
- Video Decode / Capture
- Stream Multiplexer
- Pre-processing
- TensorRT Inference
- Object Tracking
- On Screen Display
- Video Rendering / Streaming
Each stage is implemented using dedicated GStreamer plugins.
Some of the most commonly used plugins include:
- nvstreammux - batches one or multiple video streams.
- nvinfer - performs TensorRT-based inference.
- nvtracker - assigns persistent IDs to detected objects.
- nvdsosd - overlays bounding boxes, labels, and confidence scores.
- nvvideoconvert - performs hardware-accelerated color and format conversion.
- nvmsgconv and nvmsgbroker - prepare and transmit inference metadata to external systems.
This plugin-based architecture makes DeepStream highly modular, allowing developers to replace or extend individual components without redesigning the entire pipeline.
Memory Optimization Matters
For real-time applications, inference speed alone is not sufficient. Poor buffer management can introduce unnecessary latency, increased memory consumption, or dropped frames.
DeepStream addresses this through:
- Zero-copy GPU memory wherever possible
- Optimized buffer pools
- Asynchronous pipeline execution
- Parallel plugin scheduling
- Efficient batch processing
These optimizations are particularly valuable on embedded Jetson platforms where memory bandwidth is a critical resource.
Achieving Low Latency on Jetson Orin NX
Our objective was straightforward: achieve reliable real-time object detection with minimal end-to-end latency on embedded hardware.
Using the hardware and software stack described earlier, we deployed an object detection pipeline with:
- Sony FCB EV9520L camera
- Jetson Orin NX
- DeepStream 9.1
- TensorRT 10.16
- CUDA 13.2
- GStreamer 1.24.2
The Jetson was configured in Uncapped Maximum Performance (MAXN SUPER) mode to ensure all CPU and GPU resources were available for the pipeline.
After optimizing the inference pipeline, camera processing, and GPU utilization, we achieved an approximate end-to-end latency of 147 milliseconds during real-time object detection.
For high-resolution video analytics running entirely on an embedded edge device, this demonstrates the effectiveness of combining DeepStream's optimized pipeline with TensorRT acceleration.
In the demo video, we also captured this latency with a live glass-to-glass timer test: two synchronized stopwatches, one displayed on the physical scene and one on the DeepStream output monitor. The measured difference confirmed a real-world end-to-end latency in the same range as our benchmarked figure, validating the pipeline's responsiveness under actual hardware conditions rather than in a synthetic test alone.
DeepStream 9.1: What's New?
The latest DeepStream release introduces several notable improvements:
- Native support for JetPack 7.2
- Unified open-source GitHub repository
- Enhanced Python development workflows
- NVIDIA Service Maker for simplified application development
- Multi-View 3D Tracking (MV3DT)
- AutoMagicCalib for multi-camera calibration
- Improved deployment workflows for edge and server environments
These additions make DeepStream suitable not only for single-camera applications but also for enterprise-scale multi-camera AI deployments.
Lessons Learned During Deployment
Deploying production-grade Vision AI involves more than simply running a neural network. Several practical observations proved valuable during development:
- Maintaining GPU-resident buffers reduces unnecessary memory transfers.
- TensorRT optimization significantly improves inference performance.
- Proper GStreamer pipeline design has a direct impact on latency.
- Hardware configuration (such as MAXN SUPER mode) influences sustained performance.
- Selecting compatible versions of JetPack, CUDA, TensorRT, GStreamer, and DeepStream avoids integration issues and simplifies deployment.
Optimizing these factors collectively produced a stable and responsive real-time inference pipeline.
Long-Range AI Detection with InfinitEye
The same Sony FCB-EV9520L and Jetson Orin NX combination featured in this DeepStream pipeline also powers OPPILA's InfinitEye AI Platform, built specifically for long-range object detection. By pairing the camera's 30x optical zoom with the Jetson's GPU-accelerated inference, InfinitEye achieves reliable detection up to 2 km with no additional optics, extendable to 4-5 km with a 2x teleconverter lens.
The InfinitEye software stack layers YOLOv8 for detection, TensorFlow/PyTorch for training and fine-tuning, TensorRT with DeepStream and Jetson-Inference for accelerated deployment, and DeepSORT for multi-object tracking, all running on JetPack. In field testing, the platform maintained 147 ms end-to-end processing latency at Full HD 60 FPS, with detections of humans, vehicles, and aircraft.
This makes InfinitEye well suited to applications where DeepStream's short-range detection needs to extend much further out, such as:
- Border and coastal surveillance
- Traffic law enforcement
- Critical infrastructure protection
- Forest and wildlife monitoring
Watch the InfinitEye long-range detection demo:
Future Directions
DeepStream continues to evolve beyond traditional video analytics.
Emerging capabilities include:
- Vision-Language Model integration
- Multi-camera object association
- Edge-to-cloud analytics
- Distributed AI pipelines
- Agentic Vision AI workflows
- Large-scale smart city deployments
As AI applications become increasingly sophisticated, DeepStream provides a scalable foundation for deploying intelligent video analytics across embedded devices, enterprise servers, and cloud infrastructure.
Conclusion
NVIDIA DeepStream has matured into a comprehensive platform for production-grade Vision AI. By combining GPU-accelerated multimedia processing, TensorRT inference, efficient memory management, and modular pipeline architecture, it enables developers to build high-performance applications without reinventing the entire video processing stack.
Our implementation on the NVIDIA Jetson Orin NX using JetPack 7.2, CUDA 13.2, TensorRT 10.16.2.10, GStreamer 1.24.2, and DeepStream 9.1 demonstrated that optimized edge AI deployment is achievable even with high-resolution camera inputs. Using a Sony FCB camera in conjunction with a carefully tuned DeepStream pipeline running in MAXN mode, we achieved approximately 147 ms end-to-end latency for real-time object detection.
As Vision AI adoption continues to accelerate across industries, frameworks such as DeepStream will remain essential for transforming AI models into robust, scalable, and production-ready solutions.
Watch & Learn More
- DeepStream + AstrOptix Orin Super demo video: https://youtu.be/tKIsb7kH-Vg
- InfinitEye long-range detection demo video: https://youtu.be/fb79swHFroQ
- AstrOptix Orin Super Carrier Board product page: https://www.oppila.in/products/orin-nx-v2
- OPPILA: https://www.oppila.in
FAQs
NVIDIA DeepStream is a GPU-accelerated streaming analytics SDK built on top of the GStreamer multimedia framework. It combines video decoding, preprocessing, TensorRT inference, object tracking, visualization, metadata generation, encoding, and cloud connectivity into a single optimized pipeline, minimizing memory copies and maximizing GPU utilization.
The pipeline was deployed on JetPack 7.2 with CUDA 13.2, TensorRT 10.16.2.10, GStreamer 1.24.2, and DeepStream SDK 9.1. DeepStream 9.1 officially supports JetPack 7.2 on Jetson Orin platforms.
After optimizing the inference pipeline, camera processing, and GPU utilization, the deployment achieved an approximate end-to-end latency of 147 milliseconds during real-time object detection at high resolution.
In addition to benchmarking, a live glass-to-glass timer test was performed using two synchronized stopwatches, one displayed on the physical scene and one on the DeepStream output monitor. The measured difference confirmed a real-world end-to-end latency in the same range as the benchmarked figure.
The AstrOptix Orin Super Carrier Board integrates an on-board LVDS to MIPI CSI-2 bridge directly into the carrier board. This gives a direct, low-latency path from an LVDS camera such as the Sony FCB-EV9520L into the Jetson camera capture pipeline, removing the need for an external converter box or custom cabling.
The board is compatible with both Jetson Orin Nano and Jetson Orin NX modules. It supports Sony, Tamron, and KT&C cameras, uses an ultra-compact form factor for space-constrained enclosures, and is MAXN-Super ready.
Professional broadcast and industrial cameras such as the Sony FCB-EV9520L output video over LVDS, while Jetson platforms natively expect a MIPI CSI-2 camera input. The bridge converts between the two so the camera feed can enter the Jetson capture pipeline directly.
Commonly used plugins include nvstreammux for batching streams, nvinfer for TensorRT inference, nvtracker for persistent object IDs, nvdsosd for bounding box and label overlays, nvvideoconvert for hardware-accelerated format conversion, and nvmsgconv with nvmsgbroker for transmitting inference metadata to external systems.
DeepStream 9.1 adds native support for JetPack 7.2, a unified open-source GitHub repository, enhanced Python development workflows, NVIDIA Service Maker, Multi-View 3D Tracking (MV3DT), AutoMagicCalib for multi-camera calibration, and improved deployment workflows for edge and server environments.
MAXN SUPER is the Jetson uncapped maximum performance mode. Running the module in this mode makes all CPU and GPU resources available to the pipeline, which directly influences sustained inference performance and end-to-end latency.
InfinitEye uses the same Sony FCB-EV9520L and Jetson Orin NX combination but targets long-range detection. It pairs the camera's 30x optical zoom with GPU-accelerated inference to detect objects up to 2 km without additional optics, extendable to 4-5 km with a 2x teleconverter lens, using YOLOv8, TensorRT with DeepStream and Jetson-Inference, and DeepSORT for tracking.
