TensorRT 8, the eighth version of NVIDIA AI software, was released today, halving the response time for language queries, allowing developers to create the world’s best search engines, ad recommendations, and chatbots, and lead to the cloud to the perimeter…
Improvements in TensorRT 8 deliver record speeds for language applications, with BERT-Large, one of the most widely used transformer-based models in the world, in 1.2 milliseconds. Previously, companies needed to reduce the size of their models, which resulted in significantly less accurate results. Companies can now double or triple the size of their model with TensorRT 8 for significant gains in imprecision.
TensorRT has been downloaded more than 2.5 million times in the last five years by more than 350,000 developers from 27,500 companies across all industries, including healthcare, automotive, finance, and retail. TensorRT applications can be used in large-scale data centers, embedded product platforms, and automotive product platforms.
Sparsity is a new performance approach on NVIDIA Ampere architecture GPUs that improves efficiency and allows developers to accelerate neural networks while minimizing processing.
With quantization recognition training, developers can use trained models to derive INT8 precision without sacrificing precision. This significantly reduces processing and storage costs for efficient Tensor Core derivation.
Comprehensive operational support
Industry leaders have used TensorRT for deep learning information applications in conversational AI and many other fields.
Hugging Face is a leader in open-source AI, on which the world’s largest AI service providers across multiple industries depend. The company is partnering with NVIDIA to provide innovative artificial intelligence services that enable large-scale text analysis, neural search, and conversational applications.
TensorRT is being used by GE Healthcare
A global leader in medical technology, diagnostics, and digital solutions, to accelerate machine vision ultrasound applications, a critical tool for disease detection. Its innovative healthcare solutions enable physicians to provide the highest quality care.
TensorRT 8 is now widely available and free to members of the NVIDIA developer program. In addition, the latest versions of plug-ins, analysts, and samples are open source and accessible through TensorRT’s GitHub repository.
Invented by NVIDIA in 1999, the GPU fueled the growth of the computer gaming industry and redefined modern computer graphics, high-performance computing, and artificial intelligence. The company’s pioneering work in accelerated computing and artificial intelligence is reshaping trillion-dollar industries, including transportation, healthcare, and manufacturing, and fueling the growth of many others.