Demo: Optimizing Gemma inference on NVIDIA GPUs with TensorRT-LLM
Рет қаралды 2,135
Күн бұрынEven the smallest of Large Language Models are compute intensive significantly affecting the cost of your Generative AI application. Your ability to increase the throughput and reduce latency can make or break many business cases. NVIDIA TensorRT-LLM is an open-source tool that allows you to considerably speed up execution of your models and in this talk we will demonstrate its application to Gemma.
Watch more videos of Gemma Developer Day 2024 → goo.gle/440EAIV
Subscribe to Google for Developers → goo.gle/developers
#Gemma #GemmaDeveloperDay
@LuigiGoogle 2 ай бұрын
hello google, I am inspired by your company, could you create open source projects to work with people who are also interested in this, namely this topic of neural networks.
@ShayansCodeCommunity 2 ай бұрын
Hi, I am Shayan, beginner Python developer, I want to learn something from code jams or online coding challenges, can Google team will do that work for me please 🥺, I really want to learn something from Google code jam.
MLOps.community
Рет қаралды 10 М.
TheSoul Music Jam
Рет қаралды 10 МЛН
Lightning Talk: Accelerated Inference in PyTorch 2.X with Torch...- George Stefanakis & Dheeraj Peri
PyTorch
Рет қаралды 1,3 М.
freeCodeCamp.org
Рет қаралды 200 М.
CopyPasta
Рет қаралды 57 М.