In this webinar, we’re joined by Eri Rubin the VP of research and development at DeepCube a cnvrg.io customer and NVIDIA Deep Learning Solutions Architect Adam Tetelman to discuss how to optimize distributed training for multi-node and multi-GPU training to maximize performance.      

Distributed deep learning can be complex, with many factors contributing to the overall success of a deployment. Training at-scale requires a well designed data center with proper storage, networking, compute, and software design. In this webinar, we will hear from industry experts in distributed deep learning training and go over the best practices for building dynamic distributed training clusters using containers, PyTorch software tips for distributed training, and strategies for data center design and workload management to maximize NVIDIA GPU utilization. Alongside with the cnvrg.io software platform, these best-practices for deep learning software and hardware will help individual training jobs run faster while getting you a higher data center ROI and boosting cluster utilization.

We’ll follow with a live Megatron-LM example of using PyTorch in cnvrg.io. Along with DeepCube, NVIDIA and cnvrg.io CEO, Yochay Ettun, we will share performance optimization tips for: 

  • PyTorch tips and tricks for optimized distributed training for multi-node and multi GPU training
  • Building dynamic distributed training clusters using NVIDIA NGC containers, Kubernetes, OpenMPI and other open source solutions
  • Designing a topology aware GPU scheduler with emphasis on bandwidth optimization and warm/cold data tiers
  • Using real examples such as Megatron-LM, BERT, GPT-2 and other use-cases https://github.com/NVIDIA/Megatron-LM

In this webinar, we’re joined by Eri Rubin the VP of research and development at DeepCube a cnvrg.io customer and NVIDIA Deep Learning Solutions Architect Adam Tetelman to discuss how to optimize distributed training for multi-node and multi-GPU training to maximize performance.      

 

Distributed deep learning can be complex, with many factors contributing to the overall success of a deployment. Training at-scale requires a well designed data center with proper storage, networking, compute, and software design. In this webinar, we will hear from industry experts in distributed deep learning training and go over the best practices for building dynamic distributed training clusters using containers, PyTorch software tips for distributed training, and strategies for data center design and workload management to maximize NVIDIA GPU utilization. Alongside with the cnvrg.io software platform, these best-practices for deep learning software and hardware will help individual training jobs run faster while getting you a higher data center ROI and boosting cluster utilization.

We’ll follow with a live Megatron-LM example of using PyTorch in cnvrg.io. Along with DeepCube, NVIDIA and cnvrg.io CEO, Yochay Ettun, we will share performance optimization tips for: 

 

  • PyTorch tips and tricks for optimized distributed training for multi-node and multi GPU training
  • Building dynamic distributed training clusters using NVIDIA NGC containers, Kubernetes, OpenMPI and other open source solutions
  • Designing a topology aware GPU scheduler with emphasis on bandwidth optimization and warm/cold data tiers
  • Using real examples such as Megatron-LM, BERT, GPT-2 and other use-cases

    https://github.com/NVIDIA/Megatron-LM
cnvrg- Data Science Platform
cnvrg.io - ML Platform

AI:OS for Machine Learning

Get Started
Schedule Demo
small_c_popup.png

Join the innovative enterprises using cnvrg.io to build, train & deploy their models faster.

schedule a demo today

CloseMenu

Announcing CORE, a free ML Platform for the community to help data scientists focus more on data science and less on technical complexity

Download cnrvg CORE for Free

By submitting this form, I agree to cnvrg.io’s
privacy policy and terms of service.

With cnvrg.io we were able to increase our model throughput by up to 50% and on average by 30% when comparing to RESTful APIs. cnvrg.io also allows us to monitor our models in production, set alerts and retrain with high-level automation ML pipelines
Avi Gabay
Avi GabayDirector of Architecture at Playtika
As with many data science professionals, our team is hard to please. They want the flexibility to use any language, and the ability to write their own custom packages to improve performance, set configurations, etc. With cnvrg.io, this is the first time I’ve heard our data scientists and analysts say ‘when can we have it’.
Alexander Ryabov
Alexander RyabovHead of Data Services & Business Intelligence at Wargaming.net
Working in a hybrid cloud environment has major advantages but can be increasingly complex to manage, especially for AI workloads. cnvrg.io has the potential to enable us to operate in a hybrid cloud environment seamlessly. The new ML infrastructure dashboard could fill a major need in connecting our infrastructure to ML projects. It provides visibility into our on-prem GPU clusters and cloud resources, paving the way for increasing the utilization and ROI of our GPUs.
Bruce King
Bruce KingData Science Technologist at Seagate Technology Advanced Analytics Group
Lightricks is a research-driven company, which is why we’ve built a team with some of the top computer vision researchers in the country. cnvrg.io ensures our highly qualified researchers are focused on building the industry-leading AI technology that we are now world renown for, instead of spending time on engineering, configuration and DevOps.
Ofir Bibi
Ofir BibiHead of Research at Lightricks
Previous
Next