What Is a Deep Learning Workstation?

A deep learning (DL) workstation is a dedicated computer or server that supports compute-intensive AI and deep learning workloads. It offers significantly higher performance compared to traditional workstations, by leveraging multiple graphical processing units (GPUs).

Compared to just a few years ago, the demand for data science and AI has skyrocketed, driving the development of products that can handle massive amounts of data and complex deep learning workflows. In many data science projects there are security concerns, making it difficult to move data to the cloud. This has driven a growing market for specialized on-premise workstations that can handle compute-intensive AI workloads, within the confines of the local data center.

This is part of our series of articles about deep learning GPU.

In this article:

• Deep Learning Workstation: Cloud or On-premise?
• ‍Top 5 Deep Learning Workstation Options: On-Premises
• NVIDIA DGX Station
• Lambda Labs GPU Workstation
• Lenovo P Series Workstations
• Edge XT Workstation
• Data Science Workstations by 3XS
• Top 3 Deep Learning Workstation Options in the Cloud
• AWS GPU Instances
• Azure GPU VMs
• Google Cloud GPU and TPU
• Scaling Up GPU Workloads with Run:AI

Deep Learning Workstation: Cloud or On-premise?

There are four options for deploying deep learning workloads:

• Traditional cloud providers—these include major vendors like Amazon Web Services (AWS), Microsoft Azure and Google Cloud.
• Deep learning-specific cloud providers—these are cloud offerings specifically tailored to support deep learning workflows, such as focusing on software capabilities and GPU instances. An example is Paperspace.
• Pre-built, on-prem deep learning servers—deep learning workstations are available from companies like NVIDIA (e.g. DGX systems).
• ‍Proprietary DL workstations—these are built from scratch by the organization.

Top 5 Deep Learning Workstation Options: On-Premises

Here are some of the best deep learning workstations available.

Lambda Labs GPU Workstation

This is a mid-range workstation with between two and four GPUs. It is suitable for smaller teams and individual engineers looking to train machine learning models locally.

The Lambda Labs GPU workstation provides up to 4 data center grande NVIDIA GPUs (including A4000, A5000 and A6000), AMD Threadripper or Intel Core i9 CPU, up to 1 TB memory, and up to 61 TB in external storage.

Lenovo P Series Workstations

This is a range of workstations that can deliver high-level deep learning workflows. Lenovo AI workstations allow you to accelerate deep learning workflows, including preparing data, training models and visualizing results. You can execute complete analytics and data science workflows using advanced NVIDIA GPUs. Three options from the series include:

• ThinkStation P340 Tiny—intended for edge-based AI inferencing. Comes with i9-10900K processor, 32 GB system RAM, NVIDIA Quadro P2200 GPU with 5 GB RAM, and 1 TB storage.
• ThinkStation P520—for edge computing and developing AI models. Comes with Xeon W-2295 Processor, up to 256 GB system RAM, choice of GPUs up to NVIDIA Quadro RTX 8000 with 48 GB RAM, storage up to 6 TB
• ThinkStation P920—a high-end deep learning desktop for training models. Comes with Dual Intel Platinum processor, up to 28 cores, up to 1 TB system RAM, choice of GPUs up to NVIDIA Quadro RTX 8000 with 48 GB RAM, storage up to 4 TB onboard and ability to add up to 36 TB in additional drives.

Edge XT Workstation

This line of workstations enables 3D image and animation rendering. It is useful for 3D modelling, video encoding and data visualization with Kibana or Elastic. It comes in two editions:

• Edge XTa - comes with AMD Ryzen Threadripper PRO 3995WX CPU, up to 256 GB system RAM, NVIDIA RTX A4000 GPU with 16 GB RAM, 1 TB storage.
• Edge XTA - as above, with Intel i9-10920X CPU.

Data Science Workstations by 3XS

Data Science Workstations by 3XS Systems are powered by NVIDIA RTX GPU accelerators. Data scientists can use 3XS workstations during early stages, before moving to enterprise-scale training hardware.

NVIDIA-powered Data Science Workstations offer software built on NVIDIA CUDA-X AI, which contains over fifteen data processing and machine learning libraries. These allow computing applications to leverage the NVIDIA GPU-powered computing platform.

3XS provides three main editions of its workstation, each based on a different GPU - Quadro RTX 8000 with 48GB of RAM, Quadro RTX 6000 with 24GB of RAM, and Quadro GV100 with 32GB of RAM and double precision support.

Top 3 Deep Learning Workstation Options in the Cloud

AWS GPU Instances

AWS Deep Learning AMI (DLAMI) provides end-to-end solutions for cloud deep learning. The service offers a customized machine instance, which is made available in the majority of Amazon EC2 regions.

AWS DLAMI includes NVIDIA cuDNN, NVIDIA CUDA, and the latest versions of popular deep learning frameworks. You can use it with several instance types, including small CPU-only instances and high-powered multi-GPU instances:

• Amazon EC2 P3 Instances - up to 8 NVIDIA Tesla V100 GPUs.
• Amazon EC2 G3 Instances - up to 4 NVIDIA Tesla M60 GPUs.
• Amazon EC2 G4 Instances - up to 4 NVIDIA T4 GPUs.
• Amazon EC2 P4 Instances - up to 8 NVIDIA Tesla A100 GPUs.

Azure GPU VMs

Azure offers a GPU-optimized virtual machines (VM) series. Azure GPU VMs are available for use with several sizes, including multiple, single, or fractional GPUs. These sizes are designed especially for graphics and compute-intensive workloads. Here are several options:

NCv3-series and NC T4_v3-series

Both series are optimized especially for compute-intensive GPU-accelerated applications. For example, OpenCL and CUDA-based simulations, deep learning and AI. Here are the main differences between these series:

• The NC T4 v3-series - focused on inference workloads that feature NVIDIA Tesla T4 GPU as well as the AMD EPYC2 Rome processor.
• The NCv3-series - focused on high-performance computing (HPC) and AI workloads that feature NVIDIA Tesla V100 GPU.
  

ND A100 v4-series

The series is focused on scaling deep learning training as well as accelerating HPC applications. It provides eight A100 GPUs, each with 40 GB of memory, connected via 200 Gigabit InfiniBand MDR.

NV-series and NVv3-series

The sizes of this series are specifically designed and optimized to enable and support remote visualization, gaming, streaming, and encoding. It is also ideal for VDI scenarios that employ frameworks like DirectX and OpenGL. NV-series and NVv3-series VMs are powered by the NVIDIA Tesla M60 GPU.

Google Cloud GPU and TPU

Google Cloud offers two main deep learning options - GPUs and TPUs.

Google Cloud GPU

Google Cloud Compute Engine offers GPUs, which you can add to your VM instances. Google Cloud GPUs can help accelerate certain workloads running on your instances, particularly data processing and machine learning.

Google Cloud lets you use NVIDIA GRID technology to create virtual workstations for graphics-intensive workloads, including 3D rendering and 3D visualization, as well as virtual applications. You can use several GPUs, including NVIDIA K80, P4, P100, V100, A100, and T4.

Google Cloud TPU

In addition to GPUs, Google Cloud also lets you use tensor processing units (TPUs). A TPU is an application-specific integrated circuit (ASIC) device designed especially to handle the computational requirements of machine learning applications.

Cloud TPU products offer scalable cloud computing resources for machine learning engineers, developers, researchers, and data scientists who choose to run their machine learning models on Google Cloud. Cloud TPU provides over 100 petaflops of performance, enabling you to scale from a TPU v2 node with 8 cores to a full TPU v3 node with 2048 cores, for example.

Scaling Up GPU Workloads with Run:AI

Run:AI automates resource management and orchestration for machine learning infrastructure, including on GPU workstations and cloud GPU instances. With Run:AI, you can automatically run as many compute intensive experiments as needed.

Our AI Orchestration Platform for GPU-based computers running AI/ML workloads provides:

• Advanced queueing and fair scheduling to allow users to easily and automatically share clusters of GPUs,
• Distributed training on multiple GPU nodes to accelerate model training times,
• Fractional GPUs to seamlessly run multiple workloads on a single GPU of any type,
• Visibility into workloads and resource utilization to improve user productivity.
  

Run:AI simplifies machine learning infrastructure pipelines, helping data scientists accelerate their productivity and the quality of their models.

Learn more about the Run:AI GPU virtualization platform.