Cloud AI services: renting accelerators, managed models and APIs
Cloud AI services allow organizations to rent accelerator hardware, use managed models, and access AI capabilities through APIs. Providers offer purpose-built chips like TPUs and Trainium, as well as prebuilt services for speech, vision, and language. This article explains the service models, typical offerings, and how to choose between them.
Cloud computing and AI services
Cloud computing delivers on-demand access to computing resources, such as servers, storage, and software, over the internet with pay-per-use pricing. This model lets organizations avoid the expense of owning and managing physical hardware, as described in the IBM cloud computing overview.
Cloud services are typically grouped into three models: infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). IaaS offers raw compute and storage, PaaS provides a development environment, and SaaS delivers complete applications, according to the Google Cloud explanation of cloud computing.
For artificial intelligence, these models translate into renting accelerator hardware, using managed training platforms, and consuming AI APIs directly from applications. The table below summarizes these service models for AI workloads.
Renting accelerator hardware
Cloud providers offer virtual machines with specialized accelerators for AI workloads. Azure virtual machine sizes are categorized into families such as general purpose, GPU accelerated, and FPGA accelerated, with naming conventions that indicate features like CPU vendor and included accelerators, as outlined in the Azure VM sizes overview.
Google provides Tensor Processing Units (TPUs) as custom accelerators designed for training and inference. TPUs support popular frameworks like PyTorch and JAX and can be scaled to large clusters, per the Google Cloud TPU documentation.
Amazon Web Services offers Trainium chips purpose-built for AI training and inference, with a Neuron SDK that integrates with PyTorch and other frameworks without requiring code changes, as stated in the AWS Trainium product page.
Managed AI services and APIs
Managed AI services provide prebuilt and customizable APIs for common tasks like speech recognition, translation, image analysis, and content moderation. These services are accessed via REST APIs and client library SDKs, allowing developers to add AI capabilities without managing the underlying infrastructure, according to the Azure AI services overview.
Pricing for such services is typically based on the number of transactions, with each pricing tier specifying a maximum transactions per second and included features. Many services offer a free tier for testing, which helps developers evaluate the service before committing to a paid plan.
Choosing a service and checking specifications
The choice between renting accelerators and using managed services depends on the required level of control and customization. Renting hardware offers full control over the environment but demands more management effort, while managed services reduce that burden but limit flexibility.
When renting accelerators, one must examine VM size specifications for CPU, memory, and accelerator type, as explained in the size naming conventions. For managed services, developers should review the available APIs, pricing tiers, and transaction limits to match their workload. The flow below outlines a typical selection process.
| Service model | What it provides | AI usage example |
|---|---|---|
| IaaS | On-demand compute, storage, and networking | Renting virtual machines with GPUs or TPUs |
| PaaS | Hardware and software for application development | Managed model training and deployment platforms |
| SaaS | Full application stack including maintenance | AI APIs for speech, vision, and language |
Frequently asked questions
What is the difference between IaaS, PaaS, and SaaS for AI?
IaaS provides raw compute and storage, such as accelerator VMs. PaaS offers a development environment for building and deploying AI applications. SaaS delivers ready-to-use AI capabilities as web services.
What factors influence the choice between renting accelerators and using managed AI APIs?
Renting accelerators gives full control over hardware and software but requires more management. Managed AI APIs simplify development and reduce operational overhead but offer less customization. The balance between control and convenience is a primary factor.
What are typical pricing models for cloud AI services?
Accelerator VMs are typically billed per hour of usage. Managed AI services often charge per transaction, with pricing tiers that set a maximum transactions per second. Many services include a free tier for testing.
How can a VM size name indicate an included accelerator?
Azure VM size names include feature letters for accelerators, such as 'g' for GPU or 'f' for FPGA, along with other characters indicating the series and version. The Azure VM sizes overview explains these naming conventions.
What are TPUs and Trainium?
TPUs are Google's custom accelerators for AI training and inference, optimized for frameworks like PyTorch and JAX. Trainium is AWS's purpose-built chip for the same purpose, supported by the Neuron SDK for fast development.