Run GPU batch inference on Amazon ECS Managed Instances with scale to zero

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

GPU batch inference on Amazon Elastic Container Service (Amazon ECS) used to require self-managed Amazon Elastic Compute Cloud (Amazon EC2) instances with Auto Scaling groups, launch templates, Amazon Machine Image (AMI) version tracking, and manual capacity configuration. ECS Managed Instances alleviate that operational burden. You define your compute requirements, and Amazon ECS handles instance provisioning, AMI updates, NVIDIA driver management, and security patching for GPU-capable instances inside your account. In this post, you deploy a single AWS CloudFormation stack that creates a GPU batch inference pipeline on Amazon ECS for asynchronous workloads. The pipeline uses ECS Managed Instances with Amazon Simple Queue Service (Amazon SQS) for job buffering and AWS Application Auto Scaling to scale the service to zero when idle. You submit a text payload, the system provisions a GPU instance, runs inference with a 1.7-billion parameter generative AI model (text-to-speech in this example), and writes natural-sounding speech to Amazon Simple Storage Service (Amazon S3). When the queue is empty, the service scales back to zero tasks and zero instances. You pay only for active inference time. Solution overview

Figure 1: GPU batch inference pipeline on Amazon ECS. An SQS message triggers scale-out through a CloudWatch alarm and Application Auto Scaling. The ECS Managed Instances capacity provider provisions a GPU instance. The worker container polls the queue, calls the model server for inference, and uploads the result to S3. When the queue empties, the service …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://aws.amazon.com/blogs/containers/run-gpu-batch-inference-on-amazon-ecs-managed-instances-with-scale-to-zero/
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.