Spreading the load: How Salesforce met Multi-AZ HA with SageMaker Inference Components

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

When Salesforce set out to make Agentforce (Salesforce’s AI foundation for agents) highly available (HA) across multiple Availability Zones (AZs), the team faced a gap. Amazon SageMaker AI Inference Components (ICs) could cut GPU costs, but their default placement didn’t guarantee the Multi-AZ resilience Salesforce’s compliance bar required. For Salesforce, the ICs delivered an 8x reduction in infrastructure costs by co-hosting multiple models on shared GPUs. However, this cost win introduced a new question: how do you make IC endpoints highly available across multiple AZs? This post explores how Salesforce used the new IC Placement capability (surfaced through the SchedulingConfig parameter in the CreateInferenceComponent API) to meet their Multi-AZ HA compliance requirements. The challenge: Single points of failure in IC deployments By default, the SageMaker placement algorithm optimizes each IC deployment operation independently, distributing new copies evenly across instances without considering AZ balance. Even with a multi-AZ endpoint, this per-operation view means copies of a specific model can end up unevenly distributed across AZs, creating potential single points of failure: Instance-level failure: A single instance crash takes down all copies of a model. AZ-level failure: An AZ outage makes the entire model unavailable. Compliance risk: Salesforce mandates 2-AZ support for every production model. Default placement for ICs, optimized for cost alone, did not yet meet their internal 2-AZ compliance bar. The solution: SchedulingConfig AWS introduced …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://aws.amazon.com/blogs/machine-learning/spreading-the-load-how-salesforce-met-multi-az-ha-with-sagemaker-inference-components/
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.