人工智能
使用 AWS CDK 的 Orchestration SageMaker AI 端点自动缩放
更新 Amazon Sagemaker AI 实时端点配置(改变实例类型，实例计数，或环境变量)导致通过 AWS CloudFormation 或 AWS Cloud Development Kit(AWS CDK)的部署失败。 SageMaker AI要求您在更新端点前对可缩放目标进行解登记，但当您解登记时，CloudFormation 无法跟踪 AWS 应用程序自动缩放资源。
在此文章中，我们演示您如何使用 AWS CloudFormation 自定义资源的可重复使用的 CDK 构建，以在结束点更新时协调解注册和自动注册自动缩放。 您可以在一次部署中更新端点配置和自动缩放参数，而无需人工干预。
为什么会有这样的问题
当使用本地的CloudFormation自动缩放资源部署一个 SageMaker AI 端点时，然后改变端点配置并进行重新部署，会出现几个失败模式。 具体的失败取决于你改变什么，你是如何围绕这个问题工作的。
用已注册的自动缩放更改终点配置
CloudFormation试图在可扩展目标仍然注册的同时更新终点。 SageMaker AI 以以下错误拒绝更新：
SageMaker AI在CloudFormation到达自动缩放资源前屏蔽端点更新。
手动解登记自动缩放，然后只更改端点配置
如果通过AWS命令线接口(AWS CLI)对可伸缩目标进行解登记，并且只进行端点配置更改后进行重新部署，部署成功。 CloudFormation虽然在CloudFormation模板中，CloudFormation模板中的自动缩放参数没有改变，端点现在运行时没有自动缩放，没有错误显示出任何出错。
手动解登记自动缩放，然后更改自动缩放参数
如果您通过 AWS CLI 解析可缩放目标，并使用自动缩放参数变化进行调用，CloudFormation 试图更新已不存在的可缩放目标资源：
这种失败是值得注意的，因为回滚也失败了(UPDATE ROLLBACK FAILED)，使堆栈处于一个断裂状态，需要人工干预云阵控制台才能恢复。
解决方案概述
您可以使用此 CDK 构造来替换本地的 AWS:: application AutoScaled:：可扩展目标和 AWS:: application AutoScaled::: scale Political CloudFormation 资源，使用三种自定义资源。 这些自定义资源解除了注册，并在端点更新周围注册自动缩放，并明确依赖命令，由节点执行。 CDK中添加依赖性调用。 完整序列为：
- Deregister(海关资源)：在结束点更新开始前删除可扩展目标。 在第一次部署时，还没有可缩放的目标存在，所以处理者优雅地处理错误并持续。
- 端点更新(CDK管理):CDK正常更新端点资源，交换到新的端点配置。
- 注册可扩展目标(海关资源)：等待终点达到InService状态，然后注册可扩展目标。
- 附加缩放政策(海关资源)：等待可缩放目标可查询，然后应用目标跟踪政策。
注册可缩放目标并附加缩放策略使用 CDK 提供者的同步补全模式。 每个自定义资源都有一个即时返回的活动处理器，一个是完成处理器。 提供商的服务员在间隔时间(每30秒)内重新接通，提供商总时间(登记可缩放目标1小时，附加缩放政策15分钟)限制等待时间
每个自定义资源上的 ConfigHash 属性迫使 CloudFormation 当配置值发生变化时，重新启用 Lambda 函数。 端点配置和自动缩放参数一起定义，因为一个单一的ConfigHash驱动自动缩放管弦。
虽然构造定义了3个自定义资源，但部署的堆栈包含超过3个Lambda函数。 每个自定义资源都由CDK cr. Provider支持，提供方框架会生成自己的助手Lambda功能。 一个同步提供商(Deregister)新增 1框架函数。 一个同步提供商(注册可缩放目标并附加缩放政策) 新增 3 (on Event, is complete, and on Timeout) 加上驱动服务员的AWS Step函数状态机。 因此，部署的堆栈包含3个自定义资源，2个Step函数态机，12个Lambda函数。 其中5项资源持有构造本身的处理器代码(一个用于Deregister，一个用于活动，一个用于每个同步资源的完整对)，其余7项由提供者框架产生。 您不写或维护框架生成的功能，但是它们存在于堆栈中，值得在您审查部署的资源时考虑。
走过去
在这个走行部中，您会使用提供的示例堆栈使用自动缩放的 SageMaker AI 端点，然后校验配置更新是否正确。
行走包括以下步骤：
- 清除存储器并安装依赖。
- 配置端点和自动缩放设置。
- 部署和核实。
-测试最新消息
源代码可以在链接的GitHub寄存器中找到。
先决条件
要走过这段路，你需要：
- AWS账户，有Amazon SageMaker AI、AWS Lambda、AWS身份和访问管理、应用程序自动缩放和Amazon云表的许可。
- AWS CDK v2安装。
- Python 3.12或后来。
-Python依赖管理
- Amazon简易存储服务(Amazon S3)中的SageMaker AI模型文物，以及Amazon弹性集装箱登记簿(Amazon ECR)中的集装箱图像，或SageMaker AI内置算法容器。
克隆寄存器并安装依赖性
- 清除仓库：
- 安装依赖：
配置您的端点和自动缩放设置
端点和自动缩放设置在配置/配置中一起定义。 yaml 因为一个单一的配置 hash驱动自动缩放的管弦乐。 对任一部分的修改会触发完整的序列。
以您的值编辑配置/ 配置 yaml:
端点参数映射到SageMaker AI CreateModel和CreateEndpointConfig APIs. 自动缩放参数映射到应用程序自动缩放可缩放目标及PutS缩放政策 API.
部署和核查
- 如果还没有
- 部署堆：
CloudFormation创建了SageMaker AI模型，端点配置，端点，然后按顺序运行三个自定义资源来注册自动缩放。
- 核实部署是否登记了可扩展目标：
- 核实部署是否采用了缩放政策：
测试更新
更改配置/ 配置中的值。 例如，Yaml 将实例类型从 ml.m5. 大小更新到 ml.m5.x 大，并重新部署：
部署完成后，验证自动缩放是否仍在登记，并有更新的值：
与现有堆栈合并
如果您正在开始新的或想要引用执行，请使用示例堆栈。 如果您已经拥有 SageMaker AI 资源的 CDK 堆栈，请直接使用此构造。
要与已有的堆栈集成，请将 sagemaker autoscaling/ 构建/ 目录复制到您的工程中，并在您的端点资源旁添加构造：
施工方式
本节通过Sagemaker Autoscaling Construct的关键实施细节。
运动家
Deregister 自定义资源在端点更新前调用 deregister scale target. 在没有可扩展目标的情况下，它处理首次部署的ObjectNotFound排除：
依赖在CDK中建立：
可缩放目标
尽管CloudFormation在接受更新后将端点资源视为完整，但可扩展目标只能在端点到达InService后注册。 登记可缩放目标使用提供者的同步完成模式，而不是阻止Lambda在等待中援引。 事件处理器立即返回，提供者的服务员重新输入查询- 间距上的“ 完整处理器”，直至终点为 InService。 提供商总时间(1小时)限制等待
堆栈删除时，活动处理器将可扩展目标解录以清理。
附加缩放策略
在RegisterS可缩放目标返回后有一个简短的窗口，目标尚无法查询。 附加缩放策略使用相同的同步补全模式：在事件处理器上立即返回，而在可缩放目标可查询时才完成的为完成，此时它应用了策略。 提供商的总时间( 15分钟) 限制等待：
堆栈删除时，活动处理器会删除清理的缩放策略。
清理
为了避免今后发生费用，删除资源：
这删除了 SageMaker AI 端点，端点配置，模型，Lambda 函数，IAM 角色，由目标跟踪缩放策略创建的 Amazon Cloud Watch 提醒，并解除可缩放目标的记录。
结论
在此文章中，我们向您演示了如何使用可重复使用的 CDK 构造来解登记并自动注册自动缩放，以协调 SageMaker AI 端点更新。 该建筑使用3个CloudFormation自定义资源，并有明确的依赖性订单和一个ConfigHash属性，使自动缩放在整个部署中保持完好无损。
要开始，克隆样本- sagemaker-endpoint-autoscaling寄存器并遵循设置指令。 该寄存器包括一个可以直接部署的示例堆栈，或者可以将构造集成到现有的CDK堆栈中。
关于SageMaker AI应用自动缩放的更多信息，请参见自动缩放 Amazon SageMaker AI模型。 关于CloudFormation自定义资源的更多信息，请参见AWS CloudFormation用户指南中的Custom资源。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Artificial Intelligence
Orchestrating SageMaker AI endpoint auto scaling with AWS CDK
Updating the Amazon Sagemaker AI real-time endpoint configuration (changing the instance type, instance count, or environment variables) causes deployments through AWS CloudFormation or the AWS Cloud Development Kit (AWS CDK) to fail. SageMaker AI requires you to deregister the scalable target before updating the endpoint, but when you deregister it, CloudFormation can’t track the AWS Application Auto Scaling resources.
In this post, we show you how to use a reusable CDK construct with AWS CloudFormation custom resources to orchestrate deregister and register auto scaling automatically during endpoint updates. You can update your endpoint configuration and auto scaling parameters in a single deployment without manual intervention.
Why this problem occurs
When you deploy a SageMaker AI endpoint with native CloudFormation auto scaling resources, then change the endpoint configuration and redeploy, several failure modes emerge. The specific failure depends on what you change and how you work around the issue.
Changing the endpoint configuration with auto scaling registered
CloudFormation tries to update the endpoint while the scalable target is still registered. SageMaker AI rejects the update with the following error:
SageMaker AI blocks the endpoint update before CloudFormation reaches the auto scaling resources.
Manually deregistering auto scaling, then changing only the endpoint configuration
If you deregister the scalable target through the AWS Command Line Interface (AWS CLI) and redeploy with only an endpoint configuration change, the deployment succeeds. However, CloudFormation doesn’t register auto scaling even though it is in the CloudFormation template. Because the auto scaling parameters in the CloudFormation template haven’t changed, CloudFormation doesn’t register the scalable target or re-apply the scaling policy. The endpoint is now running without auto scaling, and there is no error to indicate anything went wrong.
Manually deregistering auto scaling, then changing auto scaling parameters
If you deregister the scalable target through the AWS CLI and redeploy with auto scaling parameter changes, CloudFormation tries to update the ScalableTarget resource that no longer exists:
This failure is notable because the rollback also fails (UPDATE_ROLLBACK_FAILED), leaving the stack in a broken state that requires manual intervention in the CloudFormation console to recover.
Solution overview
You can use this CDK construct to replace the native AWS::ApplicationAutoScaling::ScalableTarget and AWS::ApplicationAutoScaling::ScalingPolicy CloudFormation resources with three custom resources. These custom resources deregister and register auto scaling around the endpoint update with explicit dependency ordering, enforced by node.add_dependency calls in CDK. The full sequence is:
- Deregister (custom resource): Removes the scalable target before the endpoint update begins. On first deploy, no scalable target exists yet, so the handler handles the error gracefully and continues.
- Endpoint update (CDK-managed): CDK updates the endpoint resource normally, swapping to the new endpoint configuration.
- Register Scalable Target (custom resource): Waits for the endpoint to reach InService status, then registers the scalable target.
- Attach Scaling Policy (custom resource): Waits for the scalable target to be queryable, then applies the target tracking policy.
Register Scalable Target and Attach Scaling Policy use the CDK provider’s asynchronous completion pattern. Each custom resource has an on_event handler that returns immediately and an is_complete handler. The provider’s waiter re-invokes the is_complete handler on an interval (every 30 seconds) until its readiness check passes. The provider’s total_timeout (1 hour for Register Scalable Target, 15 minutes for Attach Scaling Policy) bounds the wait, so it isn’t limited by the timeout of any single Lambda invocation.
A ConfigHash property on each custom resource forces CloudFormation to re-invoke the Lambda functions whenever a configuration value changes. The endpoint configuration and auto scaling parameters are defined together because a single ConfigHash drives the auto scaling orchestration.
While the construct defines three custom resources, the deployed stack contains more than three Lambda functions. Each custom resource is backed by a CDK cr.Provider, and the provider framework generates its own helper Lambda functions. A synchronous provider (Deregister) adds one framework function. An asynchronous provider (Register Scalable Target and Attach Scaling Policy) adds three (onEvent, isComplete, and onTimeout) plus an AWS Step Functions state machine that drives the waiter. As a result, the deployed stack contains three custom resources, two Step Functions state machines, and twelve Lambda functions. Five of the resources hold the construct’s own handler code (one for Deregister, and an on_event and is_complete pair for each asynchronous resource), with the remaining seven generated by the provider framework. You don’t write or maintain the framework-generated functions, but they are present in the stack and worth accounting for when you review the deployed resources.
Walkthrough
In this walkthrough, you deploy a SageMaker AI endpoint with auto scaling using the provided example stack, then verify that configuration updates work correctly.
The walkthrough includes the following steps:
- Clone the repository and install dependencies.
- Configure your endpoint and auto scaling settings.
- Deploy and verify.
- Test an update.
The source code is available in the linked GitHub repository.
Prerequisites
For this walkthrough, you need the following:
- An AWS account with permissions for Amazon SageMaker AI, AWS Lambda, AWS Identity and Access Management (IAM), Application Auto Scaling, and Amazon CloudWatch.
- AWS CDK v2 installed.
- Python 3.12 or later.
- uv for Python dependency management.
- A SageMaker AI model artifact in Amazon Simple Storage Service (Amazon S3) and a container image in Amazon Elastic Container Registry (Amazon ECR), or a SageMaker AI built-in algorithm container.
Clone the repository and install dependencies
- Clone the repository:
- Install dependencies:
Configure your endpoint and auto scaling settings
The endpoint and auto scaling settings are defined together in config/config.yaml because a single config_hash drives the auto scaling orchestration. A change to either section triggers the full sequence.
Edit config/config.yaml with your values:
The endpoint parameters map to the SageMaker AI CreateModel and CreateEndpointConfig APIs. The auto scaling parameters map to the Application Auto Scaling RegisterScalableTarget and PutScalingPolicy APIs.
Deploy and verify
- Bootstrap your AWS environment if you haven’t already:
- Deploy the stack:
CloudFormation creates the SageMaker AI model, endpoint configuration, and endpoint, then runs the three custom resources in sequence to register auto scaling.
- Verify that the deployment registered the scalable target:
- Verify that the deployment applied the scaling policy:
Test an update
Change a value in config/config.yaml, for example, update the instance type from ml.m5.large to ml.m5.xlarge, and redeploy:
After the deployment completes, verify that auto scaling is still registered and has the updated values:
Integrating with an existing stack
Use the example stack if you’re starting fresh or want a reference implementation. Use the construct directly if you already have a CDK stack with SageMaker AI resources.
To integrate with an existing stack, copy the sagemaker_autoscaling/constructs/ directory into your project and add the construct alongside your endpoint resources:
How the construct works
This section walks through the key implementation details of SagemakerAutoscalingConstruct.
Deregister
The Deregister custom resource calls deregister_scalable_target before the endpoint update. It handles ObjectNotFoundException for first deployments where no scalable target exists:
The dependency is established in CDK:
Register Scalable Target
Even though CloudFormation treats the endpoint resource as complete once the update is accepted, the scalable target can only be registered after the endpoint reaches InService. Rather than block a single Lambda invocation on that wait, Register Scalable Target uses the provider’s asynchronous completion pattern. The on_event handler returns immediately, and the provider’s waiter re-invokes the is_complete handler on query_interval until the endpoint is InService. The provider’s total_timeout (1 hour) bounds the wait, not the timeout of any single Lambda invocation:
On stack deletion, the on_event handler deregisters the scalable target to clean up.
Attach Scaling Policy
There is a brief window after RegisterScalableTarget returns where the target isn’t yet queryable. Attach Scaling Policy uses the same asynchronous completion pattern: the on_event handler returns immediately, and the is_complete handler reports completion only once the scalable target is queryable, at which point it applies the policy. The provider’s total_timeout (15 minutes) bounds the wait:
On stack deletion, the on_event handler deletes the scaling policy to clean up.
Cleaning up
To avoid incurring future charges, delete the resources:
This removes the SageMaker AI endpoint, endpoint configuration, model, Lambda functions, IAM roles, Amazon CloudWatch alarms created by the target tracking scaling policy, and deregisters the scalable target.
Conclusion
In this post, we showed you how to use a reusable CDK construct to deregister and register auto scaling automatically to orchestrate SageMaker AI endpoint updates. The construct uses three CloudFormation custom resources with explicit dependency ordering and a ConfigHash property to keep auto scaling intact across deployments.
To get started, clone the sample-sagemaker-endpoint-autoscaling repository and follow the setup instructions. The repository includes an example stack you can deploy directly, or you can integrate the construct into your existing CDK stack.
For more information about Application Auto Scaling for SageMaker AI, see Automatically scale Amazon SageMaker AI models. For more information about CloudFormation custom resources, see Custom resources in the AWS CloudFormation User Guide.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.