Aws SDK v2#665
Conversation
|
Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement Learn more about why HashiCorp requires a CLA and what the CLA includes Karel Pazdera seems not to be a GitHub user. Have you signed the CLA already but the status is still pending? Recheck it. |
There was a problem hiding this comment.
Pull request overview
This pull request migrates the Amazon Packer plugin from AWS SDK for Go v1 to v2, updating EC2/IAM/SSM interactions, waiters, and related configuration/documentation to match the v2 API surface.
Changes:
- Reworked AWS client/config creation and many builder steps to use AWS SDK v2 clients, types, and waiter patterns.
- Updated state-bag conventions (e.g.,
ec2key, instance storage) and tests to reflect v2 usage. - Regenerated docs/HCL2 specs and adjusted CI tooling versions.
Reviewed changes
Copilot reviewed 115 out of 118 changed files in this pull request and generated 21 comments.
Show a summary per file
| File | Description |
|---|---|
| go.sum | Dependency updates for SDK v2 migration |
| go.mod | Module dependency updates for SDK v2 migration |
| docs-partials/builder/instance/Config-not-required.mdx | Regenerated config docs (type changes) |
| docs-partials/builder/common/volumeAttachedDetachedWaiterOptions.mdx | New generated docs for custom waiters |
| docs-partials/builder/common/RunConfig-required.mdx | Regenerated RunConfig docs (type changes) |
| docs-partials/builder/common/RunConfig-not-required.mdx | Regenerated RunConfig docs (type changes) |
| docs-partials/builder/common/Placement-not-required.mdx | Regenerated Placement docs (type changes) |
| docs-partials/builder/common/BlockDevice-not-required.mdx | Regenerated block device docs (int32 updates) |
| docs-partials/builder/common/AWSPollingConfig-not-required.mdx | Added max_timeout docs |
| docs-partials/builder/common/AMIConfig-not-required.mdx | Regenerated AMI config docs (type changes) |
| docs-partials/builder/chroot/Config-not-required.mdx | Regenerated chroot config docs (int32 updates) |
| common/step_stop_ebs_instance.go | State key updates; EC2 client usage changes |
| common/step_source_ami_info.go | ec2 types aliasing / state key updates |
| common/step_security_group.go | State key updates for EC2 client |
| common/step_run_spot_instance.go | Stores instance as pointer; state key updates |
| common/step_run_spot_instance_test.go | Updated mocks/signatures for SDK v2 |
| common/step_run_source_instance.go | Stores instance as pointer; state key updates |
| common/step_pre_validate.go | Uses ctx-aware region validation |
| common/step_network_info.go | State key updates for EC2 client |
| common/step_modify_ebs_instance.go | SDK v2 EC2 ModifyInstanceAttribute usage |
| common/step_key_pair.go | State key updates for EC2 client |
| common/step_iam_instance_profile.go | State key updates for EC2 client |
| common/step_get_password.go | SDK v2 EC2 GetPasswordData usage |
| common/step_create_ssm_tunnel.go | SDK v2 SSM/EC2IC wiring changes |
| common/step_cleanup_volumes.go | SDK v2 volume describe/delete adjustments |
| common/step_ami_region_copy.go | SDK v2 copy + waiter usage changes |
| common/step_ami_region_copy_test.go | Updated mocks for SDK v2 |
| common/ssh.go | Instance stored as pointer; ctx-aware describer |
| common/ssh_test.go | Updated tests for pointer instance + ctx |
| common/clients/ec2_client.go | Expanded EC2 client interface for SDK v2 calls |
| builder/instance/step_upload_bundle.go | Uses AWS SDK v2 config/credentials retrieval |
| builder/instance/step_register_ami.go | RegisterImage migrated to SDK v2 client/types |
| builder/instance/step_bundle_volume.go | Instance types migrated to SDK v2 |
| builder/instance/builder.hcl2spec.go | HCL2 spec types moved to SDK v2 enums |
| builder/instance/builder.go | SDK v2 config + EC2/IAM client initialization |
| builder/ebsvolume/step_tag_ebs_volumes.go | Updated EC2 client state key |
| builder/ebsvolume/step_snapshot_ebs_volumes.go | Instance now expected as pointer |
| builder/ebsvolume/step_snapshot_ebs_volumes_test.go | Updated tests for pointer instance + EC2 key |
| builder/ebsvolume/builder.go | State key updated from ec2v2 to ec2 |
| builder/ebssurrogate/step_swap_volumes.go | Updated EC2 client state key |
| builder/ebssurrogate/step_snapshot_volumes.go | Updated EC2 client state key |
| builder/ebssurrogate/step_register_ami.go | Updated EC2 client state key |
| builder/ebssurrogate/step_create_ami.go | Updated EC2 client state key |
| builder/ebssurrogate/builder.go | State key updated from ec2v2 to ec2 |
| builder/ebs/step_enable_fast_launch.go | Comment update for v2 client naming |
| builder/ebs/step_create_ami.go | Updated EC2 client state key |
| builder/ebs/builder.go | State key updated from ec2v2 to ec2 |
| builder/common/test_helper_funcs.go | FakeAccessConfig converted to SDK v2 config |
| builder/common/tags.go | Tag types migrated to SDK v2 types.Tag |
| builder/common/step_stop_ebs_instance.go | StopInstances + waiter migrated to SDK v2 |
| builder/common/step_source_ami_info.go | Source AMI describe + types migrated to v2 |
| builder/common/step_source_ami_info_test.go | Updated tests for v2 types |
| builder/common/step_security_group.go | SG operations + waiters migrated to v2 |
| builder/common/step_run_spot_instance.go | Spot flow updated for v2 client signatures |
| builder/common/step_run_spot_instance_test.go | Updated mocks/signatures for v2 |
| builder/common/step_run_source_instance.go | Source instance flow migrated to v2 types |
| builder/common/step_pre_validate.go | Region/VPC validations migrated to v2 |
| builder/common/step_pre_validate_test.go | Updated test mocks for v2 signatures/types |
| builder/common/step_network_info.go | VPC/subnet selection migrated to v2 |
| builder/common/step_network_info_test.go | Updated tests for v2 client signatures |
| builder/common/step_modify_ebs_instance.go | ModifyInstanceAttribute migrated to v2 |
| builder/common/step_modify_ami_attributes.go | ModifyImage/SnapshotAttribute migrated to v2 |
| builder/common/step_key_pair.go | CreateKeyPair migrated to v2 types |
| builder/common/step_key_pair_test.go | Updated mocks for v2 signatures/types |
| builder/common/step_iam_instance_profile.go | IAM instance profile creation migrated to v2 |
| builder/common/step_get_password.go | GetPasswordData migrated to v2 |
| builder/common/step_enable_deregistration_protection.go | Uses v2 client getter + ctx |
| builder/common/step_enable_deprecation.go | Uses v2 client getter + ctx |
| builder/common/step_deregister_ami.go | Deregister + snapshot delete migrated to v2 |
| builder/common/step_create_tags.go | Tagging AMI/snapshots migrated to v2 |
| builder/common/step_create_ssm_tunnel.go | SSM/EC2IC migrated to v2 |
| builder/common/step_cleanup_volumes.go | Cleanup volumes migrated to v2 |
| builder/common/step_ami_region_copy.go | Region copy migrated to v2 client |
| builder/common/step_ami_region_copy_test.go | Updated region copy tests for v2 |
| builder/common/state.hcl2spec.go | Added max_timeout to HCL2 spec |
| builder/common/state.go | Added MaxTimeout and v2 waiter integrations |
| builder/common/ssm/session.go | SSM session logic migrated to v2 client |
| builder/common/ssh.go | SSHHost migrated to v2 types + ctx |
| builder/common/ssh_test.go | Updated tests for ctx-aware SSHHost |
| builder/common/snapshot_config.go | Minor formatting-only change |
| builder/common/run_config.hcl2spec.go | Placement tenancy typed as v2 enum |
| builder/common/run_config.go | RunConfig fields typed with v2 enums |
| builder/common/run_config_test.go | Updated tests for v2 enum types |
| builder/common/regions.go | Region listing/validation now ctx-aware |
| builder/common/interpolate_build_info.go | Build-info extraction migrated to v2 types |
| builder/common/interpolate_build_info_test.go | Updated tests for v2 types |
| builder/common/helper_funcs.go | DestroyAMIs migrated to v2 + ctx |
| builder/common/build_filter.go | Filters migrated to v2 types.Filter |
| builder/common/build_filter_test.go | Updated tests for v2 filter slice types |
| builder/common/boot_mode_validation.go | Boot mode validation uses v2 enums |
| builder/common/boot_mode_validation_test.go | Minor formatting-only change |
| builder/common/block_device.hcl2spec.go | IOPS/throughput/size changed to int32 |
| builder/common/block_device.go | Block device mappings migrated to v2 types |
| builder/common/block_device_test.go | Updated tests for v2 types/int32 |
| builder/common/artifact.go | Artifact destroy path migrated to v2 |
| builder/common/ami_filter.go | AMI filtering migrated to v2 + ctx |
| builder/common/ami_config.go | AMI config typed with v2 enums |
| builder/common/ami_config_test.go | Updated tests for v2 DescribeRegions signature |
| builder/common/access_config.go | Access config migrated to aws-sdk-go-base/v2 |
| builder/common/access_config_test.go | Updated tests for new config handling |
| builder/chroot/step_snapshot.go | Snapshot creation/deletion migrated to v2 |
| builder/chroot/step_register_ami.go | RegisterImage migrated to v2 types |
| builder/chroot/step_register_ami_test.go | Updated tests for v2 types |
| builder/chroot/step_mount_device.go | VirtualizationType uses v2 types |
| builder/chroot/step_instance_info.go | IMDS migrated to v2 |
| builder/chroot/step_create_volume.go | CreateVolume migrated to v2 types |
| builder/chroot/step_create_volume_test.go | Updated tests for v2 types/int32 |
| builder/chroot/step_check_root_device.go | Root device checks updated to v2 enums |
| builder/chroot/step_attach_volume.go | Attach/Detach volume migrated to v2 |
| builder/chroot/builder.hcl2spec.go | HCL2 spec types moved to v2 enums/int32 |
| builder/chroot/builder.go | SDK v2 config + EC2 client initialization |
| .web-docs/components/builder/instance/README.md | Regenerated docs with new types/options |
| .web-docs/components/builder/ebsvolume/README.md | Regenerated docs with new types/options |
| .web-docs/components/builder/ebssurrogate/README.md | Regenerated docs with new types/options |
| .web-docs/components/builder/ebs/README.md | Regenerated docs with new types/options |
| .web-docs/components/builder/chroot/README.md | Regenerated docs with new types/options |
| .go-version | Go version bump |
| .github/workflows/go-validate.yml | CI lint tooling/version changes |
Files not reviewed (2)
- builder/chroot/builder.hcl2spec.go: Language not supported
- builder/common/block_device.hcl2spec.go: Language not supported
Comments suppressed due to low confidence (2)
builder/ebssurrogate/step_snapshot_volumes.go:44
instance := state.Get("instance").(ec2types.Instance)assumes the instance is stored by value, but other updated steps store*ec2types.Instancein state. If the pointer form is used, this assertion will panic. Standardize on a single convention (prefer pointer) across all steps in the workflow.
common/step_cleanup_volumes.go:37Cleanupcastsstate.Get("instance")toec2types.Instance(value), but other updated steps now store*ec2types.Instancein the state bag. This mismatch will panic at runtime during cleanup. Standardize on a singleinstancestate type (prefer*ec2types.Instance) and update this assertion/usage accordingly.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| func (c *AccessConfig) SessionRegion() string { | ||
| if c.session == nil { | ||
| panic("access config session should be set.") | ||
| if c.awsConfig == nil { | ||
| panic("access config should be set.") | ||
| } | ||
| return aws.StringValue(c.session.Config.Region) | ||
| return c.awsConfig.Region | ||
| } |
| awscfg, err := config.AccessConfig.GetAWSConfig(ctx) | ||
| region := awscfg.Region | ||
| var token string | ||
| if err == nil && accessKey == "" && secretKey == "" { | ||
| credentials, err := accessConfig.Credentials.Get() | ||
| credentials, err := awscfg.Credentials.Retrieve(ctx) | ||
| if err == nil { |
| @@ -53,10 +56,10 @@ func (s *StepStopEBSBackedInstance) Run(ctx context.Context, state multistep.Sta | |||
| }, | |||
| w := ec2.NewInstanceStoppedWaiter(ec2conn) | ||
| err = w.Wait(ctx, &ec2.DescribeInstancesInput{ | ||
| InstanceIds: []string{aws.ToString(instance.InstanceId)}, | ||
| }, time.Duration(s.PollingConfig.MaxTimeout)) | ||
| if err != nil { |
| if !s.IsRestricted { | ||
| region := state.Get("region").(*string) | ||
| ec2Tags, err := TagMap(s.Tags).EC2Tags(s.Ctx, aws.StringValue(region), state) | ||
| ec2Tags, err := TagMap(s.Tags).EC2Tags(s.Ctx, aws.ToString(region), state) | ||
| if err != nil { |
| ec2Client := state.Get("ec2").(clients.Ec2Client) | ||
| awsConfig := state.Get("aws_config").(*aws.Config) | ||
| instance := state.Get("instance").(ec2types.Instance) | ||
| ui := state.Get("ui").(packersdk.Ui) |
| profileResp, err := iamsvc.CreateInstanceProfile(ctx, &iam.CreateInstanceProfileInput{ | ||
| InstanceProfileName: aws.String(profileName), | ||
| Tags: iamProfileTags, | ||
| }) |
| func (s *StepStopEBSBackedInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction { | ||
| ec2Client := state.Get("ec2v2").(clients.Ec2Client) | ||
| ec2Client := state.Get("ec2").(clients.Ec2Client) | ||
| instance := state.Get("instance").(ec2types.Instance) | ||
| ui := state.Get("ui").(packersdk.Ui) |
| func (s *StepModifyEBSBackedInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction { | ||
| ec2Client := state.Get("ec2v2").(clients.Ec2Client) | ||
| ec2Client := state.Get("ec2").(clients.Ec2Client) | ||
| instance := state.Get("instance").(ec2types.Instance) | ||
| ui := state.Get("ui").(packersdk.Ui) |
| func (s *StepGetPassword) waitForPassword(ctx context.Context, state multistep.StateBag) (string, error) { | ||
| ec2Client := state.Get("ec2v2").(clients.Ec2Client) | ||
| ec2Client := state.Get("ec2").(clients.Ec2Client) | ||
| instance := state.Get("instance").(ec2types.Instance) | ||
| privateKey := s.Comm.SSHPrivateKey |
Description
This is the attempt to migrate plugin to AWS SDK v2 as v1 is deprecated for long time.
Resolved Issues
Closes #26
Rollback Plan
If a change needs to be reverted, we will roll out an update to the code within 7 days.
Changes to Security Controls
I am not sure if there are any relevant changes. I had to change the way how client connections are created as SDK v2 is quite different from v1.