v2 Plugin System: lambda-init, lambda-build, lambda-deploy

[sebsto]

This PR delivers the v4 plugin system, replacing the legacy single-purpose archive plugin with three focused SwiftPM command plugins that cover the end-to-end developer experience:

• lambda-init — Scaffold a new Lambda function from a template
• lambda-build — Compile and package for Amazon Linux 2023 (via Docker or Apple container)
• lambda-deploy — Deploy to AWS Lambda (create, update, or delete)

The legacy archive command is preserved as a deprecated passthrough to lambda-build.

Dependencies

Important

This PR depends on soto-project/soto-core#694 (fixes soto-project/soto-core#693) being merged and released.

Before merging this PR, the Package.swift dependency on soto-core must be reverted from the temporary branch on my fork (sebsto/soto-core, branch remove-platforms-use-availability-macro) back to the official repository:

.package(url: "https://github.com/soto-project/soto-core.git", from: "<version-with-fix>")

CI: API Breakage Check False Positives

Note

The CI "API breakage check" reports 33 false-positive "removed" types. This is a known SwiftPM bug (swiftlang/swift-package-manager#8081, #9254).

Root cause: The plugins depend on AWSLambdaPluginHelper which depends on soto-core. soto-core and its transitive dependencies bring C targets (CSotoExpat, CNIOBoringSSL, CNIOBoringSSLShims) that get built for both .host (plugin) and .target (library) destinations. The API digester receives -I paths for both builds, triggering silent "redefinition of module" errors that prevent it from loading AWSLambdaRuntime. All symbols then appear as "removed".

The AWSLambdaRuntime source code is unchanged between this branch and main. There are no actual API breakages. A partial fix landed in Swift 6.3.2 (PR #8776) for Swift targets, but Clang targets remain unfiltered. The full fix is tracked in PR #7664.

Quick Start

# Create a new project
swift package init --type executable --name MyLambda

# Scaffold a Lambda function
swift package lambda-init --allow-writing-to-package-directory

# Build for Amazon Linux
swift package --allow-network-connections docker lambda-build

# Deploy to AWS
swift package --allow-network-connections all:443 lambda-deploy

# Delete when done
swift package --allow-network-connections all:443 lambda-deploy --delete

Key Changes

Architecture

• All plugins are thin wrappers that spawn a shared AWSLambdaPluginHelper executable
• The helper dispatches to Initializer, Builder, or Deployer based on argv[1]
• AWS API calls use generated service clients (Lambda, IAM, S3, STS) built on SotoCore

Builder (lambda-build)

• Default base image changed from amazonlinux2 to amazonlinux2023
• AL2 blanket deprecation warning removed; targeted warning only when AL2 explicitly chosen
• New --cross-compile option (replaces --container-cli): docker, container, swift-static-sdk, custom-sdk
• Default binary stripping with -Xlinker -s and --no-strip opt-out
• --output-directory accepted as deprecated alias for --output-path
• Container CLI existence check with helpful install URLs

Deployer (lambda-deploy)

• Creates/updates/deletes Lambda functions with full IAM role lifecycle
• Auto-creates IAM role with AWSLambdaBasicExecutionRole
• S3 staging for archives > 50 MB
• Function URL support with IAM auth (account-scoped, not world-accessible)
• Auto-detects FunctionURLRequest usage in source code to enable URL without explicit --with-url
• Ready-to-use invocation commands in deploy output

Initializer (lambda-init)

• Detects the actual entry point file (supports both Sources/main.swift and Sources/<Name>/<Name>.swift)
• Backs up existing file before overwriting

Dependencies

• Added soto-core for AWS credential management, SigV4 signing, and HTTP transport
• Removed vendored crypto/signer/HTTP code under Vendored/
• Generated AWS service clients committed to the repository (maintainer-run generation script)

Backward Compatibility

• swift package archive preserved as deprecated alias (emits warning, delegates to lambda-build)
• --container-cli accepted as deprecated alias for --cross-compile
• --output-directory accepted as deprecated alias for --output-path
• All original archive CLI options continue to work

Documentation

• Updated DocC articles, tutorials, and quick-setup guide
• Updated all example READMEs with new plugin commands
• SAM/CDK examples preserved with their respective deployment tools

Known Limitations

• Static Linux SDK build disabled (swift-crypto/CryptoKit incompatibility with musl)
• SotoCore LoginCredentialProvider cannot persist refreshed tokens in SwiftPM sandbox (workaround: tokens refresh in-memory, PR sent to soto-core (Make LoginCredentialProvider token persistence best-effort soto-project/soto-core#692))
• SwiftPM prompts for network permission on first run (known SPM issue #9763)

Testing

• Unit tests for BuilderConfiguration and DeployerConfiguration argument parsing
• Property-based tests for correctness properties (alias equivalence, cross-compile round-trip, mutual exclusion, bucket naming, archive threshold, AL2 warning logic)
• End-to-end integration test script (scripts/integration-test.sh)
• Manually tested: full create → invoke → update → delete lifecycle on AWS

[sebsto]

Issue tracker #566

[tkrajacic]

Could we add the ability to pass build options. Sometimes it is nice to optimize for size and we can use

-Xlinker -s -Xswiftc -Osize

[sebsto]

Could we add the ability to pass build options. Sometimes it is nice to optimize for size and we can use

Thank you @tkrajacic

This is planned. This will be a default setting, with an opt-out possibility --no-strip

We also propose to automatically strip the binary of debug symbols (-Xlinker -s) to reduce the size of the ZIP file. Our tests showed that this can reduce the size by up to 50%. An option to disable stripping will be provided.

I think -Xswiftc -Osize is the default with --release, but I will double check.

UPDATE: It's difficult to get an authoritative list of compiler and linker flags passed with --release. It looks like only -O is given. I'm not sure if -O size should be a default or an opt-in. Depending on the use case, Lambda developers might favor raw performance vs cold start time. It would be interesting to measure the effect of these. We need a basic benchmark suite to measure impact of these flags before taking decisions

[tkrajacic]

It's difficult to get an authoritative list of compiler and linker flags passed with --release

I was thinking more along the lines that you can pass arbitrary flags through to the compiler.
maybe just everything at the end of the invocation and after a -- or so?
I would definitely let the user decide how they want to compile their code.

Alternatively, why not just allow the user to add any -Xswiftc xxx flags and add them to the compiler flags?

[sebsto]

Alternatively, why not just allow the user to add any -Xswiftc xxx flags and add them to the compiler flags?

I think we should provide both. 1/ a sensible default set of options (like -Xlinker -s) by default to allow the best performance for all users and 2/ a possibility to pass any command for advanced users (that will override the default)