LIFT-2453: Restore async FirehoseClient to fix player disconnections (0.3.x)

[mason-chester-rp]

Summary

• Restores fire-and-forget async Firehose behaviour on release-0.3.x, lost during the LIFT-1806 AWS SDK v2 migration
• Swaps FirehoseClient (sync) for FirehoseAsyncClient (async) in FirehoseDb — putRecordBatch() on the async client returns a CompletableFuture that is intentionally not awaited at request time
• listDeliveryStreams() in bootstrapApi() calls .join() since it is a one-time startup operation where blocking is acceptable
• Bumps version to 0.3.27

Related

• LIFT-2452 / PR LIFT-2452: Restore async FirehoseClient to fix player disconnections #216: same fix already applied to release-0.3.6.x
• liftck_heartbeat depends on rocket-inversion:0.3.26 — once this is merged and published, heartbeat should be updated to 0.3.27

Test plan

• Build passes (./gradlew test)
• liftck_heartbeat updated to 0.3.27 and deployed to dev/e2e
• Confirm request very slow warnings and SocketTimeoutException errors drop to zero
• Confirm Firehose stream delivery is unaffected

[joel-katz-rp]

putRecordBatch() on the async client returns a CompletableFuture that is currently discarded without an error handler. The existing catch block won't catch async failures, so if Firehose rejects a batch the error will be silently swallowed — no log, no exception, and the CloudWatch failure metrics won't fire either.

Would you consider attaching an .exceptionally() handler to the returned future to at least log failures? Something like:

  firehose.putRecordBatch(request)
      .exceptionally(ex -> {
          log.error("Error putting records to firehose stream '{}': {}", streamName, ex.getMessage(), ex);
          return null;
      }); 

[joel-katz-rp]

Looking into this some more, before upgrading to the latest sdk, we didn't log errors or fire exceptions in this way either. I'm honestly not sure how this code block was ever not getting a 200/201: https://github.com/RocketPartners/liftck_snooze/blob/987bdac2d66a6aa9244982cd374865c7aa535b70/src/main/java/com/liftck/snooze/api/PlayerEventHandler.java#L103-L111

[connor-brown-rp]

++ to Joel's comment above
(although, the old sdk v1 code also seemed to discard any async errors, so if we don't add the error logging, then at least its not a regression of functionality)