Skip to content

Batch operations do not use the RetryHelper settings #3512

@Baisang

Description

@Baisang

Stack trace

	2026-02-25 15:43:42.660	
	at com.google.cloud.storage.StorageException.translate(StorageException.java:230) ~[bq-sink-batch.jar:0.1.0-SNAPSHOT]
	2026-02-25 15:43:42.660	
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:360) ~[bq-sink-batch.jar:0.1.0-SNAPSHOT]
	2026-02-25 15:43:42.660	
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.access$500(HttpStorageRpc.java:119) ~[bq-sink-batch.jar:0.1.0-SNAPSHOT]
	2026-02-25 15:43:42.660	
	at com.google.cloud.storage.spi.v1.HttpStorageRpc$DefaultRpcBatch.submit(HttpStorageRpc.java:330) ~[bq-sink-batch.jar:0.1.0-SNAPSHOT]
	2026-02-25 15:43:42.660	
	at com.google.cloud.storage.StorageBatch.submit(StorageBatch.java:159) ~[bq-sink-batch.jar:0.1.0-SNAPSHOT]
	2026-02-25 15:43:42.660	
	at com.google.cloud.storage.StorageImpl.delete(StorageImpl.java:1268) ~[bq-sink-batch.jar:0.1.0-SNAPSHOT]
	2026-02-25 15:43:42.660	
	at org.apache.flink.fs.gs.storage.GSBlobStorageImpl.delete(GSBlobStorageImpl.java:246) ~[?:?]
	2026-02-25 15:43:42.660	
	at org.apache.flink.fs.gs.writer.GSRecoverableWriterCommitter.cleanupTemporaryBlobs(GSRecoverableWriterCommitter.java:244) ~[?:?]
	2026-02-25 15:43:42.660	
	at 

External references such as API reference guides

Any additional information below

#203

Hi, I wanted to reopen issue 203 as we saw it come up in our Flink application. Despite adjusting all of our other retry settings, we noticed that delete batch requests were not being retried after tracing through the code and finding the previous issue opened.

Each of these failures can cause a cascading effect when GCP buckets are under strain where that Flink checkpoint fails, and the subsequent one will submit more delete batch requests when catching up which are then more likely to run into this terminal error.

It's really confusing as a library user that the normal HTTP retry settings have no effect here so I would petition that we reopen and use #305

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: storageIssues related to the googleapis/java-storage API.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions