Could you consider making the HTTP reader resilient to dropped connections by using Range headers, similar to how wandio handles it?
Currently, long-running or delayed remote reads can trigger server-side timeouts (RIPE RIS drops connections after 10 seconds of inactivity). This can cause silent failures in downstream projects like bgpkit (for example bgpkit/bgpkit-parser#288).
I was able to fix this issue with a few additions on my fork to implement a resumable reader:
main...JustinLoye:feature/resumable-http
This is outside my comfort zone and I used mostly AI to make the POC, so if the feature interest you feel free to ignore or modify the code!
Could you consider making the HTTP reader resilient to dropped connections by using Range headers, similar to how wandio handles it?
Currently, long-running or delayed remote reads can trigger server-side timeouts (RIPE RIS drops connections after 10 seconds of inactivity). This can cause silent failures in downstream projects like bgpkit (for example bgpkit/bgpkit-parser#288).
I was able to fix this issue with a few additions on my fork to implement a resumable reader:
main...JustinLoye:feature/resumable-http
This is outside my comfort zone and I used mostly AI to make the POC, so if the feature interest you feel free to ignore or modify the code!