Improve handling of kafka offsets and capabilities of the monitor#86
Open
flowln wants to merge 11 commits intocnpem:mainfrom
Open
Improve handling of kafka offsets and capabilities of the monitor#86flowln wants to merge 11 commits intocnpem:mainfrom
flowln wants to merge 11 commits intocnpem:mainfrom
Conversation
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
It seems like development has picked up again in the original kafka-python project, and some good bug fixes are in on the newest versions. Thus, we can switch our dependency. Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Now the seeking will work on any document type, including the 'stop' document, which previously wouldn't work at all. The function was moved out of the class for ease of testing, in the next commit of this patchset. Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
flowln
added
bug
Signed-off-by: Sofia Donato Ferreira <sofia.ferreira@lnls.br>
flowln
force-pushed
the
better_kafka_monitor
branch
from
9b08288 to
2f859e7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is kind of a sinkhole where I've put everything I wanted to change in the Kafka monitor. It contains a lot of refactoring to streamline and simplify the code, as well as changes to how we interact with the
KafkaConsumerAPI, now that I understand it a bit better.Furthermore, this patch series contains the changes to
seek_start(nowseek_start_document) from #41, as well as a feature imported fromsophys-live-view,seek_back_in_timewith thehour_offsetparameter in the monitor. My intention here is to have a robust and generic implementation for these utilities, so we can share code betweensophys-common,sophys-live-view, and maybe others in the future.I've spent quite some effort in the automated tests, so that we could trust those implementation changes. I would suggest looking at them when reviewing to maybe get a better understanding of what the code is supposed to be doing!
Supersedes #41