Create summary.json of results from K6 run#8
Conversation
Signed-off-by: waleedqk <waleedqk@ibm.com> On branch wqk_prom_scrape Changes to be committed: modified: perf_test/runHowitzer.sh new file: perf_test/scripts/scraper.py
|
|
||
| # emphasize "Generating intermediate Markdown for K6 tests..." | ||
| # python3 -m src.k6.scraper -r results -s summary -p persistent-results -c $CONFIG_FILE -k configs/kingdom.dict | ||
| emphasize "Generating intermediate Markdown for K6 tests..." |
There was a problem hiding this comment.
There's another python3 -m perf_test.scripts.scraper -r results -s summary -c $CONFIG_FILE down below that basically does the same command. Is there a difference here? Maybe we only need one?
There was a problem hiding this comment.
Yeah, so testing it out locally and understanding the code as well, I think we can just remove the invocation in the loop and just running it at the end will suffice
Testing it on the perf cluster to confirm no change in behaviour
Signed-off-by: waleedqk <waleedqk@ibm.com>
…ssed and dir exists Signed-off-by: waleedqk <waleedqk@ibm.com>
Signed-off-by: waleedqk <waleedqk@ibm.com>
… scraping Signed-off-by: waleedqk <waleedqk@ibm.com>
Signed-off-by: waleedqk <waleedqk@ibm.com>
waleedqk
changed the title
Signed-off-by: waleedqk <waleedqk@ibm.com>
Signed-off-by: waleedqk <waleedqk@ibm.com>
Signed-off-by: waleedqk <waleedqk@ibm.com>
| # echo "kubectl cp ${POD_NAME}:output.md ./output.md && kubectl cp ${POD_NAME}:summary.json ./summary.json" | ||
|
|
||
| # Add a sleep at the end of the job to give time to exec into pods if need be | ||
| sleep 120 No newline at end of file |
There was a problem hiding this comment.
do you still want to keep this sleep here?
| - Then the script will run each individual k6 test in this `render` directory, storing the human readable results as text in `results` directory. | ||
| - It will also store the corresponding json file of the result into `summary` directory that we can later use for MLFlow. | ||
| - It will also store the corresponding json file of the result into `summary` directory. | ||
| - The scraper script then consolidates the results from the K6 test, the input test conditions etc. into a single `summary.json` file for the tests that were run. |
There was a problem hiding this comment.
I think since now we have multiple scripts, let's be explicit and say the perf_test.scripts.scraper script, instead of just scraper script.
| @@ -16,7 +16,8 @@ The core design principle beneath Howitzer lies in "exploding" K6 parameter iter | |||
| When Howitzer is run, the bash script will create directories `render`, `results` and `summary`: | |||
There was a problem hiding this comment.
can you change this to say
the bash script `perf_test/runHowitzer.sh`
so we can be specific
aluu317
left a comment
aluu317
left a comment
There was a problem hiding this comment.
This looks good! We should also add the unit test for scraper.py in the unit_test folder just to be complete
…used for debugging only. Signed-off-by: waleedqk <waleedqk@ibm.com>
|
/lgtm |
|
We need to fix the issue with github not being able to install python in the github action. @waleedqk Please also change the .github/workflows file to run |
Signed-off-by: waleedqk <waleedqk@ibm.com>
|
New changes are detected. LGTM label has been removed. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: waleedqk The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
@waleedqk Can you do me a favor and try with version |
3 participants
When a K6 perf test is being run, individual files are created that hold the result for each test that is part of the submitted set.
The scrapper script consolidates the results from each of the output files and joins them into a single
summary.jsonfile.