Skip to content

v0.14.4#990

Merged
Amber-Rigg merged 14 commits into
mainfrom
develop
Apr 20, 2026
Merged

v0.14.4#990
Amber-Rigg merged 14 commits into
mainfrom
develop

Conversation

@Amber-Rigg
Copy link
Copy Markdown
Contributor

Summary

This PR introduces improvements to the interpolation logic, time‑weighted averaging and fixes minor bugs identified during recent development work.
Key Changes

Testing

✅ All changes have been fully tested in the dev environment
✅ Existing test cases pass
✅ Manual verification performed for time‑weighted interpolation scenarios

Impact / Risk

Low risk: Changes are isolated to interpolation and related logic.
No breaking API or schema changes.
Backwards‑compatible for existing consumers.

Amber-Rigg and others added 14 commits February 24, 2026 13:23
…variable assignments, update with Black formatting

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
* Enhance query parameter definitions to enforce required fields and clarify default values with window length for twa as 1 day

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update tests to reflect default 1 value

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor code to improve readability by simplifying tuple unpacking and removing unnecessary line breaks

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove unnecessary blank lines for improved code cleanliness across multiple files

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to environment.yml and setup.py

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to use try-except for better error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to simplify code and improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to the main dependencies section in environment.yml

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools installation to pip section in environment.yml for turbodbc compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Ensure setuptools is installed in test workflows for compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor import handling for turbodbc and SparkConnection to improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Skip tests if SparkConnection is unavailable due to missing turbodbc dependencies

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update requirements and refactor SparkConnection import handling for improved compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove version constraint for setuptools in environment.yml for flexibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update import

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Only enable turbodbc if import fully succeeds

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update turbodbc to "turbodbc>=4.5.0"

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Fix tests with required api section.

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Common_api_setup_tasks() returns (connection, parameters), but the code only needs connection, so _ tells the linter "we know this is unused, it's intentional."

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

---------

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Signed-off-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
* Enhance query parameter definitions to enforce required fields and clarify default values with window length for twa as 1 day

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update tests to reflect default 1 value

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor code to improve readability by simplifying tuple unpacking and removing unnecessary line breaks

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove unnecessary blank lines for improved code cleanliness across multiple files

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to environment.yml and setup.py

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to use try-except for better error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to simplify code and improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to the main dependencies section in environment.yml

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools installation to pip section in environment.yml for turbodbc compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Ensure setuptools is installed in test workflows for compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor import handling for turbodbc and SparkConnection to improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Skip tests if SparkConnection is unavailable due to missing turbodbc dependencies

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update requirements and refactor SparkConnection import handling for improved compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove version constraint for setuptools in environment.yml for flexibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update import

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Only enable turbodbc if import fully succeeds

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update turbodbc to "turbodbc>=4.5.0"

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Fix tests with required api section.

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Common_api_setup_tasks() returns (connection, parameters), but the code only needs connection, so _ tells the linter "we know this is unused, it's intentional."

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor Dockerfile for improved readability and consistency

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

---------

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Signed-off-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
* Enhance query parameter definitions to enforce required fields and clarify default values with window length for twa as 1 day

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update tests to reflect default 1 value

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor code to improve readability by simplifying tuple unpacking and removing unnecessary line breaks

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove unnecessary blank lines for improved code cleanliness across multiple files

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to environment.yml and setup.py

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to use try-except for better error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to simplify code and improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to the main dependencies section in environment.yml

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools installation to pip section in environment.yml for turbodbc compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Ensure setuptools is installed in test workflows for compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor import handling for turbodbc and SparkConnection to improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Skip tests if SparkConnection is unavailable due to missing turbodbc dependencies

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update requirements and refactor SparkConnection import handling for improved compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove version constraint for setuptools in environment.yml for flexibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update import

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Only enable turbodbc if import fully succeeds

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update turbodbc to "turbodbc>=4.5.0"

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Fix tests with required api section.

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Common_api_setup_tasks() returns (connection, parameters), but the code only needs connection, so _ tells the linter "we know this is unused, it's intentional."

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor Dockerfile for improved readability and consistency

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* remove trailing spaces in dockerfile

Signed-off-by: cching95 <chloe.ching@shell.com>

---------

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Signed-off-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Signed-off-by: cching95 <chloe.ching@shell.com>
Co-authored-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Co-authored-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
* Enhance query parameter definitions to enforce required fields and clarify default values with window length for twa as 1 day

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update tests to reflect default 1 value

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor code to improve readability by simplifying tuple unpacking and removing unnecessary line breaks

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove unnecessary blank lines for improved code cleanliness across multiple files

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to environment.yml and setup.py

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to use try-except for better error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to simplify code and improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to the main dependencies section in environment.yml

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools installation to pip section in environment.yml for turbodbc compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Ensure setuptools is installed in test workflows for compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor import handling for turbodbc and SparkConnection to improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Skip tests if SparkConnection is unavailable due to missing turbodbc dependencies

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update requirements and refactor SparkConnection import handling for improved compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove version constraint for setuptools in environment.yml for flexibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update import

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Only enable turbodbc if import fully succeeds

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update turbodbc to "turbodbc>=4.5.0"

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Fix tests with required api section.

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Common_api_setup_tasks() returns (connection, parameters), but the code only needs connection, so _ tells the linter "we know this is unused, it's intentional."

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor Dockerfile for improved readability and consistency

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* remove trailing spaces in dockerfile

Signed-off-by: cching95 <chloe.ching@shell.com>

* fix: remove trailing whitespace from Dockerfile to fix build error

* fix: add cmake and pkg-config to Dockerfile build dependencies

- Add cmake and pkg-config packages required for building Python dependencies
- Resolves build errors: 'Did not find CMake' and 'Pkg-config not found'
- These are needed for turbodbc and other packages in requirements.txt

---------

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Signed-off-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Signed-off-by: cching95 <chloe.ching@shell.com>
Co-authored-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Co-authored-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
* Enhance query parameter definitions to enforce required fields and clarify default values with window length for twa as 1 day

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update tests to reflect default 1 value

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor code to improve readability by simplifying tuple unpacking and removing unnecessary line breaks

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove unnecessary blank lines for improved code cleanliness across multiple files

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to environment.yml and setup.py

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to use try-except for better error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor turbodbc import handling to simplify code and improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools dependency to the main dependencies section in environment.yml

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Add setuptools installation to pip section in environment.yml for turbodbc compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Ensure setuptools is installed in test workflows for compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor import handling for turbodbc and SparkConnection to improve error management

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Skip tests if SparkConnection is unavailable due to missing turbodbc dependencies

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update requirements and refactor SparkConnection import handling for improved compatibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove version constraint for setuptools in environment.yml for flexibility

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update import

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Only enable turbodbc if import fully succeeds

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update turbodbc to "turbodbc>=4.5.0"

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Fix tests with required api section.

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Common_api_setup_tasks() returns (connection, parameters), but the code only needs connection, so _ tells the linter "we know this is unused, it's intentional."

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor Dockerfile for improved readability and consistency

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* remove trailing spaces in dockerfile

Signed-off-by: cching95 <chloe.ching@shell.com>

* fix: remove trailing whitespace from Dockerfile to fix build error

* fix: add cmake and pkg-config to Dockerfile build dependencies

- Add cmake and pkg-config packages required for building Python dependencies
- Resolves build errors: 'Did not find CMake' and 'Pkg-config not found'
- These are needed for turbodbc and other packages in requirements.txt

* docker_file updated to resolve turboodbc issue

---------

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Signed-off-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Signed-off-by: cching95 <chloe.ching@shell.com>
Signed-off-by: srjhunjhunwalacorp <64076463+srjhunjhunwalacorp@users.noreply.github.com>
Co-authored-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Co-authored-by: Amber Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Co-authored-by: cching95 <chloe.ching@shell.com>
Signed-off-by: srjhunjhunwalacorp <64076463+srjhunjhunwalacorp@users.noreply.github.com>
* fix the dockerfile issue

* pakcage depedency install

---------

Signed-off-by: srjhunjhunwalacorp <64076463+srjhunjhunwalacorp@users.noreply.github.com>
* fix the dockerfile issue

* pakcage depedency install

* turbodc update

---------

Signed-off-by: srjhunjhunwalacorp <64076463+srjhunjhunwalacorp@users.noreply.github.com>
* fix the dockerfile issue

* pakcage depedency install

* turbodc update

* update the old config to build the image

---------

Signed-off-by: srjhunjhunwalacorp <64076463+srjhunjhunwalacorp@users.noreply.github.com>
Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
* refactor interpolate query to use FULL OUTER JOIN for improved data handling

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Black formatting

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update test with correct dummy query

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Remove debug print statement from interpolation query function

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

---------

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
Signed-off-by: cching95 <chloe.ching@shell.com>
* Enhance interpolation documentation and update mocked queries

- Added detailed warning in `interpolate.py` regarding time gradient buffering for accurate boundary interpolation.
- Updated mocked SQL queries in `sdk_test_objects.py` to include a ±5 minute buffer for the requested date range, ensuring accurate interpolation.
- Adjusted the structure of the mocked queries to reflect the new logic for handling event times and values.

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor interpolate query to include display_uom parameter and update return values

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor _build_interpolate_query to improve formatting and readability

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Enhance _build_output_query to support explicit column selection and ordering

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor _build_raw_query_for_interpolate to remove agg_method parameter and update handling in related functions

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor _build_raw_query_for_interpolate to remove time_interval_rate parameter

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Refactor _interpolation_query to remove time_interval_rate parameter

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

* Update documentation in get function to clarify buffer boundaries for sparse data

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>

---------

Signed-off-by: Amber-Rigg <amber.l.rigg25@gmail.com>
@Amber-Rigg Amber-Rigg self-assigned this Apr 20, 2026
@Amber-Rigg Amber-Rigg added the bug Something isn't working label Apr 20, 2026
@Amber-Rigg Amber-Rigg requested a review from ewatson95 April 20, 2026 12:10
Copy link
Copy Markdown
Contributor

@srjhunjhunwalacorp srjhunjhunwalacorp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test came good and confirmed by SOM

@Amber-Rigg Amber-Rigg changed the title Enhance Interpolation Logic with Time‑Weighted Averages + Minor Bug Fixes v0.14.4 Apr 20, 2026
@Amber-Rigg Amber-Rigg merged commit bcc44fe into main Apr 20, 2026
40 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants