Describe the bug
The array_element, array_any_value UDFs use a ListArray's offsets to deduce that a row is Null. If the difference between two consecutive offsets is 0 then the row should be Null.
This goes against the Arrow spec which allows positive slot lengths for Null rows.
It should be noted that a null value may have a positive slot length. That is, a null value may occupy a non-empty memory space in the data buffer. When this is true, the content of the corresponding memory space is undefined.
Expected behavior
The ListArray's null buffer should be used instead.
Describe the bug
The
array_element,array_any_valueUDFs use aListArray's offsets to deduce that a row is Null. If the difference between two consecutive offsets is0then the row should be Null.This goes against the Arrow spec which allows positive slot lengths for Null rows.
Expected behavior
The
ListArray's null buffer should be used instead.