Non-invertible matrix in interpolation

[soerenthomsen]

Discussed in #153

^{Originally posted by lily-dove July 23, 2022}

Hi all! My summer student and I are using GliderTools for a project. It's been wonderful so far! We've come across a challenge where we're trying to do the horizontal interpolation for chunks of ~50000 datapoints. It's worked for several chunks of data but we've hit one where halfway through the interpolation, we get a "LinAlgError: singular matrix" error. I'll post the full error message as an attachment. I'd appreciate any insight y'all have on how to go about troubleshooting this...

Thank you!

[callumrollo]

Hi @lily-dove.

Thanks for submitting this Issue. Interpolation isn't my strong suit, but uere are a few of things to try for troubleshooting:

• Varying the number of parallel_chunk_size and possibly n_cpus. It could be that it's making the last chunk of data too small, so you end up with a matrix of no data.
• Find the section of the dataset that is causing the problem and see if it is different from its neighbours: e.g. more nans, different depth extent, bad timestamps etc.

If these don't shed any light on the problem, make a minimal reproducible example with the data and code that generates the error and I'll take a look at it.