Fix: initialize facet to -1 in line_positive_intersect to prevent undefined behavior#457
Conversation
In HPolytope::line_positive_intersect (update_parameters overload), the variable 'facet' was declared but not initialized: int m = num_of_hyperplanes(), facet; When no valid positive intersection exists (valid_plus is all-false), the if-block is skipped so 'facet' is never assigned. The garbage value then propagates into params.facet_prev, which is used in the next reflection step to index into AA.col(params.facet_prev) causing out-of-bounds access and undefined behavior. Fixed by initializing facet = -1, consistent with all other overloads of line_positive_intersect in this file. The guard 'if (params.facet_prev >= 0)' already handles the -1 sentinel. Affected locations: 4 occurrences across hpolytope.h.
|
hi @vissarion sir, is there any change required? |
vissarion
left a comment
There was a problem hiding this comment.
Thanks for this PR. Please provide the code triggered the bug. The screenshots are not that useful.
|
hi @vissarion sir, here is the minimal reproducible c++ snippet that triggers the bug, if a ray originates at the boundary and points directly outwards, it never finds a positive intersection (lamda > 0 condition fails). // Create a simple 1D polytope: x <= 1
Eigen::MatrixXd A(1, 1); A << 1.0;
Eigen::VectorXd b(1); b << 1.0;
HPolytope<Cartesian<double>> P(1, A, b);
// Origin point on the boundary (x = 1), pointing outwards (+1 direction)
Point r(1); r.set_coord(0, 1.0);
Point v(1); v.set_coord(0, 1.0);
Eigen::VectorXd Ar, Av;
// This will return an uninitialized (garbage) integer for facet
// because no lambda > 0 is found.
std::pair<double, int> intersection = P.line_positive_intersect(r, v, Ar, Av);
|
|
in the [line_positive_intersect] function, the facet variable is declared without initialization by explicitly initializing facet = -1 at declaration, we ensure that if no positive intersection is found, the function returns a deterministic invalid state -1 rather than triggering undefined behavior with a garbage value. |
Thank you for the example. Out of curiosity how you discovered that issue? Do you have a sampling or volume example that triggers this behavior? |
…ne-positive-intersect
hi @vissarion sir, |
hi @vissarion sir
In
HPolytope::line_positive_intersect(theupdate_parametersoverload, ~line 520 ofhpolytope.h), the variablefacetis declared but never initialized:this by default takes garabge value, so it leads to error.
i have set this value to -1.
i have also attached all the proofs below.