For code like:
we currently don't require consistency between IDs when checking the state from line 1 to line 2. I.e., if the user were to mutate the object that x refers to by not changing its ID but instead incorrectly changing the current object's value to 10, then we would still consider this correct despite it potentially indicating a lack of understanding of immutability of integers.
To address this, we would need to introduce additional constraints that carry over between lines. Given that questions contain a solution at each execution line, those could potentially be leveraged for this, but when we allow for arbitrary line checking it becomes unclear what the "true" IDs are at any point in the trace.
One path forward would be to require that the user check each line in order, although this is then more tedious for long questions. For learning questions, this would be fine, but for longer questions it would mean that we either just stick with the current more permissive checking or entirely disable line checking for such questions.
Depending on the direction here, there may also be implications for the undo functionality and how we store a user's attempted solution on a question. E.g., would we actually store state per execution line or simply load up the "true" solution from a previous line if a user were to select a line that has already been correctly traced?
For code like:
we currently don't require consistency between IDs when checking the state from line 1 to line 2. I.e., if the user were to mutate the object that
xrefers to by not changing its ID but instead incorrectly changing the current object's value to 10, then we would still consider this correct despite it potentially indicating a lack of understanding of immutability of integers.To address this, we would need to introduce additional constraints that carry over between lines. Given that questions contain a solution at each execution line, those could potentially be leveraged for this, but when we allow for arbitrary line checking it becomes unclear what the "true" IDs are at any point in the trace.
One path forward would be to require that the user check each line in order, although this is then more tedious for long questions. For learning questions, this would be fine, but for longer questions it would mean that we either just stick with the current more permissive checking or entirely disable line checking for such questions.
Depending on the direction here, there may also be implications for the undo functionality and how we store a user's attempted solution on a question. E.g., would we actually store state per execution line or simply load up the "true" solution from a previous line if a user were to select a line that has already been correctly traced?