Background
The VM currently uses a Lua-style open/closed upvalue system. Captured variables start as stack slots and migrate to heap-allocated cells when their enclosing frame exits. This works correctly but carries ongoing complexity: open_upvalues, capture_upvalue, close_upvalues, CloseUpvalue opcode, and the UpvalueCell::Open/Closed distinction.
Bug B5 (materialize_upvalues_in_args) was a direct consequence of this design and was fixed by deleting the offending function (see #discussion in UPVALUE_DESIGN.md). However, the underlying open/closed mechanism remains and could produce similar bugs in future edge cases.
Proposed change
Mark captured variables during the analysis/compilation pass. Instead of starting life as stack slots and migrating to the heap on frame exit, captured variables are heap-allocated (Rc<RefCell<Value>>) from the moment of declaration. The outer frame and all inner closures hold a reference to the same heap cell from the start — no open→closed transition.
Non-captured locals are unaffected and remain cheap stack slots.
What changes
- The analyser already tracks which variables are captured by inner closures — that information is used to tag declarations
- The compiler emits
GetUpvalue/SetUpvalue for captured variables in the outer function too, not just in inner closures
UpvalueCell::Open variant is deleted — cells always hold a Valueopen_upvalues, capture_upvalue, close_upvalues, and CloseUpvalue are all removed- Loop iteration isolation (currently
CloseUpvalue) is replaced by the compiler emitting "allocate fresh cell, copy current value" at the top of each loop body for captured loop variables
Trade-offs
|
Option B (this issue) |
| Lines changed |
~200–400, multiple files |
| Risk |
Medium |
| Eliminates open/closed bug class |
Yes |
| Runtime cost |
Heap deref for captured vars (small fraction of all locals) |
References
See ndc_vm/UPVALUE_DESIGN.md for the full analysis including the alternative options that were considered.
Background
The VM currently uses a Lua-style open/closed upvalue system. Captured variables start as stack slots and migrate to heap-allocated cells when their enclosing frame exits. This works correctly but carries ongoing complexity:
open_upvalues,capture_upvalue,close_upvalues,CloseUpvalueopcode, and theUpvalueCell::Open/Closeddistinction.Bug B5 (
materialize_upvalues_in_args) was a direct consequence of this design and was fixed by deleting the offending function (see #discussion in UPVALUE_DESIGN.md). However, the underlying open/closed mechanism remains and could produce similar bugs in future edge cases.Proposed change
Mark captured variables during the analysis/compilation pass. Instead of starting life as stack slots and migrating to the heap on frame exit, captured variables are heap-allocated (
Rc<RefCell<Value>>) from the moment of declaration. The outer frame and all inner closures hold a reference to the same heap cell from the start — no open→closed transition.Non-captured locals are unaffected and remain cheap stack slots.
What changes
GetUpvalue/SetUpvaluefor captured variables in the outer function too, not just in inner closuresUpvalueCell::Openvariant is deleted — cells always hold aValueopen_upvalues,capture_upvalue,close_upvalues, andCloseUpvalueare all removedCloseUpvalue) is replaced by the compiler emitting "allocate fresh cell, copy current value" at the top of each loop body for captured loop variablesTrade-offs
References
See
ndc_vm/UPVALUE_DESIGN.mdfor the full analysis including the alternative options that were considered.