(feature) adds support for lazy importing of rows#374
(feature) adds support for lazy importing of rows#374cwilsn wants to merge 7 commits intorap2hpoutre:masterfrom
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds lazy importing functionality to handle very large import files efficiently by using Laravel's LazyCollection instead of loading all data into memory at once.
- Introduces
importLazy()method that returns a LazyCollection for memory-efficient processing - Refactors existing import logic into reusable methods to support both eager and lazy loading
- Adds comprehensive test coverage for the new lazy import functionality
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/LazyImportTest.php | Adds test cases for lazy import functionality with and without callback mapping |
| src/Importable.php | Implements lazy import method, refactors row normalization logic, and adds generator-based sheet processing |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
| } | ||
| if ($this->transpose) { | ||
| // Fallback to non-lazy processing when transposing | ||
| throw new \Exception('Transposing is not supported with lazy import.'); |
There was a problem hiding this comment.
The error message should be more descriptive and use a specific exception type. Consider using InvalidArgumentException with a message like 'Transpose functionality is not compatible with lazy importing. Use import() method instead.'
| throw new \Exception('Transposing is not supported with lazy import.'); | |
| // Transpose functionality is not compatible with lazy importing | |
| throw new \InvalidArgumentException('Transpose functionality is not compatible with lazy importing. Use import() method instead.'); |
| $cell instanceof Cell\FormulaCell => $cell->getComputedValue(), | ||
| default => $cell->getValue(), | ||
| }; | ||
| }, $rowAsObject->getCells()); |
There was a problem hiding this comment.
The cell processing logic is duplicated between importSheet() and importSheetGenerator(). Consider extracting this into a private method like processCellValue(Cell $cell) to reduce code duplication.
| }, $rowAsObject->getCells()); | |
| $row = array_map([$this, 'processCellValue'], $rowAsObject->getCells()); |
| * @param array $headers | ||
| * @param int $count_header | ||
| * | ||
| * @return array|null |
There was a problem hiding this comment.
The normalizeRow method lacks proper PHPDoc documentation. It should include @param and @return annotations with type information and descriptions for the reference parameters.
| * @return array|null | |
| * @param int $k The current row index. | |
| * @param array $row The row data to normalize. | |
| * @param array &$headers Reference to the headers array, updated when header row is encountered. | |
| * @param int &$count_header Reference to the header count, updated when header row is encountered. | |
| * | |
| * @return array|null Returns an associative array if headers exist, the raw row if not, or null to skip processing. |
Adds support for lazy importing of rows to handle very large import files quickly and memory efficiently