Paginate superuser users/projects lists with server-side search#4474
Open
sakibsadmanshajib wants to merge 12 commits intoOpenFn:mainfrom
Open
Paginate superuser users/projects lists with server-side search#4474sakibsadmanshajib wants to merge 12 commits intoOpenFn:mainfrom
sakibsadmanshajib wants to merge 12 commits intoOpenFn:mainfrom
Conversation
a7f7859 to
fcf4b25
Compare
There was a problem hiding this comment.
Pull request overview
This PR improves superuser admin list performance by moving /settings/users and /settings/projects filtering/sorting/pagination from in-memory operations to database-backed queries with URL-param-driven state, and adds trigram indexes to support efficient search.
Changes:
- Added
Accounts.list_users_for_admin/1andProjects.list_projects_for_admin/1usingRepo.paginate/2with normalized params for safe filtering/sorting. - Updated Users/Projects LiveViews to treat URL params (
page,filter,sort,dir,page_size) as the source of truth and to use Scrivener pages in table rendering. - Added
pg_trgm+ GIN trigram indexes for key search columns, plus tests covering pagination and param safety.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/lightning_web/live/user_live_test.exs | Adds LiveView pagination navigation coverage for users index. |
| test/lightning_web/live/project_live_test.exs | Adds LiveView pagination navigation coverage for projects index. |
| test/lightning/projects_test.exs | Adds coverage for admin projects search/sort and invalid param fallback. |
| test/lightning/accounts_test.exs | Adds coverage for admin users search/sort and invalid param fallback. |
| priv/repo/migrations/20260226162000_add_admin_search_trgm_indexes.exs | Adds pg_trgm extension and trigram indexes for admin search paths. |
| lib/lightning_web/live/user_live/table_component.ex | Switches users table component to Scrivener-page-driven rendering + URL patching. |
| lib/lightning_web/live/user_live/index.html.heex | Passes normalized table_params down to the table component. |
| lib/lightning_web/live/user_live/index.ex | Normalizes/validates table URL params (sort/dir/page/page_size/filter). |
| lib/lightning_web/live/user_live/components.ex | Updates users table component API to accept page + pagination_path. |
| lib/lightning_web/live/project_live/index.html.heex | Switches projects table to use Scrivener page entries + pagination URL. |
| lib/lightning_web/live/project_live/index.ex | Adds param normalization and uses Projects.list_projects_for_admin/1 for server-side table data. |
| lib/lightning/projects.ex | Implements paginated/sortable/searchable admin projects query. |
| lib/lightning/accounts.ex | Implements paginated/sortable/searchable admin users query. |
| docs/plans/2026-02-26-superuser-users-projects-pagination-search-design.md | Documents design/approach and rollout considerations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
midigofrank
requested changes
Mar 3, 2026
Collaborator
midigofrank
left a comment
There was a problem hiding this comment.
Hey @sakibsadmanshajib , great job here. The superuser interface needs some love, thank you for picking this up.
I have left some comments:
- I don't see a need for indexes on those tables. They're usually tiny
- Consider using using an embedded schema to have a single place for validations, like: https://github.com/OpenFn/lightning/blob/main/lib/lightning/workorders/search_params.ex
- With the embedded schema in place, you can try to refactor the list_users function, like: https://github.com/OpenFn/lightning/blob/main/lib/lightning/invocation.ex#L482
priv/repo/migrations/20260226162000_add_admin_search_trgm_indexes.exs
Outdated
Show resolved
Hide resolved
queries/live views refactored to use them
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds server-side pagination, sorting, and search to the superuser users/projects settings list views, and moves those list operations out of in-memory filtering into SQL-backed queries.
Closes #2913
Validation steps
mix formatmix test test/lightning/accounts_test.exs test/lightning/projects_test.exs test/lightning_web/live/user_live_test.exs test/lightning_web/live/project_live_test.exs --seed 0mix test test/lightning_web/live/audit_live_test.exs --seed 0Additional notes for the reviewer
20260226162000_add_admin_search_trgm_indexes.exs.page,filter,sort,dir) for both users and projects.AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
Pre-submission checklist
/reviewwith Claude Code):owner,:admin,:editor,:viewer)