Skip to content

fix(object-list): keep searching across all pages until exhausted - #215

Closed
ajax-bakun-n wants to merge 1 commit into
rustfs:mainfrom
ajax-bakun-n:fix/object-search-auto-paginate
Closed

ajax-bakun-n wants to merge 1 commit into
rustfs:mainfrom
ajax-bakun-n:fix/object-search-auto-paginate

Conversation

@ajax-bakun-n

@ajax-bakun-n ajax-bakun-n commented Aug 31, 2026

Copy link
Copy Markdown

Description

Object search only filtered the already-loaded page (up to 1000 objects) instead of continuing through the rest of the bucket, so a match further in was reported as not found.

This adds an automatic, debounced background pagination loop that keeps fetching subsequent pages while a search term is active, until the prefix is exhausted — with a stop/resume control so the user can cut a long scan short.

Type of Change

  • New feature (auto-continue search + stop/resume control)

Testing

  • Unit tests added/updated
  • Manual testing completed (verified against a live rustfs deployment)
pnpm test:run

Checklist

  • Code follows the project's style guidelines
  • Self-review completed
  • TypeScript types are properly defined
  • All commit messages are in English (Conventional Commits)
  • All existing tests pass
  • No new dependencies added

Related Issues

Closes rustfs/rustfs#2093

Additional Notes

@cxymds

cxymds commented Sep 1, 2026

Copy link
Copy Markdown
Member

Thanks for working on this. The user problem is valid, but I don’t think automatically fetching every continuation page in the browser is a safe solution, so I can’t approve this as-is.

The main blockers are:


  1. The search is unbounded. Any non-empty term can enumerate the entire current prefix and retain every object in React state. This can generate many S3 requests and freeze or crash the Console for large buckets, which conflicts with the bounded-loading approach established in feat: improve object browser lazy loading #152 and fix: clarify partial object sorting #206.
    
  2. “Stop search” does not reliably stop pagination. The existing IntersectionObserver still calls loadNextBatch() without checking autoSearchStopped, so scrolling or layout changes can continue the scan and bypass the debounce.
    
  3. An appended-page failure can cause an automatic retry loop. nextToken is retained and the auto-search effect does not check loadMoreError, so it retries again after loading becomes false instead of waiting for the user to click Retry.
    
  4. A stopped search can incorrectly display “No matching objects” even when nextToken shows that more objects remain unsearched. This regresses the partial-result state introduced in fix: distinguish filtered object empty states #184.
    
    This also does not fully resolve Expand Search/Sort functionality from console rustfs#2093: Delimiter: "/" means it does not recursively search the whole bucket, and global sorting is still not implemented.
    
    Thank you for taking the time to investigate this problem and submit the PR. We agree that improving object discovery in the Console would be valuable.
    
    However, automatically enumerating all continuation pages and retaining the full listing in the browser is not safe for large buckets and conflicts with the Console’s bounded-loading design. Since this concern is fundamental to the proposed approach rather than a small implementation issue, we are going to close this PR.
    
    If you have another approach that keeps requests and client-side memory bounded—such as a clearly scoped prefix search, a paginated backend capability, or another design that avoids scanning the entire bucket in the browser—we would be happy to discuss it. Please feel free to propose the design in an issue before implementing it, or submit a new PR if the approach has already been agreed upon.
    
    Thanks again for the contribution and for helping improve RustFS.

@cxymds cxymds closed this Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expand Search/Sort functionality from console

2 participants