Skip to content

Improve tree search efficiency#7941 - #8016

Open
acwhite211 wants to merge 5 commits into
mainfrom
issue-7752
Open

Improve tree search efficiency#7941#8016
acwhite211 wants to merge 5 commits into
mainfrom
issue-7752

Conversation

@acwhite211

Copy link
Copy Markdown
Contributor

Fixes #7752
Contributed by @foozleface
Based on #7941 with fixes added.

Tree search in QueryComboBox sends a LIKE query per keystroke. On tree tables with 200K+ rows, using "contains" mode generates LIKE '%pattern%' which cannot use B-tree indexes and causes full table scans. This PR changes the default search mode from "contains" to "startsWith", enabling LIKE 'pattern%' which uses B-tree indexes. It also reduces the search result limit from 1000 to 50, since a typeahead dropdown never needs that many results.

Implementation

  • Change the default value of treeSearchAlgorithm user preference from contains to startsWith in UserDefinitions.tsx
  • Export QUERY_COMBO_BOX_SEARCH_LIMIT constant set to 50 (was 1000)
  • Add tests verifying the default search operator and result limit

Testing instructions

  • Open a form with a tree-based QueryComboBox (e.g., Taxon field on a Determination)
  • Type a few characters and verify the typeahead dropdown populates quickly
  • Verify that typing "rosa" matches "Rosaceae" and "Rosales" but not "Pterosaurus" (startsWith behavior)
  • Check user preferences -- the tree search algorithm should now default to "starts with"
  • Run the frontend tests: npx jest --testPathPattern treeSearchEfficiency

foozleface and others added 3 commits April 10, 2026 09:23
Change default tree search algorithm from 'contains' (LIKE '%x%') to
'startsWith' (LIKE 'x%'), enabling B-tree index usage on 200K+ row
tree tables.

Replace hardcoded limit of 1000 with paginated batch-on-scroll loading:
- Initial fetch returns first 50 results (QUERY_COMBO_BOX_PAGE_SIZE)
- Scrolling near the bottom of the dropdown fetches the next 50
- Loading indicator shown while fetching more results
- Continues until all results are loaded

Changes:
- UserDefinitions.tsx: default treeSearchAlgorithm 'contains' -> 'startsWith'
- QueryComboBox/index.tsx: paginated fetchSource with handleScrollEnd
- AutoComplete.tsx: onScrollEnd, isLoadingMore, extraItems props
- treeSearchEfficiency.test.tsx: verify default operator and page size
@github-project-automation github-project-automation Bot moved this to 📋Back Log in General Tester Board Apr 23, 2026
@acwhite211
acwhite211 requested review from a team April 23, 2026 16:52
@acwhite211 acwhite211 added this to the 7.12.2 milestone Apr 23, 2026
@acwhite211 acwhite211 mentioned this pull request Apr 23, 2026
5 tasks

@emenslin emenslin left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Open a form with a tree-based QueryComboBox (e.g., Taxon field on a Determination)
  • Type a few characters and verify the typeahead dropdown populates quickly
  • Verify that typing "rosa" matches "Rosaceae" and "Rosales" but not "Pterosaurus" (startsWith behavior)
  • Check user preferences -- the tree search algorithm should now default to "starts with"

Switching the tree search algorithm to "contains" doesn't seem to work.

Current search behavior in main:

04-27_09.29.mp4

Issue branch:

04-27_09.28.mp4

@github-project-automation github-project-automation Bot moved this from 📋Back Log to Dev Attention Needed in General Tester Board Apr 27, 2026
@JDAM2k4
JDAM2k4 self-requested a review August 28, 2026 14:53
@rijulpoudel
rijulpoudel self-requested a review August 28, 2026 14:59

@JDAM2k4 JDAM2k4 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Open a form with a tree-based QueryComboBox (e.g., Taxon field on a Determination)
  • Type a few characters and verify the typeahead dropdown populates quickly
  • Verify that typing "rosa" matches "Rosaceae" and "Rosales" but not "Pterosaurus" (startsWith behavior)
  • Check user preferences -- the tree search algorithm should now default to "starts with"

I am having the same issue as other reviewers. It seems that no matter the selection in User Preferences, the autofill search only returns searches in the form of "starts with - case insensitive."

20260828100208.mp4

@rijulpoudel rijulpoudel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Open a form with a tree-based QueryComboBox (e.g., Taxon field on a Determination)
  • Type a few characters and verify the typeahead dropdown populates quickly
  • Verify that typing "rosa" matches "Rosaceae" and "Rosales" but not "Pterosaurus" (startsWith behavior)
  • Check user preferences -- the tree search algorithm should now default to "starts with"

The testing instructions pass but both case sensitivity and the contains options look to be broken.
Image

Image

@CarolineDenis CarolineDenis modified the milestones: 7.12.2, 7.x Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Dev Attention Needed

Development

Successfully merging this pull request may close these issues.

[Large Databases]: Make tree searches by name more efficient

6 participants