Add search-only mode #29

danmichaeljones · 2025-08-20T15:51:00Z

This adds the new search-only mode to the Query Agent client:

Adds search method to QueryAgent to retrieve the first page of search results
The next() method on the response is used to paginate the next sets of results consistently (i.e., same results set)

orca-security-eu

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Secrets	0 0 0 0	View in Orca

tsmith023 · 2025-08-27T10:15:19Z

src/query/agent.ts

+   */
+  async search<T = undefined>(
+    query: string,
+    { limit, collections }: QueryAgentSearchOnlyOptions = {},


IMO, having this be options?: QueryAgentSearchOnlyOptions would be clearer for a user on the exact nature of this functional argument. Having the object keys explicit and it having a default of {} is quite different to the rest of the TS client so might be a point of confusion for users

This makes sense, but doesn't match the existing QueryAgent methods (which I don't watch to touch now). I'll CC @augustas1 to see what he thinks once he's back, but for now I'd prefer to be consistent with the existing run / stream methods.

I've modified the signature a little though, to try and make the default argument values clearer 👍

tsmith023 · 2025-08-27T10:18:13Z

src/query/search.ts

+   * @param options.limit - The maximum number of results to return. Defaults to 20 if not specified.
+   * @param options.offset - The offset to start from. If not specified, retrieval begins from the first object.


Optional args can be signified in JSdoc using [] like so: [options.limit]

Cool - done! 👍

tsmith023 · 2025-08-27T10:23:11Z

src/query/search.ts

+   * @param options.offset - The offset to start from. If not specified, retrieval begins from the first object.
+   * @returns A SearchModeResponse object containing the results, usage, and underlying searches performed.
+   */
+  async run(options: SearchExecutionOptions): Promise<SearchModeResponse<T>> {


Should a user be able to just call await search.run()? If so, options needs to be optional here: options?: SearchExecutionOptions

This is mostly a method the end user won't touch (they should be interacting with it via the next() method on the SearchModeResponse), but I've updated to have default values (following the same convention as above) so you can now call plain await search.run().

tsmith023 · 2025-08-27T10:23:34Z

src/query/search.ts

+    }
+    return {
+      ...mappedResponse,
+      next: async ({ limit: nextLimit = 20, offset: nextOffset = 0 } = {}) =>


The function args here are different to how they are in the run outer function, might that be confusing from a UX PoV? Can the same SearchExecutionOptions type be reused here?

Yeah, that's nicer - I've moved the SearchExecutionOptions around and they're reused here.

tsmith023 · 2025-08-27T10:26:24Z

src/query/response/response.ts

+  next: (options?: {
+    limit?: number;
+    offset?: number;
+  }) => Promise<SearchModeResponse<T>>;


Likewise here as above wrt to the SearchExecutionOptions type

Same as above, now uses SearchExecutionOptions 👍

augustas1 · 2025-09-02T11:57:13Z

src/query/response/api-response.ts

+  searches?: ApiSearchResult[];
+  usage: ApiUsage;
+  total_time: number;
+  search_results: WeaviateReturn<T>;


Sadly results also probably are returned from API as underscore_case and not camelCase, so we'd have to map it as well to make it developer friendly 😢

Argh, good catch! I've adding mapping from the api snake_case to camelCase 👍 I've also removed all the generics on the types to make this work (we can add them back later if we want, but they were for typing properties but our search results are potentially multi-collection, so maybe didn't make sense anyway 🤷 ).

The types of the search result objects are also an extension of the Weaviate types, to add a collection field (which was missing from the original type).

augustas1 · 2025-09-02T12:10:43Z

src/query/response/response.ts

@@ -260,3 +262,23 @@ export type StreamedTokens = {
  outputType: "streamedTokens";
  delta: string;
 };
+
+export type MappedSearchModeResponse<T> = {


this looks like internal type and everything in this file is exported to user (see index.ts file).
So Maybe move it to response-mapping.ts?

Makes sense 👍 Moving to response-mapping.ts would end up with circular dependencies, so I've just removed this type and am using Omit<SearchModeResponse, "next"> in it's place (since it's internal anyway)

This works as well :) fyi circular dependencies are supported in JS/TS (especially fine with for types)

augustas1 · 2025-09-02T12:13:23Z

src/query/search.ts

+    }
+    return {
+      ...mappedResponse,
+      next: async (options: SearchExecutionOptions = {}) => this.run(options),


In SearchModeResponse options are declared as required, so probably = {} shouldn't be here?
On that note, if it's required, we probably expect user to always pass offset on next, so maybe offset need to be required as well (in comparison to run)?

Even better, I see run is being used only internally and we explicitly pass offset: 0, so maybe we can just make offset required on whole SearchExecutionOptions?

The SearchExecutionOptions are also used on the SearchModeResponse.next method, so I don't want to make offset required there. I've removed the = {} default value though.

isn't the whole point of next that offset should be passed? Otherwise it'll return the first page again (returned on initial search)?
Just think that users might think next is smart and will magically return next page somehow and don't pass offset if we don't require it.

Just think that users might think next is smart and will magically return next page somehow and don't pass offset if we don't require it.

Yeah, that's true.. I've updated the SearchExecutionOptions so that offset is required (and left limit optional). That's not consistent with what's in the Python version, but we can discuss whether that should be changed 👍

augustas1 · 2025-09-04T06:56:01Z

src/query/response/response-mapping.ts

+
+  return {
+    properties: object.properties,
+    metadata: metadata,


nitpick can omit variable when key has the same name. metadata,

Nice catch - fixed.

Dan Jones added 9 commits August 19, 2025 13:03

Add new types for search only mode

d9c371d

Add search-only executor class

21fb90d

Add method to agent to build searcher

9d5c9d7

Lint

04d8150

Make searcher generic to support generic WeaviateReturn

a7f980c

Update tests

11cef97

Lint

019d72f

Tidy up

368bda9

Rename methods

62dd66d

orca-security-eu bot reviewed Aug 20, 2025

View reviewed changes

danmichaeljones requested review from cdpierse and tsmith023 August 20, 2025 15:56

danmichaeljones marked this pull request as ready for review August 20, 2025 15:56

danmichaeljones marked this pull request as draft August 21, 2025 14:37

Dan Jones added 2 commits August 22, 2025 16:28

Update to new search/next design

498a659

Lint

9749bf2

danmichaeljones marked this pull request as ready for review August 22, 2025 15:35

tsmith023 reviewed Aug 27, 2025

View reviewed changes

Address PR comments

f78c508

danmichaeljones requested a review from tsmith023 August 27, 2025 14:37

augustas1 reviewed Sep 2, 2025

View reviewed changes

Dan Jones added 7 commits September 3, 2025 15:08

Add mapping of search results

8a0204f

Remove MappedSearchModeResponse

6ae894d

Remove default options

b3f53a8

Remove todo comments

85c5702

Update docs

89eaebf

Lint

e181535

Update searcher docs

b6c195f

danmichaeljones requested a review from augustas1 September 3, 2025 15:04

augustas1 reviewed Sep 4, 2025

View reviewed changes

PR comments

f43b595

danmichaeljones requested a review from augustas1 September 4, 2025 11:23

augustas1 approved these changes Sep 4, 2025

View reviewed changes

danmichaeljones merged commit 820e581 into main Sep 4, 2025
5 checks passed

danmichaeljones deleted the search_only branch September 4, 2025 13:32

		* @param options.limit - The maximum number of results to return. Defaults to 20 if not specified.
		* @param options.offset - The offset to start from. If not specified, retrieval begins from the first object.

Add search-only mode #29

Add search-only mode #29

Uh oh!

Conversation

danmichaeljones commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orca-security-eu bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Orca Security Scan Summary

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tsmith023 Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danmichaeljones Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

augustas1 Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

danmichaeljones commented Aug 20, 2025 •

edited

Loading

orca-security-eu bot left a comment •

edited

Loading

tsmith023 Aug 27, 2025 •

edited

Loading

danmichaeljones Sep 3, 2025 •

edited

Loading

augustas1 Sep 4, 2025 •

edited

Loading