core: add **kwargs to index and aindex functions for custom vector_field support #26998
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
dosubot
bot
added
size:XS
Jalmeida1994
changed the title
| @@ -198,6 +198,7 @@ def index( | |||
| source_id_key: Union[str, Callable[[Document], str], None] = None, | |||
| cleanup_batch_size: int = 1_000, | |||
| force_update: bool = False, | |||
| **kwargs: Any, | |||
There was a problem hiding this comment.
Don't use ** here to avoid confusion of which namespace kwargs belong (e.g., what if we need kwargs for delete at some point or else add other parameters to index in the future).
Instead can call is upsert_kwargs. Please document which API whey will be passed to in the doc-string
There was a problem hiding this comment.
Thank you for the feedback. I've implemented the changes as suggested:
- Replaced
**kwargswith a specificupsert_kwargs: Optional[dict[str, Any]] = Noneparameter. - Updated the docstring to clarify that these kwargs will be passed to the
add_documentsandaadd_documentsmethod of the VectorStore. - Added an example in the docstring to demonstrate how
upsert_kwargscan be used, specifically mentioning thevector_fielduse case.
There was a problem hiding this comment.
Also added some unit tests for index and aindex with upsert_kwargs.
dosubot
bot
added
size:L
| docs_to_index, | ||
| ids=uids, | ||
| batch_size=batch_size, | ||
| **(upsert_kwargs or {}), | ||
| ) | ||
| elif isinstance(destination, DocumentIndex): | ||
| destination.upsert(docs_to_index) |
There was a problem hiding this comment.
They also need to be added here -- I'll push a change in a bit
| """Test async indexing with upsert_kwargs parameter.""" | ||
| mock_aadd_documents = AsyncMock() | ||
|
|
||
| with patch.object(upserting_vector_store, "aadd_documents", mock_aadd_documents): |
There was a problem hiding this comment.
I find usage of spy here to be a bit easier on readability
There was a problem hiding this comment.
Thanks for the feedback!
dosubot
bot
added
the
lgtm
Added
**kwargsparameters to theindexandaindexfunctions inlibs/core/langchain_core/indexing/api.py. This allows users to pass additional arguments to theadd_documentsandaadd_documentsmethods, enabling the specification of a customvector_field. For example, users can now usevector_field="embedding"when indexing documents inOpenSearchVectorStore