Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): vector store indexing status #788

Closed
CollectiveUnicorn opened this issue Jul 16, 2024 · 0 comments · Fixed by #830
Closed

feat(api): vector store indexing status #788

CollectiveUnicorn opened this issue Jul 16, 2024 · 0 comments · Fixed by #830
Assignees
Labels
enhancement New feature or request

Comments

@CollectiveUnicorn
Copy link
Contributor

CollectiveUnicorn commented Jul 16, 2024

User Story

As the LeapfrogAI UI
I want to get a status update for the indexing of my files
So that I can notify the user that their docs can be chatted with

Acceptance Criteria

Given I subscribe to the vectorStoreFileStatus table
When a file is being indexed, has been successfully indexed, or failed to index
Then I can get that status via Supabase Realtime on the frontend

Additional context

The API does not need to worry about pub/sub (realtime) functionality, it only needs to create a 'vectorStoreFileStatus' table and keep it up to date with vector_store_file's ids and status. When a file is attached to the vector store and sent to the vector stores endpoint, update the status to 'processing'. When complete, change to 'complete', if there is an error, update it to 'error'.

ex.:

{
   "id": "vector_store_file_123",
   "status": "processing"
}
CREATE TABLE vector_store_file_status (
    id VARCHAR PRIMARY KEY,
    status VARCHAR CHECK (status IN ('processing', 'complete', 'error'))
);

Row Level Security (RLS) should also be setup on the table so that users can only reference their own file uploads. This may required storing the user_id on the row as well.

FYI, The frontend will subscribe to the appropriate table and handle updates in realtime with something like this (issue #671):
ex.

supabase
  .channel('vectorStoreFileStatus')
  .on('postgres_changes', { event: '*', schema: 'public', table: 'todos',  filter: 'id=in.(1, 2, 3)',}, handleUpdate)
  .subscribe()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants