Skip to content

Conversation

@jamesmcroft
Copy link
Contributor

Update to the existing NotionDBLoader to reduce duplicate calls to retrieve the metadata information.

When calling out to retrieve the database objects, the property metadata is already returned as part of the request. This change removes the secondary call out, speeding up the process of loading documents from Notion.

@rlancemartin @eyurtsev

@vercel
Copy link

vercel bot commented Jun 25, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
langchain ⬜️ Ignored (Inspect) Jun 26, 2023 7:46am

Copy link
Contributor

@rlancemartin rlancemartin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Please run make format to resolve Lint errors.

Also, worth re-running the notebook to confirm?

AFAIK, we don't have a unit test for Notion loader so notebook is probably sufficient.

@jamesmcroft
Copy link
Contributor Author

I appreciate you sorting the linting @dev2049!

@jamesmcroft
Copy link
Contributor Author

jamesmcroft commented Jun 26, 2023

Notion DB document load test on master using a database with over 250 pages (request timed out after about 10 mins):

Screenshot 2023-06-26 100838

This update change with the same dataset:

Screenshot 2023-06-26 114047

@rlancemartin
Copy link
Contributor

Notion DB document load test on master using a database with over 250 pages (request timed out after about 10 mins):

Screenshot 2023-06-26 100838

This update change with the same dataset:

Screenshot 2023-06-26 114047

Nice work. Will merge now.

@rlancemartin rlancemartin merged commit ba62276 into langchain-ai:master Jun 26, 2023
@jamesmcroft jamesmcroft deleted the feature/notion-performance branch June 26, 2023 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants