Skip to content

feat(source): compile a OneDrive path into the local wiki - #279

Open
kalyan-fireweave wants to merge 3 commits into
VectifyAI:mainfrom
kalyan-fireweave:feat/onedrive-remote-source
Open

kalyan-fireweave wants to merge 3 commits into
VectifyAI:mainfrom
kalyan-fireweave:feat/onedrive-remote-source

Conversation

@kalyan-fireweave

Copy link
Copy Markdown

Closes #276.

This branch includes the unmerged S3 work from #277 (commit 6f8d0be) and the unmerged Drive work from #278 (commit 2ebeb18). Merge those first. The OneDrive-only commit is 70065fb. This is separate from #61, which asks to store the wiki itself on S3-compatible storage.

A maintainer needs to add the connector:onedrive label so the Workbench vote search for unshipped providers can find this issue. This account cannot create labels on VectifyAI/OpenKB.

Summary

openkb source add onedrive --path Documents/Papers
  parse the path (no leading slash, no ..)
  access token (refresh via msal, or loopback sign-in)
    redirect http://localhost:7567/callback
    listener still bound to 127.0.0.1:7567
    refresh token -> <kb>/.env mode 0600
  store.py writes connectors[].path
openkb source sync
  Graph children, recursive
  download strips Authorization on a cross-host redirect
  add_single_file(identity=onedrive://{path from the drive root})

Problem

The Workbench shows OneDrive as a GitHub vote card. There is no way to bring a path from the user's own OneDrive into the local wiki through the existing compile path.

Fix

  • openkb source add onedrive --name NAME --path PATH. The path is from the drive root, with no leading slash (Documents/Papers). SharePoint sites are out of scope.
  • The Microsoft identity platform common endpoint covers personal and work or school accounts. Scope is Files.Read plus offline_access. Credentials are OPENKB_ONEDRIVE_CLIENT_ID, OPENKB_ONEDRIVE_CLIENT_SECRET, and OPENKB_ONEDRIVE_REFRESH_TOKEN. The refresh token is written by sign-in into <kb>/.env only. Secrets are not written to config.yaml, the wiki, or connector-state.json.
  • Sign-in reuses the Drive loopback listener. Microsoft's reply-URL rules accept http://localhost and ignore the port for that host, while the path must match, and the portal text box cannot add http://127.0.0.1. OpenKB therefore sends http://localhost:7567/callback and the README says to register http://localhost/callback on the Mobile and desktop applications platform. The socket stays on 127.0.0.1. --non-interactive never opens a browser.
  • A rejected refresh token runs that same sign-in once. Closing it fails every OneDrive binding and leaves other providers running. This backend does not import the Drive backend.
  • Sync lists the path recursively, downloads supported extensions, and compiles them with add_single_file. The registry key is onedrive://{path from the drive root}. Graph content downloads follow the redirect and drop the bearer token when the host changes. Transient failures retry 3 times.
  • msal==1.39.0 is the optional onedrive extra. Tests inject a fake Graph client and do not import it. CI's uv sync --locked --extra dev --extra api does not install it.
  • The API calls the same functions as the CLI, on a worker thread, with non_interactive=True. A Workbench save without a refresh token returns 400 and does not open a browser.
  • The OneDrive card flips from a vote link to the form or the list. S3 and Drive stay as shipped in feat(source): compile an Amazon S3 prefix into the local wiki #277 and feat(source): compile a Google Drive folder into the local wiki #278.

Evidence

  • Before: Settings and the Documents pane showed OneDrive as "Want it? Vote".
  • After: Settings, with no knowledge base selected, says "Open a knowledge base to connect OneDrive." There is no vote link. On a knowledge base, OneDrive shows Name, Path, and Save binding, plus the hint to run openkb source add onedrive once. Saving without a refresh token shows "Sign-in is required" and writes no binding. With a binding present, the card lists papers, onedrive://Documents/Papers/, "Not synced since this change", Sync, Remove, and Add another. Sync without a token records that sign-in error on the card. Remove clears the binding. S3 and Drive forms stay on the same page.

Verification

uv run ruff check .
uv run ruff format --check .
uv run mypy openkb
uv run pytest -q

Results here: ruff check passed, ruff format check passed, mypy reported no issues in 70 files, pytest 1293 passed.

cd frontend && npm run build

The web build passed (i18n:check, tsc -b, vite build).

Test plan

  • uv run ruff check .
  • uv run ruff format --check .
  • uv run mypy openkb
  • uv run pytest
  • cd frontend && npm ci && npm run build
  • Register http://localhost/callback on a Mobile and desktop applications client, set OPENKB_ONEDRIVE_CLIENT_ID and OPENKB_ONEDRIVE_CLIENT_SECRET in <kb>/.env, run openkb source add onedrive --path Documents/Papers, then openkb source sync
  • openkb source add onedrive --non-interactive with no refresh token fails before writing the binding
  • Confirm openkb source remove leaves compiled pages in the wiki
  • On a knowledge-base page, confirm the OneDrive form and the binding list, and that S3 and Drive still connect
  • Add the connector:onedrive label
  • Merge only after feat(source): compile an Amazon S3 prefix into the local wiki #277 and feat(source): compile a Google Drive folder into the local wiki #278

Merge Danger

Door: two-way

The command, the router, and the card can be reverted. A binding already saved under connectors: with provider: onedrive is ignored by a build that does not read it. Pages already compiled stay in the wiki, which is the same result as openkb source remove. The refresh token in <kb>/.env is left in place on revert.

Blast Radius: ingest

Local openkb add is unchanged. msal is an extra, so the core install and CI's locked dev + api sync do not install it. OneDrive reuses oauth.py from #278 and does not import the Drive backend. S3 and Drive sync behavior is unchanged.

Made with Cursor

Kalyan Chowdary and others added 3 commits November 5, 2026 18:56
Bindings stay in config and credentials stay in env files, so a remote object can recompile through the existing add path without a new storage backend.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign-in stays on loopback and writes the refresh token only into the knowledge base env file, so a binding can be saved without putting a secret in the wiki.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign-in reuses the loopback listener and sends a localhost redirect, so a personal or work account can be saved without opening a browser from the Workbench.

Co-authored-by: Cursor <cursoragent@cursor.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remote data source: ingest a OneDrive folder into the local knowledge base

1 participant