← Docgrity for VS Code home

How Docgrity for VS Code works

Last updated: 5 September 2026

The pipeline

1. Collect. Docgrity gathers your repo's markdown files (**/*.md by default, capped at 200 files). node_modules, build output, and vendor directories are always excluded, and anything in your .gitignore is skipped too. Trivial files are dropped.
2. Pair locally. Candidate document pairs are selected with local TF-IDF cosine similarity — deterministic, instant, and free. No AI is used at this stage; code blocks are ignored so shared snippets don't cause false pairing.
3. Assess with your model. Only the top pairs (25 by default) are sent to the language model — via VS Code's built-in vscode.lm API, meaning your own Copilot subscription or a local model. When both pairwise checks are enabled, duplication and contradiction are assessed in a single combined call per pair; open-question checks run only on documents that show open-question signals (TODO/TBD, placeholders, unanswered questions). Assessments run four at a time.
4. Validate everything. Model responses must be valid JSON matching a strict schema — anything else is rejected, never partially trusted. Every evidence excerpt is verified verbatim against your files; findings with unverifiable evidence are dropped (hallucination guard).
5. Show findings. Findings appear in the Docgrity activity-bar view and as diagnostics (squiggles) on the exact evidence text — the scan notification's View findings button takes you straight there. Docgrity: Open findings report generates a shareable markdown report of the whole scan. Each finding records the model and prompt version that produced it.
6. Act — with approval. Right-click a finding → Raise GitHub issue. Docgrity drafts a title and body (evidence, suggested next step, potential owner from git log) and shows it to you in a modal. Only after you click Create issue does it post, using VS Code's built-in GitHub sign-in, labelled docgrity / docgrity:<type>.

Modes: report-only vs report + issues

The docgrity.mode setting controls what the extension is allowed to do:

• report-and-issue (default) — scan, review findings, and raise GitHub issues, each previewed and human-approved before posting.
• report-only — scan and review only. The Raise GitHub issue action is hidden from menus and blocked at the command level, so the extension is guaranteed to never post anywhere. Useful for client repositories, compliance-sensitive environments, or exploratory reading.

Because it's a normal VS Code setting, you can set report-only globally in user settings and override to report-and-issue per-workspace (in .vscode/settings.json) only in repos where you want to act.

Choosing which checks run

Each check is an independent toggle — run any combination, with any model, in either mode:

• docgrity.checks.duplicates — pairwise duplicate detection (default on)
• docgrity.checks.contradictions — pairwise contradiction detection (default on)
• docgrity.checks.openQuestions — per-document unresolved-question detection (default on)

Disabling checks also speeds up scans: pair selection is skipped entirely when both pairwise checks are off, and only enabled checks cost model calls.

Viewing results

• Docgrity view — the Docgrity icon in the activity bar. Findings are grouped by type; clicking a finding jumps to its first evidence excerpt in the file.
• Problems panel — every finding is also a diagnostic on the affected files.
• Dashboard — run Docgrity: Open findings dashboard (or click Open report on the scan notification) for an interactive report: stat widgets per finding type, severity filters, expandable rows with evidence quotes, potential owners, and doc paths that open the file in the editor.
• Markdown report — run Docgrity: Open findings report (markdown) for a plain-text report — summary table, severity, confidence, files, potential owners, and evidence quotes — that you can save or share.

Potential owners

Docgrity suggests who might fix a finding by looking at the last git author of each involved file. This is always labelled potential — inferred, never asserted. It never assigns issues automatically.

Choosing your model — Copilot, Claude, GPT, or local llama

Docgrity talks to whatever model VS Code's language-model API (vscode.lm) exposes. The easiest way to switch: run Docgrity: Select AI model from the command palette — it lists every available model and saves your choice. Manual settings: docgrity.model.vendor (default copilot) and docgrity.model.family (e.g. gpt-4o, claude-sonnet-4.5, llama3.1; empty = first available).

Options, in order of simplicity:

1. GitHub Copilot (default). Sign in to Copilot — nothing to configure. Uses whichever Copilot model you have access to.
2. Claude / GPT / Gemini via Copilot. Any model enabled in Copilot's model picker is available to Docgrity — pick it with Select AI model or set docgrity.model.family.
3. Local Ollama — fully private. Install Ollama, pull a model (ollama pull llama3.1), then in Copilot Chat → Manage models → add the Ollama model. It registers under the copilot vendor; select it with Select AI model. With a local model, no document content leaves your machine at all.
4. Remote Ollama over a Cloudflare Tunnel. If your model runs on another machine (home server, GPU rig), expose it with:

   cloudflared tunnel --url http://localhost:11434

   and point Copilot's Manage models → Ollama endpoint at the generated https://….trycloudflare.com URL.
   Caveats: quick tunnels get a new URL on every restart — you must update the endpoint each time. For a stable URL, create a named tunnel on your own domain (cloudflared tunnel create docgrity-llm + a DNS route), and protect it with Cloudflare Access or an auth proxy — an unauthenticated LLM endpoint on the public internet can be abused by anyone who finds it.
5. Other provider extensions. Any VS Code extension that registers language models works: set docgrity.model.vendor to that provider's vendor id, or empty for "any available model".

Models outside the editor — CLI and GitHub Action

Copilot models are only available inside VS Code (there is no Copilot API key for terminals or CI). The CLI and Action use direct providers instead:

• ollama (CLI default) — local or tunnelled, no key, fully private.
• anthropic / openai / gemini — bring your own API key (defaults: claude-3-5-haiku-latest, gpt-4o-mini, gemini-flash). These are the vendors' direct APIs — a separate key from your Copilot subscription. In the Action, pass it as api_key: ${{ secrets.DOCGRITY_API_KEY }}.
• github-models — being retired by GitHub. If your Action runs fail with HTTP 410 (github_models_retirement), switch provider to one of the options above.

Summary: Copilot (keyless) in the editor · Ollama (keyless) everywhere · BYO key for hosted models in CI and the CLI.

No-agent mode — no AI at all

Every surface can also scan with pure algorithms — no model, no Copilot, no keys, fully offline. Extension: set docgrity.engine to no-agent. CLI: docgrity scan --provider none. Action: provider: none.

• Duplicates — verbatim shared-block detection plus TF-IDF similarity. Catches copy-paste duplication with measured confidence (the actual overlap ratio); paraphrased duplication needs AI.
• Open questions — explicit markers (TODO, TBD, FIXME, ???, "open question"…). Deterministic, so confidence is high.
• Contradictions — require AI intelligence: deciding that two documents make conflicting claims is semantic understanding, not pattern matching. In no-agent mode this check is skipped and clearly noted in the report — Docgrity never substitutes noisy heuristic guesses.

Evidence stays verbatim (extracted directly from the docs), owners still come from git history, and findings are marked method: heuristic so you always know how they were produced.

Model quality note: Docgrity's prompts demand strict JSON with verbatim evidence. Hosted frontier models handle this reliably; small local models (<8B) fail validation more often. Failed responses are rejected safely — you get fewer findings, never corrupted ones. Recommended local minimum: an 8B+ instruct model (e.g. llama3.1:8b); 70B-class or hosted models give noticeably better contradiction detection.

Cost & performance

• Pair selection is 100% local — the model is only called for the top pairs and per-doc open-question checks.
• Duplicate + contradiction share one model call per pair; documents with no open-question signals are pre-filtered before any model call; assessments run with bounded concurrency (4 at a time).
• Caps (docgrity.maxFiles, docgrity.maxPairs) keep a scan to a bounded number of model calls, and disabling checks you don't need (docgrity.checks.*) cuts calls further.
• Scans are cancellable; partial results are kept.

Security posture

• Document content is treated as untrusted input: it cannot override the agent instructions, and all model output is schema-validated in code.
• Evidence is verified verbatim against source files before a finding is recorded.
• GitHub tokens come from VS Code's authentication API, are used only for the issue-creation request you approved, and are never logged or stored.
• Git commands are read-only (git log, git remote get-url), executed without a shell.
• No telemetry, no vendor servers, no data collection. See the privacy policy.

Troubleshooting

• “No language model available” — sign in to GitHub Copilot, or configure docgrity.model.vendor/family for a local model.
• No findings — your docs may simply agree with each other; try lowering docgrity.thresholds.*, or check the corpus caps.
• Clicking a finding does nothing — update to the latest version; findings and evidence rows open the exact excerpt in the file. Results also appear in the Problems panel and via Docgrity: Open findings report.
• Scanning unwanted files — node_modules, build output, and git-ignored files are always excluded (v0.1.2+); add extra patterns with docgrity.exclude.
• Logs — Output panel → Docgrity channel. Use Developer: Set Log Level for more detail. Logs never contain document content or tokens.

← Back to home