deer-flow/.github/copilot-instructions.md

Copilot Onboarding Instructions for DeerFlow

Use this file as the default operating guide for this repository. Follow it first, and only search the codebase when this file is incomplete or incorrect.

1) Repository Summary

DeerFlow is a full-stack "super agent harness".

• Backend: Python 3.12, LangGraph + FastAPI gateway, sandbox/tool system, memory, MCP integration.
• Frontend: Next.js 16 + React 19 + TypeScript + pnpm.
• Local dev entrypoint: root Makefile starts backend + frontend + nginx on http://localhost:2026.
• Docker dev entrypoint: make docker-* (mode-aware provisioner startup from config.yaml).

Current repo footprint is medium-large (backend service, frontend app, docker stack, skills library, docs).

2) Runtime and Toolchain Requirements

Validated in this repo on macOS:

• Node.js >=22 (validated with Node 23.11.0)
• pnpm (repo expects lockfile generated by pnpm 10; validated with pnpm 10.26.2 and 10.15.0)
• Python >=3.12 (CI uses 3.12)
• uv (validated with 0.7.20)
• nginx (required for make dev unified local endpoint)

Always run from repo root unless a command explicitly says otherwise.

3) Build/Test/Lint/Run - Verified Command Sequences

These were executed and validated in this repository.

A. Bootstrap and install

1. Check prerequisites:

make check

Observed: passes when required tools are installed.

2. Install dependencies (recommended order: backend then frontend, as implemented by make install):

make install

B. Backend CI-equivalent validation

Run from backend/:

make lint
make test

Validated results:

• make lint: pass (ruff check .)
• make test: pass (277 passed, 15 warnings in ~76.6s)

CI parity:

• .github/workflows/backend-unit-tests.yml runs on pull requests.
• CI executes uv sync --group dev, then make lint, then make test in backend/.

C. Frontend validation

Run from frontend/.

Recommended reliable sequence:

pnpm lint
pnpm typecheck
BETTER_AUTH_SECRET=local-dev-secret pnpm build

Observed failure modes and workarounds:

• pnpm build fails without BETTER_AUTH_SECRET in production-mode env validation.
• Workaround: set BETTER_AUTH_SECRET (best) or set SKIP_ENV_VALIDATION=1.
• Even with SKIP_ENV_VALIDATION=1, Better Auth can still warn/error in logs about default secret; prefer setting a real non-default secret.
• pnpm check currently fails (next lint invocation is incompatible here and resolves to an invalid directory). Do not rely on pnpm check; run pnpm lint and pnpm typecheck explicitly.

D. Run locally (all services)

From root:

make dev

Behavior:

• Stops existing local services first.
• Starts LangGraph (2024), Gateway (8001), Frontend (3000), nginx (2026).
• Unified app endpoint: http://localhost:2026.
• Logs: logs/langgraph.log, logs/gateway.log, logs/frontend.log, logs/nginx.log.

Stop services:

make stop

If tool sessions/timeouts interrupt make dev, run make stop again to ensure cleanup.

E. Config bootstrap

From root:

make config

Important behavior:

• This intentionally aborts if config.yaml (or config.yml/configure.yml) already exists.
• Use make config only for first-time setup in a clean clone.

4) Command Order That Minimizes Failures

Use this exact order for local code changes:

1. make check
2. make install (if frontend fails with proxy errors, rerun frontend install with proxy vars unset)
3. Backend checks: cd backend && make lint && make test
4. Frontend checks: cd frontend && pnpm lint && pnpm typecheck
5. Frontend build (if UI changes or release-sensitive changes): BETTER_AUTH_SECRET=... pnpm build

Always run backend lint/tests before opening PRs because that is what CI enforces.

5) Project Layout and Architecture (High-Value Paths)

Root-level orchestration and config:

• Makefile - main local/dev/docker command entrypoints
• config.example.yaml - primary app config template
• config.yaml - local active config (gitignored)
• docker/docker-compose-dev.yaml - Docker dev topology
• .github/workflows/backend-unit-tests.yml - PR validation workflow

Backend core:

• backend/packages/harness/deerflow/agents/ - lead agent, middleware chain, memory
• backend/app/gateway/ - FastAPI gateway API
• backend/packages/harness/deerflow/sandbox/ - sandbox provider + tool wrappers
• backend/packages/harness/deerflow/subagents/ - subagent registry/execution
• backend/packages/harness/deerflow/mcp/ - MCP integration
• backend/langgraph.json - graph entrypoint (deerflow.agents:make_lead_agent)
• backend/pyproject.toml - Python deps and requires-python
• backend/ruff.toml - lint/format policy
• backend/tests/ - backend unit and integration-like tests

Frontend core:

• frontend/src/app/ - Next.js routes/pages
• frontend/src/components/ - UI components
• frontend/src/core/ - app logic (threads, tools, API, models)
• frontend/src/env.js - env schema/validation (critical for build behavior)
• frontend/package.json - scripts/deps
• frontend/eslint.config.js - lint rules
• frontend/tsconfig.json - TS config

Skills and assets:

• skills/public/ - built-in skill packs loaded by agent runtime

6) Pre-Checkin / Validation Expectations

Before submitting changes, run at minimum:

• Backend: cd backend && make lint && make test
• Frontend (if touched): cd frontend && pnpm lint && pnpm typecheck
• Frontend build when changing env/auth/routing/build-sensitive files: BETTER_AUTH_SECRET=... pnpm build

If touching orchestration/config (Makefile, docker/*, config*.yaml), also run make dev and verify the four services start.

7) Non-Obvious Dependencies and Gotchas

• Proxy env vars can silently break frontend network operations (pnpm install/registry access).
• BETTER_AUTH_SECRET is effectively required for reliable frontend production build validation.
• Next.js may warn about multiple lockfiles and workspace root inference; this is currently a warning, not a build blocker.
• make config is non-idempotent by design when config already exists.
• make dev includes process cleanup and can emit shutdown logs/noise if interrupted; this is expected.

8) Root Inventory (quick reference)

Important root entries:

• .github/
• backend/
• frontend/
• docker/
• skills/
• scripts/
• docs/
• README.md
• CONTRIBUTING.md
• Makefile
• config.example.yaml
• extensions_config.example.json

9) Instruction Priority

Trust this onboarding guide first.

Only do broad repo searches (grep/find/code search) when:

• you need file-level implementation details not listed here,
• a command here fails and you need updated replacement behavior,
• or CI/workflow definitions have changed since this file was written.