OpenWorker

What is this?

OpenWorker is an open-source AI coworker that lives on your desktop and delivers finished work, not just chat: your code reviewed for vulnerabilities with fixes ready to go, a polished document, a Slack reply with the numbers, a triaged inbox.

Built by Andrew Ng's team and released under the MIT license, it runs entirely on your machine. You bring your own model key — OpenAI, Anthropic, Google, DeepSeek, or a fully local Ollama model — and your data only leaves your machine through the model and integrations you choose.

What problem does it solve?

AI chatbots give you text. Real work needs execution:

Chatbot:  You → Question → Text answer → You still do the work
OpenWorker: You → Outcome wanted → Finished deliverable

A single chat window cannot:

• Scan your actual codebase for vulnerabilities and prepare fixes
• Read your inbox, calendar, Slack, and Jira to prep a customer call
• Run on a schedule (morning briefs, weekly reports, channel watches)
• Do any of this safely — with approvals, audit logs, and sandboxing

OpenWorker closes that gap: it works across your desktop, files, and connected apps, checks in before anything consequential, and hands you the finished deliverable.

How does it work?

Ask for an outcome
("prepare a customer brief")
        ↓
Task broken into steps
        ↓
Works across your files, terminal, and 25+ connectors
        ↓
Before consequential actions
(send, change calendar, run command)
        ↓
  ┌───────────────┬──────────────────┐
  │               │                  │
You approve    Auto-approve       Sandboxed
& redirect     (reviewer model     (NVIDIA
               escalates          OpenShell
               anything unsure)   container)
  └───────────────┴──────────────────┘
        ↓
Finished deliverable
(document, diff, report, reply)

Under the hood it is three layers:

┌──────────────────────────────────┐
│   OpenWorker desktop app         │  native shell + GUI
├──────────────────────────────────┤
│  local agent server (Python)     │  engine · tools · connectors
│                                  │  built on aisuite
├──────────────────────────────────┤
│ your files · your tools · your   │  everything runs with your
│ & terminal · 25+ connectors      │  keys, on your machine
└──────────────────────────────────┘

Repository Structure

andrewyng/openworker/
├── coworker/              # Python backend (the engine)
│   ├── agents/            #   agent definitions & loop
│   ├── connectors/        #   25+ integrations (GitHub, Slack, Jira...)
│   ├── providers/         #   model providers via aisuite
│   ├── tools/             #   tools the agent can call
│   ├── sandbox/           #   NVIDIA OpenShell sandbox integration
│   ├── automation/        #   scheduled/standing automations
│   ├── memory/            #   session memory
│   ├── mcp/               #   Model Context Protocol client
│   ├── skills/            #   reusable skill packs
│   └── server/            #   local HTTP agent server
├── surfaces/gui/          # Desktop app — React UI + Tauri shell
├── stt/                   # Speech-to-text sidecar (Rust)
├── packaging/             # macOS DMG / Windows installers, auto-update
├── docs/                  # Design specs and decision logs
├── tests/                 # Backend test suite
└── pyproject.toml         # Python package config

Key Features

🔐 Governed by Design — Governance is the architecture, not a plugin. Four tiers: hard floors (dangerous ops are always human-only), a ladder of earned autonomy (approvals graduate into standing rules then allowlists), a full audit trail answering "who did this, and why?", and a sandbox for what the agent runs. The agent can't grant itself permissions and no prompt can talk it past a gate.

🛡️ Security Review Coworkers — Scan a codebase and its dependencies for real risk. Findings come from deterministic scanners (like semgrep) plus model reasoning; proposed fixes are re-scanned and diff-reviewed before you approve them — the fixer is never the only checker.

🔌 25+ Integrations — GitHub, Slack, Jira, Notion, Linear, HubSpot, Outlook, monday.com, Gmail, Google Calendar, plus your terminal and local files. Any MCP-reachable tool plugs in too, with per-tool control.

🧠 Bring Your Own Model — OpenAI, Anthropic, Google Gemini, DeepSeek, Kimi, Qwen, Mistral, Grok, GLM, and more — plus open-weight models via Together/Fireworks and fully local models via Ollama.

⏱️ Standing Automations — A morning brief, a weekly report, a standing watch over a channel — on a schedule, with full transcripts. Unattended runs never self-approve: their asks park in an inbox until a human answers.

📦 NVIDIA OpenShell Sandbox — A session's commands and file tools run inside one Linux container that holds the session's folders and nothing else: no keys, no other files, only allow-listed network hosts. A prompt injection that reaches a command lands in a box that holds nothing.

💬 Slack-Native — Mention @OpenWorker in a channel; a session opens on your desktop, the work happens with your tools, and the answer comes back as a thread reply.

What You'll Learn

• How to build a production desktop agent app with a Python engine and React/Tauri shell
• How to structure agent governance (approval gates, audit trails, sandboxing) so it can't be prompt-injected away
• How to unify multiple LLM providers behind one interface using aisuite
• How to connect an agent to real tools via connectors and MCP
• How to run scheduled automations safely with human-in-the-loop escalation
• How to sandbox agent commands so a compromised tool can't reach your secrets