GitTinkerer — Comment-to-Code Automation¶

GitTinkerer implements a non-interactive, comment-triggered automation flow that turns a PR comment (or a comment submitted via a Svelte web UI) into code changes using the Codex CLI, then commits and pushes the result back to the target branch.

For a detailed setup definition with diagrams and runtime contracts, see architecture.md.

This repository contains the code that runs on a VPS and is triggered by a GitHub App webhook (and also supports a web UI trigger).

What this repo does¶

Accepts an instruction comment (from GitHub PR comments or from the web UI)
Syncs a single git repository workspace (clone / fetch / pull / checkout)
Runs Codex CLI to implement the instruction
Creates a commit and pushes it
Posts a reply back to the originating channel (PR or web UI)
Writes a full run audit bundle to artifacts/<timestamp>/agent-run/

Recent capabilities¶

Sentry observability across service and bash wrappers with run/repo/stage/user tags
Database-backed analytics with per-repo/user/day token usage, CSV export endpoints
Admin controls to pause/resume repos, retry failed runs, and monitor usage
Run history UX with runs list, run detail, and analytics dashboards (live polling)

Non-goals¶

No interactive CLI
No human-in-the-loop during execution
No PR creation (the agent pushes commits; review happens after execution)

Prerequisites¶

On the VPS that executes GitTinkerer:

Codex CLI installed and authenticated (other CLIs may be supported later)
PostgreSQL 16+ accessible to the service (set DATABASE_URL or the PG* env vars; the HTTP service will fail fast if the DB is unreachable)
A .env file present on the VPS with GitHub credentials available (see .env.example)
A GitHub App installed on the target org/repo with the issue_comment webhook enabled (see github-app-setup.md)
Provide GH_APP_ID and GH_APP_PRIVATE_KEY so the service can mint an installation token and pass it to runs as GITHUB_TOKEN (required by the CLI). The installation ID is read from webhook payloads or looked up per-repo for web mode. Set GH_APP_INSTALLATION_ID only if you need to force a specific installation or provide a fallback for custom/non-webhook flows. Alternatively, set GITHUB_TOKEN directly.
You can also drop the GitHub App private key file at service/private-key.pem; the service prefers this file and falls back to the env var.

Local webhook testing¶

For local debugging, run the service with a local .env (loaded via dotenv) and tunnel it to GitHub:

cd service
npm install
cp ../.env.example .env  # fill in your GitHub App values
npm start

Expose the service port (default 3000) with a tunnel such as ngrok http 3000 and configure the GitHub App webhook URL to the HTTPS tunnel (e.g., https://<ngrok>/api/github/webhook).

High-level flow¶

A user submits an instruction comment:
PR mode: a comment on a GitHub Pull Request
Web mode: a comment in the Svelte web UI (in web/)
In PR mode, a GitHub App webhook delivers the comment to the VPS service.
GitTinkerer:
Parses a payload (single repo per run)
Ensures a local workspace for that repo exists (clone if missing; otherwise fetch/pull)
Checks out the target ref derived from PR metadata/ref
Runs Codex CLI with a prompt that includes the comment
Computes and saves a diff for audit
Commits and pushes as the bot identity
Posts the agent reply back to the same channel (PR reply or web UI reply)
Review happens after execution by inspecting pushed commits and the run artifacts.

Bash runner layout¶

The automation is a non-interactive bash runner composed of small modules in lib/:

lib/args.sh: parses CLI arguments (payload file, timestamps, workspace flags)
lib/payload.sh: loads/validates payload JSON and exports env vars
lib/workspace.sh and lib/git.sh: manage repo checkout/branch sync
lib/codex.sh: builds the Codex prompt and runs the CLI
lib/artifacts.sh: creates and maintains the audit bundle
lib/reply/github.sh and lib/reply/web.sh: post replies back to PR or web UI
lib/log.sh, lib/metrics.sh, lib/sentry.sh, lib/env.sh: shared logging/metrics/telemetry/env helpers

A thin shim composes these modules for run --payload-file <path>; the HTTP service spawns it with the payload path.

Tech stack¶

Node.js 24 (service runtime; see service/package.json)
Fastify 5 with CORS/Helmet/RateLimit
TypeScript (ts-node/tsx for dev, tsc for build)
PostgreSQL 16+
Redis 7+
Vitest (unit/integration/coverage)

Payload schema (current implementation; single repo per run)¶

{
  "run_id": "2025-12-23T12:34:56Z-<random>",
  "source": "pr",
  "repo": {
    "clone_url": "https://github.com/OWNER/REPO.git",
    "full_name": "OWNER/REPO"
  },
  "comment_body": "Please rename Foo to Bar and update usages.",
  "comment_raw_body": "/tinker Please rename Foo to Bar and update usages.",
  "pr": {
    "number": 123,
    "base_ref": "main",
    "head_ref": "feature/branch-name",
    "head_sha": "abcdef123456...",
    "comment_id": 999999
  },
  "web": {
    "web_conversation_id": "conv_01J...",
    "user_id": "user_123"
  }
}

Notes:

source is either "pr" or "web".
comment_body and comment_raw_body are both required and kept for audit.
pr is required when source == "pr" and must include number, comment_id, head_ref, head_sha, and base_ref (all non-empty; numeric fields are integers). See service/src/types/payload.ts.
web is required when source == "web" and must include web_conversation_id and user_id. See service/src/types/payload.ts.
Validation enforces the presence of the matching metadata object based on source (superRefine). Helpers createPrPayload/createWebPayload parse and validate typed inputs. See service/src/types/payload.ts.
The target branch/ref should be derived from PR metadata/ref.

Flow diagrams¶

Compact (flowchart):

flowchart TD
  A["Webhook (GitHub) or Web UI"] --> B[Service: validate payload]
  B --> C[Service: persist run + enqueue]
  C --> D["Bash runner (lib/*.sh)"]
  D --> E[Codex CLI applies changes]
  E --> F[Git ops: commit/push]
  F --> G[Artifacts written]
  G --> H[Reply to PR or Web UI]

Detailed (sequence with alternate paths):

sequenceDiagram
  participant Webhook as GitHub Webhook
  participant WebUI as Web UI
  participant Service as Service (Fastify)
  participant Runner as Bash Runner (lib/*.sh)
  participant Git as Git Remote
  participant Store as Artifacts

  alt PR comment
    Webhook->>Service: issue_comment payload
    Service->>Service: validate payload (zod)
  else Web UI
    WebUI->>Service: POST /api/runs payload
    Service->>Service: validate payload (zod)
  end

  Service->>Service: persist run + cache
  Service->>Runner: spawn run --payload-file
  Runner->>Runner: parse args/env (lib/args.sh, env.sh)
  Runner->>Runner: load payload (payload.sh)
  Runner->>Runner: init artifacts (artifacts.sh)
  Runner->>Runner: sync workspace + checkout (workspace.sh, git.sh)
  Runner->>Runner: generate prompt + call Codex (codex.sh)
  Runner->>Git: commit + push
  Runner->>Store: write diff/prompt/reply/commit/summary
  Runner-->>Service: exit code + status
  Service-->>Webhook: reply comment (PR) or
  Service-->>WebUI: reply message (web)

Repo sync behavior¶

For the single target repo in the payload:

If the repo workspace does not exist locally: git clone
If it exists:
git fetch (and/or git pull depending on strategy)
git checkout the target derived ref
Ensure workspace is up to date before applying changes

Codex prompt contract¶

Part of the Codex prompt must include the instruction comment, plus this post-change reporting requirement:

After making changes, write a short rationale explaining:
- What was changed
- Why it was changed
- Any assumptions made

Do not include internal reasoning or deliberation.

The agent's response (including this rationale) must be:

Logged to the run artifacts
Posted back as a reply in the originating channel:
PR mode → reply comment in the PR
Web mode → reply message in the web UI conversation

Commit + push behavior¶

GitTinkerer should:

Commit changes with the identity:
Name: nntin-bot
Email: 48604375+nntin-bot@users.noreply.github.com
Push commits back to the target branch derived from PR metadata/ref

Run artifacts (audit bundle)¶

Each run writes artifacts under artifacts/<timestamp>/agent-run/ (timestamp defaults to UTC now or TIMESTAMP_ARG if provided). The folder is git-ignored and created by lib/artifacts.sh.

Files created at start:

agent-run/
├── prompt.txt            # empty until Codex prompt is written
├── pr_comment.txt        # populated from comment_raw_body (fallback to comment_body)
├── diff.patch            # empty placeholder
├── files_changed.json    # initialized to []
├── commit_sha.txt        # empty until a commit is produced
└── summary.md            # starts as "Status: running"

On failures, the runner ensures all files exist and rewrites summary.md with Status: failed, the failed stage, and a sanitized message (secrets redacted). See lib/artifacts.sh.

On success, the runner fills prompt.txt, diff.patch, files_changed.json, commit_sha.txt, and summary.md with the Codex prompt, patch, file list, pushed commit, and final reply respectively.

Service architecture (domain/usecases/infra)¶

Domain: entities for run state, repo references, and instructions with guarded transitions (pending → running → success/failed/ignored), tracking timestamps and exit codes.
Usecases: create/list/get/start runs, plus GitHub webhook verification and metrics. createRun validates payloads, allocates artifact dirs, and persists runs; startRun loads a pending run, marks it running, writes payload to disk, spawns the bash runner, and records completion.
Infra: adapters for Postgres (runs persistence), Redis (status cache/rate limits), process spawning for the runner, artifacts filesystem helpers, GitHub App auth/signature verification, and Sentry telemetry. Controllers/routes compose these pieces behind the Fastify server.

Web UI (Svelte)¶

The Svelte web client lives in web/.

Dev (VPS/internal):
npm run dev -- --host 0.0.0.0 --port 5173
Production-like preview (behind a reverse proxy):
npm run preview -- --host 0.0.0.0 --port 4173

Traefik compatibility¶

GitTinkerer's web UI is compatible with Traefik by binding to 0.0.0.0 on a known port. Traefik configuration is handled elsewhere; place Traefik in front of the web/ service and route HTTP traffic to the chosen port.

HTTP API (Node 24)¶

The web UI talks to a lightweight HTTP service running on the VPS (implemented in this repository). The UI submits runs and polls for results:

POST /api/runs
GET /api/runs/:run_id

Security & operational notes¶

Treat instruction comments as untrusted input; the system should apply least privilege where possible.
Store secrets only on the VPS (e.g., .env containing GITHUB_TOKEN). Never write secrets into artifacts.
Prefer a dedicated GitHub App token with scoped permissions for:
Reading PR metadata and comments
See operations.md for pause/retry/rate-limit behaviors and admin flows.
Pushing commits to the repo
Posting reply comments

Status¶

This repository is currently scaffolding and evolving. Payload schema and operational details may change as the implementation lands.