Playbooks, Skills, and Worktrees: A Repeatable Claude Code Framework for Data Engineering

Data engineering work is full of repeatable-but-manual sequences: branching and committing conventions, deployment steps, ticket workflows, and the investigative legwork when a production job misbehaves. Handing these to an AI agent one prompt at a time means re-explaining the project every session, and long contexts degrade the model's effectiveness. The challenge was making Claude reliably good at this specific project without stuffing everything into every conversation.

The framework lives in a dedicated AI folder inside the project repository, built from three components. A central CLAUDE.md file acts as the entry point, holding high-level project information and registering everything else. Playbooks are Markdown documents with step-by-step instructions for whole workflows, like GitHub interactions or deployments. Skills are narrow, executable instructions for specific tasks, the AI equivalent of individual functions. Each playbook carries a table of contents so Claude reads only the sections it needs, keeping context small and output sharp.

On top of this, the engineer runs Git worktrees to hold multiple branches simultaneously, letting Claude work several tickets in parallel without stashing changes. For each ticket, Claude explores the issue, drafts an implementation plan, and executes it step by step with human review at each milestone, the same discipline Lineate applies across all AI-enabled delivery. The framework also handles harder problems: during incident investigation, Claude queries databases and compares data across jobs to isolate the cause of production issues.