Agent Farm v3.4 - Chunked Write Edition
AI organism evolution and parallel task execution with tool-enabled agents. Now with Chunked Write Pattern for generating large documents and code files!
What's New in v3.4
- Chunked Write Pattern: Bugs write sections in parallel, Python assembles directly
- chunked_write: Generate large markdown/text documents (unlimited size)
- chunked_code_gen: Generate multi-function code files in parallel
- chunked_analysis: Multi-perspective analysis with synthesis
- Bypasses 500-char limit: Each bug writes small chunks, combined output is unlimited
Performance
- 8.6x faster than v3.0 (103s -> 12s for 4-task swarm)
- 1 iteration per task (was 3-5)
- 100% success rate with real tool data
- Local synthesis - qwen2.5:14b synthesizes results (no cloud tokens!)
Models
| Role | Model | VRAM | Purpose |
|---|---|---|---|
| Scout | qwen3:4b | 2.5GB | Reconnaissance |
| Worker | qwen3:4b | 2.5GB | Task execution |
| Memory | qwen3:4b | 2.5GB | Context retention |
| Guardian | qwen3:4b | 2.5GB | System monitoring |
| Learner | qwen3:4b | 2.5GB | Pattern acquisition |
| Synthesizer | qwen2.5:14b | 8.99GB | Result synthesis |
MCP Tools (30)
Colony Management
spawn_colony- Create bug colony (standard/fast/heavy/hybrid)list_colonies- List active coloniescolony_status- Detailed colony infoquick_colony- Quick health checkdissolve_colony- Remove colonycleanup_idle- Remove idle coloniesfarm_stats- Comprehensive statistics
Swarm Deployment
deploy_swarm- Deploy tasks to colonyquick_swarm- One-shot spawn + deploy
Specialized Swarms
code_review_swarm- 4-perspective code reviewcode_gen_swarm- Generate code + tests + docsfile_swarm- Parallel file operationsexec_swarm- Parallel shell commandsapi_swarm- Parallel HTTP requestskmkb_swarm- Multi-angle knowledge queries
Tool-Enabled Agents
tool_swarm- Deploy bugs with real system toolssystem_health_swarm- Quick system health checkrecon_swarm- Directory/codebase reconnaissancedeep_analysis_swarm- Deep disk/file analysisworker_task- Single worker with full tools
Direct Operations
heavy_write- Direct file write (bypasses LLM for large content)synthesize- Standalone synthesis of any JSON results
Chunked Write Pattern (NEW)
chunked_write- Generate large documents via parallel section writingchunked_code_gen- Generate code files with functions written in parallelchunked_analysis- Multi-perspective analysis with synthesis
Bug Tool Permissions
| Role | Tools |
|---|---|
| Scout | read_file, list_dir, file_exists, system_status, process_list, disk_usage, check_service, exec_cmd |
| Worker | read_file, write_file, list_dir, exec_cmd, http_get, http_post, system_status, disk_usage, check_service |
| Memory | read_file, kmkb_search, kmkb_ask, list_dir, system_status, process_list, disk_usage, check_service, exec_cmd |
| Guardian | system_status, process_list, disk_usage, check_service, read_file, list_dir, exec_cmd |
| Learner | read_file, analyze_code, list_dir, kmkb_search, system_status, process_list, disk_usage, check_service, exec_cmd |
Structured Output Details
Agent Farm v3.3 uses Ollama's structured output feature to enforce JSON schemas on model responses:
# Bug responds with guaranteed-valid JSON:
{"tool": "system_status", "arg": ""}
{"tool": "exec_cmd", "arg": "df -h"}
{"tool": "check_service", "arg": "ollama"}
The constrained decoding (GBNF grammar) masks invalid tokens during generation, ensuring:
- Always valid JSON
- Correct tool names
- Proper argument structure
- No parsing failures
Results now include a mode field showing which method was used:
structured- JSON schema enforcedstructured+autoformat- JSON + simple result formattingstructured+deep- JSON with multi-step reasoningregex- Fallback regex parsingregex+autoformat- Regex + simple result formatting
Chunked Write Pattern
The chunked write pattern solves the ~500 char output limitation of small models by decomposing large tasks:
1. PLANNER BUG (qwen2.5:14b)
|-- Creates structured JSON outline
|-- {"sections": [{"title": "...", "description": "..."}]}
2. WORKER BUGS (qwen3:4b) - IN PARALLEL
|-- Each writes one section (~300-500 chars)
|-- 4 workers = 4 sections simultaneously
3. PYTHON CONCATENATION (NO LLM)
|-- header + separator.join(sections)
|-- Zero token cost, instant assembly
4. DIRECT FILE WRITE (NO LLM)
|-- tool_write_file() saves result
|-- Bypasses any output corruption
Performance
| Tool | Output Size | Sections | Time |
|---|---|---|---|
| chunked_write | 9.6 KB | 5 | 78s |
| chunked_code_gen | 1.9 KB | 4 functions | 88s |
| chunked_analysis | Varies | 4 perspectives | ~60s |
Why It Works
- Small models excel at focused, short outputs
- Each section is within the "safe zone" (<500 chars)
- Python handles assembly (no LLM token cost)
- Parallel execution via ThreadPoolExecutor
- Structured output ensures reliable planning
Usage Examples
System Health Check
agent-farm:system_health_swarm
Custom Task Swarm
agent-farm:tool_swarm
colony_type: "heavy"
tasks: [
{"prompt": "Check CPU temperature"},
{"prompt": "List top 5 memory processes"},
{"prompt": "Check if docker is running"}
]
Large File Write (Direct)
agent-farm:heavy_write
path: "/tmp/large_output.txt"
content: "... large content ..."
Codebase Reconnaissance
agent-farm:recon_swarm
target_path: "/home/kyle/repos/my-project"
Generate Large Document (Chunked)
agent-farm:chunked_write
output_path: "/tmp/security_guide.md"
spec: "Linux server security hardening guide"
num_sections: 5
doc_type: "markdown"
Output: 9KB+ document with 5 coherent sections
Generate Code File (Chunked)
agent-farm:chunked_code_gen
output_path: "/tmp/utils.py"
spec: "File utilities: read, write, copy, delete"
language: "python"
num_functions: 4
Output: Complete Python module with 4 functions
Multi-Perspective Analysis
agent-farm:chunked_analysis
target: "/home/kyle/repos/project"
question: "What are the architectural patterns?"
num_perspectives: 4
Output: Analysis from Structure, Patterns, Quality, Performance perspectives
Installation
cd ~/repos/agent-farm
uv venv
uv pip install -e .
Claude Desktop Config
{
"mcpServers": {
"agent-farm": {
"command": "/home/kyle/repos/agent-farm/.venv/bin/python",
"args": ["-m", "agent_farm.server"]
}
}
}
Changelog
v3.4.0 (2026-01-23)
- Chunked Write Pattern - Bugs write sections in parallel, Python assembles
- chunked_write - Generate unlimited-size documents (tested: 9.6KB in 78s)
- chunked_code_gen - Generate multi-function code files in parallel
- chunked_analysis - Multi-perspective analysis with synthesis
- Bypasses 500-char bug limitation via task decomposition
- Planner uses structured JSON output for reliable outlines
v3.3.0 (2026-01-23)
- Ollama Structured Output - JSON schema enforcement via constrained decoding
- Reliable tool parsing - No more regex failures
- Mode tracking - Results show parsing method used
- Regex fallback - Legacy parsing still available as backup
- All roles get exec_cmd for complex shell queries
v3.2.0 (2026-01-22)
- Synthesizer role - qwen2.5:14b for accurate result synthesis
- synthesize parameter - Added to tool_swarm, system_health_swarm, recon_swarm, deep_analysis_swarm
- synthesize tool - Standalone synthesis of any JSON results
- No more Claude synthesis tax - bugs do ALL the work locally
v3.1.0 (2026-01-20)
- 8.6x speed improvement (103s -> 12s)
- Auto-format results skip redundant LLM calls
- Reject invalid tools instantly
- Force tool usage before answers
- Complex shell command support fixed
- All roles upgraded to qwen3:4b minimum
v3.0.0 (2026-01-19)
- Linux rebuild from Windows version
- Tool-enabled agents with role permissions
- System health, recon, worker swarms
- TRUE PARALLEL via ThreadPoolExecutor