Spike from July 2025: Creating a Programming Language using Coding Agents on GitHub

Uncategorized 8 Minutes

People using “coding agents in a loop” have discovered the joy of implementing programming language compilers using this technique. It’s a lot of fun it’s impressive how much can be achieved.

I dug out my notes from an experiment (really a “spike”) we did in July 2025.

  • Aim: Implement a programming language for teaching
  • Approach: Run multiple coding agents in GitHub Actions
  • Collaboration: Agents record work via issues and pull requests.

In this work we used shared GitHub Issues for “team” coordination and GitHub Pull Requests as the way to keep “humans in the loop” – nothing was merged without human approval.

The automation made rapid forward progress towards a goal (“implement a language for teaching”) and stayed aligned to that goal right to the end (e.g., developing an educational partnership framework and strategy, rather than investing in more programming language features or a v2.0). There was humour along the way, when, due to a configuration error, the agents suddenly discovered unit tests had not been running. 🚨📢🔔⚠️

  • GitHub Issues provide a permanent “record of action” that I can continue to link to today.
  • Agents make good use of GitHub Issues and PRs for coordination and collaboration.
  • You can get a long way using only GitHub Issues and PRs as a substrate.

Things have come a long way since July 2025, and this can now be completed by a coding agent in just a few passes. Equally, it was a fun experiment and something I learned from. I’ve shared a transcript of the conversation, tidied up a little. In the transcript you’ll see me it took a while to get things set up, but I’ve recorded that too. I instructed the coding agents to use F#, which worked remakably well all in all

Thanks to my collaborator Peli de Halleux for the enjoyable work together.

Edited transcript of one of our spikes at GitHub Next, from July 13th 2025:

dsyme  [12:40 PM] After talking with Peli on Friday, I’m now looking at “Agent Swarms running on GitHub Actions”. The concept is simple: take Markdown descriptions of agentic GitHub Actions – and add a few of these to a repo, and sit back and “let things happen”.

I’m conducting a little experiment along these lines, please feel free to follow along….

  • Repo: https://github.com/dsyme/teach-compiler
  • Project goal: Implement a small programming language for teaching
  • Project initial content: a README.md
  • Team Member #1 – Agentic Planner – creates new issues according to project goals
  • Team Member #2 – Agentic Issue Triage – standard triage analysis (won’t trigger on bot-created issues)
  • Team Member #3 – Agentic Coder 1 – chooses an issue and works on it
  • Team Member #4 – Agentic Coder 2 – same
  • Team Member #5 – Agentic Coder 3 – same
  • Team shared status report issue

Agents trigger every few minutes.  They report their progress to the shared team issue. They also report problems there if they get stuck, e.g. https://github.com/dsyme/teach-compiler/1. This uses Claude Code hosted in GitHub Actions.

dsyme  [1:02 PM] Current status: haven’t got the tool configurations quite right for the Agentic Coders

pelikhan  [1:06 PM] If the agents trigger on markdown files in a specific branch/path, we could use this for message passing…

dsyme  [1:13 PM] LOL the coder agents opened a critical issue that they’re blocked. . Now unblocked 

@pelikhan Robust “Issue Ops” is important here – the agents claim an issue by writing an issue comment, but when they fail to make progress they aren’t deleting that issue or noting that they’re not able to make progress.

pelikhan  [2:42 PM] issue ops will make those experience sticky to actions

dsyme  [3:10 PM] LOL one of the agents just got fed up that it didn’t have permission to create a branch properly, so just pushed to main [EDITOR: an inspiration for what became GitHub Agentic Workflows Safe Outputs]

dsyme [3:13 PM] I do notice that we need a big fat “PAUSE” button that just pauses everything while I work some stuff out. I’m doing a complete reset now to start things clean.

pelikhan  [3:23 PM] issue ops

dsyme  [3:26 PM] Yay we have a first PR from an Agent! I have no reviewer or approver agents at the moment, so will do that myself for now

dsyme [3:27 PM] BTW the aim of this swarm is to implement a programming language for teaching. The initial content was a single README.md:

# Teach - a simple programming language for teaching

Teach is a simple programming language designed for educational purposes. It aims to introduce programming concepts in an easy-to-understand manner, making it suitable for beginners.

The concepts in Teach are

- **Variables**: Used to store data values.
- **Functions**: Blocks of code that perform a specific task.
- **Control Structures**: Includes loops and conditionals to control the flow of the program.
- **Input/Output**: Mechanisms to interact with users.

## Example Code

```teach
// This is a simple program that prints "Hello, World!" to the console
function main() {
    print("Hello, World!");
}
main(); // Call the main function to execute the program
```

dsyme [3:31 PM] LOL I reset the content of the repository half way through and the team freaked out and raised an issue that work had been lost #24 CRITICAL: Repository Reset – Recover Lost Work

## Critical Issue Identified
The repository appears to have been reset, causing loss of previously completed work.
### Evidence:
• PR #23 was merged (created example programs) but no .teach files exist
• Repository only contains README.md and reset.sh
• Recent git history shows "reset" commit
• Previous work from completed issues is missing
### Impact:
• High Priority: Lost example programs that were completed in PR #23
• Development Blocked: Developers working on Issues #2 and #8 may be missing foundation files

dsyme  [5:28 PM] OK, I reset things in a new repo, left it for half an hour and this is what I have

I’m Agentic Coder 1 – I’ve examined the current codebase and found that the interpreter already has some basic built-in functions but is missing many functions specified in the language specification. I’m going to implement the missing standard library functions including string manipulation, array operations, and mathematical functions.

dsyme  [5:37 PM] This is all enjoyable. Things I’ve noticed:

  1. The agents do overlap and get in each others way, like real developers. When they start on a task, they may not always realise it depends on outputs of other tasks.  Managing conflict and merging etc. is a key part of a swarm.
  2. This agent swarm operates by schedule polling (starting each agent afresh every 5-10 minutes). This is not a perfect experience but has a lot of advantages as the agent can reformulate an opinion on work to be done and recent events in the repo without needing to subscribe to every event, every trigger.

dsyme  [5:40 PM] A reminder all this was done with just

  1. One README seeding the repo with initial content
  2. The generic agent descriptions including this generic coder agent description. These are not specific to the problem, but are entirely under user control.
  3. The GitHub Agentic Workflows tool/framework

dsyme  [9:11 PM] While I had dinner

  • “Agentic Planner” is getting shirty with me for taking too long to merge a PR
  • “Agentic Coder 3” reviewed and approved the PR and made a recommendation to “MERGE IMMEDIATELY”
  • “Agentic Planner” got really annoyed that multiple agents are duplicating work and raised an issue ” URGENT: Agent Coordination – Avoid Duplicate Work on PR #10″ https://github.com/dsyme/teach-compiler/issues/16

pelikhan  [9:15 PM] Bring Codex and Jules to the party.

dsyme  [10:47 PM] Between the Planner and one of the Coder agents they just ported the programming language interpreter to javascript to make a Web based learning environment

dsyme  [11:45 PM] I added a QA Engineer to the swarm

dsyme  [12:56 AM] LOL I forgot to check in a file and by the time I cam back to the repo 20min later an agent had just written it

dsyme [12:58 AM] Similarly I moved some files around and one of the agents just worked out the docs were now inconsistent.

dsyme  [4:09 PM] Picking this up again… I had a mistake in the configuration and the agents in the swarm now have proper access to “bash”. This means the QA engineer is actually doing its job now…. For example it created this issue after building the code and trying to run it https://github.com/dsyme/teach-compiler/issues/114. Likewise it filed this issue after trying to build the tests: https://github.com/dsyme/teach-compiler/issues/113

dsyme  [4:16 PM] The Planner agent is now freaking out because the QA engineer discovered the application crash on startup (now they all finally have permission to run “bash”)

🚨 URGENT: Crisis Mode – Application Cannot Run Due to Runtime Failures
This is a CRISIS SITUATION requiring immediate attention. All non-essential work should be paused until basic functionality is restored.

Just like a real PM  

dsyme  [5:06 PM] OK so the agents just fixed all the bugs and made simple samples work

$ dotnet run --project src/Teach.fsproj -- examples/hello.teach
Hello, World!

$ dotnet run --project src/Teach.fsproj -- examples/calculator.teach
x + y =
15
x - y =
5
x * y =
50
x / y =
2

dsyme  [5:45 PM] Currently the swarm are working through 107 test failures (out of 212).  Trying to break it down into buckets etc.

dsyme [5:47 PM] All hard at work

dsyme [5:47 PM] image.png

dsyme  [6:19 PM] It’s kind of weird to see a bunch of AI bots on an all night death march to grind out the remaining test failures

dsyme [6:20 PM] They’re down to 61/228

dsyme  [6:25 PM] One of the strange things about AI bots is if their work conflicts you can often just throw it away and let them try again. Humans don’t like that….

dsyme [6:25 PM] Down to 39/228

dsyme  [7:13 PM] Down to 26/228

dsyme  [11:16 PM] I went to the pub and about 10 PRs were awaiting my review when I got back.

dsyme [11:16 PM] Down to 3/228

dsyme  [2:04 AM] The agents report release 1.0.0 of the Teach programming language is ready!

“The Teach programming language project is a remarkable success story that demonstrates how effective planning, quality implementation, and team coordination can create an exceptional educational tool. The project is ready for launch and positioned for significant educational impact.”

Unknown's avatar

Published by dsyme

Published

Leave a comment