We test open-source AI projects on real work and report what holds up. We publish pieces of our own work when they are useful beyond one project. And we set build challenges anyone can take on.
We pick open-source AI repositories on GitHub, run them on real work and write up what holds up: what it does, what it costs to run, where it breaks and who should use it.
Tools, prompts and small systems from our own projects, published as public repositories when they are useful outside the project. Nothing confidential, and nothing from a client without the client's written OK.
A brief, a deadline and a public starter repository. Build the thing, open a pull request, and the strongest entries are reviewed in the next report.
The problem it solves, in one paragraph, without the README's adjectives.
The real task we gave it, the setup and the version we ran.
Hosting, model calls and the hours it took to get working.
The limits we hit, with the inputs that broke it.
Use it, watch it or skip it, and for whom.
The first repo reports and the first challenge go up here when they are ready, not before. Tell us which repository you want tested, or which problem deserves a challenge.
Send the link and the job you want it to do. If it makes the list, you get the report first.