The first real model finished training in the middle of February. 42.7% top-1 accuracy on 2,592 species. 66.1% top-5. The numbers were carefully logged, version-controlled, and decisively below anything I could ship.
This was the moment the abstract question of will this work? became a concrete answer: not yet, and not like this.
The diagnosis
The model wasn’t overfitting. The train-val gap was narrow. About a percentage point, which is healthy. It wasn’t underfitting in the way that usually means a capacity problem. The architecture was EfficientNet-B0, a standard choice for this class of task with millions of downloads behind it. The loss curves behaved. The optimiser converged.
What the numbers actually said was that the model had learned everything the data could teach it, and the data couldn’t teach it enough.
This wasn’t surprising once I looked at it clearly. The dataset was around 150,000 images across 2,592 species. About 58 images per class on average, and with a long tail: many species had under thirty, some had under ten. For fine-grained visual classification, where the model has to tell one sage from another, thirty images per class is nowhere near enough. You can train a model on that, and it will train cleanly, and it will produce loss curves that look healthy. It just won’t generalise, because it hasn’t seen enough variation to know what variation means for any given species.
The bottleneck wasn’t architecture. It wasn’t training. It was upstream of everything I’d been spending my time on. It was data.
I had thought I was done with data problems months before.
The gamble
Here’s what I should have done. I should have fixed the model at 2,500 species first. Got it to a place where I knew the approach worked. Understood what was limiting it. Understood what would scale and what wouldn’t. Then, with the smaller version known-good, scaled up.
I didn’t do that. I couldn’t.
The mobile app was being built in parallel. The mobile side needed a species contract. A list of exactly which plants the app could identify, with stable IDs, that the mobile database could be built around. A contract that could be changed later wasn’t a contract. The mobile couldn’t ship an app that might need to reindex its entire database because I’d decided to add or remove species once integration had started. The list had to be frozen, it had to be frozen soon, and it had to be frozen at the right size.
The right size wasn’t 2,500. The right size depended on who the app was for. The users were gardeners and houseplant owners, not specialists. The app needed to cover the plants those users were actually likely to photograph. That was 5,000 species, give or take. Not the world of plants. Just the world of garden plants and houseplants.
So on February 20th I doubled the class count on a model that wasn’t working. I hadn’t fixed the 2,500-species version. I didn’t know whether the problems I was seeing at 2,500 were caused by class count or by something else entirely. I was going to find out at 5,000.
It was the largest gamble I’d taken on the project. The conventional engineering move is to de-risk incrementally. Get the small version working before adding complexity. I was going the other way. I wasn’t doing this because I’d decided it was smarter. I was doing it because I had no choice. The mobile timeline needed the contract, and the contract needed to be 5,000 species, and if the model couldn’t handle 5,000 species, I wasn’t going to find out with a clean exit. I’d find out after the mobile project had built an app around a database I could no longer change.
The quieter recognition, sitting behind the engineering decision, was that this had just turned the stage gate into an all-or-nothing bet. If the 5,000 model couldn’t clear the accuracy threshold I’d set in January, the project was done. There was no fallback to a smaller-scope version of the same work. The smaller scope was what I was leaving behind. Either 5,000 species worked, or I’d told myself I’d stop.
I froze the list. I committed it to the mobile repo as a contract. And I started the work of making the data good enough to support the decision I’d just made.
The valley
From February 20th to March 27th, the work was grinding.
The collection pipeline that had been built for 2,500 species at moderate depth now needed to scale to 5,000 species at greater depth. Every species in the frozen list needed a target number of images, and the long tail was where the problems lived. Common species were easy. Tomato, basil, rosemary had tens of thousands of public-domain photographs available. The tail was harder. Species that gardeners grow but the internet hasn’t extensively documented. Species with taxonomic synonyms where images were scattered across multiple names. Species that existed in the databases but nowhere in the public image record.
I rebuilt the collection pipeline three times over eleven days in March. Each version ran, closed some species, and exposed the next blocker. The pattern became familiar: deploy the new version, watch which species closed, investigate the ones that hadn’t, find the reason they hadn’t, write a fix, deploy again.
Most of the reasons weren’t what I expected. I kept assuming the persistent tail was genuinely rare species. Plants with few images because they had few images, a data scarcity problem I couldn’t solve by writing better code. That turned out to be wrong four times out of five. The tail was mostly code bugs in my own collectors. A quality preset where the strict setting was actually less strict than the balanced setting, because someone (me) had inverted the thresholds. A shared collection budget where the first source to run would consume the whole quota before the other sources got a chance. A collector that wasn’t receiving common names, so it was searching for the Latin binomial only, on a platform where most uploads use common names. Each of these was a bug I could fix. Each of these had been silently losing images for species I’d assumed were rare.
The lesson I wrote down, somewhere in the middle of this, out of frustration, was: never assume species are rare; investigate the collector code first. I had to relearn this three or four times before it stuck.
AI tools couldn’t help me here. The bugs were in code I’d written in places I hadn’t audited. The fix in every case was specific to my own implementation choices. This was solo work, the way I’d originally wanted it to be.
I was paying for most of this out of my own pocket. The rule I’d set in the beginning: open licences, respect terms of service, pay when paying was the right model, was now showing up as line items on my bank statements. PlantCLEF access. iNaturalist API keys. Flickr subscription. By late March I added another paid image search. The long tail of rare species needed more searches than any single provider allowed. The spending wasn’t dramatic, in the hundreds of pounds total, across the whole project. But it was all coming out of my own bank account, and the rule I’d set was what required it. I was building something I’d be willing to put my name on. This was what that cost.
Somewhere in the middle of March I made a concession I wasn’t happy with. The truly rare species, plants with twenty or thirty usable images on the entire internet, couldn’t meet the 640-pixel minimum I’d set for image quality. Higher resolution would have been better for training. Higher resolution didn’t exist for these species. I dropped the minimum to 260 pixels to get any images at all. The trade was clear: smaller, lower-quality images that the model could at least see, versus no training data at all and the species getting silently dropped from the app. The 260-pixel images weren’t good. They were better than nothing, and nothing was the alternative.
The days during this stretch were uniform. Most mornings I’d check the collection dashboard to see how many species had closed overnight, find it was fewer than I’d hoped, investigate the ones that hadn’t, write a patch, deploy, wait. Most evenings I’d do the same.
The calendar was on my desk. I had printed it in January, when I’d written the roadmap. Every task mapped to the day it was meant to close, the whole plan laid out on sheets of paper. It wasn’t a schedule suggestion. It was the committed timeline, the one I’d held myself to. Every morning I could see what day it was, and what day the calendar said I should be on, and the gap between them.
The gap grew every week. A task that slipped by one day. Another that slipped by two. A planned week of collector work that turned into ten days, then twelve. By mid-March the calendar said I should have been training the model for a fortnight. I was still fixing collectors. The calendar didn’t change. The days did. I just watched the distance between where I should have been and where I was open up on a piece of paper.
The results seemed impossibly far away for most of those weeks. The model was still at 42% on 2,500 species, and the target was 75% on 5,000, and the distance between where I was and where I needed to be wasn’t closing in any way I could see from week to week. The doubts cycled in and out. Some days they were quieter. Some days they were loud. I never crossed into wanting to give up, but there were days when I genuinely couldn’t tell whether the work I was doing was going to get me there, or whether it was just a slower version of being stuck.
What kept me walking
The plan.
For two weeks in January I had stopped building and started planning. At the time, the planning felt like the thing I had to do instead of making progress. In March it was the only reason I was still making progress.
The road was laid out in front of me. Every task had acceptance criteria. Every acceptance criterion had a specific, bounded meaning. When a task was done, I knew it was done. When a task wasn’t done, I knew what was missing. The planning I’d done in January wasn’t producing progress directly, but it was letting me know, at any given moment, where on the road I was. I wasn’t lost. I was just slower than I wanted to be, and those are not the same thing. Lost is when you stop because you don’t know where to go. Slow is when you keep walking because you know where the next marker is.
The calendar on my desk was the road made visible. The gap between where I was and where I should have been was painful to look at every day. It was also the thing that told me the destination hadn’t moved. The plan hadn’t broken. I was behind on it, but I was still on it.
The stage gate hadn’t moved either. The deadline I’d written in January was still on the calendar. The work was taking longer than I’d budgeted for it, and each week in the valley was a week I wasn’t spending on training. The gate was always a little closer at the end of each week than it had been at the start. It was pressure, not panic. The plan was still good; the grind was still grinding; the gate was still out there.
The ceiling
By the end of March the data work was closing. Not finished. I don’t think the data work was ever finished, but closing. Most species were hitting their image targets. The collection pipelines had stabilised. The bugs had slowed from a daily occurrence to a weekly one. The dataset had grown more than thirty times in size from where it had been before the freeze, and the average image count per species had climbed to 1000 images. Into a range where I could plausibly ask a model to learn from it.
I told myself the data was as good as I was going to get it. I wasn’t sure. What I was sure of was that I was out of things to do on the collection side that would meaningfully move the needle. The question now was whether the model architecture and the training approach could make use of what I’d built.
I didn’t know the answer to that yet.
To be continued.