Better Protein Representations Were Not Enough: A Fly-Inspired Search That Lost

The idea was attractive. The fruit fly mushroom body turns a dense input into a very sparse code, and that sparse code underlies a well-known similarity-search algorithm called FlyHash. A reward-gated learner sitting on top of that code is a plausible way to learn which mutations help, using almost no data.

So I asked a narrow, testable question in fly-protein: can a FlyHash code and a reward-gated learner find high-fitness protein variants with only a few lab-style measurements, compared with established search strategies, when every method gets the same number of allowed measurements?

The answer is no. The more useful result is why.

The comparison had to be fair

A protein-search benchmark is easy to rig by accident, so most of the work went into the harness rather than the learner.

• Real landscapes. Ten ProteinGym deep-mutational-scanning assays from ten unrelated proteins, each with roughly 2,500 to 5,000 single mutants. The candidate set is every variant of an assay.
• A budgeted oracle. Methods receive an Oracle object and a Static object that has no fitness field. The only way to learn a fitness is to call oracle.query on a variant, and an exhausted budget raises an error. The evaluator alone reads the landscape afterwards.
• Matched conditions. Same assay, same shared start variant per assay and seed, same 20 seeds, same candidate set, and the same 15-nearest-neighbour search graph for every method that moves locally.
• Honest statistics. The unit of replication is the assay, not the run. Confidence intervals are bootstrapped over assays, and raw fitness scores are never averaged across assays because their scales differ.

Seven methods share this contract: random search and a random walk, greedy local search, regularized evolution, GP-BO, a graph-adapted AdaLead, zero-shot ESM-2, and the fly learner with and without memory.

What happened

The metric is the share of runs that reach the assay's top 1% of variants, averaged over the ten assays.

• Random search: 40.0, 65.0, 88.5, and 100.0 at 50, 100, 200, and 500 measurements.
• Graph-adapted AdaLead: 54.5, 72.5, 94.0, and 100.0.
• GP-BO: 52.0, 65.5, 92.5, and 99.5.
• Regularized evolution: 48.0, 75.5, 94.0, and 100.0.
• Fly learner: 30.5, 51.0, 76.5, and 98.5.
• Fly learner with memory: 31.5, 54.5, 78.5, and 98.0.

At 50 measurements, graph-adapted AdaLead beats the fly learner by 24.0 points, with a 95% interval of 15.5 to 32.0. GP-BO beats it by 21.5. The fly learner is level with a random walk on the same graph and at or below plain random search.

Random search is itself strong here: fifty random measurements out of roughly three thousand variants already reach about the 98th percentile on average. That is why 50 to 200 measurements is the informative regime, and why a method that cannot beat random in that window has not earned its complexity.

Was the representation the problem?

This was the first thing I suspected, and I tested it before blaming the learner.

A zero-shot ESM-2 ranking, which uses no observed fitness at all, is a strong training-free baseline. Ablations over ESM-2 size, input dimension, number of Kenyon cells, and sparsity showed that the signal is present and that FlyHash keeps most of it. Compression was not the cause.

So the representation was fine. The learner was not turning it into sample-efficient search. The title of the study became the conclusion: better protein representations were not enough, and search strategy was the bottleneck.

Four things I tried that did not rescue it

1. Tuning the learner. A sweep over learning rate, temperature, an absolute-reward rule, and replay produced no reliable improvement.
2. Cross-assay pretraining. Leave-one-assay-out transfer of the learner's weights gave a weak but consistently positive held-out ranking signal, with Spearman correlations of 0.04 to 0.14. It still failed to improve navigation and often harmed it. Weak transferable ranking information is not enough for sequential search.
3. Real hemibrain wiring. I replaced random projection wiring with real projection-neuron to Kenyon-cell synapse counts from the hemibrain connectome. It was slightly worse, not better.
4. Larger models. Bigger ESM-2 models improved neighbourhood quality but did not increase the fly learner's gain.

Making the demo as honest as the benchmark

The project site replays one recorded run, TEM-1 beta-lactamase, seed 0 of 20 fixed in advance, as the fly learner against GP-BO from the same start variant. A small three.js fly stands on the variant it currently occupies, and a circuit view shows the learner's input channels, the active Kenyon cells, its candidate scores, the chosen mutation, the measured fitness, and the recorded weight update.

Every number comes from logged state. The browser re-applies the logged weight updates and checks them against logged statistics and SHA-256 hashes. The motion is interpolated for presentation, and the page says so. The fly is a navigation marker, not a simulation of a fly brain, and the two-dimensional map is a UMAP picture that no method ever sees.

Limitations I would put first

• Ten assays, mostly growth and stability phenotypes.
• Wide assay-level confidence intervals for the deterministic zero-shot rankings.
• Search methods use ESM-2 8M embeddings while zero-shot goes up to 650M.
• Graph-adapted AdaLead, GP-BO, and regularized evolution use fixed, untuned settings, and my AdaLead is not the published implementation.
• The fly learner's best configuration was chosen earlier on a single landscape.
• This is an in-silico benchmark. It is not drug discovery and it is not wet-lab validation.

What I took away

A negative result is only worth publishing if the experiment could have said yes. The budgeted oracle, the matched seeds, and the assay-level statistics are what make this one believable.

The practical lesson is a habit rather than a finding about flies: before improving a model, decide what the fairest baseline is, make it hard to cheat, and then report the comparison even when your idea is the one that loses.

You can replay the run and read the full results, or browse the repository.