MetalSW: When My GPU Code Lost to the CPU

I wanted to see how far exact Smith–Waterman could be pushed on the GPU already sitting in my MacBook Air.

The target was deliberately narrow: one protein query against a database of many protein sequences, exact local-alignment scores, affine gaps, BLOSUM62, and no traceback. The GPU implementation uses Metal; the host side is C++17.

Before optimizing anything, I wrote a scalar CPU implementation and treated it as the oracle. That turned out to be the most important decision in the project.

Correctness before speed

The GPU was never allowed to be “probably right.”

Every GPU score was diffed against the CPU oracle. I started with hand-built cases, then moved to a pinned Swiss-Prot subset and progressively larger databases.

The final sweep covered 78,006 sequence-score comparisons with zero mismatches.

I also cross-checked the CPU oracle against Parasail before trusting it as the reference, because a fast GPU implementation that is consistently wrong is not useful.

The first benchmark was disappointing

My first serious benchmark used a 142-residue hemoglobin query against 750 Swiss-Prot sequences.

The GPU was slower than Parasail.

That was not the result I wanted, but it was the useful result.

It forced me to stop thinking in terms of “GPU should be faster” and start asking where the time was actually going.

My first hypothesis was wrong

The Metal path allocated and released a collection of buffers on every call, so my first guess was allocation overhead.

I rewrote the runner to cache and reuse those buffers.

Correctness stayed intact.

Performance barely moved.

The original run was around 0.348 GCUPS. The buffer-reuse version was around 0.345 GCUPS — effectively noise.

That experiment ruled out one explanation.

The timing data pointed somewhere else: on a small workload, fixed command-buffer submission and synchronization cost dominated the actual alignment work.

In other words, I had optimized something that was not the bottleneck.

Scale changed the result

The GPU did not become magically faster after a kernel rewrite. The workload became large enough to amortize the fixed dispatch cost.

With the same 142-residue query:

| Database size | MetalSW GPU | Parasail CPU SIMD | GPU / CPU | | --- | ---: | ---: | ---: | | 750 | 0.407 GCUPS | 0.407 GCUPS | ~1.0× | | 5,000 | 2.217 GCUPS | 0.370 GCUPS | 5.99× | | 20,000 | 5.224 GCUPS | 0.393 GCUPS | 13.30× | | 50,000 | 5.309 GCUPS | 0.392 GCUPS | 13.54× |

There is no honest single “GPU is X times faster” number here.

At small database sizes, the GPU is not clearly better. At larger sizes, there is enough parallel work for the hardware to matter.

That crossover was one of the clearest lessons from the project.

The optimization that actually mattered

The biggest useful optimization was surprisingly simple.

Protein sequences in a database vary a lot in length. Initially, neighboring GPU threads could receive very different sequence lengths.

That matters because threads in the same SIMD group move together. A short sequence can finish its useful work early and then sit idle while a longer sequence in the same group keeps going.

So I sorted database sequences by length before dispatch.

No new algorithm. No major kernel rewrite.

Just better work grouping.

The result:

| Database size | Before length sorting | After length sorting | Change | | --- | ---: | ---: | ---: | | 750 | 0.349 GCUPS | 0.407 GCUPS | +17% | | 5,000 | 2.225 GCUPS | 2.217 GCUPS | roughly flat | | 20,000 | 3.379 GCUPS | 5.224 GCUPS | +55% | | 50,000 | 3.504 GCUPS | 5.309 GCUPS | +51% |

That was the strongest optimization in the project because it attacked an actual GPU utilization problem: SIMD divergence.

Why I used two score widths

The kernel has an int8 fast path and an int16 fallback.

The idea is simple: smaller per-thread state reduces register and array pressure, but local-alignment scores can overflow an 8-bit range.

So the first pass tracks saturation. Any sequence that would overflow can be rerun through the wider path.

On every real Swiss-Prot corpus I tested — including 50,000 sequences — the int8 path handled the full workload and the fallback was not needed.

The fallback is still tested with deliberately high-scoring edge cases, because “it never happened in my benchmark” is not the same thing as “the code path is correct.”

Unified memory was not the speedup

Apple Silicon uses unified memory. The CPU and GPU can access the same physical memory, so this workload does not pay a PCIe-style host-to-device copy.

That is useful.

But it was not the reason MetalSW got faster.

The buffer-reuse experiment made that clear. Removing repeated allocations did not materially change throughput.

The bottleneck I could actually measure was dispatch and synchronization overhead at small scales, followed by SIMD-group imbalance at larger scales.

That distinction matters more than repeating “unified memory is fast.”

Power and thermals

The machine was a base M2 MacBook Air: fanless, 10-core GPU, 8 GB unified memory.

On the 50,000-sequence workload, the measured combined CPU + GPU power draw was about 6.46 W.

That works out to roughly 0.816 GCUPS/Watt.

During the sustained run, the GPU stayed near full hardware-active residency and maximum clock. I did not observe thermal warnings or a throughput collapse.

That does not mean the machine can never throttle. It means this workload, at this measured power level, did not trigger it during the test I ran.

Where MetalSW is still limited

The current version is intentionally narrow:

• score only — no traceback or reconstructed alignment path
• protein sequences only
• BLOSUM62 scoring
• one GPU thread per database sequence
• no intra-sequence wavefront parallelism
• query length capped at 512 residues
• measured on one machine: a base M2 MacBook Air

Those boundaries are part of the result.

I would rather state them clearly than hide them behind the 13.5× number.

What I took away from it

The useful part of MetalSW was not proving that a GPU can beat a CPU.

It was learning how to separate a good systems story from a good systems measurement.

The GPU initially lost.

My first optimization targeted the wrong thing.

A small workload made the architecture look worse than it was.

The change that mattered most was not more code — it was arranging work so the hardware could execute it efficiently.

That is the part of the project I trust the most because every conclusion came after a measurement that could have disagreed with me.

The code, benchmark corpus, methodology notes, and full experiment log are available in the MetalSW repository.