Tag

benchmarks

1 essay tagged benchmarks.

Benchmark · 7 min

What local models actually cost you, measured on two consumer GPUs

Six open models from 0.5B to 7.6B, measured for real on an RTX 3070 and a GTX 1660 SUPER — throughput, memory, cold starts, and which ones actually get the arithmetic right.