Shoehorn Automatically Fits Local AI Models to Available System VRAM
Chips & ComputeStaying Ahead · Aug 24

Shoehorn Automatically Fits Local AI Models to Available System VRAM

Shoehorn is a command-line tool that shrinks open-source language models to match the exact amount of free memory on a graphics card. Once resized, the application automatically launches the model locally using llama.cpp.

Read the original