Grimy , 11 hours ago

You need a GPU for any kind of performance.

For text I suggest: Ollama backend - command line interface, very easy to download models with one line of code. Supports most models and you can talk with the model inside the terminal so it’s stand alone OpenWebUI - easy install with docker and is meant to work easily with ollama. Comes with web search features and uploading pdfs. A bunch of different community tools and modules are available.

For img I suggest either: Automatic1111 - Traditional UI using gradio. Lots of extras you can download through the UI to do different things. ComfyUI - Node based UI, a bit more complicated but more powerful than automatic1111

For models, you can go on civitai and just download whatever you need and drop it into their respective folders for both auto and comfy.

For text, there’s also LMStudio which is very user friendly. It is closed source and much slower than ollama from my experience though. I have a 4060 in my laptop (8gb VRAM) and I’m getting an image every 2 secs about using stable diffusion 1.5 models and text speed is on par with chatgpt with the smaller 8b-9b model. For text I suggest gemma2 which is probably the best small model out right now.