Llama-2 FOSAI & LLM Roundup Series! (Summer 2023 Edition)
cross-posted from: lemmy.world/post/1894070
Welcome to the Llama-2 FOSAI & LLM Roundup Series!
(Summer 2023 Edition)
Hello everyone!
The wave of innovation I mentioned in our Llama-2 announcement is already on its way. The first tsunami of base models and configurations are being released as you read this post.
That being said, I’d like to take a moment to shoutout TheBloke, who is rapidly converting many of these models for the greater good of FOSS & FOSAI.
You can support TheBloke here.
Below you will find all of the latest Llama-2 models that are FOSAI friendly. This means they are commercially available, ready to use, and open for development. I will be continuing this series exclusively for Llama models. I have a feeling it will continue being a popular choice for quite some time. I will consider giving other foundational models a similar series if they garner enough support and consideration. For now, enjoy this new herd of Llamas!
All that you need to get started is capable hardware and a few moments setting up your inference platform (selected from any of your preferred software choices in the Lemmy Crash Course for Free Open-Source AIor FOSAI Nexus resource, which is also shared at the bottom of this post).
Keep reading to learn more about the exciting new models coming out of Llama-2!
8-bit System Requirements
ModelVRAM UsedMinimum Total VRAMCard ExamplesRAM/Swap to Load*LLaMA-7B9.2GB10GB3060 12GB, 3080 10GB24 GBLLaMA-13B16.3GB20GB3090, 3090 Ti, 409032 GBLLaMA-30B36GB40GBA6000 48GB, A100 40GB64 GBLLaMA-65B74GB80GBA100 80GB128 GB### 4-bit System Requirements
ModelMinimum Total VRAMCard ExamplesRAM/Swap to LoadLLaMA-7B6GBGTX 1660, 2060, AMD 5700 XT, RTX 3050, 30606 GBLLaMA-13B10GBAMD 6900 XT, RTX 2060 12GB, 3060 12GB, 3080, A200012 GBLLaMA-30B20GBRTX 3080 20GB, A4500, A5000, 3090, 4090, 6000, Tesla V10032 GBLLaMA-65B40GBA100 40GB, 2x3090, 2x4090, A40, RTX A6000, 800064 GBSystem RAM (not VRAM), is utilized to initially load a model. You can use swap space if you do not have enough RAM to support your LLM.
The Bloke
One of the most popular and consistent developers releasing consumer-friendly versions of LLMs. These active conversions of trending models allow for many of us to run these GPTQ or GGML variants at home on our own PCs and hardware.
70B
- TheBloke/Llama-2-70B-chat-GPTQ
- TheBloke/Llama-2-70B-Chat-fp16
- TheBloke/Llama-2-70B-GPTQ
- TheBloke/Llama-2-70B-fp16
13B
- TheBloke/Llama-2-13B-chat-GPTQ
- TheBloke/Llama-2-13B-chat-GGML
- TheBloke/Llama-2-13B-GPTQ
- TheBloke/Llama-2-13B-GGML
- TheBloke/Llama-2-13B-fp16
7B
- TheBloke/Llama-2-7B-GPTQ
- TheBloke/Llama-2-7B-GGML)
- TheBloke/Llama-2-7B-fp16
- TheBloke/Llama-2-7B-fp16
- TheBloke/Llama-2-7b-Chat-GPTQ
LLongMA
LLongMA-2, a suite of Llama-2 models, trained at 8k context length using linear positional interpolation scaling.
13B
7B
Also available from The Bloke in GPTQ and GGML formats:
7B
Puffin
The first commercially available language model released by Nous Research! Available in 13B parameters.
13B
Also available from The Bloke in GPTQ and GGML formats:
13B
Other Models
Leaving a section here for ‘other’ LLMs or fine tunings derivative of Llama-2 models.
7B
Getting Started w/ FOSAI!
Have no idea where to begin with AI/LLMs? Try starting here with UnderstandGPT to learn the basics of LLMs before visiting our Lemmy Crash Course for Free Open-Source AI
If you’re looking to explore more resources, see our FOSAI Nexus for a list of all the major FOSS/FOSAI in the space.
If you’re looking to jump right in, visit some of the links below and stick to models that are <13B in parameter (unless you have the power and hardware to spare).
FOSAI Resources
Fediverse / FOSAI
LLM Leaderboards
LLM Search Tools
GL, HF!
If you found anything about this post interesting - consider subscribing to !fosai where I do my best to keep you in the know about the most important updates in free open-source artificial intelligence.
I will try to continue doing this series season by season, making this a living post for the rest of this summer. If I have missed a noteworthy model, don’t hesitate to let me know in the comments so I can keep this resource up-to-date.
Thank you for reading! I hope you find what you’re looking for. Be sure to subscribe and bookmark the main post if you want a quick one-stop shop for all of the new Llama-2 models that will be emerging the rest of this summer!