Local Ai Server Setup Guides Proxmox 9 - Llama.cpp in LXC w/ GPU Passthrough

In this Local Ai setup guide I show you how to build llama.cpp in an LXC with quad 3090s, download LLMs from unsloth hosted off Huggingface, and run both CPU and GPU inference on Qwen. I show you how to connect to your new Llama.cpp server from our OpenWEBUI container providing a nice looking interface.

**IF YOU HAVE BLACKWELL 50x0 series Nvidia GPU's select the MIT and not Proprietary version during install.**

These guides are meant to be followed in this order:
▶️ Ollama Openwebui Video https://youtu.be/Met9pEfxsF8
📝 Ollama Openwebui Article https://digitalspaceport.com/how-to-setup-an-ai-server-homelab-beginners-guides-ollama-and-openwebui-on-proxmox-lxc/
▶️ 📍YOU ARE HERE📍 Llamacpp Unsloth Video
📝 Llamacpp Unsloth Article https://digitalspaceport.com/how-to-setup-an-ai-server-homelab-beginners-guides-llama-cpp-on-proxmox-9-lxc/
(Optional Guides)
▶️ vLLM Video https://youtu.be/CBNbM-RZgIQ
📝 vLLM Article https://digitalspaceport.com/how-to-setup-vllm-local-ai-homelab-ai-server-beginners-guides/
▶️ VibeVoice 7b TTS Video https://youtu.be/t7WH-86Jtn0
📝 VibeVoice Article https://digitalspaceport.com/how-to-setup-vibevoice-ai-podcast-tts-podcast-engine/

⚙️ QUAD 3090 AI HOME SERVER BUILD
GPU Rack Frame https://geni.us/GPU_Rack_Frame
Supermicro H12ssl-i MOBO (better option vs mz32-ar0) https://geni.us/MBD_H12SSL-I-O
Gigabyte MZ32-AR0 MOBO https://geni.us/mz32-ar0_motherboard
AMD 7V13 (newer, faster vs 7702) https://geni.us/EPYC_7V13_CPU
RTX 3090 24GB GPU (x4) https://geni.us/GPU3090
256GB (8x32GB) DDR4 2400 RAM https://geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) https://geni.us/PCIe4_Riser_Cable
AMD SP3 Air Cooler (easier vs water cooler) https://geni.us/EPYC_SP3_COOLER
iCUE H170i water cooler https://geni.us/iCUE_H170i_Capellix
(sTRX4 fits SP3 and retention kit comes with the CAPELLIX)
CORSAIR HX1500i PSU https://geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4, not needed for H12SSL-i) https://geni.us/SFF8654_to_SFF8654
ARCTIC MX4 Thermal Paste https://geni.us/Arctic_ThermalPaste
Thermal GPU Pads https://geni.us/Kritical-Thermal-Pads
HDD Rack Screws for Fans https://geni.us/HDD_RackScrews

▶️ Local Ai Server Builds:
Quad 3090 Ai Server Build https://youtu.be/JN4EhaM7vyw
Playlist https://www.youtube.com/playlist?list=PLarJAzZsWRGDktdCjcN51G4xVkvvjC0ZB

Ways to Support:
🚀 Join as a member for members-only content and extra perks https://www.youtube.com/c/digitalspaceport/join
☕ Buy Me a Coffee https://www.buymeacoffee.com/digitalspaceport
🔳 Patreon https://www.patreon.com/digitalspaceport
👍 Subscribe youtube.com/c/digitalspaceport?sub_confirmation=1
🌐 Check out the Website https://digitalspaceport.com

Chapters
0:00 Llama.cpp Complete Build Guide
1:22 Install NVIDIA Toolkit on Proxmox 9 HOST
4:31 Install Llama.cpp LXC Container
6:53 Install NVIDIA Driver in Llama.cpp LXC
8:27 Install NVIDIA TOOLKIT in LXC
10:27 How to take a Backup in Proxmox 9
12:45 Build Llama.cpp in a LXC
15:30 Download a LLM from Huggingface Unsloth for Llama.cpp

*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
*****

Видео Local Ai Server Setup Guides Proxmox 9 - Llama.cpp in LXC w/ GPU Passthrough канала Digital Spaceport