Local Ai Server Setup Guides Proxmox 9 - vLLM in LXC w/ GPU Passthrough

Setting up vLLM in our Proxmox 9 LXC host is actually a breeze in this video which follows on the prior 2 guides to give us a very full suite of Local Ai Server runners. We have covered in the first video setting up Proxmox 9, Ollama and OpenWEBUI in an LXC. In the second guide we setup Llama.cpp and used unsloth. We also created a base level backup LXC container which we use to rapidly deploy vLLM in this video.

IF YOU HAVE BLACKWELL 50x0 series Nvidia GPU's Use MIT Drivers and not Proprietary version during installation. The same package is downloaded, but MIT option selected during installation process.

These guides are meant to be followed in this order:
▶️ Ollama Openwebui Video https://youtu.be/Met9pEfxsF8
📝 Ollama Openwebui Article https://digitalspaceport.com/how-to-setup-an-ai-server-homelab-beginners-guides-ollama-and-openwebui-on-proxmox-lxc/
▶️ Llamacpp Unsloth Video https://youtu.be/CBNbM-RZgIQ
📝 Llamacpp Unsloth Article https://digitalspaceport.com/how-to-setup-an-ai-server-homelab-beginners-guides-llama-cpp-on-proxmox-9-lxc/
▶️ 📍YOU ARE HERE📍 vLLM Video
📝 vLLM Article https://digitalspaceport.com/how-to-setup-vllm-local-ai-homelab-ai-server-beginners-guides/
(Optional Guides)
▶️ VibeVoice 7b TTS Video https://youtu.be/t7WH-86Jtn0
📝 VibeVoice Article https://digitalspaceport.com/how-to-setup-vibevoice-ai-podcast-tts-podcast-engine/

QUAD 3090 AI HOME SERVER BUILD
GPU Rack Frame https://geni.us/GPU_Rack_Frame
Supermicro H12ssl-i MOBO (better option vs mz32-ar0) https://geni.us/MBD_H12SSL-I-O
Gigabyte MZ32-AR0 MOBO https://geni.us/mz32-ar0_motherboard
AMD 7V13 (newer, faster vs 7702) https://geni.us/EPYC_7V13_CPU
RTX 3090 24GB GPU (x4) https://geni.us/GPU3090
256GB (8x32GB) DDR4 2400 RAM https://geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) https://geni.us/PCIe4_Riser_Cable
AMD SP3 Air Cooler (easier vs water cooler) https://geni.us/EPYC_SP3_COOLER
iCUE H170i water cooler https://geni.us/iCUE_H170i_Capellix
(sTRX4 fits SP3 and retention kit comes with the CAPELLIX)
CORSAIR HX1500i PSU https://geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4, not needed for H12SSL-i) https://geni.us/SFF8654_to_SFF8654
ARCTIC MX4 Thermal Paste https://geni.us/Arctic_ThermalPaste
Thermal GPU Pads https://geni.us/Kritical-Thermal-Pads
HDD Rack Screws for Fans https://geni.us/HDD_RackScrews

Ways to Support:
🚀 Join as a member for members-only content and extra perks https://www.youtube.com/c/digitalspaceport/join
☕ Buy Me a Coffee https://www.buymeacoffee.com/digitalspaceport
🔳 Patreon https://www.patreon.com/digitalspaceport
👍 Subscribe youtube.com/c/digitalspaceport?sub_confirmation=1
🌐 Check out the Website https://digitalspaceport.com
*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
*****

Видео Local Ai Server Setup Guides Proxmox 9 - vLLM in LXC w/ GPU Passthrough канала Digital Spaceport