I Stress Tested a Micro PC with LLMs - Watch It Struggle

Can a tiny office PC handle the heavy lifting of local AI and Home Automation without breaking a sweat—or your electric bill? In this video, we deep-dive into the thermal and power performance of the Dell Optiplex 7060 Micro while running local LLMs and Whisper.

I’ve been using Dell Optiplex Micros for years to run my Node-RED instances, but adding local AI processing like Ollama and Whisper brings a whole new level of load. I’ll show you a custom Node-RED dashboard I built to track individual CPU core frequencies, fan speeds, and real-time power consumption. We also uncover a strange hardware "rabbit hole" where a simple power adapter issue was secretly throttling my entire system performance.

Key Takeaways:
- Idle vs. Load: See the surprising 5W idle power consumption compared to 100W peaks during AI tasks.
- Thermal Testing: How the 7060 manages heat and fan noise during intense Whisper transcriptions.
- The Throttling Trap: How to use the BIOS event log to diagnose power supply handshake issues.
- Node-RED Monitoring: A walkthrough of the Linux commands and MQTT setups used to build the performance dashboard.

Node-Red flow: https://github.com/nygma2004/km/blob/main/nr_load_test.json

If you like what I do here, you can support me with Youtube Thanks, or Memberships:
One time donation via PayPal: https://paypal.me/trainsmoney?country.x=HU&locale.x=en_US
Join this channel to get access to perks:
https://www.youtube.com/channel/UCdRpPAsQbIaVpn_vS2_omPw/join
TikTok channel most on trains, big and small: https://www.tiktok.com/@frontendfriday
And same in Instagram: https://www.instagram.com/hamptongardenrailway/

Chapters:
0:00 Intro
2:05 Base Performance
7:07 LLM Load
13:07 Node-Red Flow
21:16 Load Test Flow
23:49 Power Supply Rabbithole
27:02 Outro

Welcome to a new video in my home automation series and in my latest AI video, well it's not really an AI video, it's a new PC that I set up to play with local ai, with some local llms. The video showcases a new PC setup for experimenting with local LLMs, specifically a Dell Optiplex 7060 mini pc. The demonstration includes monitoring system performance metrics such as core MHz, power consumption, fan speed, and temperatures through a web interface, making it easy to run llm locally.

Видео I Stress Tested a Micro PC with LLMs - Watch It Struggle канала Csongor Varga