Ollama NVidia GPU Speed Test Comparison of RTX 4090, Tesla P40, A100 SXM 80GB, RTX 6000 Ada 48GB

This video shows a comparison of four different priced NVidia graphics cards when using Ollama, RTX 4090 24GB, Tesla P40 24GB, A100 SXM 80GB, RTX 6000 Ada 48GB.
In order to always maintain the same boundary conditions despite different VRAM sizes, two GPUs and the ollama model mixtral:8x7b-instruct-v0.1-q6_K were used in all comparison tests. Mixtral:8x7b-instruct-v0.1-q6_K fits exactly into 2 x 24 GB GPU, i.e. Tesla P40 and RTX 4090.
The video will show that even a relatively inexpensive Tesla P40 is well suited to running simple models with Ollama. The P40 is sometimes available for under $200 / €200 and if you buy two, you have a gigantic 48 GB of GPU memory.

In the video you can read RTX A6000 Ada. RTX 6000 Ada would be correct.

Видео Ollama NVidia GPU Speed Test Comparison of RTX 4090, Tesla P40, A100 SXM 80GB, RTX 6000 Ada 48GB канала Rey-Tec