Skip to content

Commit 52cdd26

Browse files
committed
Reorganized images in AI server build post with comfyui.png as hero image
1 parent 9c50c93 commit 52cdd26

1 file changed

Lines changed: 11 additions & 14 deletions

File tree

_posts/2025-10-11-ai-server-build/2025-10-11-ai-server-build.md

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,17 @@ tags: ai hardware server gpu
66

77
# 🧠 Why I Built This
88

9+
[![comfyui.png](comfyui.png){: width="700"}](comfyui.png)
10+
911
So I decided to build my own AI server. Why? Well, a few reasons.
1012
1. As part of de-googling and trying to host things away from big corp servers, this seems like the next step as we start using them more for AI services. Keeping data close to you, especially if you start talking to LLMs is worth it, IF you have the right security.
1113
2. Cost-wise you can't beat the price for the larger models in the cloud (at the moment). But strangely you can do some things that are quite expensive for their cloud versions (see more below)
1214
3. Plus, who doesn't love a good hardware project? 🤖
1315

14-
[![server_parts.jpg](server_parts.jpg){: width="500"}](server_parts.jpg)
15-
1616
The goal was simple: build a powerful AI inference server that could handle local LLM serving, fine-tuning experiments, and general ML workloads without breaking the bank.
1717

18+
[![server_parts.jpg](server_parts.jpg){: width="500"}](server_parts.jpg)
19+
1820
## 🧩 Hardware Selection
1921

2022
After plenty of research and part swapping, here's the final configuration:
@@ -91,13 +93,12 @@ After these steps, the system finally booted up! But I discovered another issue
9193

9294
Fortunately, I was able to resolve this by entering the BIOS and disabling the monitor connection requirement. This would allow the server to boot without a display attached, which was crucial for my intended use case.
9395

94-
<!-- Placeholder for first boot image -->
95-
<!-- [![first_boot.jpg](first_boot.jpg){: width="500"}](first_boot.jpg) -->
96-
9796
Those first few moments when everything finally works are always the most satisfying part of any build.
9897

9998
# ⚙️ Operating System & Software Stack
10099

100+
[![ai-server-portainer.png](ai-server-portainer.png){: width="500"}](ai-server-portainer.png)
101+
101102
The system runs a lightweight Linux environment configured for **containerized AI workloads**.
102103

103104
- **OS:** Ubuntu Server 24.04 LTS
@@ -108,8 +109,6 @@ The system runs a lightweight Linux environment configured for **containerized A
108109

109110
The containerized approach makes managing the entire stack much simpler and allows for easy updates and modifications.
110111

111-
[![ai-server-portainer.png](ai-server-portainer.png){: width="500"}](ai-server-portainer.png)
112-
113112
## 🧩 Container Setup
114113

115114
After installing the base Ubuntu Server, the first critical step was installing the NVIDIA drivers natively on the OS. This is absolutely essential before attempting to set up any Docker containers, as the containers need to access the GPU through the host system's drivers.
@@ -206,7 +205,7 @@ volumes:
206205
207206
What I love about Open WebUI is how it offers a ChatGPT-like interface but for my local models. The setup was quick, and it automatically discovered my Ollama instance thanks to the container networking.
208207
209-
[![webui_interface.png](webui_interface.png){: width="500"}](webui_interface.png)
208+
[![webui_interface.png](webui_interface.png){: width="700"}](webui_interface.png)
210209
211210
## Performance Testing
212211
@@ -224,9 +223,7 @@ Over time, I've built up a nice collection of models that balance capabilities w
224223
* **Llama3.2 3B** - Lightweight but surprisingly capable for simple tasks
225224
* **Qwen3 4B** - Another compact model great for routine interactions
226225
227-
<!-- Placeholder for performance stats image -->
228-
<!-- [![performance_stats.png](performance_stats.png){: width="500"}](performance_stats.png) -->
229-
[![single-prompt-netdata.png](single-prompt-netdata.png){: width="500"}](single-prompt-netdata.png)
226+
[![single-prompt-netdata.png](single-prompt-netdata.png){: width="700"}](single-prompt-netdata.png)
230227
231228
### Loading Times
232229
@@ -242,15 +239,15 @@ What's impressive is being able to run these models without the latency of API c
242239
243240
## Power Consumption
244241
245-
[![power_meter.png](power_meter.png){: width="500"}](power_meter.png)
242+
[![power_meter.png](power_meter.png){: width="700"}](power_meter.png)
246243
247244
This thing is hungry. According to my Netdata dashboard, the GPU alone can draw over 400W under full load when running the larger models. The RTX 3090 is definitely power-hungry, but considering the computational work it's doing, it's impressively efficient compared to running these workloads in the cloud.
248245
249246
I've found that different models have different power profiles. For instance, I'm using Granite specifically for meeting summarization tasks, and it offers a nice balance between power consumption and capability for this specific use case.
250247
251248
## Thermal Performance
252249
253-
[![temps.png](temps.png){: width="500"}](temps.png)
250+
[![temps.png](temps.png){: width="700"}](temps.png)
254251
255252
So far, temperatures are looking good! The combination of large case fans and the substantial cooling solution on the RTX 3090 itself is handling the thermal output effectively. Since the AI workloads I'm running tend to be bursty rather than sustained, the system has plenty of time to cool down between intensive operations.
256253
@@ -304,7 +301,7 @@ This is definitely a topic I'll revisit in a future blog post. The ethical consi
304301
* Local AI is incredibly satisfying when it works
305302
306303
Here's a closer look at those 180° power adapters that saved the day:
307-
[![surgery_build.jpg](surgery_build.jpg){: width="500"}](surgery_build.jpg)
304+
[![surgery_build.jpg](surgery_build.jpg){: width="700"}](surgery_build.jpg)
308305
309306
# Cost Analysis
310307

0 commit comments

Comments
 (0)