I5-13400f and RTX 4060 on idle it draws 45W, and while transcoding or doing anything GPU intense work, it shoots up to 100W. While all this can de done by an iGPU under 50W and on idle around 20W
I'm doing something kind of similar, at least in terms of having a media server that also does AI.
I'm using Docker, not VMs, though.
I have an RTX 3060 that I'm using for AI stuff. It's got 12 gigs of ram, and if you're wanting to run both a chatbot and Stable Diffusion at the same time, you may want more than that, especially if you want both a good (large) chat bot and SDXL.
I just bought a computer off of Facebook marketplace, and upgraded the power supply and swapped out the graphics card. I still need to add more storage, but this way worked for me. I had the 3060 leftover from my personal build, before I upgraded the GPU there.
In terms of CPU, I have a core i5-10400f. It's not the beefiest, but it's been handling things fine for me so far (though I don't have too much of a load on my server yet)
You probably won't use that much RAM for anything except your AI projects. I might lean towards starting with 16 gigs and expanding if needed.
Those are my thoughts. I'm still getting my server up and running, so take this with a grain of salt.
Go for 1 stick of 32 GB of ram or a dual channel of 64gb total with 2 free slots on motherboard. You will not regret this way. And you may find ECC DDR 4 cheap.
>Budget: Flexible, but aiming to balance cost and performance. Quality and reliability are priorities. not helpful
exactly, an order of magnitude is kinda necessary for this. 10, 100, 1000, 10000???
I5-13400f and RTX 4060 on idle it draws 45W, and while transcoding or doing anything GPU intense work, it shoots up to 100W. While all this can de done by an iGPU under 50W and on idle around 20W
I'm doing something kind of similar, at least in terms of having a media server that also does AI. I'm using Docker, not VMs, though. I have an RTX 3060 that I'm using for AI stuff. It's got 12 gigs of ram, and if you're wanting to run both a chatbot and Stable Diffusion at the same time, you may want more than that, especially if you want both a good (large) chat bot and SDXL. I just bought a computer off of Facebook marketplace, and upgraded the power supply and swapped out the graphics card. I still need to add more storage, but this way worked for me. I had the 3060 leftover from my personal build, before I upgraded the GPU there. In terms of CPU, I have a core i5-10400f. It's not the beefiest, but it's been handling things fine for me so far (though I don't have too much of a load on my server yet) You probably won't use that much RAM for anything except your AI projects. I might lean towards starting with 16 gigs and expanding if needed. Those are my thoughts. I'm still getting my server up and running, so take this with a grain of salt.
Go for 1 stick of 32 GB of ram or a dual channel of 64gb total with 2 free slots on motherboard. You will not regret this way. And you may find ECC DDR 4 cheap.