i plan to buy the picrel (Beelink Ser 8 with 8745HS) to be used as my new home server for hosting adguardhome, mastodon/pixelfed, random static websites/self-hosting stuff, a meta search engine, and maybe open-webui with ollama and/or sunshine+moonlight. do any one of y'all think this mini pc is a viable and logical solution for any of this? yes, a thread died for this.
for a home server you shouldnt rely on something you cant easily upgrade and fix yourself. just build a micro atx build and use that. the size alone is not worth it
>>106522691ollama sucks on this thing, not even vram unless you run the tiny models>>106522735it's a good computer and you can upgrade ram / ssd. They also have a knock off mac studio (GTR9) that comes with 128GB ram, but I think it only gets like 5-10 tokens per second on larger models. My $1000 amd gpu gets 150tokens per second for reference. These things aren't really for AI.
>>106522691>around 500$Unless you really want it for the size a small desktop build is way better, will cost you the same and is more powerful >ollamaPerformance will be bad
>>106522691These things are made in third world sweatshops, you shouldn't buy this. https://www.youtube.com/watch?v=RWXI1xItuNcInstead, but something like System76 (USA) or Tuxedo (Europe) that hire local workers and pay them fair wage, while contributing to your country's economy.
>>106522735i'm already hosting most of the things i had mentioned on an rpi4 with 2gb ram. size alone is the sole reason i'm inclined to pick up this device as it will be sitting on top the router or be bolted onto the ceiling due to space constraints. i also need something that i can throw into my backpack before bailing off of my place. >>106522769>>106522790i thought i could just throw in a gguf quant. gemma 4b model along with open-webui and get a decent local chatbot. perhaps i can host the frontend (open-webui) on it, then serve the llm from somewhere else. >>106522805i'll definitely consider that, if they're selling mini pc's.
>>106522691good plan, OP. 780m is strong like ox. Zen4 cores will make the ladies wet
>>106523074That's what i do. I have llm stuff running on a gaming computer and just access it over the net on the beelink. It can definitely do gemma, but you'll have to change the settings in the bios so that the vram available is high enough. It's really fast, I would use this as my only PC if I didn't play the occasional PC game
>>106522805>third world sweatshopsAs far as factories go, this one is actually really clean and safe.>System76 (USA) or Tuxedo (Europe)They literally sell rebadged Taiwanese Clevo laptops
>>106523074>i thought i could just throw in a gguf quanti'm pretty sure you'd just be stuck with cpu inference
>>106523280>They literally sell rebadged Taiwanese Clevo laptopsTheir desktops and keyboards are all made in Denver though, and OP isn’t looking for a laptop.
>>106522805>this videowas so fucking beautifulshit was done in harmonyabsolute cinema
>>106523280BeeLine quite literally only does the casing as you can see in the video.>>106523809Yes, but I posted the wrong video where some pajeet stole the original and did a voiceover and somehow that comes first in the search results, not the original video. Original video is here https://www.youtube.com/watch?v=ohwI3V207Ts
>>106523829I have tinnitus so I never watch videos with garbage sounds so idgaf
>>106523074Mini pcs are in a weird spot for LLMs right now. They have the ram size to host large models that are prohibitively expensive to run on a bunch of gpus but the speed is so bad it's not worth it. You can run small models pretty fast on one but a 3060 can run the same model much faster for way less $.>gemma 4bDon't. Qwen 4b is the only <12B model I would consider coherent enough to use as a googler replacement but it's pointless when you have >32GB RAM, don't run a 2GB lobotomized model on it.>perhaps i can host the frontend (open-webui) on it, then serve the llm from somewhere elseikllama backend and an IQ quant of GLM 4.5 Air. You'll get between 5 and 10 t/s until you start hitting 4k+ context then it'll slow down a lot.
>>106524813so, i should just forget about running an llm on it thats what you are saying. i have a desktop with a 3090, which is what i plan on using for serving the llm, that i do not feel comfy exposing to the internetz, hence the beefy mini pc for home server. also i could off-load models that come with open-webui, as well the kokoro-tts that i run for, well, tts. (i'm aware of edge tts.) are those smaller models that bad? i have seen people praising them in the local llm threads.
this is fucking abysmal. llama.cpp@brew + gemma 4b all running on the 5500u laptop. smokeless uma'd 8745HS with gpu+cpu split should get better rates, right? prompt eval time = 14662.54 ms / 11 tokens ( 1332.96 ms per token, 0.75 tokens per second) eval time = 65528.56 ms / 45 tokens ( 1456.19 ms per token, 0.69 tokens per second) total time = 80191.10 ms / 56 tokens
>>106526757>i have a desktop with a 3090, which is what i plan on using for serving the llmThat's a much better plan. I do basically the same thing with a 7900xtx and 64GB DDR5.>beefy mini pc for home serverIf I were you I would do the opposite, make the desktop your server and run the mini pc as your workstation. Unless you still want gayming then I would just run the desktop 24/7 and switch the model off when you game.In theory minipcs make good servers but in practice they're too limiting and you grow out of it quickly.>kokoro-ttsI run this on a separate machine and it's pretty good for streaming TTS. There's some good voice stuff that came out in the last couple weeks so keep an eye out for VibeVoice streaming or the new IndexTTS.>are those smaller models that bad?It's a 60hz vs 144hz monitor situation. You don't realize how bad it is until you try out the better option, then you can never go back. 4B is lobotomized compared to 32B, which is lobotomized compared to the 100B+ models. The main issue is you CAN run the 100B+ models, just very slowly. You're planning on using the only affordable hardware for large models for a tiny ass 4B model I run on my phone.>>106527317>5500uKek. On top of not using ik_llama.cpp you're running a 4 year old mobile cpu with DDR4 RAM. Check picrel and you'll see why it's not really a useful comparison.
>>106522805>post the video at a literal who channel>no subscribers, no comments and barely any views>steals the original which in turn has 4,4M views, lots of positive comments about the process, and even a pinned comment from the official Beelink yt accountAnon, don't shy away, you "made" a great publicity for that brand, good luck with your channel...Here's the original for anyone interested: https://www.youtube.com/watch?v=ohwI3V207TsIf you search for "mini pc factory tour" on yt, is literally the first result that pops, and there aren't that many others of this kind (all from China).How on earth the fact that they got all the balls to show the entire process, even to the smallest details, speaks bad of the brand??.I'm not a fan of Beelink, but it would be nice if other well known brands on the mini pc biz did the same (like Asus, Asrock, MSI, Gigabyte -even System76 sells mini pcs now-), so we have something to compare...
>>106523829Well, fuck me, I didn't see this one.Sorry? i guess? Lmao
>>106523829>BeeLine quite literally only does the casing as you can see in the video.>quite literallyDid we see the same video? They put the entire thing together for the motherboard, like any other manufacturer in the world does. Are you, in any case, complaining because they didn't make the PCB, CPU, capacitors, resistors, soldering paste, screws, nuts, etc? Do you think any big motherboard brand does? My god, what a spineless retard lmao.
