Oriental editionprevious >>109909067READ THE (temp)WIKI! & help by contributing:https://igwiki.lyci.de/wiki/Home_server/hsg/ is about learning and expanding your horizons. Know all about NAS? Learn virtualization. Spun up some VMs? Learn about networking by standing up a OPNsense/PFsense box and configuring some VLANs. There's always more to learn and chances to grow. Think youโre god-tier already? Setup OpenStack and report back.>What software should I run?Install Gentoo. Or whatever flavor of *nix is best for the job or most comfy for you. Jellyfin/Emby/Plex to replace Netflix, Nextcloud to replace Googlel, Ampache/Navidrome to replace Spotify, the list goes on. Look at the awesome self-hosted list and ask.>Why should I have a home server?De-botnet your life. Learn something new. Serving applications to yourself, your family, and your frens feels good. Put your tech skills to good use for yourself and those close to you. Store their data with proper availability redundancy and backups and serve it back to them with a /comfy/ easy to use interface.>Links & resourcesCool stuff to host: https://github.com/awesome-selfhosted/awesome-selfhostedhttps://reddit.com/r/datahoarderhttps://www.reddit.com/r/homelab/wiki/indexhttps://wiki.debian.org/FreedomBox/FeaturesARM-based SBCs: https://docs.google.com/spreadsheets/d/1PGaVu0sPBEy5GgLM8N-CvHB2FESdlfBOdQKqLziJLhQLow-power x86 systems: https://docs.google.com/spreadsheets/d/1LHvT2fRp7I6Hf18LcSzsNnjp10VI-odvwZpQZKv_NCISFF cases https://docs.google.com/spreadsheets/d/1AddRvGWJ_f4B6UC7_IftDiVudVc8CJ8sxLUqlxVsCz4/Cheap disks: https://shucks.top/ https://diskprices.com/PCIE info: https://files.catbox.moe/id6o0n.pdf>i226-V NICs are bad for servers>For more SATA ports, use PCIe SAS HBAs in IT modeWiFi fixing: pastebin.com/raw/vXJ2PZxnCockpit is nice for remote administrationRemember:RAID protects you from DOWNTIMEBACKUPS protect you from DATA LOSS
Reposting here:>>109969599I am also looking into noise reduction.I am thinking if I could maybe use a cooler that is 90% passive and only spins the fan up if necessary.Are there any good ones? Is motherboard support necessary?
>>109973252The cooler is the noisiest part of your system?
>>109973151hello friends, i have been setting up my home server and have been learning various sysadmin tools like ansible and more recently nix and terraformwhile i see the appeal, it seems to me that these tools are basically programming, but written in these weird formats like yaml and hcl, which poorly reimplement some existing language features (like functions, loops, conditionals, templating etc etc) and cannot be statically analyzed and debugged as they would if they were real programs.why is this the case anyway? is it because the target audience is nocoders or is there something i am missing? I see many people have the same complaint but there doesnt seem to be a solution except maybe guix system which is extremely niche as a tech
I have the cutest homelab on /g/
>>109975243>cenmate hdd baymore like cremate hdd bay
>>109975243I don't find shitty HDD caddies particularly cute.
>>109975555>>109975632
here's your flash storage bro
>>109975889now it's blocking all zpool/zfs commands indefinitely with uninterruptible sleep, even on unaffected pools, epic kernel design, maybe Tanenbaum was right all along
What would be a good double NAS backup (one at home one at an office duplicating the first) for a regular user?If it was me I would probably just use TrueNAS and setup replication to the remote via DDNS or some other tunnel with a couple miniPCs if its a small amount of stuff.But I'm looking for a more casual user friendly idea. Synology I've presonally avoided because I don't like hardware raid which I know they had in the past which can screw you if a part craps out with no replacement, but I've not looked at them in a long time. UGREEN looks promising as well. But I don't know what would offer a good simple photo/document solution that easily syncs between locations. Trouble with doing things the very manual custom way is I forget what its easy for others.
>>109973151Why the fuck would anyone install retard memeshit like gentoo instead of proxmox
>>109975975rebooted, picrel was with copies=3, it still couldn't import the pool. Good job ZFS, really putting those extra copies to good use.After recreating, syncing the data back and scrubbing, there are no errors.I'm not sure if the flash itself was the problem or the USB controller decided to shit all over my pendrive and corrupt it.All of this for 5MiB worth of data.
>>109975209>why is this the case anywaywhen you need to deploy 50 virtual machines each one with specific defaults like software installed and permissions and so on, you just fill out a recipe for each one or two or all of them.this is not like cloning where you always have the original multigigabyte static vm, it's the opposite: you can dynamically modify recipes and remotely deploy each time according to whatever requirements you need. you don't need Nginx on your machines? delete that ansible role (one or two lines of text) and you're done.
> poorly reimplement some existing language features (like functions, loops, conditionals, templating etc etc)Welcome to declarative vs imperative patterns, anon. Feel free to implement the whole checksum/indempotence/logging layer checks while you're at it. Should be fun.
>>109975243homelab more like cutebussylab
>>109975243it is cute
do you encrypt your drives?I always procrastinated but I'm thinking about encrypting my encrypted drives. It's just that I have to format them and not sure I even have a plce to offload, and also I have to change my wireguard config so I can unencrypt on reboot, and I heard a long time ago about possible issues and that it wasn't that good compared to mac disk encryption (is this true?)
>>109976899# Algorithm | Key | Encryption | Decryption aes-cbc 128b 65.5 MiB/s 74.0 MiB/sit's over
# Algorithm | Key | Encryption | Decryption aes-cbc 128b 65.5 MiB/s 74.0 MiB/s
My homeland is an rpi3b with all her holes air tight with dongles and externals.I'll wait for the ai imgens
>>109976948did you disable bluetooth and wifi
>>109977077No. I'm using both.
>>109973151is freebsd a selfhosting meme? i learned about it and that led me to take interest in selfhosting but would linux just be a better option? i already have experience on linux but maybe it's fun to learn something new?
>>109975243oh hey I have the same night stand from ikeaI'm too poor for the rest of the things in it tho
>>109977332That's not your night stand anon, I thought it was you at first too.
>>109976222zfs raid on pendrives?
>>109976222USB shits itself all the time. Never trust it for critical data unless you have easily accessible backups.Copies=2 or 3 doesn't really help things when your controller up and fucking dies. It's more of single disk protection against bitrot or sectors going bad. If your drive has a disconnect while writing, all copies could be fucked up somehow. A lot of USB thumb drives don't actually flush syncs "correctly" so ZFS might think that a sync happened, write a new uber block, and then that block gets written without the underlying stuff below it being written.I've "recovered" a lot of fucked up drives like this by rolling back to a previous uber block, but sometimes pen drives are just trash.
>>109977143pfsense and opnsense are freebsd derivatives. There's nothing wrong with using freebsd for stuff, but most general tools are more readily available/supported on linux these days.For 99% of stuff just install proxmox and have fun. Proxmox is in some aspects bloatware, but it's quite powerful and is a decent way to learn. It makes spinning up containers/vms super trivial.
>>109977373I've mirrored to thumb sticks before when I had a boot drive die and was waiting on a replacement. Went from a 2 way mirror of SSDs to a 3 way mirror with 2 thumb sticks in it. It works, but they hiccup often enough that ZFS tends to get rather angry, and really bad ones can shit themselves violently enough that it refuses to work.I've got some old 256MB ones that will hang for multiple seconds every 64MB written to them. Most file systems hang with the drive, but ZFS will treat them as dropped. It's similar to how it drops disks when using SMR drives.
>>109976899>encrypting my encrypted drivesWhat? Why?
>>109977607>t makes spinning up containers/vms super trivial.>Be me>Be porting a AWS terraform project for on-prem local dev>Need to create 2 VMs, API and data storage>Oh, you want a vlan? Better spin up a virtual bridge network first.>Oh, you want to spin up Linux VM? Better get a cloud-init capable image to set the network variables first>Oh, you want to cloud-init to call your PID1 and hand it off to Proxmox? Better install QEMU after cloud-init runs.>Oh, you want QEMU? Better install it off your package manager?>Oh, you want to run your package manager? Better get a OPNsense as your vmbr1 gateway or no wider internet>Oh, you want OPNsense? Too bad, they don't offer it in proxmox_virtual_environment_vm. You have to bash script a bz2 download/unzip/proxmox_virtual_environment_file/import
>>109977694my external drives
>>109977749all of my drives are encrypted, even the empty ones, and ones with hardware faults that are used as coasters (looking at you 4TB 870 Evo).
I have a PC dedicated for the occasional gaming and another for my jellyfin server. whenever I want to install a game onto my gaming PC from the 36TB raid0 btrfs nocow pool (shucked WD 12TB HDDs x3) on the server, I do it over the network. 2.5Gbps to 2.5Gbps via NFS. how am I doing and is there a better (faster) way?
>>109977089>uses both Bluetooth and WiFingmi
>>109977779>He thinks his files are protected because his drives are encryptedRetard. /g/ has fallen
>>109978814Oh yeah? Do you have a quantum computer on lease to break through encryption?
>>109978846Fun fact. It's all decrypted after boot-up.
>>109975209HCL is okay when you get used to it. I started using it at my previous job, coming from a Python backend background. But in any case, it's so much fucking better than trying to manually orchestrate your shit on AWS.
>>109976899>do you encrypt your drives?No, I'm too much of a little bitch.
>>109973151ni gan mah balls
>>109978375>is there a better (faster) way?
blogposting with stuff I've accomplished this weekend:- built a few custom RSS-Bridge bridges, turning a few websites into RSS feeds with the help of chabbytee- set up joplin-server to keep my joplin stuff in sync, still need to work out e2e + if it's even needed in a selfhosted context- set up planka again, because my brain thinks in terms of kanban boards rather than in to-do lists, and there's nothing better out there- set up a halfway decent docker management stack--watchtower for managing image updoots every day at 4am--ntfy to then send blorps to my phone with cutesy THIS WORKED / THIS IS FUCKED reports--uptime kuma to keep an eye on DSM, my router, and Google + Cloudflare DNS, and hopefully to keep an eye on my docker containers & Duplicati backups once I can be bothered--with all the docker molesting going on through linuxserver's socket-proxy container, thus minimising the risk of my NAS inadvertently becoming a botnet command/control node- next up i'm going to look at threading ntfy into more of my stuff: i'm thinking my torrents and Duplicati backupsi've got a few hundred physical music CDs that i'd like to catalog, like with cute Discogs integration and stuff, but to my surprise there's very little out there that'll do so. everything i've investigated thus far requires mongoDB which apparently doesn't work on my DS918+. yes this is an ideal use case for mymusiccollection.ods but still
>>109979081>too much of a little bitchwait am I a little bitch too?
>>109979363kys faggot. this is a self hosting thread. not the tranny "I need a man" version
>>109973565The 3.5" HDDs are spun down most of the time.The ones that are running 24/7 are SSDs and 2.5" HDDs, which are quite inaudible.I have 3 fans in my system, so yeah, they account to 90% of the noise.Maybe I need to remove one and just leave the CPU one and another for the case.
>>109975243That's cute.Rate mine.
>>109978375If you are already saturating the 2.5 Gbps connection, I don't see how you can accelerate it any further.>>109979363How is that any faster than fetching the games from your own network?
>>109979867what are you hosting on that thing
>>109977373no raid, just a single block device zpool for each pendrive so that i can detect corruption.>>109977600yeah, thankfully all my important data is on HDDs on HBA in raidz1 on ECC system, plus a separate raiz1 HDD offline backup.It's crazy how untested and ridden with issues all this consumer garbage is: USB NICs, USB flash, USB hubs, USB KVM switches, PCIe SATA controllers, i've had issues with every single one of them.I though that at least for 5MiB worth of gpg/passwords backup on flash it would be fine, turns out it can't even handle such a trivial usecase.
>>109979894>How is that any faster than fetching the games from your own network?i meant to say vibecode something like Steam but for your usecase.like for example https://github.com/LANCommander/LANCommander
>>109980344>I though that at least for 5MiB worth of gpg/passwords backup on flash it would be fine, turns out it can't even handle such a trivial usecase.It can work if you buy decent drives. Some of it's also going to be down to the USB implementation on whatever system you are plugging the drives into. Some motherboards have really terrible USB controllers that are daisy chained together and go through the chipset. Regardless, honestly, flash drives are a terrible long term backup solution. Most of them don't have any form of rewrite baked in so they'll eventually corrupt stuff if it sits in a drawer for a decade and leaks charges into/out of the flash.I used to use dvd-ram back in the day. Not all drives play nice with those disks, but dvd-ram is more or less exactly what you want for an archival medium. The disks are insanely durable. They're shelf stable more or less forever. You can slip them in an envelope or thin case and then throw it in a safety deposit box. I kept my business records and a handful of sentimental stuff on a handful of dvd-ram discs in a safety deposit box. I'd keep one with records in the box and rotate it with a different disk with fresh backups every month when I went into the bank for mortgages and other stuff.
>>109980344>>109981224These days I have offsite systems pulling backups. One at a relative's, and a couple at some duplexes I own. I provide internet as part of rent since it's a lot simpler for me to just maintain an account and I don't want cable techs drilling random holes in my foundations. Plus, I have access to the common area cameras so it's easy to tell when someone moves their drug dealer in against the lease terms. All the tenants know is that there's a locked closet in the basement with "the internet" in it, and that they can get Wi-Fi from anywhere even vaguely near the property. I've never met a tenant in real life that gives a fuck about the how. All they care is that the Wi-Fi works, and with 3-4 WAPS properly configured it Just Werksโข.Anyways, I'm rambling, but the point here is that it's really trivial to throw some tiny shitbox computer in a closet at some remote location you have access to and setup proper backups. Modern internet connections are fast enough and consistent enough that it's safe, and it's pretty trivial to automate. I don't think it's worth doing thumb drive backups because you have to remember to actually do them. Don't get me wrong, I still do dvd-ram backups of my tax records, but that's because I can give my lawyer a disk with everything if something happens. For most people that kind of stuff is a waste of effort.
>>109981236>Modern internet connections are fast enoughlol, if you're luckly to have one, i've only moved from ADSL to LTE a year ago, and LTE is asymetric pretty much in the same way ADSL was (order of magnitude lower upload that download).I don't have a problem with proper backups though, 5x 16TiB RAIDZ1 in a box serves me very well as my offline backup i sync once a month. I only needed some other medium for my gpg/passwords just in case (since my NAS, backup, boot drives are encrypted too etc.).Sadly optical media won't do, as i need to update those gpg/passwords every month, unless i want to go through 24 disks each year.
>>109973151any of you anons run offsite backups? is backblaze still a viable strategy in the big 2k20+6?
>>109981418'duplicati backing up /whatever to backblaze' is yet to be surpassed in terms of simplicity and reliability imo. there's a lot of room for improvement, of course, but unless something goes really peculiar i don't see it happening
>>109981371>Sadly optical media won't do, as i need to update those gpg/passwords every month, unless i want to go through 24 disks each year.Hence why I said dvd-ram. It's not your standard optical disk. It behaves more like a conventional hard drive than an optical disk. You can write and rewrite to dvd-ram without wiping the entire drive, and they show up in the system as a removable drive, not an optical one. They were used in DVRs and security camera systems for a while as well.https://en.wikipedia.org/wiki/DVD-RAMhttps://www.youtube.com/watch?v=ecH3OU0R4ls
12x 4TB IronWolf 3.5" drives for $90 each. Good deal? โ >>109967760The 10" racks are nice in theory because they can sit atop your desk and you still get all the modularity.
>>109981418>is backblaze still a viable strategy in the big 2k20+6?I'm using it as a disaster recovery plan. Never had to resort to it since it's the backup of the backup.
>>109978875>decrypted after boot-up.None of my drives decrypt without a password, what are you talking about?>>109981787>The 10" racks are nice in theory because they can sit atop your deskWhy the fuck would I want that? So I can complain about the noise here and ask for dumb suggestions to make it quiet?
>>109981787>12x 4TB IronWolf 3.5" drives for $90 each. Good deal? โ nah. at most 800 for all if you get them from the same guy. maybe stretch to 70 a piece.
>>109981787>12x 4TB IronWolf 3.5" drives for $90 each. Good deal? โ eh, if new, with current rates it isn't bad, but they're used, correct? HDDs have gone up 3x in price in the last couple of years, so keep that in mind. If you don't need all that space, may be better to just buy what you need now, and expand later if you're on a budget.
>>109981418I have like 350gb of my most important stuff in b2, i think 30 bucks a year. I might cancel and just use the free onedrive 1tb
>>109979929I've replaced it, but at the time it was hosting Nextcloud, torrents, Navidrome, Jellyfin and network shares.Maybe some other stuff I'm forgetting.
>>109981046Ah! Now that's cool!That's for the tip.
>>109981997>>109981929I was gonna buy a 8TB IronWolf for $325 otherwise but it'd be new. More expensive per TB though
>>109982091not bad at all for such a small footprint
>>109982044>onedrivewhy would you do this to yourself?>>109981829how much data roughly, my nas has a couple of terabytes but i dont really need to back up security cam footage>>109981594ty
>>109982630>how much data roughlyI have 19.4 TB on Backblaze.
>>109982630>why would you do this to yourself?its free and im poor right now
>>109979440on what hardware
>>109983142Synology DS918+, everything's running via Docker
>>109981893>Why the fuck would I want that? So I can complain about the noise here and ask for dumb suggestions to make it quiet?If you're uncertain of what your future arrangements will be, sticking with something that can always live atop your desk or even countertop ensures it'll almost be accessible. I don't want to go 19" racks and then end up living in a van.
>>109975209>while i see the appeal, it seems to me that these tools are basically programming, but written in these weird formats like yaml and hcl, which poorly reimplement some existing language features (like functions, loops, conditionals, templating etc etc) and cannot be statically analyzed and debugged as they would if they were real programs.That's why Guix does everything in Guile Scheme instead of some homebrowed domain-specific language or data format. You declaratively program your computer in one language instead of juggling a bajillion different ones.
post your racks
>>109984860I don't have one.
>>109982120i'd get 2x8 or 3x8 new for that budget and call it a day, not worth paying that much for used smaller ones. when drives had normal prices those 'dozen used 4TB drives' would just sit there for months and nobody would get them for a reason.
>>109976163Because people actually love to cause problems for themselves and then GET to complain about them. I have a proxmox cluster and I would never change it for any other hypervisor.
>>109987981Now I wait for his depression to kick in when he realizes most of the porn is lost since sites have been killing off old media.
So how do you guys keep track of your deployed system and service health, etc? I've mostly been a 'programmer' (non-professional) on a personal device, maybe at most doing some light SSH and syncing between devices, and only since getting into the NAS and whatnot stuff started needing more holistic network level views. It's been fun and I'm thankful for Nix/NixOS (formerly guix/sneeds) but it's also complicated and I feel like it's precarious with how much I have to remember right now. I'm trying to keep as much as I can in a 'gitops' style workflow so everything's declarative.
>>109984860I wish I had one. I just have a bunch of random old computers and my server and a coouple routers. I was going to try and build a dedicated opnsense ini pc and get a managed switch that supports openwrt (realtek switches are getting a ton of support lately, you should look into it if you don't want to use your switch's shitty oem firmware), but everytthing's too expensive
>>109989095at home i simply don't bother, if a system/service is down, i'll notice when i need to use it.at my job, i use prometheus blackbox-exporter + grafana alerts, you just list TCP, HTTP, ICMP endpoints to monitor them.>>109989216i just got a Zyxel XGS1250-12 flashed with OpenWRT (10x 1GbE, 4x 10GbE). I'm really liking OpenWRT so far, aiming to have a full OpenWRT network stack once i move (MF289F as my router+modem+AP, XGS1250-12 as my L3 switch, 2x hAP ac2 as L3 switch+AP).I've noticed work on some advanced hardware offloading support on Realtek in the last thread (>>109967431), sadly the one thing i wanted (MAC-based VLAN assignment) is broken.I don't know if i can recommend XGS1250-12 though, i bought a "broken" one for very cheap (just one of the 10GbE ports didn't work). The port pins look fine, disassembled the switch and the board looks clean as well, apparently 10GbE ports dying is a frequent issue on this model. Don't know if it's the controller they've used that's faulty (Aquantia AQR113c, later revision uses Realtek RTL8261N), or just a fault of something else on the switch.
>>109989287You could try your luck asking about that feature in this thread:https://forum.openwrt.org/t/support-for-rtl838x-and-rtl93xx-based-managed-switches/57875/4618There are at least two dudes working constantly on the target. It's one of the most active development threads on the site and the devs are very responsive.
>>109989332i don't think i'll bother them with my borderline usecase, especially since i've already solved the problem differenty (i set the port to untagged VLAN for AMT and tagged VLAN for OS, then tag the traffic on OS side).The development looks nice indeed, having L3 inter-VLAN routing support on the switchwould be very cool, i believe the hardware is capable of it.By the way, the stock Zyxel firmware neither supported MAC-based VLAN assignment nor inter-VLAN routing.So flashing even OpenWRT 25.12 without all those changes still doesn't loose you any functionality over stock firmware, i'm pretty sure.
>>109989287>at my job, i use prometheus blackbox-exporter + grafana alerts, you just list TCP, HTTP, ICMP endpoints to monitor them.I'll check them out, I've been reluctant to bring in external tools for bloat/complexity fears but it probably makes sense over rolling my own.I've been vibe coding up a storm today, I like the idea of dynamic graphs for registered and running services like pic rel so I can tell what's going on at a glance.
>>109989507>six most traded contractsit's fucking garbage, okay move on
>>109989533what?
>>109989554got it, you just post screencaps of ETF slop for no reason. no need to further reply back.
>>109989616lol why are you so mad? it's just showing price charts for the six most traded (0dte) contracts for SPY from databento. I'm just messing around getting a dashboard view for what I'm working on.
>>109989646>no need to further reply back
>>109989652got it boss thanks for confirming you're illiterate
>>109989646Monitoring an ETF basically means you are retarded. Their return rate is barely 2% if it is a good one. You seem to only hold poor and low performers. Generate your own portfolio. There I replied to you. But it is amazing you have build a monitor to watch your near zero growth.https://youtu.be/2IsR06zf8nQ?t=70
>>109989669yeah you don't know what you're talking about.. at all lol
>>109989675Seems to be the trope with you dims on this site, but it was amusing to watch you use the weakest cope of all when out of arguments: humour and poorly played confusion
>>109989694you apparently don't even know what an options contract is.. and the point in tracking SPY OPRA (options) is that it's the largest real time data feed in the market, so it's a good operational stress test.SPY is just the underlying, you can't infer what I'm doing from that alone.
>ssh into friend's server (used to share games and movies, we set it up years ago)>he runs some netstat command>suddenly my IP is resolved as "rlib.[name of my isp provider].com"why is that? why rlib?
>>109982622It was basically serving 2 people only, so it was enough.It had 8GB of RAM.
I've had one of these as a home server for close to 10 years now its done me well for a long time. The HDDs in it have about 80k powered on hours in a software RAID 10. Wondering if I should think about replacing these drives now? They're WD Red drives so made for this sort of thing, but I don't want to have to deal with a disk failure if that ever happens. I've been running smart long tests the last few days and as far as I can see the long test reports no issues.So should I think about replacing the drives or maybe thing about getting a new microserver? But honestly I doubt I'll find anything as good as pic related these days and this thing was really cheap when I bought it from HP too. I did manage to get some rack servers from work for free that they were throwing out, but I ended up donating them to a friend because that shit was too noisy to run anywhere in my house.
>>109990588holy fucking blog post
>should i...do whatever the fuck you want. holy shit. stop asking.
>>109989095>So how do you guys keep track of your deployed system and service health, etc?this is the explicit use case of uptime kuma. i use it to ping my wireless modem, my NAS hardware and Google DNS, and to fire a 'shit's fucked' notification to my phone if either of them aren't reachable after five retries. i probably won't ever need it, but, it's nice to have a dashboard that'll determine if my internet regularly goes down between 0300 and 0400 or something.small tip: use something like dokuwiki or joplin to note down what you're doing and why. i've been experimenting with all kinds of docker-related stuff lately, and it's nice to keep a log of what's set up (what ports are in use, all the config dirs I've specified, contents of the compose.yaml file) and what i've tried but haven't bothered keeping around. one of those 'what will I probably need to know in six months when I've forgotten why I did all this' things.
>>109990856>don't ever ask for advice
>>109989095idk but Zabbix was shit
So let me know what you think about this plan for my truenas setupCurrent: 2 X 18TB HDD's mirrored as main storage , 1 sata SSD for my boot drivePlanned additions: 2x 250gb sata ssds mirrored for applications, 1x 1TB nvme SSD for L2ARC (i know ram is better but my $80 stick is now $340), and 1x sata 500GB SSD for downloading torrents before being moved to HDDMain usecase is just a basic jellyfin/navidrome homeserver and torrenting box.im gonna need a PCIE to sata board and im also gonna throw in a fan cause my hdds are pretty warm
>>109975209>is it because the target audience is nocoders or is there something i am missing?In the case of ansible, consider the humble /etc/hosts file. In ansible, you can setup a class of 1-N servers that need a custom private NO-DNS /etc/hosts file where every time you add a new VM to the group, you make an addition to the group_vars and roll it out to each host.That shitty loop function you mentioned will go to each host assigned that role, and:>compile your /etc/hosts file off a jinja template, looping over the list of VMs that each need an entry in the file>Pull your deployment AES256 encrypted private key out of the vault>Load the key as your ssh-agent file>Open an ssh terminal to the box>Verify the file exists, creates if it doesn't>Does a diff between your src file and the dest file>Writes the diff>Returns the diff to you as part of the execution logs>Closes the connection>Deletes the unecrypted key>Writes the diff to your run changelogI could write all that, but that's a lot of fucking brittle-asd bash scripts calling python scripts calling linux applications, and I still don't have applications and hypervisors creating plugins for me.Terraform/Tofu is a whole other can of worms. The alternative there is a cluster fuck of HTTP requests to download image files and run hypervisor API endpoints that change every god damn minor version, or shit like AWS CDK in fucking TypeScript if you really hate your life.
Been switching to more power efficient hardware,And for the ones that can't be more efficient I wrote a script that watches a specified application. Tried having AI make one at first but it was dogshit and had a very limited scope. So I rewrote it.>3 minutes idle = +1 strike>10 strikes = 30 minutes>3 minutes of usage = -3 strikes
>>109991741>1x sata 500GB SSD for downloading torrents before being moved to HDDWhy not download directly to the mirrored HDDs? If it's fragmentation, make sure the BitTorrent client is configured to preallocate the space.
>>109992586Iโm not an expert but I read that you should turn off pre allocation when using ZFS or copy on write file systems which truenas uses and that a scratch ssd for downloading before moving to main hdd pool was better practice
redpill me on torrent clients with remote web interfaceswhich one is good which one is shit
>>109993080>deploy lscr.io/linuxserver/qbittorrent:latest>go to http://nas.IP.poop.fart:8080>download sissy AGP hypno vids 24/7>die alone
>Wake up>*click click click click click>what's that noise>*click click click click click oh god no>*click click click click click NOOOOOOOOOOOOOOOOOOOOODead drive :(Today will be very, very expensive
>>109993080>>109993616I use the vuetorrent frontend for qbit. It's worse in some ways to default, but it's usable from mobile
>>109993080I used transmission-daemon for a few days and its pretty good
>>109993080I use deluge because I've always used it. Had my local llm shit out some CSS so it stop burning my eyes at night, which you can have if you want.
>>109994185Doesn't mobile have a few remote Qbit apps?
>>109993080I have used Transmission and recently moved to qBittorrent because they told me it's the gold standard, but both served me well.
>>109990740Isn't this thread to share our home-server setups?
>>109993080I use qBittorrent. Has a useful API too.
>>109992962this. for seeding, an ssd is better if you're trying to avoid disk thrashing. you download to nvme or ssd and flush to HDD.
>>109975243t440p, 3 internal drives, 2 external, 1 google coral2 x 256gb internal SSDs are connected into 1 512gb btrfs volumehopefully someday component prices will drop and I'll upgrade to something more decentbut for now - it's good enough for me
>>109994816also running both zigbee and matter over thread to control all my ikea devices
>>109992962>>109994506So. For some reason you shouldn't preallocate nor download to a ZFS volume? What are you supposed to do with such a filesystem then lmao?
Hello guys. I got interested in home servers a bit after downloading docker and wsl2 on my personal computer and running penpot and wanted to ask, how big is the security risk if you opened up your server to the internet (for remote access to services) as a beginner? Do youtube tutorials make it simple to know exactly what you need to be mostly secure? How much effort does it take?
>>109997384My personal opinion is that AWS has a free tier for learning something you will likely lose interest in within a month. If not, then you can consider dealing with your ISP, routing/firewall tables, and the rest.
>>109997384>opened up your server to the internetyou cant, you're probably cgnatedtry wireguard
>>109997772>CGNated>wireguardHow is he going to get wireguard and working if he shares a public IP and cannot do port forwarding?
>>109997788Tailscale, whose very use case is dealing with that stuff while using Wireguard underneath.
>>109997801>UDP Hole Punching (STUN)Huh. Talk about some clever malicious compliance.
>>109997384I'm not comfortable with me personally punching a hole in my home network, and giving the wider internet at large a chance to get in. There's ways of managing down that risk, but it requires work and time that I'm not willing to deal with. I therefore use Tailscale to let my phone, PC, laptop and NAS to talk to each other: which, per the VPN acronym, is a virtual private network. So as long as you're logged into Tailscale on your server and on whatever device you're looking at, http://100.194.129.125:8096 will always get you into your Jellyfin instance, and because the traffic is encrypted, it's safe to use over public wifi or whatever. The trade-off is that you can't just tell someone to go to music.mydomain.com and have it Just Work right off the bat.
>"mom can we have IPMI?">"we have IPMI at home">IPMI at home:seriously, what a limited buggy piece of shit this is.>no dedicated interface, only shared with main NIC>can't tag VLAN on itself>wireless AMT is even more limited, has no static IP configuration when turned off, only DHCP, when booted is forced to share IP with OS>VNC requires a CPU with embedded graphics, and even if you have one, HDMI/DP must be plugged in for VNC to work>for some reason VNC + power on does not work on my X230, so i can't access BIOS remotely>Intel hard at work removing more and more features with each new AMT release (apparently newer ones don't even have VNC)
>>109995228>What are you supposed to do with such a filesystem then lmao?You're meant to post online about how superior it is.
>>109997384>how big is the security risk if you opened up your server to the internet (for remote access to services) as a beginner?Considerable enough that you shouldn't.
>>109997384>how big is the security risk if you opened up your server to the internet (for remote access to services) as a beginner?none if you do it right.
>>109998256>buggy piece of shithah, wait till you see AMD's "Dash"
Synology NAS>upload large file>99%...>100%.. >failed to complete
Synology NAS>renaming files>goes well>one file left>unable to perform this operation
Synology NAS>upload a file>"uploading from mounted volumes is not supported"
>synolologyyou had it coming, underneath DSM is a crippled locked down kernel from like 2013
>>109998840I'm not the one who bought it and we've had it since before 2013. Fuck this piece of shit.
are you using DSM to upload files to your NAS, and if so, why
>>109998902both dsm and the shitty drive client
>>109998256What's even the point?
>>109999170of what, AMT, or remote management in general?For remote management in general, it's super handy to be able to change BIOS options remotely, to mount OS install ISO and do the whole install process remotely, or even do stuff like typing in your root encryption password during boot if your server is encrypted. Or if you lock yourself out (by editing network config files, firewall, stopping ssh etc), you can easily fix it using remote management.The problem with AMT is that it technically supports most of it's stuff, but very poorly. If i connect to KVM/VNC on my offline Thinkpad X230 and then try to boot it, AMT locks up. It does not happen on my HP Z240, you can even mount ISOs remotely for install etc, but guess what, i got a Xeon without iGPU, so i can't use KVM/VNC. And even if i had iGPU i'd apparently have to use one of those dummy DisplayPort plugs to bring the iGPU up, because somehow AMT is not able to do it itself... At least i can still use Serial over LAN, already saved me once from fucked up network config.This and Intel constantly removing features, I've learned that older AMTs DID support VLAN tagging their traffic...
>>109994278sure. also what local llm you use if i may ask.
>>109993080qbittorrent-nox is the gold standard for servers. I use the devs' official docker image but you can use the linuxserver one too.
>>109997384If you open up the wrong thing there can be many issues. If you open up the right thing and that thing somehow gets compromised and has no segregation from the rest of your devices, there can be many issues. Ideally you should isolate the device you're opening to the internet using VLANs.
>>109992962This doesn't actually matter in practice, especially if you are using metadata specials with small blocks. Empty files compress extremely well, so it will just shove everything onto your metadata special.>>109994506This is wildly overstated information. You'll create fragmentation if you have tiny record sizes and your torrent client isn't set to download sequentially, but fragmentation with 1MB+ blocks is functionally non existent. You can download to a dataset that is heavily using small blocks to shove it onto your metadata specials and then have it moved to the main vdevs in the pool if you want. Alternatively, you can just run ZFS rewrite periodically to defrag stuff.Realistically you don't need to do any of these. It doesn't actually matter that much if your pool configuration isn't retarded. Most of the ZFS guides out there are repeating bullshit from a decade ago that has either been improved, or was never an issue to begin with, or they're conflating information for enterprise users that are slamming a pool with multiple 25 GbE links and dozens or hundreds of concurrent users. You as a homelabber are highly unlikely to even notice mild fragmentation on a media pool with large record sizes.
>>110000926now in English plz
>>110000926overcomplicating bullshit.downloading to a fast SSD/NVMe first and then moving (flushing) the completed files to slower HDDs is the golden standard for a home server, end of story
>>110000926niggah, seeding is random access. that's he hard drive killer. whenyou seed to multiple peers at onceand there all asking for different fragments of different files at thesame time the mechanical read-head inside a hard drive has to physically fly back and forth acrossthe spinning disk to find thosepieces and that's what diskthrashing is and what causes itthat shit creates massive lag slowsdown your whole system and wearsout the drive. doesn't occur in nvmeor ssddownloading and seeding activetorrents on SSD keeps all thatchaotic, random traffic off yourhard drives and when tge file is complete or you are done aggressively seeding, you move it to the HDDs
>>110001394Metadata specials massively reduce the IOP load on the RAIDZ vdev(s). You literally just add a metadata vdev, run zfs rewrite on the pool, and never have to give a shit about any of that ever again.You calling something the gold standard doesn't make it not retarded.>>110001441I've had ~200 TB of usable capacity in a RAIDZ pool with metadata specials for years now and never had any problems on roughly 150TB of media. You can push a gigabyte per second with 5 high capacity drives, and more obviously means higher. The way torrent clients actually work under the hood is you request a given piece from one or more peers and that request is broken into 16kB blocks. Thing is, those blocks are effectively requested sequentially within a given piece. On a large media torrent with 8, 16, 32, or even 64+MB blocks, you're effectively performing linear read operations within the confines of that multiMB range. Individual read requests are 16k, but if ZFS is pulling stuff out in 1-16MB operations, what actually happens is you do hundreds of read operations on a block that is already in the ARC.Keep your ZFS record sizes reasonably large for your bulk media and everything will behave close enough to linear that you won't notice problems. Sure, during writes, you'll get write amplification. When a torrent client flushes out a partial piece to disk because the peer providing it is slow, or on pieces that don't cleanly align with your ZFS records, you'll get amplification when individual sections are flushed to disk. So what? It's an utterly inconsequential amount in the grand scheme of things, and mechanical drives don't effectively have limited endurance. Do you have 25Gb fiber and have torrents that will go that fast when downloading them? No? Then stop worrying about tiny amounts of rewrites.
>>110002106>>110001441continued>Won't the write amplification cause fragmentation because of CoW and rewrites?Define fragmentation. A 100 Gigabyte 4k Remux being broken into MB+ chunks isn't "fragmented" in the sense that your performance will be torpedoed. You can still easily pull upwards of 100MB/s per disc when they're doing 1MB random reads. Set your record size so that your media is at least 512k per disc and you'll be fine.The only realistic way to have horrible fragmentation problems with torrenting is if you use small record sizes with RAIDZ because it breaks the records across drives in the vdev. If you're sticking with the default of 128k and you have 10 discs in a RAIDZ2, you're writing 16k to each of 8 disks for data, and 16k to the other two for parity. That absolutely can result in degraded performance, especially without metadata specials.If you're one of the retards that followed the old advice of using 16k records for your torrents, congratulations, you shot yourself in the dick. Assuming 4k sector size (the norm these days) on a 10 disk raidz2, each record only occupies 6 disks. 4 data disks get 4k of data, and 2 get 4k of parity. Now you do have a fragmented mess because you're doing 4k I/O and only hitting a fraction of your disks on every operation, plus you're massively increasing the amount of metadata lookup you have to do. If you don't have metadata specials, yeah, no shit your performance is going to be complete ass.