[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / r / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip / qa] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
Settings Mobile Home
/g/ - Technology


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


File: file.png (185 KB, 820x791)
185 KB
185 KB PNG
Just got off a call with my boss. He genuinely expect me to do the manual fix for each impacted machine. I am the only DevOps Engineer in my company. We have over 2,000 impacted machines. I told him that this was unreasonable and bordering on illegal as it would take me at least 48 hours straight just to mitigate the outage. He got extremely mad and cussed me out for like 10 minutes to just "do my job" and shut me up every time I tried to interject. I abruptly left the call because I started tearing up and have had 27 unanswered calls since. He's even calling me right now. I don't know what to do. He told me even if I quit or get fired I still have to fix this outage or else I'm liable for a lawsuit. I'm just sitting here shaking. What do I do?
>>
wagie wagie
>>
File: 1720434958034812.png (100 KB, 680x509)
100 KB
100 KB PNG
>>101469302
I'm sorry OP but you wanted the big boy job... time to close your eyes and swallow
>>
go fix it wagie while i watch anime
>>
>>101469302
Getting fired would be preferable I think because he doesn’t have a good reason - could collect unemployment and easily explain it to future jobs. It sounds like he is gonna be abusive if you try to stay till that happens though
>>
You're making him look bad to his boss, how dare you. You have just put your 3% annual raise in jeopardy.
>>
>>101469302
https://files.catbox.moe/frdmcm.mp4
eh it's something to do

just get to workin
>>
>>101469302
Sounds like he has no leverage and is panicked. Why don't you ask for $20k?
>>
>>101469302
>DevOps Engineer
>windows bare metal machines
Yikes
>>
You get paid overtime right? ... right?
>>
>>101469500
>>101469610
He said they can sue me for damages even if I'm fired. I'm getting calls from unknown numbers now I think he said something to the higherups. I don't understand how he can pin this all on me.

>>101469612
?

>>101469619
I don't have overtime. I'm expected to be on-call 24/7.
>>
You could just make a video of the entire process and send it off to to the wagies and tell them to do it while you work through the high priority ones
From what I've read it doesn't sound impossible for the average joe to do the fix
>>
File: 172138530404687915.gif (1.45 MB, 400x226)
1.45 MB
1.45 MB GIF
>>101469302
>>
>>101469679
Pretty sure they’ve got nothing if you didn’t do it on purpose or weren’t doing something grossly retarded
>>
>>101469690
this
>>
>>101469302
Get drunk and play video games. It's not your problem.
>>
>>101469690
He said it's my sole responsibility.

>>101469717
I told him it's a global outage and it's not my fault and he started yelling at me that I'm culpable for damages by neglect.
>>
>>101469679
Pretty sure they can't sue you for opportunity cost or whatever
If that were the case they'd charge you every time something went down
>>
>>101469740
Yeah then you work through them one by one veryyyy slowly.
Once they get back from the weekend and their computers aren't fixed what're they gonna do, sit on their ass all day?
>>
>>101469750
I don't think they can but he seemed extremely serious and now I'm screening calls from what I think might be higher ups my phone is blowing up I've stopped looking at it. Should I get a lawyer? I don't know if I can afford this.
>>
File: amazon warehouse.jpg (472 KB, 1000x667)
472 KB
472 KB JPG
maybe you should quit and get a job at an amazon warehouse?
>>
Literally not your problem. Get signed off with autism and then sit back and play vidya
>>
>I have to actually work? AAAAAAAAAAAHHHHHHH
>>
>>101469302
Just use Ansbile bro
Blast it out to all machines bro
That’s DevOps bro
>>
>be at will employee
>quit when they need you most because boss is a cunt
>no one else can do your job
>offer contractual services for the next week at 10 times your normal rate
>???????
>PROFIT!
>>
>>101469875
you can train a monkey to fix this
it's just time consuming
>>
>>101469785
Dude just pick up the call and explain to one of the higher ups that your manager was a dick and you'll work on it but don't expect it to be done anytime soon, and if they want to help to speed up the process they're welcome to help

You need to realise right now you're the one eyed man in a world of blind men
>>
I managed to send a message asking for assistance in resolving the issue on the company-wide channel by distributing the workload across all the lower-level employees. I even attached a short simple video. I received a bunch of thumbs down and cursing emoji reaction then I quit my Slack app and shut off my phone.

>>101469865
Our Ansible Tower and VPN servers are also impacted but that doesn't matter because the machines die and loop before they can ever receive an SSH command.
>>
>>101469302
Do you not automatically back up users files off their computer? Why can't you just ask employees to bring in their laptops connect their charger and an Ethernet cable then redeploy your system image. Wtf are you even doing?
>>
File: cyraxlament.jpg (425 KB, 928x1610)
425 KB
425 KB JPG
>>101469302
>I abruptly left the call because I started tearing up
fucking pussy
>>
>>101469973
I can't redeploy anything on their machines. They have to be manually switched to safe mode to even boot. Also most of the impacted machines aren't user machines.
>>
>>101469740
>>101469679
You can sue anybody for any reason. It doesn't mean they'll win. And unless they have a pretty explicit clause in your contract for this type of scenario(they 99% dont) they'll just be wasting their time and money trying to sue you. Stand up for yourself they're just trying to bully you into fixing this shit that had nothing to do with you. If you quit or they fired you, nothing will happen
>>
>>101470028
>They have to be manually switched to safe mode to even boot.
You just learned why PXE booting exists. Remember this day.
>>
>>101469679
He can't sue you for damages if you haven't caused the damage. He can sue Cuckstrike though.
>>
I can feel a panic attack coming. I'm gonna go for a walk without my phone or laptop.
>>
>>101469302
it's literally your job retard
man up.
>>
He's doing you a favor.
I would have told him to go fuck himself the instant he started screeching about this.
Just get fired and find a new job. You don't want to work for this fag anyways.
>>
CLEAN IT UP WAGIE
>>
>>101469679
Did you sign a contract saying you'd work nonstop to fix outages? Even if so I doubt that's enforceable. You should get everything in writing.
>>
>>101469302
i'd quit, unironically
it was your boss that pushed the compliance bullshit anyway, he can enjoy the result
>>
>""Dev"Ops" "Engineer"
>has to manually boot 2,000 machines
KEK. Do better in school next time.
>>
>>101469582
Is this what they call SOVL?
>>
>>101470095
If anything he could be in a position to sue them for a hostile workplace or harming his health by forcing him to work that long
>>
>quit
>lose unemployment benefits
do wagies really? is there not a "waging for dummies" book?
>>
>>101469302
Lol. That ansible bootcamp didn't help here?
>>
>>101469302
>He told me even if I quit or get fired I still have to fix this outage or else I'm liable for a lawsuit.
Call his bluff and let's see if he is willing to manually fix 2000 machines kek.
You have more power in this situation than you think.
>>
File: IMG_0672.jpg (39 KB, 600x600)
39 KB
39 KB JPG
Just got off a call with my IT guy. Told him me needs to do a manual fix for each impacted machine. He is the only IT monkey in the company so i expect him to do it ASAP. We have over 2,000 impacted machines, which should give him plenty to do. I told him that this was a reasonable request and part of his responsibilities. He got extremely mad and cussed me out for like 10 minutes complaining about the “enormity of the task” and shut me up every time I tried to interject. I told him to get on with it and abruptly left the call because I have golf in an hour and have had 27 unanswered calls since. He's even calling me right now. He told me he will quit or get fired, but i know he is a little bitch and threatened him with legal action. I'm just sitting here relaxing.
>>
This is a LARP, even a monkey would have some form of MDM at their company
>>
>>101469302
>Automating shit before review and deployment
deserved, enjoy thousands of repairs or job termination
I'm shitting as I type this, that's how little I think of your situation
>>
>>101470339
This outage bricks MDMs. The machines don't even boot.
>>
>>101469302
Fake and gay but if true i would suggest demanding an immediate raise up front and tell him that it's going to take 3 months to fix, and that "if he doesn't like those terms, he can always outsource :)"
>>
File: 1720509384950424.jpg (62 KB, 840x544)
62 KB
62 KB JPG
>>101469302
Just quit. You weren't payed to hear unsolicited insult.
Don't you have dignity?
>>
File: bocchi 'tism.png (425 KB, 1600x800)
425 KB
425 KB PNG
>>101469302
>I abruptly left the call because I started tearing up
You are literally me.
>>
>>101469865
>just use ansible bro
I CANT BOOT
>>
>>101469679
YOU didn't cause any damage, did you? Just another manager speaking out of his asshole
>>
>just quit
Hey dumb dumbs you lose all rights to benefits when you quit. You're supposed to get fired in writing.
>>
>>101469302
Just fucking quit and threaten to sue for hostility. They clearly don't put much faith in you. Even if they try to sue you it'll take time and they'll be scrambling to get back up in the meantime. Then your boss will be the one in hot water.
>>
>>101469302
This is interesting and all but you have work to do, wagecuck.
>>
File: 2a6-103617203.jpg (51 KB, 709x595)
51 KB
51 KB JPG
>>101469302
just do your best to fix as many machines as you can while you are on the job, and then go home.
>>
>>101470329
kek.

OP is softer than curry scented diarrhea. We have close to 20k machines offline, no telling how many of them are bootlooping.
>>
>>101470416
Doing a shitty job is effectively the same as quitting.
>>
Wagie wagie in a cagie
>>
I am watching anime and eating sauce and fries
>>
>>101470320
the trick is to never put yourself in a position of responsibility. You want to be a faceless forgettable engineer, not a one man army, because when shit hits the fan you're expected to do everything. Me? I'm chilling. Practically a day off and I probably make more than OP.
>>
>>101470467
I just ran the numbers and assuming the fix takes 5 minutes and there are 2,000 impacted machines then that's 166 work hours, or 20 work days. They're out for the whole month kek.
>>
OP has a lot of leverage. The company thinks they are screwed without him. He should dictate terms in a cold calm voice.
>>
>>101470360
Then network boot? No wonder you retards can't get a job lmfao
>>
Here you go, anon
https://www.reddit.com/r/sysadmin/comments/1e708o0/fix_the_crowdstrike_boot_loopbsod_automatically/
>>
>>101469302
Stop being a crybaby bitch

Figure out PXE/Netboot, make a custom Linux ISO (https://askubuntu.com/a/1193521) that runs some bash script at startup to fix the issue (https://superuser.com/a/1849624, seems like an easy fix for _most_ devices), test it on one machine, apply to all machines afterwards.

After this is over, collect all correspondence where your boss tries to pin it on you, and either sue for damages (not a lawyer idk) or use it as leverage for a promotion. If 2 doesn't work out try 1
>>
>>101469538
>3% annual raise
Lol more like 0.5%
>>
>>101469302
Record everything if he's threatening lawsuits, he can't legally force you to work for 48 hours straight.
Your boss fucked up by having one guy responsible for over 2000 machines.
I recommend quitting and offering to come back as a contractor for an exorbitant price.

Don't cry, don't shake, you have something he wants and he's desperate to get it, man the fuck up and shake him down.
>>
>>101469302
You shouldn't have let them put that retarded shit on your machines to begin with.
>>
hello saar I will fix you machin for $3.50 a pop
>>
>>101469302
Sounds like whoever decided to use crowd strike is to blame. You can fix it, will just take months.
>>
saaaaaaaaaaAAAAAAAAAAAAAAAAAAAR YOU MUST FIX THE MACHINE SAR YOU MUST FIX IT STUPID BHENCHOD
>>
Got back from my walk. No panic attack yet. Phone still turned off. I started reading up a little on what PXE is. Can I just run it on my machines with some script? Or do they need something configured beforehand? Because I don't there a way for me to configure the machines for PXE right now they don't even boot right. Also I don't even know if we're using images for our machine. Can I prepare one with the right filesystem without the Crowdstrike file and just tell the machines to boot from it? I don't really know much about any of this stuff. I'm a DevOps Engineer I didn't even realize I was supposed to be responsible for this sort of stuff.
>>
>>101469865
The machines don't even boot into OS lol. Ansibell won't save you
>>
>>101469302
Personally I'd enjoy getting paid to reboot computers right now, I've been stressing over a lot of shit. It would be nice to listen to some podcasts and clean this shit up. Its not like any IT admin can take the blame.
>>
It's over for wagies
>>
>>101470575
You use it as leverage before you give them what they want.
If there was good will maybe it would make sense to try and figure it out, but if your boss starts disrespecting you, all good will is gone, shake him down or quit and let him figure it out (he won't)
>>
File: 1697841418284647.png (41 KB, 200x200)
41 KB
41 KB PNG
>>101469302
lol
lmao even
>>
>>101469302
Fix it as slowly as you can
>>
>>101469302
What kind of DevOps engineer is you that you don't have a script to automate your machines?
>>
>>101470567
>bitlocker, another checkbox on compliance craplist, enters the chat
good luck
>>
>>101470725
DevOps is glorified internal tech support
>>
>>101469302
>He told me even if I quit or get fired I still have to fix this outage or else I'm liable for a lawsuit.
devops "engineer" finds out he is a tech janny
>>
>>101470567
>Another caveat is that this most likely will not work on systems with encrypted filesystems.
This for sure won't work if disk is encrypted.
KeK. Any real security place is encrypting filesystems.
>>
File: 1721392377887.jpg (57 KB, 364x364)
57 KB
57 KB JPG
>>101469302
Throat feeling Itchy? Covid is going around.
>>
>>101470779
I mostly handled K8S, Python and CI/CD stuff until now. I was only vaguely aware of this part of our old stack that got handed off to me, they told me not to worry and that it shouldn't cause any problems. I never even messed with this stuff before the most I did was run some Ansible for workloads that couldn't run on K8S. I only just learned what PXE is, I don't know if I can use it, I don't even know if I can boot these machines from an image, I definitely can't run a script because they don't even boot.
>>
>double my salary or do it yourself, fucknuts
there, problem solved
>>
>>101470725
>I started reading up a little on what PXE is. Can I just run it on my machines with some script? Or do they need something configured beforehand?
It's a BIOS/UEFI boot priority setting. Most enterprise / large business would set the order to PXE->SSD and explicitly block USB booting.

t. retired senior sysadmin for a major US commodities exchange/clearing house
>>
>>101470802
>>101470810
I barely even messed with this stuff before. I barely even interacted with anyone but developers before. I mostly just automated CI/CD and infra stuff in Jenkins and Python and wrote some K8S controllers in Go. I don't think this is even my job I don't know what is happening anymore.
>>
>>101469302
>Or else I'm liable for a lawsuit
That's an absurd threat. Even if you lose, you don't have the kind of money that would allow them to offset their loss. You probably don't have enough to cover the lawsuit's cost itself.
>>
>>101470902
Your job is you make the computers work. If they break it's 100% on you. Make no mistake that's literally what your boss thinks you do and is the only reason he hired you.
>>
>>101469302
YES it is your job
NO you don't have to work 48 hours straight

To your manager this is just another negotiation, he thinks you have a passive personality and you're easy to browbeat so he asks for the impossible and makes you cry so you'll try really hard. Your manager doesn't know or care what's written in your contract, he only wants to find the combination of words which produces the best outcome for him.

This outage makes YOU one of the most valuable people in the company right now, and your manager is trying his best to fuck you over. You should go over his head, call a bigger fish - get some helpers on your team to speed up the work, and ask for a new manager who understands the job you have to do. "this moronic manager thinks one man can physically fix 3000 (plausible lie) machines in-person in less than a week! it's better if you let me hire some help and manage this disaster without him interfering." Improve your position within the company and demonstrate some self-respect and decisiveness.
>>
>>101470931
>he thinks you have a passive personality and you're easy to browbeat
bro cried hung up the phone and is now crying on /g/ show me the lie
>>
>>101470848
Do you have any remote bios access? If not you better start doing it manually because if you don't have ipmi or some remote access then you have to get hands on to use pxe anyway. Unless they are all already pxe, and if they are already pxe then TPM could stop you from rolling out a fix as well.

That's kinda fucked up if it was really unclear that this sort of thing would be your responsibility. The story doesn't add up to me
>>
it's amusing that a threat of a lawsuit is enough to make wagies work 48 straight hours just to get fired
>>101470542
correct. this situation can be handled in a way that not only helps the firm but also gets him respect as an equal from this manager. however op is a wojakposting negroid and needs to be beaten to do his job so unlikely.
>>
File: 1278.jpg (59 KB, 300x477)
59 KB
59 KB JPG
Does it make the fix harder if there is a full disk encryption?
>>
File: kjk.png (123 KB, 208x268)
123 KB
123 KB PNG
shits man. the wagie bleating ITT is disrupting me from watching my comfy anime at home.

WAGE QUIETER!
>>
If he's so valuable why isn't the problem fixed? He's stated multiple times ITT he doesn't know how to fix it.
>>
>>101469302
Get fucked codemonkey
>>
>>101470929
I don't think that's fair. I'm a little more clearheaded now I realize there's no way I should be the one messing with PXE or whatever. I just read there's no way this is gonna work with encrypted filesystems I don't even know if they are encrypted. This is not my job. I don't think I can leverage this situation but I also don't think they have a legal case against me. I'm gonna tell my boss this isn't my job and hope that he just fires me, so I can do a normal DevOps or Software Engineering job somewhere else.
>>
>>101470178
I was wondering when someone with actual IT knowledge shows up and suggests the obvious solution
>>
>>101470985
Hey can I have your job then
>>
>>101470848
Wtf you talking about, do you even have physical access to all machines? I have never seen a DevOps engineer dealing with physical server. What the heck is your job exactly.
>>
>>101470971
Yes. A lot. Some encryption software don't come with any built-in option to edit an encrypted volume offline. Which make some otherwise valuable method, like plugging the HDD on another system, or booting a portable OS, useless.
>>
>>101469302
good. you deserve it. get back into the office wagie.
>>
I'd make that manager fucking apologize on that lawsuit shit, in email, and CC upper management with it, or he could go fuck himself. EFF, ACLU, or some other entity would gladly take that case.

Grow a pair of balls OP. Yes it is your responsibility to know how to fix this, efficiently. But you are not obligated to put in more than 48 hours of work a week, and likely less depending on the state.

You are in a strong negotiating position.
>>
>>101469679
Dude just install Gentoo on all 2,000 machines and flee to Argentina.
>>
>>101470955
>That's kinda fucked up if it was really unclear that this sort of thing would be your responsibility.

Sounds like a classic case of it wasn't until it was to me.
>>
>>101471028
We had an old sysadmin who died from covid about a year ago and they haven't been able to replace him. They just told me I might need to do minor technical stuff every few months but now apparently I'm the sole person responsible for this major outage.
>>
>>101471069
Based
>>
>>101471086
We're going to end up like the people living in the ruins of the Roman Empire living in a world we have no idea how they even built it
>>
>>101471061
I don't think it is my responsiblity and I'm sure they will just try to screw me even harder if I try to leverage this position. I'm certain now this company is insane and I'm just waiting for my boss to send me a message that I'm fired. I think I'm gonna have 100 unanswered calls soon lol. Just because this fucking moron is trying to pin this all on me.
>>
>>101471115
We?
Surely, you meant 'Niggers'.
I know how to fix my own shit. And I have an actually good understanding of what was left in place by people before me.
>>
>>101471115
Anon, we are already there. We're living like this, been living like this, for at least 20 years
>>
>DevOps YAML monkey knows only YAML
>never heard of PXE
>panics and cries when actual IT work is required
>likely mocked sysadmins before this
You get what you fucking deserve YAMLshitter
>>
>>101470725
I believe you said most machines aren't user facing, or was that another anon? Anyway, how many are encrypted with bitlocker? Do you have access to the bitlocker keys or is the DC also affected by the bootloop? Tbh you need to cya yesterday so you aren't pinned as the fall guy.
>>
>>101471086
So you are the sysadmin, just boot up the machines man.
>>
>>101470725
Bro this guy is toast and his manager is larping as him trying to outsource /g/ to do his job
>>
>>101471257
The manager guy should install Gentoo then, it's generally the quickest and easiest fix to all tech-related issues.
I fixed the crowd strike boot loop in my own organisation for 3000 machines within 2 hours by simply installing Gentoo.
>>
How would you guys have handled this situation in my stead? It's easy to be objective from afar, but I have a major general anxiety disorder, and I just got saddled with a major outage that I have zero knowledge how to fix because it isn't actually part of my job, it just got pushed on me because the guy who actually knew all this bullshit died and they haven't been able to replace him because they're a shit company so because they're a shit company they pushed it all on the DevOps guy thinking nothing like this could ever actually happen. Well it did and I can't fix it, and even if I could quickly study up on PXE IMI TPM Remote boot and work something up hoping to god the filesystems aren't encrypted I still wouldn't because I have no incentive, if I try to leverage my position like some have suggested I'm sure they'd just fire me and try to sue which they're going to try to do anyway. But I'm certain they have no legal case now so I'm just screening all their calls waiting for them to tell me I'm fired.
>>
>>101469302
>nooo i have to do my job
do zoomies really? no wonder no one wants to hire you retards
>>
>>101471222
I'm not the IT guy and it's not my fault this company is so shit they expect the DevOps guy to do IT work. I don't know if the filesystems are encrypted and I don't care. In fact I hope they're encrypted so the poor pajeet they outsource this mess to will not be able to fix it either and then the whole company will go under then they won't have money to even attempt to sue me.
>>
>>101469302
It's not negligence if you were following best practices and got fucked anyway. I don't see what your problem is though, obviously it's not all getting done today and then there's a weekend. You have 72 hours to get them working, and if there's 10% left to do afterward, it probably won't be that big of a deal. Just ask them to send you 3-4 guys that know computers and supervise them for a bit and you won't even have to work 16 hour

>>101469904
Everyone else at the company is untrained monkeys and their computers don't work.
>>
>>101471295
I already told you what I'd do
>>101470622
>>101470756

You're probably the only guy in the company who has any chance at fixing it, thus you have massive leverage, they couldn't replace the last guy, what makes you think they'll find someone to fix it after you're fired?
They'll scream and threaten, but they won't fire you, because they know they're fucked.
They probably have no case to sue you, but you should talk to a lawyer anyway.
Stop being a bitch and keep in mind that they need you a lot more than you need them in this moment.
>>
>>101471351
>it's not my fault this company is so shit they expect the DevOps guy to do IT work
Accurate
The issue is many companies do not have IT people anymore and many DevOps >engineers have lied about being able to do IT work
>>
What a fucking retard
Company needs you now more than ever and you shit on them. You're super fired after this.
How you should handle this at your next job is to ask your boss to get more hands and you can teach a group of people how to fix the machines and coordinate it.
>>
>>101471295
It would be good to go over your contract with a lawyer still. I know a lot of "DevOps" guys who were glorified IT guys as well, having to handle shit like this as well if there was a necessity for it.
>>
>>101471086
Kek tell them to get fucked and hire a sysadmin
>>
File: WHAT A LOVELY DAYjpg.jpg (8 KB, 259x194)
8 KB
8 KB JPG
>>101469302
>woke up
>hungover
>nonstop calls from our company about 1000s of machines down
>most of our apps in azure
>"we need this fixed right now"
>show them the entire world is fucked
>"YOUTUBE SAID WE COULD FIX THIS"

WELCOME TO HELL
>>
File: gwizu4xy1rz71.jpg (8 KB, 246x250)
8 KB
8 KB JPG
>>101469302
>I'm liable for a lawsuit.

pretty common term used to threaten new people, managers do this often lol just to make work from you
>>
>>101470398
K E K
E
K
>>
>>101471295

"Since you threatened me with legal action for a situation I did not create I have to seek the advice of counsel before I may resume working on the outage problem"

Put it back on him. Send that shit in an email. CC his boss.
>>
>>101471422
>>101471426
>>101471428
>>101471434
If I wanted to be a sysadmin then I would've been a sysadmin then I would've known what PXE and remote boot is and whether the filesystems are encrypted or not. But I didn't and I'm not. I realize there's some overlap between the demographics of DevOps and IT and that places muddy up the titles but at the end of the day I basically just write Kubernetes controllers in Go and some Python scripts. This has nothing to do with me and I don't want it to have anything to do with me. Even if I could leverage this situation I don't ever want to see my boss again or work in this field.
>>
>>101471011
>someone with actual IT knowledge
>/g/

kek
>>
>>101471295
yeah whatever man. challenges will come your way in the future and you can't hide behind an "anxiety disorder" forever.

if they're still calling you then you aren't fired.

don't talk to your manager, try to find another executive. manager has fucked himself with his pointless threats, he is fired if you can find a way to fix the problem without him.
>>
>>101471477
this.
Retards don't seems to understand that to be sued, you need to actually have something of value they can take from you.
op probably don't have any significant amount of money or asset. And it's not like putting him in prison would magically fix the company's problem.
They would lose more ressource prosecuting him than they could conceivably get back.
>>
>>101471295
In terms of covering your ass you submit a complaint to HR about the abusive behavior, cite the anxiety disorder to say that you were triggered so hard you can't work effectively. Also arrange some kind of emergency contact with a therapist as further evidence on your side.
>>
>>101471477
this. if they're threatening you, just quit.
it's not worth it. they're going to fuck you in the ass no matter what.
>>
>>101469302
I almost never go to /g/ but it's telling that this is the first post I see
>tfw freelancer
>>
>one-man-shop IT guy for a whole enterprise
>doesn't know anything about IT
>major general anxiety disorder
KEKKKK
>>
>>101471496
This is actually brilliant
>>
>>101471558
>In terms of covering your ass you submit a complaint to HR
this guy glows in the dark.

NEVER FILE SHIT TO HR FILE IT TO YOUR LAWYER AND LET THEM FILE IT TO HR. Human Resources exists to serve the investors interest not yours. You are quite expendable.
>>
>>101471496
sadly the best jewish answer. GL op and godspeed.
>>
>>101471585
This sums up this thread.

I wonder if now he will tell people he have impostor syndrome too.
>>
>>101471496
I'm sending this on the company-wide Slack channel and shutting down all communication until Monday. Then I'll go looking for a normal backend dev job or something.
>>
>>101471596
this
HR exists to protect the company, not the employees
>>
If that's how they treat you look for another job. Do as much as you can while being a professional.

Get that law suit bullshit in email and report him to his manager or just skip the email part.

Create an image without that cloudstrike and shit roll it out to a few machines

No one is going fuck you as hard as you think. Your boss sounds like a massive cunt
>>
>>101469302
Why didn't you tell him to watch his fucking mouth when talking to you? I'd go to work just to kick his ass.
>>
File: 1721394858545.jpg (48 KB, 340x340)
48 KB
48 KB JPG
>>101471496
based jew
>>
>>101471632
It's not very professional to create company-wide drama but I guess if you're gonna leave anyway fuck it
Make sure to take screenshots of every message and communication though
Don't communicate in a way you can't keep exact records of
>>
>>101471519
Just wave with your job description and tell them if they wanted you to be sysadmin they needed to change your job and salary.
>>
Kek just name the company OP. Then we can troll the shit out of them or something.
>>
>>101469973
>image
lol what year is it?
>>
>>101471685
They treated me like human garbage I didn't even bother opening up the replies to my message asking for assistance in delegating the workload. It's all on Slack I think that's good enough.
>>
File: shut-it-down-pajeet.jpg (41 KB, 630x405)
41 KB
41 KB JPG
>>101471708
Reporting in with 10 resumes ready to apply for positin, Saar. Please notify me to redeem when ready.
>>
Just talk to your union rep or works council

>America

Nevermind.
>>
You hold all the cards here anon.

Your boss doesn't know how to fix it. The enterprise doesn't have time to hire a new person.

Demand $200k bonus or just walk out and get a job in a comfy linux org. A contractor would charge them more so they'll pay you.
>>
File: goodluckfaggot.png (46 KB, 738x283)
46 KB
46 KB PNG
>>101469302
Workaround Steps:
Boot Windows into Safe Mode or the Windows Recovery Environment
Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
Locate the file matching “C-00000291*.sys”, and delete it.
Boot the host normally.


and see picrel. good luck
>>
>>101471719
They might kick you and pretend you didn't send anything. Just take screenshots, it takes zero effort and might save you some effort down the line.
>>
>>101471724
>white collar union
lol
lmao even
>>
>>101471295
>waiting for them to tell me I'm fired.
they need you they literally have nobody else lmao
if anything the higher ups are calling you to negotiate for your help because your boss was a retard
>>
>>101471069
Arch is faster. Just use archinstall, set it up with kde plasma, and go home.
>Here's your actual fix, switch to Linux so this shit doesn't happen in the future.
>>
>>101469736
/thread
>>
>>101471800
archinstall with xfce is faster than kde, archnigger
>>
>>101470178
>>101471011
based
>>
>>101471799
At this point I'm pretty certain the filesystems are encrypted which means I have to do hours upon hours of manual work even if they realize they should delegate this work to a bunch more minions. Regardless this company sucks, my boss is already burned and even the higher ups sent a cursing emoji reaction to my polite request for delegation of the workload, I don't intend to bother with this shit anymore the only thing I can leverage from this is a stack of cash to fix it then get fired but they're not offering that so I don't care. My meds are also not kicking in and I don't about anything anymore. I'm just gonna play vidya and watch anime until Monday.
>>
>>101471596
I don't know how all that works, but the point is to get evidence that your are a victim of bullying, discrimination etc. and that you are not at fault.
>>
>>101471879
Lmao based, fuck them let them burn for their incompetence. Just take screenshots for future reference if needed.
>>
>>101471295
You could be walking away from this with a 300% raise and your boss fired if you play this right. People are already posting safe mode solutions if you're interested.
>>
>>101469302
Tell your boss
>I will invoice you for 5000 USD on paypal. If you pay that I'll work 48 hours without sleep and we won't worry about the implications of this. You can say I was absent from work or whatever

$5000 is chump change and more than worth it. He'd spend more than that much hiring people off some crappy service anyway considering everyone's working overtime.

>b-b-but I have anxiety
I know, it sucks. The best way to fix this is to work freelance so you no longer feel dependent on your boss for your livelihood. You can go back to corporate working but knowing that even if you get fired, you can still survive, is a powerful thing. People who've only just got employed out of college after job searching for months are unfortunately fucked in the head, I'm sorry, you have to build that confidence.
>>
>>101471585
Yeah, this smells kinda fishy. There's this company that has 2000 computers to manage, no IT guy or MSP to help with them and anon is just a "devops" doing whatever else but he's also oncall 24/7? Why the fuck would the company even need a devops if they can't have someone do basic system administration.
>>
>>101471879
>but they're not offering that
yeah you have to ask for it, obviously they won't offer lmao
but i understand the struggle with anxiety anon, i feel for you. I just think turning off your phone won't fix anything and you will still be thinking about it all weekend, you're not going to have fun until it's done and over with.
you need to talk with some actual higher up, explain to them the extent of the issue and that you need to study on how to fix it most efficiently, and they'll listen to you. they just want their shit to work, they don't care about your boss.
>>
>>101469302
>48 hours straight just to mitigate the outage
Good thing it's friday!
>>
>>101471519
>>101471496
You already have the answer, OP. Your boss threatened to fire you for something that was outside of your responsibility. You have leverage on them now, use it.
>>
>>101471942
From the sounds of it, OP would probably need 48 days
>>
>>101469302
We have demands from the hackers.
I'm unsure what they're saying.
Can somebody translate?
https://vocaroo.com/14tf2TJZniNB

They used to work at Microsoft. I could get that much. Somebody post on hacker news.
>>
>>101469302
just say you will do your best within your contractual limits and ask for a raise.
>>
>>101471923
We had an old sysadmin who maintained all this shit but he died from covid and they couldn't replace him. They told me the maintenance was minimal and my 24/7 on-call is basically answering a few Slack messages a month, nothing like this.
>>
>>101471921
>work 48 hours without sleep
Is that physically possible? I'd just pass out at 20.
>>
>>101472011
you'll probably sleep anyway but if you have a reason to really stay up and some caffine you can make some of it.
you'll get a few naps in and just pretend you worked the whole time.
>>
>>101471938
I don't know what's wrong with this company. My boss is an abusive asshole but even the higher ups reacted with fucking cursing emojis to a simple polite request for delegation of the workload. I didn't even bother reading their replies I don't care anymore. I think they're all a bunch of boomer shitheads and their company deserves to go under, if not because of this then because of something else. I've already taken double dosage and now I'm gonna drink.
>>
File: 1721102959986318.png (143 KB, 600x600)
143 KB
143 KB PNG
>>101469302
That sucks OP.
Anyways, include me in the screencap.
>>
>>101471800
KDE 6 is buggy as fuck at the moment.
>>
>>101471708
>We are crowd strike, we hack your company. Are you afraid? Then send us your nudes and we will revert your systems to the needful benevolent
>>
>>101472171
Send that from a fake email account OP.
>>
>>101471644
based hothead
/g/ is full of spineless faggots
>>
>>101471994
Still fishy as fuck, no way a single sysadmin could handle 2000 computers what the hell
I work for a company that has under 2000 employees and we have a devops department, an "RnD Tools" department wholly separate from devops, and an IT department. Each department has at least 4 or 5 guys, for the same amount of workloads as what your old sysadmin was supposedly handling.
>>
>>101469302
So just do it, who cares if you're paid hourly. Which you are right
>>
>>101469302
LOL
>>
>>101469690
Most companies will have to resort to this. No one has the fucking manpower for this bullshit
>>
>>101472063
Might wanna update your resume too, although "bailed from the crowdstrike thing" is gonna be a hard sell for future employers.
>>
>>101472260
I don't know. Ever since he died that part of the stack was put in "maintenance mode". I mostly just had to run his scripts or read his documentation every once in a while, months could go by with me not touching any of it at all.
>>
>>101469302
>devops
>not PXE-booting the 2000+ machines to remote desktop for central management
ngmi
>>
>>101469679
Are you living in India? How can it be even possible? I hope you get paid SV salary if you aren't.
>>
>>101472398
Ask any devops guy what PXE even is I doubt even 1 in 100 would know.
>>
File: IMG_9682.jpg (130 KB, 1107x1080)
130 KB
130 KB JPG
>>101469302
>I started tearing up
>I'm just sitting here shaking
>What do I do?
grow some fucking balls first off
how fucking weak and effeminate are you? good lord
there is no way he can sue you for quitting and if he tries to you need to sue the company yourself
so maybe start looking for a lawyer
but other than that clearly he needs you more than you need him so just let him be the one that seethes and let it wash over you
take a page out of Trumpisattva's book, in His words, "Bullets were flying over us, and yet I felt serene..."
>>
>>101469679
First, no, he's talking outta his ass. Second, install a linux distro on all impacted machines with a tiling window manager then resign. :')
>>
>>101469302
You sound like the Secret Service. Just a DEI poser that drops their spaghetti when the real shit goes down.
>>
>>101470931
this
OP is a pussy and a natural born slave so its doubtful he'll be smart about it but Im rooting for this castrated bitch anyway
>>
>>101469302
Good news, OP! I had a chat with your manager. I explained the real problem to him. Your new assignment is to install Gentoo on the 2000 impacted machines and to check the source code for back doors before you compile and install it.
>>
File: 1713204253900918.png (222 KB, 478x466)
222 KB
222 KB PNG
Document everything (write down what the said, record the calls, ask him to email you the exact orders) and contact your union for advice on overtime pay etc.
>>
>>101469302
this is bullshit, but I believe it
>>
>>101469302
contact his boss and tell him you'll do it if he fires your direct boss
>>
>>101472398
>ngmi
No shit that's why he started the thread
>>
>>101469302
Tell him you need a five and a quarter inch floppy drive.
>>
>>101472398
>>101472439
Damn. That bad over there?
I mean, yeah, could've figured.
Everybody who's in the know is just trying to keep it all together, we know this train is going to derail, let's just hope not here.
t. Embedded
>>
>>101469302
>>101469679
>>101469740
op is LARP
every boomer boss saw the news and knows "there's a global computer issue"
>>
>>101472836
Any field that becomes too highly abstracted is doomed to fall in the competency crisis. WTF is devops? It's when you suck at dev and also suck at ops so you go for something where you can suck at both at the same time, and corporate figures they have to pay 1 guy instead of 2 so it's great for profits. Then you end up with people like OP who don't even know what PXE is. It can all be abstracted away by fancy crazy shit like Kubernetes until it can't. Then you're never able to reconstruct the spaceship that made it to the moon 60 fucking years ago.
>>
>>101472836
At my previous company, we just took the computer to an IT guy to fix it and he would call us when it was done. None of this remote "wake on lan" mumbo jumbo
>>
File: pepe wojak champagne.gif (62 KB, 680x553)
62 KB
62 KB GIF
clean it up wagie
>>
>>101469302
>He told me even if I quit or get fired I still have to fix this outage or else I'm liable for a lawsuit
That's not how it works, you fucking moron. They can't do shit about anything if you quit now.
>>
>>101472882
No.... I work in a big company, the sort of incompetence he is describing, his own lack of training, and just 'falling into' a position like this are disturbingly accurate for anyone who works in the contracting world
>>
>>101470539
>OP slow rolls it and works slowly on getting each computer reset
>Can threaten to quit if they try to rush him
Why is he panicking? He can use this as an excuse to get out of other work beyond doing this by saying he is 'still working on those impacted machines' in every status update for the next two months
>>
>>101472928
Because it's a wholly different specialization. DevOps has nothing to do with IT administration for retarded HR slags and dumbfuck code monkeys who don't know how to safe boot into their windows to remove a file.
I'm sure there's a lot of retarded companies like the one OP is describing who hired a devops guy thinking they'll get someone who can make them a program and fix their computer whenever they spill coffee over it at the same time, but that's not really what devops mean in the general market.
>>
>>101472928
>>101472836
DevOps should have nothing to do with maintaining a company's fleet of laptops. They're the ones managing various production/staging/test environments, ci/cd pipelines etc. Or at least thats what the job description should entail. OP's company seems fucked if they don't have an IT department or a contract with an IT services company.
>>
>>101469324
crying in his cagie
>>
File: 1714071109777847.png (339 KB, 653x490)
339 KB
339 KB PNG
>All the texts I got this morning woke me up, my phone was under my neck
>Our financial apps are down, but one of the bosses on vacation saved our ass by bringing some key servers back up
>Key stations brought back, morning process done
We got off easy. All our team needs to do now is restore every other computer, but its only like 500 or so.
>>
>>101472928
This, I've been assigned to running a project as a manager that is doing something with Kubernetes for over a year now, but I still have no idea what Kubernetes actually are
>>
>>101471069
>commit warcrime
>flee to Argentina
Always werks
>>
>>101473226
How did he do it
>>
>>101469302
You have all the power in this situation. Ignore all threats from him. Do what he tells you ofc, but in your own time - no need to stay up 48hours on it. Just do it at the pace you would normally. He is literally just panicking and unloading all his stress onto you, his threats are empty.
>>
File: 1698804465588655.jpg (248 KB, 1600x1200)
248 KB
248 KB JPG
1. Have OneDrive (or whatever the fuck) retain user data like you're supposed to
2. Use whatever backup system your company uses to roll back to pre-saar Crowdstrike
3. Don't approve any update queries, if you're not retarded you don't have anything auto-enabled and your Windows is Enterprise level
4. Wait for Crowdstrike to unfuck itself
>>
shitty larp from a dumbass op.
Spend 10 minutes training 2 monkeys to train 10 more monkeys to train 100 monkeys and have them fix the problem. Collect your paycheck. Job done.
>>
>>101469679
Maybe you shouldn’t have enabled automatic updates on your companies computers.
Why don’t you test them before rolling them out?
>>
>>101470499
>faceless engineer
>making more than OP
yeah right
unless OP is a gigaretard, there's no way
>>
>>101473357
Most companies have automatic updates on by default because they are retarded. I'd bet you money OP never thought to check if updates were automatic or not when he first got his job
>>
>>101473328
Even if you're lucky enough for your backup system itself to not be impacted, the machines still can't boot. Only remote boot + PXE + custom image can fix the issue automatically but that's only if the filesystem isn't encrypted. Now what enterprise uses Crowdstrike but doesn't encrypt its filesystems? If they are encrypted, and they are, then you can't automate shit because the Bitlocker servers are also impacted. Then you have to either manually circumvent the Bitlocker key request or input it assuming you actually printed it out on paper. Either you still have to manually fix potentially thousands of machines.
>>
>>101473256
I don't even know. My station and a few others were never down in the first place, so I was able to at least communicate with some people. Auto updates on the ones that were up are probably turned off, so he probably just needed to repair connections.

I just hit my first year here, so there's a bunch more for me to learn on how this place works exactly.
>>
>>101473428
If your system worked a week ago without BSOD then it will work now, and you will have a backup of that exact environment. Important software should also have its data backed up. Things like OneDrive will retain bookmarks and all the other shit boomers can't live without. People are now learning the value if stuff like Avamar because software is currently developed by people who don't know what they're doing, this is why you have banks in England running off Lotus 1-2-3 or something and you don't hear about them having issues. It is imperative you have something to backup important data so that being saar'd by Windows or some stupid company just means you roll back a month.
>>
>>101469944
>I received a bunch of thumbs down and cursing emoji reaction then I quit my Slack app and shut off my phone.
Top kek. This cant be real
>>
>>101473073
>tell boss it takes 30 min per machine and you have 2000 machines
>48 hours with one person
Would they not understand it's impossible?
You should be able to say "ok boss I'll work 12's until this is finished" and then demand a bonus/raise at the end

I'm glad I personally work with sane people. (Probably because it's not an office job)
>>
>>101469302
This would have never happened had you convinced your C level executives to run OpenBSD on all machines with xorg and maximized fullscreened Firefox window.
You're going to have to work 48 hours and fix the machines wagie. Do your job unless you're a nigger. BTW it's ok to take naps periodically.
>>
>>101469392
This pepe is horrendous, did you make it?
>>
File: 1570480799693.gif (354 KB, 500x491)
354 KB
354 KB GIF
>>101469582
>Fred Dibnah
Absolute legend.

>>101469619
>getting paid for overtime
>ever
>Implying
>>
>>101473509
All the data is retained but that doesn't matter because automatic rollback is made impossible by decryption of filesystems being deadlocked by impacted encryption servers.
>>
>>101473540
>Would they not understand it's impossible?
Because these companies constantly promote tech-illiterate marketing and humanities types who don't understand how computers work?

I would know, because I am one of them
>>101473517
Oh it's very real
>>
>>101473428
>you still have to
flawed premise.
Fix one machine. Print out a guide. Send it to your boss and have him fix the rest.
>>
>>101469619
>2024
>implying Americans won't work unpaid overtime
>>
>>101473558
Automatic rollback is never impossible because I can wipe an SSD, redeploy the image, omit Crowdstrike (until it's fixed) and just let all the data for the image's software repopulate because the corporate network is tiered like 27 times.
>>
Is there not a way to get administrative access and enter commands for each computer on a network at one time? If you could figure out a way to do that you could make the job way quicker.
>>
>>101473398
that management style reeks of indian wage slave, he's probably being paid a lot less than he's worth
>>
>>101473685
considering the entire company went down, he's getting paid far more than he's worth.
>>
>>101473653
>wipe an SSD, deploy the image
How are you gonna do that automatically to thousands of machines that can't boot and that request a decryption key from an impacted server?
>>
Guys I work at a nuclear facility, the control room is completely locked out of control of the reactor

Technically its fine right now because the automatic shutdown controls are analog systems and the plant can do its own thing but the operators can't manually shut the plant down, we are so fucked
>>
>>101473766
I mean that's going to depend on your environment. We have an industrial dongle so even if you smashed the SSD with a hammer we can communicate with mobo software and cycle through things until we figure out someone is beating a computer with a hammer. I'll say I don't know how to mass deploy if there's no boots on the ground whatsoever to help though.
>>
>>101473913
There's no way a fucking nuclear facility would use crowdstrike, let alone windows 11
>>
>>101473583
Is American labor law that bad? Such things happen only in Japan or China IIRC
>>
>>101469679
> He said they can sue me for damages even if I'm fired
Assuming you're not larping, this is complete nonsense and he's just trying to intimidate you. You can only be sued for damages if you caused intentional harm. You can also just walk away from your job and suffer no consequences.
>>
>>101469302
0/10, no company on the planet with 2000 Windows VMs (and not counting the Linux ones) only has 1 employee to take care of them.
You are either larping or your boss really wants to fire you, which is why he is putting the fear of God into you while others are fixing shit for real
>>
>>101473992
They can claim negligence or cite some random provision in the employment contract (who knows) tho
>>
>>101469302
start recording his abuse for a lawsuit you stupid ass
>>
>>101471011
Sure, fucking servers notoriously boot from the network you dumbfuck, why didn't everyone think of that
>>
>>101474064
Nah. It's not his negligence that the servers failed. And if it's true that this is not his normal role and he doesn't have the skillset to fix it then no court in the land will do anything to him. His boss is just redirecting his anger at a weak employee because he knows he's the one that will end up taking the blame internally.
>>
>>101474064
They can (falsely) claim whatever they want. They have 0 case for showing negligence though, and the worst consequence for violating an employment contract is termination.
>>
>>101474061
I used to manage 4000 Linux VM's by myself. With the right tools and scripts it's not that hard.
>>
>>101474061
Bruh companies have their entire IT support outsourced and zero on site personnel
This is very much possible
>>
>>101474137
It is true it can be done, but unless you are on call 24/7/365, never get sick and never leave the office, that is at the very least a retarded decision by your (former?) company. I genuinely did not believe this could be real, then again I work for M$ so my perception may be biased in favor of humongous corps
>>
>>101474064
You can just cite i don't know... the thousands of articles saying this is a global issue and your local labor laws anon, don't get intimidaded when you have both basic common sense and laws by your side
>>
>>101473950
lol the optimism
>>
I'm at the age where I'd tell the boss "no, your request is unreasonable. This is harassment and I'm going to walk unless you change your tone. Since you just threatened to fire and sue me, I'm going to ask for my salary up front for the next 48 hours you expect me to work."
>>
>>101474152
Yeah well, outsourcing still counts if they do operations and pay them to be on call
>>
>>101474120
>>101474124
I mean, they can just file a lawsuit and just claim negligence or breach of contract. It alone would financially hurt him due to legal cost, and IIRC you have to pay for your lawyer cost even if you win the case in the US.
>>
File: GQr0WlZXMAEgtFE.jpg (80 KB, 604x507)
80 KB
80 KB JPG
>fake job niggas when they have to actually do something for a change
this didn't affect my Makita kek
time to get your hands dirty you white collar fucking leech
>>
>>101469302
>DevOps
>He genuinely expect me to do the manual fix for each impacted machine
so devops are IT now?
>>
>>101473708
I don't think you understand what's happened here, anon.
>>
>>101469302
Just do it? What's the big deal, it's not difficult and you're getting paid? Since you're DevOps it's not like it's that far out of your wheelhouse or that you're getting underpaid for the task. Hell, it's practically a day off just walking around bootlooping machines.
>>
>>101473174
>>101473187
I see, I've just heard it been described as a glorified sysadmin, but that job in itself is ambiguous, as it could refer to the person deploying systems in digital virtuality or administrating digital systems in physical reality.

Apart from that: OP may have been hired as a "DevOps," but if you have to manually fix 2000 physical PCs, that is, in my "boomer" "Millenial" mindset, a sysadmin job, so I just led from there.

>>101472937
Not sure if this guy would be happy or not to receive a truckload of 2000 of them in one go, lol...


Wholly apart from that:
I've sysadmin'ed SOHO, around ~30 desk+laptops and ~15 mobile phones from 5-15 people, and if there's anymore than 8 identical client-PCs with only differing users, I'd start looking into thin-client models and how to remote administer them, and/but I'm not sure if I'm missing out or not in having never had the neccesity occur to me.
>>
>>101474224
And what are they going to do now? The pajeet remote support techs arent boarding planes to go safeboot all the windows machines
>>
>>101469302
ehh, you probably just "GOT the job without any experience.
First world nigger with no experience what so ever.

>Me: Third world Nigger, working for a Fortune 100 companies,
>Complain the shit every about things that can go wrong.
>Shit goes wrong!
>Told you, we hired an expert at day one and got this issue fix ASAP.

IMHO, you Rack Expelience at your comphy job
>>
Just take a sick leave for a month u complete retard
>>
>>101474266
a fake sysadmin left autoupdate on and paid the price.
It's a tale as old as time.
>>
>>101474232
Based blue collar chad shitting on pencil necks
>>
>>101474270
>it's practically a day off just walking around bootlooping machines
How big is your nose? Also if they aren't California cultists and part of some Jog For Niggers weekly team-building exercise, your average devops weighs more than the car he never drives to work.
>>
>>101474279
inb4 pajeet paratroopers
>>
>>101474311
So you're getting paid to do nothing. As long as you aren't salary who gives a shit just milk the clock and listen to an audiobook and collect overtime
>>
File: 1714274456560847.jpg (36 KB, 720x712)
36 KB
36 KB JPG
>>101474114
>>
>>101474271
I mean, even if you use fat-clients, strap a 64GB mSATA into a thin-client and it becomes a worthy modern Linux fat-client, now just configure one perfectly, dd image the disk and clone the others.

If you're pro, you pre-mount a copy of the disk-image using losetup -Pf disk.img, edit all the hostnames to match what you want and need, save that to the image and dd that "deployed" image into the mSATA for the fat-client.

If it just works, it just works, but if it fails, you may have to image-reset the fat-clients, which isn't toooooo critical in issue in SOHO settings, or even slightly larger companies.

Flights being grounded is an entirely different caliber, of course.
Good luck!
>>
File: nice.jpg (82 KB, 969x1024)
82 KB
82 KB JPG
>>101471496
>>
F
>>
>>101474304
There must be a lot of stupid sysadmins out there then. I thought you didn't have a choice in the autoupdate
>>
>Take Call Job
>Get A Call
>Cries
>>
File: tenor (3).gif (1011 KB, 498x498)
1011 KB
1011 KB GIF
>>101469302
>Windows DevOps
>>
>>101474474
*Preinstall an SSH server, or have one SSH server available to write to, and just cron or daemon a task to rsync regularly if users store data locally.
It's all so autonomous and independent and fucking 1337, I suppose normie management just isn't aware of what's possible and capable with just skill and the right software (and the hardware to run it on is cheap and abundant as a side-effect).
But you actually need the skill, as some r/sysadmin thread the other day proved opposite.
>>
>>101469302
Tell him that you'll do it... For a price.

If he's still stubborn then tell him to fix it himself or look for a new dev ops engineer.
>>
>>101469302
just quit brother, and no, you are not liable for a lawsuit
>>
File: image.jpg (565 KB, 2000x1333)
565 KB
565 KB JPG
>>101469302
Hey. You're stronger than you think, you know. You don't have to put up with that kind of abuse. Right what's wrong.
>>
>>101474246
The IT department deserved a culling
>wdym, why would you need a admin powershell?
>sorry, I can't just let you install sublime / vscode/ whatever, you guys should use visual studio
>sorry, we spilled water on the network switch, you might not have internet in the next few hours
>I can't enable virtualization / Wal for you, sorry
>I need to talk to your teams leader to clarify if you really need npm / python
>>
imagine interviewing a guy for a job and ask him
>so lets say theres a bug that requires reinstalling windows on the companys pc's and you're responsible for it, what would you do
>I'd send an email to my boss about hiring a lawyer, not repair the pcs and possibly quit
>wow you're hired
>>
>>101469302
>he told me even if I quit or get fired I still have to fix this outage or else I'm liable for a lawsuit
Bullshit. Anon stop being a pushover and tell him that you'll do a
It at your pace AND you want a 35% raise or you don't do it at all.
They need you more than you need them, especially right now.
>>
>>101474807
are you a bait ?
he said there are over 2k computers impacted, how is he supposed to do that manually by his own, are you one of the many retard who think you can automate it ? how can your perception of the feasibility of a task can be so completely fucked. You gotta be a neet who lives with his parent to be so clueless
>>
>>101469582
>that clip
crazy shit, but kind of inspiring. thanks for sharing
>>
>>101474892
that sounds like a ransom and won't good look on paper in a court
>>
File: GRlbHGabMAAdJKp.jpg (55 KB, 827x1019)
55 KB
55 KB JPG
>>101471496
It just might work.
>>
>>101469302
i know op is larping but
>even if I quit or get fired I still have to fix this outage or else I'm liable for a lawsuit
is this really a thing? can you be forced to work against your will like a slave?
>>
>>101469302
wagie wagie sit in cagie
ragie ragie work that 48 hour day gee!
>>
>>101470539
It's an easy as fuck fix though ain't it? OP makes a PDF with instructions, requests 10 additional monkeys from whatever department has non-retarded people and they get to work.
>>
>>101474941
and you think the reality of OP scenario is the task can only be done manually 1 computer at a time + no one else could possibly aid him in doing so out of the thousands of employees

obviously you can control multiple pcs at once and update software
>>
>>101469302
>one guy for 2000 machines LARP
if not a larp tell him to fuck himself and walk out now, you'll see him in court
>>
>>101475078
The whole point of this whole mess is that it can't be fixed automatically. Most PCs don't have PXE + network boot enabled, and even if they did if they use Crowdstrike they probably also encrypt the drive using Bitlocker whose servers are also impacted by this and most users don't even bother reading that they should keep their Bitlocker key backed up on paper.
>>
>>101474989
I mean I guess they can try? In the same sense I can try to sue somebody for saying I have shit taste in anime.
>>
>>101475078
>obviously you can control multiple pcs at once and update software
they likely can manage multiple pc at once and update software only if windows is working and each computer's agent can communicate, here the pcs won't even boot on windows so no, not possible unless the pcs are vms which is not the case or they have pxe enabled everywhere which again might not the case
>>
Just delete system32 bro
>>
>>101475168
finally someone who is not deeply retarded
>>
At the end of the day, this is what you get for injecting 3rd party dependencies. Oh let's save some money by making our dependent on a 3rd party so when they fuck up it blows our shit out of the water hurrr
>>
>>101469302
First time getting yelled at? What is this - your first week in IT?

You need to learn how to brush it off. Say "Yes sir - I'll get to work right away" then spend a couple hours making some flow charts with lots of technical jargon and acronyms, maybe even write up an 'action plan' on how you will safely and effectively restore functionality to your IT infrastructure that will ensure strong business continuity.

Your boss is a moron that doesn't understand technology - stop whining and just produce some type of information-visualization that will show him how technically illiterate he is - and then in a couple days when the CIA fixes their fuck up you take credit for all your hard work and demand a 15% raise in your performance review next year for your outstanding response to a massive worldwide security incident.
>>
>>101469302
UH OH MELTY

GET IN THE CAGIE
>>
>>101469302
>getting paid a devops salary to do some stupid shit over and over again for a few days is bad
>>
>>101474807
>so lets say theres a bug that requires reinstalling windows
okay
>in 48 hours
thank you for your time, but I have other career prospects.
>>
>>101471496
This is literally what anybody would do in GB or the EU. If he fires you it's a cast iron wrongful dismissal case here. In muttland you'll probably be executed or something.
>>
>>101469679

OP probably gives his Social security numbers when the indian scammers asked since they said he has to
>>
>>101475251
we're not talking about some dump npm module that is fucking up. We're talking about antivirus, you can't just make your own and expect it to protect your pc, you gotta be constantly looking after new threats, reverse engineer the newly discovered malware as fast as possible to make a signature out of it to maintain a up to date malware signature database, that's why there is company who does just that and nothing else, crownstrike doesn't just appear to be making antivirus they seems to have a whole range of products related to security so it's reasonable for a company to subscribe to spend good bucks on this kind of stuff
>>
>>101469302
Your a dipshit because you should've told him you need two or three data entry monkeys from whatever mail room to follow simple instructions and get the job done in 1/3rd the time. Then demand a bonus (fuck the data entry guys)
>>
>>101469302
No point in crying like a little bitch. Those machines aren't going to fix themselves IT boy.
>>
>>101475389
Anti-virus wouldn't cause boot failures, it's all the other 3rd party shit that's the problem.
>>
>>101471496
based
>>
>>101469302
Take it easy and calm down. He wont do shit. In the worst case, just take it to the court and you will win. He has no good reason to held you liable for that. Ignore all NEET genetic deadends ITT who trash you for having a life that they cant afford.
>>
>>101475043
1. He's not going to actually get that support because people are penny pinching bastards.
2. If a single monkey were to fuck up and delete the wrong file in System32 on any of the PCs, all hell breaks loose. Managers who know this will be even more reluctant to get the guy help.

The real solution would be to roll back changes for their versioned OS images and attach their decoupled drives to different compute but corpos gonna corpo. There's a reason ransomware still fucks everyone's shit up.
>>
>>101475371
>in 48 hours
*for 2000 PCs.
I mean, it's fine for one or two servers and shit, but it is inhuman to expect one man to do that, and even a judge can understand an IT task of physically having to handle 2000 PCs.

OP is most definitely a LARP, I'm not even going to entertain him, but the generic situation might exist, who knows.
It's F500 software, most would have departments, not a sole admin, but who knows.
>>
>>101474969
>court
Over what? He didn't cause the damage, no judge would side with the company. It's just some boomer acting tough with a meek nerd.
>ransom
It's called opportunity. Plus if they want him to do extra it's not unreasonable to ask for extra.
>>
>>101471496
Also good move
>>
>>101475478
>If a single monkey were to fuck up and delete the wrong file in System32 on any of the PCs, all hell breaks loose.
They'll go from bricked systems to bricked systems, so nothing changes. Not a real issue, just don't fuck up
>>
>>101475551
but it does go from a bricked system they know how to fix to one they probably won't know how to fix
>>
File: 1674860467147236.png (513 KB, 1000x1000)
513 KB
513 KB PNG
>>101469302
Take your time, I don't mind laying in bed while you fix my computer.
>>
File: 1495258501784.jpg (125 KB, 680x680)
125 KB
125 KB JPG
OP just do this you newfag
>>
File: smile.jpg (126 KB, 1200x630)
126 KB
126 KB JPG
I wouldn't know, never had a job, hahahahahaha.
>>
>>101475402
if he's in the usa he could easily gather an army of crackheads who usually wander around downtown and will follow any instruction you ask them to do for a buck, then from there he put two cackheads for one pc so they can assist each other when one is too much high, this will significantly reduce the time needed for such task given he manage to recruit hundreds of crackheads/homeless, this dude isn't even trying

>>101475486
if you ask for more money or else you won't fix what is causing the company a huge amount of revenue loss then even if you're not responsible for the situation that is still some form of blackmail
>>
>>101471295
> have a major general anxiety disorder
That's just nuspeak for being meek
>not my job
You said it yourself, this isn't your responsibility.
>sue me
They won't, just an empty threat, managers do this all the time to people easily intimidated. No judge would side with them.
>fire me
Nigger stop being a retard. They are pressuring you so much precisely because they NEED YOU BAD atm. Fire you? And then what? Not operate at all?
Besides, they are asking a lot of you, more work=more pay - it's perfectly reasonable for you to ask for extra, they are asking for extra as well.
>>
>>101469302
you stupid computer nerds have it so easy. bitching about press some buttons
how about you try fighting on the frontlines eh, limp-dicked poindexter?
>>
>>101475718
enjoy getting killed on duty meanwhile we are comfy in our a/c equipped open space, have to start working at 9:30 am and more importantly making three times your income
>>
>>101474580
Business windows lets you deploy whenever and delay for as long as your company needs.
>>
>>101475709
>blackmail
Not unless you caused said revenue loss. If they want you to do more work, you ask for more pay. By your logic, not doing unpaid overtime would be blackmailing the company.
>>
>>101471295
boomer execs will say WHATEVER they can to get you to give something your 1000%. it's THEIR job that's on the line and they can't do anything other than yell at you, so it's a given.
>>
>>101475846
well then my bad you're correct
>>
>>101475871
don't believe their lies, the supreme court will rule not doing unpaid overtime is illegal and carries a mandatory 10 year prison sentence of additional unpaid overtime
>>
>>101470178
I have just a bad news 4u. This shit is usually on machines with SecureBoot and TPM-ciphered volumes, so it's necessary to boot the original system to access things inside. Usual megacorpo policy by their security teams. The correct solution is to call whatever idiots want this shit installed and let THEM to repair and pay for damages. Not OP.
>>
>>101470779
yeah bro just run a script on a BSOD looping machine
dumb fuck
>>
>>101476035
It's actually worse than that. Safe mode means manual input of Bitlocker key is required and no one knows that shit. You either pray to God that you have a key recovery mechanism and that it doesn't depend on a server that's also bricked, or you just cut your losses and wipe the drive clean.
>>
>>101475871
I am European tho, that IS how it works here but maybe in the US it's different - labour laws over there are kinda...funny.
>>101475942
Only 70% sure this is a joke
>>
>>101475475
Seriously, this is why disaster recovery protocols exist.
The fact this needs to me manually remediated on every single machine is fucking hilarious, and almost all of my clients are down today with >100,000 machines soft-bricked.
>>
File: 1711095025549714.png (114 KB, 370x360)
114 KB
114 KB PNG
>we have the Bitlocker keys for all stations stored in AD
Oh thank goodness
>>
>>101473428
good. Bitlocker cucks deserve it.
LUKS was right and won.
faux bullshit security doesn't work and becomes a footgun in a critical failure path.
>>
>>101476280
I'm european too so unsure, yeah it's crazy out there they can fire you for practically no reason at least here everything is shit but our labour laws are pretty good compared to them
>>
>>101473983
Yeah, basically America labor laws have collapsed and we are trending towards late 1800s gilded age politics again
>>
So now that Crowdstrike is going down in flames, who will be the next market leader?
>>
File: 1721406689616575.jpg (1.99 MB, 3024x4032)
1.99 MB
1.99 MB JPG
I just found OP
>>
>>101469302
You heard the man, get to it IT monkey!
>>
>>101476206
>or you just cut your losses and wipe the drive clean
This is it.

We're going to get a lot of nice thin-clients on the market, because they're just going to buy the next system they'll be ready to wipe and start over anytime it is convenienced.

If we're lucky, one or two might learn about the value of technical autonomy (or technical autonomous systems) from this mess.
>>
File: 1711270645881891.jpg (67 KB, 526x700)
67 KB
67 KB JPG
>>101469302

I can't believe OP doesn't realize this is the opportunity of a lifetime

>I'll do it, but now I make 2x my previous salary, and I want the executive's health and dental coverage, plus 6 weeks PTO. Let me know when the paperwork's ready, I'll come right up and sign it
>or, I quit, and enjoy going through the hiring process with every computer in the building dead, hope you can get someone within a week
>>
>>101476855
that would just get one fired a week later
>>
>>101476968
sounds like 6 months' severance needs to go in that new contract too

I mean, imagine being the only person in that building who is at that moment completely irreplaceable and not realizing it. You're the only one who can fix what's crippled the business. They can't replace you. They should be throwing money and benefits at you.

That idiot middle manager drone is trying to scare the one absolutely key employee in the entire organization into saving them all. The ONLY response to that has to be "oh, if that's how you want to play..." and then you *unzip dick* and jam it right down their throats
>>
>>101476757
>familiar with computers
>dreads
>black
This dude loves anime
>>
>>101471496
kek this guy has to be jewish
based nonetheless
>>
>>101471938
>you will still be thinking about it all weekend
this anon gets it, I have the same issue
unresolved issues just hurt nonstop, doesn't matter how small. I envy people who can just ignore shit
>>
>>101477111
goes to show you need to write in clauses that you cant renegotiate contracts during an emergency and if you try to blackmail the company the employee needs to be liable for the damages they cause
>>
>>101471906
this, only if anon wasn't as retarded
>>
>>101476853
There are many machines where that step is not an option. On large companies. So sysop's nightmare.
>>
>>101471719
>>101472063
lmao mans had the opportunity to double his salary, get benefits and PTO and literally anything else he could think to ask for, and instead blew up his career because some middle manager tried to bluff him and it hurt his feelings. the manager's an idiot but OP is ngmi.

>>101477222
that's not how it works. they should've offered, but drone manager decided to try to swing his dick at a guy he clearly and correctly perceives as a huge pussy. he went all in on a busted flush, hoping OP would fold. the correct move is to call, OP instead flipped the table and stormed out of the game to go get hammered
learn to play poker, it's the real life training manual for every manager you'll ever meet, it's how they think about situations
>>
>>101469679
Sounds like you're gonna get sued either way. Lawyer up and resign, ASAP.
>>
>>101477269
Guess it's time the upper echelons stop using tech for their games.

I'm saying (ultimately) all global tech infrastructure like undersea internet relays should belong to a global tech union like the UN (despite its negative connotations, I need this association to place its standing among the executive parts of nations' governments) that all countries basically just agree to not fuck around with because the losses incurred by that are greater than any gain that can be had from it.

Well, that is maybe utopian or not.
I guess I have this kink of imagining utopias and wondering what kind of compromises one must make with reality to realize a practical implementation of them.
>OnlyPlanets
Take that, Rick!
>>
>>101477410
he's not going to get sued jesus christ how do any of you survive in real life
>>
>>101477337
>mans had the opportunity to double his salary, get benefits and PTO and literally anything else he could think to ask for
and then get laid off a few months later in favor of some other devops monkey willing to work for his old salary and no benefits
>>
>>101469302
Here's the solution to your problem:
>Write documented fix process
>Send to managers of departments, or offices
Tell boss to help you with the remaining, if he refuses tell him you quit. No you're not liable for a 3rd party company pushing out an update that broke your computers



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.