Amazon Web Services big outage was caused by human error.
“Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended.”
Whoops, indeed, but been there, done that.
Amazon Web Services big outage was caused by human error.
“Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended.”
Whoops, indeed, but been there, done that.
So, your incompetence has cost a company or shareholders millions, or inconvenienced a large percentage of the customer base?
That’s the kinda thing that gets people fired, and rightly so.
In 2001 I opened an e-mail from a buddy of mine with Anna Kournikova in the title. It was the AK virus of course and my Outlook automatically sent it to everyone in my 500 person engineering firm. Good times. Lol
What level of incompetence does software protect against…. The back door is wide open & evil will find its way into stupid….SHHHHHHHHHHHHHHHHH Its all a toy game for children with a time line measured in weeks
Yep…if the server reboots fast enough, nobody will even notice, right? No need to sent out a broadcast message and deal with the barrage of individual whingings…
Well … that’s just ONE MORE job … for the Robots to do
You gonna fire a guy for doing a ‘-a’ instead of a ‘-s’ arg, Yeahwell? You know they pay the tier 1’s shit. You know s/he didn’t write the script. You know they’re following the scrip.
And I call BS on your number, pulled out of a hat, much?
Yep, I’m gonna fire a guy for making a mistake, if the mistake is big enough.
If tier 1 workers are given the necessary authentication credentials to take down server farms, well I’d fire some management folk, too.
“You gonna fire a guy for doing a ‘-a’ instead of a ‘-s’ arg, Yeahwell?”
The classic excuse. Like a surgeon saying “It was just a small clamp that I left inside the guy when I closed up”.
Up and running in no time…unlike Gov of Can Phoenix program.
today’s comic relief:
back in the dinosaur era of mainframes, I wuz 5 minutes away from end of shift when I allowed a compact tape to fall from my grip and bounce off the control switches. followed by a keeeRASH of the operating system.
oopsie.
std solution wuz to reboot and document the restart. but then all those dozens of users would lose their unsaved work followed by an invitation to the , uh, ‘red carpet’ to expliquez things.
instead I hit the continue switch which gloriously got the mainframe running where it left off. then I tracked down the printer utility which took the brunt of the ‘damage’ and restarted it. so far so good.
then I issued a message over the intermediary system which handled communications with the dumb terminal users to ‘carry on’, and then reconnected it with the now functioning mainframe. presto !
and walked out the door. never told anybody for months after. my workplace adage then became ‘you really know your stuff when you can hide your mistakes’.
maybe Rance has never heard of the “Gimli Glider”. Some mistakes can not be just brushed off:-)))