BOFH moment
Awesome comedy moment today in uni. Around about 4.50pm a few of us were left in the office just doing work and so forth. Suddenly the lights flicker and dim for a split second then come back up. At the same time a few of the computer fans in nearby unused workstations power down. I figured we had a power spike or cut temporarily. As everything else is still working, I carry on typing since no computers have crashed.
About 20 seconds later the "A network connection was unplugged" balloon appears in the notification area which is when I knew the system has gone down big style somewhere.
5 seconds later three sets of heavy footsteps can be heard thundering past the office and barging through the stairwell doors.
The computer support office is located right by my office on the first floor. All the machine rooms and servers are in the basement. That was the sound of all remaining support staff sprinting for the machine room having realised that the entire network has crashed and that they have to run to ensure the main servers haven't been destroyed and to safely terminate any other equipment that is still running. I just cracked up in the lab because it just reminded me of scenes from reading BOFH.
At this point realising that we have no internal network/ no internet access / no anything I quickly saved my work to the local disk as one power dip can often be followed up by others and our office computers might not have taken a second hit.
For those who've never heard of it (probably most of you) BOFH stands for the Bastard Operator From Hell - a fictional system administrator at a university whose sole goal in life is to make use of the computer equipment hell for his users. http://bofh.ntk.net/Bastard_Indexes.html for the stories. Causing scenarios like this is what he spends most of his time doing.
We found out later that all the file servers and infrastructure were OK but one of the main compute servers has a "hardware fault" (Ohhhh! That explains everything...nice and specific!) which is due to repaired Monday. Not that this matters as we have several of them. Also maybe its time the department considered a UPS system for our servers so they could shutdown gracefully when this sort of thing happens.
About 20 seconds later the "A network connection was unplugged" balloon appears in the notification area which is when I knew the system has gone down big style somewhere.
5 seconds later three sets of heavy footsteps can be heard thundering past the office and barging through the stairwell doors.
The computer support office is located right by my office on the first floor. All the machine rooms and servers are in the basement. That was the sound of all remaining support staff sprinting for the machine room having realised that the entire network has crashed and that they have to run to ensure the main servers haven't been destroyed and to safely terminate any other equipment that is still running. I just cracked up in the lab because it just reminded me of scenes from reading BOFH.
At this point realising that we have no internal network/ no internet access / no anything I quickly saved my work to the local disk as one power dip can often be followed up by others and our office computers might not have taken a second hit.
For those who've never heard of it (probably most of you) BOFH stands for the Bastard Operator From Hell - a fictional system administrator at a university whose sole goal in life is to make use of the computer equipment hell for his users. http://bofh.ntk.net/Bastard_Indexes.html for the stories. Causing scenarios like this is what he spends most of his time doing.
We found out later that all the file servers and infrastructure were OK but one of the main compute servers has a "hardware fault" (Ohhhh! That explains everything...nice and specific!) which is due to repaired Monday. Not that this matters as we have several of them. Also maybe its time the department considered a UPS system for our servers so they could shutdown gracefully when this sort of thing happens.