Unforgettable work moments that don't crack the nod for my CV

 

A CV does not really record your work experience. It doesn’t tell about those unforgettable moments…

• I was called to a motor car factory where the paint shop was reported as having intermittent network problems. When I went to the network cabinet at the paint shop it was covered in pigeon poo. The poo was corroding the network chassis. That was a moment of being in sh1t.
• A colleague used a desktop fan to cool the telecommunications equipment. The bearings failed and the resultant smoke caused the gas to be dumped in the data centre. All this on 31 December 1999. That was when the sh1t hit the fan.
• Someone installed equipment in a cabinet and did not put the tops on the electrical power plugs. I reached in to reset a router. It was a shocking experience.
• An air conditioner iced up in the DC and over time, unnoticed it created a small pond underneath the floor. Finding nothing else to use to pump out the water we appropriated the industrial wet and dry vacuum cleaner from the facilities department and sucked it out. We broke the vacuum cleaner but solved the problem.
• While working at an ISP a building was being constructed next door. A customer's car was encased in concrete from a bucket that failed from 10 floors up.
• A pipe starting leaking in a network room. The closet thing we had to prevent a disaster was to place an umbrella over the network switch.
• The water mains ruptured on the top floor of the building. Operations shut down for a day…
• The water reservoir on the roof sprung a leak. Same result as above. Ceiling boards all collapsed and ended up on our desks.
• A user in one of the departments installed a server on his desk. The whole department started using it and when it crashed he came over to ask where the backup is?
• The building was remodelled. The kitchen was placed above the data centre. The geyser in the kitchen burst…
• The data centre burnt down and the fire department arrived and finished off the equipment by drowning it all.
• Air conditioners were installed in the data centre but their start cycle was not sequenced. On the first power failure the generator started, all the air conditioners came on-line, overload the circuit and the generator shut down. This repeated itself until the starter battery on the generator exploded. I was standing a few metres away and had to change my shorts afterwards.
• The first time in the tunnels under the airport to test a cable and I encountered a rat the size of a dog. I nearly screamed for my mommy.
• Cable spaghetti is common, but add torn off pieces of paper attached as labels to the cables using discarded telephone cable to tie it on, and you have the winner of messiest data centre in the world.
• There was the battle of Luthuli house and when the AK47’s started shooting, I did the office floor leopard crawl.
• Same car factory as the poo incident. A Peterbilt truck collided with the data centre. It felt like an earth quake.
• I received a R250,000 mainframe processing bill for creating a branch line report.
• The network failed because the 10Base2 terminator heated up in the sun and increased the resistance.
• The network switch that was covered in fat from cooking as it was located above the stove in the canteen.
• Spanning the core switch trunks to each other and the whole network going dark.
• Loading OS/2 with 17 1.44" diskettes, multiple times and sometimes on the same server!
• Putting my foot in the dustbin during a change management meeting and being unable to extract it (all people in the meeting LMAOROFL)
• Reading a network magazine and seeing the phone number of a video conference unit in one of the case study pictures. Proceeded to test it and dialled into the Anglo-American Board meeting.
• A new guy starting at work. He was trouble-shooting why the Windows net message wasn’t working for a user. He logged onto the company domain controller and broadcast “WHY THE F**K DOES THIS NOT WORK?” to all computer screens in the company.
• We decided to install a fake blue screen saver on the main company file server as a joke. The administrator walked past, saw the screen, and switched the power off on the server.
• There was a screen saver called Johnny Castaway that killed the NIC on a PC. On a token-ring network this would result in shot gunning as the relays open and closed trying to resolve the problem.
• Being at a building site when the cladding and scaffolding failed. Two builders died and the dust cloud and noise in still etched in my memory. The dust cloud was like a tsunami. For a moment, I nearly did something involuntarily in my pants.
• The boss swearing and kicking the wall leaving a boot print on the wall of the main meeting room.
• A vendor arrived at the data centre and decided to charge his phone. The charger did not quite fit snugly into the strip plug so he put his foot on it and forced it in, The main circuit breaker tripped with the short circuit and the whole data centre went dark!
• A techie was sent to a remote location to troubleshoot a link. When he arrived he unplugged the main router to charge his phone. Three hours later the NOC finally worked out what went wrong. In all this time they were talking to him on his phone.
• I was at a red light just outside work. I noticed a taxi driving towards the intersection with sparks flying out from beneath it. In horror, it next ploughed into a group of pedestrians at the intersection. It hit one guy so hard that he flew up into the air, hitting a large tree about halfway up. I jumped out of the car and ran over. He looked dead. A few seconds later a doctor who was in the traffic arrived and the guy started groaning. The guy's life was saved but he was severely disabled. The taxi driver ran away.
• ICASA raiding WBS with a warrant to confiscate radio equipment but taking the fibre equipment instead. The data centre manager at Gallo Manor allows them to do that without bothering to phone us and tell us. The company for whom the data centre manger works suffers the biggest loss of connectivity as the location is a single point of failure for them which was never addressed even though the risk had been highlighted.
• During the same ICASA raid DNS stops working on the iBurst radio network. The VAS guys had told us pork pies about the DNS servers being resilient across two data centres.We changed the DNS servers to 8.8.8.8 and voila, up and running.
• A bolt falling from the roof of a vendor neutral data centre. It smashed the windscreen of a car. A few minutes before that a number of people including myself had walked past that point which was a few metres from the entrance.
• Deciding that it was a good idea to embed a red alert wav file in the p0rn0 blocking software so that the culprit would catch a skrik. It was very funny as when it happened all the person's work colleagues around him knew what had happened. That was until it happened to the groot baas...
• The file server running out of space. An audit of the home directories shows a user has copied a large number of gigs of pics and movies to the server. It was selfies of a very hot co-worker and they were naked and... let us leave it there...
• I learnt to programme on a ZX81! This was in the early eighties and within ten years of starting on a ZX81, I was on an IBM mainframe.
• The facilities department deciding to drill through the dry walling in the network rooms and covering all the network equipment in a fine white dust. Needless to say, the switch failed...
• The IT Security bunch decided to run a vulnerability scan in the middle of the day. They forced the network switches in the dealing room to fail and reboot. The dealers shouted blue murder!
• When LANs first started, menus where the in thing. Also there were these p0rn0 .exe files and someone named one of them 123.exe to hide it as a spreadsheet programme and then copied it to the LAN server. The selection of the spreadsheet menu item resulted in... (you can guess the rest!)
• This has happened to me many times. I often work on networks with fault tolerant paths. One of these paths fail and every is 100% still connected as the alternative path is being utilized. A network engineer is dispatched to fix the broken path but instead unplugs or works on the remaining live path. Next moment, total black out!!!!
• The IT systems guys typing in the specification for the launch of new products using an article from MyBroadband because no-one emailed them the internal product description.
• An employee phoning from New Zealand, saying he had immigrated and could he carry on working from home? He had asked to take a day off work because he had broken his toe (doctor's note in tow) and then disappeared. He never resigned, he really thought he could telecommute.... Fired his ass!
• Downing a bottle of Tequila before starring as Dr Frankenfurter in a Rocky Horror skit at a team building function.
• The Internet being burnt down in 6th October City.
• The surveillance camera being disabled in the NOC because the recording of activities was a constitutional violation of rights.
• The air conditioners in the data centre failed, again. The data centre overheated. It was decided to leave the servers on and open the doors for ventilation!
• Most data centres have designated hot and cold isles. After a night of mounting servers, the server team realized that they had mounted the servers the wrong way around when they switched them on. The hot air was blowing into the cold aisle. They left it like that instead of correcting the problem.
• The two cabling contractors who disconnected the SAA mainframe when pulling through a CAT 5 cable under the data centre floor boards. I then helped the IT manager with our first mainframe cold boot attempt.
• The employee who after a drunken office party decided to take a dump on the Head of IT's desk (and not realizing there was a security camera!)
• Trying to persuade the data centre manager of a leading ISP in South Africa to use IoT. He said he didn't need it as he has security guards monitoring an LED panel with HVAC alarms on it.
• Discovering that the top of the range Cisco firewalls at the Telkom campus took 20 minutes to boot as they were configured with two million rules.
• Being accused of inappropriate behaviour in a zoom call when commenting about a noisy hadeda by international conference participants who have no idea what a hadeda is...
• Chasing rogue DHCP servers!
• Scratching my head and troubleshooting network problems for months till I'm blue in the face. Log a dozen tickets against the ISP only to discover a dodgy demarc install by the ISP onsite.
• Persuading the High school headmaster to buy an Apple IIe which was then used for people to learn how to program. One of those was the guy who gave the world the Cisco licensing scheme.
• Seeing my life flash before my eyes when a building's scaffolding collapsed. Being a network engineer can be a dangerous occupation.

How does a person list these experiences?

Ronald Bartels works for Fusion Broadband is driving SD-WAN adoption in South Africa.

This article was published over at LinkedIn: Unforgettable work moments that don't crack the nod for my CV