Tweet
As mentioned in the unsupportable thread, I worked an internship with the local government in Texas, and they weren't exactly up to date hardware or software wise. They had their moments, like a fiber link between the IT building and the courthouse, but they also had extremes in the other direction, like using Pentium 33MHz machines in the law library, connected in a token ring network. This means that the least technologically literate people had a network topology that doesn't play nice with "But I wanted the computer moved over THERRRREEEE!!!" This was also before wireless became really prevalent - 802.11G released while I was there - only higher end laptops had wireless at this point, so they couldn't make the wireless claim either.
Broken Ring tales of woe
Note for those unfamiliar with this network topology: The entire system is connected in a big ring using these huge connectors like this:
If any of those connectors came undone, the ENTIRE network goes offline, because the ring is broken. At the computer was a serial port, since that connector doesn't fit into a PCI/ISA slot form Factor.
We had a ticket system entirely on paper. Every day I'd get in, I'd have a stack of tickets to address on site. Some days, I'd have a stack 40-50 pages high, 50% of which was "THE LAW LIBRARY IS OFFLINE! I HAVE A PAPER DUE IN 5 MINUTES!!!!!!!!"
Some days, they'd make it easy on me, and leave stuff as it was when stuff stopped working. By "stopped working", I mean, try to move one of the workstations to the other end of the room, then freak out when the other 5 machines stop working too. This made it easy.
Others, they'd try to play computer tech and try to swap stuff around to see if it'd start working again, instead of doing the logical thing and undoing what they've done. 5 computers means 10 connections between computer and switch would have to be checked. Sometimes, they'd get smart, and try to plug it back in. Usually, they wouldn't lock the connector, and when they let go, it'd go loose again, and disconnect everything. This became daily for some law students, even though we banned them from the computer system, rather than just upgrade the network there.
Random Spring Storm
I was sent down to the work program office, where inmates who are good are allowed to go work on a limited basis outside the jail, usually to clean up trash along the freeway, or in the vicinity of the jail, to test a computer that was rebooting on its own at random intervals (read: power supply finally choked to death on cigarette smoke and dust)
I grabbed some hardware, and wandered down there. The sky was partly cloudy, and the sun was out, even if normal West Texas winds were gusting > 60MPH, I didn't think anything of it. After 15 minutes of troubleshooting and swapping out the power supply, I hear a LOUD clap of thunder, then the sound of hail on the roof.
I go to leave, and it's still pouring rain, although the hail's stopped. One of the guys told me to just wait out the storm and brought me a cup of coffee. Of course, right then, my walkee talkee crackles, and my boss calls me back to the IT building, saying it's urgent and to get there ASAP. I acknowledge him, finish my coffee, and sprint across the street and 150 feet down the sidewalk to the main building (where IT is, on the 5th floor), get in the elevator, and it's still pouring rain. I look like I fell in a pool. I'm totally drenched, and dripping water. I had long hair, which made it look worse. By the time the elevator reaches IT department, it's stopped raining and it's sunny again.
Boss: Whoa. what happened to you?
Me: Random spring storm. Looks like it's over now too. Figures. What did you need me for?
*boss tells me one of his infamous puns that's beyond groan worthy*
Me: Was that it?
Boss: Nope. Need you to go to the data center and swap out a tape drive in the robotic backup unit *hands me a SCSI tape drive*
Me: The data center's 60F, and I'm dripping wet. Mind if I dry off and get coffee first?
Boss: Sure, as long as it's done before backups run in the next 20 minutes.
Me: Sure thing. *sighs as I get other looks from everyone in the office walking to the coffee pot.*
Inventory...oh the humanity!
My next task was to get an inventory of the 500 computers we had just replaced with Dell's infamous Optiplex GX270 with the exploding caps. I get a list of all the info on each machine, wipe the hard drives with a special floppy I made that auto wipes the drive with no confirmation (Labeled with a skull and crossbones and a "DO NOT BOOT!!!!!!"), strip useful parts, like RAM, and a few hard drives for use as spares, make a spreadsheet, and go to save. Right then, SuSE decides to auto update, prevents me from cancelling, and keeps me out of all my applications.
It removes YaST (auto updater), the kernel, base_layout (commands like copy, move, change directory, and list directory structure - cp, mv, cd, and ls, respectively), then reboots, meanwhile, I have no real way to save my work. I'm the only linux guy there in an office of windows zealots, and of course, I get endlessly made fun of, since I now get to redo the last 20 minutes of work. Good timing, SuSE, and you're still the reason I hate RPM based distributions!
It's MEELLLLTING!!!!
I was sent out to upgrade the BIOS in a group of computers in the sheriff's office since it was causing problems with the upgrade to windows 2000 (this was 2004.) After 50 or so machines, I got to the point where I didn't even look at the computer, I knew where the floppy drive was. I sit down at the next computer, go to shove the floppy in the machine, and the floppy doesn't go in.
Me: Hrm...maybe it's upside down. *looks at floppy* Nope.
Employee: What's wrong? Don't you know how to use a floppy drive?
Me:
Yes. It's not fitting in the drive. *looks at computer to see...something's...off*
Me: The front of your computer's melted. Both floppy and CD drive are totally destroyed. Do you use a space heater?
Employee: Well...Err...Sometimes?
Me: you realize policy prohibits that, right?
Employee: I get cold!
Me: So put on a sweater. The space heater ban is there for exactly this reason. *puts the computer on the cart and takes it back to IT for an assessment from the Bosslady
Bosslady: She was using a space heater, wasn't she?
Me: Yep.
Bosslady: Tell her the replacement will come out of her paycheck, and get a new one from the store room.
Me: Ok.
Back on site...
Me: Ok, I have a new computer for you with the latest OS, and according to *bosslady*, you will have to cover the damages.
Employee: But!
Me: No more space heater!
Employee: Fine!
Forgotten, but still phoning home
I was handed a sheet with an IP address, switch it's on, and MAC address of a computer that's not in inventory, can't be connected to, but is still answering to pings and some requests...odd. It's in the jail, in the women's cell block. Great...
So I wind my way back to C block, go to the control room, and check MAC address/IP Address on all computers. After not finding it, I ask if that's it.
Sgt: Yep. That's it. Just the 3 machines.
Me: I have a 4th machine in this room, wired to the same switch. It's only halfway responding.
*Another Sgt I know from outside work wanders in. We'll call her Sgt H.*
Sgt H: What's going on, mister IT man?
*explains situation*
Sgt H: You know, I just found a computer yesterday that doesn't look like it's been touched in a while. *pulls desk out of corner, revealing a P1 Dell that looks like it's been there and totally untouched for years*
Me: Whoa... That's likely it. *plugs a monitor in, showing a win95 screen that's been locked up for who knows how long.*
After unplugging the computer, I see the dirt's pretty much welded the computer to the floor, and I had to have the Sgt H pull it up from the back since I couldn't get to it. I take it up to IT to show the boss, and he at it, before sending me up to strip it. When I get there, there's so much dust inside it that it's full from motherboard to top, with a small air channel going from grille at the front, by the CPU, and out the power supply. Everything else is totally covered. It was REALLY epic. Who knows how long it's been there. It still had an IP address, and would respond to hits from the remote management software we wrote for in house use, but was otherwise totally locked up.
And finally:
Exploding power supplies
During the inventory, a coworker found an old 486 server with 4 redundant power supplies in it. We needed to determine whether or not it worked, so he plugged it in.
Coworker: (grinning) Stand back, in case it blows up! *presses the power button*
Server: *WhirrBANG BANG BANG BANG!......*
Coworker:
Me:
You killed it! You murderer!
*Coworker unplugs it, shoves it back in the corner he found it in, fans the smoke a bit, and wanders off to see what can be done.*
Meanwhile, I go back to getting serial numbers off of the stack of 10MB MFM Seagate drives in the other corner. Job was fun, but you never knew what you'd find.
Broken Ring tales of woe
Note for those unfamiliar with this network topology: The entire system is connected in a big ring using these huge connectors like this:
If any of those connectors came undone, the ENTIRE network goes offline, because the ring is broken. At the computer was a serial port, since that connector doesn't fit into a PCI/ISA slot form Factor.
We had a ticket system entirely on paper. Every day I'd get in, I'd have a stack of tickets to address on site. Some days, I'd have a stack 40-50 pages high, 50% of which was "THE LAW LIBRARY IS OFFLINE! I HAVE A PAPER DUE IN 5 MINUTES!!!!!!!!"
Some days, they'd make it easy on me, and leave stuff as it was when stuff stopped working. By "stopped working", I mean, try to move one of the workstations to the other end of the room, then freak out when the other 5 machines stop working too. This made it easy.
Others, they'd try to play computer tech and try to swap stuff around to see if it'd start working again, instead of doing the logical thing and undoing what they've done. 5 computers means 10 connections between computer and switch would have to be checked. Sometimes, they'd get smart, and try to plug it back in. Usually, they wouldn't lock the connector, and when they let go, it'd go loose again, and disconnect everything. This became daily for some law students, even though we banned them from the computer system, rather than just upgrade the network there.
Random Spring Storm
I was sent down to the work program office, where inmates who are good are allowed to go work on a limited basis outside the jail, usually to clean up trash along the freeway, or in the vicinity of the jail, to test a computer that was rebooting on its own at random intervals (read: power supply finally choked to death on cigarette smoke and dust)
I grabbed some hardware, and wandered down there. The sky was partly cloudy, and the sun was out, even if normal West Texas winds were gusting > 60MPH, I didn't think anything of it. After 15 minutes of troubleshooting and swapping out the power supply, I hear a LOUD clap of thunder, then the sound of hail on the roof.
I go to leave, and it's still pouring rain, although the hail's stopped. One of the guys told me to just wait out the storm and brought me a cup of coffee. Of course, right then, my walkee talkee crackles, and my boss calls me back to the IT building, saying it's urgent and to get there ASAP.
I acknowledge him, finish my coffee, and sprint across the street and 150 feet down the sidewalk to the main building (where IT is, on the 5th floor), get in the elevator, and it's still pouring rain. I look like I fell in a pool. I'm totally drenched, and dripping water. I had long hair, which made it look worse. By the time the elevator reaches IT department, it's stopped raining and it's sunny again.Boss: Whoa. what happened to you?
Me: Random spring storm. Looks like it's over now too. Figures. What did you need me for?
*boss tells me one of his infamous puns that's beyond groan worthy*
Me: Was that it?
Boss: Nope. Need you to go to the data center and swap out a tape drive in the robotic backup unit *hands me a SCSI tape drive*
Me: The data center's 60F, and I'm dripping wet. Mind if I dry off and get coffee first?
Boss: Sure, as long as it's done before backups run in the next 20 minutes.
Me: Sure thing. *sighs as I get other
looks from everyone in the office walking to the coffee pot.*Inventory...oh the humanity!
My next task was to get an inventory of the 500 computers we had just replaced with Dell's infamous Optiplex GX270 with the exploding caps. I get a list of all the info on each machine, wipe the hard drives with a special floppy I made that auto wipes the drive with no confirmation (Labeled with a skull and crossbones and a "DO NOT BOOT!!!!!!"), strip useful parts, like RAM, and a few hard drives for use as spares, make a spreadsheet, and go to save. Right then, SuSE decides to auto update, prevents me from cancelling, and keeps me out of all my applications.
It removes YaST (auto updater), the kernel, base_layout (commands like copy, move, change directory, and list directory structure - cp, mv, cd, and ls, respectively), then reboots, meanwhile, I have no real way to save my work. I'm the only linux guy there in an office of windows zealots, and of course, I get endlessly made fun of, since I now get to redo the last 20 minutes of work. Good timing, SuSE, and you're still the reason I hate RPM based distributions!
It's MEELLLLTING!!!!
I was sent out to upgrade the BIOS in a group of computers in the sheriff's office since it was causing problems with the upgrade to windows 2000 (this was 2004.) After 50 or so machines, I got to the point where I didn't even look at the computer, I knew where the floppy drive was. I sit down at the next computer, go to shove the floppy in the machine, and the floppy doesn't go in.
Me: Hrm...maybe it's upside down. *looks at floppy* Nope.
Employee: What's wrong? Don't you know how to use a floppy drive?
Me:
Yes. It's not fitting in the drive. *looks at computer to see...something's...off*Me: The front of your computer's melted. Both floppy and CD drive are totally destroyed. Do you use a space heater?
Employee: Well...Err...Sometimes?
Me: you realize policy prohibits that, right?
Employee: I get cold!
Me: So put on a sweater. The space heater ban is there for exactly this reason. *puts the computer on the cart and takes it back to IT for an assessment from the Bosslady
Bosslady: She was using a space heater, wasn't she?
Me: Yep.
Bosslady: Tell her the replacement will come out of her paycheck, and get a new one from the store room.
Me: Ok.
Back on site...
Me: Ok, I have a new computer for you with the latest OS, and according to *bosslady*, you will have to cover the damages.
Employee: But!
Me: No more space heater!
Employee: Fine!
Forgotten, but still phoning home
I was handed a sheet with an IP address, switch it's on, and MAC address of a computer that's not in inventory, can't be connected to, but is still answering to pings and some requests...odd. It's in the jail, in the women's cell block. Great...
So I wind my way back to C block, go to the control room, and check MAC address/IP Address on all computers. After not finding it, I ask if that's it.
Sgt: Yep. That's it. Just the 3 machines.
Me: I have a 4th machine in this room, wired to the same switch. It's only halfway responding.
*Another Sgt I know from outside work wanders in. We'll call her Sgt H.*
Sgt H: What's going on, mister IT man?
*explains situation*
Sgt H: You know, I just found a computer yesterday that doesn't look like it's been touched in a while. *pulls desk out of corner, revealing a P1 Dell that looks like it's been there and totally untouched for years*
Me: Whoa... That's likely it. *plugs a monitor in, showing a win95 screen that's been locked up for who knows how long.*
After unplugging the computer, I see the dirt's pretty much welded the computer to the floor, and I had to have the Sgt H pull it up from the back since I couldn't get to it. I take it up to IT to show the boss, and he
at it, before sending me up to strip it. When I get there, there's so much dust inside it that it's full from motherboard to top, with a small air channel going from grille at the front, by the CPU, and out the power supply. Everything else is totally covered. It was REALLY epic. Who knows how long it's been there. It still had an IP address, and would respond to hits from the remote management software we wrote for in house use, but was otherwise totally locked up.And finally:
Exploding power supplies
During the inventory, a coworker found an old 486 server with 4 redundant power supplies in it. We needed to determine whether or not it worked, so he plugged it in.
Coworker: (grinning) Stand back, in case it blows up! *presses the power button*
Server: *WhirrBANG BANG BANG BANG!......*
Coworker:
Me:
You killed it! You murderer!*Coworker unplugs it, shoves it back in the corner he found it in, fans the smoke a bit, and wanders off to see what can be done.*
Meanwhile, I go back to getting serial numbers off of the stack of 10MB MFM Seagate drives in the other corner. Job was fun, but you never knew what you'd find.
We still have entire buildings running on 2000.
.
Comment