Master of the Domain

So I get in this morning, and all's cool. All the jobs ran, and no major errors. People seem to be having issues logging into the web server, but that's nothing new. (4 day weekend, perhaps?) So it looks like a light day.

*ring* *ring* Dammit....

ME: 'Company X', this is Sammich...

GOSAS: Morning, Sammich. It's Good-Old-So-And-So.

ME: (Genuinely happy. She's got a clue!) Morning, GOSAS. What's up?

GOSAS: Well... nothing.

ME: As in...?

GOSAS: Application1, Application2, Application3... nothing.

ME: (sigh) and how many users are affected?

GOSAS: Everyone.

ME: 'K. I'll check it out and give you a call.

So I hit the logs. Usually, when everything is down, it's a network issue. Everything is fine from my side, so I go log into the client side and take a look. Nothing's there. This means two things. 1) It's either a domain controller issue or a firewall issue. 2) There's no way out of this. I'm going to have to call... the help desk.

Now, having been on help desks in the past, I can say that it is the most thankless job in the world. And, one of the most difficult. Still, I was thrilled to have someone tell me exactly what to do, and where to send the ticket when they called. Made my day, since I didn't have to do any troubleshooting with clueless users, or make tons of executive decisions. Not so with this desk apparently. I can tell them exactly what to put in the ticket, and where to send it. Does not matter. It will go off into the ether with contradictory information, about 12 out of 13 times. I say 12 out of 13, since today was my 13th call to them, and they finally did something right. They transferred me directly to the department I wanted the ticket to go to. Now that's service.

So, we narrow things down to the fact that a client-side domain controller isn't responding. This jives with what I see in the logs. (Session setup failure between my DC and theirs. If everything on my side is authenticating fine, then it's gotta be theirs.) No response from their side. Not even a ping. So... I walk over to the local hell desk and ask if the RD can come over and reboot the server. After much discussion on wether or not this is possible, allowable, authorized and whatever, he hits the power button twice. 5 Minutes later, we're good to go.

Now here's the punch line. The first time this error showed up on my side? 10:30 AM yesterday. You'd think someone would be watching these things. Like me. Considering that I got no calls yesterday, I assume most people were in with cached credentials. That works fine until you reboot or your session times out.

Anyway. All's well in the world now. On to security auditing!