Exchange Server Disaster Before CNY

Never imagine one of my clients Exchange server would be busted before Chinese New Year. Well, this is a sign to work during holiday. There is no way to escape and the reason is simple. They only have one server and manage everything. This is what I fear the most working with people leading the department who have stingy mindset and not bother to foreseen such situation. There is no excuse on myself too not spending time monitoring the server health using some free tools such as Spiceworks. It just simply too hard to be working a bit server administrator, then IT Technician and then admin job nowadays. When you are exhausted, you going to forget your next pending job sooner or later if you don't focus on one thing properly.

Despite being good at maintaining Windows Server 2012, but when it comes with Microsoft Exchange. I am nothing with no single knowledge managing and understand how Exchange really works. Using search engine looking for troubleshooting and solutions from time to time does help. Not much helpful when it comes to serious problem. I tried to study Exchange, so far not much progress except learning how to set up does give me some experience.

Looking at the problem my client server is facing, for one, the Exchange storage hit with low disk space. Secondly, the external mail entirely stop coming due to its high pressure suffering low disk space. Lastly, I found Outlook 2007-2013 no longer able to connect to Exchange and kept getting errors. I do not know if Exchange shutdown all its running services causing such errors. My mind is telling me this is not good.

I don't denied my part miscalculate the partition size back in 2013 I personally first setup the server. Back then, I am confidence the mail database won't expand when the Outlook would download everything into the user computers. However, later I found out the fucking new Microsoft Outlook 2013 option to deliver Exchange emails to a pst-file has been officially removed. This backlash is the result of fast growing mail database size in one year. The solution to such problem either can either create POP3 account or add rules to do the job. To me, this is just unnecessary extra works.

Another way, is run Exchange maintenance mode and perform compacting mail database to squeeze the size maybe once or every few months. Honestly, this is something I trying avoid and even well suggested by many Exchange expert because it's very dangerous task that can corrupt the mail database. Again, I have no good knowledge and best not to touch such maintenance duty when never once done any recovery mail database experience before. Why take that risk?

My friend told we can take it slowly to find solutions to fix this problem instead jumping straight suggesting reformat the server. The problem here is our clients have no patience playing waiting games during their crucial time working with upcoming new tenders and projects. So it's not good idea to drag any longer before they start working on Tuesday (after Brunei National day). Moreover, there's no guarantee we can solve the problem because none of us are even trained Exchange admin in the first place. The only thing already perform are the necessary solutions to get the Exchange working again for two days straight and seeing so many weird errors showing up in the events log drive me crazier before the CNY holiday starts. Reformatting and rework the server is the best option during the holiday time.

I bring it back to the office on CNY eve and starts working on the Friday and Saturday. On Monday afternoon, I send back and starts rejoin all domains and slowly warp up the remaining pending setting until Thursday this week. It was tiring and stressful reworking from scratch. I revised the Exchange storage size from 200GB to 700GB by sacrificing 80% data size. I also upgrade to Service Pack 1 and Cumulative Update 7 for Exchange Server 2013. There are no second thought or regrets doing so since everything now fresh installed. Why not just throws in everything? I did somehow notice Microsoft puts out quite a few new options or features in the Exchange console. Nonetheless, my priority are as long the emails start coming in and out. Those shitty new features or not are not my concerns.

By laws, Exchange must need to have its own powerful server to run and separate from the main server. It was not supposed to run with single server in the first place. Like I said earlier, stingy mindset brought the problem. If I in charge on the sales, there definitely going have two servers rather one. In case my client can only afford one server, I would convince them have their Emails hosted on Cloud. Even though we have crappy ISP in Brunei, why bother with so much worrying when you get 100% up time, 24hrs support and can use anywhere anytime?

What do I learned from this? I lost my holiday mood and unnecessary stress. That's it. It was supposed my resting month. Things just doesn't turn out to be after this incident. Hard to say how many times had repeated itself. I seriously really need to rethink my role if its wise to continue working as multitasking job this year with the company.