I assume most here now mainly use cloud deployments of exchange however just in case there are any of you that use on premise and have a redundant site I thought I would share with you what can only be described as the weekend from hell.
We have 2 exchange servers, 1 on site A and 1 on site B. It hasn't been uncommon in the past that without warning, emails just stop appearing. This is usually down to the 2 servers stopping talking (for no reason) and a restart of both and it processes the missing emails.
The same happened yesterday morning, but the usual restart did nothing. I got error after error sending on all sorts of wild goose chases. Database's wouldn't mount, they would be out of sync, certificate errors, services set to auto wouldn't start, started services couldn't be stopped. You name it, it errored. No matter what I tried it just would not work. Comms between the 2 sites perfect and all other replications working a treat as normal.
Whilst searching pages and pages of forums I noticed hidden in one complain that MTU had caused a similar issue which lit a bulb in my head. Site A is leased line, high end Cisco routers etc etc but site B is just plain old vdsl and the PPPOE session created by the firewall. Due to issues with Dualcom, I have had to put a Draytek 2860 to managed the VDSL connection so that I can add a Cisco 5505 to mange the connection to CSL and the existing firewall do the rest. I did this on Friday with no problems but it put the idea in my head that these issues had only occurred since this work had been done although it worked fine for 24 hours in this setup and comms down the tunnel were fine.
Dissed the Draytek and made the current firewall manage PPPOE as it had done before and all these pages of errors disappeared. I am going to see if this is an MTU issue or if the IPSEC doesn't like it and try Open VPN to see what the root error was.
For alarm notifications and support emails these don't touch Exchange due to previous strops it throws with anything non Microsoft so we use a very reliable, simple email program called MDaemon for that. Ironic that the reliable solution costs a tenth of Exchange.
An unusual problem that has caused me some serious migraines this weekend so though I would share in case anyone else has the unfortunate job of administering Exchange.
All healthy now