We had an IT network system failure recently and our email was down for several days due to data corruption. Over the years the data base to our Exchange email had grown to a pretty big size so running the necessary tools to clean the corruption and recover our system took a long time. We're talking many hours spread over several days.
Once recovery occured going through the long list of material, up to five days old (covered a weekend) it is really interesting to see how out of date and non mission critical it becomes. It really shows how fast things move in the current environment and how non urgent much of our email traffic can be.
What seems critical at the time of writing quickly becomes stale when seen five days later.
What is the lesson in this?
I think one of the lessons is the push to instant communication that started with email and that has grown with chat and text messaging is largely artificial. Much of what we deal with 'urgently' is really not important nor urgent at all. The false sense of urgency makes the communication less compelling.
In discussing this with a colleague today he described how his car was broken into one night just before he headed out on a business trip for four days. Not having time to get a replacement phone he was without this immediate communication for the duration of his trip. He was able to get email in the hotel at night, but no constant interruptions, no immediate responses. The only call he got in the four days (routed through a colleague travelling with him) was from his son and it was not urgent. This colleague is a senior marketing executive who you would expect really required constant availability. In his words it was a nice break and in the end the lack of the phone did not affect his performance or those of others at all.
Inconvenience is not a critical issue?
During out outage we were able to put in place a few workarounds that helped to mitigate the lack of our system. Using personal emails and web based tools we were able to accomplish many of the things that would have naturally gravitated to email. There were items missed and some catchup is natural but the outage was not a total disaster and for a large part we have gotten through it okay.
Of course we will be doing a systems and procedures check to see what we can do to avoid such an occurance in the future. One thing for sure is we will be altering the way we store back information in emails and attachments to reduce the size of our Exchange data base.
Once a full analysis is done on what caused the outage, we suspect a faulty hard drive in the RAID storage that did not fully fail but introduced corruption into our records, we will be able to put steps in place to avoid such issues. We will also be able to provide advixe to clients and others who might face similar challenges.
The conclusion that I have come to is that email is generally important but not usually critical. Missing it for a few hours or even a few days will probably not cause undue harm. There are exceptions to this, we were really concerned about our support email address and whether we had missed or delayed any support needs because we did not receive the emails on a timely basis, but they are few and with some planning they can be corrected.
How critical is email to your business processes? Can you function with an interruption? Has the impact of a failure and prolonged outage been analysed and mitigating plans put in place?
Your comments are welcomed.