2. Customer Impact
Not all failures of systems have impact that our customers see — but where they do, the CoE must cleanly and clearly articulate what customers saw, what their experience was, and consider the impact that had on them.
Any financial losses or impact, as a result of the failure, should be clearly stated in this section.
It is important to view any failure or outage from the point of view of customers.
What did they see on our site? What was their experience of the outage or failure? If they were directly impacted, how many and what exactly did we do to remediate them?
Not all outages or failures may have a directly visible impact — but where they do, this should be articulated in detail. If customers saw an error, were unable to communicate or transact with us, or worse, both of these things seemed to be happening but were not, that should be detailed in the CoE.
Three failures of the same system, written up as our customers experienced them.
No customer was affected. The failure was contained to a standby database which serves no customer traffic, and the site continued to be served throughout. We are writing this CoE because the failover we rely on did not work, and we would not have known that until we needed it.
Customers were not shown an error, and 1,400 of them waited between 9 and 40 seconds for a page that normally takes under a second. 380 gave up and left before it loaded. Nobody was prevented from transacting with us, but a number of people will have concluded that our site is slow, and they are right.
For 38 minutes, 214 customers who signed in were shown another customer's account page, carrying that person's name, postal address and balance. Nobody was able to act on the account they were shown. Four customers contacted us while it was happening, and the first is the reason we knew at all. We wrote to all 255 customers involved on 5 March.
The first two are the ones usually left out. An outage nobody saw still tells us something, and a failure that made customers wait is a customer impact even though nothing broke.
The numbers in this section are the ones a reader will quote back to us, and they are usually the last thing we are certain of. Care should be taken to write only what we can evidence at the time of issue — some customers is honest during the incident, and 214 customers is honest the following morning, once the logs have been read. A number published early and corrected later does more damage to confidence in the process than a Summary that admits we are still counting.
It is sometimes the case that failures or particularly, the next actions required, are not priortised or paid attention to. By ensuring we capture the exact customer experience during the failure, we can ensure that their needs and views are taken account of, particularly when it comes to Lessons Learned and Next Actions.
There are many classes of incident that ONLY customers see - in those cases where internal observations or monitoring do not contain details of the failure, the customer impact, the reports they made to us become our primary observations.
CoEs have life and are living documents, issued and revised over time. The full customer impact may not initally be understood.