Data retention

From The Jolly Contrarian
Revision as of 16:05, 28 December 2018 by Amwelladmin (talk | contribs)
Jump to navigation Jump to search
JC pontificates about technology
An occasional series.
Index: Click to expand:
Tell me more
Sign up for our newsletter — or just get in touch: for ½ a weekly 🍺 you get to consult JC. Ask about it here.

Scarcely a day goes by where a well-meaning information technology professional inveigles to a Luddite lawyer on the subject of data retention, specifically its risks and costs. Their favourite routines are:

  • Data retention is expensive.
  • Data retention is risky.

IT folk spend less time waxing lyrical about the value of that email data. They overstate the risk and misunderstand the benefit.

“Data retention is expensive”

Well, everything comes at a cost. But given the ever decreasing cost of processing power and storage capacity, it is hard to believe the physical cost of storage of data is that expensive — I mean, how much data have you got? A terabyte is estimated to cost a couple of cents these days — much less that this cost is greater than the value of keeping that data.

That is to say, data retention is not a straight cost question but a trade-off: cost versus value. IT folk are good at articulating — and inflating — the cost of data storage. They are not so adept at understanding its value because IT people don’t understand what other office workers do. It people are fond of saying “email is just a messaging system. It is not a file management system.”

Yeah, but it is a file management system, and quite a good one.

The benefit of keeping access to email data is not often articulated, but it should be obvious - in the real world every business is desperate to get its hands on this kind of bog data to mine it for commercial advantage. Maybe we should make an effort to quantify it. In any case, it increases the more information we have. IT infrastructure controllers are not the only interested parties here.

“Data retention is risky”

The “liability created by over-retaining email” is certainly something that lawyers are qualified to talk about, not that they’re ever asked. This, too, is not a straight decision but a trade-off:

  • Wwhat is the liability of retaining email (given that we are legally obliged to retain it for seven years anyway), against
  • What is the benefit from retaining email.

Liability

I am not sure what the risk of keeping business records is (except to the extent it reveals employee misbehaviour which leads to contractual, reputational or regulatory loss – but even in that case what is the sense in adopting a “hear no evil, see no evil speak no evil” approach? Shouldn’t we, as risk prudent risk controllers and compliance managers, want to keep evidence of misbehaviour?).

In any case, this is undoubtedly a tail risk: if the risk doesn't materialize in seven years, what are the odds it will come back to bite later on. They're there, for sure - but low.

And let’s be clear here: a pristine email record is not a risk in itself, but just evidence of it. The actual risk is of employee misconduct. If (as seems to be the case) we assume that employees will misbehave, won’t systematic destruction of our internal record of their misbehaviour only encourage misbehaviour? How will it help root this behaviour out?

On the other hand, the benefit of being able to mine “good” information is immediate, direct and obvious, and accrues as soon as we put the ability to search the database in employee’s hands. For every one incriminating email, there are hundreds of thousands containing valuable information about how this firm has dealt with novel situations in the past. Who wouldn't want access to that?