Data retention

JC pontificates about technology An occasional series. →Go uptown Technology Tell me more Sign up for our newsletter — or just get in touch: for ½ a weekly 🍺 you get to consult JC. Ask about it here.

Scarcely a day goes by where a well-meaning information technology professional inveigles to a Luddite lawyer on the subject of data retention, specifically its risks and costs. Their favourite routines are:

• Data retention is expensive.
• Data retention is risky.

IT folk spend less time waxing lyrical about the value of that email data. They overstate the risk and misunderstand the benefit.

“Data retention is expensive”

Well, everything comes at a cost. But given the ever decreasing cost of processing power and storage capacity, it is hard to believe the physical cost of storage of data is that expensive — I mean, how much data have you got? A terabyte is estimated to cost a couple of cents these days — much less that this cost is greater than the value of keeping that data.

That is to say, data retention is not a straight cost question but a trade-off: cost versus value. IT folk are good at articulating — and inflating — the cost of data storage. They are not so adept at understanding its value because IT people don’t understand what other office workers do. It people are fond of saying “email is just a messaging system. It is not a file management system.”

Yeah, but it is a file management system, and quite a good one.

The benefit of keeping access to email data is not often articulated, but it should be obvious - in the real world every business is desperate to get its hands on this kind of bog data to mine it for commercial advantage. Maybe we should make an effort to quantify it. In any case, it increases the more information we have. IT infrastructure controllers are not the only interested parties here.

“Data retention is risky”

The “liability created by over-retaining email” is certainly something that lawyers are qualified to talk about, not that they’re ever asked. This, too, is not a straight decision but a trade-off:

• Wwhat is the liability of retaining email (given that we are legally obliged to retain it for seven years anyway), against
• What is the benefit from retaining email.

Liability

I am not sure what the risk of keeping business records is, except where it reveals employee misbehaviour or (more likely) incompetence, which leads to contractual, reputational or regulatory loss. But even then, what is the sense in a “hear no evil, see no evil speak no evil” approach? Shouldn’t we, as risk prudent risk controllers, want to keep evidence of misbehaviour?

In any case, this is undoubtedly a tail risk: if the risk doesn't materialize in seven years, what are the odds it will come back to bite later on. They’re there, for sure - but low.

And let’s be clear here: a pristine email record is not a risk in itself, but just evidence of it. The actual risk is of employee misconduct in writing the email. If (as seems to be the case) we assume that employees will misbehave out of all proportion with the competent discharge of their duties, won’t systematic destruction of the internal record of their misbehaviour at the first opportunity only encourage their misbehaviour? How will it help root this behaviour out?

On the other hand, the benefit of being able to mine “good” information — and for that matter find and act to mitigate bad information — is immediate, direct and obvious, and accrues as soon as we put the ability to search the database in employee’s hands. For every one incriminating email, there are hundreds of thousands containing valuable information about how this firm has dealt with novel situations in the past. Who wouldn't want access to that?