Desktops, metadata and filing
In 1973, Xerox’s Palo Alto Research Center released the “Alto” personal computer. This was the first machine equipped with a graphical user interface (GUI) instead of the traditional character user interface.[1]
|
JC pontificates about technology
An occasional series.
|
A bad metaphor: the desktop
To lessen the cognitive burden on users — at the time, bowler-hatted bureaucrats, sleeve-gartered clerks and others whose mental framework comprised a typing pool and boys running memoranda between office in-trays in reusable manila envelopes, and whose idea of “information technology” was a pneumatic tube system that launched invoices around the clanking pipes of the organisation like mortar bombs — Xerox came up with a visual metaphor.
If users were going to be asked to give up their card catalogue system and stare at a computer screen all day then best make it as familiar as possible. Thus, the Alto interface was modelled on a “desktop”: not an impenetrable wall of green code and a flashing cursor, but a cartoonish depiction of a literal desktop with its familiar iconography: folders, a blotter, filing cabinets, in-trays, out-trays and even a dinky little waste-paper basket.
All designed to reassure the meatware —perennially fearful of incipient obsolescence as it was, and still is — that the change journey from the analogue to the atomic age would not be so bad after all.
A better metaphor: the spreadsheet
In 1979, Dan Bricklin and Bob Frankston created a new application for the Apple II computer. They called it “VisiCalc”. It was a grid of cells that you could input numbers and text into and then run calculations on by reference to cell coordinates. VisiCalc was the first spreadsheet program: a primitive ancestor to that beast we all now know and love as Microsoft Excel.
VisiCalc’s brilliant innovation was to separate the data you wanted to manipulate — the numbers and text in the cells — from the logical operations you wanted to manipulate them with — quasi-mathematical formulae — which referenced just the coordinates of the cells holding the data, not the data itself. You could therefore change the data without upsetting the calculation parameters. VisiCalc established a rudimentary form of programming language. A spreadsheet is a sort of programme. This may seem redolent of a smart contract, by the way. That is because it is. But let us not be distracted.
It might not have seemed much in 1979, but VisiCalc and its heirs would revolutionise business computing. While not nearly as intuitive as the Alto’s “desktop” — there was no graphic user interface or anything like that — VisiCalc was a much purer expression of what a personal computer could do. It promised even modest undertakings a powerful means of storing, augmenting, filtering, analysing and manipulating unprecedented amounts of information as structured data.
A spreadsheet is a much better way of thinking about how to organise digital information than a desktop because it is apparently infinite: digital information has no physical dimension. It is not constrained by a physical “substrate” — usually paper — that analogue information is embedded in. An empty spreadsheet stretches endlessly away in two directions.
The desktop was designed around that physical problem: how to manage bits of paper. In printed information, paper is “form”, text is “substance”. A desktop is obliged to prioritise form over substance because the substance does not exist independently of the form.
Paper must be put somewhere. Unless you physically copy it, it can only be in one place.
Copying and transporting paper is expensive, slow and “lossy”. Each copy loses fidelity and increases storage costs.[2]
Digital information has (almost) no form.[3] It does not occupy physical space. It costs nothing to store. We can copy and move it costlessly, instantly, and with no loss of fidelity. At least when compared with physical information, digital information can be everywhere at once. We are not constrained by space or time when we store or move digital information. Yet to file it, we use a metaphor that assumes we are.
Not being constrained by the physical limitations of a desktop a spreadsheet extends:
- Downwards: You can add items to your filing system without limit, unconstrained by the area of your desk or the volume of your filing cabinet, where each item occupies one of an infinite number of rows.
- Across: You can categorise each item however you like by creating new columns. There is no limit to columns, and no set hierarchy between columns. They need not even bear any relation to each other as long as they relate to the original item. Whereas a subfolder is necessarily a sub-division of the folder it sits in, this is not true of a new column.
Division versus multiplication
In a “desktop” structure, subfolders are sub-divisions, each further level down more fine-grained and subordinate than the last, and less important relative to the formal hierarchy. We prioritise the hierarchy over the item. The hierarchy explains and contextualises everything.
All columns in a spreadsheet have equal standing — they are, well, pari passu — and their combination has a multiplicative effect: if an existing column, or an artful combination of columns, doesn’t yield the information you need, you can always add more columns. In a spreadsheet, we prioritise the item over the hierarchy. The hierarchy is incidental.
A front in the battle between substance and form
The desktop prioritises form.
A spreadsheet prioritises substance.
The last thing to notice is our old friend the struggle between form and substance: if we take it that, whatever your metaphor of choice, the “item” — the thing being filed — is the substance and the organising system it goes into is the form, we can see that the desktop and the spreadsheet have fundamentally opposed philosophies.
What should matter to your organisation more? Substance or form?
The desktop priorities form — the “item” is buried at the bottom of a rigid formal structure of folders and subfolders which, once created, cannot easily be altered. This is why it is so hard to find things you have misfiled. You cannot put anything into the database until you have fully specified its folder path.
By contrast, a spreadsheet prioritises substance. The “item” is the first thing to go in the database, without any formal structure. It sits at the top of the structure. Only once it is in situ can you assign it any formal properties. The item therefore wears the properties we assign it lightly. Its position and identity does not change if we later alter, remove or augment the values we assign to it.
Metadata
Each desktop folder spreadsheet column represents metadata — literally, “information about information” — about the item being filed.[4]
A folder structure generates a very narrow, wan sort of metadata in the form of folder names: they are limited to a finite number of text characters. It is so limited that it is hardly worth thinking of it as metadata at all. Indeed, the Windows operating system could, but doesn’t, treat folder names as metadata, which is mad.
A spreadsheet, by contrast, uses metadata powerfully, putting few limits on what form it can take: text, calculable numbers and dates, checkboxes, people,[5] colours, flags, choices, lookups, comments, or calculations. It can be validated, managed, controlled, compulsory, optional, pre-populated or free-form. You can then filter, group, sort, chart, pivot and triangulate. Your imagination is the limit.
The more metadata you have, the more ways you can look at the data. Each separate value represents a new and distinct way of organising information. Even if the metadata is wrong, the inconsistencies between it and other correct fields on the same item allow us to triangulate and identify problematic data. We can, in this way, generate metadata about metadata. This is meta-metadata.
The desktop clings on
Yet even in our modern, hyper-networked, cloud-based work environment — even though we have had Microsoft Excel for nearly 40 years, the desktop metaphor hangs on.
We still call them “desktops”, though now for the prosaic reason that they generally are the only thing that sits on top of our desk. Look: the desktop was a nice, quaint idea. It got old geezers in green visors to sit down at keyboards — for that, the change managers of the world can be truly grateful — but the metaphor has well-outlived its purpose now.
It assumes each item to be filed has a unique physical location as if it were shelved in a physical library. Older readers may remember the Dewey decimal system, which numbered the entire corpus of non-fiction wisdom from zero to 1,000.[6]
When information is digital and has no physical dimension this is an unnecessary constraint. Duplicating items to suit multiple hierarchies creates basis risk. Which was the canonical version of the document? How can we be sure they are the same? What happens if one, but not the other, gets updated?
Where the document is a “living thing,” plotting its own miserable trajectory through the cosmos — say, a contract under negotiation, or a maintained legal template — then running multiple copies multiplies the job of maintaining all copies as the document changes, and that introduces the risk of human error. There may be miskeys. A document may be forgotten. Version control is a pain.
Also, a preferred hierarchy can change. Personnel, managers, business priorities, and circumstances change. They change the priorities of formal organisation. Changing your preferred hierarchy means completely re-engineering your folder structure.
Substrate neutrality
These are all problems of the physical realm; the spreadsheet metaphor shows us we need not be troubled by them in the digital realm. Here, the physical “substrate” — the hard copy — is irrelevant. What matters is the ASCII code embedded in it. In the digital realm, it has been abstracted and floats free of the substrate.
Across a diverse network of collaborators, the freedom to create multiple organising hierarchies on the fly, without upsetting other users, is immensely empowering.
Our reverence for the sacred substrate fell away, but not entirely. We still revere wet ink, for some reason counterparts clauses, and the dear old desktop.
For still, as we file, we cannot resist the siren call of folders. Folders in folders in folders in folders in folders.
Why do we persist with folders?
More than twenty years ago Tom Zingale taught a young JC a valuable lesson. Battling with some byzantine folder structure, and losing, JC cried out in anguish, and Tom said this:
JC: How on earth am I meant to organise all this?
Tom: With metadata.
JC: Er, with what?
Tom: Metadata. The answer to your question is metadata. Metadata, metadata, metadata. Whatever your question is, the answer is metadata.
JC: Well, my question is, “How do I use metadata to fix this filing problem?
Tom: Oh, right. Simple: SharePoint.
Wait: SharePoint?
Now a lot of good people viscerally hate SharePoint. To be sure, Microsoft seems to have gone out of its way to foment this hatred. It seems to have conducted a programme, over 20 years, to make SharePoint as hard to love as it can.
But at the same time, Microsoft has rebuilt its entire Office 35 Suite around the SharePoint platform. It is, to be sure monumentally confusing, the Teams integration is baffling. The utterly dismal online versions of its Office suite drive people righteously up the wall.
But, still, a good part of the enmity for SharePoint arises from the users’ basic misunderstanding of SharePoint’s fundamental architecture.
SharePoint is the first, philosophically, digitally native operating system. It has abandoned the desktop metaphor. SharePoint uses the spreadsheet metaphor.
In SharePoint you organise by metadata, not by folders.
DO NOT USE FOLDERS IN SHAREPOINT. DO NOT COMPLAIN THAT SHAREPOINT SUCKS IF YOU USE FOLDERS. IF YOU USE FOLDERS IN SHAREPOINT, THAT MEANS YOU SUCK, NOT SHAREPOINT.
Folders are top-down. Metadata is bottom-up. Folders prefer form over substance. Metadata prefers substance over form.
SharePoint allows you to do exactly the same thing with a document library as Excel allows you to do with a spreadsheet.
So it is odd — isn’t it? We intuitively understand the power of metadata when we are presented with a spreadsheet. But the same power does not occur to us when we are presented with a file management system. The desktop metaphor is burned on our retina.
Even though it is, in essence, a supercharged online spreadsheet, SharePoint continues to be resented by almost everyone.
See also
References
- ↑ It was well ahead of its time: the GUI would not become mainstream until Apple released its Macintosh a decade later, in 1984.
- ↑ All physical information is eventually destined for the Iron Mountain.
- ↑ Okay: almost no form. Compared with physical information. In this section take the word “almost” as read.
- ↑ Grammar pedants’ corner: Even though “data” is plural, “metadata” is generally treated as a singular mass noun. Please direct your letters to the Royal Statistical Society — not because it is their fault: rather, they might keep metadata about this sort of thing.
- ↑ As in, a lookup to an object in a people directory, and not just a text name.
- ↑ My favourite was 001.9.