Template:M intro technology metadata: Difference between revisions

From The Jolly Contrarian
Jump to navigation Jump to search
← Older edit
VisualWikitext
No edit summary
No edit summary
 
(15 intermediate revisions by the same user not shown)
Line 5: Line 5:
If potential users — bowler-hatted bureaucrats who didn’t use computers ''at all'' — were to be persuaded to give up their card catalogue systems, typing pools and reusable manila envelopes and instead stare at a screen all day, the system would need to look as familiar as possible.  
If potential users — bowler-hatted bureaucrats who didn’t use computers ''at all'' — were to be persuaded to give up their card catalogue systems, typing pools and reusable manila envelopes and instead stare at a screen all day, the system would need to look as familiar as possible.  


To lessen the cognitive burden, Xerox came up with a visual metaphor. The Alto’s interface was modelled on a “[[desktop]]”. Instead of an impenetrable wall of green code and a flashing cursor, users were presented with a cartoonish depiction of a ''literal'' desktop and all its familiar iconography: documents, folders, a blotter, filing cabinets, in-trays, out-trays and even a dinky little waste-paper basket. <ref>The metaphor even extended to categories of data that did not exist in an analogue office: Emails were depicted as little envelopes with a stamp and a wax seal.</ref>
And so, to lessen the cognitive burden, Xerox came up with a visual metaphor. The Alto’s graphic user interface was modelled on a “[[desktop]]”. Instead of an impenetrable wall of green code and a flashing cursor, users were presented with a cartoonish depiction of a ''literal'' desktop and all its familiar iconography: documents, folders, a blotter, filing cabinets, in-trays, out-trays and even a dinky little waste-paper basket. This even extended to types of document that did not exist in an analogue office: Emails were depicted as little envelopes with a stamp and a wax seal.


All designed to reassure the [[meatware]] — as fearful of incipient obsolescence then as it is now — that the [[change journey]] from the comfy old analogue world to the coming atomic age would not be so bad after all.
All was designed to reassure the [[meatware]] — as fearful of incipient obsolescence then as it is now — that the [[change journey]] from the comfy old analogue world to the coming atomic age would not be so bad after all.


====The spreadsheet====
====The spreadsheet====
Line 18: Line 18:
'''Across''': You can ''categorise'' each artefact however you like by creating new ''columns''. There is no limit to the number of columns and no necessary hierarchy between them. }}
'''Across''': You can ''categorise'' each artefact however you like by creating new ''columns''. There is no limit to the number of columns and no necessary hierarchy between them. }}


The desktop is designed to manage physical properties that digital information does not have. In printed information, paper is “form”, text is “substance”. A desktop must prioritise form because in a physical system ''substance cannot exist independently of form''. Data must be printed on paper. Paper must be ''put'' somewhere. Unless you physically copy it, you can only put a piece of paper in one place at a time. Older readers may remember the [[Dewey decimal system]], which numbered the entire corpus of non-fiction wisdom from zero to 1,000.<ref>My favourite was [[001.9]]: “mysteries and the unexplained”.</ref> This addressed an exclusively physical problem. We don’t need the Dewey system any more.<ref>Entertainingly, courtesy of some well-meaning rube, the [[Dewey decimal system]] limped on in the information age, in the shape of [https://www.oclc.org/en/dewey/webdewey.html webdewey]. This may be the librarian’s equivalent of the [[Lehmans]] online Amish supplies store<ref>''It’s [[Lehman]], not [[Lehmans]]''. Well, it ''was'', any rate. Who knew that an online store for people living off the grid would outlive a Wall Street behemoth?</ref></ref> We don’t need the desktop, either.
The desktop is designed to manage physical properties that digital information does not have. In printed information, paper is “form”, text is “substance”. A desktop must prioritise form because in a physical system ''substance cannot exist independently of form''. Data must be printed on paper. Paper must be ''put'' somewhere. Unless you physically copy it, you can only put a piece of paper in one place at a time.  


Furthermore, storing paper is expensive.<ref>All physical information is eventually destined for the [[Iron Mountain]].</ref> Copying and transporting paper is expensive, slow and “''lossy''”. The embedded data degrades. Each copy increases storage costs at the [[Iron Mountain]].
Older readers may remember the [[Dewey decimal system]], by which libraries numbered the entire corpus of non-fiction wisdom from zero to 1,000. My favourite was [[001.9]]: “mysteries and the unexplained”. But the Dewey system addressed an exclusively physical problem: where to put things that could only be in one place at a time. We don’t need it online.<ref>Entertainingly, courtesy of some well-meaning rube, the [[Dewey decimal system]] limped into the information age, in the shape of [https://www.oclc.org/en/dewey/webdewey.html webdewey]. This may be the librarian’s equivalent of an online Amish supplies store.</ref>


Whereas a desktop subfolder is necessarily a sub-division of the folder it sits in, spreadsheet columns have no hierarchy and need not bear any relation to each other, as long as they all relate to the original artefact.
For the same reason, we don’t need the desktop. It was designed around other purely physical constraints, too. Storing physical information, on paper, is expensive.<ref>All physical information is eventually destined for the [[Iron Mountain]].</ref> Copying and transporting paper is expensive, slow and “''lossy''”. With each copy we make we increase our storage costs and lose some fidelity.


But ''digital'' information has (almost) no form.<ref>Okay: ''almost'' no form. Compared with physical information. In this section take the word “almost” as read. </ref> It does not occupy physical space. It costs nothing to store. We can copy and move it costlessly, instantly, and with no loss of fidelity. At least when compared with physical information, digital information can be everywhere at once. We are not constrained by space or time when we store or move digital information. Yet to file it, we use a metaphor that assumes we are. 
But ''digital'' information has — ''almost''<ref>''Compared with physical information''. Yes, it does take up some space and as we get carried away into our brute force computing AI swoon the energy cost of data is no-longer negligible, but that is a peripheral issue in “normal” computing. Nevertheless, in this section take the word “almost” as read. </ref> — no “form” at all. It does not occupy physical space. It costs nothing to store. We can copy and move it costlessly, instantly, and with no loss of fidelity. At least when compared with physical information, digital information can be everywhere, and nowhere, at once. We are not constrained by space or time when we store or move digital information.  
====Division versus multiplication====
{{drop|I|n a “desktop”}} structure, subfolders are sub-''divisions'', each further level down more fine-grained and ''subordinate'' than the last, and less important relative to the formal hierarchy. A folder
path is a rabbit hole.  ''Hierarchy'' takes priority over the ''artefact''. The hierarchy explains and contextualises everything. The more extensive the hierarchy, the less significant an individual artefact.  


Whereas folders are ''divisive'' in nature, spreadsheet columns are ''multiplicative''. All columns in a spreadsheet have equal standing — they are, well, ''[[pari passu]]'' — so they can be multiplied without limit: if an existing column, or an artful combination of them, doesn’t yield the information you need, you can always add ''more'' columns. In a spreadsheet, ''the artefact takes priority over the hierarchy''. The hierarchy is incidental. ''Formal''.
Yet to file it, we insist upon using a metaphor that assumes we are.
 
But naturally we ''like'' hierarchy. Hierarchies place objects in a permanent relationship to each other. This is comforting in a changing world. It feels ''tangible'' in a virtual realm that natively is ''not'' tangible. We naturally organise ourselves into hierarchies — for better or worse — so there is no surprise that we like our information served up with similar structure.
 
Flexibility is good for experts, virtuosi, and improvisers: it intimidates those who are not. Structure works better where we want dependability and reliability.


====A front in the battle between substance and form====
====A front in the battle between substance and form====
{{quote|
{{quote|
The desktop prioritises ''form''. <br>A spreadsheet prioritises ''substance''.}}
The desktop prioritises ''form''. <br>A spreadsheet prioritises ''substance''.}}
{{Drop|H|ere we find}} our old friend the [[Der Sieg der Form über Substanz|struggle between form and substance]]: if we take it that, whatever your metaphor of choice, the “artefact” — the thing being filed — is the ''substance'' and the organising system it goes into is the ''form'', we can see that the desktop and the spreadsheet have fundamentally opposed philosophies.  
{{drop|I|n a “desktop”}} structure, subfolders are sub-''divisions'', each further level down more fine-grained and ''subordinate'' than the last. The deeper the folder structure, the less significant the artefact relative to the formal hierarchy. The folder structure is logically prior to the artefact because you cannot put anything into the system until you have fully specified its folder path. The end-folder has to be there to put the document into it.
 
The desktop therefore ''buries'' documents at the ''bottom'' of the filing structure. If you want to retrieve one, down the folder-path rabbit hole you must go. Heaven help you if your document has been misfiled.
 
In a spreadsheet, the substance takes priority. The artefact is the ''first'' thing to go in the database, and sits at the top. Only then do we apply formal properties to our artefact. These formal properties are thus incidental.
 
And where folders are ''divisive'' in nature — a subfolder is necessarily a sub-division of the folder it sits in — spreadsheet columns have no hierarchy, need not bear any relation to each other, and are therefore ''multiplicative'': they can be multiplied without limit: if an existing column, or an artful combination of them, doesn’t yield the exact information you need, you can always add ''more'' columns.  


The desktop priorities ''form'' — the artefact is buried at the ''bottom'' of a rigid formal structure of folders and subfolders which, once created, cannot easily be altered. This is why it is so hard to find things you have misfiled. You cannot put anything into the database until you have fully specified its folder path.  
Now, naturally we ''like'' hierarchies. There is not a social structure on the face of the earth that doesn’t have one. Hierarchies place things in a permanent, graspable relation to each other. This is comforting. It feels ''tangible'' in a virtual realm that natively is ''not'' tangible. Flexibility is good for experts, improvisers and virtuosi, but it intimidates everyone else. Hierarchy is important when we want dependability and reliability.


By contrast, a spreadsheet prioritises ''substance''. The artefact is the first thing to go in the database, before any formal structure is applied to it.<ref>Other than mandatory metadata ''about'' the act of filing: a date stamp, a file name, the person who filed it, etc.</ref> It sits at the ''top'' of the structure. Only once it is ''in situ'' can you assign it any formal properties. The artefact therefore wears the properties we assign it ''lightly''. Its position and identity does not change if we later alter, remove or augment the values we assign to it.
And here we find our old friend the [[Der Sieg der Form über Substanz|struggle between form and substance]]: if we take it that, whatever your metaphor of choice, the “artefact” — the thing being filed — is the ''substance'' and the organising system it goes into is the ''form'', we can see that the desktop and the spreadsheet are fundamentally opposed philosophies.  


==== Metadata ====
==== Metadata ====
{{drop|E|ach desktop folder}} or spreadsheet column is ''[[metadata]]'' about its artefact — literally, “information ''about'' information”.<ref>Grammar pedants’ corner: Even though “data” is plural, “metadata” is generally treated as a singular mass noun. Please direct your letters to the Royal Statistical Society — not because it is their fault: rather, they might keep metadata about this sort of thing.</ref> A folder structure generates a limited, anaemic sort of metadata in the shape of folder 260-character labels: so limited that the Windows operating system does not treat folder names as metadata at all.<ref>This, by the way, is mad.</ref> A spreadsheet, by contrast, imposes few limits on what form metadata can take: text, calculable numbers and dates, checkboxes, people,<ref>As in, a lookup to an object in a people directory, and not just a text name.</ref> colours, flags, choices, lookups, comments, or calculations. It can be validated, managed, controlled, compulsory, optional, pre-populated or free-form. You can filter, group, sort, chart, or pivot it.  
{{drop|E|ach desktop folder}} or spreadsheet column is ''[[metadata]]'' about its artefact — literally, “information ''about'' information”.<ref>Grammar pedants’ corner: Even though “data” is plural, “metadata” is generally treated as a singular mass noun. Please direct your letters to the Royal Statistical Society — not because it is their fault: rather, they might keep metadata about this sort of thing.</ref> A folder structure generates a limited, anaemic sort of metadata in the shape of the folder name: an alphanumeric label of up to 260 characters that is so limited in what it can be used for that the Windows operating system does not treat it as metadata at all.<ref>This, by the way, is mad.</ref> A spreadsheet, by contrast, imposes few limits on what form metadata can take: text, calculable numbers and dates, checkboxes, people,<ref>As in, a lookup to an object in a people directory, and not just a text name.</ref> colours, flags, choices, lookups, comments, concatenations or calculations. Spreadsheet metadata can be compulsory, optional, pre-populated or free-form. You can validate, manage, control, filter, group, sort, chart, or pivot it.
 
The more metadata you have, the more ways you can play with it. Each separate value represents a new and distinct way of organising your information. Even if your metadata is ''wrong'', you can triangulate the inconsistencies to identify problematic data. Whereas a misfiled folder is lost forever, a mis-tagged cell reveals itself for self-cleansing.  


The more metadata you have, the more ways you can look at your data. The more ways you can ''access'' your data. Each separate value represents a new and distinct way of organising information. Even if your metadata is ''wrong'', you can triangulate the inconsistencies to identify problematic data. You can, in this way, generate metadata ''about'' metadata. This is ''meta''-[[metadata]].
You can, in this way, generate metadata ''about'' metadata. This is ''meta''-[[metadata]].


==== The desktop clings on ====
==== The desktop clings on ====
{{drop|Y|et even in}} our modern, hyper-networked, cloud-based work environment — even though we have had Microsoft Excel for nearly 40 years, the desktop [[metaphor]] hangs on. We still call them “desktops”, for the prosaic reason that they are the only thing still allowed on the desktop in our clear-desk, humans-as-fungible-cogs-in-the-machine modern office environment. (Is it any wonder firms are struggling to get staff back to the office, by the way?)
{{drop|Y|et even in}} our modern, hyper-networked, cloud-based work environment; even though we have had Microsoft Excel for nearly 40 years and we’re quite good at it now, the desktop [[metaphor]] hangs on. We still call them “desktops”, for the prosaic reason that they are the only thing still allowed on the desktop in our clear-desk, humans-as-fungible-cogs-in-the-machine modern office environment. (Is it any wonder firms are struggling to get staff back to the office, by the way?)


The desktop was a nice, quaint idea. It got old geezers in green visors to sit down at keyboards. For that, the [[change manager]]s of the world can be grateful, but the metaphor has long since outstayed its welcome now. Enough already of the dinky desktop.
The desktop was a nice, quaint idea. It got old geezers in green visors to sit down at keyboards. For that, the [[change manager]]s of the world can be grateful. But the metaphor has long since outstayed its welcome. Enough already of the dinky desktop.


When information is digital and has no physical dimension it is an unnecessary constraint. Duplicating artefacts to suit multiple hierarchies creates ''[[basis]] risk''. Which was the canonical version of the document? How can we be sure they are the same? What happens if one, but not the other, gets updated?  
When information is digital and has no physical dimension it is an unnecessary constraint. Duplicating artefacts to suit multiple hierarchies creates ''[[basis]] risk''. Which was the canonical version of the document? How can we be sure they are the same? What happens if one, but not the other, gets updated?  
Line 61: Line 62:


==== Substrate neutrality ====
==== Substrate neutrality ====
{{drop|T|hese are all}} problems of the ''physical'' realm; the spreadsheet metaphor shows us we need not be troubled by them in the digital realm. Here, the ''physical'' “[[substrate]]” — the hard copy — is ''irrelevant''. What matters is the ASCII code embedded in it. In the digital realm, it has been abstracted and floats free of the substrate.  
{{drop|T|hese are all}} problems of the ''physical'' realm; the spreadsheet metaphor shows us we need not be troubled by them in the digital realm. Here, the ''physical'' “[[substrate]]” — the hard copy — is ''irrelevant''. What matters is the ASCII code embedded in it. In the digital realm, it has been abstracted and floats free of the papery [[substrate]].  


Across a [[diverse]] network of collaborators, the freedom to create multiple organising hierarchies on the fly, without upsetting other users, is immensely empowering.
Across a [[diverse]] network of collaborators, the freedom to create multiple organising hierarchies on the fly, without upsetting other users and without needlessly duplicating documents, is immensely empowering.


Our reverence for the sacred substrate fell away, but not ''entirely''. We still revere [[Wet signature|wet ink]], for some reason [[counterparts]] clauses, and the dear old desktop.  
Our reverence for the sacred substrate has fallen away, but not ''entirely''. We still revere [[Wet signature|wet ink]], for some reason [[counterparts]] clauses, and the dear old desktop.  


For still, as we file, ''we cannot resist the siren call of folders''. Folders in folders in folders in folders in folders.  
For still, as we file, ''we cannot resist the siren call of folders''. Folders in folders in folders in folders in folders.  
Line 71: Line 72:
Why do we persist with folders?
Why do we persist with folders?


More than twenty years ago Tom Zingale taught a young JC a valuable lesson. Battling with some byzantine folder structure, and losing, JC cried out in anguish, and Tom said this:{{dialogue|
More than twenty years ago Mr T. Zingale taught the young JC a valuable lesson. Battling with some byzantine folder structure, and losing, JC cried out in anguish to his technologist friend:{{dialogue|
{{dia|JC|How on earth am I meant to organise all this?}}
{{dia|JC|How on earth am I meant to organise all this?}}
{{dia|Tom|With [[metadata]].}}
{{dia|Mr. T|With [[metadata]].}}
{{dia|JC|Er, with ''what''?}}
{{dia|JC|Er, with ''what''?}}
{{dia|Tom|[[Metadata]]. The answer to your question is [[metadata]]. Metadata, [[metadata]], [[metadata]]. Whatever your question is, the answer is [[metadata]].}}
{{dia|Mr. T|[[Metadata]]. The answer to your question is [[metadata]]. Metadata, [[metadata]], [[metadata]]. Whatever your question is, the answer is [[metadata]].}}
{{dia|JC|Well, my question is, “How do I use [[metadata]] to fix this filing problem?}}
{{dia|JC|Well, my question is, “How do I use [[metadata]] to fix this filing problem?}}
{{dia|Tom|Oh, right. Simple: [[SharePoint]].}}
{{dia|Mr. T|Oh, right. Simple: [[SharePoint]].}}
}}
}}


Wait: ''SharePoint''?
Wait: did somebody say, ''SharePoint''?


==== About SharePoint ====
==== About SharePoint ====
{{Drop|N|ow a lot}} of good people viscerally ''hate'' [[SharePoint]]. To be sure, Microsoft seems to have gone out of its way to foment this hatred. It seems to have conducted a programme, over 20 years, to make [[Desktops, metadata and filing|SharePoint]] as hard to love as it can.  
{{Quote|
I know I’m unlovable<br>
You don’t have to tell me.<br>
Message received loud and clear <br>
[...]<br>
Loud and clear <br>
If I seem a little strange well <br>
That’s because I am. <br>
:—The Smiths, ''Unlovable''}}
 
{{Drop|N|ow a lot}} of good people viscerally ''hate'' [[SharePoint]]. To be sure, Microsoft has apparently gone out of its way to foment this emotion, waging a sustained campaign, over 20 years, to render it as unlovable as can be.
 
But still, SharePoint was the first — and remains the only — ''philosophically'' digital operating system. It abandoned the dinky [[desktop]] [[metaphor]] and embraced the ''[[spreadsheet]]'' approach. Think of it as an online version of [[Excel]] that can also hold documents in a spreadsheet format. You can create as many new metadata columns about each document, just like you can in Excel.
 
And Microsoft must realise this: it has rebuilt its entire productivity suite around SharePoint. To be sure, the Office 365 platform — or whatever they call it now — is ''monumentally'' confusing: the Teams integration is baffling. The interaction between OneDrive, Teams, File Explorer, SharePoint and Outlook seems wilfully designed to create as much confusion, duplication and data loss as possible. Where once there was a single place for documents, communications and schedules, now there are three or four of each. Did you get that file I sent? I chatted it to you in Teams. Or, hang on, did I email it? Or Share a link from OneDrive?


But at the same time, Microsoft has rebuilt its entire Office 35 Suite around the SharePoint platform. It is, to be sure ''monumentally confusing'', the Teams integration is baffling. The utterly dismal online versions of its Office suite drive people righteously up the wall.  
Disaster.
====The unbearable tolerance of spreadsheets====
{{Drop|A|nd this is}} the ''downside'' of the spreadsheet metaphor. A spreadsheet is a blank page: it leaves hierarchy and database design up to whoever wants to have a go at it. SharePoint leaves the discipline of getting the database design right, and then enforcing it, up to the user.  


But, still, a good part of the enmity for [[SharePoint]] arises from the users’ basic misunderstanding of SharePoint’s fundamental architecture.  
But this pluralism, this multiplicity, this total ambivalence to how users want to work, this tolerance of all the different possible ways users ''do'' work, however dopey, this total disarming ''agnosticism'', that will even let users embrace the desktop metaphor if that’s what they really want — is a recipe for organisational ''chaos''.  


SharePoint is the first, philosophically, digitally native operating system. It has abandoned the [[desktop]] metaphor. SharePoint uses the ''[[spreadsheet]]'' metaphor.
Because, as we should know by now, most users ''will'' embrace the desktop metaphor if you give them half a chance. Because they don’t know any better. Why ''would'' they? Database design optimisation is its own branch of computer engineering. You can even get a {{pl|https://www.gisci.org/Study-Guide/Database-Design-and-Management|certificate}} in it.


In SharePoint you organise by ''[[metadata]]'', not by ''folders''.
''Most users do not have such a certificate''. Nor do they ''want'' one. They have not thought much about database design optimisation. Nor do they ''want'' to.


{{quote|DO NOT USE FOLDERS IN SHAREPOINT. DO NOT COMPLAIN THAT SHAREPOINT SUCKS IF YOU USE FOLDERS. IF YOU USE FOLDERS IN SHAREPOINT, THAT MEANS ''YOU'' SUCK, NOT SHAREPOINT.}}
Rather, when they entered the workforce, or first clapped eyes on a computer, they saw a ''desktop'' and they ''liked'' it. They understood it and got very used to it.  


Folders are top-down. Metadata is bottom-up. Folders prefer form over substance. Metadata prefers substance over form.
So the [[desktop]] [[metaphor]] has proven resilient. It will remain hard to dislodge.  


''SharePoint allows you to do exactly the same thing with a document library as Excel allows you to do with a spreadsheet''.  
Yet it is odd — isn’t it? We intuitively understand the power of metadata when we are presented with a spreadsheet. But the same power does not occur to us when we are presented with a file management system. The desktop metaphor is burned on our retina.


So it is odd — isn’t it? We intuitively understand the power of metadata when we are presented with a spreadsheet. But the same power does not occur to us when we are presented with a file management system. The desktop metaphor is burned on our retina.
Even though it is, in essence, a supercharged online spreadsheet, SharePoint continues to be resented by almost everyone. This is no more than Microsoft deserves for its terrible implementation, but at least give it a try.  


Even though it is, in essence, a supercharged online spreadsheet, SharePoint continues to be resented by almost everyone.
Cross the Rubicon. Redpill yourself. Leave the desktop metaphor behind and finally go truly digital.

Latest revision as of 18:18, 30 September 2024

The desktop

In 1973, Xerox’s Palo Alto Research Center released the “Alto”. This was the first personal computer equipped with a “graphical user interface” (GUI) — computing with pictures — instead of the traditional “character user interface”.

If potential users — bowler-hatted bureaucrats who didn’t use computers at all — were to be persuaded to give up their card catalogue systems, typing pools and reusable manila envelopes and instead stare at a screen all day, the system would need to look as familiar as possible.

And so, to lessen the cognitive burden, Xerox came up with a visual metaphor. The Alto’s graphic user interface was modelled on a “desktop”. Instead of an impenetrable wall of green code and a flashing cursor, users were presented with a cartoonish depiction of a literal desktop and all its familiar iconography: documents, folders, a blotter, filing cabinets, in-trays, out-trays and even a dinky little waste-paper basket. This even extended to types of document that did not exist in an analogue office: Emails were depicted as little envelopes with a stamp and a wax seal.

All was designed to reassure the meatware — as fearful of incipient obsolescence then as it is now — that the change journey from the comfy old analogue world to the coming atomic age would not be so bad after all.

The spreadsheet

In 1979, Dan Bricklin and Bob Frankston created a new application for the Apple II computer. They called it “VisiCalc”. It was a grid of cells that you could input numbers and text into and then run calculations on by reference to cell coordinates. VisiCalc was the first spreadsheet program: a primitive ancestor to that beast we all now know and love as Microsoft Excel.

VisiCalc’s brilliant innovation was to separate the data you wanted to manipulate — the numbers and text in the cells — from the logical operations you wanted to manipulate them with — quasi-mathematical formulae — which referenced just the coordinates of the cells holding the data, not the data itself. You could therefore change the data without upsetting the calculation parameters. VisiCalc established a rudimentary form of programming language. A spreadsheet is a sort of programme. This may seem redolent of a smart contract, by the way. That is because it is. But let us not be distracted.

It might not have seemed much in 1979, but VisiCalc and its heirs would revolutionise business computing. While not nearly as intuitive as the Alto’s “desktop” — there was no graphic user interface or anything like that — VisiCalc was a much purer expression of what a personal computer could do. It promised even modest undertakings a powerful means of storing, augmenting, filtering, analysing and manipulating unprecedented amounts of information as structured data.

Good and bad metaphors

A spreadsheet is a much better way of thinking about how to organise digital information than a desktop because it is not constrained by physical space. Whereas the information on a traditional desktop is embedded in a physical “substrate” — usually paper — digital information has no such limitations. An empty spreadsheet stretches endlessly away in two directions:

Downwards: You can add items — “artefacts” — to your filing system without limit, unconstrained by the area of your desk or the volume of your filing cabinet. Each new artefact occupies a new row. There is an infinite number of rows.
Across: You can categorise each artefact however you like by creating new columns. There is no limit to the number of columns and no necessary hierarchy between them.

The desktop is designed to manage physical properties that digital information does not have. In printed information, paper is “form”, text is “substance”. A desktop must prioritise form because in a physical system substance cannot exist independently of form. Data must be printed on paper. Paper must be put somewhere. Unless you physically copy it, you can only put a piece of paper in one place at a time.

Older readers may remember the Dewey decimal system, by which libraries numbered the entire corpus of non-fiction wisdom from zero to 1,000. My favourite was 001.9: “mysteries and the unexplained”. But the Dewey system addressed an exclusively physical problem: where to put things that could only be in one place at a time. We don’t need it online.[1]

For the same reason, we don’t need the desktop. It was designed around other purely physical constraints, too. Storing physical information, on paper, is expensive.[2] Copying and transporting paper is expensive, slow and “lossy”. With each copy we make we increase our storage costs and lose some fidelity.

But digital information has — almost[3] — no “form” at all. It does not occupy physical space. It costs nothing to store. We can copy and move it costlessly, instantly, and with no loss of fidelity. At least when compared with physical information, digital information can be everywhere, and nowhere, at once. We are not constrained by space or time when we store or move digital information.

Yet to file it, we insist upon using a metaphor that assumes we are.

A front in the battle between substance and form

The desktop prioritises form.
A spreadsheet prioritises substance.

In a “desktop” structure, subfolders are sub-divisions, each further level down more fine-grained and subordinate than the last. The deeper the folder structure, the less significant the artefact relative to the formal hierarchy. The folder structure is logically prior to the artefact because you cannot put anything into the system until you have fully specified its folder path. The end-folder has to be there to put the document into it.

The desktop therefore buries documents at the bottom of the filing structure. If you want to retrieve one, down the folder-path rabbit hole you must go. Heaven help you if your document has been misfiled.

In a spreadsheet, the substance takes priority. The artefact is the first thing to go in the database, and sits at the top. Only then do we apply formal properties to our artefact. These formal properties are thus incidental.

And where folders are divisive in nature — a subfolder is necessarily a sub-division of the folder it sits in — spreadsheet columns have no hierarchy, need not bear any relation to each other, and are therefore multiplicative: they can be multiplied without limit: if an existing column, or an artful combination of them, doesn’t yield the exact information you need, you can always add more columns.

Now, naturally we like hierarchies. There is not a social structure on the face of the earth that doesn’t have one. Hierarchies place things in a permanent, graspable relation to each other. This is comforting. It feels tangible in a virtual realm that natively is not tangible. Flexibility is good for experts, improvisers and virtuosi, but it intimidates everyone else. Hierarchy is important when we want dependability and reliability.

And here we find our old friend the struggle between form and substance: if we take it that, whatever your metaphor of choice, the “artefact” — the thing being filed — is the substance and the organising system it goes into is the form, we can see that the desktop and the spreadsheet are fundamentally opposed philosophies.

Metadata

Each desktop folder or spreadsheet column is metadata about its artefact — literally, “information about information”.[4] A folder structure generates a limited, anaemic sort of metadata in the shape of the folder name: an alphanumeric label of up to 260 characters that is so limited in what it can be used for that the Windows operating system does not treat it as metadata at all.[5] A spreadsheet, by contrast, imposes few limits on what form metadata can take: text, calculable numbers and dates, checkboxes, people,[6] colours, flags, choices, lookups, comments, concatenations or calculations. Spreadsheet metadata can be compulsory, optional, pre-populated or free-form. You can validate, manage, control, filter, group, sort, chart, or pivot it.

The more metadata you have, the more ways you can play with it. Each separate value represents a new and distinct way of organising your information. Even if your metadata is wrong, you can triangulate the inconsistencies to identify problematic data. Whereas a misfiled folder is lost forever, a mis-tagged cell reveals itself for self-cleansing.

You can, in this way, generate metadata about metadata. This is meta-metadata.

The desktop clings on

Yet even in our modern, hyper-networked, cloud-based work environment; even though we have had Microsoft Excel for nearly 40 years and we’re quite good at it now, the desktop metaphor hangs on. We still call them “desktops”, for the prosaic reason that they are the only thing still allowed on the desktop in our clear-desk, humans-as-fungible-cogs-in-the-machine modern office environment. (Is it any wonder firms are struggling to get staff back to the office, by the way?)

The desktop was a nice, quaint idea. It got old geezers in green visors to sit down at keyboards. For that, the change managers of the world can be grateful. But the metaphor has long since outstayed its welcome. Enough already of the dinky desktop.

When information is digital and has no physical dimension it is an unnecessary constraint. Duplicating artefacts to suit multiple hierarchies creates basis risk. Which was the canonical version of the document? How can we be sure they are the same? What happens if one, but not the other, gets updated?

Where the document is a “living thing,” plotting its own miserable trajectory through the cosmos — say, a contract under negotiation, or a maintained legal template — then running multiple copies multiplies the job of maintaining all copies as the document changes, and that introduces the risk of human error. There may be miskeys. A document may be forgotten. Version control is a pain.

Also, a preferred hierarchy can change. Personnel, managers, business priorities, and circumstances change. They change the priorities of formal organisation. Changing your preferred hierarchy means completely re-engineering your folder structure.

Substrate neutrality

These are all problems of the physical realm; the spreadsheet metaphor shows us we need not be troubled by them in the digital realm. Here, the physical “substrate” — the hard copy — is irrelevant. What matters is the ASCII code embedded in it. In the digital realm, it has been abstracted and floats free of the papery substrate.

Across a diverse network of collaborators, the freedom to create multiple organising hierarchies on the fly, without upsetting other users and without needlessly duplicating documents, is immensely empowering.

Our reverence for the sacred substrate has fallen away, but not entirely. We still revere wet ink, for some reason counterparts clauses, and the dear old desktop.

For still, as we file, we cannot resist the siren call of folders. Folders in folders in folders in folders in folders.

Why do we persist with folders?

More than twenty years ago Mr T. Zingale taught the young JC a valuable lesson. Battling with some byzantine folder structure, and losing, JC cried out in anguish to his technologist friend:

JC: How on earth am I meant to organise all this?

Mr. T: With metadata.

JC: Er, with what?

Mr. T: Metadata. The answer to your question is metadata. Metadata, metadata, metadata. Whatever your question is, the answer is metadata.

JC: Well, my question is, “How do I use metadata to fix this filing problem?

Mr. T: Oh, right. Simple: SharePoint.

Wait: did somebody say, SharePoint?

About SharePoint

I know I’m unlovable
You don’t have to tell me.
Message received loud and clear
[...]
Loud and clear
If I seem a little strange well
That’s because I am.

—The Smiths, Unlovable

Now a lot of good people viscerally hate SharePoint. To be sure, Microsoft has apparently gone out of its way to foment this emotion, waging a sustained campaign, over 20 years, to render it as unlovable as can be.

But still, SharePoint was the first — and remains the only — philosophically digital operating system. It abandoned the dinky desktop metaphor and embraced the spreadsheet approach. Think of it as an online version of Excel that can also hold documents in a spreadsheet format. You can create as many new metadata columns about each document, just like you can in Excel.

And Microsoft must realise this: it has rebuilt its entire productivity suite around SharePoint. To be sure, the Office 365 platform — or whatever they call it now — is monumentally confusing: the Teams integration is baffling. The interaction between OneDrive, Teams, File Explorer, SharePoint and Outlook seems wilfully designed to create as much confusion, duplication and data loss as possible. Where once there was a single place for documents, communications and schedules, now there are three or four of each. Did you get that file I sent? I chatted it to you in Teams. Or, hang on, did I email it? Or Share a link from OneDrive?

Disaster.

The unbearable tolerance of spreadsheets

And this is the downside of the spreadsheet metaphor. A spreadsheet is a blank page: it leaves hierarchy and database design up to whoever wants to have a go at it. SharePoint leaves the discipline of getting the database design right, and then enforcing it, up to the user.

But this pluralism, this multiplicity, this total ambivalence to how users want to work, this tolerance of all the different possible ways users do work, however dopey, this total disarming agnosticism, that will even let users embrace the desktop metaphor if that’s what they really want — is a recipe for organisational chaos.

Because, as we should know by now, most users will embrace the desktop metaphor if you give them half a chance. Because they don’t know any better. Why would they? Database design optimisation is its own branch of computer engineering. You can even get a certificate in it.

Most users do not have such a certificate. Nor do they want one. They have not thought much about database design optimisation. Nor do they want to.

Rather, when they entered the workforce, or first clapped eyes on a computer, they saw a desktop and they liked it. They understood it and got very used to it.

So the desktop metaphor has proven resilient. It will remain hard to dislodge.

Yet it is odd — isn’t it? We intuitively understand the power of metadata when we are presented with a spreadsheet. But the same power does not occur to us when we are presented with a file management system. The desktop metaphor is burned on our retina.

Even though it is, in essence, a supercharged online spreadsheet, SharePoint continues to be resented by almost everyone. This is no more than Microsoft deserves for its terrible implementation, but at least give it a try.

Cross the Rubicon. Redpill yourself. Leave the desktop metaphor behind and finally go truly digital.

  1. ↑ Entertainingly, courtesy of some well-meaning rube, the Dewey decimal system limped into the information age, in the shape of webdewey. This may be the librarian’s equivalent of an online Amish supplies store.
  2. ↑ All physical information is eventually destined for the Iron Mountain.
  3. ↑ Compared with physical information. Yes, it does take up some space and as we get carried away into our brute force computing AI swoon the energy cost of data is no-longer negligible, but that is a peripheral issue in “normal” computing. Nevertheless, in this section take the word “almost” as read.
  4. ↑ Grammar pedants’ corner: Even though “data” is plural, “metadata” is generally treated as a singular mass noun. Please direct your letters to the Royal Statistical Society — not because it is their fault: rather, they might keep metadata about this sort of thing.
  5. ↑ This, by the way, is mad.
  6. ↑ As in, a lookup to an object in a people directory, and not just a text name.
Retrieved from "https://jollycontrarian.com/index.php?title=Template:M_intro_technology_metadata&oldid=91292"