Imagine if you could also listen to these documents, with the author reading, and the text tracking.
I believe the effect of that one simple addition would amplify their effectiveness (or at least reach) by an order of magnitude.
After the massive work that their authors put in, the effort of one good read-through would be marginal. If the above surmise is true, you'd be in power law territory.
But you'd need a tool to make such an additional channel as trivial as doing one good read-through.
I'm a little surprised that Bret Victor doesn't do this for his pieces, considering his emphasis on the use of more human capabilities[0].
Implementing the read-back part is trivial [1], and available on all "modern" browsers.
Trickier is making the product that lets authors add this channel without manual tagging. But even this is becoming trivial with speech-to-text API's.
Anyway, I wholly agree with the OP that the first focus of documents in dynamic media should be improved exposition and, by extension argumentation.
Can you expand on why adding a narrative would amplify the effectiveness of these explorable explanations by so much? The whole core of explorable explanations is that you can play with the models yourself to gain insights and reach conclusions the author may not have come to herself, to me, adding a narrative would only add to the linearity of the text itself, and therefor take away some of the "exploration" part
First, consider the difference between two people reading aloud: one is new to the material, concepts, and associated vocabulary, and the other is the author. They are reading the same text, but the author's rendition will carry an understanding of the material that is absent in the other one—and not found in the text, either. Doesn't that difference comprise relevant information?
Shakespearean actors do this for a living: they stick to the words on the page, but they add another layer of signals to help unlock them for people. Technical and expository writing can benefit from this effect as well. Don Norman argues, for example, that thinking requires emotion, and the voice is an easy way to add an emotional connection.[0]
So I believe that any document, explorable or otherwise, is amplified by a human reading, when that human really understands and cares about the material. And again, for subjects that warrant these specialized explanations—which tend to require some specialized language—the value added is particularly great, because it teaches people how to read something new.
Second, I strongly believe in an "all-in" approach for things that you care about. Clearly these authors care about reaching people, and I think there can be no doubt that the ability to click "play" brings in listeners who would never have been readers. And by giving a passive "flow" to the presentation, it helps sustain many readers who would have given up.
Adding a channel does not "take away" anything. Everything, including the interactivity and the opportunity for non-linear reading, is still there. (And note that the recording doesn't have to be linear, anyway.) The authors are not competing with themselves, they're competing with YouTube and notifications and other low-friction distractions.
So I'm excited about the OP. I would like this community—mature, passionate thinkers and teachers—to be the avant garde of future media.
> They are reading the same text, but the author's rendition will carry an understanding of the material that is absent in the other one—and not found in the text, either. Doesn't that difference comprise relevant information?
Not to me, as someone who's never run into the material before. There's no knowledge-carrying particle that's transmitted vocally.
In fact, to me it would be a distraction. I learn better visually; I can stop reading, and start again, and quickly re-scan what I'd read before to catch up. It's harder to do that with a recording of a voice, and takes longer.
Other people might say the same about the graphics, or the interactions.
The point is that it's not zero-sum—no one loses out by the presence of other channels, if the document is dynamic. But if some people do benefit, then there's a value proposition.
I imagine that the reason for adding narrative is to add another dimension that's proven to be engaging in other mediums; one akin to explanation videos like those found on Khan Academy or the educational sections of YouTube. Sometimes jumping into an explorable explanation without some guide to begin with (outside of text) may be overwhelming.
That being said, I don't know how a spoken narrative would integrate into a piece of text. You would likely not want someone to read through the text as written because a spoken explanation will generally differ in phrase structure (shorter, less verbose) than written text.
The thing about doing this kind of work is you have one sort of explanation in mind as you do it, and like any software, you work until you're there and then you call it done. They're not designed and built with every possible form of accessibility and exposition in mind. It's entirely likely no-one had previously considered doing a narration.
Instead of isolated examples that happen to use math, would it make sense (or even be possible) to start a platform for visualization of mathematics? Something like context-augmented LaTeX. Not a computation engine (like mathematica or sage), but something closer to a github for math (visualization and exploration).
Consider how much time it took you to learn Gaussian elimination from a book or lecture vs how long it would take to see an animated example and then pick up the details.
Or consider how ambiguous mathematic notation can be [1].
So much of mathematics (purportedly the most structured knowledge produced by humankind) sleeps in mountains of unread journals and inside the brains (software) of mathematicians that are trained to do these visualizations on wetware.
It is a feature of math that these aren't hard-coded, as in the abstract one can conjure up any edits the imagination desires without the complex implementation details typically required by a computer. And in some cases a visualization (such as geometry) can be misleading and reduce generality [2].
However, there could be much to be gained by the majority from the hypothetical platform. Computation packages like mathematica only incidentally help visualize, as a byproduct.
Take for instance the videos by 3blue1brown [3]. Their appeal (and those in the parent link) is that they reduce the individual mental overhead and that a carbon copy of the visual is imprinted in the learner.
There is no punchline to this comment. And I'm certainly not an expert on the matter. I still find the 'movement' fascinating.
I believe the effect of that one simple addition would amplify their effectiveness (or at least reach) by an order of magnitude.
After the massive work that their authors put in, the effort of one good read-through would be marginal. If the above surmise is true, you'd be in power law territory.
But you'd need a tool to make such an additional channel as trivial as doing one good read-through.
I'm a little surprised that Bret Victor doesn't do this for his pieces, considering his emphasis on the use of more human capabilities[0].
Implementing the read-back part is trivial [1], and available on all "modern" browsers.
Trickier is making the product that lets authors add this channel without manual tagging. But even this is becoming trivial with speech-to-text API's.
Anyway, I wholly agree with the OP that the first focus of documents in dynamic media should be improved exposition and, by extension argumentation.
[0] "The Humane Representation of Thought" https://vimeo.com/115154289
[1] For example, see a crude implementation at https://willshake.net/plays/Ado/1.1 (click a play arrow in the blue column.) The tracking code is at https://willshake.net/about/text_tracking