Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Make It Ephemeral: Software Should Decay and Lose Data (pocoo.org)
31 points by BerislavLopac on Nov 2, 2024 | hide | past | favorite | 40 comments


This was an excellent read and it resonates with me a lot.

I have written here before[1] about the tyranny of "read it later" apps and how liberating it felt to ditch them and move to a "read it now or read it never" (RINORIN) mindset.

Although this articles talks about digital ephemera in different contexts such as shared spaces, wikis, runbooks etc., I believe the underlying idea is the same.

[1]: https://news.ycombinator.com/item?id=41991376


Not everything important is equally urgent. https://en.wikipedia.org/wiki/Getting_Things_Done recognizes the need for a trustworthy "I know I am going to get back to this at the right time" process, so that I don't continually reserve a bit of my brain worrying about forgetting.


It's already ephemeral and decays by itself.

Honestly as a polyglot, only a Go project can run without being touched for 5 years.

The rest of them get vulnerabilities discovered in the packages, which requires you to upgrade, which changes the feature set and takes re-work.

Also design ages, UX paradigms change over time, look at the "web 2.0" look today and the "web 1.0" of before. Skeuomorphic design aged as fast as Garage and Jungle music did in the 90s. This will be the same with flat design and neuomorphic design in 5-10 years, if there is even an UI then.

All in all, software does decay and we lose data every day. Try go through your oldest bookmarks, many sites will be gone. Or run software that used to run on windows 98 or DOS or amiga or classic mac, you need to jump through hoops, the UX will be terrible, that's aging.

Software doesn't need artificial aging, it erodes well all by itself. Even fast moving code in a startup changes so much from week to week, your merges can be difficult. That's like a river bed redefining itself during a rain storm.

As humans go through time, so does software as an extension.


> Honestly as a polyglot, only a Go project can run without being touched for 5 years.

Go CVE archive [1] clearly indicate otherwise. What distinguishes a Go project from any other programming language?

[1] https://pkg.go.dev/vuln/list


> What distinguishes a Go project from any other programming language?

My experience with Go, as user and open source maintainer, is that many libraries just work even if untouched for several years.

There is a culture around building small libraries, with minimal usage of dependencies beyond the standard library. This leads to most CVEs existing in the standard library, which combined with Go's commitment to backward compatibility, makes improving your security posture simple by just upgrading the toolchain.


The physical world does impose itself on software in some ways. Software is prone to going out of fashion, becoming disused and unmaintained, and eventually forgotten. Peering at a long dead document format in a hex editor is the digital equivalent of wiping dust and grime from the wall of a forgotten tomb.


I don't think I could disagree more. He compares software to "the physical world", but photos in photo albums don't just disappear because they get old, and if I put something in a filing cabinet I know it'll be there in 1 year or 10 years.


That's because you took the care to put them in the photo album where they would be protected. If you left it sitting naked on your desk it would fade, accrue coffee stains, etc. Same goes with the filing cabinet. That's what Ronacher is suggesting - save the things that people care about, discard the things that are no longer relevant.

I don't think it's a dogmatic position either, I don't think the suggestion is to always do this but to do this far more often than it's done now (almost never). And they are pretty explicit that they think this should be a default which can be overriden, and that you should be able to save things indefinitely if you so choose.


What are some concrete examples of this?

- Linear automatically cancels tickets which haven't been touched in 6 months

- Trello would visually "age" cards, making them increasingly yellow and tattered

- Datadog dashboards have a "popularity" score based on usage


As jwz points out in "The CADT Model," abandoning reported bugs is just telling your users not to bother reporting anything.


In my experience, things don't get done just because they are hanging about in the backlog.

Most auto-closed issues in my small team are of the varieties "the PM had a nice idea one day" and "long standing annoyance which we would need to re-architect to solve and we'll get to it one day".

And I don't even necessarily disagree with the CADT post, but... most users are not, for better or worse, so easily dissuaded.


For this reason, my browser's Downloads folder is /tmp.


> For this reason, my browser's Downloads folder is /tmp

OK. But my desktop reaches uptime of six months very easily (its not common that a security update mandating a reboot happens with Linux).

Once in a while I clean my browser's Download folder. When I do that the very first thing I do, before the mess creeps in everywhere in "proper" directories is to deduplicate the browser's Download folder.

The amount of time I'll have bankingstatement20241103.pdf, bankingstatement20241104-1.pdf (which should be really be "-2" btw instead of that zero-based non-sense but I digress), bankingstatement20241104-2.pdf, etc. in my Download folder is insane.


> I'm referring to making a deliberate design decision to discard data at a later time. I'm willing to bet that your cloud storage or SaaS applications likely serve as dumping grounds for outdated, forgotten files and artifacts. This doesn’t have to be the case.

IMO there is no way to do this programmatically that satisfies all of your users. What the author describes is organizational debt, and the only way to deal with it effectively is to have the organizational will to periodically clean up that debt. It's not easy, and any shortcut comes with it's own drawbacks (see corporate data retention policies that inevitably nuke documentation or context).


Decay and losses are just two of the many constraints that could be mixed like paints to create new systems.

Limitations, artificial or not, are not always bad. Walls, for example, can be seen as limitation or protection depending on how it's used.


Thought about this. I agree 100%. The incumbent belief in software that machines should always remember (embedded in postgresql logo for example) has a dystopian sense of inmortality desires.

I would like to play with the idea of a filesystem layer solution that tiers out information based on staleness. The closest I've seen to this is AWS S3 tiers, you've got standard S# at 2.2 cents per GB month, then infrequent, then glacier and some tiers in between.


Methinks the mindfulness cult has gone too far


That's a very feel-good "why" that doesn't answer on "how", which is 99% of the work. How exactly do you select what to forget?


The attempts I've seen at this are usually in support of a corporate data classification and retention policy and are pretty hard to pull off successfully. The attempts I've seen usually involve adding metadata or tagging the data either manually or using automated classifiers.


I mean, retention policies exist.


This is a strange and awful take. Physical objects decay to the fundamental forces of nature, but that is a flaw, not something we should emulate.

Software is a tool. Software is infrastructure. Degradation is desirable in neither.

Data has metadata - creation time, access time, etc. - that can power workflows to identify and deal with old data if that is desirable, but in most cases they are not desirable. I haven't looked at the directory full of my wedding pictures in a decade, but I don't want my photo editing software to decide they're irrelevant and delete or hide them.

Having access to old documents and data is often important and beneficial. It's why people look at inscriptions in old books, why they keep photo albums, why finding a forgotten item when moving furniture can be delightful. It's why we're able to identify trends.

Most crucially, it is incredibly difficult to predict in advance which data will be useful in the future. Proactively discarding things when storage is incredibly cheap is a net negative.


Disagree, I'm with Kay on this one, life is much more powerful than current machines.

When was the last time your mind ran into an out of storage issue and crashed?

You talk about metadata being important, yet more than half of the storage used today isn't used to store books, or medical records or whatever business case, but traffic logs and virtualized package dependencies and shit like that.


It's called a senior moment, and you'll have them too if you've never had one before. In a middle of a conversation, you'll just forget what you were saying because you got distracted.


First of all that's memory, not storage.

And second, it's a graceful decay, you are not completely shut down from storing imporant information for later use. If in that moment something memorable happens, you will remember that.


The disagreement here seems to be about what kind of data should be considered ephemeral. Traffic logs from a decade ago? A ten-year-old package lock? I'm sure I don't care anymore. Documents I wrote in the 90s? Yeah, I might still want them. I might not need them everyday, but I can say from experience that I've needed to track down files that are more than a decade old, and I'm glad I was able to find them.


We strictly talking documents? It's very unlikely they will suffer from any decay since they are so lightweight. And to the extent that it's important you probably accessed it or shared it sometime in that period.

A video from the 90s that you never read or accessed in that period? Yes, a candidate for deletion, or at least a decay. For example, if it was a video, the transcript might be saved, but the video might be lost. We may also get some written interpretation on the visual elements of the video.


I'm talking documents in a very general sense. It might be a text file, audio, video, a software installer, source code, or anything at all.

> A video from the 90s that you never read or accessed in that period? Yes, a candidate for deletion, or at least a decay. For example, if it was a video, the transcript might be saved, but the video might be lost. We may also get some written interpretation on the visual elements of the video.

My point is that the video shouldn't decay, either. It might not be playable in modern media software for a variety of reasons, but that doesn't automatically make it a candidate for deletion.


If I learned of a old video of my grandparents, but a robot destroyed it, I would be furious. (Photos are what actually remain from their era.)


Oh yeah, taking screenshots would be a nice intermediate decay, instead of the whole video.

You have to think, in general, that we have finite resources, and the alternative is not remember everything in perfect detail, but forgetting things that you do not choose to forget.

Better to choose what we forget.


After forty-ish years, the limit on personal storage is ever higher. The sum of every filesystem I ever owned will probably fit on two of today's very affordable hard drives. (I should get on that for pre-cloud stuff.)

And decayed versions wouldn't get the author much mercy from bereaved families.


1- moore's law will break eventually. Depending on storage growing is unsustainable. We will need to develop solutions that work when storage growth becomes linear.

2- even if you can save the bits and bytes, there's fidelity loss in soft and hard standards, namely video codecs and connector standards. Not sure how planned decay would fit here, but relevant considering that document from the 90s needs is probably on a weird codec in a pre pata drive. Or even worse an analog medium.

Even if you don't admit that it will happen in this generation, it will happen in the next. We have companies that offer free storage like google, the economics can't sustain that forever, youtube videos are already being purged, google drive limited.

The cost of storing forever is unsustainable over the decades.


If you're trying to upgrade or rebuild ten year old software, it's very important to know exactly what its dependencies were and where to find them, because there are too many other versions to choose from and most of them won't work. New software only needs a lockfile because it will eventually get old (if useful).


Fair point. But that just means the breadth of files that might need to be preserved is even broader than I suggested.


Author here.

> Physical objects decay to the fundamental forces of nature, but that is a flaw, not something we should emulate.

That is explicitly called out in the article and I do not agree with you. Useful information is replicated, less useful information tends to die out. That we threw away that concept entirely in the digital world without a good replacement has some pretty strong downsides from my experience.

And if you read what I wrote in more detail you can see that I'm very open to the idea of soft deletes and intentional information hiding.


Information that is known to be useful is sometimes replicated. The https://en.wikipedia.org/wiki/Rosetta_Stone#Rediscovery was abused and damaged; only four incomplete copies were gradually found by luck after two millennia. There are entire academic fields trying to learn things about our history by digging up stuff that was thrown away or abandoned.


[flagged]


I don't see how the headline is "inflammatory". You might disagree with the headline and the content and that is fine, however both headline and content reflects what I believe a core capability of (at least a significant number) of software products should be.


You think a core capability of software should be that it loses your data? That's like saying a core capability of shoes should be tacks under the sole.

Of course you don't really think that. What software do you use that loses your data. Don't say something that cycles data out of a cache or lets you delete data, because that wouldn't be losing it.


No one forced them to write a comment without reading the article. And it’s not a small point that the author corrected them on; multiple paragraphs addressed thoughts like archiving to make searching old things intentional, and soft deletes.

Also I didn’t read a scolding tone in the author’s response. And calling it an “inflammatory headline” is quite a stretch: this is a post tagged “thoughts” in someone’s personal blog, not a headline in the New York Times.


You can view software as tools or art, it is arguably both.


>It's why people look at inscriptions in old books, why they keep photo albums, why finding a forgotten item when moving furniture can be delightful. It's why we're able to identify trends.

I think it's often the opposite. Drawing confusing or bad analogies to the past rests on the false assumption that the future resembles the past, and the reason people love old photo albums or forgotten items is usually a kind of nostalgia, not because they hold relevance towards the future.

This is also why forgetting or deliberately wiping the slate clean is fundamentally a good thing, it forces people to rebuild things from first principles without being bogged down in false assumptions carried over from the past, it's why the people who build new things tend to be young rather than old. If past information was so valuable in determining the future most visionaries would be 60. Hanging on to old information is a form of digital hoarding.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: