> is there any evidence to suggest technology based solutions by themselves lead to any of those obviously bad false positive outcomes?
Specific to this type of technology? YouTube and their Content ID comes to mind - stories abound about people getting random videos flagged or demonetized because of a match to something in a background, or false positive match to their own original rendition, or to their own original rendition of their own work they own copyright to, or even against noise. Some of those stories end up harming those YouTubers financially. It's happening often enough that pretty much every channel I've watched has a video commenting or complaining about this at this point.
Then, related, random bans of Google accounts, or people having their apps kicked off Apple's App store or Google's Play Store for seemingly no reason (following some of those stories over the years, I estimate it's 3:1 false positive to the person actually violating ToS). Google account bans, in particular, can easily make your life very difficult for some time, or even kill your company, and they have a nasty feature of being transitive (if you have multiple Google Accounts connected in an obvious way, e.g. by recovery number, having one banned tends to be followed by having the rest banned too).
Point being, those big cloud companies don't have particularly good reputation when it comes to algorithmic moderation. So when one of those companies wants to deploy another set of automated scans, with a twist that a false positive now has a chance of quickly and irreversibly derailing your whole life, it's hopefully understandable why people are apprehensive.
> whatever the specific technology in question, it's difficult to imagine it playing out in the current legal system of any of country that isn't already based around just semi-randomly throwing people in jail.
With this particular crime, you don't have to go into jail to have your life destroyed. You don't even have to go to court. All it takes is that a rumor leaks out that you're accused of consuming or producing CSAM - it'll do irreparable damage to your relationship with others, as they're always be wondering whether there wasn't something to those rumors.
In a way, this technology being criticized so loudly and broadly is a form of mitigating the damage it could cause: the more people are aware it's liable to spurious false positives, the more chance the victims of such false positives have that others will believe them.
Specific to this type of technology? YouTube and their Content ID comes to mind - stories abound about people getting random videos flagged or demonetized because of a match to something in a background, or false positive match to their own original rendition, or to their own original rendition of their own work they own copyright to, or even against noise. Some of those stories end up harming those YouTubers financially. It's happening often enough that pretty much every channel I've watched has a video commenting or complaining about this at this point.
Then, related, random bans of Google accounts, or people having their apps kicked off Apple's App store or Google's Play Store for seemingly no reason (following some of those stories over the years, I estimate it's 3:1 false positive to the person actually violating ToS). Google account bans, in particular, can easily make your life very difficult for some time, or even kill your company, and they have a nasty feature of being transitive (if you have multiple Google Accounts connected in an obvious way, e.g. by recovery number, having one banned tends to be followed by having the rest banned too).
Point being, those big cloud companies don't have particularly good reputation when it comes to algorithmic moderation. So when one of those companies wants to deploy another set of automated scans, with a twist that a false positive now has a chance of quickly and irreversibly derailing your whole life, it's hopefully understandable why people are apprehensive.
> whatever the specific technology in question, it's difficult to imagine it playing out in the current legal system of any of country that isn't already based around just semi-randomly throwing people in jail.
With this particular crime, you don't have to go into jail to have your life destroyed. You don't even have to go to court. All it takes is that a rumor leaks out that you're accused of consuming or producing CSAM - it'll do irreparable damage to your relationship with others, as they're always be wondering whether there wasn't something to those rumors.
In a way, this technology being criticized so loudly and broadly is a form of mitigating the damage it could cause: the more people are aware it's liable to spurious false positives, the more chance the victims of such false positives have that others will believe them.