itch.io store was taken down by Funko due to "trash AI Powered" phishing report

As someone who worked on the "other side" (in fact, I personally automated some abuse handling), I'd like to offer a bit of a different point of view.

Firstly internet in general is fundamentally flawed in that most of the protocols and technologies in use today were developed by some nerds in a uni/garage. They were never intended to be used at such scale nor were they designed to be abuse-proof.

As a result, internet is full of trash. I worked for some small registrar/host and we would get hundreds of abuse reports daily, week-end included. And the sad truth is that 99.9% of them were legitimate: hacked websites, domain registered with fake/stolen info for phishing, servers used to send spam, the list is endless.

Additionally, the bad actors are not dumb, abuses are often obfuscated (like displaying only if the referrer is google or at certain times of the day) to make detection harder. Checking whether a website is really compromised may require some technical skills (and thus be expensive).

It sucks for itch and imo, their registrar is still at fault if itch did indeed reply (should have trigger a human check). But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

As someone who worked on the "other side" (in fact, I personally automated some abuse handling), I'd like to offer a bit of a different point of view.

Firstly internet in general is fundamentally flawed in that most of the protocols and technologies in use today were developed by some nerds in a uni/garage. They were never intended to be used at such scale nor were they designed to be abuse-proof.

As a result, internet is full of trash. I worked for some small registrar/host and we would get hundreds of abuse reports daily, week-end included. And the sad truth is that 99.9% of them were legitimate: hacked websites, domain registered with fake/stolen info for phishing, servers used to send spam, the list is endless.

Additionally, the bad actors are not dumb, abuses are often obfuscated (like displaying only if the referrer is google or at certain times of the day) to make detection harder. Checking whether a website is really compromised may require some technical skills (and thus be expensive).

It sucks for itch and imo, their registrar is still at fault if itch did indeed reply (should have trigger a human check). But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

I agree. The economics at play simply make automation the only realistic option. I guess what we should wish for, in the case of the Itch.io story, would be a more sophisticated automated system. E.g. such that long-standing pages get a longer grace period before being killed or get higher priority human (or very good automated) verification.

The problem is that the "penalty of perjury" part of DMCA is never enforced on false reports. If it was, we'd have way less false positive reports

we *did* take the disputed page down as soon as we got the notice because it's not worth fighting stuff like that

Curious, what did itch put in the "disputed page" to trigger this?

"shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

I agree with all that you said, and even with your sentence i just quoted up there, because it's obvious, but it's also obvious that is a principle that cannot be used as a justification.

So “we're too lazy to sanitize our inputs” is supposed to be a good thing? And we were worried about DDOS attacks…

It sucks for itch and imo, their registrar is still at fault if itch did indeed reply (should have trigger a human check). But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

It can also be easily abused to get rid of legitimate sites. The only question is how long it will take before it happens.

But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

To play devil's advocate: why do people need protecting? Who's job is it to protect them? If these protectors constantly try to sanitize the internet, the users will never be exposed to scams. While this might be a good thing, consider your immune system. It gets stronger by being exposed to everything and anything. If you get scammed, like touching a hot stove, you might be more sensitive to scams in the future. Over years and decades people would be *forced* to learn to avoid scams.

I suppose my point is that there is so much nanny-ing in the current world that I fear we are actually doing a great disservice to most people in not giving them the opportunity to fend for themselves. I think we can see this manifesting in the younger generations that got participation trophies and were never faced with failure. When they got out of uni and into the "real world" they often times melt (ala snowflakes).

As someone who worked on the "other side" (in fact, I personally automated some abuse handling), I'd like to offer a bit of a different point of view.

Firstly internet in general is fundamentally flawed in that most of the protocols and technologies in use today were developed by some nerds in a uni/garage. They were never intended to be used at such scale nor were they designed to be abuse-proof.

As a result, internet is full of trash. I worked for some small registrar/host and we would get hundreds of abuse reports daily, week-end included. And the sad truth is that 99.9% of them were legitimate: hacked websites, domain registered with fake/stolen info for phishing, servers used to send spam, the list is endless.

Additionally, the bad actors are not dumb, abuses are often obfuscated (like displaying only if the referrer is google or at certain times of the day) to make detection harder. Checking whether a website is really compromised may require some technical skills (and thus be expensive).

It sucks for itch and imo, their registrar is still at fault if itch did indeed reply (should have trigger a human check). But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

If they want to accept some false positives, I'd say it is fine... if they take liability for it. Revenue lost, deadlines missed, possibly moral damages. Do they? Is it viable for a small business or individual to get it? Otherwise, they should have to prove they did due diligence before taking it down.

But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

To play devil's advocate: why do people need protecting? Who's job is it to protect them? If these protectors constantly try to sanitize the internet, the users will never be exposed to scams. While this might be a good thing, consider your immune system. It gets stronger by being exposed to everything and anything. If you get scammed, like touching a hot stove, you might be more sensitive to scams in the future. Over years and decades people would be *forced* to learn to avoid scams.

I suppose my point is that there is so much nanny-ing in the current world that I fear we are actually doing a great disservice to most people in not giving them the opportunity to fend for themselves. I think we can see this manifesting in the younger generations that got participation trophies and were never faced with failure. When they got out of uni and into the "real world" they often times melt (ala snowflakes).

Good devil's advocate. I would reply that in times past when there was no such thing as "public health measures", not only did plagues regularly sweep the land killing large percentages of the population in horrible ways, but in general cities in normal times had death rates so high they had to be replenished with ongoing in-migration just to stay the same size. When people live densely packed past a certain point, it would seem the disease organisms evolve faster than people's immune systems adapt. There is no point where people have toughened up and the result is better than having the public health measures.

Going back from the analogy, I would say that if you do nothing to stop cyber attacks you don't get to a state where people are attack-proof, you just continue to have a state where lots of people get successfully attacked.


Last edited by Purple Library Guy on 9 Dec 2024 at 5:23 pm UTC

The problem is that the "penalty of perjury" part of DMCA is never enforced on false reports. If it was, we'd have way less false positive reports

DMCA is very different from abuse reports. DMCA is an american legal system intended to protect copyright holders (more often than not, big companies with lots of money).

Abuse reports aren't a legal system, they're just a way for people or companies to report bad actions (phishing, miners, viruses, etc) to another company (usually a registrar or host). There are some common practices (such as the [email protected] mailbox) but there is no law or set process to react to reports. Therefore there is no "penalty of perjury" either.

If they want to accept some false positives, I'd say it is fine... if they take liability for it. Revenue lost, deadlines missed, possibly moral damages. Do they? Is it viable for a small business or individual to get it? Otherwise, they should have to prove they did due diligence before taking it down.

In the case of the company I worked at, liability was waived as part of the contract customers accepted. For the basic, cheap, competitive offer that is.

There were also offers which included manual review of any action or even remediation, as well as access to a qualified employee 24/7 and liability for SLA. However, these were not cheap, typically £250 to £10k per month.


Last edited by hell0 on 9 Dec 2024 at 7:37 pm UTC

i already hated funko pop before, i mean, why purchase funko if you can get an nendroid wich is way more cute?
anyway... now i hate funko even more, thanks stupid company!

But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

To play devil's advocate: why do people need protecting? Who's job is it to protect them? If these protectors constantly try to sanitize the internet, the users will never be exposed to scams. While this might be a good thing, consider your immune system. It gets stronger by being exposed to everything and anything. If you get scammed, like touching a hot stove, you might be more sensitive to scams in the future. Over years and decades people would be *forced* to learn to avoid scams.

I suppose my point is that there is so much nanny-ing in the current world that I fear we are actually doing a great disservice to most people in not giving them the opportunity to fend for themselves. I think we can see this manifesting in the younger generations that got participation trophies and were never faced with failure. When they got out of uni and into the "real world" they often times melt (ala snowflakes).

Good devil's advocate. I would reply that in times past when there was no such thing as "public health measures", not only did plagues regularly sweep the land killing large percentages of the population in horrible ways, but in general cities in normal times had death rates so high they had to be replenished with ongoing in-migration just to stay the same size. When people live densely packed past a certain point, it would seem the disease organisms evolve faster than people's immune systems adapt. There is no point where people have toughened up and the result is better than having the public health measures.

Going back from the analogy, I would say that if you do nothing to stop cyber attacks you don't get to a state where people are attack-proof, you just continue to have a state where lots of people get successfully attacked.

Well, replace the idea of a stronger immune system with the knowledge of hygiene. It wasn't "public health policy" but more that people gained an understanding of how to get less sick by sterilizing things and washing their hands. You could put vaccines and medicine in general in the same category. The health policy likely was a proponent of good hygiene in such the same way that a policy today could help teach how to prevent getting scammed. Either analogy still falls a bit short because we are talking about death versus losing money.

I tend to agree that doing nothing would result it more successful attacks, but I simply am not sure at what cost. Giving an organization or government power to shut things down they deem as scams feels like the greater evil than "allowing" some people to get scammed. It probably comes down to a liberty vs security argument and there will be valid points on both sides.

It sucks for itch and imo, their registrar is still at fault if itch did indeed reply (should have trigger a human check). But in general "shoot first, check second" is sadly the most efficient way to protect people from scams, phishing and other bad stuff.

Your first statement is true. In the age of algorithmic process, a human still needs to be in the chain of command.

However, the attitude in the second sentence is a problem. Accepting "shoot first" is a great way for people to weaponize any take-down process. Just look at DMCA removals on a lot of platforms.

AI vs AI is not a real solution.


Last edited by no_information_here on 10 Dec 2024 at 3:20 am UTC

Curious, what did itch put in the "disputed page" to trigger this?

From this article:
"Itch.io co-founder Leaf Corcoran offered more context in a post on Hacker News and said the phishing reported filed by Funko and Brand Shield related to a fan page for a video game called Funko Fusion, which is based on the Funko Pop! toy line.

"From what I can tell, some person made a fan page for an existing Funko Pop video game (Funko Fusion), with links to the official site and screenshots of the game," wrote Corcoran."

That makes it even more ridiculous.

Well, replace the idea of a stronger immune system with the knowledge of hygiene. It wasn't "public health policy" but more that people gained an understanding of how to get less sick by sterilizing things and washing their hands.

Really it wasn't either. The sterilizing-and-washing-hands thing made a huge difference in hospitals. Maybe in slaughterhouses if there were government regulations enforced. It wasn't really something that saved tons of lives in people's day to day home lives. What made the really big difference was plumbing--water and sewage systems. This, again, was not something individuals on their own could do, but rather a social/government thing. Without the plumbing, the safe water supply, washing your hands could as likely give you cholera as save you.

Really the biggest problem with the internet is that it is basically a "public good" in the economic sense but is being operated privately for profit.