People on HN have a hard time understanding that the barrier to entry matters. Photoshop has existed for decades but the level of skill needed to make realistic looking nudes in Photoshop is not something your average freak with a fetish possesses. New technology puts that power into the hands of every single person.
It's not your average freak with a fetish that is even the primary issue. The volume will come from average anybody (although mostly men). That's the real problem. Random co-workers in the office that want to deepfake their crush. Your average teenage boy with a decent GPU that wants to deepfake their classmates just to experiment. There have already been several instances of this catching the news in Europe for example, so it's obviously going on globally.
Oh but the average guy isn't that perverted? Yes they are. We know this from the vast amount of porn they consume and of what type they consume.
Actually nailing someone for revenge porn or sexual harassment seems like a borderline impossible task for police departments if the person doing it has even a minor degree of anonymity. Enlarging the potential pool of people who can do this and the pool of victims they can target by an order of magnitude by lowering both the skill it took to make convincing fakes before and the time taken to make each fake doesn't seem to me like a good idea.
I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI. Yes, they've been around for well over a decade, but never before has their creation (& thus volume) been so accessible to the average layperson. A simple 10 minutes on CivitAI will tell you all you need to know about how prolific this problem is.
IMO it is only a matter of time before this same issue comes to the forefront in the US (I'm kinda surprised it hasn't already become a bigger topic). I could easily see custom celeb LORAs/emmbeddings and realistic SD/SDXL checkpoints being banned outright for this reason. Yes, it would be an infringement on free speech, but weighed against the interests & safety of the public, the courts might see a move like that as reasonable.
It's just the new reality. Porn is not even the worst thing. If anything it can make real photos/videos harder to get viral.
But augmentation will be (if not already is) used to make very tiny adjustments, like you make one candidate/celebrity face slightly more symmetrical and the other one less, change their gait just a little bit. These changes are non-perceptible, are not interesting enough to make a scandal, and they can have huge impact on how people perceive other people. I mean, I am not suggesting that it isn't a puppet show without those, but the game is evolving.
> ban photoshop all over again. It's just the new reality.
We didn't solve teens doctoring inappropriate images of their classmates by banning Photoshop or accepting it as the new normal. We solved it by passing laws which criminalise the behaviour and enforcing them to deter the behaviour.
Technology influences society. But plenty of social mores are almost universally respected despite being difficult to enforce. Possible and difficult to enforce doesn't force permissibility.
I was a teen post photoshop and it was just... never an issue. It never occured to me. No one ever mentioned it to me as a possibility. If someone did it would have struck me as weird and as way too much work. Photography class in school (in which we learned to use photoshop) didn't mention any acceptable use guidelines or anything. As far as I know no one in my school ever did anything distasteful with photoshop.
I don't accept the idea that "passing laws and enforcing them to deter behavior" was the cause of the lack of issues, because to be a deterrent us teens would have had to been aware of the laws.
> was a teen post photoshop and it was just... never an issue. It never occured to me
Same. And same.
Drawing inappropriate pictures of your classmates was creepy before. Making Photoshop or AI porn remains so now. Most people won't do it. But there is active social reinforcement that keeps it from becoming "the new reality."
> don't accept the idea that "passing laws and enforcing them to deter behavior" was the cause of the lack of issue
Fair enough.
The jargon was cyberbullying, and there was absolutely legislative activity around horrific examples. But the principal mechanism of action probably wasn't a direct fear of getting caught / punishment, but the prompted discussion reinforcing and extending the norm.
This is just a certificate of authenticity that one either trusts or not, based on whatever one knows about secure enclaves, cryptography, or the device/person issuing the key/certificate.
I think the reality is that photographs and video are now like text, you trust the source or not.
I can write about Cleopatra's ability to snowboard and play Xbox, or I can make a photo with the caption "Cleopatra doing a 1080", and now I can make a video of it (probably).
I can also give you a certificate/key/blockchain whatever to prove it's "valid".
> devices need to cryptographically sign the original video with keys stored in some kind of Secure Enclave
This is, in practice, verifiable by nobody. It's a neat trick for e.g. judicial purposes. But it's almost worthless in the public domain and absolutely worthless for someone refuting that a video purporting to be of them is fake.
> will be a time when the Trump "grab them by the you know what" style scandal will just be met with sceptism
Politicians since the dawn of time have claimed their scandals were smears. The era of irrefutable recordings and photography was somewhere between short and non-existent.
Photoshop was never built as an easy tool so powerful that even grandma can produce photo-realistic fake images in seconds. The new wave of AI powered editing tools are clearly on a new level.
This is like bringing your howitzer to a gun debate because they are kinda similar.
Generative AI is much easier than photoshop the same way that photoshop was much easier than trying to alter photos any other way. In both cases it is a completely new level.
A centrally-managed app will always be easier to use than a local self-service tool. But those local apps will keep trying to converge towards the ease of the hosted apps. Over time I expect the difference will become negligible (i.e. a 1-click install and it actually works)
Photoshopping something convincing has a much bigger bar to entry than (as far as I understand it) generation of deepfakes now. Lots of time/practice invested even for one convincing image.
From what I can see the issue here is that this problem with AI can potentially become much much more widespread because the amount of work required is pretty much none and gets lower as models get better. It sounds like someone using deepfakes to ie extort people can just cast a much wider net by just scraping/dumping masses of images into a generator to create a multitude of convincing pictures of many different people instead of having to focus their time and effort on picking a single target and creating fakes manually.
I don't think of myself as on the e/acc side, but I think that deepfakes are here to stay and we'll just have to get used to them. I don't see any way to put the genie back in the bottle but I also don't think it'll take that long to adapt.
South Korea has a very bad gender relations problem partially stemming from an extremely lopsided gender ratio. In the 90s it reached as high as 115 male children to 100, and now all those extra young men are angry adults. https://theconversation.com/south-koreas-gender-imbalance-is...
Fair point, technologists will always innovate faster than politicians are able to regulate. Cat's outta the bag now, but that only means the problem will continue to escalate until more drastic measures are deemed necessary.
It's that or we will come to accept some level of societal degradation due to the abuse/misuse of AI tools becoming normalized.
As AI image generation (and soon video generation) gets to be a more saturated space, we will end up in the state that you cannot trust anything you see online, even if you are educated, well-informed, and meticulously research the topic.
It has already been inching that way for a long time, but generative AI definitely is adding new fuel to the fire.
I mean, you already kind of cannot trust anything you see online, it's just that so many people still haven't got the message. Maybe everyone needs a deepfake porno of themselves sent to them so they finally grok that everything online should be by default considered fake.
> I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI.
I don't think they're a concern at all, other than that they will ruin the evidentiary value of photographs. They'll do that whether or not the general public has access to them.
The concern about fake pictures of people naked is stupid. The same process mentioned above happens to naked pictures of people: they lose evidentiary value. General access to the ability to make deepfakes not only makes it so people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem. You can claim it's a fake, and no one can disprove it. This is a reason for celebration.
The standard response for that has been for people to simply claim that it won't matter, and that people will destroy the life of some young innocent girl if a mean boy spreads fake pictures of her naked. The answer is that people who harass a young girl are guilty of harassment, and should be jailed. You shouldn't be harassing people even if naked pictures of them are real, which they often are. And if you'd attack a person over pictures that you could fake up yourself in five minutes, you're either a) a moron or b) somebody using the photos as an excuse, and it would be a benefit for society for you to be locked away from it.
I suspect a lot of people really worried about deepfake porn deep down are worried about their ability to properly identify and harass the harlots if people won't believe the pictures that you send to their family and friends anymore. From my perspective, they're literally fighting to preserve revenge porn.
> The concern about fake pictures of people naked is stupid. The same process mentioned above happens to naked pictures of people: they lose evidentiary value.
This is too narrow: it presumes rational contexts where evidence will be evaluated critically, and it assumes a mild, uncreative attacker but harassers are good at exploiting social norms. The damage caused by this kind of harassment isn’t seeing yourself nude as much as wondering or knowing what other people will think, and there’s no way to undo that. Even if people _say_ they believe that the pictures are fake, the victim may be wondering whether that’s really true for the rest of their life.
The other thing to think about is that we’re really just getting started. Most of the deepfakes which people outside of the scene have seen are fairly mainstream porn nudes: there’s a huge control issue but in general there are worse things than people thinking you have a star’s body. We’re going to see those worse things soon: it won’t be a porn fantasy but things designed to degrade and more targeted - if the target had body issues, exaggerating them; punishing the person who rejected someone by faking them looking not like a porn star but less attractive and having degrading sex with someone unpopular, knowing that a bunch of people are going to think “well, he would say that”; make some teacher’s life hell by faking them having sex with a student; etc.
Because the attacks are being directed by humans, they’re going to adjust and find what works. Anything which forces an official investigation is going to be especially bad because we can’t just ignore things like child abuse so it’s going to be like SWATing where there’s an ugly period where the wrong teenager can get an incredibly high-leverage response with a single message.
I think the problem is that people aren't necessarily logical or rational about such things. I know someone (family friend) who was the victim of similar harassment with (clearly) photoshopped images being sent anonymously to her work, family, friends on social media. The fact that they were fake didn't diminish the humiliation or stress she went through and I don't think she ever really recovered from the trauma of having some anonymous asshole harassing and stalking her and her family online. It eventually stopped but she was afraid of it starting again for a long time. "It's fake" isn't much of a comfort to someone being harassed and being afraid of harassment escalating in one way or another. And it getting easier and easier for crazy people to do this with less and less effort doesn't help the issue at all regardless of what laws are passed. Finding better ways to deal with these people is a good thing but enabling them to make it easier isn't, in my opinion.
I think for a lot of people it's easy to die on the hill of technological progress and freedom at any cost when the consequences haven't happened to you or a loved one.
We definitely need to crack down more on harassment and stalking and not worry so much about what form that harassment and stalking takes. It's insane that you can have someone all over you, sending you unwanted messages, following you, or generally targeting you with misery that falls short of physical assault, and the police won't do anything about it until you're physically hurt. It shouldn't matter if the harassment is phone calls at 4AM or deepfakes. Harassment is harassment.
I agree. But the internet age feels like it unfortunately has made this much more difficult than it should be. People over share on social media and make themselves too visible because they aren't aware of the consequences- or don't assume there are people out there sociopathic enough to harass someone to that extent. The pool of potential suspects when they're acting anonymously has grown from whoever is in your local community to whoever has an internet connection in the world. Not uncommon to hear of such harassment, blackmailing cases and such where the perpetrator doesn't even live in your own country where police could do anything about it even if they had their name and real evidence.
This is going to be a monumental issue when the fact that we can't trust any photographs fully takes hold. Maybe watermarking will help with that. More likely people will develop tools to circumvent watermarking technology (either on the imprinting side, or on the verification side).
> people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem.
This may be a comforting way to think about it, but it won't work in all cases. Consider instead of revenge porn, the case of CSAM – that will never be a solved problem until 100% of it is wiped from the face of the earth and it is never produced again. Making it less believable because there is so much AI-generated equivalents out there is a not a solution to the problem, it exacerbates the problem.
> when the fact that we can't trust any photographs fully takes hold
Full faith in photography was never a thing. If anything, the advent of photography came alongside yellow journalism. Trust is an institution. You can't build institution on solely technology.
Good context. But the fact is, a great many people – perhaps even the majority of people – are quite used to looking at a photograph and fully trusting that it depicts something which actually happened. As they say, seeing is believing.
Even with CGI being what it has been since the 70s, it's still appropriate to apply Occam's Razor in many cases, and to simply say that because it looks real, it probably is.
I think freedom of speech is mainly about freedom of thought and just plain freedom, and the ability to criticize the government is a side benefit. And that it should also include the freedom to augment our imaginations with the prosthetics of our choice.
> Freedom of speech is mainly so that we can criticize the government
Freedom of speech is a bigger concept than the First Amendment. Technically curtailing one's ability to make AI porn of your colleagues is a restriction on speech. That said, so is every spam filter, fraud statute and retaliation against a declaration of war. A right doesn't have to be absolute to be good (or vice versa).
Freedom of speech is against government intervention, not against private agreements.
Rather in fact an absolute form of freedom of speech would protect against the government from stopping you from forwarding all your correspondence to someone else, and would protect against the government stopping that someone else from removing the spam and sending the rest back to you. Non-government run spam filters aren't just not offending freedom of speech, they are protected by it.
> Freedom of speech is against government intervention, not against private agreements
You're conflating the First Amendment and freedom of speech [1]. If you're in my house and I prohibit you from expressing certain views, that is fine by the First Amendment but not in the spirit of free speech.
No I'm not. You're conflating freedom of speech with a duty to listen. Freedom of speech entails a freedom to send spam. It doesn't entail a duty to receive spam, to read spam, to not have someone else filter your communications for spam, etc.
And as I argued above, since every step in having someone filter your communications for spam is simply speech, it protects spam filters.
You restricted it to "government intervention, not...private agreements." The freedom of speech extends to the private sphere.
> You're conflating freedom of speech with a duty to listen
These are inextricably linked in the classical form. In the Athenian tradition, there was a duty to listen [1]. (Though even then practicality made it far from absolute.) That traces through to the Protestation of 1621 [2] and the Founders' debates around the First Amendment (namely, between Madison and Hamilton).
Practically, of course, no duty is absolute. But a society where everyone screams into a void isn't adhering to the principles of free speech and expression. (I'll note that alongside the duty to listen is an implied duty on the part of the speaker to speak truly [3]. Spam is the epitome of the cessation of the latter revoking the former.)
I think you're confusing something along the lines of "parliamentary privilege" for freedom of speech. Various forms of government have imposed as part of their decision making process a duty to listen to various forms of official speech. That's not a component of freedom of speech though, rather it's a component of that specific form of governments process.
I'm not sure where you find a duty to listen in Athens, my best guess is you are pointing to the duty to attend the assembly, which fits this form of "parliamentary privilege" rather than a broader duty to listen. To contrast, by my understanding political commentary in the form of plays and the like were quite common in Athenian culture, and I'm fairly sure there was absolutely no duty to pay attention to those. I'd also note that even with the assembly "the crowd might raise a clamor and refuse to listen to a speaker advocate an unpopular proposal" (your source) which suggests little duty to actively listen even there.
The protestation of 1621 is directly related to parliamentary privilege (and why I labelled this the way I did, though I don't think the label fits perfectly, since it applies to many smaller government meetings that parliament).
I can't say I'm familiar with the first amendment debate you are discussing.
> You restricted it to "government intervention, not...private agreements."
I don't think this is the cause of our disagreement, but I'll grant you that that phrase was slightly inartful. E.g. there are authorities other than governments, like a local gang, who could impinge on freedom of speech with threats of retaliation, and to the extent that private agreements are enforced by an authority it's possible for them to do so.
Information intentionally communicated by a person to another is speech by the strictest classical definition. (I guess technically the creation of the AI porn isn't speech in the way a falling tree in a deserted forest doesn't make a perceived sound. But it's a schoolyard riddle for a reason.)
Freedom of speech is there to protect objectionable/offensive speech, whether it is directed towards the government or otherwise. Objectionable speech is frequently in the crosshairs of pro-censorship folks, and many people think of deepfakes as objectionable – reasonably so.
I feel as though I'm the only one who doesn't care about this.
Curious to know if 20-30 years ago, as graphic design tools got better, were people freaking out about being able to superimpose another person's face on a the body of p*rnstar?
Superimposing people's faces on to porn star's bodies happened/s, but does anyone care really even notice or care about it now days? For the most part, of course not.
So why should we care about porn videos? What makes them so different? The whole concern smells like typical media hype (i.e. yay - we get to write about AI + Porn - instant clickbait) that nobody will be able to stop and nobody will even care about in 1-2 years: e.g. "oh, yah, another deepfake of <(politician|actress|colleague^)> in a porn film - how incredibly creative".
^ Incidentally, this was made light of in the British version of The Office where a modified image of the boss, David Brent, was sent around the office. If it's joked about harmlessly on mainstream TV, it's probably not a huge concern.
It's more problematic than photoshopping a person's face was decades ago because we didn't have pervasive online social media decades ago. Computers weren't in your pocket, and an offending image couldn't be beamed to everyone you know instantly.
> If it's joked about harmlessly on mainstream TV, it's probably not a huge concern.
There are plenty of mainstream TV jokes about horrendous things. Our ability to joke about something is a poor way to gauge how concerned we should be.
> an offending image couldn't be beamed to everyone you know instantly
Is your concern that recipients would think it's real? Or is it that people will get an email/chat message containing NSFW material? (because the latter happens now, and is nothing more than background noise at most). What is your actual, specific concern? (genuinely, I have no idea, but want to try to understand - to me it feels like panic about a problem that isn't actually a real problem).
I'm not sure what my personal concerns are specifically, so I don't have a good answer to that question.
But I think there's a difference between a crud photoshop picture that a person has on their PC, and a (more or less) convincing deepfake video of a person being broadcast.
I think you're likely right that there is an element of panic. I don't know a lot about Korean culture, but I imagine the concern is that deepfakes can have real-life negative impact on people's lives, either because people think it's real or because it creates a hyper-sexualised social environment that endangers women. Both are worthy of concern in my view.
Photoshop has for decades been able to produce fake images indistinguishable from real life.
There's still no case for why fake video is apparently so much worse than fake images. Both achieve the same effects when sent maliciously to a relative or contacts list.
I'll (try to) steelman the case for why video's worse: because people don't realise how easy fake videos are to make. But in 1-2 years... meh... most of society will have it figured out.
When I grew up, my friends dad had a picture taken of his British public school class taken circa 1955. In this photo, which was taken by a slowly rotating panorama camera, a lad started at the beginning of the photo, on the viewers left. Once the camera panned enough that he was out of frame, he sprinted behind the bleachers to the viewers far right, and took his place on the bleachers before the camera panned to take his picture for a second time.
Kindly, if you grew up after this picture was taken, and you believed the guideline “a photo never lies,” you were wrong.
Makes sense and I can see this becoming a global trend if it's not already. But the irony of the news agency taking and posting their photo/video is a facepalm.
EDIT: I can foresee a dark future where surveillance video is monetized for such salacious purpose and it becomes inescapable as the cameras are everywhere.
Could be worse than that. With all of the facial recognition and videoing that's going on by the government, if the police wanted to, they could insert you into any footage committing any crimes they want.
I feel like this is a problem without a solution, or at least without a solution more expensive than the problem:
* I suspect that, like prohibition or DMCA, criminalizing behavior will lead to worse outcomes than accepting deepfake porn sometimes happens, at least if laws are passed targeting this in isolation.
* I suspect that before acceptance, many people will be harmed, on both sides. Humiliated teen girls. Immature hormone-driven teen boys.
* I suspect that we'll see a range of bigger problems such as real scams where, for example, a deepfake child calls their elderly parents and empties their bank account. Or, perhaps, a deepfake general calls someone in a nuclear bunker.
* Deepfakes are at the tip of technological problems which need solving. We have other things, like the cheap and easy availability of technology to make the next Covid-19. The point is we have the technology to make it in a lab, and the price point is going down by an almost Moore's Law like curve.
I don't see a good solution short of completely rethinking democracy and capitalism from scratch. The first time around took the Age of Enlightenment and the Age of Revolution.
It's perhaps time to do that.
Prove me wrong. Seriously. If there are incremental solutions which stop deepfake porn, deepfake scams, personal drone strikes, and super-viruses, please let me know what they are.
I honestly don’t understand the problem with deepfake porn. It’s the same as a fantasy you might have in your head about someone you know. As long as it is not passed off as real, which might be defamatory, I don’t have a moral problem with it. My bigger concern is that a sudden spike in puritanism is going to be used to justify limitations on free speech - which is what a lot of AI regulation is.
> As long as it is not passed off as real, which might be defamatory
That’s basically it. AI is extremely accessible and still relatively unknown. Virtually anyone can create a deepfake photo for free in seconds, post it anywhere and a lot of people will assume it is genuine.
I find deepfakes inherently non-sexy. The whole point of a nude or porn is to reveal something hidden accurately, what’s the point if it is just made up?
Right now you can generate hundreds of realistic fake nudes of celebrity XYZ and no one cares. Celebrity XYZ accidentally leaks a nude photo from their phone and the entire world loses their mind.
I think after the initial moral panic and once the novelty wears off, this issue will fade into obscurity. And currently existing laws should cover sharing such imagery/videos anyway.
What are your thoughts on hentai then? It's all made up after all.
Meta-point is that different people get off on different things, so using seemingly objective language ("the whole point of A is B") to describe something extremely subjective is imo counterproductive.
I think the point is often to harm/exert control over the person depicted, e.g. when boys are making deepfake nudes of girls at their school, as happened in Spain and the USA recently.
Some people don't care about the fakery and are happy with a low quality way to beat off.
Other people don't care because their goal is not to get aroused but to embarrass the persons whose real faces have been faked onto the AI-generated or AI-edited porno footage.
Some people say 'logically we all know it is fake so just ignore it'. This demonstrates something of a lack of imagination, an inability to picture oneself or a beloved family member being put in that position. It also overlooks that different societies have different social standards and that social mobs tend toward the lowest common IQ denominator.
South Korea is a very conservative society, whether you agree with this conservatism or not. Being in a deepfake is easily enough to ruin a person's social or professional reputation, and schoolchildren are especially vulnerable to such pressures. It's not so different from the US problem with sextortion, where teens are tricked into sending nudes to an apparent romantic prospect, and then blackmailed. In both countries, such abuse can end in suicide.
In even more conservative countries like Islamic societies, the victims of such fakes could conceivably suffer criminal penalties if authorities don't believe (or deliberately subvert) their protestations of innocence.
> Being in a deepfake is easily enough to ruin a person's social or professional reputation
This is the problem with Korea, not deepfakes. That they have US-adjacent employment laws, and will fire people based on arbitrary anonymous bile. This shouldn't happen even when the photos are real. The idea that women who have been naked shouldn't work is repulsive.
People on HN have a hard time understanding that the barrier to entry matters. Photoshop has existed for decades but the level of skill needed to make realistic looking nudes in Photoshop is not something your average freak with a fetish possesses. New technology puts that power into the hands of every single person.
It's not your average freak with a fetish that is even the primary issue. The volume will come from average anybody (although mostly men). That's the real problem. Random co-workers in the office that want to deepfake their crush. Your average teenage boy with a decent GPU that wants to deepfake their classmates just to experiment. There have already been several instances of this catching the news in Europe for example, so it's obviously going on globally.
Oh but the average guy isn't that perverted? Yes they are. We know this from the vast amount of porn they consume and of what type they consume.
Seems like revenge porn and sexual harassment should already cover the use case of distribution.
What perverted things someone does with photos in the privacy of their own home seems like it should be their own business to me.
Actually nailing someone for revenge porn or sexual harassment seems like a borderline impossible task for police departments if the person doing it has even a minor degree of anonymity. Enlarging the potential pool of people who can do this and the pool of victims they can target by an order of magnitude by lowering both the skill it took to make convincing fakes before and the time taken to make each fake doesn't seem to me like a good idea.
I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI. Yes, they've been around for well over a decade, but never before has their creation (& thus volume) been so accessible to the average layperson. A simple 10 minutes on CivitAI will tell you all you need to know about how prolific this problem is.
IMO it is only a matter of time before this same issue comes to the forefront in the US (I'm kinda surprised it hasn't already become a bigger topic). I could easily see custom celeb LORAs/emmbeddings and realistic SD/SDXL checkpoints being banned outright for this reason. Yes, it would be an infringement on free speech, but weighed against the interests & safety of the public, the courts might see a move like that as reasonable.
It's ban photoshop all over again.
It's just the new reality. Porn is not even the worst thing. If anything it can make real photos/videos harder to get viral.
But augmentation will be (if not already is) used to make very tiny adjustments, like you make one candidate/celebrity face slightly more symmetrical and the other one less, change their gait just a little bit. These changes are non-perceptible, are not interesting enough to make a scandal, and they can have huge impact on how people perceive other people. I mean, I am not suggesting that it isn't a puppet show without those, but the game is evolving.
> ban photoshop all over again. It's just the new reality.
We didn't solve teens doctoring inappropriate images of their classmates by banning Photoshop or accepting it as the new normal. We solved it by passing laws which criminalise the behaviour and enforcing them to deter the behaviour.
Technology influences society. But plenty of social mores are almost universally respected despite being difficult to enforce. Possible and difficult to enforce doesn't force permissibility.
Since when did photoshopping people's faces onto adult actors become criminalized? I don't recall societies ever "solving" this.
Did we criminalize doctoring a photo and make it a crime?
I thought we have general laws around sexual harassment and revenge porn which criminalize the point of harm.
I bring this up because most of the enforcement difficulty seems to stem from a desire to to control behavior which would otherwise be undetectable.
I believe we already have means of enforcement for students sharing pornography of their classmates.
I was a teen post photoshop and it was just... never an issue. It never occured to me. No one ever mentioned it to me as a possibility. If someone did it would have struck me as weird and as way too much work. Photography class in school (in which we learned to use photoshop) didn't mention any acceptable use guidelines or anything. As far as I know no one in my school ever did anything distasteful with photoshop.
I don't accept the idea that "passing laws and enforcing them to deter behavior" was the cause of the lack of issues, because to be a deterrent us teens would have had to been aware of the laws.
> was a teen post photoshop and it was just... never an issue. It never occured to me
Same. And same.
Drawing inappropriate pictures of your classmates was creepy before. Making Photoshop or AI porn remains so now. Most people won't do it. But there is active social reinforcement that keeps it from becoming "the new reality."
> don't accept the idea that "passing laws and enforcing them to deter behavior" was the cause of the lack of issue
Fair enough.
The jargon was cyberbullying, and there was absolutely legislative activity around horrific examples. But the principal mechanism of action probably wasn't a direct fear of getting caught / punishment, but the prompted discussion reinforcing and extending the norm.
Laws can help, but I'm most concerned with states leveraging this to influence foreign campaigns.
There will be a time when the Trump "grab them by the you know what" style scandal will just be met with sceptism.
Where do you go when even video evidence can't be seen as the truth?
The devices need to cryptographically sign the original video with keys stored in some kind of Secure Enclave
If the video is edited or comes from a source that doesn’t have trusted keys then it’s no better than some random tweet
This is just a certificate of authenticity that one either trusts or not, based on whatever one knows about secure enclaves, cryptography, or the device/person issuing the key/certificate.
I think the reality is that photographs and video are now like text, you trust the source or not.
I can write about Cleopatra's ability to snowboard and play Xbox, or I can make a photo with the caption "Cleopatra doing a 1080", and now I can make a video of it (probably).
I can also give you a certificate/key/blockchain whatever to prove it's "valid".
> devices need to cryptographically sign the original video with keys stored in some kind of Secure Enclave
This is, in practice, verifiable by nobody. It's a neat trick for e.g. judicial purposes. But it's almost worthless in the public domain and absolutely worthless for someone refuting that a video purporting to be of them is fake.
> will be a time when the Trump "grab them by the you know what" style scandal will just be met with sceptism
Politicians since the dawn of time have claimed their scandals were smears. The era of irrefutable recordings and photography was somewhere between short and non-existent.
Photoshop was never built as an easy tool so powerful that even grandma can produce photo-realistic fake images in seconds. The new wave of AI powered editing tools are clearly on a new level.
This is like bringing your howitzer to a gun debate because they are kinda similar.
Generative AI is much easier than photoshop the same way that photoshop was much easier than trying to alter photos any other way. In both cases it is a completely new level.
> new wave of AI powered editing tools are clearly on a new level
For the time being, the easiest-to-use tools are centrally run. That makes monitoring and enforcement easier.
A centrally-managed app will always be easier to use than a local self-service tool. But those local apps will keep trying to converge towards the ease of the hosted apps. Over time I expect the difference will become negligible (i.e. a 1-click install and it actually works)
Photoshopping something convincing has a much bigger bar to entry than (as far as I understand it) generation of deepfakes now. Lots of time/practice invested even for one convincing image.
From what I can see the issue here is that this problem with AI can potentially become much much more widespread because the amount of work required is pretty much none and gets lower as models get better. It sounds like someone using deepfakes to ie extort people can just cast a much wider net by just scraping/dumping masses of images into a generator to create a multitude of convincing pictures of many different people instead of having to focus their time and effort on picking a single target and creating fakes manually.
The above sounds like the opinion of someone who doesn’t have a daughter.
Even trying to be helpful here you're still just centering men's feelings about "their" women.
I don't think of myself as on the e/acc side, but I think that deepfakes are here to stay and we'll just have to get used to them. I don't see any way to put the genie back in the bottle but I also don't think it'll take that long to adapt.
This is giving up a little quickly on watermarking image generation.
I do think it’s a hard problem (especially if we want robustness to compression etc) but amenable to research.
South Korea has a very bad gender relations problem partially stemming from an extremely lopsided gender ratio. In the 90s it reached as high as 115 male children to 100, and now all those extra young men are angry adults. https://theconversation.com/south-koreas-gender-imbalance-is...
Banning Loras/Embedding won't help that..cause you can generate realistic images with only one reference photo these days...
Fair point, technologists will always innovate faster than politicians are able to regulate. Cat's outta the bag now, but that only means the problem will continue to escalate until more drastic measures are deemed necessary.
It's that or we will come to accept some level of societal degradation due to the abuse/misuse of AI tools becoming normalized.
It won’t be a problem if it gets saturated, people bored of things easily
As AI image generation (and soon video generation) gets to be a more saturated space, we will end up in the state that you cannot trust anything you see online, even if you are educated, well-informed, and meticulously research the topic.
It has already been inching that way for a long time, but generative AI definitely is adding new fuel to the fire.
I mean, you already kind of cannot trust anything you see online, it's just that so many people still haven't got the message. Maybe everyone needs a deepfake porno of themselves sent to them so they finally grok that everything online should be by default considered fake.
Ah yes, “waiting for the sexual abuser to get bored” sounds like a good tactic and very comforting for young girls
> I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI.
I don't think they're a concern at all, other than that they will ruin the evidentiary value of photographs. They'll do that whether or not the general public has access to them.
The concern about fake pictures of people naked is stupid. The same process mentioned above happens to naked pictures of people: they lose evidentiary value. General access to the ability to make deepfakes not only makes it so people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem. You can claim it's a fake, and no one can disprove it. This is a reason for celebration.
The standard response for that has been for people to simply claim that it won't matter, and that people will destroy the life of some young innocent girl if a mean boy spreads fake pictures of her naked. The answer is that people who harass a young girl are guilty of harassment, and should be jailed. You shouldn't be harassing people even if naked pictures of them are real, which they often are. And if you'd attack a person over pictures that you could fake up yourself in five minutes, you're either a) a moron or b) somebody using the photos as an excuse, and it would be a benefit for society for you to be locked away from it.
I suspect a lot of people really worried about deepfake porn deep down are worried about their ability to properly identify and harass the harlots if people won't believe the pictures that you send to their family and friends anymore. From my perspective, they're literally fighting to preserve revenge porn.
> The concern about fake pictures of people naked is stupid. The same process mentioned above happens to naked pictures of people: they lose evidentiary value.
This is too narrow: it presumes rational contexts where evidence will be evaluated critically, and it assumes a mild, uncreative attacker but harassers are good at exploiting social norms. The damage caused by this kind of harassment isn’t seeing yourself nude as much as wondering or knowing what other people will think, and there’s no way to undo that. Even if people _say_ they believe that the pictures are fake, the victim may be wondering whether that’s really true for the rest of their life.
The other thing to think about is that we’re really just getting started. Most of the deepfakes which people outside of the scene have seen are fairly mainstream porn nudes: there’s a huge control issue but in general there are worse things than people thinking you have a star’s body. We’re going to see those worse things soon: it won’t be a porn fantasy but things designed to degrade and more targeted - if the target had body issues, exaggerating them; punishing the person who rejected someone by faking them looking not like a porn star but less attractive and having degrading sex with someone unpopular, knowing that a bunch of people are going to think “well, he would say that”; make some teacher’s life hell by faking them having sex with a student; etc.
Because the attacks are being directed by humans, they’re going to adjust and find what works. Anything which forces an official investigation is going to be especially bad because we can’t just ignore things like child abuse so it’s going to be like SWATing where there’s an ugly period where the wrong teenager can get an incredibly high-leverage response with a single message.
I think the problem is that people aren't necessarily logical or rational about such things. I know someone (family friend) who was the victim of similar harassment with (clearly) photoshopped images being sent anonymously to her work, family, friends on social media. The fact that they were fake didn't diminish the humiliation or stress she went through and I don't think she ever really recovered from the trauma of having some anonymous asshole harassing and stalking her and her family online. It eventually stopped but she was afraid of it starting again for a long time. "It's fake" isn't much of a comfort to someone being harassed and being afraid of harassment escalating in one way or another. And it getting easier and easier for crazy people to do this with less and less effort doesn't help the issue at all regardless of what laws are passed. Finding better ways to deal with these people is a good thing but enabling them to make it easier isn't, in my opinion.
I think for a lot of people it's easy to die on the hill of technological progress and freedom at any cost when the consequences haven't happened to you or a loved one.
We definitely need to crack down more on harassment and stalking and not worry so much about what form that harassment and stalking takes. It's insane that you can have someone all over you, sending you unwanted messages, following you, or generally targeting you with misery that falls short of physical assault, and the police won't do anything about it until you're physically hurt. It shouldn't matter if the harassment is phone calls at 4AM or deepfakes. Harassment is harassment.
I agree. But the internet age feels like it unfortunately has made this much more difficult than it should be. People over share on social media and make themselves too visible because they aren't aware of the consequences- or don't assume there are people out there sociopathic enough to harass someone to that extent. The pool of potential suspects when they're acting anonymously has grown from whoever is in your local community to whoever has an internet connection in the world. Not uncommon to hear of such harassment, blackmailing cases and such where the perpetrator doesn't even live in your own country where police could do anything about it even if they had their name and real evidence.
> ruin the evidentiary value of photographs
This is going to be a monumental issue when the fact that we can't trust any photographs fully takes hold. Maybe watermarking will help with that. More likely people will develop tools to circumvent watermarking technology (either on the imprinting side, or on the verification side).
> people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem.
This may be a comforting way to think about it, but it won't work in all cases. Consider instead of revenge porn, the case of CSAM – that will never be a solved problem until 100% of it is wiped from the face of the earth and it is never produced again. Making it less believable because there is so much AI-generated equivalents out there is a not a solution to the problem, it exacerbates the problem.
> when the fact that we can't trust any photographs fully takes hold
Full faith in photography was never a thing. If anything, the advent of photography came alongside yellow journalism. Trust is an institution. You can't build institution on solely technology.
Good context. But the fact is, a great many people – perhaps even the majority of people – are quite used to looking at a photograph and fully trusting that it depicts something which actually happened. As they say, seeing is believing.
Even with CGI being what it has been since the 70s, it's still appropriate to apply Occam's Razor in many cases, and to simply say that because it looks real, it probably is.
> Yes, it would be an infringement on free speech
Freedom of speech is mainly so that we can criticize the government. How does deepfake porn or impersonation have anything to do with speech?
I think freedom of speech is mainly about freedom of thought and just plain freedom, and the ability to criticize the government is a side benefit. And that it should also include the freedom to augment our imaginations with the prosthetics of our choice.
But people have always been free to talk about the weather, the killer feature is being able to talk about your leaders.
Cannot wait to augment the imaginations of his coworkers with a realistic video of this guy fucking a goat.
> Freedom of speech is mainly so that we can criticize the government
Freedom of speech is a bigger concept than the First Amendment. Technically curtailing one's ability to make AI porn of your colleagues is a restriction on speech. That said, so is every spam filter, fraud statute and retaliation against a declaration of war. A right doesn't have to be absolute to be good (or vice versa).
> spam filter
Your other examples, yes. This one, not at all.
Freedom of speech is against government intervention, not against private agreements.
Rather in fact an absolute form of freedom of speech would protect against the government from stopping you from forwarding all your correspondence to someone else, and would protect against the government stopping that someone else from removing the spam and sending the rest back to you. Non-government run spam filters aren't just not offending freedom of speech, they are protected by it.
> Freedom of speech is against government intervention, not against private agreements
You're conflating the First Amendment and freedom of speech [1]. If you're in my house and I prohibit you from expressing certain views, that is fine by the First Amendment but not in the spirit of free speech.
[1] https://en.wikipedia.org/wiki/Freedom_of_speech
No I'm not. You're conflating freedom of speech with a duty to listen. Freedom of speech entails a freedom to send spam. It doesn't entail a duty to receive spam, to read spam, to not have someone else filter your communications for spam, etc.
And as I argued above, since every step in having someone filter your communications for spam is simply speech, it protects spam filters.
> No I'm not
You restricted it to "government intervention, not...private agreements." The freedom of speech extends to the private sphere.
> You're conflating freedom of speech with a duty to listen
These are inextricably linked in the classical form. In the Athenian tradition, there was a duty to listen [1]. (Though even then practicality made it far from absolute.) That traces through to the Protestation of 1621 [2] and the Founders' debates around the First Amendment (namely, between Madison and Hamilton).
Practically, of course, no duty is absolute. But a society where everyone screams into a void isn't adhering to the principles of free speech and expression. (I'll note that alongside the duty to listen is an implied duty on the part of the speaker to speak truly [3]. Spam is the epitome of the cessation of the latter revoking the former.)
[1] https://www.stoa.org/demos/article_democracy_overview@page=a...
[2] https://en.wikipedia.org/wiki/Protestation_of_1621
[3] https://en.wikipedia.org/wiki/Parrhesia
I think you're confusing something along the lines of "parliamentary privilege" for freedom of speech. Various forms of government have imposed as part of their decision making process a duty to listen to various forms of official speech. That's not a component of freedom of speech though, rather it's a component of that specific form of governments process.
I'm not sure where you find a duty to listen in Athens, my best guess is you are pointing to the duty to attend the assembly, which fits this form of "parliamentary privilege" rather than a broader duty to listen. To contrast, by my understanding political commentary in the form of plays and the like were quite common in Athenian culture, and I'm fairly sure there was absolutely no duty to pay attention to those. I'd also note that even with the assembly "the crowd might raise a clamor and refuse to listen to a speaker advocate an unpopular proposal" (your source) which suggests little duty to actively listen even there.
The protestation of 1621 is directly related to parliamentary privilege (and why I labelled this the way I did, though I don't think the label fits perfectly, since it applies to many smaller government meetings that parliament).
I can't say I'm familiar with the first amendment debate you are discussing.
> You restricted it to "government intervention, not...private agreements."
I don't think this is the cause of our disagreement, but I'll grant you that that phrase was slightly inartful. E.g. there are authorities other than governments, like a local gang, who could impinge on freedom of speech with threats of retaliation, and to the extent that private agreements are enforced by an authority it's possible for them to do so.
All speech is information but not all information is speech
> information != speech
Information intentionally communicated by a person to another is speech by the strictest classical definition. (I guess technically the creation of the AI porn isn't speech in the way a falling tree in a deserted forest doesn't make a perceived sound. But it's a schoolyard riddle for a reason.)
Freedom of speech is there to protect objectionable/offensive speech, whether it is directed towards the government or otherwise. Objectionable speech is frequently in the crosshairs of pro-censorship folks, and many people think of deepfakes as objectionable – reasonably so.
I feel as though I'm the only one who doesn't care about this.
Curious to know if 20-30 years ago, as graphic design tools got better, were people freaking out about being able to superimpose another person's face on a the body of p*rnstar?
Superimposing people's faces on to porn star's bodies happened/s, but does anyone care really even notice or care about it now days? For the most part, of course not.
So why should we care about porn videos? What makes them so different? The whole concern smells like typical media hype (i.e. yay - we get to write about AI + Porn - instant clickbait) that nobody will be able to stop and nobody will even care about in 1-2 years: e.g. "oh, yah, another deepfake of <(politician|actress|colleague^)> in a porn film - how incredibly creative".
^ Incidentally, this was made light of in the British version of The Office where a modified image of the boss, David Brent, was sent around the office. If it's joked about harmlessly on mainstream TV, it's probably not a huge concern.
It's more problematic than photoshopping a person's face was decades ago because we didn't have pervasive online social media decades ago. Computers weren't in your pocket, and an offending image couldn't be beamed to everyone you know instantly.
> If it's joked about harmlessly on mainstream TV, it's probably not a huge concern.
There are plenty of mainstream TV jokes about horrendous things. Our ability to joke about something is a poor way to gauge how concerned we should be.
> an offending image couldn't be beamed to everyone you know instantly
Is your concern that recipients would think it's real? Or is it that people will get an email/chat message containing NSFW material? (because the latter happens now, and is nothing more than background noise at most). What is your actual, specific concern? (genuinely, I have no idea, but want to try to understand - to me it feels like panic about a problem that isn't actually a real problem).
I'm not sure what my personal concerns are specifically, so I don't have a good answer to that question.
But I think there's a difference between a crud photoshop picture that a person has on their PC, and a (more or less) convincing deepfake video of a person being broadcast.
I think you're likely right that there is an element of panic. I don't know a lot about Korean culture, but I imagine the concern is that deepfakes can have real-life negative impact on people's lives, either because people think it's real or because it creates a hyper-sexualised social environment that endangers women. Both are worthy of concern in my view.
Photoshop has for decades been able to produce fake images indistinguishable from real life.
There's still no case for why fake video is apparently so much worse than fake images. Both achieve the same effects when sent maliciously to a relative or contacts list.
I'll (try to) steelman the case for why video's worse: because people don't realise how easy fake videos are to make. But in 1-2 years... meh... most of society will have it figured out.
But who gives a fuck about people who's lives are ruined in the interim 1-2 years, right?
When I grew up "a photo never lies" was a rule you could rely on. Yes really!!
Photoshop and other advances long since killed that certainty, and now we have a healthy distrust of pictures we're not sure where they came from.
I expect videos will now walk a similar path, and life will go on with no major disasters.
Doctored photos have been around for most of the last century.
https://en.wikipedia.org/wiki/Censorship_of_images_in_the_So...
The appearance of doctored photographs coincides with the appearance of photography.
When I grew up, my friends dad had a picture taken of his British public school class taken circa 1955. In this photo, which was taken by a slowly rotating panorama camera, a lad started at the beginning of the photo, on the viewers left. Once the camera panned enough that he was out of frame, he sprinted behind the bleachers to the viewers far right, and took his place on the bleachers before the camera panned to take his picture for a second time.
Kindly, if you grew up after this picture was taken, and you believed the guideline “a photo never lies,” you were wrong.
> When I grew up "a photo never lies" was a rule you could rely on.
It was a rule you thought you could rely on.
I would refer you to an interesting book called “The Commissar Vanishes: The Falsification of Photographs and Art in Stalin’s Russia”.
https://en.wikipedia.org/wiki/The_Commissar_Vanishes
Makes sense and I can see this becoming a global trend if it's not already. But the irony of the news agency taking and posting their photo/video is a facepalm.
EDIT: I can foresee a dark future where surveillance video is monetized for such salacious purpose and it becomes inescapable as the cameras are everywhere.
That is the dark present in South Korea as they have a huge problem with cameras hidden in hotel rooms and even toilets :-/
Could be worse than that. With all of the facial recognition and videoing that's going on by the government, if the police wanted to, they could insert you into any footage committing any crimes they want.
I feel like this is a problem without a solution, or at least without a solution more expensive than the problem:
* I suspect that, like prohibition or DMCA, criminalizing behavior will lead to worse outcomes than accepting deepfake porn sometimes happens, at least if laws are passed targeting this in isolation.
* I suspect that before acceptance, many people will be harmed, on both sides. Humiliated teen girls. Immature hormone-driven teen boys.
* I suspect that we'll see a range of bigger problems such as real scams where, for example, a deepfake child calls their elderly parents and empties their bank account. Or, perhaps, a deepfake general calls someone in a nuclear bunker.
* Deepfakes are at the tip of technological problems which need solving. We have other things, like the cheap and easy availability of technology to make the next Covid-19. The point is we have the technology to make it in a lab, and the price point is going down by an almost Moore's Law like curve.
I don't see a good solution short of completely rethinking democracy and capitalism from scratch. The first time around took the Age of Enlightenment and the Age of Revolution.
It's perhaps time to do that.
Prove me wrong. Seriously. If there are incremental solutions which stop deepfake porn, deepfake scams, personal drone strikes, and super-viruses, please let me know what they are.
I honestly don’t understand the problem with deepfake porn. It’s the same as a fantasy you might have in your head about someone you know. As long as it is not passed off as real, which might be defamatory, I don’t have a moral problem with it. My bigger concern is that a sudden spike in puritanism is going to be used to justify limitations on free speech - which is what a lot of AI regulation is.
> As long as it is not passed off as real, which might be defamatory
That’s basically it. AI is extremely accessible and still relatively unknown. Virtually anyone can create a deepfake photo for free in seconds, post it anywhere and a lot of people will assume it is genuine.
I find deepfakes inherently non-sexy. The whole point of a nude or porn is to reveal something hidden accurately, what’s the point if it is just made up?
Right now you can generate hundreds of realistic fake nudes of celebrity XYZ and no one cares. Celebrity XYZ accidentally leaks a nude photo from their phone and the entire world loses their mind.
I think after the initial moral panic and once the novelty wears off, this issue will fade into obscurity. And currently existing laws should cover sharing such imagery/videos anyway.
What are your thoughts on hentai then? It's all made up after all.
Meta-point is that different people get off on different things, so using seemingly objective language ("the whole point of A is B") to describe something extremely subjective is imo counterproductive.
[dead]
I think the point is often to harm/exert control over the person depicted, e.g. when boys are making deepfake nudes of girls at their school, as happened in Spain and the USA recently.
Some people don't care about the fakery and are happy with a low quality way to beat off.
Other people don't care because their goal is not to get aroused but to embarrass the persons whose real faces have been faked onto the AI-generated or AI-edited porno footage.
Some people say 'logically we all know it is fake so just ignore it'. This demonstrates something of a lack of imagination, an inability to picture oneself or a beloved family member being put in that position. It also overlooks that different societies have different social standards and that social mobs tend toward the lowest common IQ denominator.
South Korea is a very conservative society, whether you agree with this conservatism or not. Being in a deepfake is easily enough to ruin a person's social or professional reputation, and schoolchildren are especially vulnerable to such pressures. It's not so different from the US problem with sextortion, where teens are tricked into sending nudes to an apparent romantic prospect, and then blackmailed. In both countries, such abuse can end in suicide.
In even more conservative countries like Islamic societies, the victims of such fakes could conceivably suffer criminal penalties if authorities don't believe (or deliberately subvert) their protestations of innocence.
> Being in a deepfake is easily enough to ruin a person's social or professional reputation
This is the problem with Korea, not deepfakes. That they have US-adjacent employment laws, and will fire people based on arbitrary anonymous bile. This shouldn't happen even when the photos are real. The idea that women who have been naked shouldn't work is repulsive.