mar24
It seems like only months ago deepfakes were still just a curiosity. Now, deepfakes are a real and present danger. And in an election year, the influence of AI-manipulated content could be disastrous.
During a recent Washington Post Live event, Anne Neuberger, deputy national security adviser for cyber and emerging technologies at the White House, commented on the rising risk of deepfakes. Incidents have already occurred, such as the recent fake-Biden robocall meant to discourage voters ahead of the New Hampshire primary.
What are the potential consequences of deepfake attacks in an election year? And could watermarking make a difference in mitigating deepfake attacks?
Ultra-real deepfakes are here
How realistic are deepfakes now? Consider the case of the clerk who fell for a deepfake while working for the Hong Kong branch of a multinational company. In January 2024, the clerk transferred HK$200 million (USD 25.58M) of the firm’s money to fraudsters after being tricked into joining a video conference where all the other participants were AI-generated deepfakes.
Acting senior police superintendent Baron Chan said, “I believe the fraudster downloaded videos in advance and then used artificial intelligence to add fake voices to use in the video conference.”
In another case, using a technique called audio-jacking, cyber researchers were able to modify the details of a live financial conversation occurring between two people with the assistance of generative AI. In this staged exchange, money was diverted to a fake adversarial account without the speakers realizing their call was compromised.
Meanwhile, AI itself can be fooled with prompt injection attacks that manipulate large language models (LLMs). This can result in tricking an LLM into performing unintended actions, circumventing content policies to generate misleading or harmful responses, or revealing sensitive information.
Can watermarking save the day?
AI watermarking works by embedding a unique signal into an artificial intelligence model’s output. This signal can be an image or text, and it’s intended to identify the content as AI-generated.
Some types of watermarks include:
- Visible watermarks: Can be seen by the human eye, such as logos, images, copyrighted text and personal signatures.
- Invisible watermarks: Cannot be seen and may utilize stenographic techniques and watermark extraction algorithms.
- Public watermarks: Not secure and can be modified by anyone using certain algorithms.
- Frequency and spatial watermarks: A form of domain watermarking that defines images as pixels. This provides improved watermarking quality and imperceptibility.
During the Washington Post event, Neuberger touched upon watermarking as a way to mitigate risks posed by deepfakes. She mentioned that watermarking could be effective for platforms that comply with mandates like the White House’s AI Executive Order. For example, on Facebook, any AI-generated content might display an icon that clearly states the content was generated with artificial intelligence.
While watermarking would be useful on compliant platforms, “there will always be platforms… that are not interested in being responsible. And for that, researchers and companies are looking at and need to do more to build the technology to identify what are deepfakes,” said Neuberger.
Election year impact
With approximately 4.2 billion people expected to vote in elections around the world in 2024, AI creators, scholars and politicians said in interviews that standards on the watermarking of AI-generated content must be established quickly. Otherwise, AI-generated fake content could have an impact on election results.
While standards would be welcome, nefarious actors and extremist or nuisance groups certainly won’t be watermarking their deepfakes. If anything, they will develop ways to hide or remove watermarks from their malicious content.
Perhaps the solution to AI deepfakes can be found in the cause. Maybe AI-driven deepfake detectors will be deployed by social media platforms. Or maybe, someday, you will be able to download an app that detects deepfakes for you.
CONTINUE READING
https://securityintelligence.com/news/watermarking-2024-election-deepfake/
fv24
Digital watermarks may be the most hyped solution to many of the social problems posed by generative AI.
https://www.theverge.com/2024/2/13/24067991/watermark-generative-ai-deepfake-copyright
dez23
The crux of the problem is that image-generating tools like DALL-E 2 and Midjourney make it easy for anyone to create realistic-but-fake photos of events that never happened, and similar tools exist for video. While the major generative-AI platforms have protocols to prevent people from creating fake photos or videos of real people, such as politicians, plenty of hackers delight in “jailbreaking” these systems and finding ways around the safety checks. And less-reputable platforms have fewer safeguards.
Against this backdrop, a few big media organizations are making a push to use the C2PA’s content credentials system to allow Internet users to check the manifests that accompany validated images and videos. Images that have been authenticated by the C2PA system can include a little “cr” icon in the corner; users can click on it to see whatever information is available for that image—when and how the image was created, who first published it, what tools they used to alter it, how it was altered, and so on. However, viewers will see that information only if they’re using a social-media platform or application that can read and display content-credential data.
https://spectrum.ieee.org/deepfakes-election
Meta requires political advertisers to mark when deepfakes used
While much work at Pindrop research has gone into developing tools for accurate deepfake detection, we believe that more can be done to protect users from malicious or misleading use of deepfakes. One such path is to use digital audio watermarking to aid the distinction between live and synthetically generated speech. The vision here is that all synthetically generated speech is watermarked but like any other opportunity, it does not come without its own challenges. Most watermarking technology has been applied to images and it is already used for AI generated images1.
- Michigan lawmakers want to require disclaimers on political ‘deepfakes’ and campaign ads that use artificial intelligence
- Bills would establish fines and criminal penalties for creators or distributors who fail to include disclaimers
- Legislation is before the full Michigan House after Tuesday approval by the House Elections Committee
Some viral TikTok videos may soon show a new type of label: that it’s made by AI.
The ByteDance-owned app is developing a tool for content creators to disclose they used generative artificial intelligence in making their videos, according to a person with direct knowledge of the efforts. The move comes as people increasingly turn to AI-generated videos for creative expression, which has sparked copyright battles as well as concerns about misinformation.
https://www.theinformation.com/articles/tiktok-is-developing-ai-generated-video-disclosures-as-deepfakes-rise
ab23
In Europe, an AI-generated “interview” with Formula 1 racing legend Michael Schumacher highlights the danger (Schumacher has not been seen in public since his brain injury in a skiing accident in December 2013.)
The Schumacher family will now take legal action against Die Actuelle, a German tabloid magazine that failed to mark the interview as an AI creation. The magazine fired the editor responsible, but the damage was already done — and out in public.
https://www.ravepubs.com/video-deepfakes-theres-a-new-sheriff-in-town/Can YOU spot a deepfake from a real person? World's first 'certified' deepfake warns viewers not to trust everything they see online
- An AI studio has created the world's first cryptographically signed deepfake
- Its tamper-evident seal declares that the video contains AI-generated content
- It is hoped this will eliminate confusion as to where online videos originate
- https://www.dailymail.co.uk/sciencetech/article-11940965/Worlds-certified-deepfake-warns-viewers-not-trust-online.html
Does a 'fake news' label help audiences identify false information?
Para evitar confusões, China exige que deepfakes tenham marca d’água
Building trust in content must allow for anonymity and redaction. A human rights documenter in Ukraine cannot be required to publicly share their name and exact location as a prerequisite for use. They must also be able to blur faces or create redacted versions of their media. Immutability and inability to edit are recipes for disaster. Lessons from platform policies around ‘real names’ tell us many people — for example, survivors of domestic violence — face similar circumstances that we should learn from. We support how the C2PA specifications focus on protecting privacy and don’t require identity disclosures, but we should be cautious of how implementation may compromise this.
Platforms must likewise consider how they integrate these types of signals. Facebook, TikTok, YouTube and other platforms currently offer no ready consumer-oriented options to help viewers understand if videos or images are being decontextualized and misconstrued. They provide neither reverse image and video search that shows previous versions of media existing online nor signaling based on the C2PA specifications. However, turn to #Ukraine, and we see videos from years ago or different locations claimed as current, and audio remixed over different videos on TikTok. Increased transparency about how media is created and where it comes from is a critical aid to media literacy skills
https://thehill.com/opinion/technology/3513054-to-battle-deepfakes-our-technologies-must-lead-us-to-the-truth/
One promising approach involves tracking a video’s provenance, “a record of everything that happened from the point that the light hit the camera to when it shows up on your display,” explained James Tompkin, a visual computing researcher at Brown.
But problems persist. “You need to secure all the parts along the chain to maintain provenance, and you also need buy-in,” Tompkin said. “We’re already in a situation where this isn’t the standard, or even required, on any media distribution system.”
And beyond simply ignoring provenance standards, wily adversaries could manipulate the provenance systems, which are themselves vulnerable to cyberattacks. “If you can break the security, you can fake the provenance,” Tompkin said. “And there’s never been a security system in the world that’s never been broken into at some point.”
Given these issues, a single silver bullet for deepfakes appears unlikely. Instead, each strategy at our disposal must be just one of a “toolbelt of techniques we can apply,” Tompkin said. https://brownpoliticalreview.org/2021/11/hunters-laptop-deepfakes-and-the-arbitration-of-truth/
A team of researchers has put together a new initiative with an available open-source code to help better detect deepfakes that have been edited to remove watermarks with the goal of avoiding the spread of misinformation. Inpainting — also known as “Content-Aware Fill” for Photoshop users — is a method that uses machine-learning models to reconstruct missing pieces of an image or to remove unwanted objects. Although it is generally used as a tool among creatives to “clean up” the image for a more fine-tuned result, this technology can also used for malicious intentions, such as removing watermarks, reconstructing the reality by removing people or certain objects in the photos, adding false information, and more. This type of technology has greatly developed in recent years, with the notable example of NVIDIA’s AI-powered “Content-Aware Fill”, which goes a step further than Photoshop’s already advanced tools. Manipulating images with malicious intent can cause not only profit loss from image theft by removing watermarks or other visual copyright identifying factors, but it can also lead to the spread misinformation in the case of its ability to remove a person from a crime scene photo, scam people or businesses, even destabilize politics in a case earlier reported by PetaPixel. https://petapixel.com/2021/06/16/new-markpainting-tech-blocks-watermark-removal-fights-deepfakes/
Some major firms, as well as civil society and media organisations, have teamed up under the Content Authenticity Initiative to create a standard for digital content provenance. This is a good start, but many more firms should get involved and civil society should to apply to join, making sure global perspectives are reflected. We need to be careful, for example, that provenance tools cannot be abused to detect brave citizen journalists who document human rights abuses. We can do more in the EU as well. Positively, the European Commission addressed the threat in the recently-unveiled proposal for an AI Act proposed in April.The draft obliges users of AI systems that generate or manipulate visual content to disclose when image, video or audio content has been manipulated through automated means. https://euobserver.com/opinion/151935
From Deepfakes to TikTok Filters: How Do You Label AI Content? In a future full of media manipulated by artificial intelligence, we will need methods like this to indicate what is real and what has been faked with AI. And this won’t just be a question of ethics. The EU may require that users be told when AI has generated or manipulated something they are viewing. (...) Watermarks are icons or filters that visually cover content to advertise or cite the tool used to create it. They are also often automatically applied by commercial AI manipulation tools, such as impressions.app, as a means of promotion. This offers insight into the fact that the media was edited and a specific tool used to do so.
Mar21
Os termos de utilização da Avatarify dizem que não ela pode ser usada de forma obscena ou odiosa, mas não tem sistemas para o verificar. Mais ainda, a appnão limita o que podes pôr as pessoas a dizer ou a fazer. “Não limitamos porque procuramos os motivos de utilização — e elas são maioritariamente entretenimento”, Aliev diz. “Se formos demasiado preventivos podemos perder alguma coisa.”
Hany Farid, um professor de ciência da computação na Universidade da Califórinia em Berkeley, diz que já ouviu este discurso de “mexer rápido e quebrar coisas” antes, vindo de empresas como o Facebook. “Se a tua tecnologia vai levar ao perigo — e é razoável prever esse perigo — penso que tens de ser responsabilizado”, diz.
Que cautelas podem mitigar o perigo? Ben-Zion Benkhin, CEO da Wombo, diz que os criadores das apps de deepfake devem ser “muito cuidadosos” ao dar às pessoas o poder de controlar o que sai da boca de outras pessoas. A sua app é limitada a animações deepfake de uma colecção de videoclipes, com movimentos de cabeça e dos lábios gravados por actores. “Não podes escolher algo que seja super ofensivo ou que possa ser mal interpretado”, diz Benkhin.
A MyHeritage não deixa adicionar movimentos dos lábios ou vozes nos vídeos — ainda que tenha quebrado a sua própria regra ao usar a tecnologia para produzir uma publicidade com um Abraham Lincoln falso.
Também há problemas de privacidade relativos à partilha de caras com uma app, uma lição que aprendemos depois da controvérsia em 2019 com a FaceApp, um serviço russo que requeria acesso às nossas fotos para usar IA e envelhecer rostos. A Avatarify (também russa) diz que nunca recebe as fotos dos utilizadores porque funciona integralmente no telemóvel — mas a Wombo e MyHeritage usam as tuas fotos e processam-nas na cloud.
As lojas de aplicações que distribuem esta tecnologia podiam estar a fazer muito mais para definir standards. A Apple removeu a Avatarify da App Store chinesa, explicando que violava uma lei não especificada. Mas a app está disponível nos Estados Unidos e no resto do mundo — e a Apple diz que não tem regras específicas para as apps de deepfakes, além das proibições gerais de conteúdo difamatório, discriminatório ou de má-fé.
Logótipos ou marcas de água que tornem claro quando estamos a olhar para um deepfake poderiam também ajudar. Estes três serviços incluem marcas de água visíveis, ainda que a Avatarify a remova com uma subscrição do serviço premium, que custa 2,50 dólares por semana.
Melhor ainda seria que estas marcas de água fossem mais difíceis de eliminar, ajudando a identificar deepfakes. Os três criadores dizem que acham essa uma boa ideia — mas que precisam que alguém desenvolva os standards.
As redes sociais teriam também um papel determinante em assegurar que os deepfakes não são usadas para prejuízo de alguém. As suas políticas geralmente tratam os deepfakes como qualquer outro conteúdo que desinforme ou possa levar pessoas a prejudicarem-se: a política do Facebook e do Instagram diz que devem ser removidos conteúdos “manipulados”, ainda que tenha uma excepção para paródias. A política do TikTok diz também que devem ser removidas “falsificações digitais” que desinformem e prejudiquem o sujeito do vídeo ou a sociedade, assim como informação sanitária errada. As políticas de “prática enganosa” do Youtube proíbem conteúdo tecnicamente manipulado que engane e possa significar um risco sério.
Público. Agora, qualquer pessoa com um iPhone pode fazer um deepfake. E não estamos prontos para o que vai acontecer Geoffrey A. Fowler