Imran Khan—Pakistan’s Jailed Ex-Leader—Uses AI Deepfake To Address Online Election Rally
Former Pakistani Prime Minister Imran Khan, who is serving a three-year prison sentence, used AI-generated voice and video in a clip to campaign for his party ahead of the country’s upcoming general election, spotlighting the potential use of AI and deepfakes as major polls are scheduled in the U.S., India, European Union, Russia, Taiwan and beyond in 2024.
Khan’s party Pakistan Tehreek-e-Insaf (PTI) held an online campaign rally featured an AI-generated video of the former leader addressing his supporters and urging them to vote in large numbers for his party.
, which is about four minutes long, features an AI-generated voice resembling Khan’s delivering the speech and briefly includes an AI-generated deep fake video of the jailed leader sitting in front of a Pakistani flag.
In the video, Khan tells his supporters that his party has been barred from holding public rallies and talks about his party members being targeted, kidnapped and harassed.
PTI social media lead Jibran Ilyas said the content of the video is based on notes provided by Khan from prison, adding that the AI-generated voice feels like a “65-70%” match of the former Prime Minister.
The nearly five-hour
livestream of the virtual rally has clocked up more than 1.5 million views on YouTube, although Khan’s AI-generated clip is only around four and half minutes long.
https://www.forbes.com/sites/siladityaray/2023/12/18/imran-khan-pakistans-jailed-ex-leader-uses-ai-deepfake-to-address-online-election-rally/?sh=6e5c48e55903
DEZ23
As two billion people around the world go to voting stations next year in fifty countries, there is a crucial question: how can we build resilience into our democracy in an era of audiovisual manipulation? When AI can blur the lines between reality and fiction with increasing credibility and ease, discerning truth from falsehood becomes not just a technological battle but a fight to uphold democracy.
From conversations with journalists, activists, technologists and other communities impacted by generative AI and deepfakes, I have learnt that the effects of synthetic media on democracy are a mix of new, old, and borrowed challenges. https://www.cfr.org/blog/protect-democracy-deepfake-era-we-need-bring-voices-those-defending-it-frontlines
dez23
Deepfakes for $24 a month — how AI is disrupting Bangladesh’s election
Ahead of South Asian nation going to the polls in January, AI-generated disinformation has become a growing problem
Policymakers around the world are worrying over how AI-generated disinformation can be harnessed to try to mislead voters and inflame divisions ahead of several big elections next year.
In one country it is already happening: Bangladesh.
The South Asian nation of 170mn people is heading to the polls in early January, a contest marked by a bitter and polarising power struggle between incumbent Prime Minister Sheikh Hasina and her rivals, the opposition Bangladesh Nationalist party.
Pro-government news outlets and influencers in Bangladesh have in recent months promoted AI-generated disinformation created with cheap tools offered by artificial intelligence start-ups.
In one purported news clip, an AI-generated anchor lambasts the US, a country that Sheikh Hasina’s government has criticised ahead of the polls. A separate deepfake video, which has since been removed, showed an opposition leader equivocating over support for Gazans, a potentially ruinous position in the Muslim-majority country with strong public sympathy for Palestinians.
Public pressure is rising on tech platforms to crack down on misleading AI content ahead of several big elections expected in 2024, including in the US, the UK, India and Indonesia. In response, Google and Meta have recently announced policies to start requiring campaigns to disclose if political adverts have been digitally altered.
But the examples from Bangladesh show not only how these AI tools can be exploited in elections but also the difficulty in controlling their use in smaller markets that risk being overlooked by American tech companies.
Miraj Ahmed Chowdhury, the managing director of Bangladesh-based media research firm Digitally Right, said that while AI-generated disinformation was “still at an experimentation level” — with most created using conventional photo or video editing platforms — it showed how it could take off.
“When they have technologies and tools like AI, which allow them to produce misinformation and disinformation at a mass scale, then you can imagine how big that threat is,” he said, adding that “a platform’s attention to a certain jurisdiction depends on how important it is as a market”.
Global focus on the ability to use AI to create misleading or false political content has risen over the past year with the proliferation of powerful tools such as OpenAI’s ChatGPT and AI video generators.
Earlier this year, the US Republican National Committee released an attack ad using AI-generated images to depict a dystopian future under President Joe Biden. And YouTube suspended several accounts in Venezuela using AI-generated news anchors to promote disinformation favourable to President Nicolás Maduro’s regime.
In Bangladesh, the disinformation fuels a tense political climate ahead of polls in early January. Sheikh Hasina has cracked down on the opposition. Thousands of leaders and activists have been arrested in what critics warn amounts to an attempt to rig polls in her favour, prompting the US to publicly pressure her government to ensure free and fair elections.
In one video posted on X in September by BD Politico, an online news outlet, a news anchor for “World News” presented a studio segment — interspersed with images of rioting — in which he accused US diplomats of interfering in Bangladeshi elections and blamed them for political violence.
The video was made using HeyGen, a Los Angeles-based AI video generator that allows customers to create clips fronted by AI avatars for as little as $24 a month. The same anchor, named “Edward”, can be seen in HeyGen’s promotional content as one of several avatars — which are themselves generated from real actors — available to the platform’s users. X, BD Politico and HeyGen did not respond to requests for comment.
Screenshot of an anti-opposition disinformation video posted on Facebook featuring an deepfake of exiled BNP leader Tarique Rahman
Other examples include anti-opposition deepfake videos posted on Meta’s Facebook, including one that falsely purports to be of exiled BNP leader Tarique Rahman suggesting the party “keep quiet” about Gaza to not displease the US. The Tech Global Institute, a think-tank, and media non-profit Witness both concluded the fake video was likely AI-generated.
AKM Wahiduzzaman, a BNP official, said that his party asked Meta to remove such content but “most of the time they don’t bother to reply”. Meta removed the video after being contacted by the Financial Times for comment.
In another deepfake video, created by utilising Tel Aviv-based AI video platform D-ID, the BNP’s youth wing leader Rashed Iqbal Khan is shown lying about his age in what the Tech Global Institute said was an effort to discredit him. D-ID did not respond to a request for comment.
A primary challenge in identifying such disinformation was the lack of reliable AI-detection tools, said Sabhanaz Rashid Diya, a Tech Global Institute founder and former Meta executive, with off-the-shelf products particularly ineffective at identifying non-English language content.
She added that the solutions proposed by large tech platforms, which have focused on regulating AI in political adverts, will have limited effect in countries such as Bangladesh where ads are a smaller part of political communication.
“The solutions that are coming out to address this onslaught of AI misinformation are very western-centric.” Tech platforms “are not taking this as seriously in other parts of the world”.
The problem is exacerbated by the lack of regulation or its selective enforcement by authorities. Bangladesh’s Cyber Security Act, for example, has been criticised for giving the government draconian powers to crack down on dissent online.
Bangladesh’s internet regulator did not respond to a request for comment about what it is doing to control online misinformation.
Recommended
The Big Read
Will Bangladesh come to regret its dash for gas?
A greater threat than the AI-generated content itself, Diya argued, was the prospect that politicians and others could use the mere possibility of deepfakes to discredit uncomfortable information.
In neighbouring India, for example, a politician responded to a leaked audio in which he allegedly discussed corruption in his party by alleging that it was fake — a claim subsequently dismissed by fact-checkers.
“It’s easy for a politician to say, ‘This is deepfake’, or ‘This is AI-generated’, and sow a sense of confusion,” she said. “The challenge for the global south . . . is going to be how the idea of AI-generated content is being weaponised to erode what people believe to be true versus false.”
Additional reporting by Jyotsna Singh in New Delhi
dez23
https://www.youtube.com/watch?v=eeLAT6MIf04
nov23
Urgent general election warning as sinister deepfakes 'will play a part' in 2024 vote
EXCLUSIVE: The UK's former top civil servant has warned that Britain must be vigilant to the real threat of artificial intelligence powered technologies before next year's election.
https://www.express.co.uk/news/politics/1837015/deepfakes-uk-general-election-2024-simon-mcdonald-spt
nov23
Buenos Aires — In the final weeks of campaigning, Argentine president-elect Javier Milei published a fabricated image depicting his Peronist rival, Sergio Massa, as an old-fashioned communist in military garb, his hand raised aloft in salute.
The apparently AI-generated image drew about 3-million views when Milei posted it on a social media account, highlighting how the rival campaign teams used artificial intelligence (AI) technology to catch voters’ attention in a bid to sway the race.
“There were troubling signs of AI use” in the election, said Darrell West, a senior fellow at the Center for Technology Innovation at the Washington DC-based Brookings Institution.
“Campaigners used AI to deliver deceptive messages to voters, and this is a risk for any election process,” he said.
Right-wing libertarian Milei won Sunday’s run-off with 56% of the vote as he tapped into voter anger with the political mainstream — including Massa’s dominant Peronist party, but both sides turned to AI during the fractious election campaign.
Massa’s team distributed a series of stylised AI-generated images and videos through an unofficial Instagram account named “AI for the Homeland”.
In one, the centre-left economy minister was depicted as a Roman emperor. In others, he was shown as a boxer knocking out a rival, starring on a fake cover of New Yorker magazine and as a soldier in footage from the 1917 war film.
Other AI-generated images set out to undermine and vilify Milei, portraying the wild-haired economist and his team as enraged zombies and pirates.
The use of increasingly accessible AI tech in political campaigning is a global trend, tech and rights specialists say, raising concerns about the potential implications for important upcoming elections in countries including the US, Indonesia and India in 2024.
nov23
Britain’s cybersecurity agency said Tuesday that artificial intelligence poses a threat to the country’s next national election, and cyberattacks by hostile countries and their proxies are proliferating and getting harder to track.
The National Cyber Security Center said “this year has seen the emergence of state-aligned actors as a new cyber threat to critical national infrastructure” such as power, water and internet networks.
The center — part of Britain’s cyberespionage agency, GCHQ — said in its annual review that the past year also has seen “the emergence of a new class of cyber adversary in the form of state-aligned actors, who are often sympathetic to Russia’s further invasion of Ukraine and are ideologically, rather than financially, motivated.”
https://apnews.com/article/uk-cyber-threats-ai-elections-b6482d3127ae524551e15887b3fdb01b
nov23
Meta, parent company of Instagram and Facebook, will require political advertisers around the world to disclose any use of artificial intelligence in their ads, starting next year, the company said Wednesday, as part of a broader move to limit so-called “deepfakes” and other digitally altered misleading content.
The rule is set to take effect next year, the company added, ahead of the 2024 US election and other future elections worldwide.
https://edition.cnn.com/2023/11/08/tech/meta-political-ads-ai-deepfakes/index.html
out23
Election Integrity in the Age of Generative AI: Fact vs. Fiction
Misinformation and falsehoods about election processes and integrity, spread before and since the 2020 election, have eroded trust in American democracy.
In the last year, great attention has been devoted to the ways that powerful new artificial intelligence tools might disrupt the economy and the wider society.If misused, AI could add new dimensions to the challenges election officials already face. This article, the first of two, addresses specific examples. The second will discuss potential remedies.
https://www.governing.com/security/election-integrity-in-the-age-of-generative-ai-fact-vs-fiction
out23
Deepfake technology could be harnessed by hostile states to sway the forthcoming general election, the head of MI5 has warned.
Ken McCallum said he was concerned that the technology could be used to create ‘all kinds of dissension and chaos in our societies’.
Speaking at a security summit in California, the Director General warned that artificial technology was also being used to amplify terrorism, disseminate propaganda and teach people how to build bombs
https://www.dailymail.co.uk/news/article-12645927/Deepfake-technology-UKs-enemies-election-chaos-MI5-chief.html
out23
The United Kingdom wants to lead the world on AI safety, but at home it is struggling with its most urgent threat.
Fears over the proliferation of AI-generated media, known as deepfakes, intensified this weekend as an audio clip appearing to show U.K. opposition leader Keir Starmer swearing at staffers went viral.
MPs from across the British political spectrum swiftly warned the clip was fake on Sunday. But by Monday afternoon it was still gathering views on X, formerly known as Twitter, and approaching nearly 1.5 million hits.
https://www.politico.eu/article/uk-keir-starmer-labour-party-deepfake-ai-politics-elections/
out23
Just two days before Slovakia’s elections, an audio recording was posted to Facebook. On it were two voices: allegedly, Michal Šimečka, who leads the liberal Progressive Slovakia party, and Monika Tódová from the daily newspaper Denník N. They appeared to be discussing how to rig the election, partly by buying votes from the country’s marginalized Roma minority.
Šimečka and Denník N immediately denounced the audio as fake. The fact-checking department of news agency AFP said the audio showed signs of being manipulated using AI. But the recording was posted during a 48-hour moratorium ahead of the polls opening, during which media outlets and politicians are supposed to stay silent. That meant, under Slovakia’s election rules, the post was difficult to widely debunk. And, because the post was audio, it exploited a loophole in Meta’s manipulated-media policy, which dictates only faked videos—where a person has been edited to say words they never said—go against its rules.
The election was a tight race between two frontrunners with opposing visions for Slovakia. On Sunday it was announced that the pro-NATO party, Progressive Slovakia, had lost to SMER, which campaigned to withdraw military support for its neighbor, Ukraine.
https://www.wired.co.uk/article/slovakia-election-deepfakes
https://www.bloomberg.com/news/newsletters/2023-10-04/deepfakes-in-slovakia-preview-how-ai-will-change-the-face-of-elections
