《RAND：2024生成式人工智能對信息完整性的威脅和潛在的政策應對措施（英文版）（24頁）.pdf》由會員分享，可在線閱讀，更多相關《RAND：2024生成式人工智能對信息完整性的威脅和潛在的政策應對措施（英文版）（24頁）.pdf（24頁珍藏版）》請在三個皮匠報告上搜索。

1、The advent of generative artificial intelligence(AI)and the growth of broader AI capabilities have resulted in many questions about the technologys impact on the body politic and the role of information and trust in a func-tioning democracy.Even before the 2022 generative AI breakthrough when OpenAI

2、 released ChatGPT and DALL-E,the United States and other nations suf-fered from what RAND researchers have referred to as truth decay,or the decline of the role of facts and analysis in public life.1 However,generative AI opens up even newer avenues of both malicious use and unwitting misuse of info

3、rmation,which can lead to sweeping implications for election cycles,the empowerment of danger-ous nonstate actors,the spread of both misinformation and disinformation,and the potential to undermine electoral processes.2 The purpose of this paper is to provide policymakers and scholars with a brief a

4、nd high-level review of potential threats that generative AI might pose to a trust-worthy information ecosystem.3 We offer a summary of policy initiatives that could mitigate these threats.In this paper,we avoid offering specific recommendations TODD C.HELMUS,BILVA CHANDRAGenerative Artificial Intel

5、ligence Threats to Information Integrity and Potential Policy Responses Expert InsightsPERSPECTIVE ON A TIMELY POLICY ISSUEApril 20242AbbreviationsAIartificial intelligenceEUEuropean UnionLLMlarge language modelNISTNational Institute of Standards and TechnologyNPDnonconsensual pornographic deepfakef

6、or policy initiatives,but we try to summarize the strengths and limitations of major proposals.We conducted this review by examining a variety of published resources on generative AI threats to the information space and papers that highlight potential policy options.4 For a more com-prehensive take

7、on policy initiatives,we encourage readers to seek additional sources.In accordance with the purpose of this paper,we first review generative AI threats to information integrity.We examine potential threats that can hail from large language models(LLMs),AI-generated images,and deepfakes.We then prov

8、ide a brief review of policy options for address-ing those threats.In the policy review,we examine options that are associated with content moderation,transpar-ency initiatives that provide for the testing and disclosure of risks,content provenance approaches associated with detection and watermarki

9、ng of synthetic content,and con-tent authenticity of nonsynthetic content.We also review approaches for media literacy,enhanced U.S.government research on AI threats,and privacy initiatives.The new and evolving capabilities of generative AI have the opportunity to significantly undermine trust in me

10、dia and information.LLM hallucinations,or false or misleading content created by LLMs,and the willful mali-cious misuse of LLMs by foreign and domestic actors to mass-produce text that fuels false social media or other web-based content will both likely contribute to truth decay.Generative AI can al

11、so produce deepfake content in the form of false and misleading synthetic images,videos,or voice-based content,which have already taken a toll on democratic processes and have affected information integ-rity during various global conflicts.Policymakers have several tools at their disposal to counter

12、 these potential malign effects.Moderation controls and provenance-based techniques that can warn audiences when they are interacting with synthetically generated con-tent(such as through AI detection systems or watermark-ing)or inform audiences when they are interacting with authentic media content

13、(such as with the Content Authen-ticity Initiative)offer several opportunities of intervention for policymakers and AI platforms.The government and private stakeholders could also promote media literacy initiatives that educate audiences about the risk of genera-tive AI and fund research into AI saf

14、ety.Finally,privacy legislation that imposes greater civil or criminal penalties against nonconsensual deepfakes could further mitigate the potential harms of generative AI.One broad observation from this set of potential policy tools is that it is likely no single tool will provide sufficient prote

15、ction against AI threats to information integrity.Nei-ther provenance-based approaches nor media literacy,for example,are going to be sufficient.Policymakers should consequently consider developing a cohesive set of diverse initiatives that draw on the broad set of tools highlighted in this report.3

16、Artificial Intelligence Information Threats In this section,we review the potential threats to informa-tion integrity that can hail from LLMs,deepfakes,and AI-generated images and audio.We also briefly highlight some potential positive impacts of AI on information integrity as a counterweight to the

17、 reviewed information threats of AI.Large Language ModelsLLMs,such as OpenAIs ChatGPT or Googles Bard,are sizable generative AI models that are trained on a mas-sive corpus of text data scraped from the internet and can generate humanlike text from prompts.5 For example,the most-recent iteration of

18、ChatGPT,GPT-4 was trained on 45 gigabytes of data.6 The model is subsequently fine-tuned on specific datasets to improve its efficacy in specific sub-jects.For example,ChatGPT is an LLM that was fine-tuned to function as a chatbot enabled by user prompting.LLMs can undermine information integrity by

19、 gener-ating false content that is often referred to as hallucinations or confabulations,which can fuel the creation of misin-formation and could contribute to existing polarization or truth decay.7 LLMs can also undermine information integrity by allowing users to generate disinformation or content

20、 that is intentionally false and designed to mislead audiences,which can be used by malicious actors to shape democratic outcomes.8HallucinationsThe term hallucinations is broadly used to describe when LLMs produce false or inaccurate outputs without the use of adversarial human prompting.9 It is vi

21、tal to note that the term is not ideal,given that LLMs are not sentient and do not possess human-level qualities.However,LLMs do not need to be sentient to produce false content.In November 2022,Meta launched its LLM,Galactica,which was built to help scientists summarize research papers,solve math p

22、roblems,and generate wiki articles.The model was taken down by Meta after three days because it generated biased and incorrect results:Fake papers were sometimes attrib-uted to real authors,and the LLM“generated wiki articles about the history of bears in space as readily as ones about protein compl

23、exes and the speed of light.”10 The model was ultimately deemed too inaccurate to be functional.Fur-thermore,reputational risks can arise from hallucinations that spread falsehoods about individuals.For example,ChatGPT generated content about law professor Jonathan Turley committing sexual assault,c

24、iting a nonexistent 2018 Washington Post article.11 The Federal Trade Commis-sion is investigating OpenAI,partly because of ChatGPTs potential to cause reputational harm to consumers.12 LLM hallucinations are a serious issue that has yet to be solved by researchers or large AI developers.Disinformat

25、ionLLMs are effective tools to craft false narratives that are appealing to targeted audiences in various languages.LLMs are trained to produce text that mirrors text pro-duced by humans and function through predictive mecha-nisms based on the data on which they are trained.Com-mercial LLMs are not

26、built with fact-checking components,and they largely rely on preexisting information on which they have been trained or external information that might be retrieved from the internet and other sources.These 4LLMS are not explicitly built to fact-check,although there are methods that can be used to r

27、educe the output of false or incorrect information.In 2021,researchers at the Center for Security and Technology assessed GPT-3 for its poten-tial to generate disinformation.The authors concluded that GPT-3 performed extremely well in such categories as narrative reiteration,generating short message

28、s or head-lines,narrative manipulation,rewriting news articles from a new perspective or tone,narrative seeding,and devising new narratives to support conspiracy theories.13 Research-ers at NewsGuard,an institution that“provides transpar-ent tools to counter misinformation for readers,brands,and dem

29、ocracies,”wrote in a nonacademic report that they asked ChatGPT 3.5 to respond to a series of prompts related to 100 known false narratives from their catalog of provably false online narratives.14 It responded and gener-ated these false narratives 80 percent of the time about such topics as the cor

30、onavirus disease 2019 pandemic,the war in Ukraine,and school shootings.The newest iteration of the model,GPT-4,is equipped with more-robust safeguards against harmful content,such as enhanced Reinforcement Learning from Human Feed-back,and more-tailored safety research on such topics as dangerous in

31、structions.15 However,it is also trained with more than twice the volume of data as its predecessor was,which increases its effectiveness,creativity,and nuance in generating targeted and impactful content.16 Furthermore,malicious actors,internet hobbyists,and white hat hack-ers have discovered how t

32、o jailbreak GPT-4,or how to use prompt engineering to dismantle model safeguards to harmful results,such as accessing personally identifiable information or generating hateful content.17 Ways in Which Large Language Models Can Undermine Information IntegrityThe issues of LLM hallucinations and the a

33、bility to use LLMs to create disinformation could make them an ideal tool to undermine democratic processes at scale,increase polarization and the spread of false or inaccurate information,and undermine trust in media and critical institutions.The issue of LLM hallucinations could affect democ-racy,

34、especially if users place a disproportionate amount of trust in LLM outputs.LLM hallucinationssuch as creat-ing fake citations or confidently stating that specific living LLMs could be an ideal tool to undermine democratic processes at scale,increase polarization and the spread of false or inaccurat

35、e information,and undermine trust in media and critical institutions.5public figures are deadcan be dangerous,especially when models are used without much public access to media or AI literacy training.Model-generated misinforma-tion without avenues for fact-checking could sway public opinion in imm

36、easurable ways,especially when unwitting participants then spread false model-generated information across social media platforms or the rest of the internet.For example,if an LLM chatbot gives an individual false information about where to vote or encourages an indi-vidual who is jaded with U.S.pol

37、itics to not vote at all,it could limit participation in the electoral process.Although the effects of this activity on people have not been mea-sured at scale,studies show that existing detection models cannot classify and detect LLM-generated misinformation as effectively as they can classify huma

38、n-generated misin-formation,which also creates more bottlenecks to identify-ing this form of misinformation on the wider internet.18 Hallucinations can have second-and third-order effects on democracy by simply eroding access to objectively true and factual information on the internet.Users trusting

39、 models to always provide them with reliable or objective informa-tion could also have harmful outcomes.LLM-enabled disinformation could have more direct and profound effects on a trustworthy information eco-system.LLMs can be deployed in multiple languages to intentionally generate false informatio

40、n targeted at specific audiences;a 2023 RAND paper highlights how the Chinese Communist Party appears to be interested in using the technology for just this purpose.19 Even unsophisticated foreign actors will easily and more cheaply be able to create high-fidelity,English-language propaganda content

41、.LLMs also can be used to create large volumes of false social media content that can overwhelm the volume of tradi-tional forms of accurate information.20 LLMs could be used in conjunction with detailed and personal information col-lected by data brokers to disseminate individually crafted and targ

42、eted messages to mass audiences.Finally,LLMs could undercut key components of participatory democ-racy by allowing malicious actors to fake large volumes of constituent feedback or public input into government agency decisionmaking.21Furthermore,in the previously mentioned 2023 paper on generative A

43、I and disinformation,RAND researchers highlighted the viability of using LLMs to greatly improve astroturfing campaigns,or campaigns that create the false perception of grassroots support through inauthentic and deceptive practices due to their ability to rapidly scale targeted content.22 This use w

44、ould be particularly advanta-geous for malign authoritarian state actors.Deepfakes and AI-Generated Images Deepfakes are synthetic media that have been manipulated to deceptively alter the appearance,voice,and disposition of an individual.AI-generated images are any form of syn-thetic imagery create

45、d through an image generation tool,such as DALL-E or Midjourney.Using voice cloning,even short samples of an individuals speech can be cloned and made to generate synthetic speech from text inputs.Gener-ating deepfakes or AI-generated images is quite simple and can be done at low cost:With Midjourne

46、y,users can create images for free or spend 60 dollars per month for a more advanced subscription.23 Furthermore,the guardrails used within these tools are not particularly effective.A Logi-cally.AI report revealed that Midjourney,DALL-E 2,and Stable Diffusion accepted more than 85 percent of prompt

47、s that sought to generate evidence for misinformation or 6disinformation claims.24 Inconsistent moderation has resulted in the generation of images of some public figures but not others.OpenAI prohibits DALL-E from generating public figures;this guardrail functions when prompting for images of Presi

48、dent Joseph Biden and President Donald Trump but not when prompting for images of many less visible public figures.25 Another example of inconsistency is Midjourneys ban on words about the human reproduc-tive system in an effort to prevent the generation of porno-graphic content.26 However,banning w

49、ords at the prompt level is a not a holistic approach to content moderation.AI-generated video content and deepfake videos are also of concern.There are currently three main approaches for generating such content.First,it is possible to create face-swap deepfake videos in which the facial features a

50、nd voice of one person are superimposed on the video of an actor.27 Such videos can be highly realistic;however,they require the support of a trained actor who can mimic the movements of the impersonated individual,and the manual removal of distortions in the video currently takes several weeks or l

51、onger.Second,it is possible to create fully synthetic videos by compiling video and audio footage of an actor or a public official,for example.28 The resulting avatar can then narrate a script and do so in a variety of languages or accents.The firm Synthesia has commer-cialized a similar product by

52、providing avatars and AI-generated voices in more than 120 languages,although the avatars do not perfectly resemble human video footage.29 Lastly,maybe most significantly,OpenAI announced the development of Sora,its text-to-video generative model.The model can generate highly realistic videos with s

53、ome light but still noticeable distortions.The models develop-ment was heralded as a major breakthrough in text-to-video generation.The technology is likely to continue its advancements and enable user-friendly,faster,and more-accurate video generation tools to produce photoreal deep-fake and AI-gen

54、erated video content.30How Deepfakes and AI-Generated Images Could Undermine Information IntegrityDeepfakes and AI-generated images,video,and voiceovers could threaten financial markets,undermine elections,and facilitate foreign propaganda campaigns.In May 2023,an AI-generated image of an explosion

55、at the Pentagon went viral and was quickly debunked as a fake.However,the fleeting phenomenon caused the stock market to briefly plunge,and many speculate that such images can be used to purposely short the stock market.31 Already,deepfake images have been used to sully political opponents,and a wel

56、l-timed deepfake in an October Surprisea deepfake timed to occur immediately before an election and affect its outcomecould theoretically change the outcome of a closely contested election.32 In Slovakia,for example,a deepfake recording was released that falsely depicted offi-cials in the progressiv

57、e party discussing how they would purchase votes from the countrys marginalized Roma party.33 The technology has also been used in foreign pro-paganda campaigns.Graphika reported on a proChinese Communist Party influence operation that used deepfake technology from Synthesia to spread political cont

58、ent.34 In addition,China has reportedly shared misattributed social media content that included AI-generated images and the false assertion that forest fires in Maui were the result of U.S.government“weather weapons.”35 Such efforts reflect a desire by known malicious actors to experiment with 7deep

59、fake technology and use it to generate low-cost and scalable content.Deepfake content could introduce other costs.Such content could exacerbate social divisions in the United States by promoting political polarization or spoofing racially stereotyped behaviors.The mass proliferation of false content

60、 could foment a liars dividend,which occurs when authentic content is slandered as fake and further undermines trust in media.36 This content also poses particular threats to women,given the proliferation of deepfake nude content,and such content could be used to attack and undermine female public f

61、igures.37 Finally,the social and democratic costs to the United States could be dwarfed by costs for more-fragile democracies or develop-ing countries whose populations have lower levels of edu-cation and media literacy and where interethnic tensions are already high.38 A Caveat on the Potential Ris

62、ksAlthough LLMs or deepfake images and voice recordings could taint the information environment and trick audi-ences into believing that they are authentic,the true impact of this content on information integrity and democratic resilience remains to be seen.Dystopian depictions of the consequence of

63、 deepfakes often assume that audiences will continue to remain gullible to deepfake content.However,as this content proliferates and as examples of misuse gain publicity,it seems likely that audiences will adjust the levels of trust they otherwise confer on text,images,video,and voice content.We can

64、 also assume that journalistic out-lets and fact-checking institutions will be able to quickly debunk a share of circulating deepfake materials.In addi-tion,commentators have highlighted the inherent chal-lenges of using deepfakes to influence elections,particu-larly noting how the electorate is so

65、polarized that it will be difficult for any October Surprise video to substantially change individual votes.39 In addition,although deepfake images and videos have been used in recent modern con-flicts,such as the war in Ukraine and the Israeli war in Gaza,few of these have been high-quality deepfak

66、es.For all the technology invested into generative AI,audiences are still relying on cheapfakeswhich are manually altered,selectively edited,or out-of-context content intended to mislead an audienceto influence users.40 Hence,it is con-ceivable that some of the darkest visions of generative AIs impa

67、ct might not come to pass.Deepfakes and AI-generated images,video,and voiceovers could threaten financial markets,undermine elections,and facilitate foreign propaganda campaigns.8Potential Positive Impacts of Generative AI on Information IntegrityBeyond the potential negative impacts of generative A

68、I,there might also be more-positive impacts on informa-tion integrity.For example,fine-tuned LLMs could be trained on civic content,bills passed by Congress,and other publicly available content to communicate informa-tion on key policy issues,voting procedure and locations,congressional activities,a

69、nd more.Such models could also be trained to communicate to nonEnglish-speaking audiences.Doing so would allow individuals with limited English-speaking skills to civically engage more effec-tively and lower barriers to education for information or media literacy and democratic participation.The off

70、ice of New York City Mayor Eric Adams has used AI to share public service announcements in several languages that the mayor does not speak,and the city is launching an AI chatbot to answer questions for small business owners.41 When deployed correctly(i.e.,for narrow tasks,and with guardrails in pla

71、ce),these models could be used to support stronger information ecosystems and could help overcome the echo chambers of social media.Generative AI can also be used to moderate or limit audience exposure to harmful content.In 2023,OpenAI released a blog post about using GPT-4 for content modera-tion t

72、hat notes how the LLM can be effective in classifying and detecting harmful content at scale and could reduce the costs of using human moderators and the burden of human moderation of egregious content.42 However,the efficacy of these classifiers on gray-area contentsuch as satire or political conte

73、ntcan be questionable,and it is unknown how they perform compared with human-generated labels.The LLM can also be used to help social media and AI platforms draft wording for new content moderation policies,which,according to OpenAI,can help produce more-consistent labels for offensive content and i

74、ncrease the speed associated with what is currently a laborious cycle of developing policy updates.43 These tech-niques are novel and have not been thoroughly evaluated for effectiveness,but they could provide a promising start for AI developers and social media companies.Policy Options for Managing

75、 AI Threats to Information IntegrityIn the remainder of this paper,we provide an overview of potential steps that the U.S.government can take to address the potential risks posed by generative AI.We offer both a review of conceptual issues related to these approaches and steps that the U.S.governmen

76、t can take to strengthen their implementation.We begin this review by summarizing Beyond the potential negative impacts of generative AI,there might also be more-positive impacts on information integrity.9current regulatory initiatives in the European Union(EU)and the United States.We then examine o

77、ptions associated with content moderation,transparency initiatives,content provenance,media literacy,enhanced U.S.government research on AI threats,and privacy initiatives.Current European Union and U.S.Regulatory ApproachesThe new EU rules approach applies standards to AI sys-tems according to thei

78、r risk levels.Some unacceptable risk systems will be banned,including the use of AI to support social scoring,in which AI systems would be used to rate citizens according to certain characteristics or behaviors.AI systems that pose potential risks to democracy,such as generative AI,are classified as

79、 high risk.44 For example,according to these rules,which were approved on Decem-ber 9,2023,AI platforms are required to inform audiences when they are interacting with a chatbot,label deepfakes and AI-generated content,and develop systems to allow AI-generated content,such as deepfakes,to be detecte

80、d.45 Transparency requirements are also built into the system;these requirements include providing detailed technical documentation and information on content used for train-ing AI models.46 In contrast,U.S.stakeholders have so far taken a mea-sured approach through voluntary commitments and,most si

81、gnificantly,the longest executive order in history on AI.47 The White House designates key agencies to move the needle on AI developments.From a regulatory per-spective,the White House is requiring compute reporting requirements for incredibly large model training runs to be led by the Department of

82、 Commerce.It also requires the Department of Commerce,under the National Institute of Standards and Technology(NIST)to produce a report on synthetic content that covers such issues as content authen-tication and watermarking,as well as potential measure-ment science techniques around synthetic conte

83、nt.These actions taken by the United States are contingent on desig-nated agency actions to complete these tasks in the stated time frame and do not carry the same level of regulatory weight as congressional legislation.There is particular con-cern that the executive order could be overturned by a n

84、ew president and might not be permanent.Consequently,there might be value for the U.S.Congress to consider more per-manent legislative approaches that can establish necessary regulations or provide funding to support other initiatives,such as media literacy and dedicated AI safety research.Content M

85、oderationAI platforms often implement content moderation policies that impose limits on the kind of content generated by their algorithms.The types of content targeted for moderation varies according to platform policies,although common types include sexually explicit materials,graphic violence,and

86、hateful material.AI developers use a variety of tactics as moderation solutions,including filtering out certain types of content(such as sexually explicit images)from being used to train the algorithms,introducing in-model controls by using human feedback to fine-tune a models weights,using input fi

87、lters to limit the types of search terms or prompts that can be used to query the algorithms,and filtering outputs or the generation of content surfaced to the user.48 Midjourney,for example,employs prompt-level modera-tion to block unsavory content.Unfortunately,none of these approaches are foolpro

88、of or comprehensive,although 10they can help mitigate some forms of misuse.49 Research by OpenAI suggests that GPT-4 can be used to support con-tent moderation,but,as we noted previously,this approach remains untested.50Content moderation cannot fill gaps for open-source models,or models that have t

89、heir code available on the internet,or for closed models that provide fine-tuning access through an application programming interface.For these models,content moderation is not potent,and more-concerning harms are possible,especially given the dif-fused nature of open-source models.Any malicious act

90、or can likely remove model safeguards on an open-source model and fine-tune that model to create harm because there is not any direct content moderation on model inputs and fine-tuning methods.51 Traditional content moderation methods are helpful for larger,closed models,although they are not a pana

91、cea for safety issues and adversarial abuse and have little to no effect on open-source models that can be fine-tuned by opportunistic actors.Government Approaches to Address Content ModerationThe United States could set requirements for basic content moderation regimes.The first task to do so would

92、 be to identify the types of content that should be subject to con-tent moderation.For example,LLMs could be forbidden from generating large volumes of content that resemble tweets,and image and deepfake video generators could limit users ability to generate images of high-profile politi-cal actors.

93、Clearly,some types of content moderation would be subject to intense debate among policymakers and the public,and such moderation regulations risk legal chal-lenges on the basis of the First Amendment.Furthermore,as the false image of the Pentagon explosion demonstrates,it will be difficult to predi

94、ct and moderate all types of potentially harmful material.Regardless of which content is subject to moderation,regulations could focus on the ways in which AI platforms perform moderation,such as by stipulating require-ments for training data or requiring input/output filters.Although limiting the t

95、ype of content used to train the algorithms is often considered the most robust approach,it can also be extremely cumbersome because existing AI systems would require expensive and time-consuming retraining or filtering of existing data.52 Despite the limita-tions of safety filters,they are consider

96、ed more agile and can more easily be implemented on existing systems than retraining or filtering.53 The United States could further mandate approaches to open-source codes and limit ways in which such codes could be manipulated to undermine the code that drives content moderation.54 An alternative

97、or complementary approach to mandat-ing content moderation could be to ensure that AI devel-opers can be held liable for the content that their models produce.It remains unclear whether AI developers are protected by Section 230,and a bipartisan bill in Congress would clarify that Section 230 does n

98、ot apply to AI devel-opers.55 This approach would create a legal incentive for AI developers to limit potentially dangerous forms of content,although it could simultaneously subject them to signifi-cant levels of costly litigation and provide an advantage to major incumbent firms that can afford the

99、 inevitable litigation.11Transparency:AI Platforms Could Test for and Disclose Risks The specter or the absence of such moderation controls raises a key transparency issue.Platforms can play a role in managing problematic content by disclosing the types of content that are and are not subject to mod

100、eration,and they could rigorously test their algorithms to ensure that the designed moderation controls are effective.Such disclo-sures can be fraught,however because they depend on the rigor of the testing regimes.In addition,many such disclo-sures often go unread,and they can be too complicated to

101、 be understood by the typical user.56 Government Approaches to Testing and Disclosure of RisksThe White House has already secured commitments from leading AI firms to voluntarily internally test AI systems before their release,facilitate third-party assessments fol-lowing release,and publicly report

102、 AI system capabilities and limitations.57 Some of the larger AI labs have joined together to form the Frontier Model Forum to ensure“safe and responsible development”of AI models through an industry body to coordinate safety efforts and work directly with government and civil society.58 The actual

103、implementation of such voluntary standards remains unclear,as does whether in-house testing and disclosures can sufficiently address potential risks.As a result,one common recommendation is the creation of an“FDA-like”approval and licensing process in which a government agency would work with outsid

104、e experts to rigorously test potentially dangerous new AI systems and certify that they meet minimum federal safety standards.59 The pro-cess could also require that the platforms receive a federal license before large-scale deployment and mandate contin-ued auditing after the AI system is publicly

105、released.The obvious benefit of this approach is that it would ensure some universal basic minimum standard of AI safety across AI labs.However that standard could come at the risk of enshrining the position of market leaders,which,in contrast to upstart firms,are better positioned to both influence

106、 regulations and meet the burdens of those regu-lations.60 This regulatory regime might also struggle to keep pace with rapid advances in AI.61 Furthermore,AI developers could create information-sharing regimes between each other and social media platforms by hashing harmful content(e.g.,known disin

107、-formation outputs,extremist terrorist content)and sharing hashes in order to better track the proliferation of this con-tent on the internet.Hashing is a method to create unique digital signatures for content in its raw form,which allows for the detection of the raw content across the internet via

108、hash disclosure.This is a common practice used for identifying terrorist content.62 The Global Internet Forum to Counter Terrorism curates a hash-sharing database and disseminates hashes of known extremist social media content across social media platforms,thus allowing those platforms to more easil

109、y identify and remove the offend-ing content.63 A similar method can be used by large AI developers and social media companies to hash and label harmful AI-generated disinformation,extremist content,and other forms of egregious content that would be stored in a secure database for social media platf

110、orms to track the dissemination of harmful AI content.Disclosing content moderation and policy enforcement efforts through trans-parency reporting and information-sharing would be a vital first step in a coordinated system.12Data transparency laws focused on social media platforms might be helpful i

111、n mitigating the threats of deepfakes.Proposed transparency legislation,for example,would mandate that social media firms divulge informa-tion on platform policies,which would help consumers understand the degree to which the platforms screen for and either remove or label deepfake content.64 Anothe

112、r suggested policy option would allow select researchers access to sensitive social media so that they could assess how platform algorithms promote deepfake content.65Content Provenance:Detection and WatermarkingContent provenance approaches help inform an audience about the origin of particular med

113、ia content.One approach to content provenance is to identify synthetic content.One method for doing so is to use AI-driven detection technol-ogy,which can scan linguistic,video,audio,or still-image content and identify whether that content is created with AI.However,the tools might not be capable of

114、 reliably detecting AI-created content.AI-detection tools for text content,for example,have low accuracy rates and have the tendency to detect authentic text as AI-generated.66 Fur-thermore,it has been shown that Optics“AI or Not”tool is quite inaccurate with compressed images,67 and Hany Farid,a do

115、main expert and professor at UC Berkeley,has stated that image detectors have trouble with“highly struc-tured shapes and straight lines,”which further reduces detection accuracy.68 Although detection capabilities for deepfakes have significantly improved over the past several years,so has the develo

116、pment of deepfake content.The result is an arms race that is decidedly in favor of those creating the deepfake content because content creators can adapt to new forms of detection.69 An alternative approach involves watermarking AI-generated media content.The simplest version of a water-mark is a lo

117、go overlaid on an image,such as DALL-Es rainbow watermark.Alternatively,imperceptible patterns of image pixels can be embedded into the image at the point of generation that computers can then read and use to identify the image as synthetically generated.Currently,implementation of such imperceptibl

118、e watermarks is not enforced.Initial evidence suggested that watermarks were a relatively robust measure against image manipulations.However,more-recent studies suggest that experts can not only successfully remove watermarks from digitally altered content but also add fabricated watermarks on authe

119、ntic content(thus rendering real content seemingly fake),which can be manipulated by bad actors.70 Although the technol-ogy for creating and hacking watermarks is advancing,there might still be utility in developing an approach in Data transparency laws focused on social media platforms might be hel

120、pful in mitigating the threats of deepfakes.13which some forms of deepfake content can be effectively identified and labeled.71 An alternative approach to provenance is for social media or search engines to offer more information about the provenance of an image.For example,Google released an“About

121、This Image”feature for its search engine.This feature provides users with information about when the image and similar images were first indexed by Google,where it might have appeared first,and where else it has been seen online,including fact-checking websites.72 Government Approaches to Promote Wa

122、termarkingLeading AI companies have publicly committed to develop-ing robust mechanisms,such as watermarking,to inform users when they are viewing AI-generated content.73 The government might have a vested interest in working with these companies to establish a common framework for watermarking cont

123、ent and could mandate the application of this framework to major AI platforms.The government and AI labs could also work with social media platforms to develop processes by which detection regimes can be effectively applied,recognizing that some platforms,par-ticularly those with a small market shar

124、e,might not be able to consistently screen all shared content for synthetically generated materials.That said,social media platforms routinely screen for child pornographic content for legal compliance,and similar processes might be applicable to AI-generated imagery.74The U.S.government could furth

125、er promote or man-date labeling regimes so that the public could be warned when they are viewing synthetically generated images,videos,or voice recordings.The Federal Election Commis-sion,for example,has already initiated a petition seeking to regulate AI images used in political ads that misreprese

126、nt political opponents.75 Furthermore,a series of legislative bills would expand the authority of the Federal Election Commission and mandate that it require“clear and con-spicuous”notice when political advertisements contain AI-generated content.76 Beyond these approaches,the government could requi

127、re that social media companies develop visually distinctive contextual labels or warnings for AI-generated content.One challenge of labeling regimes is deciding which types of content would require labeling.With the rise of generative AIsupported photo editing software,for example,it can be expected

128、 that many photos uploaded to social media will include innocuous forms of editing,such as turning a gray sky into a blue sky.In addition,political campaigns and advertisers could increasingly use generative images as a cheaper alternative to photographic images.Therefore,a question arises as to whi

129、ch forms of AI-generated material require warning labels.If warn-ing labels are attached to all such images,then the labels attached to more deceptive content could become less effective.Content Provenance:Content Authenticity InitiativeThrough the Content Authenticity Initiative,Adobe,Qual-comm,Tru

130、epic,the New York Times,and other collabora-tors have developed a way to digitally capture and present the provenance of images,videos,and voice recordings.For example,the Content Authenticity Initiative devel-oped a way for photographers to use a secure mode on their smartphones;this mode embeds cr

131、itical information into the metadata of the digital image.The approach uses“cryptographic asset hashing,”which provides“verifiable,14tamper-evident signatures that the image and metadata hasnt been unknowingly altered,”to track and record information about the image and any resulting edits to the co

132、ntent.77 When images are shared on collaborating social media platforms or on participating news websites,when and where the photo was taken and the types of edits made to the resulting image can be identified.78 Furthermore,the Coalition for Content Provenance and Authority open framework,which hel

133、ps establish the cryptographic hash-ing,can detect when an image has been manipulated or is not in its original state because the technology is“tamper-evident.”79 Ultimately,the technology provides a hallmark of trust for authenticated content and ensures that the rise of deepfakes cannot undermine

134、trust in all forms of media.A key challenge for content provenance relates to adoption of the technology.The Coalition for Content Provenance and Authority has established the technical standards that will guide the implementation of con-tent provenance for creators,editors,publishers,media platform

135、s,and consumers of the technology.80 For this approach to be successful,manufacturers would need to outfit cameras,mobile phones,and other recording devices with the technology,and social media platforms and other key media outlets would need to host and employ the authentication technology.Governme

136、nt Approaches to Advancing Content AuthenticityThe U.S.government could take steps to promote adoption of content authenticity.First,the government could model adoption by using provenance-based content to show trans-parency and authenticity of government created and trans-mitted photographic and vi

137、deo content.The Senate version of the fiscal year 2024 National Defense Authorization Act,for example,calls for a pilot program to assess the feasi-bility of implementing content provenance standards for photographic and visual content released by the Defense Visual Information Distribution Service.

138、81 Such initiatives could be expanded across the government.Furthermore,President Bidens Executive Order on AI designates the Department of Commerces NIST to build a comprehensive report on current provenance and watermarking efforts and potentially translate them into future standards or guidelines

139、.82 NIST,in collaboration with other agencies,could also work with social media platforms and publishers of media content to collaborate on the adoption of content provenance verification technology for their systems and to build effective best practices around adoption for different industry stakeh

140、olders.Media LiteracyMedia literacy initiatives can help audiences assess the accuracy and credibility of media and think twice before sharing content online.Such initiatives would need to build public awareness of the threat of deepfakes and generated text and the rising proliferation of this techn

141、ology.These The U.S.government could take steps to promote adoption of content authenticity.15initiatives could help audiences become increasingly savvy consumers of information and learn to recognize when certain types of content are suspiciously inauthentic(e.g.,a generative AI image of Pope Franc

142、is wearing a white puffy jacket).83 Government Approaches to Advancing Media LiteracyThe U.S.government could further oversee national edu-cation efforts to inform the public about the threats of AI and improve peoples capacity to navigate those threats and make informed choices on how to use AI.Doi

143、ng so could include the creation of public service announcements and broad digital campaigns,as well as more-grassroots approaches in local communities.The Department of Edu-cation could further work to promote AI media literacy in classrooms.Audiences would need,as part of such literacy,to better u

144、nderstand foundational AI models,such as LLMs,and recognize the capabilities and limits that such models have.The government could also find ways to support civil society organizations that are already serving at the fore-front of efforts to enhance information integrity.One criti-cal set of tools t

145、hat civil society and news organizations rely on to verify the authenticity of online content are called Open Source Intelligence tools.These tools can help inves-tigators conduct online searches for images(i.e.,reverse image search),separate video content into individual frames,and validate image c

146、ontent by identifying historical weather patterns and even authenticating shadow angles.84 Continued development of such tools and the expansion of initiatives that train civil society actors both in the United States and abroad on their use could prove critical in the fight against deepfakes.Likewi

147、se,efforts that educate and empower civil society to advance media literacy education might complement and enhance the effects of government education initiatives.ResearchFinally,continued academic and private-sector research will prove critical to advancing protections against the threats of AI-gen

148、erated content.Government Approaches to Advancing Research on AIA 2022 roadmap for research on AI threats to informa-tion integrity recommended four lines of research that are critical to addressing such threats.85 These consist of the need to analyze,model,and measure information eco-systems;invest

149、igate safeguards that assist people;identify technical approaches to enhance high-integrity informa-tion exchange;and establish effective strategies to address manipulated-information campaigns.The U.S.government could consider a concerted and coordinated strategy for funding this research.The resea

150、rch roadmap also called for further improve-ments in research collaboration and infrastructure.86 There are several opportunities to accomplish this:The U.S.government could foster collaborationpossibly through a national or international research consortiumamong experts on AI,democracy,and informat

151、ion and cogni-tive security.It could further enable this consortium with crowdsourced access to expensive computational equip-ment that would otherwise be cost prohibitive for individ-ual researchers and research institutions.87 It could likewise enable research access to generative-AI platforms and

152、 16sensitive social media data,although doing so will require some form of transparency legislation that would mandate such access.88Privacy InitiativesGenerative AI has the potential to upend the privacy of U.S.citizens.As of 2021,the most common application of deepfake technology involved the crea

153、tion of noncon-sensual pornographic images.89 Democracy is particularly threatened when synthetic images and videos fabricate the image of public officials and other real citizens.Synthetic nonconsensual intimate imagery can play a critical role in such democratic threats,given that female politicia

154、ns or government leaders might be particularly vulnerable to having images of them or their likeness being manipu-lated.90 Finally,both generative and narrow AI can be used to power highly targeted political and other advertising campaigns by drawing on the large trove of consumer data.Efforts that

155、address these and related privacy threats could prove critical.Government Approaches to Enhancing PrivacyThere are several policy options that the U.S.government could consider for addressing AI threats to privacy.First,an online safety bill could address the threat of nonconsen-sual deepfake conten

156、t,particularly nonconsensual porno-graphic deepfakes(NPDs).Bills of this variety have been passed in a few states.These bills vary significantly;the state of California affords monetary damages in civil court for NPDs,whereas the Commonwealth of Virginia offers criminal liability but only in cases f

157、or which the distribu-tion was intended to“coerce,harass,or intimidate,”which can be difficult to prove.91 Some advocates call for a stricter federal criminal law that forgoes requirements to prove intent and makes distribution of NPDs punishable with time in prison.The enactment of data-privacy law

158、s is another option to address the threat of generative AI.A data-privacy law that affords consumers the right to see what types of infor-mation data brokers have amassed and the right to opt-in to such data collection,for example,could limit the poten-tial threats of AI-assisted targeted advertisin

159、g.92 17Notes1 Kavanagh and Rich,Truth Decay.2 For examples of how generative AI could be used to undermine elec-tions,see Posard et al.,The 2024 U.S.Election,Trust,and Technology.3 We have drafted this paper primarily in the context of the U.S.infor-mation environment and the U.S.policy process;howe

160、ver,many aspects of both threats and policy solutions could have international relevance.4 We note that we did not conduct a systematic review.Systematic reviews are review papers that use systematic methods to collect indi-vidual,published studies and analyze and synthesize the findings of those pa

161、pers.Instead,we first consulted published literature reviews on the topic of AI and synthetically generated media and then searched for research papers that address the specific policy options reviewed in this paper.5 Okerlund et al.,Whats in the Chatterbox?6 Sarkar,“GPT3 vs GPT4-Battle of the Holy

162、Grail of AI Language Models.”7 RAND Corporation,“Countering Truth Decay.”8 Misinformation is false information that is unintentionally or unwit-tingly spread.Disinformation is false information that is knowingly or maliciously spread.9 Yao et al.,“LLM Lies”10 Heaven,“Why Metas Latest Large Language

163、Model Only Survived Three Days Online.”11 Nelson,“ChatGPT Wrongly Accuses Law Professor of Sexual Assault.”12 Kerr,“FTC Investigating ChatGPT over Potential Consumer Harm.”13 Buchanan et al.,Truth,Lies,and Automation.14 NewsGuard,“About NewsGuard.”15 Aerin,“RLHF:Reinforcement Learning from Human Fee

164、dback.”RLHF refers to a machine learning technique that uses human feedback to directly train a reward model.16 OpenAI,GPT-4 System Card.17 For an example of actors who have managed to jailbreak the model,see Reddit,“r/ChatGPTJailbreak.”18 See,for example,Zhou et al.,“Synthetic Lies.”19 Marcellino e

165、t al.,The Rise of Generative AI and the Coming Era of Social Media Manipulation 3.0.20 China,for example,seems to have such a tactic to overwhelm the hashtag“#Xinjiang,”which references the Chinese region known for the forced labor and reeducation of Chinas Muslim Uyghur population.Instead of findin

166、g tweets addressing human rights abuses,a reader is just as likely to see tweets depicting cotton,which is one of Xinjiangs greatest exports,and the fields in which it is grown(Linvill and Warren,“Understanding the Pro-China Propaganda and Disinformation Tool Set in Xinjiang”).See also DiResta,“The

167、Supply of Disinformation Will Soon Be Infinite.”21 Panditharatne and Weiner,“Artificial Intelligence,Participatory Democracy,and Responsive Government.”22 Marcellino et al.,The Rise of Generative AI and the Coming Era of Social Media Manipulation 3.0.23 Ulmer and Tong,“Deepfaking It.”24 Walter,Testi

168、ng Multimodal Generative AI.25 Ulmer and Tong,“Deepfaking It.”26 Heikkil,“AI Image Generator Midjourney Blocks Porn by Banning Words About the Human Reproductive System.”27 Sample,“What Are Deepfakesand How Can You Spot Them?”28 Deepbrain AI,homepage.29 Synthesia,homepage.30 OpenAI,“Sora.”31 Marcelo

169、,“Fake Image of Pentagon Explosion Briefly Sends Jitters Through Stock Market.”32 In general,October Surprise refers to any newsworthy event that hap-pens prior to a U.S.election that has the potential to affect the outcome of the election.33 Meaker,“Slovakias Election Deepfakes Show AI Is a Danger

170、to Democracy.”1834 Graphika,Deepfake it Till You Make It.35 Sanger and Myers,“China Sows Disinformation About Hawaii Fires Using New Techniques.”36 Chesney and Citron,“Deep Fakes.”37 Jankowicz et al.,Malign Creativity.38 In India,for example,several popular news anchors have been used in deepfake vi

171、deos that purported to have them selling diabetes medica-tion.A fake of a well-known Indian industrialist had him selling“dubi-ous financial investment opportunities”(Dasgupta,“Deepfake Videos Raise Concern in India Ahead of General Election”).Deepfakes also appeared in Burkina Faso showing a“divers

172、e group of people urging Burkinabe to support”the military junta(“Concern Grows as Deep-fakes Spread Misinformation”).39 Carlyon,“Deepfakes Arent the Disinformation Threat Theyre Made Out to Be.”40 Carlyon,“Deepfakes Arent the Disinformation Threat Theyre Made Out to Be.”41 Coltin,“Greetings from Ma

173、yor Adams,Generated by AI,in Different Languages.”42 Hurtz,“OpenAI Wants GPT-4 to Solve the Content Moderation Dilemma.”43 Hurtz,“OpenAI Wants GPT-4 to Solve the Content Moderation Dilemma.”44 European Parliament,“EU AI Act.”45 Heikkil,“Five Things You Need to Know About the EUs New AI Act,”46 Europ

174、ean Parliament,“Artificial Intelligence Act.”47 Biden,“Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence.”48 Hao et al.,“Safety and Fairness for Content Moderation in Genera-tive Models.”49 Lambert,et al.,“Illustrating Reinforcement Learning from Human Feedback(RLHF).”50 Hur

175、tz,“OpenAI Wants GPT-4 to Solve the Content Moderation Dilemma.”51 Qi et al.,”Fine-Tuning Language Models Compromises Safety Even When Users Do Not Intend to!”52 These approaches often come with their own caveats.Models trained on harmful training data can still generate harmful outputs even with sa

176、fety filters because of hallucinations and other black box glitches within a system.53 Hao et al.,“Safety and Fairness for Content Moderation in Genera-tive Models.”54 This approach would likely be achieved by using a safety-by-design approach with vetted training data(Rando et al.,“Red-Teaming the

177、Stable Diffusion Safety Filter”).55 Section 230 is a“law that shields internet companies from liabil-ity for the content posted to their platforms”(Paul,“Bipartisan U.S.Bill Would End Section 230 Immunity for Generative AI”).See also Macpherson,“Lies,Damn Lies,and Generative Artificial Intelligence.

178、”56 Meehan,“Transparency Wont Be Enough for AI Accountability.”57 White House,“Biden-Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI.”58 Heath,”New Group To Represent AI Frontier Model Pioneers.”59 Marcus,replies to U

179、.S.Senate queries,p.3.60 Meehan,“Transparency Wont Be Enough for AI Accountability.”61 Thierer and Chilson,“The Problem with AI Licensing and an FDA for Algorithms.”62 Thorley,“Advances in Hashing for Counterterrorism.”63 Thorley,“Advances in Hashing for Counterterrorism.”64 MacCarthy,“Transparency

180、Is Essential for Effective Social Media Regulation.”65 Perrino,“Platform Accountability and Transparency Act Reintro-duced in Senate.”66 Kelly,“ChatGPT Creator Pulls AI Detection Tool Due to Low Rate of Accuracy.”67 Kovtun,”Testing AI or Not.”1968 Maiberg,“AI Image Detectors Are Being Used to Discre

181、dit the Real Horrors of War.”69 Studies suggest that detectors of deepfaked content struggle when confronted with high-quality deepfakes or deepfakes generated with novel forms of technology that were not part of the detector training dataset.Still,some new detectors that,for example,measure blood f

182、low patterns on facial images claim superior performance.It is unclear whether such claims have been independently evaluated or whether new forms of deepfake technology can render the detection system useless(Beckmann,Hilsmann,and Eisert,“Fooling State-of-the-Art Deepfake Detection with High-Quality

183、 Deepfakes”;Helmus,Artificial Intelli-gence,Deepfakes,and Disinformation;Intel,“How Intels New Deepfake Detection Works”).70 Saberi et al.,“Robustness of AI-Image Detectors”;Yu et al.,“Artifi-cial Fingerprinting for Generative Models.”71 Knibbs,“Researchers Tested AI Watermarksand Broke All of Them.

184、”72 Dunton,“Get Helpful Context with About This Image.”73 White House,“Biden-Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI.”74 National Center for Missing and Exploited Children,“Child Sexual Abuse Material.”75 Swen

185、son,“FEC Moves Toward Potentially Regulating AI Deepfakes in Campaign Ads.”76 Shea et al.,“Tech Policy Trifecta.”77 Content Authenticity Initiative,“How It Works.”78 Content Authenticity Initiative,homepage.79 Coalition for Content Provenance and Authenticity,“C2PA Specifications.”80 Coalition for C

186、ontent Provenance and Authenticity,“C2PA Specifications.”81 Public Law 118-31,National Defense Authorization Act for Fiscal Year 2024.82 Biden,“Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence.”83 Novak,“That Viral Image of Pope Francis Wearing a White Puffer Coat is Totall

187、y Fake”;Helmus,Artificial Intelligence,Deepfakes,and Disinformation.84 Gregory,“Deepfakes and Synthetic Media”;Helmus,Artificial Intel-ligence,Deepfakes,and Disinformation.85 Information Integrity R&D Interagency Working Group,Roadmap For Researchers on Priorities Related to Information Integrity Re

188、search and Development.86 Information Integrity R&D Interagency Working Group,Roadmap For Researchers on Priorities Related to Information Integrity Research and Development.87 Wanless and Shapiro offer a vision for such a social media research collaboration that could apply well to AI(Wanless and S

189、hapiro,A CERN Model for Studying the Information Environment).88 See Marcus and Reuel,“The World Needs an International Agency for Artificial Intelligence,Say Two AI Experts.”89 Hao,“Deepfake Porn Is Ruining Womens Lives.Now the Law May Finally Ban It.”90 Jankowicz et al.,Malign Creativity.91 Willia

190、ms,“Tightening Restrictions on Deepfake Porn.”92 Klosowski,“The State of Consumer Data Privacy Laws in the U.S.(and Why It Matters).”20References Aerin,Kim,“RLHF:Reinforcement Learning from Human Feedback,”Towards Data Science,October 11,2023.Anote,“LLMs:Eight Things to Know About Large Language Mod

191、els,”Medium,May 25,2023.As of March 29,2024:https:/anote- Hilsmann,and Peter Eisert,“Fooling State-of-the-Art Deepfake Detection with High-Quality Deepfakes,”arXiv,May 16,2023.Biden,Joseph R.,Jr.“Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence,”Executive Order 14110,Octobe

192、r 30,2023.Buchanan,Ben,Andrew Lohn,Micah Musser,and Katerina Sedova,Truth,Lies,and Automation:How Language Models Could Change Disinformation,Center for Security and Emerging Technology,May 2021.Carlyon,Peter,“Deepfakes Arent the Disinformation Threat Theyre Made Out to Be,”RAND Blog,December 19,202

193、3.As of February 22,2024:https:/www.rand.org/pubs/commentary/2023/12/deepfakes-arent-the-disinformation-threat-theyre-made.htmlChesney,Bobby,and Danielle Citron,“Deep Fakes:A Looming Challenge for Privacy,Democracy,and National Security,”California Law Review,Vol.107,December 2019.Coalition for Cont

194、ent Provenance and Authenticity,“C2PA Specifications,”webpage,undated.As of February 22,2024:https:/c2pa.org/public-draft/Coltin,Jeff,“Greetings from Mayor Adams,Generated by AI,in Different Languages,”Politico,October 16,2023.“Concern Grows as Deepfakes Spread Misinformation,”Africa Defense Forum,A

195、pril 11,2023.Content Authenticity Initiative,homepage,undated.As of February 22,2024:https:/contentauthenticity.orgContent Authenticity Initiative,“How It Works,”webpage,undated.As of February 22,2024:https:/contentauthenticity.org/how-it-works Dasgupta,Debarshi,“Deepfake Videos Raise Concern in Ind

196、ia Ahead of General Election,”Straits Times,December 18,2023.DeepBrain AI,homepage,undated.As of March 11,2024:https:/www.deepbrain.ioDiResta,Renee,“The Supply of Disinformation Will Soon Be Infinite,”The Atlantic,September 20,2020.Dunton,Cory,“Get Helpful Context with About This Image,”The Keyword

197、blog,Google,May 10,2023.European Parliament,“Artificial Intelligence Act:Deal on Comprehensive Rules for Trustworthy AI,”press release,December 9,2023.European Parliament,“EU AI Act:First Regulation on Artificial Intelligence,”last updated December 19,2023.Graphika,Deepfake It Till You Make It:Pro-C

198、hinese Actors Promote AI-Generated Video Footage of Fictitious People in Online Influence Operation,February 2023.Gregory,Sam,“Deepfakes and Synthetic Media:What Should We Fear?What Can We Do?”WITNESS blog,undated.Hao,Karen,“Deepfake Porn Is Ruining Womens Lives.Now the Law May Finally Ban It,”MIT T

199、echnology Review,February 12,2021.Hao,Susan,Piyush Kumar,Sarah Laszlo,Shivani Poddar,Bhaktipriya Radharapu,and Renee Shelby,“Safety and Fairness for Content Moderation in Generative Models,”arXiv,June 9,2023.Heath,Ryan,“New Group To Represent AI Frontier Model Pioneers,”Axios,July 26,2023.Heaven,Wil

200、l Douglas,“Why Metas Latest Large Language Model Only Survived Three Days Online,”MIT Technology Review,November 18,2022.Heikkil,Melissa,“AI Image Generator Midjourney Blocks Porn by Banning Words About the Human Reproductive System,”MIT Technology Review,February 24,2023.Heikkil,Melissa,“Five Thing

201、s You Need to Know About the EUs New AI Act,”MIT Technology Review,December 11,2023,Helmus,Todd C.,Artificial Intelligence,Deepfakes,and Disinformation:A Primer,RAND Corporation,PE-A1043-1,2022.As of February 22,2024:https:/www.rand.org/pubs/perspectives/PEA1043-1.htmlHurtz,Simon,“OpenAI Wants GPT-4

202、 to Solve the Content Moderation Dilemma,”The Verge,August 15,2023.21Information Integrity R&D Interagency Working Group,Roadmap For Researchers on Priorities Related to Information Integrity Research and Development,Office of Science and Technology Policy,December 2022.Intel,“How Intels New Deepfak

203、e Detection Works,”webpage,undated,As of February 22,2024:https:/ Jankowicz,Nina,Jillian Hunchak,Alexandra Pavliuc,Celia Davies,Shannon Pierson,and Zoe Kaufmann,Malign Creativity:How Gender,Sex,and Lies Are Weaponized Against Women Online,Wilson Center,January 2021.Kavanagh,Jennifer,and Michael D.Ri

204、ch,Truth Decay:An Initial Exploration of the Diminishing Role of Facts and Analysis in American Public Life,RAND Corporation,RR-2314-RC,2018.As of February 26,2024:https:/www.rand.org/pubs/research_reports/RR2314.htmlKelly,Samantha Murphy,“ChatGPT Creator Pulls AI Detection Tool Due to Low Rate of A

205、ccuracy,”CNN Business,July 25,2023.Kerr,Dara,“FTC Investigating ChatGPT over Potential Consumer Harm,”NPR,July 13,2023.Klosowski,Thorin,“The State of Consumer Data Privacy Laws in the U.S.(and Why It Matters),”New York Times,September 6,2021.Knibbs,Kate,“Researchers Tested AI Watermarksand Broke All

206、 of Them,”Wired,October 3,2023.Kovtun,Dennis,“Testing AI or Not:How Well Does an Image Detector Do Its Job?”Bellingcat,September 11,2023.Lambert,Nathan,Louis Castricato,Leandro von Werra,and Alex Havrilla,“Illustrating Reinforcement Learning from Human Feedback(RLHF),”Hugging Face blog,December 9,20

207、22.Linvill,Darren,and Patrick Warren,“Understanding the Pro-China Propaganda and Disinformation Tool Set in Xinjiang,”Lawfare,December 1,2021.MacCarthy,Mark,“Transparency Is Essential for Effective Social Media Regulation,”Brookings Institution,November 1,2022.Macpherson,Lisa,“Lies,Damn Lies,and Gen

208、erative Artificial Intelligence:How GAI Automates Disinformation and What We Should Do About It,”Public Knowledge,August 7,2023.Maiberg,Emanuel,”AI Image Detectors Are Being Used to Discredit the Real Horrors of War,”404 Media,October 13,2023.Marcellino,William,Nathan Beauchamp-Mustafaga,Amanda Kerr

209、igan,Lev Navarre Chao,and Jackson Smith,The Rise of Generative AI and the Coming Era of Social Media Manipulation 3.0:Next-Generation Chinese Astroturfing and Coping with Ubiquitous AI,RAND Corporation,PE-A2679-1,September 2023.As of February 26,2024:https:/www.rand.org/pubs/perspectives/PEA2679-1.h

210、tmlMarcelo,Philip,“Fake Image of Pentagon Explosion Briefly Sends Jitters Through Stock Market,”Associated Press,May 23,2023.Marcus,Gary,replies to U.S.Senate queries,June 13,2023.Marcus,Gary,and Anka Reuel,“The World Needs an International Agency for Artificial Intelligence,Say Two AI Experts,”The

211、Economist,April 18,2023.Meaker,Morgan,“Slovakias Election Deepfakes Show AI Is a Danger to Democracy,”Wired,October 3,2023.Meehan,Elizabeth,“Transparency Wont Be Enough for AI Accountability,”Tech Policy Press,May 17,2023.National Center for Missing and Exploited Children,“Child Sexual Abuse Materia

212、l,”webpage,undated.As of February 26,2024:https:/www.missingkids.org/theissues/csam Nelson,Jason,“ChatGPT Wrongly Accuses Law Professor of Sexual Assault,”Emerge,April 7,2023.NewsGuard,“About NewsGuard,”webpage,undated.As of February 26,2024:https:/ Viral Image of Pope Francis Wearing a White Puffer

213、 Coat Is Totally Fake,”Forbes,March 26,2023.Okerlund,Joanna,Evan Klasky,Aditya Middha,Sujin Kim,HannahRosenfeld,Molly Kleinman,and Shobita Parthasarathy,Whats in the Chatterbox?Large Language Models,Why They Matter,and What We Should Do About Them,University of Michigan,April 2022.OpenAI,“Sora,”webp

214、age,undated.As of March 12,2024:https:/ System Card,March 23,2023.Panditharatne,Mekela,Daniel I.Weiner,and Douglas Kriner,“Artificial Intelligence,Participatory Democracy,and Responsive Government,”Brennan Center for Justice,November 3,2023.Paul,Katie,“Bipartisan U.S.Bill Would End Section 230 Immun

215、ity for Generative AI,”Reuters,June 14,2023.22Perrino,John,“Platform Accountability and Transparency Act Reintroduced in Senate,”Tech Policy Press,June 8,2023.Posard,Marek N.,Todd C.Helmus,Michelle Woods,and Bilva Chandra,The 2024 U.S.Election,Trust,and Technology:Preparing for a Perfect Storm of Th

216、reats to Democracy,RAND Corporation,PE-A3073-1,forthcoming.Public Law 118-31,National Defense Authorization Act for Fiscal Year 2024,December 22,2023.Qi,Xiangyu,Yi Zeng,Tinghao Xie,Pin-Yu Chen,Ruoxi Jia,Prateek Mittal,and Peter Henderson,“Fine-Tuning Language Models Compromises Safety,Even When User

217、s Do Not Intend to!”arXiv,October 5,2023.RAND Corporation,“Countering Truth Decay,”webpage,undated.As of March 29,2024:https:/www.rand.org/research/projects/truth-decay.htmlRando,Javier,Daniel Paleka,David Lindner,Lennart Heim,and Florian Tramr,“Red-Teaming the Stable Diffusion Safety Filter,”arXiv,

218、November 10,2022.Reddit,“r/ChatGPTJailbreak,”webpage,undated.As of February 26,2024:https:/ Sankar Sadasivan,Keivan Rezaei,Aounon Kumar,Atoosa Chegini,Wenxiao Wang,and Soheil Feizi,“Robustness of AI-Image Detectors:Fundamental Limits and Adversarial Attacks,”arXiv,September 29,2023.Sample,Ian,“What

219、Are Deepfakesand How Can You Spot Them?”The Guardian,January 13,2020.Sanger,David E.,and Steven Lee Myers,“China Sows Disinformation About Hawaii Fires Using New Techniques,”New York Times,September 11,2023.Sarkar,Daivi,“GPT3 vs GPT4-Battle of the Holy Grail of AI Language Models,”ProjectPro blog,Oc

220、tober 12,2023.Shea,Gabrielle,Sabine Neschke,Sean Long,and Tom Romanoff,“Tech Policy Trifecta:Data Privacy,AI Governance,and Content Moderation,”Bipartisan Policy Center,July 18,2023.Swenson,Ali,“FEC Moves Toward Potentially Regulating AI Deepfakes in Campaign Ads,”PBS NewsHour,August 10,2023.Synthes

221、ia,homepage,undated.As of February 26,2024:https:/www.synthesia.io Thierer,Adam,and Neil Chilson,“The Problem with AI Licensing and an FDA for Algorithms,”FedSoc Blog,June 5,2023.Thorley,Tom,“Advances in Hashing for Counterterrorism,”Global Internet Forum to Counter Terrorism,March 29,2023Ulmer,Alex

222、andra,and Anna Tong,“Deepfaking It:Americas 2024 Election Collides with AI Boom,”Reuters,May 30,2023.Walter,Kyle,Testing Multimodal Generative AI:Generating Election Mis-and-Disinformation Evidence,Logically,2023.Wanless,Alicia,and Jacob N.Shapiro,A CERN Model for Studying the Information Environmen

223、t,Carnegie Endowment for International Peace,November 2022.White House,“Biden-Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI,”fact sheet,July 21,2023.Williams,Kaylee,“Tightening Restrictions on Deepfake Porn:What US

224、Lawmakers Could Learn from the UK,”Tech Policy Press,July 25,2023.Yao,Jia-Yu,Kun-Peng Ning,Zhen-Hui Liu,Mu-Nan Ning,and Li Yuan,“LLM Lies:Hallucinations Are Not Bugs,but Features as Adversarial Examples,”arXiv,October 4,2023.Yu,Ning,Vladislav Skripniuk,Sahar Abdelnabi,and Mario Fritz,“Artificial Fin

225、gerprinting for Generative Models:Rooting Deepfake Attribution in Training Data,”arXiv,October 7,2021.Zhou,Jiawei,Yixuan Zhang,Qianni Luo,Andrea G.Parker,and Munmun De Choudhury,“Synthetic Lies:Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human Solutions,”Proceedings of t

226、he 2023 CHI Conference on Human Factors in Computing Systems,April 2328,2023.23AcknowledgmentsWe thank Grace Evans,legislative analyst at RAND,for her helpful guid-ance.We also thank the leadership of the RAND Technology and Secu-rity Policy Center,Jeff Alstott and Emma Westerman,for their guidance

227、on this publication.Special thanks go to the reviewers of this report,Doowan Lee,of Trust in Media and Georgetown University,and Marek Posard,of RAND,who offered helpful critiques.Any and all errors in this report are the sole responsibility of the authors.About the AuthorsTodd C.Helmus is a senior

228、behavioral scientist at RAND and a profes-sor of policy analysis at the Pardee RAND Graduate School.Helmus specializes in the use of data and evidence-based strategies to under-stand and counter disinformation and extremism.Helmus holds a Ph.D.in clinical psychology.Bilva Chandra wrote this report w

229、hen she was a Technology and Security Policy Fellow at RAND.She is currently a senior policy adviser at NIST,where she leads synthetic content policy efforts.Prior to RAND,she led product safety efforts for DALL-E(image generation)at OpenAI and led disinformation and influence operations(as well as

230、election integrity)efforts at LinkedIn.Chandra has an M.A.in security studies.www.rand.orgPE-A3089-1About This PaperThis paper highlights the ecosystem of generative artificial intelligence(AI)threats to information integrity and democracy and potential policy responses to mitigate the nexus of thos

231、e evolving threats.We focus on the information environment and how generative AIsuch as large language models or AI-generated images and audiois able to accel-erate existing harms on the internet and beyond.The policy options that could address these complex problems are vast,varying from much-neede

232、d social media reforms to using federal agencies to create sweeping standards for AI-generated content.The goal of this paper is to provide a meaningful overview of the risks that generative AI pres-ents to democratic systems,as well as tangible and detailed whole-of-government and societal solution

233、s to mitigate these risks at scale.FundingFunding for this work was provided by gifts from RAND supporters.Technology and Security Policy CenterRAND Global and Emerging Risks is a division at RAND that develops novel methods and delivers rigorous research on potential catastrophic risks confronting

234、humanity.This work was undertaken by the divisions Technology and Security Policy Center,which explores how high-consequence,dual-use technologies change the global competition and threat environment,then develops policy and technology options to advance the security of the United States,its allies

235、and partners,and the world.For more information,contact tasprand.org.RAND is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure,healthier and more prosperous.RAND is nonprofit,nonpartisan,and committed to th

236、e public interest.Research IntegrityOur mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior.To help ensure our research and a

237、nalysis are rigorous,objective,and nonpartisan,we subject our research publications to a robust and exacting quality-assurance process;avoid both the appearance and reality of financial and other conflicts of interest through staff training,project screening,and a policy of mandatory disclosure;and

238、pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations,disclosure of the source of funding of published research,and policies to ensure intellectual independence.For more information,visit www.rand.org/about/researc

239、h-integrity.RANDs publications do not necessarily reflect the opinions of its research clients and sponsors.is a registered trademark.Limited Print and Electronic Distribution RightsThis publication and trademark(s)contained herein are protected by law.This representation of RAND intellectual proper

240、ty is provided for noncommercial use only.Unauthorized posting of this publication online is prohibited;linking directly to its webpage on rand.org is encouraged.Permission is required from RAND to reproduce,or reuse in another form,any of its research products for commercial purposes.For information on reprint and reuse permissions,please visit www.rand.org/pubs/permissions.For more information on this publication,visit www.rand.org/t/PEA3089-1.2024 RAND Corporation