《挪威消費者協會:生成式人工智能對消費者的危害(英文版)(75頁).pdf》由會員分享,可在線閱讀,更多相關《挪威消費者協會:生成式人工智能對消費者的危害(英文版)(75頁).pdf(75頁珍藏版)》請在三個皮匠報告上搜索。
1、1Norwegian Consumer CouncilJune 2023Ghost in the machineRunning headerGHOST IN THE MACHINEAddressing the consumer harms of generative AIJUNE 2023TABLE OF CONTENTSEXECUTIVE SUMMARY 51 INTRODUCTION 61.1 An overview of generative artificial intelligence 7 1.1.1 Examples of generative AI models 81.1.2Th
2、egenerativeartificialintelligenceactorchain111.1.3Opensourceorclosedsourcemodels 12 1.1.4 General purpose AI 121.2 Consumer applications 132 HARMS AND CHALLENGES OF GENERATIVE ARTIFICIAL INTELLIGENCE 142.1 Structural challenges of generative AI 152.1.1IdentifyingtheconcreterisksofgenerativeAI 152.1.
3、2Technologicalsolutionism 172.1.3Concentratingpowerinthehandsofbigtech 172.1.4Opaquesystemsandlackof accountability 192.2 Manipulation 222.2.1Mistakesandinaccurateoutput222.2.2ThepersonificationofAImodels 232.2.3Deepfakesanddisinformation252.2.4DetectingAI-generatedcontent 272.2.5Generativeartificia
4、lintelligenceinadvertising282.3 Bias,discrimination,and content moderation 29 2.3.1 Bias in training data 29 2.3.2 Content moderation 302.4 Privacy and data protection 312.4.1Privacychallengesrelatedtodatasetsusedformodeltraining312.4.2Privacychallengesrelatedtogeneratedcontent322.5 Security vulnera
5、bilities and fraud 322.6 Replacing humans in consumer-facing applications with generative AI,wholly or in part 33 2.6.1Challengesrelatedtocombininghuman-andautomateddecision-making33 2.7 Environmental impact 342.7.1Climateimpact34 2.7.2 Water footprint 362.7.3Greenwashing&hopesforGreenAI37 2.8 Impac
6、t on labour 372.8.1Labourexploitationandghostwork372.8.2Labourautomationandthreatstojobs 38 2.9 Intellectual property 383 REGULATIONS 403.1 Data protection law 45 3.1.1Datasubjectrights 46 3.1.2TheItalianDPAsdecisionconcerningChatGPT 47 3.2 Consumer law 483.2.1ConsumerlawinaU.S.setting 49 3.3 Genera
7、l product safety law 503.3.1TheGeneralProductSafetyDirective503.3.2TheGeneralProductSafetyRegulation 50 3.4 Competition law 513.5 Content moderation 51 3.6 The draft Artificial Intelligence Act 52 3.6.1 The EU Commissions proposal 523.6.2TheEUCouncilspositionontheAIA53 3.6.3 The EU Parliaments posit
8、ion on the AIA 533.6.4TheArtificialIntelligenceActmustprotectconsumers54 3.7 Liability 543.7.1ProductLiabilityDirective543.7.2RevisedProductLiabilityDirective553.7.3AILiabilityDirective55 3.8 Industry standards and guidelines 564 THE WAY FORWARD 574.1 Consumer rights principles that are key for safe
9、 and responsible AI 594.2 Policy recommendations 604.2.1Callstoactionandempowermentofenforcementagencies604.2.2Decisionmakersstrategicmeasures614.2.3Newlegislativemeasures61ENDNOTES 65Ghost in the machine Addressing the consumerharmsofgenerativeAINorwegianConsumerCouncilJune 2023www.forbrukerradet.n
10、o/aiDesignby:VonKommunikasjon5Norwegian Consumer CouncilJune 2023Ghost in the machine Executive summaryTherehasbeenanexplosionofconsumerfacinggener-ativeartificialintelligenceservices.Theseapplicationscanbeusedtogeneratesynthetictext,images,sound,orvideothatcloselyresemblehuman-madecontent.Asgenerat
11、iveAIsystemsbecomeintegratedintopopu-larplatformsandtools,adoptionofthetechnologyispoisedtokeeprising.Meanwhile,anumberofemergingchallengeshavespurrednumerousdebatesabouthowtoensurethatgenerativeAIissafe,reliable,andfair.Thisreportisacontributiontothesediscussions,andaimstoprovidepolicymakers,lawmak
12、ers,enforcementagencies,andotherrelevantentitieswitharobuststart-ingpointtoensurethatgenerativeAIdoesnotcomeattheexpenseofconsumerandhumanrights.Wecannotknowforcertainhowthetechnologywilldevelopinthemonthsandyearstocomebutbelievethatthedirectionoftechnologicaladvancementshouldhappenonsoci-etysterms.
13、Therefore,wepresentanumberofoverar-chingprinciplesthatcanhelpdefinehowgenerativeAIsystemscanbedevelopedandusedinaconsumer-andhuman-centredway.Wealsostronglyurgegovernments,enforcementagen-ciesandpolicymakerstoactnow,usingexistinglawsandframeworksontheidentifiedharmsthatautomatedsystemsalreadyposetod
14、ay.Newframeworksandsafe-guardsshouldbedevelopedinparallel,butconsumersandsocietycannotwaitforyearswhiletechnologiesarebeingrolledoutwithoutappropriatechecksandbalances.ThefirstchapterofthisreportprovidesanexplanationofgenerativeAIanditsuses,alongsideseveralexamplesand illustrations.Inchaptertwo,wesu
15、mmarizevariouscurrentandemergingchallenges,risks,andharmsofgenerativeAI.Thisincludeschallengesrelatedto Power,transparency,andaccountability,wrongorinaccurateoutput,usingtechnologytomanipulateormislead consumers,biasanddiscrimination,privacyandpersonalintegrity,securityvulnerabilities,automatinghuma
16、ntasks,environmentalimpact,labourexploitation.Chapterthreecontainsanoverviewofthepatchworkofexistingandupcomingrulesandregulationsthatmayapplytothedevelopment,deployment,anduseofgener-ativeAIsystems.ThisismostlycentredaroundEUlegis-lation,butwithsomereferencestoongoingprocessesinthe United States.Th
17、efinalchaptercontainsnumerousrecommendationsonhowtoaddresstheproblematicissuesofgenerativeAI.Thisincludes:Enforcementofexistinglawsand regulations,ensuringsufficientresourcesfor enforcementbodies,strongerconsumerprotection,robustgovernmentpolicies,newlegislativemeasures,strongobligationsthatcoverdev
18、elopersand deployers of generative AI systems.Executive summary1.INTRODUCTION7Norwegian Consumer CouncilJune 2023Ghost in the machine1.IntroductionConsumerfacingartificialintelligencesystemshavebeenaroundinvariousformsfordecades,andareusedforpersonalizingsocialmedia,filteremails,recommendstreamingco
19、ntent,texttranslation,andmuchmore.Someofthesepurposesarebenignanddiscreet,andmostpeoplemayneverrealizethattheyareinteractingwithanAI-poweredsystem.AnewwaveofAI-poweredsystemsinconsumerfacingapplicationsisfastapproaching,withthemassdeploy-mentandadoptionofgenerativeartificialintelligence(generativeAI
20、)systems.GenerativeAIisasubsetofartificialintelligencethatcangeneratesyntheticcontentsuchastext,images,audio,orvideo,whichcancloselyresemblehuman-createdcontent.Suchsystemsarepoisedtochangemanyoftheinterfacesandcontentconsumersmeettoday.InNovember2022,aprototypeofthechatbotChatGPTwasreleasedtothepub
21、lic.Theapplicationquicklygainedworldwideattention,becomingthefastestgrowingdigitalserviceofalltimewithinamonthofitsrelease.1Inthefollowingmonths,otherservicesforgeneratingtext,images,sound,andvideo,werequicklydeployedanditeratedupon,sparkingasortofarmsraceforgenerativeAIsystems.Consumerswereprovided
22、accesstothesecontentgeneratorsdirectlyinwebin-terfaces,whilecompaniesstartedtoembedthecontentgeneratorsintheirapplicationsandservices.ThesuddenandwidespreaddeploymentandadoptionofgenerativeAIsystemssparkedpublicdiscourseaboutthepromisesandperilsofthetechnology.ThedebatehasrangedfromhowgenerativeAIma
23、ybeusedtoincreaseefficiencyintheworkforceandtosparkcre-ativity,tohowitcanbeusedtospreaddisinformation,manipulateindividualsandsociety,displacejobs,andchallengeartistscopyright.Thediscussionabouthowtocontrolorregulatethesesystemsisongoing,withpolicymakersacrosstheworldtryingtoengagewiththepromisesand
24、challengesofgenerativeAI.Thisreportisacontributiontothesediscussionsbyprovidingananalysisofthemostpressingissuesfromtheconsumerangle,alongwithanumberofpossiblesolutionsandwaystoproceedfrombothalegal,ethical,andpoliticalperspective.Althoughwedonotpretendtohavetheanswerstoallthequestionsraisedbygenera
25、tiveAI,webelievethatmanyoftheemergingorongoingissuescanbeaddressedthroughacombina-tionofregulation,enforcement,andconcretepoliciesdesignedtosteerthetechnologyinaconsumer-andhuman-friendlydirection.As the development of generative AI seems to move at a breakneckpace,thedescriptionsthroughoutthisrepor
26、tmustbeseenasasnapshotofanemergingtechnology.ThereportwaswrittenbetweenFebruaryandMay2023,anddoesnotincludeanynewinformationfrompaperspublishedafterJune1st.TheNorwegianConsumerCouncilisapubliclyfunded,independentconsumerorganization,thatrepresentsconsumerinterests.Thisreportwaswrittenwithcontri-buti
27、onsfromBEUC,MiikaBlinnfromVZBV,KrisShrishakfromtheIrishCouncilforCivilLiberties,DanielLeuferfromAccessNow,JonWorth,MarijaSlavkovik,andAnjaSalzmannfromtheUniversityofBergen.1.1 AnoverviewofgenerativeartificialintelligenceGenerative AI isablankettermusedtodescribealgorith-micmodelsthataretrainedtogene
28、ratenewdata,suchastext,images,andsound.Whiletheseapplicationsrelyondifferenttypesofinputdata,thegeneralprinciplesbehindhowtheyaretrainedaresimilar.TheemergenceofadvancedgenerativeAIispossibleduetoanenor-mousamountofcontentavailableontheinternet,com-binedwithadvancesinmachinelearningandcomputingpower
29、.GenerativeAImodelsworkbyanalysinglargeamountsofinformationtopredictandgeneratethenextwordinasentence,featureofanimage,etc.Thisisdonebydetectingpatternsinandrelationshipsbetweendatapointsinthetrainingdata,whichinturnallowsthesystemtoreplicatesimilarpatternstogeneratesyn-theticcontent,forexampleapiec
30、eofwriting,music,oravideoclip.Thisprocesscanalsobedescribedasacomplexmash-upofcontentfromthedatathesystemwastrainedon.Inotherwords,theyarepredictivemod-8Norwegian Consumer CouncilJune 2023Ghost in the machine1.1Anoverviewofgenerativeartificialintelligenceelsthataretrainedto“connectthedots”betweendat
31、apointsinexistingcontenttogeneratesyntheticcontent.Generatedcontentisprobabilisticallyandrandomlygen-eratedbasedoncertaininput(orprompts),whichareusuallywrittenbyahuman.Therefore,theoutputofanygivengenerativeAImodelislikelytobedifferentforeachpersonpromptingthemodelandmaybothresemblepatternsinthetra
32、iningdataorappeartobesomethingcompletelynew.1.1.1 EXAMPLES OF GENERATIVE AI MODELSTherearevarioustypesofgenerativeAImodels,includ-inglargelanguagemodels(LLM)thatcanrespondtotextbygeneratingnewtext,andmulti-modalmodelsthatcangeneratemorethanonetypeofoutputorrespondtomorethanonetypeofinput,forexamplec
33、hatbotsthatcanalsogenerateimageswhenpromptedtodoso.AshortintroductionofthemostpopulargenerativeAImodelsonthemarkettodayispresentedbelow,accom-paniedbysomerelevantexamples.1.1.1.1 Text generatorsText generators are a type of generative AI model that cangeneratetextpassagesbasedonpredictiveanalysis,wh
34、icharebuiltonlargelanguagemodels.2 These models are usually trained on enormous amount of textscrapedfromtheinternet,includingbooks,forums,newssites,socialmedia,etc.Textgeneratorscanbeusedforwritingessays,coding,chatbots,andaugmentingsearchengines,amongstotherthings.Inmanycases,textgeneratorsaremean
35、ttogeneratetextthatappearstobewrittenbyahuman,forexamplebygeneratingtextwritteninafirst-personperspective,usingemojis,orbywritingtextindicatingthatithasthecapabilitytoexperiencehumanemotions.Sometextgeneratorsaremultimodalandcangeneratetextbasedonimages.Although text generators have existed in somef
36、ormforseveralyears,forexampleaspredictiontoolsfortypingtextmessag-es,thediscussionaroundthetechnologygainedmomentumduringtheautumnof2022,withthepublicreleaseofChatGPT,whichisownedandoper-atedbythecompanyOpenAI(whichisalsotheownerofDALL-E,seebelow).ChatGPT3isavailableonlineforthosewhocreateafreeaccou
37、nt,whilethemorepower-fulChatGPT4modelisavailableforamonthlysubscrip-tion fee.3 InJanuary2023Microsoftannouncedamajorinvest-mentinChatGPTandlaunchednewfeaturesintheBingsearchenginepoweredbythetechnology.4MicrosofthasannouncedthatitintendstointegrateChatGPTintoitsotherservices,includingtheMicrosoftOff
38、icesuiteofapplications,forexampletoautomaticallytakenotesduringmeetingsinMicrosoftTeams.5GooglehasalsodevelopedaLLMthatcangeneratetext,calledLaMDA.InthewakeofMicrosoftsinvestmentinChatGPT,GooglerolledoutsimilarfeaturestoitssearchenginewithatextgeneratorcalledBard.6 Google is alsoplanningtointroducev
39、ariousAI-poweredfeaturessuchasdraftingandsummarizingemails,aswellasbrainstormingandwritingdocumentsinitsWorkplaceapplications.7 2LargelanguagemodelsaresophisticatedAImodelsthataredesignedto generatetextthatresembleshumanlanguage.Theyarenormallytrainedonvastamountsoftextsourcesto“learn”patternsandgra
40、mmar.LLMscanbeusedfortaskssuchasmachinetranslation,sentimentanalysis,hu-man-machineinteraction,proofreading,andmanyotherpurposes.Poem about the consumer issues of generative AI,ChatGPT.9Norwegian Consumer CouncilJune 2023Ghost in the machineMetahasdevelopedtheLLMGalacticatrainedonsci-entificarticles
41、andmaterials,whichismeantto“store,combineandreasonaboutscientificknowledge”.AfterthemodelwasreleasedasapublicdemoinNovember2022,thepublic-facingdemowasquicklyremovedduetogeneratingtextcontainingmultipleerrorsandbiases.8 InFebruary2023,MetareleasedanotherLLM,calledLLaMa(Large Language Model Meta AI).
42、LlaMa is an open sourcemodel,whichwasinitiallyreleasedtoresearchersbasedonanaccessapplicationprocess.InMarch2023themodelwasleakedonapublicmessageboard,mean-ingthatanyonewitharelativelypowerfulcomputercandownload,use,andadaptthemodel.9 TherearealsoseveralopensourceLLMsthataredevel-opedandmaintainedby
43、smalleractors.Forexample,thetextgeneratorBLOOMisavailablethroughthecompanyHuggingFace,10whileStabilityAIhasreleasedopenmod-elsunderthemonikerStableLM.111.1.1.2 Image generatorsGenerative AI models that are trained to generate images canbecollectivelyclassifiedasimagegenerators.Theycancreateimagesfro
44、mtext prompts(text to image)or from existing images(image to image).Image generators workbyanalysinghugeamountsofexistingpictures,suchasphotographs,paintings,etc.,whichareoftenscrapedfromvarioussourcesontheinternet.By training the algorithm on thesedatasets,themodelcangenerate images of different ob
45、jects(achair,atrain),people(ayoungwoman,JerrySein-feld),andstyles(impressionism,inthestyleofEdwardMunch).Themostwidelyusedimagegenerators as of June 2023 are Midjourney,12 DALL-E,13 and Sta-ble Diffusion.14 MidjourneyisavailablethroughthechatserviceDiscord.ItispossibletojointheofficialMidjourneyDisc
46、ordserv-er,toaskaMidjourneybotto“imagine”apicturebasedonvariousprompts.Forexample,thepersonpromptingthesystemcouldtype“/imagine hyper realistic photo of political advisor writing a paper on generative ai,in an open office plan”.Thebotrespondsinthechatwithfourgeneratedpictures.WhileMidjourneywasfreet
47、otryforthefirstfewmonthsafteritsrelease,foralimitednumberofgeneratedimages,ithassincebecomeapaidsubscriptionservice.ThecompanyMidjourneyInc.ownsthegenerativeAImodelthatgeneratestheimagesandrunsandcontrolsboththemodelitselfandtheserversthatitishostedon.Thismeansthatthecompanycanrestrictaccess,changet
48、hemodel,andaddcontentfilterstocontrolwhatkindofimagesthemodelcanandcannotgenerate.ThegenerativeAImodelDALL-EisaccessiblethroughthewebsiteofitsownerOpenAI.Individualscancreateanaccountandreceivealimitednumberoftokenseachmonththatcanbeusedtogenerateimages.ImagesareHyper realistic photo of political ad
49、visor writing a paper on generative ai,in an open office plan,Midjourney.1.1Anoverviewofgenerativeartificialintelligence10Norwegian Consumer CouncilJune 2023Ghost in the machinegeneratedbyenteringdifferentpromptsintothewebsiteinterface.Ifthepersonrunsoutoftokens,theycanpaytoreceivemore.LikeMidjourne
50、y,themodelbehindDALL-Eisowned,controlled,andoperatedbyitsparentcompany.ThegenerativeAImodelStableDiffusionisdevelopedbythecompanyStabilityAI.UnlikeMidjourneyandDALL-E,StableDiffusionisanopensourcemodelthatcanbefreelydownloadedbyanyone,anddoesnotrequireasubscriptionoraccesstotheinternettouse.Oncethen
51、ecessarysoftwareisdownloaded,unlimitedimagescanbegeneratedlocally.RunningStableDiffusionlocallyonlyrequiresacomputerwithrelativelypowerfulcon-sumer-gradegraphicscard.StabilityAIonlytrainsanddistributesthebasicmodelsforStableDiffusion,whileitispossibleforanyonewithaccesstothemtocontinuetraininganddev
52、elopingnewmodelsthatarebasedontheoriginalStableDiffusionmodels.Thesenewmodelscanthenbedistributedtoothers.Inpractice,thismeansthatthecompanydoesnotcontrolthemodelnoritsoutput.1.1.1.3 Audio generatorsAudiogeneratorsusegenerativeAItechnologiestocreateaudioclipsbasedontextprompts(forexampletexttovoice)
53、.Suchmodelsaretrainedonexistingvoicedata,music,etc.AudiogeneratorscanbeusedtocreateAI-generatedmusic15andvoices,andtherearemodelscapableofrecreatingthevoiceandpitchofindividualindividuals.16 Forexample,thecompanyElevenLabshasreleasedamodelthatallowsanyonetoconvertshorttextinputintovoiceclips,inasele
54、ctionofdifferentvoices.17Micro-softhasannouncedthegenerativeAImodelVALL-E,whichthecompanyclaimscangeneraterealisticvoicesbasedonathreesecondvoicesample.18AsofMay2023,VALL-Ehasnotbeenreleasedtothepublic.1.1.1.4 Video generatorsVideogeneratorscanbeusedtocreatevideoclipsbasedontextprompts(texttovideo),
55、images(imagetovideo),orexistingclips(videotovideo).Asitismorecompli-catedtogenerateauthenticlookingvideofootagethantomakestillimages,thistechnologyissomewhatlessdevelopedatthetimeofwriting.However,thismaychangeinthenearfuture,asseveralmajorcompaniesareactivelyworkingonmodelsforgenerating video.An oi
56、l painting by Munch of a policy advisor writing a paper on generative AI,DALL-E.A photo of a consumer policy advisor writing a paper about the consumer challenges of generative AI,in an open office plan,Stable Diffusion 1.5.1.1Anoverviewofgenerativeartificialintelligence11Norwegian Consumer CouncilJ
57、une 2023Ghost in the machineMeta has developed a model to turn short texts into video clips,20andGooglehasannouncedasimilarsystem.21 As ofMay2023,neitherofthesesystemshavebeenmadeavailabletothepublic.StabilityAI,thecompanybehindStableDiffusion,hasreleased a model for generating animations from text
58、prompts and images.22ThecompanyRunwayhasre-leasedamobileappthatcanbeusedtogenerateshortvideoclipsfromexistingvideos.231.1.2 THE GENERATIVE ARTIFICIAL INTELLIGENCE ACTOR CHAINFromassemblingdatasetsfortrainingmodelstode-ployingandpromptinggenerativeAIsystems,therearepotentiallymanydifferentactors.Thes
59、eactorsmayallinfluencethesystemorhowitisusedinvariousways.Therelevantactorsareshownintheillustrationjustbelow.Eachboxrepresentsadifferentactorintheactorchain.Theseactorsmayallbewithinoneorganisationbutwilloftenbespreadacrossseveralorganisations.Thearrowsareonlypointingonewaytomaintainasimplerepre-se
60、ntation;itisofcoursepossiblethattherecouldbefeedbackloopsbetweendifferentactors.ThedatasetassemblercollectsandsystemizesthenecessarydatatotrainagenerativeAImodel.Therearenumerousavailableopensourcedatasetsthathavebeencompiledandlabelledbyscrapingahugenumberofsourcesonline.Inmanycases,suchdatasetsare
61、compiledforresearchpurposes,andmadefreelyavailable.Acom-panydevelopingagenerativeAImodelcanthustraintheirmodelondatasetsthatsomeoneelsehasassembled.ThedevelopersofgenerativeAImodelscreateabaselinemodel,whichmaythenbetrainedandtunedforcertainmorespecificcontextsorapplicationsbydownstreamde-velopers.I
62、nsomecases,thisfine-tuningofthemodelmaybedonebythesameactorasthetrainingofthebaselinemodel,whileinothercases,thefine-tuningmaybedonebyaseparateactorentirely.Thismaybeanothercompany,orinthecaseofopensourcemodels,themodelcanbefine-tunedbyanyonewitharelativelypowerfulcomputer.Tofurthercomplicatethematt
63、er,asgeneral-purposegenerativeAImodelsarebeingintegratedintootherapplications,thecompanyorentitydeployingthesystemmaybeseparatefromthecompanydevelopingthemod-eland/orfine-tuningit.Finally,thereareendusers,engagingwiththedeployedmodel.Inconsumerusecases,theconsumerwilltypi-callybeanactor,bypromptingt
64、hemodeltogenerate,forexample,atextoranimage.Consumersmayalsoindi-rectlyencountergenerativeAIsystemswheninteractingwithabusiness,forexampleifacustomerserviceagentusesatextgeneratortogenerateanswerstoconsumerqueries,orifaconsumerispromptedtomakecertainqueriesbasedonAI-generatedrecommendations.Thereare
65、numerousactorsinthegenerativeAIsystemactorchain.ItiscrucialtounderstandtherelationshipbetweentheseactorstounderstandhowgenerativeAIshouldberegulated,andatwhichpointtheactorchaindifferent harms arise.Illustration of different actors in the generative AI actor chain.Data set assemblerDeveloper of base
66、linemodelDownstream developerDeployerEnd userFrame from video generated in Make-A-Video,Meta AI.191.1Anoverviewofgenerativeartificialintelligence12Norwegian Consumer CouncilJune 2023Ghost in the machine1.1.3 OPEN SOURCE OR CLOSED SOURCE MODELSTherearesignificantdifferencesinhowgenerativeAImodelsared
67、istributedandcontrolled.Manymodelsareproprietaryclosedsourcemodelsthatrunoncloudserverscontrolledbytheownerofthesystem(the“sys-temowner”).Thismeansthatconsumerscanaccessthemodelthroughtheinternet,andthatthesystemprovidercanchangethemodelatanytime,addcontentfilters,re-strictaccess,etc.Forclosedsystem
68、s,thesystemownerprovidestheprocessingpowerrequiredbothtotrainthemodel,andtogeneratesyntheticcontent.ForaclosedsourcegenerativeAImodel,itisnotpos-sibletoknowhowthemodelworks,whatdataitwastrainedon,andhowparametersareweighed,unlessthecompanybehindthemodelpublishessufficientdocu-mentationorprovidesacce
69、sstothirdpartyauditors,enforcementagencies,orresearchers.Inmanycases,suchinformationmaybekeptsecretduetosecurityand/orbusinessinterests.Ontheotherhand,somegenerativeAImodelsarereleasedasopensource,whichmaytakevariousforms.Differentpartsofthesystem,suchasthedataset,sourcecode,modelparametersandweight
70、s,maybemadeavailabletothirdparties.Whenthesourcecodeismadeavailabletothepublic,itcanbeused,studied,tested,modified,anddistributedbyanyone.Thismeansthatitcanbeinspectedforerrorsandvulnerabilities.Itcanalsobeimprovedanditerateduponcollaboratively.Using,modifying,anddistributingopensourcesoftwareisgene
71、rallygovernedbylicenseterms.Giventheopensourcesoftwareandthedataset,anyonewithsufficientcomputingresourcecouldrepro-ducethegenerativeAImodel,althoughthecomputingresourcesneededtodosoaresignificantenoughthatinpracticeitwilllikelybelimitedtolargecompanies.However,forgenerativeAIsystemstobetrulyopensou
72、rce,themodelitselfshouldbemadeavailabletothepublic.Insuchacase,applicationsdevelopedbasedonthesemodelsmightalsobereleasedasopensourceap-plication,suchastheimagegeneratorStableDiffusion,oritcanbeadaptedintoaclosedsourceapplication.OpensourcegenerativeAImodelscanbedownloadedbyanyone.Withapowerfulenoug
73、hcomputer,itcanbeusedto generate data and update the model as one pleases.Thesourcecode,parameters,etc.ofsuchmodelscanbeinspectedbyanyone.However,thisdoesnotnecessarilymeanthattheycanunderstandhowthemodelworks,assufficientlycomplexmodelsmaybemoreorlessimpos-sibletounderstandtheinnerworkingsof.Oncean
74、opensourcegenerativeAImodelisreleasedtothepublic,thereispracticallynothingthedeveloperofthebaselinemodelcandotoinfluencehowthemodelfunctions.Thismeansthatanycontentfiltersandotherartificiallimitersplacedonthemodelmaybealteredorremovedbydownstreamdevelopersordeployers.Thiscreatesbothbenefitsandsignif
75、icantdrawbacks,whichwillbeelaboratedonbelow.1.1.4 GENERAL PURPOSE AIWhilesomeAImodelsaredesignedwithaspecificpurposeandusecaseinmind,suchasearlydiscoveryofcancercells,manygenerativeAImodelsareexamplesofso-calledgeneralpurposeartificialintelligence.Thismeansthatthebasicsystem,suchasalargelanguagemode
76、l,istrainedtobeabletorespondtoavastvarietyofsituationsandinteractionsandcanbeadaptedtobeusedinnewcontexts.Unlikeamodelwithaspecificpurpose,itisextremelydifficultorimpossibleforthedevelopersofageneralpurposeAImodeltoforeseethepossibleusesandabus-esofthetechnology.Thismakesitparticularlyimportantthats
77、uchmodelsaresubjecttotechnical,scientific,legislative,andregulatoryscrutinybeforetheyarewidelyadopted.However,applicationssuchasChatGPThavealreadybeenreleasedtothewiderpublicwithoutrigorousevaluation,impactassessmentorscrutiny,whilebeingincreasinglyopaqueandinaccessibletothirdpartyauditorsandresearc
78、hers.24Itisworthconsideringwhetherthisleadstoadesirablefuture,consideringthemanyharmstoucheduponinchapter2ofthisreport.1.1Anoverviewofgenerativeartificialintelligence13Norwegian Consumer CouncilJune 2023Ghost in the machine1.2Consumerapplications1.2 ConsumerapplicationsSeveraldifferenttypesofgenerat
79、iveartificialintelli-gencearepubliclyavailabletoconsumerstoday.Manyofthesearereadilyavailabletousebyanyonewithaninternetconnection,anddonotrequireexperttechnicalknowledgetouse.Someofthemaredirectlyaccessiblethroughwebinterfaces,whilegenerativeAItechnologyisalsoincreasinglybeingintegratedintootherser
80、vicessuchasonlinesearch,learningandadministrationsoft-ware,andsocialmedia.AsofMay2023,themostpopularconsumerusesforgenerative AI models are text and image generation.However,withmajorconsumer-facingcompaniessuchasMicrosoft,Meta,andGoogleinvestingheavilyinthetechnology,theusecasesarelikelytoexpandint
81、hecomingmonths,asgenerativeAImodelsareimplement-edinvariousservices.Forexample,textgeneratorscanbeausefultooltostreamlineand/oroptimizemundanetasks,functioningasakindofmulti-purposedigitalassistant.Thismayincludechanginginternetsearchfunctions,automatingcertaintaskssuchaswritingcode,transcribingvoic
82、emessages,orpersonalizingservicesinvariousways.Whilesuchapplicationsmaybeusefulandefficientincertaincontexts,therearealsosignificantrisksanddrawbacks,whichwillbeexploredfurtherinthefollow-ingchapters.Asthetechnologyisdevelopedandadopted,generativeAImaybeusedtoautomatetediousandtime-consum-ingprocess
83、esthatpreviouslyhadtobedonemanually,forexamplebywritingconcisetexts,fillinginforms,generatingschedulesorplans,orwritingsoftwarecode.Ithasthepotentialtomakeservicesmorecost-efficient,whichmayalsolowercostsforconsumers,forexamplewhensolicitinglegaladvice.25Ontheotherhand,theproliferationoflow-costAI-g
84、eneratedcontentmayre-placehumanlabourandhuman-generatedcontent,thusloweringthequalityofconsumer-facingservicessuchascustomersupport.Thetechnologyalsoopensnewavenuesforconsumermanipulationinareassuchasad-vertisingorproductrecommendationsandcanfacilitateorobfuscatediscriminatorypractices.2.HARMS AND C
85、HALLENGES OF GENERATIVE ARTIFICIAL INTELLIGENCE“From violations of privacy and personal integrity to the creation of fraud and misinformation,generative AI models introduce vast risks and challenges,while turbocharging others.”15Norwegian Consumer CouncilJune 2023Ghost in the machineTherehavebeensev
86、eralcontroversiessurroundingthedevelopmentanduseofgenerativeartificialintelli-gence.Fromviolationsofprivacyandpersonalintegritytothecreationoffraudandmisinformation,generativeAImodelsintroducevastrisksandchallenges,whileturbochargingothers.Concreteandhighlyrelevantexamplesofthisarechatbotsandsearche
87、nginesprovid-ingincorrectbutconvincinginformation,theabuseofcheaplabourintheglobalsouthforcontentmoderation,andasignificantenvironmentalimpactduetoresourceconsumption.Itisessentialthattheseproblemsaresuf-ficientlyaddressedbyenforcing,applying,andestablish-inglawsandregulationsthatservetoprotectconsu
88、mersfromvariousnegativeconsequences.TheissuesdiscussedthroughoutthisreportareoftennotneworuniquetogenerativeAI.Algorithmiccomput-ersystemshaveexistedforacentury,whilethetech-nologypopularlyreferredtoasartificialintelligencehasbeenaroundsincethe1950s.Inthe1960s,thecomputerscientistJosephWeizenbaumcre
89、atedELIZA,amodelthatsimulatedhumaninteraction,usingrule-basedalgorithms.26PeoplewhointeractedwithELIZAat-tributedhumanattributesandemotionstothemodel,eventhoughtheywereinformedthatthesystemhadnosuchcapability,mirroringsomeusecasesofgenerativeAI-poweredchatbotstoday.Issuesrelatedtocontentmoderation,a
90、lgorithmicbias,privacy,anddisinformationhavebeendebatedatalmosteveryjunctionasdigitaltechnologyevolvesandiswidelyused.However,thedeploymentandpublicadoptionofsystemssuchasChatGPT,bothfortechnicallyadeptconsumersandthegeneralpublic,alongsideitseaseofuseandwide-scaleavailability,meansthatmanyoftheseis
91、sueshavebecomeurgentlyrelevanttoanalysefromaconsumerperspective.Asdescribedinthefollow-ingchapters,anumberoftheseissuesmaybeaddress-ablebyenforcingapplicablelawsandregulations,whileother may require other solutions or remedies.2.1 StructuralchallengesofgenerativeAIAtabasiclevel,generativeAImodelsare
92、fundamentallydesignedtoreproduceexistingmaterial,althoughinpotentiallynovelways.Thismeansthatsuchmodelsareinherentlypronetoreproducingexistingbiasesandpow-erstructures.Therefore,whilethemodelshavenoun-derstanding,mind,orintentionoftheirown,thedecisiontodevelop,deployandusethemisinherentlypolitical.I
93、tisnotsufficienttoascribeneutralityorobjectivitytotheoperationsoroutputsofagenerativeAImodel,becauseitstrainingdataandalgorithmsstemsfromhumanbe-ings,withallthatthisentails.AsgenerativeAImodelsarebeingintroducedintoallsectorsofsociety,sofarwithlittleornoregulatoryoversight,therearefundamentalissuest
94、hatneedtobeaddressed.GenerativeAImodelsaredependentonlargeamountsofdatathatistakenfromamultitudeofsources,usuallywithouttheknowledgeorconsentoftheoriginatorofthedata,beitapieceofart,anewsarticle,oraselfie.Informationissiphonedandgatheredtobeusedindifferentways,withanendgoalofenrichingasmallnumberofc
95、ompanies.Thisraisesquestionsofvaluedistribution,usagepermission,privacy,account-ability,intellectualproperty,andhumanrights.27 2.1.1 IDENTIFYING THE CONCRETE RISKS OF GENERATIVE AIAswithanynewtechnologies,thediscoursearoundgen-erativeAIismuddledwithamixoffacts,concerns,andalot of hype and enthusiasm
96、.28ManyAIsystemsarebeingtoutedasbeingcapableofsolvingalmostanytask,oftenwithoutevidencetobackuptheclaims,aphenomenonthatcanbedescribedasAIsnakeoil.29 When addressing problematicandhazardousissueswiththetechnology,itisimportanttobeabletosortfactsfromfiction.2.Harmsandchallengesofgenerativeartificiali
97、ntelligence“At a basic level,generative AI models are fundamentally designed to reproduce existing material,al-though in potentially novel ways.This means that such models are inherently prone to reproducing existing biases and power structures.”16Norwegian Consumer CouncilJune 2023Ghost in the mach
98、ine2.1StructuralchallengesofgenerativeAIHeavilypublicizedwarningsaboutthedangersofarti-ficialintelligencehaveconcentratedonhypotheticalrisksofdevelopinganartificialgeneralintelligence(AGI),meaningasystemthatisabletoperformintellectualtasksthatarecomparabletotheabilityofhumanbeings.Theoretically,such
99、systemsshouldbeabletothinkandreason,andbeabletoperformabroadrangeoftasksthatequalhumancapacityforthinking.Thisdifferscon-siderablyfromgenerativeAImodels,whichhavenosuchcapacities.AsAGIsystemsdonotcurrentlyexist,andthereareseriousdisputesaboutwhethertheycaneverberealized,suchsystemswillnotbefurtherco
100、nsideredin this report.Therehavebeencallsforvoluntarymoratoriumsorpausing of developing generative AI models.Some of thesecallshavefocusedonapotentialfuturewhereAIsystemshavebecomesopowerfulthattheyposeanex-istential threat to humanity.30Whilesuchcallsacknowl-edgeissuesrelatedtoaccountability,safety
101、,andcontroloverAIsystemsingeneral,thereareseriousconcernsthatfocusingonpotentiallong-termscenariosaredraw-ingattentionawayfrommanyofthecurrentpressingissuesofgenerativeAI,potentiallyleavingtheseissuesinsufficientlyregulated.31 Theargumentthatahypotheticalgeneralartificialintelli-genceisanexistential
102、threattohumanityimpliesthatconcernsaboutcurrentissuessuchasdiscrimination,privacy,andfairness,areinconsequentialandmarginal.32 Inotherwords,thenarrativeconcerningapotential“AIsupermind”mayserveasadistractionfrompressingissuesthatarealreadypresentintodaysapplicationofgenerativeartificialintelligence.
103、Itiscrucialthatnarra-tivesabouthypotheticalexistentialthreatstohumanitydonotcomeinthewayofproposingconcretesolutionstotheveryrealissuesposedbythetechnologythatexists today.33“There are serious concerns that focusing on potential long-term scenarios are drawing attention away from many of the current
104、 pressing issues of generative AI,potentially leaving these issues insufficiently regulated.”17Norwegian Consumer CouncilJune 2023Ghost in the machine2.1.2 TECHNOLOGICAL SOLUTIONISMArtificialintelligenceisoftenlaudedasthesolutionforavastnumberofissuesacrosssectors,fromhealthcareandpublicadministrati
105、ontolegalassistance.Whilethisnarrativeisattractivetobothprivateenterpriseslookingtosellsoftwaresolutions,andtopolicymakerssearchingforsimpleremediestopoliticalorregulatoryailments,itneedscriticalexamination.Thebeliefthatalmostanyissuecanbeimprovedorsolvedusingtechnologyisknownastechnologicalsolution
106、ism.AtermcoinedbythetechcriticEvgenyMo-rozov,technologicalsolutioniststendtoglossovercom-plexandmultifacetedsocialproblemsinfavourofsimplemathematicalorengineeringsolutions.34Thisreduc-tionistbeliefisattractivetoservice-providersbecauseitallowsthemtoadvertisemiraclecuresAIsnakeoilandtopolicymakersbe
107、causetechnologicalquickfixesaretangibleandusuallyappearmorecost-efficientthanexaminingcomplicatedandoftendeep-rootedsocialandpoliticalconflictsorinequalities.AsMorozovpointsout,technologicalsolutionismisdangerousbecauseitoftensimplydoesnotwork.Bypresentingmultifacetedandcomplexissuesasamereengineeri
108、ngissuetobesolvedinalab,solutionismmis-representssocialproblemsandmissestheunderlyingcauses.Whenpresentedasproblemsthatcanbesolvedbytechnology,solutioniststendtodisregardthesocial,political,andculturalcontextthatisthebackboneofoursocieties.Whenconsideringtheproliferationofartificialintelli-gencemode
109、lsthatarerapidlybeingdeployedacrosssectors,itisworthkeepingthefollyoftechnologicalsolutionisminmind.ThisisparticularlyimportantifgenerativeAImodelsorsimilartechnologiesarebeingpushedasaremedyorsolutiontoinequalities,suchasprovidingaccesstomentalhealthtoolstopeoplewhootherwisewouldnotbeabletoaffordit
110、.Whileitmayappearattractivetooutsourcementalhealthcaretoalargelanguagemodelthatcanbedeployedandaccessedatarelativelylowmonetarycost,thisapproachrisksreducingthecomplexityofmentalhealthandthevalueofhumancontacttoaquestionofpredictiveanalysisand language modelling.35Similarly,beforedecidingtodeployate
111、xtgeneratorasasolutionforoverworkedcasehandlersinthepublicsector,forexample,itiscrucialtoconsiderthecontextandcausesoftheproblem,ratherthanadoptingdevelopingtechnologiesasablanketsolution.Ifsuchtechnologicalquickfixesareadoptedattheexpenseofinvestmentintoproveneffectivemeasures,whichareoftencostlyan
112、ddifficulttoimplement,thismaycomeatsignificantcoststomarginalizedgroupswhoriskbeingdeprivedofeffectivetreatmentandmeasuresbecausetheirissuesarepurportedlybeingaddressedbytheuseoftechnology.2.1.3 CONCENTRATING POWER IN THE HANDS OF BIG TECHAtthebaseofthediscoursearoundgenerativeartificialintelligence
113、isaquestionofpower.GenerativeAImodelsareproductsofculturalandpoliticalcontexts,acontextthatisembeddedineverythingfromthedecisiontode-velopthemodel,thechoiceoftrainingdata,thetuningofmodels,andthegivenpurposesfordeployment.Assuch,thealreadypowerfulcanpotentiallyentrenchexistingpowerstructuresthrought
114、hetechnology,whilethedisenfranchisedwillremainsounlessthereisoutsideintervention.ThisbecomesapparentwhenagenerativeAImodelgeneratesbiasedordiscriminatorycontent,butalsomanifestsinaspectssuchascontentmoderationpracticesandinwhohasaccesstothesystems.AsgenerativeAImodelsareoftentrainedondatacol-lectedf
115、romanyavailablesources,someactorsarerais-ingquestionsaboutwhetherprivatecompaniesshouldbeallowedtousethecollectiveknowledgeofhumanitytoturnaprofit.Thevastamountofinformationthatcanbefoundopenlyavailableonlinecanbedescribedasadigitalcommons,asitisabodyofresourceswherepracticallyeveryoneisacontributor
116、,fromindividualpiecesofdatatothepublicinfrastructureoftheinter-net.Ifthedigitalcommonsaresiphonedtodevelopandtrainproprietarymodels,thisraisesethicalconcernsabouthowvaluegeneratedonthebasisofthesecom-monresourcesshouldbedistributed.36Theseconcernsextendtodatagovernanceissuesregardingwhoshouldcontrol
117、howdataisused,suchasifatechcompanywantstocommercializeAImodelstrainedonindigenouslanguages.37ThequestionofwhocontrolsthedevelopmentandtrainingofgenerativeAImodelsandhowtheyareusedisoffundamentalimportance.Thosewhocontrolthetechnologyhavesignificantpotentialtocreatede-pendencies,setthetermsofuse,andd
118、ecidewhohasaccess.Thisentrenchmentofpowercreatesoverarch-ingconcernsaboutleadingtechcompaniesbecominggatekeepersthatcanexcluderivalsandotherwiseabusetheirincreasinglydominantmarketpositions.38 While 2.1StructuralchallengesofgenerativeAI18Norwegian Consumer CouncilJune 2023Ghost in the machineopensou
119、rcemodelsmaylowerthebarrierofentryforcertaintypesofgenerativeAI,39suchmodelsarestilllargelydependentonafoundationalmodelthathasbeendevelopedbyactorswithaccesstosignificantcomput-ingresourcesandtrainingdata.ThismeansthatalreadydominanttechnologycompaniessuchasMicrosoft,Google,andMeta,arewellpositione
120、dtoseizethemarketforgenerativeAIs.Withproprietaryclosedmodel,thesystemownerhascontroloverwhocanaccessthetechnology,whatitcosts,itsfeatures,andhowitmaybeused.Thismayinturnaffectaca-demia,attractingwhatcouldhavebeenindependentresearcherstoworkwithinthecloseddomainsofbigtechnologycompanies,wheretheyhav
121、eaccesstostate-of-the-arttechnologyinthefield.Overall,thismeansthatthetechgiantscanfurtherleveragetheirdominantpositionsacrossdifferentonlinemarketsinthefieldofgenerative AI.40 WithonlyafewgenerativeAImodelsavailableonthemarket,thesemodelsareintegrated into a variety of services,providingthe modelow
122、nerswithsignifi-cantpower.Modelscanbepatchedorotherwisemodi-fied,functionalitycanbeaddedorremoved,andcontentcanbebanned,filtered,orotherwiserestricted.Ifthesystemownersetsthetermsforhowitstechnologymayormaynotbeused,endusersorthird-partycompaniesintegratingthemodelareatthemercyoftheowner.Somecompeti
123、tionconcernsmaybesomewhatalleviatedinthecaseofopensourcegenerativeAImodels,whicharenotnecessarilybeholdentothebusinessmodel,ob-jectives,orwhimsofamodelsoriginalcreator.However,eveninsuchcases,manycompaniesdonothavethemeanstocompetewithbigtechfirmsinofferinggenera-tiveAIsolutionstoconsumers.Largecomp
124、aniesbenefitsignificantlyfromnetworkeffects,asmoreusersmeansmoredata,whichleadstobetterservices.Incaseswheremodelsarefurthertrainedonconsumerinterac-tionsorfeedback,thecompaniescanfurtherimproveandfine-tunethemodelsataratethatisunattainableforsmallercompetitors.Dominantactorscanfurtherentrenchtheirp
125、owerbyintegratinggenerativeAIintotheirownservicesthatarealreadyusedbymillionsacrosstheworld.Forexample,byrollingoutitschatbotBardaspartofitssearchen-gine,Googlealreadyhasamassiveglobaluserbasethatcanbeleveragedtoboosttheadoptionofthechatbot.Similarly,asMicrosoftimplementsChatGPT-basedmod-elsintoitsO
126、fficesuiteofapplications,thecompanyalreadyhasauserbasethatcompetitorscanonlydreamof.CompaniescanalsoleveragetheirmarketpositionbymakingtheusageofagenerativeAImodeldependentonusingadifferentservicefromthesamecompany,bybundlingservicestogether.Forexample,tohaveaccesstoMicrosoftsBingchatbotfunctionalit
127、y,consumersmustuseMicrosoftsEdgebrowser.41 TheintegrationofgenerativeAImodelsintoservicessuchassearchenginescanalsosignificantlylimitconsumerchoice.Forexample,inaregularonlinesearchengine,theconsumerispresentedwithnumeroussearchresultsthattheymaychoosebetween.Ifthesearchengineisreplacedbyatextgenera
128、torthatprovidesasingleanswertoanyquery,thispoten-tially limits the information available.Ifsimilarmodelsareusedforonlineshopping,thiscreatesnewavenuesforplatformstoself-preferenceproducts,byensuringthattheplatformspreferredproductistheonlyortheprimarilysuggestedpurchase.Iftheconsumerqueries“whatisth
129、ebestcoffeemachinefortosuitmyneeds?”,itwillbenecessarytomonitorandcontrolhowachatbotor“shoppingassistant”landsataparticularresultorrecommendation.2.1.3.1 Walled gardens and its downstream effectsInordertomaximizeconsumerengagement,manydig-italserviceprovidershaveafinancialincentivetokeepconsumersont
130、heirplatformsaslongaspossible.Thisgoalcanbeattainedbyintegratingandbundlingasmanyservicesintotheplatformaspossible,whilecreatingbarrierssuchasnotprovidingserviceinteroperabilitytodisincentivizeconsumerstoleavetheplatform.Plat-formsandservicesdesignedtokeeptheconsumerfromleavingarecalledwalledgardens
131、.42 Theintegrationofgenerativeartificialintelligenceintovariousplatformsalreadyseemstofacilitateawalled2.1StructuralchallengesofgenerativeAI“Those who control the technology have significant potential to create dependencies,set the terms of use,and decide who has access.”19Norwegian Consumer Council
132、June 2023Ghost in the machinegardenapproachthatmayhaveseriousanticompetitiveeffectsonbothdirectcompetitorstothelargeonlineplatforms,andacrossmarkets.SnapchatisforinstanceintroducingrecommendationsforrestaurantsorrecipesinitsAI-chatbot,43whichwouldreduceconsumersneedtoaccessotherservices,suchastradit
133、ionalsearchengines,forthesekindsofqueries.Thisislikelyasignofthingstocome,asmajorplatformsare slated to integrate generative AI models into their services.Ascompaniescompetetodevelop“killerapps”,servicesthatintegrateasmanyfunctionsandpurpos-esaspossible,theseproblemsaresettoexacerbate.Newcomersmayfi
134、nditincreasinglydifficulttoprovidestand-aloneservicestoconsumers,astheyhavefewerreasonstoexittheirapplications.Thiswouldservetoconcentratepowerwithalreadyestablishedactors,effectivelyharmingtheconsumermarket.TheintegrationofgenerativeAIintosearchenginesissparkingmajorconcernsforpublishersandadvertis
135、ers,sincesuchintegrationcaneffectivelycreatewalledgardens.44Withtraditionalsearchengines,theconsum-ercouldsearchforatopicandbepresentedwithalistoflinkstowebsitesthatcontaininformationaboutthetopic.Theconsumerwillthenclickononeormoreofthelinksandberedirectedtoawebsite.Consequently,thewebsiteownergene
136、ratesrevenuebydisplayingadstotheconsumer.ThisdynamicmaychangewiththeintroductionofgenerativeAI.Googleisforexampleexperimentingwithintroducinggenerated,summarizedcontentinitssearchengine,whichfillsupthefirstpageonasmallerscreen(suchasaphone).45Inshort,ifconsumerscansimplyaskachatbotaboutasubjectandre
137、ceiveanswersinthesameinterface,theincentivetovisitathird-partywebsiteisreduced.Iftheconsumerdoesnotvisitthethird-partywebsite,theoriginatorofthecontentcannotmonetizethecontentbydisplayingads.Alackoftrafficposesaproblemtopublishers,whosecontentmaybescrapedtodisplayinformationinthroughthechatbotinterf
138、ace,butwhomaystruggletomonetizethecontent.Thiscanthereforehavedownstreameffectsbyreducingtheincentivetoproducequalitycontent,po-tentiallyleadingtocontent-productionbecomingautoma-tizedaspartsofcost-cuttingmeasures.Itisalsowell-es-tablishedthatpowerconcentrationamongafewactorsisseldomlybeneficialtohe
139、althyconsumersmarkets.2.1.3.2 Data colonialismIf generative AI models are trained on data sets that wereindiscriminatelyscrapedfromtheinternet(digitalcommons),thismayalsoentaillargeamountsofdatafrom indigenous and other minority groups.The informa-tioncanthenberepackagedandusedinnewways,forexampleto
140、selltechnologyorservicesbasedonthedatabacktothegroupsfromwhomthedataoriginated.Theprocessbywhichorganizationsandcorporationsclaimownershipoverdataproducedorharvestedfrompeopleiscalleddatacolonialism.Theconceptofdatacolonial-ismishighlyrelevantwhendiscussingtheoperationsofgenerative AI models.Forexam
141、ple,indigenouscommunitiesinNewZealandhaveexpressedconcernsregardingthedevelopmentoflargelanguagemodelsbeingtrainedonhundredsofhoursofMoriindigenouslanguage.Communityleadersandresearchersfearthat“ifIndigenouspeoplesdonthavesovereigntyoftheirowndata,theywillsimplybere-colonizedinthisinformationsociety
142、”.46Thelanguageharvestedwithoutconsentcanbedistort-edandleadtoabuseanddeprivecommunitiesoftheirrights.Accordingtoindigenouscommunities,itisnotforBigTechtoplaywithsuchheritage.2.1.4 OPAQUE SYSTEMS AND LACK OF ACCOUNTABILITY Modelssuchaslargelanguagemodelsaregenerallyverytechnologicallycomplex,butthey
143、arenotimpossibletounderstandorexplain.Therearefundamentalscientif-icprinciplesrelatingtotransparency,peerreviewandrigorousqualitycontrolthatapplyinfieldssuchasthepharmaceuticalandaviationindustries,whichshouldalsoapplytodevelopersofAImodels.Informationabouthowtrainingdataiscollected,howthedataislabe
144、lled,howtestingisperformed,whatdecisionsaremadere-gardingcontentmoderation,andtheenvironmentalandsocialimpactsofthemodelsarejustafewareaswheretransparencyisnecessarytoensurethatrisksaremiti-gatedandthatclaimsaboutthetechnologyareaccurate.48 HypotheticalAIsystemthatdemonstrateshuman-levelintelligence
145、andautonomy.Doesnotcurrentlyexist.2.1StructuralchallengesofgenerativeAI20Norwegian Consumer CouncilJune 2023Ghost in the machine2.1.4.1 Opaque systems reduce accountabilityUnfortunately,therearealreadytendenciesfromcertainAIdeveloperstocloseofftheirsystemsfromexternalscrutiny.Forexample,Googlehascom
146、mittedtoapolicychangewherethecompanywillonlysharepapersaftertheirresearchhasbeenturnedintoproducts.47MicrosoftresearchershavemadegrandclaimsaboutitsownAIsystemsshowsignsofartificialgeneralintelligence,48 whilenotprovidingresearcherswithaccesstothemodelinordertoverifyordisputetheclaims.49Finally,Chat
147、GPTownerOpenAIhasclaimedthatthecompanysAIsystems,includingwhattrainingdataisused,howthemodelworks,etc.,shouldnotbeopenforexternalreviewbecausegivingothersaccesswouldposeacompetitionandsafetyrisk.50Whilethelackoftransparencyisanissuethatappearsacrossthespectrumofthesoftwareindustry,OpenAIsowndescript
148、ionoftherisksoftheirproductstendtoborderonexistential,withitsCEOSamAltmanstatingthatthecompanyis“scared”bythepotentialharmsthatmaystemfromitsownsystems.51Suchclaimsasapre-tenceforclosingdowngenerativeAIsystemsforexternalauditingandreviewareworryingtendenciesthatcanmaskanumberofdownstreameffects,andw
149、hichposeenforcementagenciesandresearchersmajorchallenges.ResearchersatPrincetonUniversityhaveclaimedthatOpenAImightbemisrepresentingthecapabilitiesoftheirsystems,butthisisimpossibletoproveduetothesystembeingclosedtoexternalscrutiny.52Theresearch-erswarnthatthisalsosignificantlyhindersattemptsatrepro
150、ducibilityofanyclaimsmadebythecompany.53 2.1.4.2 Trade agreements as barriers to transparencyWhilecompaniesthemselvesareattemptingtocloseofftheirsystemsfromexternalscrutiny,lawmakersmaybeincreasinglylimitedfromrequiringtransparencybytradeagreements.InternaldocumentsfromtheEUCommis-sionshowthatdigita
151、ltradeagreementsbetweentheEUandUSlimitEuropeanlawmakersabilitytorequirethird-partyaccesstothesourcecodeofAI.54Closed-doornegotiationsaffectinglawmakersabilitytocreateawell-bal-ancedandconsumer-friendlymarketarehighlyproblematic.Thisbarscivilsoci-etyandotherstakeholdersfromprovidingimportantinputanda
152、ppearsatoddswithcrucialdemocraticprinciples.2.1.4.3 Actor chain transparencyThelackoftransparencyalsobecomesproblematicwhenserviceprovidersimplementthird-partygenera-tiveAImodelsintotheirservices.Thismayincreasetheriskoferrorsorunexpectedbehaviourfromthemodel.55 Thedeveloperofthebaselinemodeldoesnot
153、necessarilyseeorunderstandthedownstreamcontextsinwhichthemodelisused,whiletheserviceproviderorotherdownstreamdevelopersdonotsufficientlyunderstandthe limitations of the model.Iftheserviceproviderisnotprivytothedatasetsusedtotrainthemodels,ortohowthemodelactuallyworks,theserviceproviderwillnotbeablet
154、ogivetheconsumeranexplanationaboutwhyacertainoutputwasgener-ated.AssupplychainsforgenerativeAIsystemsmaybecomplex,withoneactorcollectingandlabellingdatasets,whileitcanbeotheractorsdevelopingthealgo-rithms,trainingthemodel,orintegratingitintoservices,itbecomesdifficulttoattributeliabilityandaccountab
155、il-itytotherightentity.Fortheconsumer,thismayhaveanegativeimpactontherighttoanexplanation,aswellascontestabilityandgeneraltransparencyobligations.Forexample,theretailbank,payments,andshoppingserviceKlarnahasannouncedacollaborationwithOpenAI,withplanstointegrateChatGPTintoitsservicestoprovidea“highly
156、personalizedandintuitiveshoppingexperiencebyprovidingcuratedrecommendations”.56 Ifthissystemprovidesflawedrecommendationsforproducts,orranksproductsinaskewedway,itwillbeessentialthatconsumers,nottomentionenforcementagencies,areabletoaccessandassessdataonhowtherecommendationsystemaffectstheconsumer.T
157、hisbecomesimpossibleifOpenAIasathird-partyserviceproviderdoesnotprovideexternalactorsthenecessaryinformationabouttheAIsystem.Withoutsuchinforma-tion,itisentirelyplausiblethattheservicesandprod-uctsshouldnotbeonthemarketatall.2.1StructuralchallengesofgenerativeAI“There are fundamental scientific prin
158、ciples relating to transparency,peer review and rigorousquality control that apply in fields such as the pharmaceutical and aviation industries,which should also apply to developers of AI models.”21Norwegian Consumer CouncilJune 2023Ghost in the machine2.1.4.4 Opaque systems exacerbate consumer harm
159、s and hinder consumer rightsThegenerallackoftransparencyinsomegenerativeAIsystemsmayhavesignificanteffectsonconsumers.AsgenerativeAIsystemsarebeingadoptedbyconsumersforvarioususecases,thepotentialforharmsrises,aselaboratedinothersectionsbelow.Forexample,manytextgeneratorsarepronetoprovidingfalseorin
160、accurateinformation.Thismayhavedirecteffectsonconsumers,forexampleifachatbotprovidesbadfinancialadvice.Theoften-complicatedactorchainsbehindaconsumerfacinggenerativeAIsystemmayalsomakeitexceedinglydifficultforconsumerstogetintouchwiththeresponsibleentityincasesomethinggoeswrong.Thiscouldalsobeproble
161、maticwhenitcomestoclaimsforcompensation.Withoutacertaintransparencyintohowthesystemworks,suchaslimitationsonthesystemsintendeduse,alongsidedisclosuresaboutpossibleinaccuracies,thepotentialforharmbecomeslarger.Theotherharmsthatarecoveredinthefollowingsubsectionsofthisreportareexacerbatedwhenconsumers
162、arekeptinthedarkofthe systems potential for harms and harmful uses.Itisimportantthatcompaniesprovidetransparentsys-temsandapplicationtoconsumers.However,thepowerasymmetrybetweencompaniesandconsumersindigitalenvironments,57meansthatanytransparencymea-suresdirectedatconsumerstoreduceharmsmustbeimpleme
163、ntedinadditiontoothermeasures,ratherthanasastand-alonemeasure.TheresponsibilityofensuringfairandlegitimateuseofgenerativeAImustbeonthecompanies,andnevershiftedontoconsumersthroughtransparencymeasures.2.1.4.5 The limits and restrictions of corporate AI ethicsEthicalandlegalconsiderationsplayafundamen
164、talroletoensurethatmodelsaredeveloped,trained,deployed,andusedinaresponsibleway,fromthedevelopmentstageandthroughoutthelifecycleofthemodel.Asethicalnormsandvaluesdiffersignificantlydepend-ingonculturalcontexts,itisalsoworthnotingthatthedecisionofwhichethicalstandardstoconsiderandapplyisapoliticalcho
165、ice.Similarly,legalframeworksarenotuniversal,whichmayproveaserioushurdleasgenera-tiveAImodelsarerolledoutonaglobalscale.WhilemanycompaniesworkingongenerativeAImodelshaveemployedAIethicsteamstohelpdefineguardrailsandredlinesforAIdevelopment,therearedoubtsabouthoweffectivethishasbeenincaseswhereethica
166、lcon-cernsconflictwiththecompanysprofitmotives.Famously,GooglefiredmembersofitsAIethicsteamafterresearchersfromtheteampublishedthepaperOn the Dangers of Stochastic Parrots:Can Language Models Be Too Big?.Thepaperposedcriticalquestionsabouthowlargesuchmodelsshouldbe,alongsidecriticaleval-uationsofinh
167、erentbiasesandtheenvironmentalimpactofthemodels.Afterrefusingtoretractthepaper,theresearcherswereaskedtoresignfromthecompany.58Amongstthetechcompanylayoffsin2022/2023,AIethicsorresponsibleAIteamsatcompaniesincludingGoogle,Twitter,Microsoft,andMetawerealsolaidoff.Thisraisesconcernsaboutwhetherethical
168、concernsareignoredorheavilydownprioritizedbycompaniescompetinginagenerativeAIgoldrush.59Somecompaniesarecallingforregulationofgenera-tiveAI,notablyOpenAI,60seeminglywishingtoabidebylawmakersrequirements.Atthesametime,OpenAIhasthreatenedtoleavetheEUiftheprovisionsofthenewAIActaretoostrict.61Thisindic
169、atesadesirefromcompa-niestoshaperegulationsinaccordancewiththeirownprofitmotives.2.1StructuralchallengesofgenerativeAI“As ethical norms and values differ significantly depending on cultural contexts,it is also worth noting that the decision of which ethical standards to consider and apply is a polit
170、ical choice.”22Norwegian Consumer CouncilJune 2023Ghost in the machine2.2 ManipulationGenerativeartificialintelligencemodelshavethecapabilitytogeneratesyntheticcontentthatcloselyresemblesrealcontent,includingdialogue,voices,photographs,andvideo.Studieshaveshownthattextgeneratorsthatsimulatehumandial
171、oguecaninfluencepeoplesfeelings,dispositions,andopinions.62 As gener-ativeAImodelbecomemorepowerful,thepotentialformanipulationbecomesgreater.Lowqualitycontentcanalsomisleadormanipulate,bothonpurposeandbecauseoflowqualitydataandmodels.IfagenerativeAImodelproducesinaccurateorfalseinformation,thiscanh
172、aveharmfulconsequencesforconsumers.Ifsuchmodelsaredeployedandusedmaliciously,thismayleadtoconsumersbeingtricked,misled,orotherwisemanipulated.2.2.1 MISTAKES AND INACCURATE OUTPUTGenerativeAImodelsarecomplexsystemstrainedonvastamountsofmaterial,whichmaygiveanimpressionofinfallibility.However,asthemod
173、elsdonot“under-stand”contextandthecontentitproduces,theyhaveatendencytoproducecontentthatlooksconvincingandcorrectbutisfactuallyincorrect.Thisparticularlyapplies to text generators.Forexample,ChatGPTcanproducetextthatlooksveryconvincingandfact-basedbutcontainsfactualerrorsorfallacies.63Thishasledcri
174、ticstocallthesystema“confidentbullshitter”.64Similarly,GoogleemployeeshaslabelledthecompanysowntextgeneratorBarda“pathologicalliar”.65Itcanbedifficultforthepersonpromptingthesystemtonoticeorrevealtheseerrorsiftheyarenotalreadyfamiliarwiththefactsoftherelevantsubject.Somesystems,suchasBing,citessourc
175、esforthegeneratedinformation,apparentlytoalleviatesomeoftheseissues.However,themodelshavebeenproneto“makeup”non-existingsources,eitherthroughpresentingsourcesthatdonotinfactexistorpresentingsourcesthatdocontaintherelevantcontenttosupportthegeneratedcontent.66 Mistakesandinaccuraciesareexacerbatedasg
176、en-erativeAImodelsarepluggedintotheworkstreamindifferentareas.Amidstsinkingrevenue,soonafterthewidespreadadoptionofChatGPT,publisherswerequicktoannouncethattheywouldstartusingthemodelforcontentproduction.67However,whenthenewssiteCnetusedatextgeneratortogeneratejournalisticcontent,itwassoondiscovered
177、thatthepublishedoutputwasriddledwithfactualerrors.68TherearealsoconcernsthattheuseofgenerativeAImodelsasareplacementfortraditionalinternetsearchengineswillmakeitsignifi-cantlyhardertoidentifyinaccurateorincorrectinforma-tion,whilealsohavingnegativeeffectsoninformationliteracy.69 Aslargelanguagemodel
178、sbecomeincreasinglysophis-ticated,theycanadoptmoreauthoritativeandconvinc-ingsyntax.Combinedwithadjustmentsofanswerstoincreasepersuasivenessandengagement,itbecomesmoredifficulttodetectmistakes.Whilefactualerrorsmaybeironedoutthroughtechnologicaladvancements,thismayalsomakeithardertoknowwheninformati
179、onisincorrect.Forexample,ifaLLMprovidedsophisticatedandaccurateanswers99times,itbecomesharderfortheendusertoknowthatitwasinaccurateorcompletelywrongthe100thtime.Inaccuratelygeneratedinformationcouldhaveharmfulconsequences,bothasstandalonemodelsandwhenthegenerativeAIisembeddedinothersystems.For Bing
180、pretending to feel good.(23.03.2023)2.2Manipulation23Norwegian Consumer CouncilJune 2023Ghost in the machine2.2Manipulationexample,ifanAI-poweredchatbotisusedbyaconsumertosolicitmedicaladvice,andtheadviceiswrong,thiscouldleadtoreallifeharm.Similarly,textgeneratorsarereportedlybeingusedbyconsumersfor
181、mentalhealthpurposes,whichmayalsohaveseriousconsequences,alsobecausethemodelsdonotfollowanyethicalorlegalguidelines or rules.70Finally,textgeneratorsthatareusedtofindinformationaboutconsumerrightsmayendupprovidingfalseinformationthatenduprenderingthecon-sumerunawareoforunabletoexercisetheirlegalrigh
182、ts.InMarch2023,thePortuguesegovernmentannouncedthatitwoulduseanadaptedversionofChatGPTtopro-videlegaladvicetocitizens.71 Although the model is only meanttoprovidegeneraladviceincertainareas,andwillnotreplacedecisionmakers,itshouldbeexpectedthatendusersareconditionedtotrusttheoutputofthemod-elregardl
183、essofitsactualfactualaccuracy.Whensuchmodelsareusedbypublicinstitutions,theadditionalveneeroflegitimacymaymakeerrorsevenhardertode-tect.Thisisalsoacontextwheremistakeswilladverselyaffectpeopleinavulnerablesituation,whosereasonforaccessingtheinformationistheirneedforlegaladvice.Suchvulnerabilitymayal
184、soenhanceotherrisks,suchastheriskofbeingmisled.Iforganizationswithinthepressor inthepublicsectorbegindeploy-ing and relying on generative AI models,theproductionoffalse,misleading,orinaccurateinfor-mationcanbecomeasignificanttrustissue.Forexample,ifagovernment-promotedservicegivescitizensbadlegaladv
185、ice,thishasariskoferodingtrustinpublicinstitutions.Similarly,anewspaperusingatextgeneratortoproducearticlescontainingfalseinforma-tion undermines readers faith in theveracityofallinformationthepaperpublishes,perhapseventhepressmorebroadly.2.2.2 THE PERSONIFICATION OF AI MODELSManyconsumersarealready
186、becomingusedtointeractingwithgenerativeAImodels.Suchmodelsare often designed to emulate humanspeechpatterns,behaviours,andemotions.Thiscreatessignificantpotentialformanipulationanddeception,whichmayexploitandunderminecognitivefreedoms.72 LargelanguagemodelssuchasLaMDAorChatGPTaretrained on enormous
187、amounts of text gathered from the internet,whichmeansthattheyhavehugerepositoriesofdatatodrawpredictionsfrom.Thisalsomeansthatthemodelsareabletosimulatehumanpatternsinthetextsthataregeneratedafterall,theymayhavebeentrainedonavastnumberofconversationsbetweenrealpeople.Theexhibitionofhuman-likebehavio
188、ur,emotions and traits are not inherent to generative AI models,theseareattributesthatdeveloperscanchoosetoincludeornot.Forexample,theuseofcasualcon-versationallanguageandemojismaybeawaytoeaseconsumersintointeractingwithachatbot,butcanalsobeexploitedtomakeconsumersfeelguiltyaboutnottakingcertainacti
189、ons,manipulatethemintopayingforaservice,etc.TherearefundamentalissueswithreleasinggenerativeAImodelstothepublicwithoutplacingrestrictionsonitsabilitiestoemulatehumanbehaviour.73 If the model generatescontentthatsimulateshumanemotion,thisisinherently manipulative.My AI simulating human emotion and be
190、haviour.24Norwegian Consumer CouncilJune 2023Ghost in the machine2.2ManipulationAshumans,ourcognitivebiasesmakeusassignhumantraitsandabilitiestoanimalsorobjectsthatexhibitsomesignsofhumanity,suchasfacialexpressions,be-haviouralpatterns,orapparentpersonalitytraits.Thisisarecurringphenomenonforpeoplei
191、nteractingwithgen-erativeAImodels,particularlytextgenerators.Humansascribecommunicativeintentwhenonthereceivingendoforalorwrittennaturallanguage,regardlessofwhetherthecontributorhassuchintent.Thiscanoccurevenwhenoneisfactuallyawarethatthemodeldoesnotactuallyhavehumanattributes.74 Misunderstandingsab
192、outthecapabilitiesofgenerativeAImodelsarealsoinfluencedbydeliberatemarketingstrategiesfromthecompaniesdevelopingthemodels,75 andbyusingvagueormisleadinglanguagetodescribewhatthemodeldoes.76Finally,featuresofhuman-likebehaviour,suchasonusingemojisinconversationorgeneratingtextinthefirst-person,canals
193、oservetoincreaseconsumersattributinghumantraitstothemodels.77In2022,aGoogleengineererroneouslyclaimedpubliclythattheLaMDAchatbothadbecomesentient,i.e.,capa-bleoffeelinghumanemotions.78In2023,betatestersofBingssearchengineimplementationofChatGPTwereshockedtoseethemodelrespondtoquerieswithappar-entlyu
194、nhingedandmentallyunstablerants.79Bothcaseswerefollowedbydiscussionsaboutwhetherthemodelsmayhavebecomesufficientlyadvancedtoresemblehumanintelligence.Suchdiscussions,wherehumanemotionsandmotivesareassignedtoagenerativeAImodel,revealsafunda-mentalmisunderstandingofhowthistechnologyworks.Inreality,gen
195、erativeAImodelsarenotsentient,anddonot have feelings or desires.Generative AI models are predictivealgorithmicsystemsthatcanstatisticallypredicthowpiecesofdatafittogether.Thiscanbeexemplifiedbypredictivetext models that are stan-dardonmostsmartphones,wherethemodelistrainedtopredictorguessthenextword
196、inasequenceofwordsforexample,themodelmaypredictbasedonitstrainingthatthenextwordinthesentence“Ilove”islikelytobe“you”,or“coffee”,or“therain”.Amoresophis-ticatedmodelmaybeabletomoreaccuratelyguessthatbecausethesentenceispartofaconversationaboutItaliancuisine,“pasta”isthemostlikelynextwordinthesentenc
197、e.AsdescribedbyBenderetal.,“alanguagemodel is a system for hap-hazardlystitchingtogetherScreenshots from Replika.“There are fundamental issues with releasing generative AI models to the public without placing restrictions on its abilities to emulate human behaviour.If the model generates content tha
198、t simulates human emotion,this is inherently manipulative.“25Norwegian Consumer CouncilJune 2023Ghost in the machine sequencesoflinguisticformsithasobservedinitsvasttrainingdata,accordingtoprobabilisticinformationabouthowtheycombine,butwithoutanyreferencetomeaning:astochasticparrot”.80Arecentstudyha
199、sshownthathumansareunabletodistinguishbetweenhuman-generatedandAI-gener-ated text.81Accordingtothestudy,humanbeingsarenotequippedtoaccuratelyrecognizeAI-generatedlanguage,andtestsubjectswerepronetolabellingAI-generatedtextashumanatahigherratethanactualhuman-generatedtext.Thiscanbeexploitedtomanip-ul
200、ateordeceivepeoplebyemployingtextgeneratorsmasqueradingashumanbeings.Aspeoplemaytendtotrusthumansmorethanachatbot,advancedlanguagemodelsmaybesuitedtodeceiveconsumersintogivinguppersonalinformation,spendmoney,orperformcer-tainactions.82 Manipulation may happen due to the end user not knowingthattheya
201、reinfactinteractingwithamachine,butevenifthisismadeclearandobvious,anthropomor-phizedgenerativeAImodelscanstillbeeffectivetoolsformanipulation.Thiscanalsooccurincaseswherethe“human-like”behaviourisamainfeatureofthemodel,suchasAI-basedassistantsoremergingAIromanticcompanions.Forexample,theapplication
202、ReplikausesgenerativeAItosimulateapartner,oftenwithemphasisonroman-ticoreroticconversation.TheAImodel“remembers”conversations,simulatesfeelingsbyprofessinglovefortheconsumer,andappearstobesadifthepersonrarelyusestheservice.Therearemanymicrotransactionsintheapp,whichcanbepurchasedtounlockfeaturessuch
203、asnewpersonalities,“Selfies”(receiveexclusiveselfiesfromReplika),andevenavirtualmarriage.AllthesefeaturesadduptoahighlymanipulativeexperiencewheretheconsumersaresubjectedtoAI-generatedcommercialand emotional pressure.InFebruary2023,theItalianDataProtectionAuthorityfoundthatReplikawascollectingperson
204、aldatafromchildrenwithoutalegalbasis,andthatthecompanywasinbreachoftheGeneralDataProtectionRegula-tion(GDPR).Asaresponse,Replikaaddedsignificantrestrictionsinfeaturesoftheapp,wherefeatureswerediminishedorentirelyremoved.Reportedly,thecompan-ionwouldnolongerrememberpastconversationsandwouldrefusetota
205、lkaboutvarioussubjects.Asaresult,peoplewhoweresimulatingromanticpartnershipwiththeAIcompanionwereleftheartbrokenandbereft.83 Inthiscase,eventhoughReplikaneverpretendedthattheappwasanythingmorethananAIsystem,consum-ersneverthelessformedgenuinebondswithit,leadingtosignificantnegativepsychologicalimpac
206、toncethedeveloperchangedhowthesystemworked.ThesocialmediaplatformSnapchathasalsointroducedanAIcompanioncalledMyAI.84Itwasinitiallyintroducedasapremiumservice,butwithinfewmonthsthechatbotwasrolledouttoallusers,alongwithamessagealertingconsumersaboutthisnewfeature.Soonafterlaunch,MyAIwassubjecttosigni
207、ficantcriticismforitslackofguardrails.Forexample,themodelcheerilyofferedadvicetoaresearcherposingasa13-year-oldgirlthataskedabouthavingsexwitha31-year-oldpartner,whileajournalistposingasaminorreceivedadviceabouthowtomaskthesmellofalcoholandmarijuana.85 Inadditiontothesafetyissuesthatsuchcasesmaypose
208、,itisgenerallymorallyandlegallydubioustorolloutexperimentalAI-drivenfeaturesinanappusedbymanyminors.Therearealsopotentiallysignificantrisksposedbygivingpeople,particularlychildren,artificial“friendsasaservice”thattheymustpayasubscriptiontokeeptalkingto,orthatcouldbeplacedbehindapaywallatalatertime.T
209、herisksofchildreninteractingwithamachinethattheybelievetobehumanmayincludedevelopingunhealthyemotionaldependencies,manipu-lation,andtheextractionofdata.86Thiscanbeexploitedbycompaniesforprofit,forexamplethroughadvertisingorotherwisesponsoredcontent.2.2.3 DEEPFAKES AND DISINFORMATIONAsgenerativeAImod
210、elskeepgettingincreasinglypow-erful,itbecomeseasiertousethemtocreaterealisticsyntheticimages,text,orvoicerecordingsthatcanbemistakenforrealcontent.Itcanhelplowerthethresholdforproducingdeliberatelymisleadingcontent(disinfor-mation),orforcreatingfakeimagesorvoiceclipsofrealpeopleincompromisingsituati
211、ons,orforimitatingrealpeople(deepfakes).87 A2022Europolreportestimatesthatby2026,about90%ofonlinecontentmaybeAIgenerated.88 Asthevolumeofsyntheticcontentgrows,itbecomesdifficulttotrustonesowneyesandears.Thelong-termeffectsofthiscanbedevastatingfortrustininstitutions2.2Manipulation26Norwegian Consume
212、r CouncilJune 2023Ghost in the machineandeachother.Theproliferationofdeep-fakedcontentcanleadtosignificanterosionoftrust,aspeoplewillnotbeabletoknowwhetheranimage,text,sound,orvideoisrealorsynthetic.Evenifthecontentwasnotoriginallygeneratedtospreaddisinformation,thenatureoftheinternetmeansthatcontex
213、tanddisclaimersarequicklystrippedawayascontentissharedacrossplatforms.89 Assyntheticcontentproliferates,thismayalsoprovideplausibledeniabilityinthecaseofauthenticcontent.Forexample,ifawhistle-blowerleaksinformationexposingcorruption,theaccusedindividualorinsti-tutionmayplausiblyclaimthattheleakedmat
214、erialisfake.Asub-categoryofdeepfakesthacanhaveaparticularlydevastatingeffectonvictims,isdeepfakepornography.AccordingtoastudybythecompanySensity,96%ofdeepfakeimagesaresexuallyexplicitpicturesofwomenwhodidnotconsenttotheimagegeneration.90Asdescribedabove,opensourcemodelssuchasStableDiffusionmakesitpo
215、ssibleforanyonetotrainmodels,whichmeansthatpeoplecanbedeepfakedeveniftheyarenotapublicpersonwithmanyavailableimages already part of the training data.WhilethegenerativeAImodelscanbeusedtointention-allyproduceandspreaddisinformation,theemploymentofinaccurategenerativeAImodelsinconsumerfacingproductsc
216、analsoaccidentallyleadtothespreadoffalsehoods.Asdiscussedmorein-depthinchapter2.2.1,prominenttextgeneratorsarepronetoproducingveryconvincing,falseinformation,aswellasreferenc-ingsourcesthatdonotbackupitsclaims.91 AsmoreadvancedgenerativeAImodelsbecomein-creasinglyefficientatgeneratingtextthatseemscr
217、ed-ible,thismayleadtodisinformationbecomingmoredifficulttodetect.AstudyfromMarch2023foundthatChatGPT4ismorelikelythanitspredecessortogen-eratemisinformationwhenprompted,includingfalsenarrativesconcerningvaccines,conspiracytheories,and propaganda.92Thiscouldmakethetechnologyanefficienttoolforrapidlyp
218、roducingconvincingtextthatcanbedisseminatedwithpotentiallyharmfuleffects.Discussionsonhowdeepfakesanddisinformationwillplayoutinelectionsandthedemocraticprocessisincreasinginintensityaswell.93 President Donald Trump crying in front of the White House,Midjourney.A lifelike photograph of a woman,DALL-
219、E.Note the watermark in the bottom right corner.2.2Manipulation27Norwegian Consumer CouncilJune 2023Ghost in the machine2.2.4 DETECTING AI-GENERATED CONTENTOne suggested solution to the deluge ofsyntheticcontent,istowater-markorotherwiseclearlylabelthatapieceofcontentwasgeneratedusing a generative A
220、I model.This canbedoneeitherbyaddingavisuallabelthatindicatesthatanimageorvideoisAI-generated,throughimperceptiblewater-markssuchassinglepixels,orbyaddinginformationtothemetadatathatcanbeusedtoshowtheoriginsofthecontent.94Forexample,GoogleisimplementingafeaturetoautomaticallylabelAI-generatedcontent
221、inthemetadataofpictures,andaddcontexttowheretheimage originated.95Whilewatermarkingcanbeusefultoquicklyidenti-fythatanimageorvideoisnotauthentic,therearesignificantlimitationstothisapproach.Awatermarkingsystemonlyworksaslongasthesystemdeveloperand/orthepersonusingthemodelchoosestoabidebythewatermark
222、ingstandards.Closedsourceimagegenera-torssuchasDALL-EandMidjourneymaychoosetoaddmandatorywatermarkstothemetadataofallgeneratedpictures,butthiscanbecircumventedforexamplebytakingascreenshotandsharingthescreenshotinsteadoftheoriginalimage.Visualwatermarks,suchastheonesusedbyDALL-E,canbecroppedoutofthe
223、picture,unlessthewatermarkissoobtrusivethatitsignifi-cantlydetractsfromtheimagequality.Imperceptiblewatermarkssuchasindividualpixelscanberemovedbyslightlychangingthecolourgradingoftheimage.ForopensourcemodelssuchasStableDiffusion,attempts ataddingwatermarkstogeneratedimagescanberemovedfromthemodelby
224、anyonewhowantstodeliberatelypassoffsyntheticcontentasreal.Thiscanbeaddressedifasignifi-cantpartofthetrainingdataforthemodeliswatermarked,butevenifthiswasthecase,thewatermarkingcouldbecircumventedasdescribedabove.Inadditiontoissuesofmisinformation,inaccuraciesandauthenticityrelatingtogeneratingimages
225、,therearesignificantquestionsabouthowtodetectplagiarism,forexamplewhenstudentsusetextgeneratorsinacademicsettings.ChatGPThasbeenwidelyusedtogeneratees-saysandanswerotherschoolassignments,raisingalarmsaboutcheatingandnegativeeffectsonlearning.96 Watermarkingoftextismorecomplexthanforimagesandvideos,a
226、sanytextcopiedfromatextgeneratordoesnothaveanymetadatatoappendwatermarksto.ThereareongoingeffortstocreatetextualsignaturestotextgeneratedbyChatGPT,butthismaybecircumvent-edbymakingchangestothetextorbyfeedingthetextthrough another text generator.97 Systemsthataresupposedtodetectandflagwhetheratextwas
227、writtenbyatextgeneratororahumanhavebeennotoriouslyinaccurate,98andarenotascalablesolutionasalltextneedstobefedintothedetectorsys-tem.Forexample,OpenAIhavereleasedagenerativeAImodelwiththepurposeofdetectingwhetheratexthasbeenwrittenbyChatGPT,butthismodelonlyhada26%accuracyrate.99Detectingwhethersomet
228、hingwasgener-atedbyagenerativeAImodelrequiresmoretechnicallycomplexsolutionsthangeneratingnewcontent,whichmeansthatthedetectionsystemswillseeminglyalwaysbelaggingbehindinthisarmsrace.ThisleadstoquestionsaboutwhatrecourseapersonhasinacasewhereanAImodelfalselyaccusesthemofplagiarism.Identifyingfalsefl
229、agsmayalsobecomplextask,whichmeansthatateacherusingasystemtode-tectplagiarismmaynotbeabletoaccuratelydoso.Iftheenduseroftheflaggingsystem(forexampleateacher)cannotsuccessfullydeterminethatatextorimagewasfalselyflaggedasplagiarism,thisputsthestudentinadifficultposition,atitisvirtuallyimpossibletoprov
230、ethatChatGPTdidinfactnotwriteyouressay.Similarly,ifpeopleareflaggedascheaters,untrust-worthy,orsimplyasnon-human,theymayexperiencesignificantnegativeeffectswithfewmeanstorecourse.Forexample,ifaplatformhasaflaggingsysteminplacetoidentifyandremovecontentthatappearstobeAI-generated,thesesystemsmayerron
231、eouslyflagcon-tent,leadingtoconsequencestotheconsumerthathascontentremoved.Summedup,watermarkinganddetectiontoolsaretechnologicalsolutionsthatmightworkincertainlimitedsettings,suchasdemonstratingthataphotographorig-inated from a real photographer or from an image genera-2.2Manipulation“The prolifera
232、tion of deepfaked content can lead to significant erosion of trust,as people will not be able to know whether an image,text,sound,or video is real or synthetic.”28Norwegian Consumer CouncilJune 2023Ghost in the machinetor,forexampleinadvertisingorwhenusedbymediaorpublicinstitutions.Itcanbeausefultoo
233、ltoquicklydeterminethatapictureisactuallyfromGettyImagesorfromDALL-E,whichmayalleviatesomeharmsrelatedtoaccidentalspreadofmisinformation.However,abeliefthatwatermarkingwillsolvethein-formationcrisisisatitscoreatechnologicalsolutionistapproach.Evenifitwastechnicallypossibletoaccu-ratelywatermarkallAI
234、-generatedcontent,thedelugeofsyntheticcontentanddisinformationisnotsolvablebyaddinganothertechnicallayer,particularlyifcontentisdeliberatelymeanttomislead.Thelackoftrustinthemediaandpublicinstitutionsisnotsolelyamatterofnotbeingabletotellsyntheticfromauthenticcontent.Furthermore,itisunreasonabletoex
235、pectpeopletoscaneverypieceofmediatheyseeonlinetodetectwhetheritissynthetic.Sincethereisnotechnologicalquickfix,itiscrucialtolookatothersolutions,suchasrobustmedialiteracyandtrustedmediainstitutions.Thisshouldalsobegivenconsiderableattentionbymedia-andsocialsciencesresearchers,whocanprovidepolicymake
236、rsandotherswithsustainable,long-termsolutions.2.2.5 GENERATIVE ARTIFICIAL INTELLIGENCE IN ADVERTISINGThepromiseofgenerativeAImodelshasalsoreachedthe advertising industry.100Thetechnologyisalreadybe-ingusedtogenerateadcopy,101creatingsyntheticstockphotosandmodels,102andaspartofmarketingstunts.103 The
237、seusecasesmayreducelabourintheadvertisingsectorbutcanalsohaveadverseeffectsonconsumers,particularlybymakingiteasierandmoreefficienttomanipulatepeoplethroughcreatingpersonalizedand/orconversationaladvertising.TheintroductionofpubliclyavailablegenerativeAIhasbeenlargelyad-free,butthisispoisedtochange.
238、InMarch2023,MicrosoftannouncedthatitwouldberollingoutpaidadsintheBingchatbot.104InMay,GoogleannouncedthatitwouldintegrateadvertisingintheirgenerativeAIproducts.105IfconsumersdependontextgeneratorssuchasBingtoprovideaccurateandfactualinformation,theplacementofadvertisingintheanswersitprovidesmaybemis
239、leading.Thepotentialforbehaviouralmanipulationwheninteractingwithalargelanguagemodelmayenablemoreeffectiveadvertisingatthecostofconsumeragency.106ImplementinggenerativeAImodelsmayalsoexacerbateseveralproblematicissuesrelatedtosurveillance-basedadvertising,suchasdiscrimination,fraud,andprivacyviolati
240、ons,bymakingiteasiertogeneratetailoredadstoparticulargroupsorcategoriesofpeople,whichmayinturnmakeiteasiertoconvincesomeonetopurchaseaproductorbelieveastatement.107Thisfeaturewouldaccelerateandfacilitatecompaniesabilitytotargetthecontentofadsautomatically,ratherthanjusttheadsthemselves.CombinedwithA
241、/B-testing,thiscouldin-creasethemanipulativecharacterofsurveillance-basedadvertising.2.2.5.1 Using chatbots to collect personal dataTherearerisingconcernsaboutgenerativeAIinchat-botsandtheirabilitytotrickconsumersintosharingper-sonaldata,whichmayberepurposedtoservetargetedadvertisingortomanipulateco
242、nsumersintopurchasingproductsorservices.Whilethischallengeechoesabroaderdebateabouttherepurposingofpersonaldataforbusinessgains,108themanipulativeaspectsofgener-ativeAImodelspretendingtobehumans,asmentionedabove,couldexacerbatetheproblems.Thisisespeciallyrelevantinthecaseofvulnerablegroupssuchaschil
243、-drenorlonelypeople,whomaybemorelikelytosharesensitiveinformationaboutthemselvesinconversationwiththegenerativeAI.Forinstance,chatapplicationslikeReplikaandSnap-chatsMyAI,bothofwhichwerediscussedinchapter2.2.2onpersonificationofAImodels,explicitlyinviteenduserstoshareinformationaboutthemselves.Simil
244、arly,generativeAImodelsusedtoembedsearchinvariousapplicationsmaybeusedtocollectandstorequerydata,suchaswhetherthepersonqueryingthemodeliscur-rentlyinterestedinlocalrestaurantsorshoes,depend-ingonthequery.Personaldataisthebasisformassivebusinessmodels,andthesetextgeneratorscanimprovebusinessesability
245、toobtainhighlyrelevantconsumerinformation.2.2Manipulation“A belief that watermarking will solve the information crisis is at its core a technological solutionist approach.“29Norwegian Consumer CouncilJune 2023Ghost in the machine2.3 Bias,discrimination,and content moderation2.3 Bias,discrimination,a
246、nd content moderationAswithotherformsofartificialintelligence,generativeAImodelsmaycontain,perpetuate,orcreatenewbiases.Models that are trained on vast amount of information takenfromtheinternetwillinheritthebiasesofitstrainingdata.Assuch,themodelsmaygeneratecontentthatreproducenegativeorunwantedten
247、dencies.Thishasledtomanyservice-providersaddingcontentfilterstomoderatewhatispossibletogenerate,andtoflagproblematiccontentinthetrainingdataofthemodels.2.3.1 BIAS IN TRAINING DATAAsmentionedabove,generativeAImodelscangener-atesyntheticcontentthatresembleshuman-createdcontentbecausetheyhavebeentraine
248、donlargedatasetsofexistingcontent.This means that the contentinthedatasetsisofcrucialimportance.There are several steps tocreatingandcuratinga training data set for generativeAImodels,rangingfromscrapingdataonline,selectionandlabelling,tocon-tentmoderation.Withoutcarefulvetting,labelling,andcleaning
249、ofthetrainingdata,datasetsscrapedfromtheinternetcanleadtoseriousdownstreameffects.Forexample,theimagegeneratorStableDiffusionistrainedonanopensourcedatasetfromtheGermannon-profitorganizationLAION.109 The LAION data sets donotcontainanyactualimagesbutisratherasetofURLsthatpointtoimagesfromacrossthewe
250、b.LAIONhasreceivedcriticismforalackofaccountabilityandforinsufficientcurationofcontent(suchasexcludingharm-fulorpotentiallyillegalmaterial),forexamplewhenitwasfoundtoincludeURLspointingatconfidentialmedicalinformation in its data sets.110 AsgenerativeAImodelsaretrainedonhistoricaldata,discriminatory
251、factorsinthedatasetscanbereinforcedbybeingreproducedinthetext,imagesorsoundthatisgenerated.Furthermore,suchmodelscanonlybetrainedonrecordeddata,whichmeansthatphenom-enaoreventsthatarenot(orcannotbe)recordedandquantifiedasdatacannotberecognizedbythemodel.Assuch,generativeAImodelsarepredisposedtoencod
252、edbiasesthatamplifyorentrenchexistinginjusticesandpowerstructures.111 Generative AI models are primarily trained on images andtextscrapedfromtheinternet,whichmeansthereisselectionbiasalreadyatthetrainingstage.Populationsegmentsandgroupsthatlackinternetaccess,forex-ampleindigenousgroups,willlikelybeu
253、nderrepresentedinthetrainingdata,whichcanhavedownstreamdis-criminatoryeffects.Furthermore,ifonlinecommunitieswherecertainpopulationgroupsareoverrepresentedareprominentinthetrainingdata,thismaycontributetoafeedbackloopthatcontinuouslylessenstheimpactofdatafromhistoricallyunderrepresented populations.
254、112Trainingdatacollectedfrom the internet tend toincludepornograph-ic,racist,andste-reotypicalcontent.Ifthe data sets are not curatedandcleaned,thesefactorsmaybecomeembeddedinthemodel.Forexample,imagegeneratorstendtosexualizewomen,particularlywomenofcolour,atamuchhigherratethanmen.113Similarly,promp
255、tssuchas“Africanworkers”tendtogeneratepicturesofmanuallabourer,while“Europeanworkers”resultsinpicturesofwhite-collarjobs.114 A Washington Post investigation found that Googles C4 dataset,whichisusedastrainingdataforbothGoogleandMetaslargelanguagemodels,includedmassiveamountsoftextscrapedfromtheopenw
256、eb,includingWikipedia,Redditandalargeamountofotherdiscussionforums,newspublishers,governmentwebsites,andmuchmore.115 This means that any generative AI model trainedonthissetwill“learn”fromcontentthatmaycon-taineverythingfromhatespeechtoadvertising,whichmayhaveanimpactonthetextitisabletogenerate.If,f
257、orexample,dataisscrapedfromaninternetforumthatcontainsalotofracistorotherwisetoxiccontent,anymodelstrainedonthedatasetruntheriskofrecreatingsimilar material.“The selection and labelling of training data is not neutral.Certain groups of people may be overrepresented in the data,while how the company
258、chooses to label images may reflect biases.”30Norwegian Consumer CouncilJune 2023Ghost in the machine2.3 Bias,discrimination,and content moderationTheselectionandlabellingoftrainingdataisnotneutral.Certaingroupsofpeoplemaybeoverrepresentedinthedata,whilehowthecompanychoosestolabelimagesmayreflectbia
259、ses.Forexample,adevelopermightchoosehowmanycategoriesofethnicitiesand/orgen-derstoincludeintrainingdatalabelsorcouldchoosenottoincludetheseattributesaslabelsatall.IfmodelsaretrainedonotherAIgeneratedcontent,thisrunstheriskoffurtherreinforcingbiases.Asaresult,theremaybefeedbackloopswhereeachtrainings
260、essionstrengthensabiasedordiscriminatorysequenceofdata.ManyAImodelshaveissueswithrecognizingandlabel-lingimagesofnon-whitepeople,likelypartiallybecauseoftrainingthemodelsondatasetswherewhitepeopleare overrepresented.Both Google116 and Meta117 have beenunderfireaftertheirimage-recognitionalgorithmsla
261、belleddarker-skinnedpeopleasgorillasorprimates.LanguageprocessingmodelssuchasBERThasalsobeenshowntoconnectdisabledpersonswithwordsofamore negative sentiment.1182.3.1.1 Discriminatory outcomesBiasedordiscriminatoryoutcomesfromtheuseofgen-erativeAImodelsisnotonlyaproblemrelatedtotrainingdata.Thereareh
262、umanandsystemicbiasesthatcanbeembeddedorstrengthenedfromhowcompaniesandpeoplechoosetouseornotusethemodels.119 For exam-ple,iftheuseoftextgeneratorsbecomesarequirementforvariousjobs,thismayindirectlyexcludelesstechni-callyproficientgroupsofpeople.When AI models are implemented in an attempt to solve
263、complexissues,thereisarealriskthatmoreeffectivesolutions,whichmaybemorecostlyand/orcomplicat-ed,aredownprioritized,asdiscussedmorein-depthinchapter2.1.2.Forexample,theWorldHealthOrgani-zationhaswarnedthattheuseofAImodelsinhealth-caremayhavenegativeeffectsonolderpeople,unlesscertainissuesareaddressed
264、.120ConcernsincludethatAImodelsmaybetrainedondatathatcontainageiststereotypes,andthatolderpeopleareoftenunderrepre-sented in training data.This may help perpetuate ageism andunderminethequalityofhealthandsocialcareforolder populations.2.3.2 CONTENT MODERATIONWhentrainedonlargeenoughdatasets,thereare
265、fewlimitsonwhatmaterialagenerativeAImodelmayproduce.Asnotedabove,manysuchmodelscanbeusedtogenerateillegal,discriminatory,andotherwiseunacceptablesyntheticcontent,sincetheyaretrainedondatasetsthatmayincludeavarietyofdubiouscontent.Inanattempttoalleviatetheseissues,manygenerativeAImodelshavecontentmod
266、erationinplacetofilteroutandflagcertaincontent,orotherwiseintroducelimitstohowthetechnologycanbeused.Whilecontentfilterscanbeusedtolimitthegenerationofcertaintypesofcontent,thisisanapproachwithnumerousshortcomings.Firstofall,contentmoderationgivesthesystemownersignificantpowertodecidewhatmaterialish
267、armfulandwhatispermitted,unlessthisisclearlydefinedbylaw-makers.Forexample,OpenAIhascomeunderfirewithclaimsthatChatGPTrestrictscertainpointsofview,byrefusingtogeneratetextaboutcertainpoliticizedtopics.Thiscanleadtoabusesofpowerthatmayhavesignifi-cantdownstreamconsequences,asprivatecompaniesincreaset
268、heirabilitytodecidewhatisdeemedaccept-ablecontent.Aswithcontentmoderationpracticesonsocialme-diaplatforms,contentfiltersongenerativeAImodelsrunstheriskofover-moderation,whereinnocuousorimportantcontentisfilteredoutorbanned.Thiscanhappenbothbyaccidentorbydesign.Forexample,theimagegeneratorMidjourneyb
269、eganfilteringoutscientificanatomicaltermstoclampdownonendusersgener-atingpornographiccontent.121ThecompanyalsoaddedcontentfilterstopreventconsumersfromgeneratingimagesofXiJinpinginordertoavoidbeingblockedinChina,andendedupdiscontinuingitsfreetrialversionafteraMidjourney-generatedpicturesofDonaldTrum
270、pbeingarrestedwentviral.Thecompanydoesnotdis-closepubliclywhatwordsorpromptsarebannedfromtheplatform,to“minimizedrama”.122Therearealsotechnicallimitationsonwhatcontentfil-terscando.WherevercontentfiltersareusedtorestrictgenerativeAImodels,thereareattemptstobypassorjailbreakthemodels.Peoplehavediscov
271、ereddifferentpromptsthatmaybeusedtogeneratebannedcontent,forexamplebyinstructingthemodeltosimulatecharac-tersthatareallowedtobypassthecontentfilter.123 This armsraceislikelytoleadtomoreover-moderation,ascompaniesrushtocloseanyperceivedloopholes.Content moderation of generative AI models may also cre
272、ateorenforcediscriminatorypractices.Asanexam-ple,wordsreferringtoLGBTQI+communitiesorother31Norwegian Consumer CouncilJune 2023Ghost in the machine2.4 Privacy and data protectionminoritygroupsmaybeflaggedtoremovehatespeechordiscriminatorycontentformthetrainingdata.How-ever,suchattemptsatremovingunac
273、ceptablecontentcouldalsoleadtotheremovalofcontentwhichisinfactshowcasingpositivesidesandsentimentsrelatingtoLGBTQI+communities.Contentmoderationcouldinthiswayreinforceunderrepresentation.Choosingtoaddressthebiasedoutputratherthanthebiasinherentinthedatasetsorthemodel,isinherent-lyproblematic.Moderat
274、ionattemptswillrequirethesuppressionofeachtypeofbiasedoutput,effectivelyapproachingbiasingenerativeAIasagameofwhack-a-mole.124Increasingtheattentiongiventothecurationofdatasetstoreducetheirinherentharmfulbiasisneces-sary,insteadofrelyingonpost-hoccontentmoderation.2.3.2.1 Cultural contextContentmode
275、rationisnotaneutralpractice,andun-derstandingthecontextofcontentiscrucial.Differentculturalcontextthereforepresentsasignificantbarrierforcontentmoderationatscale.Forexample,thereisariskofover-orunder-moderationbecauseofinsuffi-cienttrainingdataormoderatorsforcertainlanguagesordialects.Awidelyusedlan
276、guagesuchasEnglishwillhavealargercorpusoftextinitstrainingdatatoprovidemoreaccurateinformation,andconsequentlypotentiallybettermoderation.Otherlanguagesandculturesareoftenunder-represent-edinthetrainingdata,meaningthatmoderationislikelytobelessaccurateornon-existent.Minoritygroupalsotendtobeseverely
277、underrepresentedamongthepeopledeveloping and training the models.125Furthermore,therearesignificantissuesrelatedtoculturalcontextandnationallegislation,aswhatissociallyacceptableorlegalinoneplaceandcontextmaybetabooorillegalsomewhereelse.Thecontextualcomplicationsaroundcontentmodera-tionalsomakesita
278、taskthatmaybeillsuitedforauto-mation.Theworkofmoderatingoutputandannotatingtrainingdataareautomatedinsomecases,butalsoofteninvolvesmanualwork.Inmanycases,processessuchasdatacleaning,contentclassificationandcontentmoderationinvolvesmentallytaxinghumanlabour.Thisiselaborateduponbelowinthesectiononlabo
279、urexploitation.2.3.2.2 Open source models and the limits of content filtersInpractice,contentmoderationonlyworksoncentral-izedclosedsourcemodels.Inopensourcemodels,suchasStableDiffusion,itispracticallyimpossibletocontrolwhatcontentthemodelcanproduce.Downstreamdevelopers,includingindividuals,cantrain
280、andsharemodelsthatcanmakeanykindofimages,regardlessoflegality.Asthemodelsrunlocallywithoutrequiringaninternetconnectionoraccesstoacloudserver,thecompanywhoreleasedthemodelcannotinterceptorlimitwhatitisabletogenerate.2.4 Privacy and data protectionTherighttoprivacyisoneofthecorevaluesofdemo-craticsoc
281、ieties.Privacyencompassesmanydifferentaspects,suchasprivacyofcorrespondencewithothers,privacyofidentityandthoughts,andprivacyofdataandinformationaboutoneself.Dataprotectionisasub-stantialandimportantpartofprivacy,especiallyinthecontextofonlineservices,butprivacycoversamuchbroaderrangeofindividualpro
282、tections.Personaldatahaslongbeencovetedashighlyvaluableforbusinessesandmaybeusedtotargetadvertisingtoindividualsandgroups,tomeasureengagementortoimprovecompaniesservices,amongotherpurpos-es.When generative AI models are trained on material scrapedfromtheinternet,thetrainingdatausuallycontainsalargea
283、mountofpersonaldata.AsgenerativeAIisdevelopedanddeployed,theseissuesrelatedtodataprotectionandpersonaldatacanleadtosubstantialprivacyharms.2.4.1 PRIVACY CHALLENGES RELATED TO DATA SETS USED FOR MODEL TRAININGImage generators are usually trained on huge datasets thatincludeimagesofrealpeople.Theseima
284、gescan,forexample,betakenfromsocialmediaandsearchengines,withoutalawfullegalbasisorknowledgebythepeopleinthepictures.Similarly,textgeneratorsaretraineddata-setsthatcouldincludepersonaldataaboutindividuals,orconversationsbetweenindividuals.32Norwegian Consumer CouncilJune 2023Ghost in the machineIf a
285、 generative AI model is trained on personal data that wastakenoutofcontext,thismayviolatethecontextualintegrityofindividualconsumers.Whenapersonup-loadsaphotoofthemselvesonline,forexampleonsocialmedia,theycouldnotforeseethatthiswouldbeusedtotrainanAImodel.Theindividualwasneverinformedthiswouldhappen
286、,neverconsentedtosuchuseoftheirlikenessandwilllikelynotbeawarethattheirprivacyandpersonaldatarightswereviolated.AsthepublicawarenessgrowsabouthowgenerativeAImodelsaretrained,theuseofpersonaldatafortrainingmaycreatechillingeffects.UnlessauthoritiesenforcecurrentlegislationsuchastheGDPRagainstcompa-ni
287、esdeployinggenerativeAImodels,andguardrailsandrestrictionsforusingimagesofpeopleareinplace,theonlyrealchoiceforconsumerswhodonotwanttheirimagesusedfortrainingdataistostoppostingpicturesonline.Thisisclearlyaninsufficientsolution.2.4.2 PRIVACY CHALLENGES RELATED TO GENERATED CONTENTItisparticularlypro
288、blematicifagenerativeAImodelcangeneratenewimagesofanindividual,suchasdeep-fakes.Thisinvolvescreating“new”personaldataabouttheindividual,inawaythepersoncanhavenocontrolover.Thisviolatestheintegrityoftheindividualwhoisdepictedinthegeneratedcontent,potentiallyinveryinvasiveorharmfulways.Sometimesapictu
289、recanbeaccuratelyreproducedbythegenerativeAImodel.Thishappensifthemodelwasovertrainedoncertaindata.Forexample,theMonaLisaislikelytobeoverrepresentedinatrainingdatasetcontainingart,becauseitissuchafamousworkofart.Ifthishappens,themodelmayovertrainonthefaceoftheMonaLisa,andthereforemaybelikelytoreprod
290、ucethepaintingquiteaccurately.Overtrainingonpicturesofcertainpeoplewillhavethesameeffect,meaningthatitismorelikelytoreproduceaphotoofahigh-profilecelebritythanarandominternetuser.WithopensourcemodelssuchasStableDiffusion,however,anydown-streamdeveloper,includingindividuals,cantrainmodelsonthefacesof
291、anyone,whichcanbeusedtocreatedeepfakes.Inadditiontothegenerationofpicturesandtheadverseeffectsthismayhaveonconsumers,itisalsopossibletogeneratetextaboutindividuals.Thisincludestextgeneratorsgeneratingfalseand/orlibellousclaimsaboutpeople.Forexample,ChatGPThasgeneratedtextwithpotentiallyhazardousresu
292、ltsforthepeoplewhomitcon-cerns,suchasfalseclaimsaboutaprofessorsinvolve-mentinasexualharassmentscandalorfalseclaimsthata mayor had served prison time.126 2.5 SecurityvulnerabilitiesandfraudGenerativeAImodelscanbeabusedbymaliciousactorstoaugmentorsuperchargecriminalactivities.Aswithotherareas,generat
293、iveAIcanbeusedmakefraud,scams,andotheractivitiesmoreefficient.Themodelscanalsoposechallengestoexistingsecuritysystems.WhilethetypesofcybercrimethatcanbeundertakenusinggenerativeAIarenotnew,theubiquitousnessandeaseofuseofthetechnologymayleadtoanupscalingofsuchattacks.Largelanguagemodelscanbeusedbysca
294、mmerstogen-eratealargeamountofconvincing-lookingtexttodeceivevictims.Similarly,catfishingscams,wherethescammerbuildstrustwiththevictimovertimethroughregularcon-tact,canpotentiallybeautomatizedconvincinglybytheuseofadvancedchatbots.Thismeansthatthecriminalcanef-fectivelyscammorevictimsusinglesstimean
295、dresources.Deepfakingcanalsobeusedtobypasssecuritymea-sures.Whenpicturesandvoicescanbeconvincinglyfaked,thismakesitpossibletoengageinfraudinnewways.Forexample,areporterwasabletofakeclipsofhisownvoicetobypassthevoicerecognitionbiometricidentificationonhisbankaccount.127Similarly,audiogeneratorshavere
296、portedlybeenusedtoimpersonatefamilymembersforcriminalpurposes.128 Largelanguagemodelsarevulnerabletoexploitstoby-passfiltersandsecuritymeasures(jailbreaking),delib-eratelymanipulatingthetrainingdata(datapoisoning),andhiddencommandsthatspurthemodelsintotakingcertainactions,forexamplethroughhiddentext
297、inan2.5Securityvulnerabilitiesandfraud33Norwegian Consumer CouncilJune 2023Ghost in the machine2.6Replacinghumansinconsumer-facingapplicationswithgenerativeAI,whollyorinparte-mail(promptinjection).129Thesesecurityvulnerabili-tiesmayprovetobegrievous,ascompaniestrytostayaheadofthecurvebyintegratingge
298、nerativeAIrapidlyintovariousservices,potentiallywithoutsufficientsecuritytesting.Cybersecurityexpertshavewarnedthattextgeneratorscanalsobeweaponizedbyusingthetechnologytowritemaliciouscodesuchasmalware.130Thismeansthatcy-bercriminalscanpotentiallygeneratevirusesandotherharmfulcodewithoutneedingthete
299、chnicalproficiencytraditionallyassociatedwithsuchactivities.Similarly,AImodelsbuiltfordrugdiscoverycanpotentiallyalsobeusedfordesigningbiologicalweapons.131 Europol has alsowarnedaboutthepotentialforlargelanguagemod-elstobeusedinvarioustypesofcybercrime.Accordingtotheagency,contentmoderationmaybeins
300、ufficientastherearenumerouswaystobypasssuchrestrictionsorjailbreakthemodels.132ThelackoftransparencyabouthowcompaniessuchasOpenAIusedatahasalsosparkedconcernsabouthowconfidentialinformationmaybeabused.Sever-alhigh-profilecompanieshavebannedorwarneditsemployeesagainstinputtingbusinessinformationintoC
301、hatGPT.133Amazonhasreportedlyobservedthetextgeneratorgeneratingtextthatcloselymatchedinternalcompanydocuments.134Thisindicatesthatthereisariskthatconfidentialinformationisleakedthroughgenera-tive AI models.2.6 Replacinghumansinconsumer-facingapplications withgenerativeAI,whollyorinpartWhengenerative
302、AImodelswereinitiallyintroducedtothepublic,theywereprimarilystand-alonesystems,withwhichenduserscouldgeneratecontent.Astheinterestforthesesystemsrose,thesystemownersintroducedthepossibilitytoincorporatethemintootherapplicationsandsystemsthroughAPIs.Thiscouldentailadd-onstorecreationalapplicationsand
303、systems,butitisalsopossibletoenvisagetheminpartlyorfullyauto-mateddecision-makingsystems,orasreplacementsofhumaninteractioninconsumerfacingservices.Thismayhavefar-reachingimplications.Forexample,OpenAI founder Sam Altman has argued that in the future generativeAImodelsmayfunctionasmedicaladvisersfor
304、peoplewhoaretoopoortoaffordhealthcare.135 In May2023theUSnon-profitorganizationforsupportingpeoplewitheatingdisorderslaidoffstaffandvolunteersforitshelpline,tobereplacedbyanAIchatbot.136 While aspokespersonfortheorganizationclaimedthatthechatbotwasnotadirectreplacementforthehelpline,itneverthelessac
305、companiedtheshutdownofthehelplineservice,leavingpeoplewithoutrealhumanstotalkto.Automatingsuchtasksmaymultiplytheriskoffatalmistakesifthereareproblemsinthetrainingdataorinthe model itself.Foryears,companieshaveattemptedtoautomatecon-sumerinteractions,forinstancebyautomatingcustomerservicethroughchat
306、bots.Manycompaniesmakeitdifficultforconsumerstogetincontactwithhumans,whichadverselyaffectconsumerswhodonothavestandardizedproblemsthatareaddressedinFAQsandsimilardocuments.WiththeriseofgenerativeAI,thereisariskthatcompanieswillmakeitevenmoredifficultforconsumerstogetittouchwithrealhumans.2.6.1 CHAL
307、LENGES RELATED TO COMBINING HUMAN-AND AUTOMATED DECISION-MAKINGAutomatedsystemsdonothavethecapacityforethicalreflection,sympathy,orunderstanding.Generally,peoplearenotpersecutedforminorinfractions,butautomat-edsystemsarenotabletodistinguishbetweenminor“The lack of transparency about how companies su
308、ch as OpenAI use data has also sparked concerns about how confidential information may be abused.“34Norwegian Consumer CouncilJune 2023Ghost in the machineandaggravatedinfractions.Ifaconsumermissedtheirpaymentbyaday,ahumanmightconsiderwhetherthecustomerrelationshouldbeprioritizedoverstrictcompliance
309、withtherules,andthereforeallowforalatepaymentwithnoadditionalcosts.Theautomatedsystemwouldnotbeabletomakesuchconsiderations.Sympa-thyandprinciplesoffairnesscouldthereforebelostinthetransitofautomatingprocesses.Fully automated systems are usually regulated through additionallegalprovisionsandprotecti
310、ons,toaccountforadditionalrisksrelatedtosuchdecisionmaking.This caninsomecasesentailrequirementsofhumaninvolve-ment,137orleadtocompaniesintroducingahumaninthelooptoavoidlegalscrutiny.138Keepinghumansintheloopishoweveracomplexmeasure,withseveralpitfalls.Humanscanbothover-relyandunder-relyontheoutputo
311、fautomatedsystems,139andtheproblemisparticularlyprominentinautomatedcomputersystemsthatdonotproduceexplainableorinterpretabledecisions.Itishow-everover-relianceonautomatedsystemsoutputthatinvolvesthemostnovelchallenges,asopposedtoanin-dividualover-relyingonherowndecisions,whichismoresimilartoawholly
312、manualdecision-makingprocess.Inwhollyorpartiallyautomatedsystemsover-reliancecanaffectdifferentpeople:the“humanintheloop”mightnotchallengethesystem,evenwhenitwouldbeprudent,whilethepersonwhoisaffectedbythedecisionmightnotlodgeacomplaintordemandahumanreviewofthedecision.Inbothcases,theinterestsofthep
313、ersonaffectedbythedecisionareputatrisk.Asdescribedinprevioussections,theoutputoftextgeneratorssuchasChatGPThaveprovenverycon-vincing.Iftextgeneratorsareusedindecision-makingprocessesaffectingconsumers,theriskofover-relianceontheoutputmightincrease.Theseeffectsmaybefurthercompoundedbyendusersbelievin
314、gthattheyareinteractingwithasentientintelligentbeingratherthanaprobabilistictextgenerator.Eveniftheinterlocutorsunderstandadecision,deemituntrustworthy,andthereforeconsideroverturningit,therecanbeadditionalhurdles.Fromabusinessperspective,thereisefficiencytogainfromautomatingwholeorpartsofaprocess.I
315、fthedecisionsfromthemachinearegenerallyupheld,overturningadecisionmightrequiremorein-depthargumentsthanacceptingthedecision.Aninterlocutorwhorepeatedlyoverturnsthedecisions,thushaltingefficiency,maybeseenasatroublemaker.Astoresponsibilityandliability,overturningdecisionscouldprovedifficultforindivid
316、ual,humaninterlocutors.Whileawrongfuldecisionfromacomputersystemcanbeblamedonthatsystem,overturningthedecisioncouldsignificantlyheightentheinterlocutorssenseofriskbecausetheinterlocutorassumesresponsibilityforthedecision.Liabilityregimescanenhancetheactualandperceivedriskforinterlocutors.2.7 Environ
317、mentalimpactAnincreasingnumberofpeopleintheresearchandscientificcommunityareraisingtheissueoftheimpactof generative AI model development on the environment.Inacontextwhereclimatechangeandscarcityofnaturalresourcesareaglobalchallenge,adilemmaarisesbetweenclaimsthatgenerativeAIcansolveclimatechange,an
318、dtheactualenvironmentalimpactofsuchtechnologies.ThissectiontakesacloserlookatsomeoftheseclaimsandprovidesacriticalexaminationoftherealisticimpactthatgenerativeAIontheenvironmentbothtodayandinthenearfuture.Itshouldbenotedthatmanyoftheseimpactsalsoapplytolargeswathesofthebroadertechsector,butitisimpor
319、tantthatthisperspectiveisnot lost in the hype surrounding generative AI.2.7.1 CLIMATE IMPACTSomeactorsinthegenerativeAIfieldclaimthatthetechnologyhasthepotentialtosaveusfromtheperilsofclimatechange.140However,thecurrentlyavailabledatashowsthatdeployinggenerativeAIinthesamecontext2.7Environmentalimpa
320、ct“If text generators are used in decision-making processes affecting consumers,the risk of over-reliance on the output might increase.”35Norwegian Consumer CouncilJune 2023Ghost in the machine2.7Environmentalimpactthatlargetechcompanieshasbeenoperatinguntilnow,ismoreofaproblemthanasolutiontoissuess
321、uchasclimatechange,watershortages,andhighenergyconsumption.TheTechindustryisalreadyemittingasubstantialamountofcarbon.AccordingtoUNEP,in2021,thetechindustrysemissionsaccountedfor2to3%oftheworldscarbonemissions.141InNovember2022,theMITreportedthat“thecloudhasnowalargercarbonfootprintthantheentireairl
322、ineindustry”.GenerativeAIisnoexceptiontothis negative trend.InMay2023,AIreportedly“usesmoreenergythanotherformsofcomputing,andtrainingasinglemodelcangobbleupmoreelectricitythan100U.S.householdsuseinanentireyear”.142Datacentresareknowntouseanincredibleamountofenergy,andalreadyfiveyearsago,itwaspredic
323、tedthattheenergydemandsofworldwidecomputingcouldexceedthetotalworldelectricitypowergenerationwithinadecade143.Thiswasbeforetherapiddevelopment and deployment of generative AI.144 With theexponentialgrowthofgenerativeAImodelsandin-vestmentininfrastructuretosupportthisgrowth,energyuseandcarbonemission
324、sareexpectedtoskyrocket.Forbesrecentlyreportedthat“generativeAIisbreakingthedatacentre”.145Indeed,basedonaresearchbyTiriasResearch,datacentreinfrastructureandoperatingcostsareprojectedtoincreasetooverUSD$76billionby2028duetoAIdevelopment.TiriasResearchestimatesthat“thisisthecostofmorethantwicetheest
325、imatedannualoperatingcostofAmazonscloudserviceAWS,whichtodaycontrolsonethirdoftheglobalcloudinfrastruc-tureservicesmarket”.146Thisexponentialgrowthhasapricefortheenvironment.Theexactpriceisyettobecalculated,butasanindication,whendeployed,planstointegratelargelanguagemodelsintosearchenginesmayinvolve
326、afourfoldincreaseinenergyusageperindividualsearchquery.147Inotherwords,itisclearthatAItechnologycomeswithahighcarbonfootprint,148 and that energy is needed every stepofthewaywhendesigning,training,developing,de-ploying,andusinggenerativeAImodels.149Theproblemisthatthereisstillalackofdataavailableont
327、heamountof energy needed for generative AI development.150 At the timeofwriting,nocompanieshavedisclosednumbersonhowmuchenergywasrequiredforthelifecycleofagenerative AI model.TheenergyconsumptionofgenerativeAIisexponentialandwillhopefullybemoreresearchedwithprojectionsforthenextfivetotenyears,whichw
328、illallowconsumersaccesstoinformation,andpolicymakerstoregulatehowmuchthisindustryshouldemit.Forexample,theamountofcomputingpowerusedtotraindeeplearningmodelsin-creased300,000timesin6yearsbetween2012and2018.151ThereiscurrentlynostandardizedwaytomeasurecarbonemissionsofAImodels,andnogoodwillfromAI-foc
329、usedtechcompaniestoreleasethenecessaryin-formation.WhereasestablishedtechcompaniessuchasMeta,GoogleandMicrosoftpublishyearlySustainabilityreportswheretheyself-reportenergyandwateruseaswellascarbonemissions,AIcompaniessuchasOpenAIdonotpublishanykindofinformationontheirenviron-mentalimpactandhowtheymi
330、tigateit.Asasidenote,itseemslikelythatevenwhentheydomaketheeffortofreporting,largetechcompaniesun-derreporttheirownemissions,accordingtoa2021studyfromtheTechnicalUniversityofMunich.“Across a sample of 56 major tech companies surveyed,more than half of these emissions were excluded from self-reportin
331、g in 2019.At approximately 390 megatons carbon dioxide equivalents,the omitted emissions are in the same ballpark as the carbon footprint of Australia”.152 Thereisalackofinterestinthetechindustryincalcu-latingcarbonemissionsgeneratedbygenerativeAI,astheindustryisinterestedinobtaininghigherresultsina
332、ccuracythroughmassivecomputationalpower,153 at the costofallotherconsiderations154.FortheAIcommunity,itseemsliketherehasbeenaconstantpushfor“biggerisbetter”,wheretheexponentialsizeofmodelsanddatasetsisvaluedabovealmostallelse.155Unfortunately,thisapproachisnotsustainable.Researchershavecalledthisphe
333、nomenon“RedAI”,156whichresultsinrapidlyescalating“Data shows that deploying generative AI in the same context that large tech companies has been operating until now,is more of a problem than a solution to issues such as climate change,water shortages,and high energy consumption.”36Norwegian Consumer CouncilJune 2023Ghost in the machine2.7Environmentalimpactcomputationalandthuscarboncosts.Theresear