《挪威消費者協會：生成式人工智能對消費者的危害（英文版）（75頁）.pdf》由會員分享，可在線閱讀，更多相關《挪威消費者協會：生成式人工智能對消費者的危害（英文版）（75頁）.pdf（75頁珍藏版）》請在三個皮匠報告上搜索。

1、1Norwegian Consumer CouncilJune 2023Ghost in the machineRunning headerGHOST IN THE MACHINEAddressing the consumer harms of generative AIJUNE 2023TABLE OF CONTENTSEXECUTIVE SUMMARY 51 INTRODUCTION 61.1 An overview of generative artificial intelligence 7 1.1.1 Examples of generative AI models 81.1.2Th

2、egenerativeartificialintelligenceactorchain111.1.3Opensourceorclosedsourcemodels 12 1.1.4 General purpose AI 121.2 Consumer applications 132 HARMS AND CHALLENGES OF GENERATIVE ARTIFICIAL INTELLIGENCE 142.1 Structural challenges of generative AI 152.1.1IdentifyingtheconcreterisksofgenerativeAI 152.1.

3、2Technologicalsolutionism 172.1.3Concentratingpowerinthehandsofbigtech 172.1.4Opaquesystemsandlackof accountability 192.2 Manipulation 222.2.1Mistakesandinaccurateoutput222.2.2ThepersonificationofAImodels 232.2.3Deepfakesanddisinformation252.2.4DetectingAI-generatedcontent 272.2.5Generativeartificia

4、lintelligenceinadvertising282.3 Bias,discrimination,and content moderation 29 2.3.1 Bias in training data 29 2.3.2 Content moderation 302.4 Privacy and data protection 312.4.1Privacychallengesrelatedtodatasetsusedformodeltraining312.4.2Privacychallengesrelatedtogeneratedcontent322.5 Security vulnera

5、bilities and fraud 322.6 Replacing humans in consumer-facing applications with generative AI,wholly or in part 33 2.6.1Challengesrelatedtocombininghuman-andautomateddecision-making33 2.7 Environmental impact 342.7.1Climateimpact34 2.7.2 Water footprint 362.7.3Greenwashing&hopesforGreenAI37 2.8 Impac

6、t on labour 372.8.1Labourexploitationandghostwork372.8.2Labourautomationandthreatstojobs 38 2.9 Intellectual property 383 REGULATIONS 403.1 Data protection law 45 3.1.1Datasubjectrights 46 3.1.2TheItalianDPAsdecisionconcerningChatGPT 47 3.2 Consumer law 483.2.1ConsumerlawinaU.S.setting 49 3.3 Genera

7、l product safety law 503.3.1TheGeneralProductSafetyDirective503.3.2TheGeneralProductSafetyRegulation 50 3.4 Competition law 513.5 Content moderation 51 3.6 The draft Artificial Intelligence Act 52 3.6.1 The EU Commissions proposal 523.6.2TheEUCouncilspositionontheAIA53 3.6.3 The EU Parliaments posit

8、ion on the AIA 533.6.4TheArtificialIntelligenceActmustprotectconsumers54 3.7 Liability 543.7.1ProductLiabilityDirective543.7.2RevisedProductLiabilityDirective553.7.3AILiabilityDirective55 3.8 Industry standards and guidelines 564 THE WAY FORWARD 574.1 Consumer rights principles that are key for safe

9、 and responsible AI 594.2 Policy recommendations 604.2.1Callstoactionandempowermentofenforcementagencies604.2.2Decisionmakersstrategicmeasures614.2.3Newlegislativemeasures61ENDNOTES 65Ghost in the machine Addressing the consumerharmsofgenerativeAINorwegianConsumerCouncilJune 2023www.forbrukerradet.n

10、o/aiDesignby:VonKommunikasjon5Norwegian Consumer CouncilJune 2023Ghost in the machine Executive summaryTherehasbeenanexplosionofconsumerfacinggener-ativeartificialintelligenceservices.Theseapplicationscanbeusedtogeneratesynthetictext,images,sound,orvideothatcloselyresemblehuman-madecontent.Asgenerat

11、iveAIsystemsbecomeintegratedintopopu-larplatformsandtools,adoptionofthetechnologyispoisedtokeeprising.Meanwhile,anumberofemergingchallengeshavespurrednumerousdebatesabouthowtoensurethatgenerativeAIissafe,reliable,andfair.Thisreportisacontributiontothesediscussions,andaimstoprovidepolicymakers,lawmak

12、ers,enforcementagencies,andotherrelevantentitieswitharobuststart-ingpointtoensurethatgenerativeAIdoesnotcomeattheexpenseofconsumerandhumanrights.Wecannotknowforcertainhowthetechnologywilldevelopinthemonthsandyearstocomebutbelievethatthedirectionoftechnologicaladvancementshouldhappenonsoci-etysterms.

13、Therefore,wepresentanumberofoverar-chingprinciplesthatcanhelpdefinehowgenerativeAIsystemscanbedevelopedandusedinaconsumer-andhuman-centredway.Wealsostronglyurgegovernments,enforcementagen-ciesandpolicymakerstoactnow,usingexistinglawsandframeworksontheidentifiedharmsthatautomatedsystemsalreadyposetod

14、ay.Newframeworksandsafe-guardsshouldbedevelopedinparallel,butconsumersandsocietycannotwaitforyearswhiletechnologiesarebeingrolledoutwithoutappropriatechecksandbalances.ThefirstchapterofthisreportprovidesanexplanationofgenerativeAIanditsuses,alongsideseveralexamplesand illustrations.Inchaptertwo,wesu

15、mmarizevariouscurrentandemergingchallenges,risks,andharmsofgenerativeAI.Thisincludeschallengesrelatedto Power,transparency,andaccountability,wrongorinaccurateoutput,usingtechnologytomanipulateormislead consumers,biasanddiscrimination,privacyandpersonalintegrity,securityvulnerabilities,automatinghuma

16、ntasks,environmentalimpact,labourexploitation.Chapterthreecontainsanoverviewofthepatchworkofexistingandupcomingrulesandregulationsthatmayapplytothedevelopment,deployment,anduseofgener-ativeAIsystems.ThisismostlycentredaroundEUlegis-lation,butwithsomereferencestoongoingprocessesinthe United States.Th

17、efinalchaptercontainsnumerousrecommendationsonhowtoaddresstheproblematicissuesofgenerativeAI.Thisincludes:Enforcementofexistinglawsand regulations,ensuringsufficientresourcesfor enforcementbodies,strongerconsumerprotection,robustgovernmentpolicies,newlegislativemeasures,strongobligationsthatcoverdev

18、elopersand deployers of generative AI systems.Executive summary1.INTRODUCTION7Norwegian Consumer CouncilJune 2023Ghost in the machine1.IntroductionConsumerfacingartificialintelligencesystemshavebeenaroundinvariousformsfordecades,andareusedforpersonalizingsocialmedia,filteremails,recommendstreamingco

19、ntent,texttranslation,andmuchmore.Someofthesepurposesarebenignanddiscreet,andmostpeoplemayneverrealizethattheyareinteractingwithanAI-poweredsystem.AnewwaveofAI-poweredsystemsinconsumerfacingapplicationsisfastapproaching,withthemassdeploy-mentandadoptionofgenerativeartificialintelligence(generativeAI

20、)systems.GenerativeAIisasubsetofartificialintelligencethatcangeneratesyntheticcontentsuchastext,images,audio,orvideo,whichcancloselyresemblehuman-createdcontent.Suchsystemsarepoisedtochangemanyoftheinterfacesandcontentconsumersmeettoday.InNovember2022,aprototypeofthechatbotChatGPTwasreleasedtothepub

21、lic.Theapplicationquicklygainedworldwideattention,becomingthefastestgrowingdigitalserviceofalltimewithinamonthofitsrelease.1Inthefollowingmonths,otherservicesforgeneratingtext,images,sound,andvideo,werequicklydeployedanditeratedupon,sparkingasortofarmsraceforgenerativeAIsystems.Consumerswereprovided

22、accesstothesecontentgeneratorsdirectlyinwebin-terfaces,whilecompaniesstartedtoembedthecontentgeneratorsintheirapplicationsandservices.ThesuddenandwidespreaddeploymentandadoptionofgenerativeAIsystemssparkedpublicdiscourseaboutthepromisesandperilsofthetechnology.ThedebatehasrangedfromhowgenerativeAIma

23、ybeusedtoincreaseefficiencyintheworkforceandtosparkcre-ativity,tohowitcanbeusedtospreaddisinformation,manipulateindividualsandsociety,displacejobs,andchallengeartistscopyright.Thediscussionabouthowtocontrolorregulatethesesystemsisongoing,withpolicymakersacrosstheworldtryingtoengagewiththepromisesand

24、challengesofgenerativeAI.Thisreportisacontributiontothesediscussionsbyprovidingananalysisofthemostpressingissuesfromtheconsumerangle,alongwithanumberofpossiblesolutionsandwaystoproceedfrombothalegal,ethical,andpoliticalperspective.Althoughwedonotpretendtohavetheanswerstoallthequestionsraisedbygenera

25、tiveAI,webelievethatmanyoftheemergingorongoingissuescanbeaddressedthroughacombina-tionofregulation,enforcement,andconcretepoliciesdesignedtosteerthetechnologyinaconsumer-andhuman-friendlydirection.As the development of generative AI seems to move at a breakneckpace,thedescriptionsthroughoutthisrepor

26、tmustbeseenasasnapshotofanemergingtechnology.ThereportwaswrittenbetweenFebruaryandMay2023,anddoesnotincludeanynewinformationfrompaperspublishedafterJune1st.TheNorwegianConsumerCouncilisapubliclyfunded,independentconsumerorganization,thatrepresentsconsumerinterests.Thisreportwaswrittenwithcontri-buti

27、onsfromBEUC,MiikaBlinnfromVZBV,KrisShrishakfromtheIrishCouncilforCivilLiberties,DanielLeuferfromAccessNow,JonWorth,MarijaSlavkovik,andAnjaSalzmannfromtheUniversityofBergen.1.1 AnoverviewofgenerativeartificialintelligenceGenerative AI isablankettermusedtodescribealgorith-micmodelsthataretrainedtogene

28、ratenewdata,suchastext,images,andsound.Whiletheseapplicationsrelyondifferenttypesofinputdata,thegeneralprinciplesbehindhowtheyaretrainedaresimilar.TheemergenceofadvancedgenerativeAIispossibleduetoanenor-mousamountofcontentavailableontheinternet,com-binedwithadvancesinmachinelearningandcomputingpower

29、.GenerativeAImodelsworkbyanalysinglargeamountsofinformationtopredictandgeneratethenextwordinasentence,featureofanimage,etc.Thisisdonebydetectingpatternsinandrelationshipsbetweendatapointsinthetrainingdata,whichinturnallowsthesystemtoreplicatesimilarpatternstogeneratesyn-theticcontent,forexampleapiec

30、eofwriting,music,oravideoclip.Thisprocesscanalsobedescribedasacomplexmash-upofcontentfromthedatathesystemwastrainedon.Inotherwords,theyarepredictivemod-8Norwegian Consumer CouncilJune 2023Ghost in the machine1.1Anoverviewofgenerativeartificialintelligenceelsthataretrainedto“connectthedots”betweendat

31、apointsinexistingcontenttogeneratesyntheticcontent.Generatedcontentisprobabilisticallyandrandomlygen-eratedbasedoncertaininput(orprompts),whichareusuallywrittenbyahuman.Therefore,theoutputofanygivengenerativeAImodelislikelytobedifferentforeachpersonpromptingthemodelandmaybothresemblepatternsinthetra

32、iningdataorappeartobesomethingcompletelynew.1.1.1 EXAMPLES OF GENERATIVE AI MODELSTherearevarioustypesofgenerativeAImodels,includ-inglargelanguagemodels(LLM)thatcanrespondtotextbygeneratingnewtext,andmulti-modalmodelsthatcangeneratemorethanonetypeofoutputorrespondtomorethanonetypeofinput,forexamplec

33、hatbotsthatcanalsogenerateimageswhenpromptedtodoso.AshortintroductionofthemostpopulargenerativeAImodelsonthemarkettodayispresentedbelow,accom-paniedbysomerelevantexamples.1.1.1.1 Text generatorsText generators are a type of generative AI model that cangeneratetextpassagesbasedonpredictiveanalysis,wh

34、icharebuiltonlargelanguagemodels.2 These models are usually trained on enormous amount of textscrapedfromtheinternet,includingbooks,forums,newssites,socialmedia,etc.Textgeneratorscanbeusedforwritingessays,coding,chatbots,andaugmentingsearchengines,amongstotherthings.Inmanycases,textgeneratorsaremean

35、ttogeneratetextthatappearstobewrittenbyahuman,forexamplebygeneratingtextwritteninafirst-personperspective,usingemojis,orbywritingtextindicatingthatithasthecapabilitytoexperiencehumanemotions.Sometextgeneratorsaremultimodalandcangeneratetextbasedonimages.Although text generators have existed in somef

36、ormforseveralyears,forexampleaspredictiontoolsfortypingtextmessag-es,thediscussionaroundthetechnologygainedmomentumduringtheautumnof2022,withthepublicreleaseofChatGPT,whichisownedandoper-atedbythecompanyOpenAI(whichisalsotheownerofDALL-E,seebelow).ChatGPT3isavailableonlineforthosewhocreateafreeaccou

37、nt,whilethemorepower-fulChatGPT4modelisavailableforamonthlysubscrip-tion fee.3 InJanuary2023Microsoftannouncedamajorinvest-mentinChatGPTandlaunchednewfeaturesintheBingsearchenginepoweredbythetechnology.4MicrosofthasannouncedthatitintendstointegrateChatGPTintoitsotherservices,includingtheMicrosoftOff

38、icesuiteofapplications,forexampletoautomaticallytakenotesduringmeetingsinMicrosoftTeams.5GooglehasalsodevelopedaLLMthatcangeneratetext,calledLaMDA.InthewakeofMicrosoftsinvestmentinChatGPT,GooglerolledoutsimilarfeaturestoitssearchenginewithatextgeneratorcalledBard.6 Google is alsoplanningtointroducev

39、ariousAI-poweredfeaturessuchasdraftingandsummarizingemails,aswellasbrainstormingandwritingdocumentsinitsWorkplaceapplications.7 2LargelanguagemodelsaresophisticatedAImodelsthataredesignedto generatetextthatresembleshumanlanguage.Theyarenormallytrainedonvastamountsoftextsourcesto“learn”patternsandgra

40、mmar.LLMscanbeusedfortaskssuchasmachinetranslation,sentimentanalysis,hu-man-machineinteraction,proofreading,andmanyotherpurposes.Poem about the consumer issues of generative AI,ChatGPT.9Norwegian Consumer CouncilJune 2023Ghost in the machineMetahasdevelopedtheLLMGalacticatrainedonsci-entificarticles

41、andmaterials,whichismeantto“store,combineandreasonaboutscientificknowledge”.AfterthemodelwasreleasedasapublicdemoinNovember2022,thepublic-facingdemowasquicklyremovedduetogeneratingtextcontainingmultipleerrorsandbiases.8 InFebruary2023,MetareleasedanotherLLM,calledLLaMa(Large Language Model Meta AI).

42、LlaMa is an open sourcemodel,whichwasinitiallyreleasedtoresearchersbasedonanaccessapplicationprocess.InMarch2023themodelwasleakedonapublicmessageboard,mean-ingthatanyonewitharelativelypowerfulcomputercandownload,use,andadaptthemodel.9 TherearealsoseveralopensourceLLMsthataredevel-opedandmaintainedby

43、smalleractors.Forexample,thetextgeneratorBLOOMisavailablethroughthecompanyHuggingFace,10whileStabilityAIhasreleasedopenmod-elsunderthemonikerStableLM.111.1.1.2 Image generatorsGenerative AI models that are trained to generate images canbecollectivelyclassifiedasimagegenerators.Theycancreateimagesfro

44、mtext prompts(text to image)or from existing images(image to image).Image generators workbyanalysinghugeamountsofexistingpictures,suchasphotographs,paintings,etc.,whichareoftenscrapedfromvarioussourcesontheinternet.By training the algorithm on thesedatasets,themodelcangenerate images of different ob

45、jects(achair,atrain),people(ayoungwoman,JerrySein-feld),andstyles(impressionism,inthestyleofEdwardMunch).Themostwidelyusedimagegenerators as of June 2023 are Midjourney,12 DALL-E,13 and Sta-ble Diffusion.14 MidjourneyisavailablethroughthechatserviceDiscord.ItispossibletojointheofficialMidjourneyDisc

46、ordserv-er,toaskaMidjourneybotto“imagine”apicturebasedonvariousprompts.Forexample,thepersonpromptingthesystemcouldtype“/imagine hyper realistic photo of political advisor writing a paper on generative ai,in an open office plan”.Thebotrespondsinthechatwithfourgeneratedpictures.WhileMidjourneywasfreet

47、otryforthefirstfewmonthsafteritsrelease,foralimitednumberofgeneratedimages,ithassincebecomeapaidsubscriptionservice.ThecompanyMidjourneyInc.ownsthegenerativeAImodelthatgeneratestheimagesandrunsandcontrolsboththemodelitselfandtheserversthatitishostedon.Thismeansthatthecompanycanrestrictaccess,changet

48、hemodel,andaddcontentfilterstocontrolwhatkindofimagesthemodelcanandcannotgenerate.ThegenerativeAImodelDALL-EisaccessiblethroughthewebsiteofitsownerOpenAI.Individualscancreateanaccountandreceivealimitednumberoftokenseachmonththatcanbeusedtogenerateimages.ImagesareHyper realistic photo of political ad

49、visor writing a paper on generative ai,in an open office plan,Midjourney.1.1Anoverviewofgenerativeartificialintelligence10Norwegian Consumer CouncilJune 2023Ghost in the machinegeneratedbyenteringdifferentpromptsintothewebsiteinterface.Ifthepersonrunsoutoftokens,theycanpaytoreceivemore.LikeMidjourne

50、y,themodelbehindDALL-Eisowned,controlled,andoperatedbyitsparentcompany.ThegenerativeAImodelStableDiffusionisdevelopedbythecompanyStabilityAI.UnlikeMidjourneyandDALL-E,StableDiffusionisanopensourcemodelthatcanbefreelydownloadedbyanyone,anddoesnotrequireasubscriptionoraccesstotheinternettouse.Oncethen

51、ecessarysoftwareisdownloaded,unlimitedimagescanbegeneratedlocally.RunningStableDiffusionlocallyonlyrequiresacomputerwithrelativelypowerfulcon-sumer-gradegraphicscard.StabilityAIonlytrainsanddistributesthebasicmodelsforStableDiffusion,whileitispossibleforanyonewithaccesstothemtocontinuetraininganddev

52、elopingnewmodelsthatarebasedontheoriginalStableDiffusionmodels.Thesenewmodelscanthenbedistributedtoothers.Inpractice,thismeansthatthecompanydoesnotcontrolthemodelnoritsoutput.1.1.1.3 Audio generatorsAudiogeneratorsusegenerativeAItechnologiestocreateaudioclipsbasedontextprompts(forexampletexttovoice)

53、.Suchmodelsaretrainedonexistingvoicedata,music,etc.AudiogeneratorscanbeusedtocreateAI-generatedmusic15andvoices,andtherearemodelscapableofrecreatingthevoiceandpitchofindividualindividuals.16 Forexample,thecompanyElevenLabshasreleasedamodelthatallowsanyonetoconvertshorttextinputintovoiceclips,inasele

54、ctionofdifferentvoices.17Micro-softhasannouncedthegenerativeAImodelVALL-E,whichthecompanyclaimscangeneraterealisticvoicesbasedonathreesecondvoicesample.18AsofMay2023,VALL-Ehasnotbeenreleasedtothepublic.1.1.1.4 Video generatorsVideogeneratorscanbeusedtocreatevideoclipsbasedontextprompts(texttovideo),

55、images(imagetovideo),orexistingclips(videotovideo).Asitismorecompli-catedtogenerateauthenticlookingvideofootagethantomakestillimages,thistechnologyissomewhatlessdevelopedatthetimeofwriting.However,thismaychangeinthenearfuture,asseveralmajorcompaniesareactivelyworkingonmodelsforgenerating video.An oi

56、l painting by Munch of a policy advisor writing a paper on generative AI,DALL-E.A photo of a consumer policy advisor writing a paper about the consumer challenges of generative AI,in an open office plan,Stable Diffusion 1.5.1.1Anoverviewofgenerativeartificialintelligence11Norwegian Consumer CouncilJ

57、une 2023Ghost in the machineMeta has developed a model to turn short texts into video clips,20andGooglehasannouncedasimilarsystem.21 As ofMay2023,neitherofthesesystemshavebeenmadeavailabletothepublic.StabilityAI,thecompanybehindStableDiffusion,hasreleased a model for generating animations from text

58、prompts and images.22ThecompanyRunwayhasre-leasedamobileappthatcanbeusedtogenerateshortvideoclipsfromexistingvideos.231.1.2 THE GENERATIVE ARTIFICIAL INTELLIGENCE ACTOR CHAINFromassemblingdatasetsfortrainingmodelstode-ployingandpromptinggenerativeAIsystems,therearepotentiallymanydifferentactors.Thes

59、eactorsmayallinfluencethesystemorhowitisusedinvariousways.Therelevantactorsareshownintheillustrationjustbelow.Eachboxrepresentsadifferentactorintheactorchain.Theseactorsmayallbewithinoneorganisationbutwilloftenbespreadacrossseveralorganisations.Thearrowsareonlypointingonewaytomaintainasimplerepre-se

60、ntation;itisofcoursepossiblethattherecouldbefeedbackloopsbetweendifferentactors.ThedatasetassemblercollectsandsystemizesthenecessarydatatotrainagenerativeAImodel.Therearenumerousavailableopensourcedatasetsthathavebeencompiledandlabelledbyscrapingahugenumberofsourcesonline.Inmanycases,suchdatasetsare

61、compiledforresearchpurposes,andmadefreelyavailable.Acom-panydevelopingagenerativeAImodelcanthustraintheirmodelondatasetsthatsomeoneelsehasassembled.ThedevelopersofgenerativeAImodelscreateabaselinemodel,whichmaythenbetrainedandtunedforcertainmorespecificcontextsorapplicationsbydownstreamde-velopers.I

62、nsomecases,thisfine-tuningofthemodelmaybedonebythesameactorasthetrainingofthebaselinemodel,whileinothercases,thefine-tuningmaybedonebyaseparateactorentirely.Thismaybeanothercompany,orinthecaseofopensourcemodels,themodelcanbefine-tunedbyanyonewitharelativelypowerfulcomputer.Tofurthercomplicatethematt

63、er,asgeneral-purposegenerativeAImodelsarebeingintegratedintootherapplications,thecompanyorentitydeployingthesystemmaybeseparatefromthecompanydevelopingthemod-eland/orfine-tuningit.Finally,thereareendusers,engagingwiththedeployedmodel.Inconsumerusecases,theconsumerwilltypi-callybeanactor,bypromptingt

64、hemodeltogenerate,forexample,atextoranimage.Consumersmayalsoindi-rectlyencountergenerativeAIsystemswheninteractingwithabusiness,forexampleifacustomerserviceagentusesatextgeneratortogenerateanswerstoconsumerqueries,orifaconsumerispromptedtomakecertainqueriesbasedonAI-generatedrecommendations.Thereare

65、numerousactorsinthegenerativeAIsystemactorchain.ItiscrucialtounderstandtherelationshipbetweentheseactorstounderstandhowgenerativeAIshouldberegulated,andatwhichpointtheactorchaindifferent harms arise.Illustration of different actors in the generative AI actor chain.Data set assemblerDeveloper of base

66、linemodelDownstream developerDeployerEnd userFrame from video generated in Make-A-Video,Meta AI.191.1Anoverviewofgenerativeartificialintelligence12Norwegian Consumer CouncilJune 2023Ghost in the machine1.1.3 OPEN SOURCE OR CLOSED SOURCE MODELSTherearesignificantdifferencesinhowgenerativeAImodelsared

67、istributedandcontrolled.Manymodelsareproprietaryclosedsourcemodelsthatrunoncloudserverscontrolledbytheownerofthesystem(the“sys-temowner”).Thismeansthatconsumerscanaccessthemodelthroughtheinternet,andthatthesystemprovidercanchangethemodelatanytime,addcontentfilters,re-strictaccess,etc.Forclosedsystem

68、s,thesystemownerprovidestheprocessingpowerrequiredbothtotrainthemodel,andtogeneratesyntheticcontent.ForaclosedsourcegenerativeAImodel,itisnotpos-sibletoknowhowthemodelworks,whatdataitwastrainedon,andhowparametersareweighed,unlessthecompanybehindthemodelpublishessufficientdocu-mentationorprovidesacce

69、sstothirdpartyauditors,enforcementagencies,orresearchers.Inmanycases,suchinformationmaybekeptsecretduetosecurityand/orbusinessinterests.Ontheotherhand,somegenerativeAImodelsarereleasedasopensource,whichmaytakevariousforms.Differentpartsofthesystem,suchasthedataset,sourcecode,modelparametersandweight

70、s,maybemadeavailabletothirdparties.Whenthesourcecodeismadeavailabletothepublic,itcanbeused,studied,tested,modified,anddistributedbyanyone.Thismeansthatitcanbeinspectedforerrorsandvulnerabilities.Itcanalsobeimprovedanditerateduponcollaboratively.Using,modifying,anddistributingopensourcesoftwareisgene

71、rallygovernedbylicenseterms.Giventheopensourcesoftwareandthedataset,anyonewithsufficientcomputingresourcecouldrepro-ducethegenerativeAImodel,althoughthecomputingresourcesneededtodosoaresignificantenoughthatinpracticeitwilllikelybelimitedtolargecompanies.However,forgenerativeAIsystemstobetrulyopensou

72、rce,themodelitselfshouldbemadeavailabletothepublic.Insuchacase,applicationsdevelopedbasedonthesemodelsmightalsobereleasedasopensourceap-plication,suchastheimagegeneratorStableDiffusion,oritcanbeadaptedintoaclosedsourceapplication.OpensourcegenerativeAImodelscanbedownloadedbyanyone.Withapowerfulenoug

73、hcomputer,itcanbeusedto generate data and update the model as one pleases.Thesourcecode,parameters,etc.ofsuchmodelscanbeinspectedbyanyone.However,thisdoesnotnecessarilymeanthattheycanunderstandhowthemodelworks,assufficientlycomplexmodelsmaybemoreorlessimpos-sibletounderstandtheinnerworkingsof.Oncean

74、opensourcegenerativeAImodelisreleasedtothepublic,thereispracticallynothingthedeveloperofthebaselinemodelcandotoinfluencehowthemodelfunctions.Thismeansthatanycontentfiltersandotherartificiallimitersplacedonthemodelmaybealteredorremovedbydownstreamdevelopersordeployers.Thiscreatesbothbenefitsandsignif

75、icantdrawbacks,whichwillbeelaboratedonbelow.1.1.4 GENERAL PURPOSE AIWhilesomeAImodelsaredesignedwithaspecificpurposeandusecaseinmind,suchasearlydiscoveryofcancercells,manygenerativeAImodelsareexamplesofso-calledgeneralpurposeartificialintelligence.Thismeansthatthebasicsystem,suchasalargelanguagemode

76、l,istrainedtobeabletorespondtoavastvarietyofsituationsandinteractionsandcanbeadaptedtobeusedinnewcontexts.Unlikeamodelwithaspecificpurpose,itisextremelydifficultorimpossibleforthedevelopersofageneralpurposeAImodeltoforeseethepossibleusesandabus-esofthetechnology.Thismakesitparticularlyimportantthats

77、uchmodelsaresubjecttotechnical,scientific,legislative,andregulatoryscrutinybeforetheyarewidelyadopted.However,applicationssuchasChatGPThavealreadybeenreleasedtothewiderpublicwithoutrigorousevaluation,impactassessmentorscrutiny,whilebeingincreasinglyopaqueandinaccessibletothirdpartyauditorsandresearc

78、hers.24Itisworthconsideringwhetherthisleadstoadesirablefuture,consideringthemanyharmstoucheduponinchapter2ofthisreport.1.1Anoverviewofgenerativeartificialintelligence13Norwegian Consumer CouncilJune 2023Ghost in the machine1.2Consumerapplications1.2 ConsumerapplicationsSeveraldifferenttypesofgenerat

79、iveartificialintelli-gencearepubliclyavailabletoconsumerstoday.Manyofthesearereadilyavailabletousebyanyonewithaninternetconnection,anddonotrequireexperttechnicalknowledgetouse.Someofthemaredirectlyaccessiblethroughwebinterfaces,whilegenerativeAItechnologyisalsoincreasinglybeingintegratedintootherser

80、vicessuchasonlinesearch,learningandadministrationsoft-ware,andsocialmedia.AsofMay2023,themostpopularconsumerusesforgenerative AI models are text and image generation.However,withmajorconsumer-facingcompaniessuchasMicrosoft,Meta,andGoogleinvestingheavilyinthetechnology,theusecasesarelikelytoexpandint

81、hecomingmonths,asgenerativeAImodelsareimplement-edinvariousservices.Forexample,textgeneratorscanbeausefultooltostreamlineand/oroptimizemundanetasks,functioningasakindofmulti-purposedigitalassistant.Thismayincludechanginginternetsearchfunctions,automatingcertaintaskssuchaswritingcode,transcribingvoic

82、emessages,orpersonalizingservicesinvariousways.Whilesuchapplicationsmaybeusefulandefficientincertaincontexts,therearealsosignificantrisksanddrawbacks,whichwillbeexploredfurtherinthefollow-ingchapters.Asthetechnologyisdevelopedandadopted,generativeAImaybeusedtoautomatetediousandtime-consum-ingprocess

83、esthatpreviouslyhadtobedonemanually,forexamplebywritingconcisetexts,fillinginforms,generatingschedulesorplans,orwritingsoftwarecode.Ithasthepotentialtomakeservicesmorecost-efficient,whichmayalsolowercostsforconsumers,forexamplewhensolicitinglegaladvice.25Ontheotherhand,theproliferationoflow-costAI-g

84、eneratedcontentmayre-placehumanlabourandhuman-generatedcontent,thusloweringthequalityofconsumer-facingservicessuchascustomersupport.Thetechnologyalsoopensnewavenuesforconsumermanipulationinareassuchasad-vertisingorproductrecommendationsandcanfacilitateorobfuscatediscriminatorypractices.2.HARMS AND C

85、HALLENGES OF GENERATIVE ARTIFICIAL INTELLIGENCE“From violations of privacy and personal integrity to the creation of fraud and misinformation,generative AI models introduce vast risks and challenges,while turbocharging others.”15Norwegian Consumer CouncilJune 2023Ghost in the machineTherehavebeensev

86、eralcontroversiessurroundingthedevelopmentanduseofgenerativeartificialintelli-gence.Fromviolationsofprivacyandpersonalintegritytothecreationoffraudandmisinformation,generativeAImodelsintroducevastrisksandchallenges,whileturbochargingothers.Concreteandhighlyrelevantexamplesofthisarechatbotsandsearche

87、nginesprovid-ingincorrectbutconvincinginformation,theabuseofcheaplabourintheglobalsouthforcontentmoderation,andasignificantenvironmentalimpactduetoresourceconsumption.Itisessentialthattheseproblemsaresuf-ficientlyaddressedbyenforcing,applying,andestablish-inglawsandregulationsthatservetoprotectconsu

88、mersfromvariousnegativeconsequences.TheissuesdiscussedthroughoutthisreportareoftennotneworuniquetogenerativeAI.Algorithmiccomput-ersystemshaveexistedforacentury,whilethetech-nologypopularlyreferredtoasartificialintelligencehasbeenaroundsincethe1950s.Inthe1960s,thecomputerscientistJosephWeizenbaumcre

89、atedELIZA,amodelthatsimulatedhumaninteraction,usingrule-basedalgorithms.26PeoplewhointeractedwithELIZAat-tributedhumanattributesandemotionstothemodel,eventhoughtheywereinformedthatthesystemhadnosuchcapability,mirroringsomeusecasesofgenerativeAI-poweredchatbotstoday.Issuesrelatedtocontentmoderation,a

90、lgorithmicbias,privacy,anddisinformationhavebeendebatedatalmosteveryjunctionasdigitaltechnologyevolvesandiswidelyused.However,thedeploymentandpublicadoptionofsystemssuchasChatGPT,bothfortechnicallyadeptconsumersandthegeneralpublic,alongsideitseaseofuseandwide-scaleavailability,meansthatmanyoftheseis

91、sueshavebecomeurgentlyrelevanttoanalysefromaconsumerperspective.Asdescribedinthefollow-ingchapters,anumberoftheseissuesmaybeaddress-ablebyenforcingapplicablelawsandregulations,whileother may require other solutions or remedies.2.1 StructuralchallengesofgenerativeAIAtabasiclevel,generativeAImodelsare

92、fundamentallydesignedtoreproduceexistingmaterial,althoughinpotentiallynovelways.Thismeansthatsuchmodelsareinherentlypronetoreproducingexistingbiasesandpow-erstructures.Therefore,whilethemodelshavenoun-derstanding,mind,orintentionoftheirown,thedecisiontodevelop,deployandusethemisinherentlypolitical.I

93、tisnotsufficienttoascribeneutralityorobjectivitytotheoperationsoroutputsofagenerativeAImodel,becauseitstrainingdataandalgorithmsstemsfromhumanbe-ings,withallthatthisentails.AsgenerativeAImodelsarebeingintroducedintoallsectorsofsociety,sofarwithlittleornoregulatoryoversight,therearefundamentalissuest

94、hatneedtobeaddressed.GenerativeAImodelsaredependentonlargeamountsofdatathatistakenfromamultitudeofsources,usuallywithouttheknowledgeorconsentoftheoriginatorofthedata,beitapieceofart,anewsarticle,oraselfie.Informationissiphonedandgatheredtobeusedindifferentways,withanendgoalofenrichingasmallnumberofc

95、ompanies.Thisraisesquestionsofvaluedistribution,usagepermission,privacy,account-ability,intellectualproperty,andhumanrights.27 2.1.1 IDENTIFYING THE CONCRETE RISKS OF GENERATIVE AIAswithanynewtechnologies,thediscoursearoundgen-erativeAIismuddledwithamixoffacts,concerns,andalot of hype and enthusiasm

96、.28ManyAIsystemsarebeingtoutedasbeingcapableofsolvingalmostanytask,oftenwithoutevidencetobackuptheclaims,aphenomenonthatcanbedescribedasAIsnakeoil.29 When addressing problematicandhazardousissueswiththetechnology,itisimportanttobeabletosortfactsfromfiction.2.Harmsandchallengesofgenerativeartificiali

97、ntelligence“At a basic level,generative AI models are fundamentally designed to reproduce existing material,al-though in potentially novel ways.This means that such models are inherently prone to reproducing existing biases and power structures.”16Norwegian Consumer CouncilJune 2023Ghost in the mach

98、ine2.1StructuralchallengesofgenerativeAIHeavilypublicizedwarningsaboutthedangersofarti-ficialintelligencehaveconcentratedonhypotheticalrisksofdevelopinganartificialgeneralintelligence(AGI),meaningasystemthatisabletoperformintellectualtasksthatarecomparabletotheabilityofhumanbeings.Theoretically,such

99、systemsshouldbeabletothinkandreason,andbeabletoperformabroadrangeoftasksthatequalhumancapacityforthinking.Thisdifferscon-siderablyfromgenerativeAImodels,whichhavenosuchcapacities.AsAGIsystemsdonotcurrentlyexist,andthereareseriousdisputesaboutwhethertheycaneverberealized,suchsystemswillnotbefurtherco

100、nsideredin this report.Therehavebeencallsforvoluntarymoratoriumsorpausing of developing generative AI models.Some of thesecallshavefocusedonapotentialfuturewhereAIsystemshavebecomesopowerfulthattheyposeanex-istential threat to humanity.30Whilesuchcallsacknowl-edgeissuesrelatedtoaccountability,safety

101、,andcontroloverAIsystemsingeneral,thereareseriousconcernsthatfocusingonpotentiallong-termscenariosaredraw-ingattentionawayfrommanyofthecurrentpressingissuesofgenerativeAI,potentiallyleavingtheseissuesinsufficientlyregulated.31 Theargumentthatahypotheticalgeneralartificialintelli-genceisanexistential

102、threattohumanityimpliesthatconcernsaboutcurrentissuessuchasdiscrimination,privacy,andfairness,areinconsequentialandmarginal.32 Inotherwords,thenarrativeconcerningapotential“AIsupermind”mayserveasadistractionfrompressingissuesthatarealreadypresentintodaysapplicationofgenerativeartificialintelligence.

103、Itiscrucialthatnarra-tivesabouthypotheticalexistentialthreatstohumanitydonotcomeinthewayofproposingconcretesolutionstotheveryrealissuesposedbythetechnologythatexists today.33“There are serious concerns that focusing on potential long-term scenarios are drawing attention away from many of the current

104、 pressing issues of generative AI,potentially leaving these issues insufficiently regulated.”17Norwegian Consumer CouncilJune 2023Ghost in the machine2.1.2 TECHNOLOGICAL SOLUTIONISMArtificialintelligenceisoftenlaudedasthesolutionforavastnumberofissuesacrosssectors,fromhealthcareandpublicadministrati

105、ontolegalassistance.Whilethisnarrativeisattractivetobothprivateenterpriseslookingtosellsoftwaresolutions,andtopolicymakerssearchingforsimpleremediestopoliticalorregulatoryailments,itneedscriticalexamination.Thebeliefthatalmostanyissuecanbeimprovedorsolvedusingtechnologyisknownastechnologicalsolution

106、ism.AtermcoinedbythetechcriticEvgenyMo-rozov,technologicalsolutioniststendtoglossovercom-plexandmultifacetedsocialproblemsinfavourofsimplemathematicalorengineeringsolutions.34Thisreduc-tionistbeliefisattractivetoservice-providersbecauseitallowsthemtoadvertisemiraclecuresAIsnakeoilandtopolicymakersbe

107、causetechnologicalquickfixesaretangibleandusuallyappearmorecost-efficientthanexaminingcomplicatedandoftendeep-rootedsocialandpoliticalconflictsorinequalities.AsMorozovpointsout,technologicalsolutionismisdangerousbecauseitoftensimplydoesnotwork.Bypresentingmultifacetedandcomplexissuesasamereengineeri

108、ngissuetobesolvedinalab,solutionismmis-representssocialproblemsandmissestheunderlyingcauses.Whenpresentedasproblemsthatcanbesolvedbytechnology,solutioniststendtodisregardthesocial,political,andculturalcontextthatisthebackboneofoursocieties.Whenconsideringtheproliferationofartificialintelli-gencemode

109、lsthatarerapidlybeingdeployedacrosssectors,itisworthkeepingthefollyoftechnologicalsolutionisminmind.ThisisparticularlyimportantifgenerativeAImodelsorsimilartechnologiesarebeingpushedasaremedyorsolutiontoinequalities,suchasprovidingaccesstomentalhealthtoolstopeoplewhootherwisewouldnotbeabletoaffordit

110、.Whileitmayappearattractivetooutsourcementalhealthcaretoalargelanguagemodelthatcanbedeployedandaccessedatarelativelylowmonetarycost,thisapproachrisksreducingthecomplexityofmentalhealthandthevalueofhumancontacttoaquestionofpredictiveanalysisand language modelling.35Similarly,beforedecidingtodeployate

111、xtgeneratorasasolutionforoverworkedcasehandlersinthepublicsector,forexample,itiscrucialtoconsiderthecontextandcausesoftheproblem,ratherthanadoptingdevelopingtechnologiesasablanketsolution.Ifsuchtechnologicalquickfixesareadoptedattheexpenseofinvestmentintoproveneffectivemeasures,whichareoftencostlyan

112、ddifficulttoimplement,thismaycomeatsignificantcoststomarginalizedgroupswhoriskbeingdeprivedofeffectivetreatmentandmeasuresbecausetheirissuesarepurportedlybeingaddressedbytheuseoftechnology.2.1.3 CONCENTRATING POWER IN THE HANDS OF BIG TECHAtthebaseofthediscoursearoundgenerativeartificialintelligence

113、isaquestionofpower.GenerativeAImodelsareproductsofculturalandpoliticalcontexts,acontextthatisembeddedineverythingfromthedecisiontode-velopthemodel,thechoiceoftrainingdata,thetuningofmodels,andthegivenpurposesfordeployment.Assuch,thealreadypowerfulcanpotentiallyentrenchexistingpowerstructuresthrought

114、hetechnology,whilethedisenfranchisedwillremainsounlessthereisoutsideintervention.ThisbecomesapparentwhenagenerativeAImodelgeneratesbiasedordiscriminatorycontent,butalsomanifestsinaspectssuchascontentmoderationpracticesandinwhohasaccesstothesystems.AsgenerativeAImodelsareoftentrainedondatacol-lectedf

115、romanyavailablesources,someactorsarerais-ingquestionsaboutwhetherprivatecompaniesshouldbeallowedtousethecollectiveknowledgeofhumanitytoturnaprofit.Thevastamountofinformationthatcanbefoundopenlyavailableonlinecanbedescribedasadigitalcommons,asitisabodyofresourceswherepracticallyeveryoneisacontributor

116、,fromindividualpiecesofdatatothepublicinfrastructureoftheinter-net.Ifthedigitalcommonsaresiphonedtodevelopandtrainproprietarymodels,thisraisesethicalconcernsabouthowvaluegeneratedonthebasisofthesecom-monresourcesshouldbedistributed.36Theseconcernsextendtodatagovernanceissuesregardingwhoshouldcontrol

117、howdataisused,suchasifatechcompanywantstocommercializeAImodelstrainedonindigenouslanguages.37ThequestionofwhocontrolsthedevelopmentandtrainingofgenerativeAImodelsandhowtheyareusedisoffundamentalimportance.Thosewhocontrolthetechnologyhavesignificantpotentialtocreatede-pendencies,setthetermsofuse,andd

118、ecidewhohasaccess.Thisentrenchmentofpowercreatesoverarch-ingconcernsaboutleadingtechcompaniesbecominggatekeepersthatcanexcluderivalsandotherwiseabusetheirincreasinglydominantmarketpositions.38 While 2.1StructuralchallengesofgenerativeAI18Norwegian Consumer CouncilJune 2023Ghost in the machineopensou

119、rcemodelsmaylowerthebarrierofentryforcertaintypesofgenerativeAI,39suchmodelsarestilllargelydependentonafoundationalmodelthathasbeendevelopedbyactorswithaccesstosignificantcomput-ingresourcesandtrainingdata.ThismeansthatalreadydominanttechnologycompaniessuchasMicrosoft,Google,andMeta,arewellpositione

120、dtoseizethemarketforgenerativeAIs.Withproprietaryclosedmodel,thesystemownerhascontroloverwhocanaccessthetechnology,whatitcosts,itsfeatures,andhowitmaybeused.Thismayinturnaffectaca-demia,attractingwhatcouldhavebeenindependentresearcherstoworkwithinthecloseddomainsofbigtechnologycompanies,wheretheyhav

121、eaccesstostate-of-the-arttechnologyinthefield.Overall,thismeansthatthetechgiantscanfurtherleveragetheirdominantpositionsacrossdifferentonlinemarketsinthefieldofgenerative AI.40 WithonlyafewgenerativeAImodelsavailableonthemarket,thesemodelsareintegrated into a variety of services,providingthe modelow

122、nerswithsignifi-cantpower.Modelscanbepatchedorotherwisemodi-fied,functionalitycanbeaddedorremoved,andcontentcanbebanned,filtered,orotherwiserestricted.Ifthesystemownersetsthetermsforhowitstechnologymayormaynotbeused,endusersorthird-partycompaniesintegratingthemodelareatthemercyoftheowner.Somecompeti

123、tionconcernsmaybesomewhatalleviatedinthecaseofopensourcegenerativeAImodels,whicharenotnecessarilybeholdentothebusinessmodel,ob-jectives,orwhimsofamodelsoriginalcreator.However,eveninsuchcases,manycompaniesdonothavethemeanstocompetewithbigtechfirmsinofferinggenera-tiveAIsolutionstoconsumers.Largecomp

124、aniesbenefitsignificantlyfromnetworkeffects,asmoreusersmeansmoredata,whichleadstobetterservices.Incaseswheremodelsarefurthertrainedonconsumerinterac-tionsorfeedback,thecompaniescanfurtherimproveandfine-tunethemodelsataratethatisunattainableforsmallercompetitors.Dominantactorscanfurtherentrenchtheirp

125、owerbyintegratinggenerativeAIintotheirownservicesthatarealreadyusedbymillionsacrosstheworld.Forexample,byrollingoutitschatbotBardaspartofitssearchen-gine,Googlealreadyhasamassiveglobaluserbasethatcanbeleveragedtoboosttheadoptionofthechatbot.Similarly,asMicrosoftimplementsChatGPT-basedmod-elsintoitsO

126、fficesuiteofapplications,thecompanyalreadyhasauserbasethatcompetitorscanonlydreamof.CompaniescanalsoleveragetheirmarketpositionbymakingtheusageofagenerativeAImodeldependentonusingadifferentservicefromthesamecompany,bybundlingservicestogether.Forexample,tohaveaccesstoMicrosoftsBingchatbotfunctionalit

127、y,consumersmustuseMicrosoftsEdgebrowser.41 TheintegrationofgenerativeAImodelsintoservicessuchassearchenginescanalsosignificantlylimitconsumerchoice.Forexample,inaregularonlinesearchengine,theconsumerispresentedwithnumeroussearchresultsthattheymaychoosebetween.Ifthesearchengineisreplacedbyatextgenera

128、torthatprovidesasingleanswertoanyquery,thispoten-tially limits the information available.Ifsimilarmodelsareusedforonlineshopping,thiscreatesnewavenuesforplatformstoself-preferenceproducts,byensuringthattheplatformspreferredproductistheonlyortheprimarilysuggestedpurchase.Iftheconsumerqueries“whatisth

129、ebestcoffeemachinefortosuitmyneeds?”,itwillbenecessarytomonitorandcontrolhowachatbotor“shoppingassistant”landsataparticularresultorrecommendation.2.1.3.1 Walled gardens and its downstream effectsInordertomaximizeconsumerengagement,manydig-italserviceprovidershaveafinancialincentivetokeepconsumersont

130、heirplatformsaslongaspossible.Thisgoalcanbeattainedbyintegratingandbundlingasmanyservicesintotheplatformaspossible,whilecreatingbarrierssuchasnotprovidingserviceinteroperabilitytodisincentivizeconsumerstoleavetheplatform.Plat-formsandservicesdesignedtokeeptheconsumerfromleavingarecalledwalledgardens

131、.42 Theintegrationofgenerativeartificialintelligenceintovariousplatformsalreadyseemstofacilitateawalled2.1StructuralchallengesofgenerativeAI“Those who control the technology have significant potential to create dependencies,set the terms of use,and decide who has access.”19Norwegian Consumer Council

132、June 2023Ghost in the machinegardenapproachthatmayhaveseriousanticompetitiveeffectsonbothdirectcompetitorstothelargeonlineplatforms,andacrossmarkets.SnapchatisforinstanceintroducingrecommendationsforrestaurantsorrecipesinitsAI-chatbot,43whichwouldreduceconsumersneedtoaccessotherservices,suchastradit

133、ionalsearchengines,forthesekindsofqueries.Thisislikelyasignofthingstocome,asmajorplatformsare slated to integrate generative AI models into their services.Ascompaniescompetetodevelop“killerapps”,servicesthatintegrateasmanyfunctionsandpurpos-esaspossible,theseproblemsaresettoexacerbate.Newcomersmayfi

134、nditincreasinglydifficulttoprovidestand-aloneservicestoconsumers,astheyhavefewerreasonstoexittheirapplications.Thiswouldservetoconcentratepowerwithalreadyestablishedactors,effectivelyharmingtheconsumermarket.TheintegrationofgenerativeAIintosearchenginesissparkingmajorconcernsforpublishersandadvertis

135、ers,sincesuchintegrationcaneffectivelycreatewalledgardens.44Withtraditionalsearchengines,theconsum-ercouldsearchforatopicandbepresentedwithalistoflinkstowebsitesthatcontaininformationaboutthetopic.Theconsumerwillthenclickononeormoreofthelinksandberedirectedtoawebsite.Consequently,thewebsiteownergene

136、ratesrevenuebydisplayingadstotheconsumer.ThisdynamicmaychangewiththeintroductionofgenerativeAI.Googleisforexampleexperimentingwithintroducinggenerated,summarizedcontentinitssearchengine,whichfillsupthefirstpageonasmallerscreen(suchasaphone).45Inshort,ifconsumerscansimplyaskachatbotaboutasubjectandre

137、ceiveanswersinthesameinterface,theincentivetovisitathird-partywebsiteisreduced.Iftheconsumerdoesnotvisitthethird-partywebsite,theoriginatorofthecontentcannotmonetizethecontentbydisplayingads.Alackoftrafficposesaproblemtopublishers,whosecontentmaybescrapedtodisplayinformationinthroughthechatbotinterf

138、ace,butwhomaystruggletomonetizethecontent.Thiscanthereforehavedownstreameffectsbyreducingtheincentivetoproducequalitycontent,po-tentiallyleadingtocontent-productionbecomingautoma-tizedaspartsofcost-cuttingmeasures.Itisalsowell-es-tablishedthatpowerconcentrationamongafewactorsisseldomlybeneficialtohe

139、althyconsumersmarkets.2.1.3.2 Data colonialismIf generative AI models are trained on data sets that wereindiscriminatelyscrapedfromtheinternet(digitalcommons),thismayalsoentaillargeamountsofdatafrom indigenous and other minority groups.The informa-tioncanthenberepackagedandusedinnewways,forexampleto

140、selltechnologyorservicesbasedonthedatabacktothegroupsfromwhomthedataoriginated.Theprocessbywhichorganizationsandcorporationsclaimownershipoverdataproducedorharvestedfrompeopleiscalleddatacolonialism.Theconceptofdatacolonial-ismishighlyrelevantwhendiscussingtheoperationsofgenerative AI models.Forexam

141、ple,indigenouscommunitiesinNewZealandhaveexpressedconcernsregardingthedevelopmentoflargelanguagemodelsbeingtrainedonhundredsofhoursofMoriindigenouslanguage.Communityleadersandresearchersfearthat“ifIndigenouspeoplesdonthavesovereigntyoftheirowndata,theywillsimplybere-colonizedinthisinformationsociety

142、”.46Thelanguageharvestedwithoutconsentcanbedistort-edandleadtoabuseanddeprivecommunitiesoftheirrights.Accordingtoindigenouscommunities,itisnotforBigTechtoplaywithsuchheritage.2.1.4 OPAQUE SYSTEMS AND LACK OF ACCOUNTABILITY Modelssuchaslargelanguagemodelsaregenerallyverytechnologicallycomplex,butthey

143、arenotimpossibletounderstandorexplain.Therearefundamentalscientif-icprinciplesrelatingtotransparency,peerreviewandrigorousqualitycontrolthatapplyinfieldssuchasthepharmaceuticalandaviationindustries,whichshouldalsoapplytodevelopersofAImodels.Informationabouthowtrainingdataiscollected,howthedataislabe

144、lled,howtestingisperformed,whatdecisionsaremadere-gardingcontentmoderation,andtheenvironmentalandsocialimpactsofthemodelsarejustafewareaswheretransparencyisnecessarytoensurethatrisksaremiti-gatedandthatclaimsaboutthetechnologyareaccurate.48 HypotheticalAIsystemthatdemonstrateshuman-levelintelligence

145、andautonomy.Doesnotcurrentlyexist.2.1StructuralchallengesofgenerativeAI20Norwegian Consumer CouncilJune 2023Ghost in the machine2.1.4.1 Opaque systems reduce accountabilityUnfortunately,therearealreadytendenciesfromcertainAIdeveloperstocloseofftheirsystemsfromexternalscrutiny.Forexample,Googlehascom

146、mittedtoapolicychangewherethecompanywillonlysharepapersaftertheirresearchhasbeenturnedintoproducts.47MicrosoftresearchershavemadegrandclaimsaboutitsownAIsystemsshowsignsofartificialgeneralintelligence,48 whilenotprovidingresearcherswithaccesstothemodelinordertoverifyordisputetheclaims.49Finally,Chat

147、GPTownerOpenAIhasclaimedthatthecompanysAIsystems,includingwhattrainingdataisused,howthemodelworks,etc.,shouldnotbeopenforexternalreviewbecausegivingothersaccesswouldposeacompetitionandsafetyrisk.50Whilethelackoftransparencyisanissuethatappearsacrossthespectrumofthesoftwareindustry,OpenAIsowndescript

148、ionoftherisksoftheirproductstendtoborderonexistential,withitsCEOSamAltmanstatingthatthecompanyis“scared”bythepotentialharmsthatmaystemfromitsownsystems.51Suchclaimsasapre-tenceforclosingdowngenerativeAIsystemsforexternalauditingandreviewareworryingtendenciesthatcanmaskanumberofdownstreameffects,andw

149、hichposeenforcementagenciesandresearchersmajorchallenges.ResearchersatPrincetonUniversityhaveclaimedthatOpenAImightbemisrepresentingthecapabilitiesoftheirsystems,butthisisimpossibletoproveduetothesystembeingclosedtoexternalscrutiny.52Theresearch-erswarnthatthisalsosignificantlyhindersattemptsatrepro

150、ducibilityofanyclaimsmadebythecompany.53 2.1.4.2 Trade agreements as barriers to transparencyWhilecompaniesthemselvesareattemptingtocloseofftheirsystemsfromexternalscrutiny,lawmakersmaybeincreasinglylimitedfromrequiringtransparencybytradeagreements.InternaldocumentsfromtheEUCommis-sionshowthatdigita

151、ltradeagreementsbetweentheEUandUSlimitEuropeanlawmakersabilitytorequirethird-partyaccesstothesourcecodeofAI.54Closed-doornegotiationsaffectinglawmakersabilitytocreateawell-bal-ancedandconsumer-friendlymarketarehighlyproblematic.Thisbarscivilsoci-etyandotherstakeholdersfromprovidingimportantinputanda

152、ppearsatoddswithcrucialdemocraticprinciples.2.1.4.3 Actor chain transparencyThelackoftransparencyalsobecomesproblematicwhenserviceprovidersimplementthird-partygenera-tiveAImodelsintotheirservices.Thismayincreasetheriskoferrorsorunexpectedbehaviourfromthemodel.55 Thedeveloperofthebaselinemodeldoesnot

153、necessarilyseeorunderstandthedownstreamcontextsinwhichthemodelisused,whiletheserviceproviderorotherdownstreamdevelopersdonotsufficientlyunderstandthe limitations of the model.Iftheserviceproviderisnotprivytothedatasetsusedtotrainthemodels,ortohowthemodelactuallyworks,theserviceproviderwillnotbeablet

154、ogivetheconsumeranexplanationaboutwhyacertainoutputwasgener-ated.AssupplychainsforgenerativeAIsystemsmaybecomplex,withoneactorcollectingandlabellingdatasets,whileitcanbeotheractorsdevelopingthealgo-rithms,trainingthemodel,orintegratingitintoservices,itbecomesdifficulttoattributeliabilityandaccountab

155、il-itytotherightentity.Fortheconsumer,thismayhaveanegativeimpactontherighttoanexplanation,aswellascontestabilityandgeneraltransparencyobligations.Forexample,theretailbank,payments,andshoppingserviceKlarnahasannouncedacollaborationwithOpenAI,withplanstointegrateChatGPTintoitsservicestoprovidea“highly

156、personalizedandintuitiveshoppingexperiencebyprovidingcuratedrecommendations”.56 Ifthissystemprovidesflawedrecommendationsforproducts,orranksproductsinaskewedway,itwillbeessentialthatconsumers,nottomentionenforcementagencies,areabletoaccessandassessdataonhowtherecommendationsystemaffectstheconsumer.T

157、hisbecomesimpossibleifOpenAIasathird-partyserviceproviderdoesnotprovideexternalactorsthenecessaryinformationabouttheAIsystem.Withoutsuchinforma-tion,itisentirelyplausiblethattheservicesandprod-uctsshouldnotbeonthemarketatall.2.1StructuralchallengesofgenerativeAI“There are fundamental scientific prin

158、ciples relating to transparency,peer review and rigorousquality control that apply in fields such as the pharmaceutical and aviation industries,which should also apply to developers of AI models.”21Norwegian Consumer CouncilJune 2023Ghost in the machine2.1.4.4 Opaque systems exacerbate consumer harm

159、s and hinder consumer rightsThegenerallackoftransparencyinsomegenerativeAIsystemsmayhavesignificanteffectsonconsumers.AsgenerativeAIsystemsarebeingadoptedbyconsumersforvarioususecases,thepotentialforharmsrises,aselaboratedinothersectionsbelow.Forexample,manytextgeneratorsarepronetoprovidingfalseorin

160、accurateinformation.Thismayhavedirecteffectsonconsumers,forexampleifachatbotprovidesbadfinancialadvice.Theoften-complicatedactorchainsbehindaconsumerfacinggenerativeAIsystemmayalsomakeitexceedinglydifficultforconsumerstogetintouchwiththeresponsibleentityincasesomethinggoeswrong.Thiscouldalsobeproble

161、maticwhenitcomestoclaimsforcompensation.Withoutacertaintransparencyintohowthesystemworks,suchaslimitationsonthesystemsintendeduse,alongsidedisclosuresaboutpossibleinaccuracies,thepotentialforharmbecomeslarger.Theotherharmsthatarecoveredinthefollowingsubsectionsofthisreportareexacerbatedwhenconsumers

162、arekeptinthedarkofthe systems potential for harms and harmful uses.Itisimportantthatcompaniesprovidetransparentsys-temsandapplicationtoconsumers.However,thepowerasymmetrybetweencompaniesandconsumersindigitalenvironments,57meansthatanytransparencymea-suresdirectedatconsumerstoreduceharmsmustbeimpleme

163、ntedinadditiontoothermeasures,ratherthanasastand-alonemeasure.TheresponsibilityofensuringfairandlegitimateuseofgenerativeAImustbeonthecompanies,andnevershiftedontoconsumersthroughtransparencymeasures.2.1.4.5 The limits and restrictions of corporate AI ethicsEthicalandlegalconsiderationsplayafundamen

164、talroletoensurethatmodelsaredeveloped,trained,deployed,andusedinaresponsibleway,fromthedevelopmentstageandthroughoutthelifecycleofthemodel.Asethicalnormsandvaluesdiffersignificantlydepend-ingonculturalcontexts,itisalsoworthnotingthatthedecisionofwhichethicalstandardstoconsiderandapplyisapoliticalcho

165、ice.Similarly,legalframeworksarenotuniversal,whichmayproveaserioushurdleasgenera-tiveAImodelsarerolledoutonaglobalscale.WhilemanycompaniesworkingongenerativeAImodelshaveemployedAIethicsteamstohelpdefineguardrailsandredlinesforAIdevelopment,therearedoubtsabouthoweffectivethishasbeenincaseswhereethica

166、lcon-cernsconflictwiththecompanysprofitmotives.Famously,GooglefiredmembersofitsAIethicsteamafterresearchersfromtheteampublishedthepaperOn the Dangers of Stochastic Parrots:Can Language Models Be Too Big?.Thepaperposedcriticalquestionsabouthowlargesuchmodelsshouldbe,alongsidecriticaleval-uationsofinh

167、erentbiasesandtheenvironmentalimpactofthemodels.Afterrefusingtoretractthepaper,theresearcherswereaskedtoresignfromthecompany.58Amongstthetechcompanylayoffsin2022/2023,AIethicsorresponsibleAIteamsatcompaniesincludingGoogle,Twitter,Microsoft,andMetawerealsolaidoff.Thisraisesconcernsaboutwhetherethical

168、concernsareignoredorheavilydownprioritizedbycompaniescompetinginagenerativeAIgoldrush.59Somecompaniesarecallingforregulationofgenera-tiveAI,notablyOpenAI,60seeminglywishingtoabidebylawmakersrequirements.Atthesametime,OpenAIhasthreatenedtoleavetheEUiftheprovisionsofthenewAIActaretoostrict.61Thisindic

169、atesadesirefromcompa-niestoshaperegulationsinaccordancewiththeirownprofitmotives.2.1StructuralchallengesofgenerativeAI“As ethical norms and values differ significantly depending on cultural contexts,it is also worth noting that the decision of which ethical standards to consider and apply is a polit

170、ical choice.”22Norwegian Consumer CouncilJune 2023Ghost in the machine2.2 ManipulationGenerativeartificialintelligencemodelshavethecapabilitytogeneratesyntheticcontentthatcloselyresemblesrealcontent,includingdialogue,voices,photographs,andvideo.Studieshaveshownthattextgeneratorsthatsimulatehumandial

171、oguecaninfluencepeoplesfeelings,dispositions,andopinions.62 As gener-ativeAImodelbecomemorepowerful,thepotentialformanipulationbecomesgreater.Lowqualitycontentcanalsomisleadormanipulate,bothonpurposeandbecauseoflowqualitydataandmodels.IfagenerativeAImodelproducesinaccurateorfalseinformation,thiscanh

172、aveharmfulconsequencesforconsumers.Ifsuchmodelsaredeployedandusedmaliciously,thismayleadtoconsumersbeingtricked,misled,orotherwisemanipulated.2.2.1 MISTAKES AND INACCURATE OUTPUTGenerativeAImodelsarecomplexsystemstrainedonvastamountsofmaterial,whichmaygiveanimpressionofinfallibility.However,asthemod

173、elsdonot“under-stand”contextandthecontentitproduces,theyhaveatendencytoproducecontentthatlooksconvincingandcorrectbutisfactuallyincorrect.Thisparticularlyapplies to text generators.Forexample,ChatGPTcanproducetextthatlooksveryconvincingandfact-basedbutcontainsfactualerrorsorfallacies.63Thishasledcri

174、ticstocallthesystema“confidentbullshitter”.64Similarly,GoogleemployeeshaslabelledthecompanysowntextgeneratorBarda“pathologicalliar”.65Itcanbedifficultforthepersonpromptingthesystemtonoticeorrevealtheseerrorsiftheyarenotalreadyfamiliarwiththefactsoftherelevantsubject.Somesystems,suchasBing,citessourc

175、esforthegeneratedinformation,apparentlytoalleviatesomeoftheseissues.However,themodelshavebeenproneto“makeup”non-existingsources,eitherthroughpresentingsourcesthatdonotinfactexistorpresentingsourcesthatdocontaintherelevantcontenttosupportthegeneratedcontent.66 Mistakesandinaccuraciesareexacerbatedasg

176、en-erativeAImodelsarepluggedintotheworkstreamindifferentareas.Amidstsinkingrevenue,soonafterthewidespreadadoptionofChatGPT,publisherswerequicktoannouncethattheywouldstartusingthemodelforcontentproduction.67However,whenthenewssiteCnetusedatextgeneratortogeneratejournalisticcontent,itwassoondiscovered

177、thatthepublishedoutputwasriddledwithfactualerrors.68TherearealsoconcernsthattheuseofgenerativeAImodelsasareplacementfortraditionalinternetsearchengineswillmakeitsignifi-cantlyhardertoidentifyinaccurateorincorrectinforma-tion,whilealsohavingnegativeeffectsoninformationliteracy.69 Aslargelanguagemodel

178、sbecomeincreasinglysophis-ticated,theycanadoptmoreauthoritativeandconvinc-ingsyntax.Combinedwithadjustmentsofanswerstoincreasepersuasivenessandengagement,itbecomesmoredifficulttodetectmistakes.Whilefactualerrorsmaybeironedoutthroughtechnologicaladvancements,thismayalsomakeithardertoknowwheninformati

179、onisincorrect.Forexample,ifaLLMprovidedsophisticatedandaccurateanswers99times,itbecomesharderfortheendusertoknowthatitwasinaccurateorcompletelywrongthe100thtime.Inaccuratelygeneratedinformationcouldhaveharmfulconsequences,bothasstandalonemodelsandwhenthegenerativeAIisembeddedinothersystems.For Bing

180、pretending to feel good.(23.03.2023)2.2Manipulation23Norwegian Consumer CouncilJune 2023Ghost in the machine2.2Manipulationexample,ifanAI-poweredchatbotisusedbyaconsumertosolicitmedicaladvice,andtheadviceiswrong,thiscouldleadtoreallifeharm.Similarly,textgeneratorsarereportedlybeingusedbyconsumersfor

181、mentalhealthpurposes,whichmayalsohaveseriousconsequences,alsobecausethemodelsdonotfollowanyethicalorlegalguidelines or rules.70Finally,textgeneratorsthatareusedtofindinformationaboutconsumerrightsmayendupprovidingfalseinformationthatenduprenderingthecon-sumerunawareoforunabletoexercisetheirlegalrigh

182、ts.InMarch2023,thePortuguesegovernmentannouncedthatitwoulduseanadaptedversionofChatGPTtopro-videlegaladvicetocitizens.71 Although the model is only meanttoprovidegeneraladviceincertainareas,andwillnotreplacedecisionmakers,itshouldbeexpectedthatendusersareconditionedtotrusttheoutputofthemod-elregardl

183、essofitsactualfactualaccuracy.Whensuchmodelsareusedbypublicinstitutions,theadditionalveneeroflegitimacymaymakeerrorsevenhardertode-tect.Thisisalsoacontextwheremistakeswilladverselyaffectpeopleinavulnerablesituation,whosereasonforaccessingtheinformationistheirneedforlegaladvice.Suchvulnerabilitymayal

184、soenhanceotherrisks,suchastheriskofbeingmisled.Iforganizationswithinthepressor inthepublicsectorbegindeploy-ing and relying on generative AI models,theproductionoffalse,misleading,orinaccurateinfor-mationcanbecomeasignificanttrustissue.Forexample,ifagovernment-promotedservicegivescitizensbadlegaladv

185、ice,thishasariskoferodingtrustinpublicinstitutions.Similarly,anewspaperusingatextgeneratortoproducearticlescontainingfalseinforma-tion undermines readers faith in theveracityofallinformationthepaperpublishes,perhapseventhepressmorebroadly.2.2.2 THE PERSONIFICATION OF AI MODELSManyconsumersarealready

186、becomingusedtointeractingwithgenerativeAImodels.Suchmodelsare often designed to emulate humanspeechpatterns,behaviours,andemotions.Thiscreatessignificantpotentialformanipulationanddeception,whichmayexploitandunderminecognitivefreedoms.72 LargelanguagemodelssuchasLaMDAorChatGPTaretrained on enormous

187、amounts of text gathered from the internet,whichmeansthattheyhavehugerepositoriesofdatatodrawpredictionsfrom.Thisalsomeansthatthemodelsareabletosimulatehumanpatternsinthetextsthataregeneratedafterall,theymayhavebeentrainedonavastnumberofconversationsbetweenrealpeople.Theexhibitionofhuman-likebehavio

188、ur,emotions and traits are not inherent to generative AI models,theseareattributesthatdeveloperscanchoosetoincludeornot.Forexample,theuseofcasualcon-versationallanguageandemojismaybeawaytoeaseconsumersintointeractingwithachatbot,butcanalsobeexploitedtomakeconsumersfeelguiltyaboutnottakingcertainacti

189、ons,manipulatethemintopayingforaservice,etc.TherearefundamentalissueswithreleasinggenerativeAImodelstothepublicwithoutplacingrestrictionsonitsabilitiestoemulatehumanbehaviour.73 If the model generatescontentthatsimulateshumanemotion,thisisinherently manipulative.My AI simulating human emotion and be

190、haviour.24Norwegian Consumer CouncilJune 2023Ghost in the machine2.2ManipulationAshumans,ourcognitivebiasesmakeusassignhumantraitsandabilitiestoanimalsorobjectsthatexhibitsomesignsofhumanity,suchasfacialexpressions,be-haviouralpatterns,orapparentpersonalitytraits.Thisisarecurringphenomenonforpeoplei

191、nteractingwithgen-erativeAImodels,particularlytextgenerators.Humansascribecommunicativeintentwhenonthereceivingendoforalorwrittennaturallanguage,regardlessofwhetherthecontributorhassuchintent.Thiscanoccurevenwhenoneisfactuallyawarethatthemodeldoesnotactuallyhavehumanattributes.74 Misunderstandingsab

192、outthecapabilitiesofgenerativeAImodelsarealsoinfluencedbydeliberatemarketingstrategiesfromthecompaniesdevelopingthemodels,75 andbyusingvagueormisleadinglanguagetodescribewhatthemodeldoes.76Finally,featuresofhuman-likebehaviour,suchasonusingemojisinconversationorgeneratingtextinthefirst-person,canals

193、oservetoincreaseconsumersattributinghumantraitstothemodels.77In2022,aGoogleengineererroneouslyclaimedpubliclythattheLaMDAchatbothadbecomesentient,i.e.,capa-bleoffeelinghumanemotions.78In2023,betatestersofBingssearchengineimplementationofChatGPTwereshockedtoseethemodelrespondtoquerieswithappar-entlyu

194、nhingedandmentallyunstablerants.79Bothcaseswerefollowedbydiscussionsaboutwhetherthemodelsmayhavebecomesufficientlyadvancedtoresemblehumanintelligence.Suchdiscussions,wherehumanemotionsandmotivesareassignedtoagenerativeAImodel,revealsafunda-mentalmisunderstandingofhowthistechnologyworks.Inreality,gen

195、erativeAImodelsarenotsentient,anddonot have feelings or desires.Generative AI models are predictivealgorithmicsystemsthatcanstatisticallypredicthowpiecesofdatafittogether.Thiscanbeexemplifiedbypredictivetext models that are stan-dardonmostsmartphones,wherethemodelistrainedtopredictorguessthenextword

196、inasequenceofwordsforexample,themodelmaypredictbasedonitstrainingthatthenextwordinthesentence“Ilove”islikelytobe“you”,or“coffee”,or“therain”.Amoresophis-ticatedmodelmaybeabletomoreaccuratelyguessthatbecausethesentenceispartofaconversationaboutItaliancuisine,“pasta”isthemostlikelynextwordinthesentenc

197、e.AsdescribedbyBenderetal.,“alanguagemodel is a system for hap-hazardlystitchingtogetherScreenshots from Replika.“There are fundamental issues with releasing generative AI models to the public without placing restrictions on its abilities to emulate human behaviour.If the model generates content tha

198、t simulates human emotion,this is inherently manipulative.“25Norwegian Consumer CouncilJune 2023Ghost in the machine sequencesoflinguisticformsithasobservedinitsvasttrainingdata,accordingtoprobabilisticinformationabouthowtheycombine,butwithoutanyreferencetomeaning:astochasticparrot”.80Arecentstudyha

199、sshownthathumansareunabletodistinguishbetweenhuman-generatedandAI-gener-ated text.81Accordingtothestudy,humanbeingsarenotequippedtoaccuratelyrecognizeAI-generatedlanguage,andtestsubjectswerepronetolabellingAI-generatedtextashumanatahigherratethanactualhuman-generatedtext.Thiscanbeexploitedtomanip-ul

200、ateordeceivepeoplebyemployingtextgeneratorsmasqueradingashumanbeings.Aspeoplemaytendtotrusthumansmorethanachatbot,advancedlanguagemodelsmaybesuitedtodeceiveconsumersintogivinguppersonalinformation,spendmoney,orperformcer-tainactions.82 Manipulation may happen due to the end user not knowingthattheya

201、reinfactinteractingwithamachine,butevenifthisismadeclearandobvious,anthropomor-phizedgenerativeAImodelscanstillbeeffectivetoolsformanipulation.Thiscanalsooccurincaseswherethe“human-like”behaviourisamainfeatureofthemodel,suchasAI-basedassistantsoremergingAIromanticcompanions.Forexample,theapplication

202、ReplikausesgenerativeAItosimulateapartner,oftenwithemphasisonroman-ticoreroticconversation.TheAImodel“remembers”conversations,simulatesfeelingsbyprofessinglovefortheconsumer,andappearstobesadifthepersonrarelyusestheservice.Therearemanymicrotransactionsintheapp,whichcanbepurchasedtounlockfeaturessuch

203、asnewpersonalities,“Selfies”(receiveexclusiveselfiesfromReplika),andevenavirtualmarriage.AllthesefeaturesadduptoahighlymanipulativeexperiencewheretheconsumersaresubjectedtoAI-generatedcommercialand emotional pressure.InFebruary2023,theItalianDataProtectionAuthorityfoundthatReplikawascollectingperson

204、aldatafromchildrenwithoutalegalbasis,andthatthecompanywasinbreachoftheGeneralDataProtectionRegula-tion(GDPR).Asaresponse,Replikaaddedsignificantrestrictionsinfeaturesoftheapp,wherefeatureswerediminishedorentirelyremoved.Reportedly,thecompan-ionwouldnolongerrememberpastconversationsandwouldrefusetota

205、lkaboutvarioussubjects.Asaresult,peoplewhoweresimulatingromanticpartnershipwiththeAIcompanionwereleftheartbrokenandbereft.83 Inthiscase,eventhoughReplikaneverpretendedthattheappwasanythingmorethananAIsystem,consum-ersneverthelessformedgenuinebondswithit,leadingtosignificantnegativepsychologicalimpac

206、toncethedeveloperchangedhowthesystemworked.ThesocialmediaplatformSnapchathasalsointroducedanAIcompanioncalledMyAI.84Itwasinitiallyintroducedasapremiumservice,butwithinfewmonthsthechatbotwasrolledouttoallusers,alongwithamessagealertingconsumersaboutthisnewfeature.Soonafterlaunch,MyAIwassubjecttosigni

207、ficantcriticismforitslackofguardrails.Forexample,themodelcheerilyofferedadvicetoaresearcherposingasa13-year-oldgirlthataskedabouthavingsexwitha31-year-oldpartner,whileajournalistposingasaminorreceivedadviceabouthowtomaskthesmellofalcoholandmarijuana.85 Inadditiontothesafetyissuesthatsuchcasesmaypose

208、,itisgenerallymorallyandlegallydubioustorolloutexperimentalAI-drivenfeaturesinanappusedbymanyminors.Therearealsopotentiallysignificantrisksposedbygivingpeople,particularlychildren,artificial“friendsasaservice”thattheymustpayasubscriptiontokeeptalkingto,orthatcouldbeplacedbehindapaywallatalatertime.T

209、herisksofchildreninteractingwithamachinethattheybelievetobehumanmayincludedevelopingunhealthyemotionaldependencies,manipu-lation,andtheextractionofdata.86Thiscanbeexploitedbycompaniesforprofit,forexamplethroughadvertisingorotherwisesponsoredcontent.2.2.3 DEEPFAKES AND DISINFORMATIONAsgenerativeAImod

210、elskeepgettingincreasinglypow-erful,itbecomeseasiertousethemtocreaterealisticsyntheticimages,text,orvoicerecordingsthatcanbemistakenforrealcontent.Itcanhelplowerthethresholdforproducingdeliberatelymisleadingcontent(disinfor-mation),orforcreatingfakeimagesorvoiceclipsofrealpeopleincompromisingsituati

211、ons,orforimitatingrealpeople(deepfakes).87 A2022Europolreportestimatesthatby2026,about90%ofonlinecontentmaybeAIgenerated.88 Asthevolumeofsyntheticcontentgrows,itbecomesdifficulttotrustonesowneyesandears.Thelong-termeffectsofthiscanbedevastatingfortrustininstitutions2.2Manipulation26Norwegian Consume

212、r CouncilJune 2023Ghost in the machineandeachother.Theproliferationofdeep-fakedcontentcanleadtosignificanterosionoftrust,aspeoplewillnotbeabletoknowwhetheranimage,text,sound,orvideoisrealorsynthetic.Evenifthecontentwasnotoriginallygeneratedtospreaddisinformation,thenatureoftheinternetmeansthatcontex

213、tanddisclaimersarequicklystrippedawayascontentissharedacrossplatforms.89 Assyntheticcontentproliferates,thismayalsoprovideplausibledeniabilityinthecaseofauthenticcontent.Forexample,ifawhistle-blowerleaksinformationexposingcorruption,theaccusedindividualorinsti-tutionmayplausiblyclaimthattheleakedmat

214、erialisfake.Asub-categoryofdeepfakesthacanhaveaparticularlydevastatingeffectonvictims,isdeepfakepornography.AccordingtoastudybythecompanySensity,96%ofdeepfakeimagesaresexuallyexplicitpicturesofwomenwhodidnotconsenttotheimagegeneration.90Asdescribedabove,opensourcemodelssuchasStableDiffusionmakesitpo

215、ssibleforanyonetotrainmodels,whichmeansthatpeoplecanbedeepfakedeveniftheyarenotapublicpersonwithmanyavailableimages already part of the training data.WhilethegenerativeAImodelscanbeusedtointention-allyproduceandspreaddisinformation,theemploymentofinaccurategenerativeAImodelsinconsumerfacingproductsc

216、analsoaccidentallyleadtothespreadoffalsehoods.Asdiscussedmorein-depthinchapter2.2.1,prominenttextgeneratorsarepronetoproducingveryconvincing,falseinformation,aswellasreferenc-ingsourcesthatdonotbackupitsclaims.91 AsmoreadvancedgenerativeAImodelsbecomein-creasinglyefficientatgeneratingtextthatseemscr

217、ed-ible,thismayleadtodisinformationbecomingmoredifficulttodetect.AstudyfromMarch2023foundthatChatGPT4ismorelikelythanitspredecessortogen-eratemisinformationwhenprompted,includingfalsenarrativesconcerningvaccines,conspiracytheories,and propaganda.92Thiscouldmakethetechnologyanefficienttoolforrapidlyp

218、roducingconvincingtextthatcanbedisseminatedwithpotentiallyharmfuleffects.Discussionsonhowdeepfakesanddisinformationwillplayoutinelectionsandthedemocraticprocessisincreasinginintensityaswell.93 President Donald Trump crying in front of the White House,Midjourney.A lifelike photograph of a woman,DALL-

219、E.Note the watermark in the bottom right corner.2.2Manipulation27Norwegian Consumer CouncilJune 2023Ghost in the machine2.2.4 DETECTING AI-GENERATED CONTENTOne suggested solution to the deluge ofsyntheticcontent,istowater-markorotherwiseclearlylabelthatapieceofcontentwasgeneratedusing a generative A

220、I model.This canbedoneeitherbyaddingavisuallabelthatindicatesthatanimageorvideoisAI-generated,throughimperceptiblewater-markssuchassinglepixels,orbyaddinginformationtothemetadatathatcanbeusedtoshowtheoriginsofthecontent.94Forexample,GoogleisimplementingafeaturetoautomaticallylabelAI-generatedcontent

221、inthemetadataofpictures,andaddcontexttowheretheimage originated.95Whilewatermarkingcanbeusefultoquicklyidenti-fythatanimageorvideoisnotauthentic,therearesignificantlimitationstothisapproach.Awatermarkingsystemonlyworksaslongasthesystemdeveloperand/orthepersonusingthemodelchoosestoabidebythewatermark

222、ingstandards.Closedsourceimagegenera-torssuchasDALL-EandMidjourneymaychoosetoaddmandatorywatermarkstothemetadataofallgeneratedpictures,butthiscanbecircumventedforexamplebytakingascreenshotandsharingthescreenshotinsteadoftheoriginalimage.Visualwatermarks,suchastheonesusedbyDALL-E,canbecroppedoutofthe

223、picture,unlessthewatermarkissoobtrusivethatitsignifi-cantlydetractsfromtheimagequality.Imperceptiblewatermarkssuchasindividualpixelscanberemovedbyslightlychangingthecolourgradingoftheimage.ForopensourcemodelssuchasStableDiffusion,attempts ataddingwatermarkstogeneratedimagescanberemovedfromthemodelby

224、anyonewhowantstodeliberatelypassoffsyntheticcontentasreal.Thiscanbeaddressedifasignifi-cantpartofthetrainingdataforthemodeliswatermarked,butevenifthiswasthecase,thewatermarkingcouldbecircumventedasdescribedabove.Inadditiontoissuesofmisinformation,inaccuraciesandauthenticityrelatingtogeneratingimages

225、,therearesignificantquestionsabouthowtodetectplagiarism,forexamplewhenstudentsusetextgeneratorsinacademicsettings.ChatGPThasbeenwidelyusedtogeneratees-saysandanswerotherschoolassignments,raisingalarmsaboutcheatingandnegativeeffectsonlearning.96 Watermarkingoftextismorecomplexthanforimagesandvideos,a

226、sanytextcopiedfromatextgeneratordoesnothaveanymetadatatoappendwatermarksto.ThereareongoingeffortstocreatetextualsignaturestotextgeneratedbyChatGPT,butthismaybecircumvent-edbymakingchangestothetextorbyfeedingthetextthrough another text generator.97 Systemsthataresupposedtodetectandflagwhetheratextwas

227、writtenbyatextgeneratororahumanhavebeennotoriouslyinaccurate,98andarenotascalablesolutionasalltextneedstobefedintothedetectorsys-tem.Forexample,OpenAIhavereleasedagenerativeAImodelwiththepurposeofdetectingwhetheratexthasbeenwrittenbyChatGPT,butthismodelonlyhada26%accuracyrate.99Detectingwhethersomet

228、hingwasgener-atedbyagenerativeAImodelrequiresmoretechnicallycomplexsolutionsthangeneratingnewcontent,whichmeansthatthedetectionsystemswillseeminglyalwaysbelaggingbehindinthisarmsrace.ThisleadstoquestionsaboutwhatrecourseapersonhasinacasewhereanAImodelfalselyaccusesthemofplagiarism.Identifyingfalsefl

229、agsmayalsobecomplextask,whichmeansthatateacherusingasystemtode-tectplagiarismmaynotbeabletoaccuratelydoso.Iftheenduseroftheflaggingsystem(forexampleateacher)cannotsuccessfullydeterminethatatextorimagewasfalselyflaggedasplagiarism,thisputsthestudentinadifficultposition,atitisvirtuallyimpossibletoprov

230、ethatChatGPTdidinfactnotwriteyouressay.Similarly,ifpeopleareflaggedascheaters,untrust-worthy,orsimplyasnon-human,theymayexperiencesignificantnegativeeffectswithfewmeanstorecourse.Forexample,ifaplatformhasaflaggingsysteminplacetoidentifyandremovecontentthatappearstobeAI-generated,thesesystemsmayerron

231、eouslyflagcon-tent,leadingtoconsequencestotheconsumerthathascontentremoved.Summedup,watermarkinganddetectiontoolsaretechnologicalsolutionsthatmightworkincertainlimitedsettings,suchasdemonstratingthataphotographorig-inated from a real photographer or from an image genera-2.2Manipulation“The prolifera

232、tion of deepfaked content can lead to significant erosion of trust,as people will not be able to know whether an image,text,sound,or video is real or synthetic.”28Norwegian Consumer CouncilJune 2023Ghost in the machinetor,forexampleinadvertisingorwhenusedbymediaorpublicinstitutions.Itcanbeausefultoo

233、ltoquicklydeterminethatapictureisactuallyfromGettyImagesorfromDALL-E,whichmayalleviatesomeharmsrelatedtoaccidentalspreadofmisinformation.However,abeliefthatwatermarkingwillsolvethein-formationcrisisisatitscoreatechnologicalsolutionistapproach.Evenifitwastechnicallypossibletoaccu-ratelywatermarkallAI

234、-generatedcontent,thedelugeofsyntheticcontentanddisinformationisnotsolvablebyaddinganothertechnicallayer,particularlyifcontentisdeliberatelymeanttomislead.Thelackoftrustinthemediaandpublicinstitutionsisnotsolelyamatterofnotbeingabletotellsyntheticfromauthenticcontent.Furthermore,itisunreasonabletoex

235、pectpeopletoscaneverypieceofmediatheyseeonlinetodetectwhetheritissynthetic.Sincethereisnotechnologicalquickfix,itiscrucialtolookatothersolutions,suchasrobustmedialiteracyandtrustedmediainstitutions.Thisshouldalsobegivenconsiderableattentionbymedia-andsocialsciencesresearchers,whocanprovidepolicymake

236、rsandotherswithsustainable,long-termsolutions.2.2.5 GENERATIVE ARTIFICIAL INTELLIGENCE IN ADVERTISINGThepromiseofgenerativeAImodelshasalsoreachedthe advertising industry.100Thetechnologyisalreadybe-ingusedtogenerateadcopy,101creatingsyntheticstockphotosandmodels,102andaspartofmarketingstunts.103 The

237、seusecasesmayreducelabourintheadvertisingsectorbutcanalsohaveadverseeffectsonconsumers,particularlybymakingiteasierandmoreefficienttomanipulatepeoplethroughcreatingpersonalizedand/orconversationaladvertising.TheintroductionofpubliclyavailablegenerativeAIhasbeenlargelyad-free,butthisispoisedtochange.

238、InMarch2023,MicrosoftannouncedthatitwouldberollingoutpaidadsintheBingchatbot.104InMay,GoogleannouncedthatitwouldintegrateadvertisingintheirgenerativeAIproducts.105IfconsumersdependontextgeneratorssuchasBingtoprovideaccurateandfactualinformation,theplacementofadvertisingintheanswersitprovidesmaybemis

239、leading.Thepotentialforbehaviouralmanipulationwheninteractingwithalargelanguagemodelmayenablemoreeffectiveadvertisingatthecostofconsumeragency.106ImplementinggenerativeAImodelsmayalsoexacerbateseveralproblematicissuesrelatedtosurveillance-basedadvertising,suchasdiscrimination,fraud,andprivacyviolati

240、ons,bymakingiteasiertogeneratetailoredadstoparticulargroupsorcategoriesofpeople,whichmayinturnmakeiteasiertoconvincesomeonetopurchaseaproductorbelieveastatement.107Thisfeaturewouldaccelerateandfacilitatecompaniesabilitytotargetthecontentofadsautomatically,ratherthanjusttheadsthemselves.CombinedwithA

241、/B-testing,thiscouldin-creasethemanipulativecharacterofsurveillance-basedadvertising.2.2.5.1 Using chatbots to collect personal dataTherearerisingconcernsaboutgenerativeAIinchat-botsandtheirabilitytotrickconsumersintosharingper-sonaldata,whichmayberepurposedtoservetargetedadvertisingortomanipulateco

242、nsumersintopurchasingproductsorservices.Whilethischallengeechoesabroaderdebateabouttherepurposingofpersonaldataforbusinessgains,108themanipulativeaspectsofgener-ativeAImodelspretendingtobehumans,asmentionedabove,couldexacerbatetheproblems.Thisisespeciallyrelevantinthecaseofvulnerablegroupssuchaschil

243、-drenorlonelypeople,whomaybemorelikelytosharesensitiveinformationaboutthemselvesinconversationwiththegenerativeAI.Forinstance,chatapplicationslikeReplikaandSnap-chatsMyAI,bothofwhichwerediscussedinchapter2.2.2onpersonificationofAImodels,explicitlyinviteenduserstoshareinformationaboutthemselves.Simil

244、arly,generativeAImodelsusedtoembedsearchinvariousapplicationsmaybeusedtocollectandstorequerydata,suchaswhetherthepersonqueryingthemodeliscur-rentlyinterestedinlocalrestaurantsorshoes,depend-ingonthequery.Personaldataisthebasisformassivebusinessmodels,andthesetextgeneratorscanimprovebusinessesability

245、toobtainhighlyrelevantconsumerinformation.2.2Manipulation“A belief that watermarking will solve the information crisis is at its core a technological solutionist approach.“29Norwegian Consumer CouncilJune 2023Ghost in the machine2.3 Bias,discrimination,and content moderation2.3 Bias,discrimination,a

246、nd content moderationAswithotherformsofartificialintelligence,generativeAImodelsmaycontain,perpetuate,orcreatenewbiases.Models that are trained on vast amount of information takenfromtheinternetwillinheritthebiasesofitstrainingdata.Assuch,themodelsmaygeneratecontentthatreproducenegativeorunwantedten

247、dencies.Thishasledtomanyservice-providersaddingcontentfilterstomoderatewhatispossibletogenerate,andtoflagproblematiccontentinthetrainingdataofthemodels.2.3.1 BIAS IN TRAINING DATAAsmentionedabove,generativeAImodelscangener-atesyntheticcontentthatresembleshuman-createdcontentbecausetheyhavebeentraine

248、donlargedatasetsofexistingcontent.This means that the contentinthedatasetsisofcrucialimportance.There are several steps tocreatingandcuratinga training data set for generativeAImodels,rangingfromscrapingdataonline,selectionandlabelling,tocon-tentmoderation.Withoutcarefulvetting,labelling,andcleaning

249、ofthetrainingdata,datasetsscrapedfromtheinternetcanleadtoseriousdownstreameffects.Forexample,theimagegeneratorStableDiffusionistrainedonanopensourcedatasetfromtheGermannon-profitorganizationLAION.109 The LAION data sets donotcontainanyactualimagesbutisratherasetofURLsthatpointtoimagesfromacrossthewe

250、b.LAIONhasreceivedcriticismforalackofaccountabilityandforinsufficientcurationofcontent(suchasexcludingharm-fulorpotentiallyillegalmaterial),forexamplewhenitwasfoundtoincludeURLspointingatconfidentialmedicalinformation in its data sets.110 AsgenerativeAImodelsaretrainedonhistoricaldata,discriminatory

251、factorsinthedatasetscanbereinforcedbybeingreproducedinthetext,imagesorsoundthatisgenerated.Furthermore,suchmodelscanonlybetrainedonrecordeddata,whichmeansthatphenom-enaoreventsthatarenot(orcannotbe)recordedandquantifiedasdatacannotberecognizedbythemodel.Assuch,generativeAImodelsarepredisposedtoencod

252、edbiasesthatamplifyorentrenchexistinginjusticesandpowerstructures.111 Generative AI models are primarily trained on images andtextscrapedfromtheinternet,whichmeansthereisselectionbiasalreadyatthetrainingstage.Populationsegmentsandgroupsthatlackinternetaccess,forex-ampleindigenousgroups,willlikelybeu

253、nderrepresentedinthetrainingdata,whichcanhavedownstreamdis-criminatoryeffects.Furthermore,ifonlinecommunitieswherecertainpopulationgroupsareoverrepresentedareprominentinthetrainingdata,thismaycontributetoafeedbackloopthatcontinuouslylessenstheimpactofdatafromhistoricallyunderrepresented populations.

254、112Trainingdatacollectedfrom the internet tend toincludepornograph-ic,racist,andste-reotypicalcontent.Ifthe data sets are not curatedandcleaned,thesefactorsmaybecomeembeddedinthemodel.Forexample,imagegeneratorstendtosexualizewomen,particularlywomenofcolour,atamuchhigherratethanmen.113Similarly,promp

255、tssuchas“Africanworkers”tendtogeneratepicturesofmanuallabourer,while“Europeanworkers”resultsinpicturesofwhite-collarjobs.114 A Washington Post investigation found that Googles C4 dataset,whichisusedastrainingdataforbothGoogleandMetaslargelanguagemodels,includedmassiveamountsoftextscrapedfromtheopenw

256、eb,includingWikipedia,Redditandalargeamountofotherdiscussionforums,newspublishers,governmentwebsites,andmuchmore.115 This means that any generative AI model trainedonthissetwill“learn”fromcontentthatmaycon-taineverythingfromhatespeechtoadvertising,whichmayhaveanimpactonthetextitisabletogenerate.If,f

257、orexample,dataisscrapedfromaninternetforumthatcontainsalotofracistorotherwisetoxiccontent,anymodelstrainedonthedatasetruntheriskofrecreatingsimilar material.“The selection and labelling of training data is not neutral.Certain groups of people may be overrepresented in the data,while how the company

258、chooses to label images may reflect biases.”30Norwegian Consumer CouncilJune 2023Ghost in the machine2.3 Bias,discrimination,and content moderationTheselectionandlabellingoftrainingdataisnotneutral.Certaingroupsofpeoplemaybeoverrepresentedinthedata,whilehowthecompanychoosestolabelimagesmayreflectbia

259、ses.Forexample,adevelopermightchoosehowmanycategoriesofethnicitiesand/orgen-derstoincludeintrainingdatalabelsorcouldchoosenottoincludetheseattributesaslabelsatall.IfmodelsaretrainedonotherAIgeneratedcontent,thisrunstheriskoffurtherreinforcingbiases.Asaresult,theremaybefeedbackloopswhereeachtrainings

260、essionstrengthensabiasedordiscriminatorysequenceofdata.ManyAImodelshaveissueswithrecognizingandlabel-lingimagesofnon-whitepeople,likelypartiallybecauseoftrainingthemodelsondatasetswherewhitepeopleare overrepresented.Both Google116 and Meta117 have beenunderfireaftertheirimage-recognitionalgorithmsla

261、belleddarker-skinnedpeopleasgorillasorprimates.LanguageprocessingmodelssuchasBERThasalsobeenshowntoconnectdisabledpersonswithwordsofamore negative sentiment.1182.3.1.1 Discriminatory outcomesBiasedordiscriminatoryoutcomesfromtheuseofgen-erativeAImodelsisnotonlyaproblemrelatedtotrainingdata.Thereareh

262、umanandsystemicbiasesthatcanbeembeddedorstrengthenedfromhowcompaniesandpeoplechoosetouseornotusethemodels.119 For exam-ple,iftheuseoftextgeneratorsbecomesarequirementforvariousjobs,thismayindirectlyexcludelesstechni-callyproficientgroupsofpeople.When AI models are implemented in an attempt to solve

263、complexissues,thereisarealriskthatmoreeffectivesolutions,whichmaybemorecostlyand/orcomplicat-ed,aredownprioritized,asdiscussedmorein-depthinchapter2.1.2.Forexample,theWorldHealthOrgani-zationhaswarnedthattheuseofAImodelsinhealth-caremayhavenegativeeffectsonolderpeople,unlesscertainissuesareaddressed

264、.120ConcernsincludethatAImodelsmaybetrainedondatathatcontainageiststereotypes,andthatolderpeopleareoftenunderrepre-sented in training data.This may help perpetuate ageism andunderminethequalityofhealthandsocialcareforolder populations.2.3.2 CONTENT MODERATIONWhentrainedonlargeenoughdatasets,thereare

265、fewlimitsonwhatmaterialagenerativeAImodelmayproduce.Asnotedabove,manysuchmodelscanbeusedtogenerateillegal,discriminatory,andotherwiseunacceptablesyntheticcontent,sincetheyaretrainedondatasetsthatmayincludeavarietyofdubiouscontent.Inanattempttoalleviatetheseissues,manygenerativeAImodelshavecontentmod

266、erationinplacetofilteroutandflagcertaincontent,orotherwiseintroducelimitstohowthetechnologycanbeused.Whilecontentfilterscanbeusedtolimitthegenerationofcertaintypesofcontent,thisisanapproachwithnumerousshortcomings.Firstofall,contentmoderationgivesthesystemownersignificantpowertodecidewhatmaterialish

267、armfulandwhatispermitted,unlessthisisclearlydefinedbylaw-makers.Forexample,OpenAIhascomeunderfirewithclaimsthatChatGPTrestrictscertainpointsofview,byrefusingtogeneratetextaboutcertainpoliticizedtopics.Thiscanleadtoabusesofpowerthatmayhavesignifi-cantdownstreamconsequences,asprivatecompaniesincreaset

268、heirabilitytodecidewhatisdeemedaccept-ablecontent.Aswithcontentmoderationpracticesonsocialme-diaplatforms,contentfiltersongenerativeAImodelsrunstheriskofover-moderation,whereinnocuousorimportantcontentisfilteredoutorbanned.Thiscanhappenbothbyaccidentorbydesign.Forexample,theimagegeneratorMidjourneyb

269、eganfilteringoutscientificanatomicaltermstoclampdownonendusersgener-atingpornographiccontent.121ThecompanyalsoaddedcontentfilterstopreventconsumersfromgeneratingimagesofXiJinpinginordertoavoidbeingblockedinChina,andendedupdiscontinuingitsfreetrialversionafteraMidjourney-generatedpicturesofDonaldTrum

270、pbeingarrestedwentviral.Thecompanydoesnotdis-closepubliclywhatwordsorpromptsarebannedfromtheplatform,to“minimizedrama”.122Therearealsotechnicallimitationsonwhatcontentfil-terscando.WherevercontentfiltersareusedtorestrictgenerativeAImodels,thereareattemptstobypassorjailbreakthemodels.Peoplehavediscov

271、ereddifferentpromptsthatmaybeusedtogeneratebannedcontent,forexamplebyinstructingthemodeltosimulatecharac-tersthatareallowedtobypassthecontentfilter.123 This armsraceislikelytoleadtomoreover-moderation,ascompaniesrushtocloseanyperceivedloopholes.Content moderation of generative AI models may also cre

272、ateorenforcediscriminatorypractices.Asanexam-ple,wordsreferringtoLGBTQI+communitiesorother31Norwegian Consumer CouncilJune 2023Ghost in the machine2.4 Privacy and data protectionminoritygroupsmaybeflaggedtoremovehatespeechordiscriminatorycontentformthetrainingdata.How-ever,suchattemptsatremovingunac

273、ceptablecontentcouldalsoleadtotheremovalofcontentwhichisinfactshowcasingpositivesidesandsentimentsrelatingtoLGBTQI+communities.Contentmoderationcouldinthiswayreinforceunderrepresentation.Choosingtoaddressthebiasedoutputratherthanthebiasinherentinthedatasetsorthemodel,isinherent-lyproblematic.Moderat

274、ionattemptswillrequirethesuppressionofeachtypeofbiasedoutput,effectivelyapproachingbiasingenerativeAIasagameofwhack-a-mole.124Increasingtheattentiongiventothecurationofdatasetstoreducetheirinherentharmfulbiasisneces-sary,insteadofrelyingonpost-hoccontentmoderation.2.3.2.1 Cultural contextContentmode

275、rationisnotaneutralpractice,andun-derstandingthecontextofcontentiscrucial.Differentculturalcontextthereforepresentsasignificantbarrierforcontentmoderationatscale.Forexample,thereisariskofover-orunder-moderationbecauseofinsuffi-cienttrainingdataormoderatorsforcertainlanguagesordialects.Awidelyusedlan

276、guagesuchasEnglishwillhavealargercorpusoftextinitstrainingdatatoprovidemoreaccurateinformation,andconsequentlypotentiallybettermoderation.Otherlanguagesandculturesareoftenunder-represent-edinthetrainingdata,meaningthatmoderationislikelytobelessaccurateornon-existent.Minoritygroupalsotendtobeseverely

277、underrepresentedamongthepeopledeveloping and training the models.125Furthermore,therearesignificantissuesrelatedtoculturalcontextandnationallegislation,aswhatissociallyacceptableorlegalinoneplaceandcontextmaybetabooorillegalsomewhereelse.Thecontextualcomplicationsaroundcontentmodera-tionalsomakesita

278、taskthatmaybeillsuitedforauto-mation.Theworkofmoderatingoutputandannotatingtrainingdataareautomatedinsomecases,butalsoofteninvolvesmanualwork.Inmanycases,processessuchasdatacleaning,contentclassificationandcontentmoderationinvolvesmentallytaxinghumanlabour.Thisiselaborateduponbelowinthesectiononlabo

279、urexploitation.2.3.2.2 Open source models and the limits of content filtersInpractice,contentmoderationonlyworksoncentral-izedclosedsourcemodels.Inopensourcemodels,suchasStableDiffusion,itispracticallyimpossibletocontrolwhatcontentthemodelcanproduce.Downstreamdevelopers,includingindividuals,cantrain

280、andsharemodelsthatcanmakeanykindofimages,regardlessoflegality.Asthemodelsrunlocallywithoutrequiringaninternetconnectionoraccesstoacloudserver,thecompanywhoreleasedthemodelcannotinterceptorlimitwhatitisabletogenerate.2.4 Privacy and data protectionTherighttoprivacyisoneofthecorevaluesofdemo-craticsoc

281、ieties.Privacyencompassesmanydifferentaspects,suchasprivacyofcorrespondencewithothers,privacyofidentityandthoughts,andprivacyofdataandinformationaboutoneself.Dataprotectionisasub-stantialandimportantpartofprivacy,especiallyinthecontextofonlineservices,butprivacycoversamuchbroaderrangeofindividualpro

282、tections.Personaldatahaslongbeencovetedashighlyvaluableforbusinessesandmaybeusedtotargetadvertisingtoindividualsandgroups,tomeasureengagementortoimprovecompaniesservices,amongotherpurpos-es.When generative AI models are trained on material scrapedfromtheinternet,thetrainingdatausuallycontainsalargea

283、mountofpersonaldata.AsgenerativeAIisdevelopedanddeployed,theseissuesrelatedtodataprotectionandpersonaldatacanleadtosubstantialprivacyharms.2.4.1 PRIVACY CHALLENGES RELATED TO DATA SETS USED FOR MODEL TRAININGImage generators are usually trained on huge datasets thatincludeimagesofrealpeople.Theseima

284、gescan,forexample,betakenfromsocialmediaandsearchengines,withoutalawfullegalbasisorknowledgebythepeopleinthepictures.Similarly,textgeneratorsaretraineddata-setsthatcouldincludepersonaldataaboutindividuals,orconversationsbetweenindividuals.32Norwegian Consumer CouncilJune 2023Ghost in the machineIf a

285、 generative AI model is trained on personal data that wastakenoutofcontext,thismayviolatethecontextualintegrityofindividualconsumers.Whenapersonup-loadsaphotoofthemselvesonline,forexampleonsocialmedia,theycouldnotforeseethatthiswouldbeusedtotrainanAImodel.Theindividualwasneverinformedthiswouldhappen

286、,neverconsentedtosuchuseoftheirlikenessandwilllikelynotbeawarethattheirprivacyandpersonaldatarightswereviolated.AsthepublicawarenessgrowsabouthowgenerativeAImodelsaretrained,theuseofpersonaldatafortrainingmaycreatechillingeffects.UnlessauthoritiesenforcecurrentlegislationsuchastheGDPRagainstcompa-ni

287、esdeployinggenerativeAImodels,andguardrailsandrestrictionsforusingimagesofpeopleareinplace,theonlyrealchoiceforconsumerswhodonotwanttheirimagesusedfortrainingdataistostoppostingpicturesonline.Thisisclearlyaninsufficientsolution.2.4.2 PRIVACY CHALLENGES RELATED TO GENERATED CONTENTItisparticularlypro

288、blematicifagenerativeAImodelcangeneratenewimagesofanindividual,suchasdeep-fakes.Thisinvolvescreating“new”personaldataabouttheindividual,inawaythepersoncanhavenocontrolover.Thisviolatestheintegrityoftheindividualwhoisdepictedinthegeneratedcontent,potentiallyinveryinvasiveorharmfulways.Sometimesapictu

289、recanbeaccuratelyreproducedbythegenerativeAImodel.Thishappensifthemodelwasovertrainedoncertaindata.Forexample,theMonaLisaislikelytobeoverrepresentedinatrainingdatasetcontainingart,becauseitissuchafamousworkofart.Ifthishappens,themodelmayovertrainonthefaceoftheMonaLisa,andthereforemaybelikelytoreprod

290、ucethepaintingquiteaccurately.Overtrainingonpicturesofcertainpeoplewillhavethesameeffect,meaningthatitismorelikelytoreproduceaphotoofahigh-profilecelebritythanarandominternetuser.WithopensourcemodelssuchasStableDiffusion,however,anydown-streamdeveloper,includingindividuals,cantrainmodelsonthefacesof

291、anyone,whichcanbeusedtocreatedeepfakes.Inadditiontothegenerationofpicturesandtheadverseeffectsthismayhaveonconsumers,itisalsopossibletogeneratetextaboutindividuals.Thisincludestextgeneratorsgeneratingfalseand/orlibellousclaimsaboutpeople.Forexample,ChatGPThasgeneratedtextwithpotentiallyhazardousresu

292、ltsforthepeoplewhomitcon-cerns,suchasfalseclaimsaboutaprofessorsinvolve-mentinasexualharassmentscandalorfalseclaimsthata mayor had served prison time.126 2.5 SecurityvulnerabilitiesandfraudGenerativeAImodelscanbeabusedbymaliciousactorstoaugmentorsuperchargecriminalactivities.Aswithotherareas,generat

293、iveAIcanbeusedmakefraud,scams,andotheractivitiesmoreefficient.Themodelscanalsoposechallengestoexistingsecuritysystems.WhilethetypesofcybercrimethatcanbeundertakenusinggenerativeAIarenotnew,theubiquitousnessandeaseofuseofthetechnologymayleadtoanupscalingofsuchattacks.Largelanguagemodelscanbeusedbysca

294、mmerstogen-eratealargeamountofconvincing-lookingtexttodeceivevictims.Similarly,catfishingscams,wherethescammerbuildstrustwiththevictimovertimethroughregularcon-tact,canpotentiallybeautomatizedconvincinglybytheuseofadvancedchatbots.Thismeansthatthecriminalcanef-fectivelyscammorevictimsusinglesstimean

295、dresources.Deepfakingcanalsobeusedtobypasssecuritymea-sures.Whenpicturesandvoicescanbeconvincinglyfaked,thismakesitpossibletoengageinfraudinnewways.Forexample,areporterwasabletofakeclipsofhisownvoicetobypassthevoicerecognitionbiometricidentificationonhisbankaccount.127Similarly,audiogeneratorshavere

296、portedlybeenusedtoimpersonatefamilymembersforcriminalpurposes.128 Largelanguagemodelsarevulnerabletoexploitstoby-passfiltersandsecuritymeasures(jailbreaking),delib-eratelymanipulatingthetrainingdata(datapoisoning),andhiddencommandsthatspurthemodelsintotakingcertainactions,forexamplethroughhiddentext

297、inan2.5Securityvulnerabilitiesandfraud33Norwegian Consumer CouncilJune 2023Ghost in the machine2.6Replacinghumansinconsumer-facingapplicationswithgenerativeAI,whollyorinparte-mail(promptinjection).129Thesesecurityvulnerabili-tiesmayprovetobegrievous,ascompaniestrytostayaheadofthecurvebyintegratingge

298、nerativeAIrapidlyintovariousservices,potentiallywithoutsufficientsecuritytesting.Cybersecurityexpertshavewarnedthattextgeneratorscanalsobeweaponizedbyusingthetechnologytowritemaliciouscodesuchasmalware.130Thismeansthatcy-bercriminalscanpotentiallygeneratevirusesandotherharmfulcodewithoutneedingthete

299、chnicalproficiencytraditionallyassociatedwithsuchactivities.Similarly,AImodelsbuiltfordrugdiscoverycanpotentiallyalsobeusedfordesigningbiologicalweapons.131 Europol has alsowarnedaboutthepotentialforlargelanguagemod-elstobeusedinvarioustypesofcybercrime.Accordingtotheagency,contentmoderationmaybeins

300、ufficientastherearenumerouswaystobypasssuchrestrictionsorjailbreakthemodels.132ThelackoftransparencyabouthowcompaniessuchasOpenAIusedatahasalsosparkedconcernsabouthowconfidentialinformationmaybeabused.Sever-alhigh-profilecompanieshavebannedorwarneditsemployeesagainstinputtingbusinessinformationintoC

301、hatGPT.133Amazonhasreportedlyobservedthetextgeneratorgeneratingtextthatcloselymatchedinternalcompanydocuments.134Thisindicatesthatthereisariskthatconfidentialinformationisleakedthroughgenera-tive AI models.2.6 Replacinghumansinconsumer-facingapplications withgenerativeAI,whollyorinpartWhengenerative

302、AImodelswereinitiallyintroducedtothepublic,theywereprimarilystand-alonesystems,withwhichenduserscouldgeneratecontent.Astheinterestforthesesystemsrose,thesystemownersintroducedthepossibilitytoincorporatethemintootherapplicationsandsystemsthroughAPIs.Thiscouldentailadd-onstorecreationalapplicationsand

303、systems,butitisalsopossibletoenvisagetheminpartlyorfullyauto-mateddecision-makingsystems,orasreplacementsofhumaninteractioninconsumerfacingservices.Thismayhavefar-reachingimplications.Forexample,OpenAI founder Sam Altman has argued that in the future generativeAImodelsmayfunctionasmedicaladvisersfor

304、peoplewhoaretoopoortoaffordhealthcare.135 In May2023theUSnon-profitorganizationforsupportingpeoplewitheatingdisorderslaidoffstaffandvolunteersforitshelpline,tobereplacedbyanAIchatbot.136 While aspokespersonfortheorganizationclaimedthatthechatbotwasnotadirectreplacementforthehelpline,itneverthelessac

305、companiedtheshutdownofthehelplineservice,leavingpeoplewithoutrealhumanstotalkto.Automatingsuchtasksmaymultiplytheriskoffatalmistakesifthereareproblemsinthetrainingdataorinthe model itself.Foryears,companieshaveattemptedtoautomatecon-sumerinteractions,forinstancebyautomatingcustomerservicethroughchat

306、bots.Manycompaniesmakeitdifficultforconsumerstogetincontactwithhumans,whichadverselyaffectconsumerswhodonothavestandardizedproblemsthatareaddressedinFAQsandsimilardocuments.WiththeriseofgenerativeAI,thereisariskthatcompanieswillmakeitevenmoredifficultforconsumerstogetittouchwithrealhumans.2.6.1 CHAL

307、LENGES RELATED TO COMBINING HUMAN-AND AUTOMATED DECISION-MAKINGAutomatedsystemsdonothavethecapacityforethicalreflection,sympathy,orunderstanding.Generally,peoplearenotpersecutedforminorinfractions,butautomat-edsystemsarenotabletodistinguishbetweenminor“The lack of transparency about how companies su

308、ch as OpenAI use data has also sparked concerns about how confidential information may be abused.“34Norwegian Consumer CouncilJune 2023Ghost in the machineandaggravatedinfractions.Ifaconsumermissedtheirpaymentbyaday,ahumanmightconsiderwhetherthecustomerrelationshouldbeprioritizedoverstrictcompliance

309、withtherules,andthereforeallowforalatepaymentwithnoadditionalcosts.Theautomatedsystemwouldnotbeabletomakesuchconsiderations.Sympa-thyandprinciplesoffairnesscouldthereforebelostinthetransitofautomatingprocesses.Fully automated systems are usually regulated through additionallegalprovisionsandprotecti

310、ons,toaccountforadditionalrisksrelatedtosuchdecisionmaking.This caninsomecasesentailrequirementsofhumaninvolve-ment,137orleadtocompaniesintroducingahumaninthelooptoavoidlegalscrutiny.138Keepinghumansintheloopishoweveracomplexmeasure,withseveralpitfalls.Humanscanbothover-relyandunder-relyontheoutputo

311、fautomatedsystems,139andtheproblemisparticularlyprominentinautomatedcomputersystemsthatdonotproduceexplainableorinterpretabledecisions.Itishow-everover-relianceonautomatedsystemsoutputthatinvolvesthemostnovelchallenges,asopposedtoanin-dividualover-relyingonherowndecisions,whichismoresimilartoawholly

312、manualdecision-makingprocess.Inwhollyorpartiallyautomatedsystemsover-reliancecanaffectdifferentpeople:the“humanintheloop”mightnotchallengethesystem,evenwhenitwouldbeprudent,whilethepersonwhoisaffectedbythedecisionmightnotlodgeacomplaintordemandahumanreviewofthedecision.Inbothcases,theinterestsofthep

313、ersonaffectedbythedecisionareputatrisk.Asdescribedinprevioussections,theoutputoftextgeneratorssuchasChatGPThaveprovenverycon-vincing.Iftextgeneratorsareusedindecision-makingprocessesaffectingconsumers,theriskofover-relianceontheoutputmightincrease.Theseeffectsmaybefurthercompoundedbyendusersbelievin

314、gthattheyareinteractingwithasentientintelligentbeingratherthanaprobabilistictextgenerator.Eveniftheinterlocutorsunderstandadecision,deemituntrustworthy,andthereforeconsideroverturningit,therecanbeadditionalhurdles.Fromabusinessperspective,thereisefficiencytogainfromautomatingwholeorpartsofaprocess.I

315、fthedecisionsfromthemachinearegenerallyupheld,overturningadecisionmightrequiremorein-depthargumentsthanacceptingthedecision.Aninterlocutorwhorepeatedlyoverturnsthedecisions,thushaltingefficiency,maybeseenasatroublemaker.Astoresponsibilityandliability,overturningdecisionscouldprovedifficultforindivid

316、ual,humaninterlocutors.Whileawrongfuldecisionfromacomputersystemcanbeblamedonthatsystem,overturningthedecisioncouldsignificantlyheightentheinterlocutorssenseofriskbecausetheinterlocutorassumesresponsibilityforthedecision.Liabilityregimescanenhancetheactualandperceivedriskforinterlocutors.2.7 Environ

317、mentalimpactAnincreasingnumberofpeopleintheresearchandscientificcommunityareraisingtheissueoftheimpactof generative AI model development on the environment.Inacontextwhereclimatechangeandscarcityofnaturalresourcesareaglobalchallenge,adilemmaarisesbetweenclaimsthatgenerativeAIcansolveclimatechange,an

318、dtheactualenvironmentalimpactofsuchtechnologies.ThissectiontakesacloserlookatsomeoftheseclaimsandprovidesacriticalexaminationoftherealisticimpactthatgenerativeAIontheenvironmentbothtodayandinthenearfuture.Itshouldbenotedthatmanyoftheseimpactsalsoapplytolargeswathesofthebroadertechsector,butitisimpor

319、tantthatthisperspectiveisnot lost in the hype surrounding generative AI.2.7.1 CLIMATE IMPACTSomeactorsinthegenerativeAIfieldclaimthatthetechnologyhasthepotentialtosaveusfromtheperilsofclimatechange.140However,thecurrentlyavailabledatashowsthatdeployinggenerativeAIinthesamecontext2.7Environmentalimpa

320、ct“If text generators are used in decision-making processes affecting consumers,the risk of over-reliance on the output might increase.”35Norwegian Consumer CouncilJune 2023Ghost in the machine2.7Environmentalimpactthatlargetechcompanieshasbeenoperatinguntilnow,ismoreofaproblemthanasolutiontoissuess

321、uchasclimatechange,watershortages,andhighenergyconsumption.TheTechindustryisalreadyemittingasubstantialamountofcarbon.AccordingtoUNEP,in2021,thetechindustrysemissionsaccountedfor2to3%oftheworldscarbonemissions.141InNovember2022,theMITreportedthat“thecloudhasnowalargercarbonfootprintthantheentireairl

322、ineindustry”.GenerativeAIisnoexceptiontothis negative trend.InMay2023,AIreportedly“usesmoreenergythanotherformsofcomputing,andtrainingasinglemodelcangobbleupmoreelectricitythan100U.S.householdsuseinanentireyear”.142Datacentresareknowntouseanincredibleamountofenergy,andalreadyfiveyearsago,itwaspredic

323、tedthattheenergydemandsofworldwidecomputingcouldexceedthetotalworldelectricitypowergenerationwithinadecade143.Thiswasbeforetherapiddevelopment and deployment of generative AI.144 With theexponentialgrowthofgenerativeAImodelsandin-vestmentininfrastructuretosupportthisgrowth,energyuseandcarbonemission

324、sareexpectedtoskyrocket.Forbesrecentlyreportedthat“generativeAIisbreakingthedatacentre”.145Indeed,basedonaresearchbyTiriasResearch,datacentreinfrastructureandoperatingcostsareprojectedtoincreasetooverUSD$76billionby2028duetoAIdevelopment.TiriasResearchestimatesthat“thisisthecostofmorethantwicetheest

325、imatedannualoperatingcostofAmazonscloudserviceAWS,whichtodaycontrolsonethirdoftheglobalcloudinfrastruc-tureservicesmarket”.146Thisexponentialgrowthhasapricefortheenvironment.Theexactpriceisyettobecalculated,butasanindication,whendeployed,planstointegratelargelanguagemodelsintosearchenginesmayinvolve

326、afourfoldincreaseinenergyusageperindividualsearchquery.147Inotherwords,itisclearthatAItechnologycomeswithahighcarbonfootprint,148 and that energy is needed every stepofthewaywhendesigning,training,developing,de-ploying,andusinggenerativeAImodels.149Theproblemisthatthereisstillalackofdataavailableont

327、heamountof energy needed for generative AI development.150 At the timeofwriting,nocompanieshavedisclosednumbersonhowmuchenergywasrequiredforthelifecycleofagenerative AI model.TheenergyconsumptionofgenerativeAIisexponentialandwillhopefullybemoreresearchedwithprojectionsforthenextfivetotenyears,whichw

328、illallowconsumersaccesstoinformation,andpolicymakerstoregulatehowmuchthisindustryshouldemit.Forexample,theamountofcomputingpowerusedtotraindeeplearningmodelsin-creased300,000timesin6yearsbetween2012and2018.151ThereiscurrentlynostandardizedwaytomeasurecarbonemissionsofAImodels,andnogoodwillfromAI-foc

329、usedtechcompaniestoreleasethenecessaryin-formation.WhereasestablishedtechcompaniessuchasMeta,GoogleandMicrosoftpublishyearlySustainabilityreportswheretheyself-reportenergyandwateruseaswellascarbonemissions,AIcompaniessuchasOpenAIdonotpublishanykindofinformationontheirenviron-mentalimpactandhowtheymi

330、tigateit.Asasidenote,itseemslikelythatevenwhentheydomaketheeffortofreporting,largetechcompaniesun-derreporttheirownemissions,accordingtoa2021studyfromtheTechnicalUniversityofMunich.“Across a sample of 56 major tech companies surveyed,more than half of these emissions were excluded from self-reportin

331、g in 2019.At approximately 390 megatons carbon dioxide equivalents,the omitted emissions are in the same ballpark as the carbon footprint of Australia”.152 Thereisalackofinterestinthetechindustryincalcu-latingcarbonemissionsgeneratedbygenerativeAI,astheindustryisinterestedinobtaininghigherresultsina

332、ccuracythroughmassivecomputationalpower,153 at the costofallotherconsiderations154.FortheAIcommunity,itseemsliketherehasbeenaconstantpushfor“biggerisbetter”,wheretheexponentialsizeofmodelsanddatasetsisvaluedabovealmostallelse.155Unfortunately,thisapproachisnotsustainable.Researchershavecalledthisphe

333、nomenon“RedAI”,156whichresultsinrapidlyescalating“Data shows that deploying generative AI in the same context that large tech companies has been operating until now,is more of a problem than a solution to issues such as climate change,water shortages,and high energy consumption.”36Norwegian Consumer CouncilJune 2023Ghost in the machine2.7Environmentalimpactcomputationalandthuscarboncosts.Theresear