《云安全聯盟（CSA）：2024AI模型風險管理框架報告（英文版）（54頁）.pdf》由會員分享，可在線閱讀，更多相關《云安全聯盟（CSA）：2024AI模型風險管理框架報告（英文版）（54頁）.pdf（54頁珍藏版）》請在三個皮匠報告上搜索。

1、AI Model RiskManagement FrameworkTable of ContentsTable of Contents.2Acknowledgments.4Executive Summary.5Intended Audience.6Scope.6Introduction.7The Need and Importance of MRM.7The Four Pillars:Model Cards,Data Sheets,Risk Cards,Scenario Planning.12Benefits of a Comprehensive Framework.15Core Compon

2、ents.181.Model Cards:Understanding the Model.182.Data Sheets:Examining the Training Data.223.Risk Cards:Identifying Potential Issues.244.Scenario Planning:The“What If”Approach.28Combining Techniques:A Holistic Approach.361.Leveraging Model Card Information for Risk Cards.362.Using Data Sheets to Enf

3、orce Model Understanding.363.Using Risk Cards to Inform Scenario Planning.374.Scenario Planning Feedback to Risk Management and Development.415.AI MRM in Action.43Conclusion and Future Outlook.49References.50Appendix 1:AI Frameworks,Regulations,and Guidance.52 Copyright 2024,Cloud Security Alliance.

4、All rights reserved.2The permanent and official location for the AI Technology and Risk Working Group ishttps:/cloudsecurityalliance.org/research/working-groups/ai-technology-and-risk.2024 Cloud Security Alliance All Rights Reserved.You may download,store,display on yourcomputer,view,print,and link

5、to the Cloud Security Alliance at https:/cloudsecurityalliance.org subject tothe following:(a)the draft may be used solely for your personal,informational,noncommercial use;(b)the draft may not be modified or altered in any way;(c)the draft may not be redistributed;and(d)thetrademark,copyright or ot

6、her notices may not be removed.You may quote portions of the draft aspermitted by the Fair Use provisions of the United States Copyright Act,provided that you attribute theportions to the Cloud Security Alliance.Copyright 2024,Cloud Security Alliance.All rights reserved.3AcknowledgmentsLead AuthorsM

7、aria SchwengerVani MittalContributorsEric TierlingHadir LabibMichael RozaRenata BudkoReviewersCandy AlexanderDaniel CElier CruzHarie Srinivasa Bangalore Ram ThilakKaran GoenkaKenneth Thomas MorasNamal KulathungaNicolas RayOtto SulinRohit ValiaSanitra AngramTom BowyerVaibhav MalikYuvaraj MadheswaranC

8、o-ChairsChris KirschkeMark YanalitisCSA Global StaffJosh BukerMarina BregkouStephen Smith Copyright 2024,Cloud Security Alliance.All rights reserved.4Executive SummaryThe widespread adoption of sophisticated machine learning(ML)models presents exciting opportunitiesin fields like predictive maintena

9、nce,fraud detection,personalized medicine,autonomous vehicles,andsmart supply chain management1.While these models hold the potential to unlock significant innovationand drive efficiency,their increasing use also introduces inherent risks,specifically those stemming fromthe models themselves.Unmitig

10、ated model risks can lead to substantial financial losses,regulatory issues,and reputational harm.To address these concerns,we need a proactive approach to risk management.Model Risk Management(MRM)is a key factor for fostering a culture of responsibility and trust indeveloping,deploying,and using a

11、rtificial intelligence(AI)and ML models,enabling organizations toharness their full potential while minimizing risks.This paper explores the importance of MRM in ensuring the responsible development,deployment,anduse of AI models.It caters to a broad audience with a shared interest in this topic,inc

12、luding practitionersdirectly involved in AI development and business and compliance leaders focusing on AI governance.The paper highlights the inherent risks associated with AI models,such as data biases,factualinaccuracies or irrelevancies(known colloquially as“hallucinations”or“fabrications”2),and

13、 potentialmisuse.It emphasizes the need for a proactive approach to ensure a comprehensive MRM framework.This framework is built on four interconnected pillars:Model Cards,Data Sheets,Risk Cards,and ScenarioPlanning.These pillars work together to identify and mitigate risks and improve model develop

14、ment andrisk management through a continuous feedback loop.Specifically,Model Cards and Data Sheets informrisk assessments,and Risk Cards guide Scenario Planning.Scenario Planning refines risk managementand model development.By implementing this framework,organizations can ensure the safe and benefi

15、cial use of ML models withkey benefits such as:Enhanced transparency and explainabilityProactive risk mitigation and“security by design”Informed decision-makingTrust-building with stakeholders and regulatorsThis paper emphasizes the importance of MRM for harnessing the full potential of AI and ML wh

16、ileminimizing risks.2NIST AI 600-1“Artificial Intelligence Risk Management Framework:Generative Artificial Intelligence Profile”1McKinsey&Company“The state of AI in 2023:Generative AIs breakout year”Copyright 2024,Cloud Security Alliance.All rights reserved.5Intended AudienceAI MRM is designed for a

17、 broad audience with a common interest in the responsible development anddeployment of ML models.It bridges the gap between technical and non-technical stakeholders,cateringto those directly involved in the technical aspects of AI development and those concerned with itsgovernance and oversight.The

18、target audience can be segmented into the following two primary groups.1.Practitioners in AI Model Development and ImplementationML Engineers and Data Scientists:This group benefits from the detailed explanations ofModel Cards and Data Sheets and how they contribute to model understanding anddevelop

19、ment.Understanding these components empowers them to build more transparent andaccountable models.AI Developers and Project Managers:This group will find tools to help them anticipatepotential issues throughout the AI model lifecycle,ensuring responsible deployment fromconception to implementation.2

20、.Stakeholders in AI Governance and OversightRisk Management Professionals,Compliance Officers,and Auditors:This group will findsections on the importance of MRM and its alignment with common industry frameworks mostrelevant to establishing,enforcing,and assessing effective governance practices.Busin

21、ess Leaders and Executives:The introduction and conclusion sections will be particularlyvaluable for them,as they highlight the importance of MRM in fostering responsible AI adoptionwithin the organization.Communications and Public Relations Professionals:This group will benefit from sectionson comm

22、unicating AI model risks and benefits,stakeholder engagement,and reputationmanagement,as well as learning how to craft resonant messages for diverse audiences.ScopeThis paper explores MRM and its importance for responsible AI development.It examines closely the fourpillars of an effective MRM framew

23、ork and how they work together to create a holistic approach to MRM.We discuss how these techniques foster transparency,accountability,and responsible AI development.Copyright 2024,Cloud Security Alliance.All rights reserved.6The paper emphasizes the role of MRM in shaping the future of ethical and

24、responsible AI.Note that thispaper focuses on the conceptual and methodological aspects of MRM,and does not address thepeople-centric aspects,such as roles,ownership,RACI,and cross-functional involvement,which arecovered in the CSA publication“AI Organizational Responsibilities-Core Security Respons

25、ibilities”.IntroductionThe Need and Importance of MRMToday,we witness the adoption of complex AI/ML models at an unprecedented rate across diverseindustries.On the one hand,the increasing reliance on ML models holds the promise of unlocking vastpotential for innovation and efficiency gains.On the ot

26、her hand,it introduces inherent risks,particularlythose associated with the models themselves model risks.If left unchecked,they can lead to significantfinancial losses,regulatory sanctions,and reputational damage.Biases in training data,factualinaccuracies in model outputs(called“hallucinations”or“

27、fabrications”3),and the potential for misuse,alongside privacy risks and intellectual property(IP)concerns,necessitate a proactive approach to riskmanagement.AI MRM emerges as a critical discipline to ensure the responsible and trustworthydevelopment,deployment,and utilization of these models.MRM is

28、 a term commonly used in industries like finance,where it traditionally refers to managing risksassociated with quantitative models.In this paper,however,this established concept outlines a frameworkfor managing the risks associated with AI models.AI MRM helps safeguard against the complexities,unce

29、rtainties,and vulnerabilities associated with AImodels,bolstering confidence among users,stakeholders,and regulators in the dependability andfairness of AI-driven decisions.As AI continues to evolve and permeate more sectors,MRM will play anincreasingly vital role in shaping the future of responsibl

30、e AI deployment,benefiting businesses andindustries.At its core,model risks arise from the inherent limitations of the models themselves.Several of the mostfrequently seen sources of AI model risks are:Data Quality Issues:The foundation of any model is its data.Inaccurate,incomplete,or biaseddata ca

31、n lead to a flawed model,resulting in unreliable outputs and erroneous conclusions.Forexample,if a model is trained to predict loan defaults using historical data that underrepresentshigh-risk borrowers,it might underestimate the risk of future defaults,potentially leading tofinancial losses.3NIST A

32、I 600-1“Artificial Intelligence Risk Management Framework:Generative Artificial Intelligence Profile”Copyright 2024,Cloud Security Alliance.All rights reserved.7Model Selection,Tuning,and Design Flaws:Choosing the wrong model architecture oremploying inappropriate algorithms for the given task can s

33、ignificantly impact the modelseffectiveness and reliability.For instance,using a linear regression model to predict a highlynon-linear phenomenon like stock market volatility would likely yield misleading results.It is alsoimportant to ensure the models integrity,particularly when using open-source

34、models,as endusers should be able to verify the models signature to ensure they are using the correct modeland that the Model Card accurately represents the models capabilities and limitations.Inherent Risks to the Best-in-Class Models:Even top-performing models released bybig-name vendors may carry

35、 intrinsic risks based on the shortcomings of the model themselves,such as hallucination,harmful language,bias,and data leakage.These risks can have far-reachingimpacts,affecting not only individual organizations but also society as a whole4.Implementation and Operational Errors:A well-designed mode

36、l can be compromised duringimplementation.Incorrect coding,inadequate controls,or improper integration with existingsystems can introduce model deployment errors.For instance,a credit scoring model may becorrectly developed,but its implementation into the loan processing system may be flawed,leading

37、 to inaccurate assessments and unfair loan denials.Security is another key operational setof risks.Those risks can be both well-established,such as application-level and access-levelvulnerabilities,and new in the GenAI era,such as prompt injections5.AI models also increase therisks of threat actors

38、aiming to change the decision-making that model users aim to use themodel for.Evolving External Factors:Models are often trained on historical data,assuming a certain levelof stability in the underlying environment.However,the real world is constantly evolving.Economic downturns,new regulations,or u

39、nforeseen events can render historical data irrelevant,leading to unreliable predictions by the model.For example,a model trained to predict customerchurn based on past purchasing habits might struggle if consumer preferences shift due to aglobal pandemic.Similarly,a model trained to predict loan de

40、faults based on historical data mayalso struggle if consumer behavior shifts due to unforeseen events such as a global pandemic,changes in economic policies,or unexpected changes in loan activity(new,refinance,andrenegotiation of terms).Both examples illustrate how models can be vulnerable to unexpe

41、ctedchanges in the underlying environment,highlighting the need to monitor and update models toensure their effectiveness.An MRM framework is a structured approach to identifying,assessing,mitigating,and monitoring the risksassociated with ML models,especially in decision-making processes.Establishi

42、ng this is a proactivepractice that safeguards the benefits of ML models while minimizing potential downsides.It acts as aroadmap for organizations to ensure the responsible and trustworthy development,deployment,and useof these models.Its important to note that the specific risks and their severity

43、(risk level)will varydepending on the nature of the organization,industry,business unit,and the models intended use.5World Wide Technology Secure Your Future:A CISOs Guide to AI4CNBC The biggest risk corporations see in gen AI usage isnt hallucinations Copyright 2024,Cloud Security Alliance.All righ

44、ts reserved.8A well-designed MRM framework enables customization by establishing a structured process to identifyand assess model-specific risks.This ongoing process is built on several essential components,whichinclude the following:1.GovernanceThe governance of AI and ML models within an organizat

45、ion is critical for ensuring that these modelsare effectively managed and aligned with the strategic goals and regulatory requirements.Thisinvolves setting clear objectives,maintaining a detailed inventory,defining ownership roles,andestablishing approval processes.Key components of governance inclu

46、de:Business Approach:Defines the organizations overall AI strategy and business goals to identifyareas where AI can be leveraged to improve productivity,efficiency,decision-making,or delivernew user experiences.Model Inventory:Establishes a comprehensive list of all models used within the organizati

47、on,categorizing them by purpose,complexity,risk level,and alignment with the established businessapproach.A well-structured model inventory enables targeted risk assessment and monitoring ofhigh-risk or critical models through categorization based on risk level and potential impact.Model Lifecycle M

48、anagement:Clearly define roles and responsibilities for each models lifecycle,from design and testing to development and deployment,ongoing monitoring andmaintenance,and deprecation.Clear ownership enables efficient knowledge transfer anddocumentation,reducing the risk of knowledge gaps or silos tha

49、t could hinder the modelslong-term maintenance and evolution.Model Approvals:Establishes a formal process and criteria for approving models beforedeployment,ensuring they meet business needs,align with the business architecture,and complywith regulatory requirements.The approval process also evaluat

50、es models for potential biases,ethical concerns,and adherence to responsible AI principles,promoting fairness,transparency,and trustworthiness.2.Model Development StandardsEstablishing robust model development standards is essential for ensuring that AI models are builton high-quality data,adhere to

51、 best practices and comply with relevant regulations.This includesmanaging data quality,following standardized design and development practices,and implementingthorough validation and testing processes.Key components of model development standardsinclude:Copyright 2024,Cloud Security Alliance.All ri

52、ghts reserved.9Data Quality Management:Defines practices to ensure high-quality data for model training byrequiring accuracy,completeness,minimal bias,and minimization(ensuring data is fit for thepurpose and limited to only necessary information)through data diversification and adherence tointellect

53、ual property and privacy protection measures.Model Design and Development:Outlines standards for model architecture,developmentmethodologies,and documentation practices.Align model development standards with existinggovernance and compliance frameworks,including regulatory guidelines.For a list of t

54、he mostprominent guidance,see“Appendix 1:AI Frameworks,Regulations,and Guidance.”Model Validation and Testing:Establishes processes for rigorously testing models to assesstheir performance,accuracy,safety and robustness.Governance and Compliance Frameworks:Align model development standards with exis

55、tinggovernance and compliance frameworks,including recommendations by regulatory guidelines(e.g.,GDPR,CCPA),industry standards(e.g.,ISO 27001,ISO 42001),and organizational policies.For guidance on ensuring adherence to legal,ethical,and risk management requirements,refer tothe CSA publication“Princi

56、ples to Practice:Responsible AI in a Dynamic RegulatoryEnvironment”.3.Model Deployment and UseModel Monitoring:Implements procedures for continuously monitoring model performance inproduction,detecting any degradation in accuracy or unexpected behavior.Model Change Management:Defines a transparent p

57、rocess for changing deployed models,ensuring proper testing and validation before implementation,and providing rollback anddeprecation mechanisms for models no longer in use.Model Communication and Training:Establishes protocols for communicating modellimitations and capabilities to stakeholders and

58、 providing training for proper model usage.4.Model Risk AssessmentModel risk assessment is essential for identifying and mitigating potential risks in AI and ML models,bothdeveloped internally and acquired externally.This process addresses risks across financial,supply chain,legal,regulatory,and cus

59、tomer domains.Key components include:Risk Scope:Risk assessment process applies not only to models developed and used within anorganization,but also to models acquired from third parties or outside organizations.It definesthe types of risks the organization would like to address at all levels,such a

60、s financial,supplychain,legal and regulatory,customer retention,and so on.Copyright 2024,Cloud Security Alliance.All rights reserved.10Risk Identification:This is the initial step in effectively managing risks associated with MLmodels.It involves employing techniques to uncover potential issues thro

61、ughout the modellifecycle systematically.Some key factors considered during risk identification include dataquality,model complexity,intended use,training data acquisition and use of personal data,andmodel protection mechanisms.Risk Assessment:Evaluate the severity and likelihood of identified risks

62、,allowing forprioritization of mitigation efforts.Risk assessment may use qualitative or quantitative methods,such as FAIR-AI6.Risk Mitigation:Develop strategies to address the identified risks,including data cleansing,model improvements,implementing security and privacy controls,and protecting inte

63、llectualproperty.Prioritize the efforts based on the balance of risk reduction of those efforts againsttheir costs and practicability within the organizations environment.5.Documentation and ReportingThorough documentation and regular reporting are vital for maintaining transparency and accountabili

64、tyin model risk management.These practices ensure that all aspects of the model lifecycle arewell-documented and communicated to relevant stakeholders.Key components include:Model Documentation:Maintains comprehensive documentation throughout the modellifecycle,capturing development steps,assumption

65、s,limitations,and performance metrics.Model Risk Reporting:Regularly reports to relevant stakeholders on identified model risks,mitigation strategies,and overall model performance.A robust MRM framework ensures trustworthy development,deployment,and ongoing use of ML models.By proactively identifyin

66、g,assessing,and mitigating these risks,organizations can harness the power ofmodels while safeguarding themselves and their customers and users from potential pitfalls.This ensuresthe reliability and accuracy of model-driven decisions and fosters trust and transparency.6A FAIR Artificial Intelligenc

67、e(AI)Cyber Risk Playbook Copyright 2024,Cloud Security Alliance.All rights reserved.11The Four Pillars:Model Cards,DataSheets,Risk Cards,Scenario PlanningThis framework can be built by combining four key components:Model Cards:Offer a clear and concise window into an ML model.They detail the modelsp

68、urpose,training data,capabilities,Adversarial AI resistance,limitations,and performance,promoting transparency and informed use.Data Sheets:Function as detailed descriptions of a dataset used to train an ML model.Theydocument the creation process,composition(data types,formats),intended uses,potenti

69、albiases,limitations,and any ethical considerations associated with the data.Risk Cards:Summarizes the key risks associated with an AI model.It systematically identifies,categorizes,and analyzes potential issues,highlights observed risks during development ordeployment,explains current and planned r

70、emediations,and outlines expected user behavior toensure responsible use of the model.Scenario Planning:Explores hypothetical situations where a model could be misused ormalfunctioning,helping identify unforeseen risks and develop mitigation strategies.Figure 1.Framework Pillars for Responsible and

71、Well-Informed Use of AI/ML Copyright 2024,Cloud Security Alliance.All rights reserved.12These techniques work together to create a holistic approach.In a nutshell,information from Model Cardsinforms risk assessments;building on the foundation of Model Cards and Data Sheets provide additionalcontext

72、for understanding model strengths and limitations.Risk Cards guide Scenario Planning exercises,and Scenario Planning outcomes feed back into risk management,creating a continuous feedback loop.Note:The difference between the Training Data category in Model Cards and the Technical Specificationssecti

73、on in Data Sheets is that the Training Data category in Model Cards refers to the specific dataset usedto train a machine learning model,including its sources,size,quality,and preprocessing steps.On the otherhand,the Technical Specifications section in Data Sheets provides a detailed description of

74、a datasetstechnical construction and operational characteristics,including data schema,processing steps,andtechnical dependencies,which are not limited to model architecture.Understanding this distinction isessential to effectively utilize both Model Cards and Data Sheets for managing and maintainin

75、g machinelearning models and datasets.By combining these techniques,organizations can create a comprehensive Risk Management Framework(RMF)that fosters:Transparency and Explainability:Model Cards,Data Sheets,and clear communicationempower stakeholders to understand model capabilities and limitations

76、.Techniques like LocalInterpretable Model-agnostic Explanations(LIME),SHapley Additive exPlanations(SHAP),Integrated Gradients,Concept Activation Vectors(CAVs),and model distillation can provide localexplanations,identify high-level semantic concepts,and create interpretable surrogate models,respect

77、ively,to enhance transparency and explainability of complex models.Proactive Risk Management:A multi-faceted approach is key to effective risk management.This includes utilizing Model Cards to document potential biases and limitations,leveraging DataSheets to understand the training data,conducting

78、thorough risk assessments(based on the RiskCards)to identify general risks,and engaging in Scenario Planning to explore potential futurechallenges.Additionally,adversarial testing,stress testing,edge-case analysis,and regularizationtechniques like dropout,L1/L2 regularization,or adversarial training

79、 can help identifyvulnerabilities,blind spots,and improve model robustness,enabling proactive risk management.Consistent Risk Management:Consistency in the risk management process is focused onensuring that risk assessments are reproducible and allow comparison and tracking of AI modelsperformance a

80、nd safety over time.Consistent risk assessments help accurately monitor risksevolution and mitigation strategies effectiveness,fostering continuous improvement in AIsystems.Informed Decision-Making:A comprehensive understanding of model risks empowersstakeholders to make informed decisions about mod

81、el deployment and use.Building Trust,Credibility,and Ethical Use:Transparency and responsible risk managementpractices build trust and foster ethical use of ML models.Implementing privacy-preservingtechniques,obtaining certifications for ethical AI practices,establishing governance frameworks Copyri

82、ght 2024,Cloud Security Alliance.All rights reserved.13and ethical AI committees,and conducting third-party audits can build trust,credibility,andfoster ethical use of ML models.Continuous Monitoring and Improvement:Ongoing monitoring and the ability to adapt basedon new information ensures the mode

83、ls sustained effectiveness and safety.Some techniquesinclude adopting Machine Learning,Security,and Operations(MLSecOps)practices.Thisinvolves setting up monitoring pipelines to track model performance,data drift,feedback loops,and unintended consequences.Additionally,implementing online or continuo

84、us learningtechniques is important.Establishing processes for incorporating user feedback,incident reports,and lessons learned can ensure sustained effectiveness,safety,and continuous improvement ofAI systems.Copyright 2024,Cloud Security Alliance.All rights reserved.14Benefits of a ComprehensiveFra

85、meworkA comprehensive risk management framework(RMF)for ML models offers several benefits,definedbelow.Enhanced Transparency,Explainability,and AccountabilityModel Cards,Data Sheets,and Risk Cards are pivotal for transparency,explainability,and accountabilitywithin MRM.Data Sheets document the datas

86、 origin,acquisition composition,and pre-processingmethods,providing crucial context for understanding a models inputs,limitations,and role.Thisdocumentation can help you understand the inner workings of a model to a degree,allowing for someassessment of its strengths,weaknesses,and potential bias.Wh

87、at is available for proprietary models istypically much more restricted when compared to open-source models.Proactive Risk Assessment and Scenario AnalysisData Sheets complement Scenario Planning by detailing the data-specific characteristics that couldinfluence model performance under different sce

88、narios.This information is vital for conducting thoroughrisk assessments and ensures that scenario analyses consider data quality and other factors relevant tothe company.Development of Risk Mitigation StrategiesIncorporating insights from Data Sheets into the risk mitigation process allows for more

89、 targetedstrategies.Understanding data limitations and biases helps in designing effective mitigations,such asdata cleansing,augmentation,or re-balancing techniques,which are critical for addressing potential risksidentified by Risk Cards.Informed Decision-Making and Model GovernanceData Sheets,whic

90、h detail the training data and model characteristics,are critical in informing governancepractices.This detailed understanding ensures well-founded,documented,and transparent decisionsaround model deployment.While training data can be swapped out,its quality directly impacts themodels behavior.Data

91、Sheets help identify potential limitations and biases within the data,as well as the Copyright 2024,Cloud Security Alliance.All rights reserved.15characteristics of the model that might influence its outputs.This comprehensive information leads toinformed decisions about model deployment.Data Sheets

92、 provide essential information that influencesdecision-making processes by highlighting data-related constraints and opportunities.In MRM,thisdetailed understanding of data characteristics informs governance practices,ensuring decisions aroundmodel deployment are well-founded,documented,and justifia

93、ble.Robust Model ValidationRobust model validation is integral to the MRM framework,ensuring that models perform as expectedand adapt to real-world conditions.This involves rigorous testing with diverse datasets that reflectreal-world scenarios.Information from Data Sheets,such as the data distribut

94、ion and potential biases,caninform the selection of these datasets for a more comprehensive validation process.Techniques likediversity testing,stress testing,and generalizability metrics are crucial for this validation process.Byincorporating these validations,the framework ensures that models main

95、tain effectiveness and avoidunexpected performance issues or biased outcomes in real-world applications.Building Trust and Enhancing Model AdoptionData Sheets form the groundwork for trust by ensuring data clarity.However,building confidence requiresa multi-layered approach.Model Cards provide deepe

96、r insights into the models inner workings,and RiskCards proactively address potential biases or limitations.This promotes transparency and responsible AIdevelopment,ultimately leading to increased user and regulatory trust in adopting the model.Thesedocuments provide transparent and honest communica

97、tion about model capabilities and performanceexpectations.This clarity is crucial for gaining the confidence of users and regulators,particularly insectors where data provenance and integrity are critical.Continuous Monitoring and ImprovementContinuous monitoring is integral to the MRM framework,ens

98、uring that models operate as expected andadapt to changes over time.This involves regular updates to Model Cards,Risk Cards,and Data Sheets toreflect changes in the models performance or operational environment.For example,metrics such asaccuracy,precision,and F1 score may be tracked to measure perf

99、ormance,while Mean Absolute Error(MAE)and Mean Squared Error(MSE)may be used to evaluate model drift.Continuous monitoring helpsidentify when a model might be drifting from its intended performance or when changes in the externalenvironment necessitate model adjustments or deployment strategy.This o

100、ngoing vigilance helps ensuresustained compliance,efficacy,and safety of ML models in dynamic operational contexts.Copyright 2024,Cloud Security Alliance.All rights reserved.16Positive Societal and Ethical ImpactData Sheets are foundational for addressing societal and ethical bias in ML models.Docum

101、enting thetraining datas origin,composition,and preprocessing methods provides crucial transparency to identifypotential biases,which is crucial for developing fair and equitable ML models.By ensuring that datahandling practices are aligned with ethical standards,organizations can better manage the

102、broaderimpacts of their technologies.Strong Governance and OversightStrong governance and oversight,built on a foundation of controls that ensure alignment with theorganizations objectives,guarantee transparent,explainable,and accountable AI model development,use,and maintenance,guided by ethically

103、aware and competent individuals.They establish robustenforcement mechanisms that ensure ethical guidelines and responsible data practices are followed.Effective governance involves clear roles and responsibilities,defined decision-making processes,andescalation procedures for resolving conflicts or

104、disputes.Regular audits provide a layer of accountability,verifying stakeholder commitment to these principles.Rigorous change management procedures,controlupdates,retraining,and deployment decisions promote oversight and proactively mitigate potential risks.Clear communication and collaboration amo

105、ng stakeholders,including users,data scientists,engineers,and business leaders,are crucial for successful governance and oversight.Copyright 2024,Cloud Security Alliance.All rights reserved.17Core Components1.Model Cards:Understanding the ModelModel Cards provide a transparent overview of a model.Th

106、ey detail the models purpose,training data,capabilities,limitations,and performance metrics.This information helps developers,deployers,riskmanagement professionals,compliance officers,and end-users understand the models strengths andweaknesses,forming the foundation for risk assessment.Key elements

107、 of a Model Card usually include:Models Details and Intended Purpose:This clarifies the models function and goal.Training Data Details:This describes the composition of the data used to train the model,including its source,size,how it was acquired(consent,donation,etc.),ethical considerations,andpot

108、ential biases.A link to a Data Sheet(if available)can be provided for further details.Intended Use Cases and Limitations:This explains what the model can be used for and whereit might not perform well.Performance Metrics(Evaluation Metrics):This outlines how well the model performs onrelevant tasks,

109、using clear metrics like accuracy and generalizability.Evaluation Methodologies Employed:This describes the methods used to assess the modelsperformance.Model Explainability and Bias:This section describes techniques for understanding themodels decision-making process and identifying potential biase

110、s.It also details methods formitigating bias and ensuring fair outcomes across different groups.Known Limitations:This acknowledges potential shortcomings of the model,such assusceptibility to specific prompts or factual errors.Sustainability and Environmental Aspects(Optional):If available,this est

111、imates theenvironmental impact of training the model(e.g.,carbon emissions).Adversarial Resistance(Performance Metrics under Adversarial AttackOptional):Although specific details of adversarial training are not usually documented in the Model Cards,based on our experience,we recommend including adve

112、rsarial resistance metrics in theevaluation section of the Model and Risk Cards.Data scientists can demonstrate a modelsresilience by reporting accuracy metrics under simulated adversarial attacks,providing a morecomprehensive understanding of the models performance and potential vulnerabilities.Cop

113、yright 2024,Cloud Security Alliance.All rights reserved.18Benefits of Model CardsModel Cards offer a wealth of advantages that contribute to responsible AI development and deploymentand serve as a foundation of risk management,including:Insights and Transparency:Model Cards guide stakeholders,helpin

114、g them understand themodels design,development process,and deployment.They illuminate the training data and themodels performance metrics,allowing users to grasp its capabilities and limitations.Identifying Potential Risks:By outlining the composition of the training data,Model Cards canreveal poten

115、tial issues like bias when the outputs may be influenced in unfair or discriminatoryways,copyright violations,limited generalizability when the model might not perform well incontexts different from its training data,factual errors stemming from inaccuracies in the trainingdata,and others.Reproducib

116、ility/Accountability:Model Cards document the development process,enablingothers to recreate the model and independently assess its risks.Foundation for Risk ManagementModel Cards serve as the cornerstone for effective risk management of ML models,providing keyinformation about a model,including:Tra

117、ining Data Characteristics:Revealing potential privacy breaches,copyright infringement,and biases.Behavior and Performance Limitations:Anticipating situations where models might generateunreliable or misleading outputs.Benefits for Risk MitigationTailored Mitigation Strategies:Knowing the types of r

118、isks allows seeking relevant mitigationstrategies and then focusing on the ones with the highest risk reduction potential at acceptableimplementation complexity,for example,developing specific safeguards against risks likegenerating harmful contentCommunication and Transparency:Facilitating stakehol

119、der communication and responsibleuseGuiding Prompt Design:Designing prompts for safe and accurate responses Copyright 2024,Cloud Security Alliance.All rights reserved.19Compliance and Trust:Assessing compliance with regulations,fostering trust,and ensuringinformed decisions about model trustworthine

120、ss and safetyTraining Data Curation:Ensuring data quality and fairnessImplementing Guardrails:Documenting techniques to prevent unintended outputsIn essence,Model Cards act as a comprehensive record,promoting responsible AI development anddeployment and establishing the foundation for risk managemen

121、t and mitigation.Creating and Updating Model CardsModel Card Creation EssentialsEffective Model Card creation requires a collaborative and automated approach to ensure accuracy andefficiency.The most common best practices include the following:Process and Ownership:A clear process and ownership for

122、creating and maintaining ModelCards must be established within the organization.Key leaders are responsible for enforcing thisprocess and appointing a specific owner for each Model Card.The selected owners should havethe skills to ask the right questions,gather necessary information,and lead collabo

123、rations acrossthe organization.Ideally,they should have experience building Model Cards or be able to learnquickly,with sufficient technical knowledge.Not every model may require a Model Card,so clear guidelines should be defined forwhen Model Cards are necessary,for example,for models used by over

124、100 people or inproduction or testing.Collaboration:Involve cross-functional teams in the creation process to ensure comprehensivecoverage.Template:Use a standardized template to ensure consistency and ease of use.Automation:Leverage automation tools to generate Model Cards,reducing manual effort an

125、dincreasing accuracy.Version Control:Utilize version control systems to track changes and maintain a clear record ofupdates.Model Card Repositories:Establish a centralized repository for Model Cards,ensuring easyaccess and management.Copyright 2024,Cloud Security Alliance.All rights reserved.20Keepi

126、ng Model Cards Up to DateRegular updates are critical to ensure Model Cards remain accurate and relevant.Implementing astreamlined update process reduces manual effort,increases efficiency,and should include:Regular Reviews:Conduct regular reviews of Model Cards to reflect changes in the model ordat

127、a.Automated Updates:Utilize automation tools to update Model Cards,reducing manual effortand increasing accuracy.Change Management:Establish a process to document and approve updates properly.Audit Trail:Maintain an audit trail of all updates and changes to ensure transparency andaccountability.Some

128、 additional advanced techniques can be leveraged to create a streamlined and efficient process forcreating and updating Model Cards.For example,ML algorithms can analyze model performance andupdate Model Cards dynamically,while natural language processing algorithms can generate Model Cardcontent au

129、tomatically.Visualization tools can provide a graphical representation of model performanceand updates,making complex data easier to understand.Integrating Model Cards with other tools andsystems,such as version control and collaboration platforms,can enhance collaboration and reducemanual effort.Th

130、ese approaches can improve the process with enhanced accuracy,efficiency,andcollaboration.Limitations of Model CardsCompleteness and Accuracy:The details rely solely on how thoroughly and accurately theModel Card is filled out.This leaves a risk of misleading or incomplete information,especiallywhen

131、 this process is primarily manual.For this reason,we advocate automating data collection asmuch as possible.However,ensuring completeness and accuracy also requires a cultural shiftwithin the organization,sponsored and enforced by management,to prioritize Model Cardupdates and maintenance.Without le

132、adership buy-in,even well-intentioned developers maydeprioritize Model Card creation and updates,hindering the effectiveness of this riskmanagement tool.Static Representation:Model Cards offer a valuable snapshot of a model at a specific time,buttheir static nature can pose challenges.As models are

133、updated and improved,the informationdocumented in the Model Cards may become outdated.This necessitates regular review andupdates to the Model Card to ensure it accurately reflects the models current state.Subjectivity in Evaluation:Models focusing on fairness or ethical consideration can beinherent

134、ly subjective as no standardized benchmarks or evaluation criteria exist.Copyright 2024,Cloud Security Alliance.All rights reserved.21Limited Scope:While Model Cards provide technical details like architecture,training data,andperformance metrics,they often fall short of comprehensively addressing t

135、he models impact.This limited scope can overlook potential biases,ethical considerations,and social implicationsthat arise from the models real-world application.Varying Levels of Detail:Theres no standardized format for Model Cards.The level of detailand clarity can vary,making comparisons and risk

136、 assessment across different models difficult.Model Cards are valuable tools for understanding ML models and their potential risks.They promotetransparency and allow developers and users to understand the models strengths and weaknesses.2.Data Sheets:Examining the Training DataData Sheets of model b

137、lueprints provide an in-depth technical description of an ML model.They serve asa reference document for developers,risk managers,and auditors,detailing the models constructionparameters and operational characteristics.This information is crucial for understanding the modelspotential strengths,weakn

138、esses,and inherent risks.The Need for Data SheetsWhile Model Cards and Risk Cards offer valuable insights for risk management,an essential element stillneeds to be added:a transparent view of the models internal logic.Data Sheets bridge this gap as afoundational document for effective model risk man

139、agement.Heres how Data Sheets foster trust andenable more informed risk assessments:Model Transparency:Understanding how a model arrives at its decisions is crucial for riskmanagement.While Model Cards provide a high-level overview and Risk Cards highlight potentialissues,they dont delve into the mo

140、dels inner workings.Data Sheets address this gap by lookingdeeper into the models logic.This transparency fosters trust in the model and empowers riskmanagers to make more informed assessments of its limitations and potential biases.Risk Assessment:By understanding the models construction and traini

141、ng data,risk managerscan effectively evaluate potential sources of model risk,such as data quality issues,overfitting,oralgorithm bias.Model Governance:Data specifications serve as a cornerstone for model governance practices,facilitating ongoing monitoring,maintenance,and retraining of the model as

142、 needed.Reproducibility:Detailed specifications ensure independent parties can recreate and validatethe model,promoting trust and confidence in its outputs.Copyright 2024,Cloud Security Alliance.All rights reserved.22The Role of Data Sheets in MRMBeyond simply documenting the models logic,Data Sheet

143、s empower proactive risk management andensure model fit.They provide the roadmap for ongoing improvement and compliance,fulfilling criticalfunctions in the MRM lifecycle as follows:Risk Identification and Mitigation:Data specifications enable risk managers to proactivelyidentify potential failure po

144、ints within the model and develop mitigation strategies.Model Validation and Refinement:The documented training process and performance metricsallow for rigorous validation of the models effectiveness and generalizability.Data specificationsalso provide a basis for ongoing calibration and refinement

145、 of the model to address identifiedbiases or performance limitations.Regulatory Compliance:Comprehensive data specifications can play a vital role indemonstrating compliance with relevant regulations and ethical guidelines for AI/ML modeldevelopment and deployment.Key Elements of a Data SheetData Sh

146、eets provide a concise and accessible overview of the models inner workings,including:Model Purpose and Scope:Clear definition of what the model is designed to achieve and thelimitations of its use.Data Inputs and Assumptions:A detailed listing of all input features the model uses,includingdata sour

147、ces/types/formats and any pre-processing transformation steps applied,together withany underlying assumptions made.Model Architecture:A technical description of the models architecture(e.g.,decision tree,neural network),including hyperparameter settings(learning rate,number of layers)and thechosen a

148、lgorithm.Model Development Process:Briefly outline the steps to build and train the model,includingany relevant algorithms used.Training Data Characteristics:A breakdown of the training data used to develop themodel,encompassing data source(s),size,distribution characteristics,and any dataquality ch

149、ecks performed.Training Process:Documentation of the training process,including the chosenoptimization algorithm,objective for success,and convergence criteria.Copyright 2024,Cloud Security Alliance.All rights reserved.23Performance Metrics:This is a comprehensive set of indicators used to evaluate

150、themodels effectiveness at the training and validation datasets(e.g.,accuracy,precision,recall,F1 score).Model Outputs and Interpretation:A clear definition of the models output format,includingdata types and how the interpretations of the generated results should be understood.Assumptions and Limit

151、ations:Transparent disclosure of any assumptions made during modeldevelopment and any limitations inherent to the chosen model architecture or training data.Limitations of Data SheetsWhile Data Sheets offer significant advantages,its crucial to acknowledge their limitations to ensure theyare used ef

152、fectively.Data Sheets can present challenges in complexity and scope and keep pace with theevolving field of AI/ML.Some of these limitations include:Complexity:Depending on the specific components of the AI/ML framework,including thetraining dataset,selected algorithm,machine learning operations(MLO

153、ps)control regime,andperformance metrics measured,the data specifications can become highly technical,requiringML expertise to comprehend fully.Limited Scope:Data specifications primarily focus on the technical aspects of the model.Theymay not fully capture the broader business context or potential

154、societal implications of themodels outputs.Evolving Field:As AI/ML rapidly evolves,data specification best practices may need to becontinually adapted to incorporate new technologies and methodologies.Common Limitations with Model Cards,such as completeness and accuracy,becoming acompany culture,and

155、 static/outdated representation,also apply to Data Sheets.Data Sheets are an essential tool for managing model risk.By providing a technical roadmap for themodels construction and operation,they empower risk management professionals to effectively assess,mitigate,and govern the risks associated with

156、 ML models.3.Risk Cards:Identifying Potential IssuesRisk Cards delve deeper into potential issues associated with AI models.They systematically identify,categorize,and analyze potential risks.Think of them like flashcards for potential model risks.Each carddescribes a specific risk,potential impact,

157、and mitigation strategies.Similar to flashcards,they provide aquick and structured way to understand and address model vulnerabilities.Copyright 2024,Cloud Security Alliance.All rights reserved.24Risk Cards typically encompass a range of potential concerns,including:Safety and Ethical Risks:These en

158、compass issues such as privacy,generation of harmfulcontent,and the promotion of bias.Security Risks:Data breaches,manipulation attempts,and other security vulnerabilities fallunder this category.Societal Risks:Job displacement or the misuse of AI for propaganda are examples of societalrisks.Environ

159、mental Risks:AI models can use a lot of electric power,and thus increase thegeneration of harmful gasses.Even models that use clean energy take that energy away fromother social uses,thus forcing them to generate harmful gasses.Operational Risks:Models can face challenges related to limited training

160、 data,computeintensity,integration with existing systems,and so on.Regulatory and Legal Risks:The organization may fall foul of laws,rules,regulations(LRR)dueto its initial implementation or due to LRR changing over time.Or use of input data may bechallenged by owners of intellectual property rights

161、.Financial Risks:Costs of serving the model can increase unexpectedly,such as using agenticworkflows.Supply Chain Risks:Relate to risks carried from outside the organization and ones with potentialto carry from our model to partners.Reputation Risks:Inappropriate model usage can lead to negative pre

162、ss,and so on.Note that risk categories may differ for your organization,or at least the depth of focus on each riskcategory may differ.For example,the NIST AI RMF7focuses on risks to a model being“valid and reliable,safe,secure and resilient,accountable and transparent,explainable and interpretable,

163、privacy-enhanced,and fair with harmful bias managed”.Structure of Risk CardsEach Risk Card follows a well-defined structure to ensure a focused and informative approach towardunderstanding the specific risk and developing targeted mitigation strategies.The following elements canbe typically found in

164、 each Risk Card:Risk Category:Classify risks(e.g.,bias,factual errors,misuse)7NIST AI 100-1“Artificial Intelligence Risk Management Framework(AI RMF 1.0)”Copyright 2024,Cloud Security Alliance.All rights reserved.25Risk Description:A concise description of the potential problem,such as bias,factual

165、errors,orgenerating harmful contentImpact:The potential consequences of the risk,considering factors like reputational damage,user harm,or legal issuesSeverity Level:Assess the potential impact of the risk(high,medium,low)Likelihood:Evaluate the probability of the risk occurringMitigation Strategies

166、:Actionable steps to reduce the likelihood or severity of the risk couldinvolve data filtering techniques,improved training data,user prompts guiding the modelsdevelopment toward safer outputs,and operational and organizational strategiesThe table below presents an example of a Risk Card.RiskDescrip

167、tionImpactMitigation StrategiesBias&FairnessModel outputs biasedcontent based on trainingdataPromotesdiscrimination and apotential forreputational damageUse diverse training dataImplement fairness checks in themodelProvide transparency on limitationsThis Risk Card highlighted the potential for unint

168、ended bias in the ML model used by a retail company forgenerating marketing and social media content.With a clear description and High potential impact(highseverity),the data team prioritized addressing the issue.The company conducted a dual review of thetraining data and the model architecture to i

169、nvestigate potential biases.The data team analyzed datademographics,identified skews in representation,and examined the sources of the training data forpotential bias.They also discussed fairness metrics to quantify potential biases and used techniques likeinterpretability methods to understand how

170、the model arrives at its outputs.Based on this analysis,several mitigation strategies were implemented:Data De-biasing:Balancing the training data through oversampling/under-sampling andremoving non-essential sensitive attributes were used to create a more balanced dataset.Thecompany is also explori

171、ng using synthetic data to address bias further.Fairness in Training:Fairness constraints were incorporated into the training process to penalizebiased outputs and reinforce appropriate outputs.Post-processing Filters:Deploying sentiment analysis and fact-checking tools to identify andflag potential

172、ly biased content after generation.Copyright 2024,Cloud Security Alliance.All rights reserved.26Beyond these mitigation strategies,the company also developed a well-thought-out contingency plan toreinforce the teams defense against bias.This contingency plan included:Flag and Address Biased Outputs:

173、A process to clearly flag and address biased outputsinvolves human reviewers who can identify and correct biased content.Incident Response Protocol:When a Risk Card scenario is triggered,it is highly beneficial if theorganizations already have a pre-established incident response protocol that can be

174、 leveraged byAI/ML Ops teams to ensure swift investigation and mitigation.Actions may include retraining themodel with a more balanced dataset,as in the case of bias detection.Communication Protocols:Cross-company communication protocols,regarding potential bias,ensure transparency and foster trust

175、with users and stakeholders,promoting responsible modelusage throughout the organization.By implementing these mitigation strategies,particularly the focus on data diversity and algorithmicfairness,the team took a proactive stance against bias in the models outputs.This established afoundation for b

176、uilding trustworthy and ethical AI systems across the organization,enabling the companyto promote inclusivity,transparency,and accountability in its AI applications.Benefits of Risk CardsRisk Cards offer a structured and dynamic approach to managing the ever-evolving landscape of modelrisk.They prov

177、ide a systematic way to identify,categorize,and prioritize model risks and act as a powerfulcommunication tool,facilitating discussions among developers,users,and stakeholders.This collaborativeenvironment fosters a deeper understanding of potential issues,leading to the development of actionableins

178、ights such as mitigation strategies and contingency plans.Beyond these core benefits,Risk Cards offer significant advantages specifically for MRM which include:Proactive Approach:Risk Cards help identify potential issues before they occur,allowing forpreemptive solutions.This approach enables evalua

179、ting each strategys potential risk reductionbenefit versus its complexity and costs,ensuring proactive mitigation with the best return oninvestment.Stress Testing:Risk Cards facilitate the process of stress testing the model under variousconditions by prompting discussions and brainstorming around p

180、otential risks.Risk Cards are astarting point for stress testing.Actual stress testing involves applying quantitative andqualitative techniques to analyze how the model would behave under those risks identified in theRisk Cards.The results of the stress tests are generally not recorded in the Risk C

181、ards but mayinform another iteration of the Risk Cards.Copyright 2024,Cloud Security Alliance.All rights reserved.27Improved Decision-Making:Through comprehensive risk identification and analysis,Risk Cardsempower organizations to make informed choices about deploying the model and selectingappropri

182、ate use cases.This ensures the model is utilized effectively while minimizing associatedrisks.Limitations of Risk CardsLimited Scope:Risk Cards typically focus on a predefined set of potential issues.This can bebeneficial for covering common risks,but it might not capture all the unique vulnerabilit

183、iesspecific to your AI model.This limitation also includes inadequate quantification,which hindersthe assessment of risk impact and likelihood,making it challenging to prioritize mitigation efforts.Additionally,complex or nuanced risks might be oversimplified or condensed,which could lead tounderest

184、imating the severity or mitigation challenges.Dynamic Nature of AI:AI models constantly evolve,and new risks can emerge.Risk Cards needto be able to keep pace with the rapid development in the field.Inadequate Quantification:While Risk Cards provide a qualitative assessment of risks,they may fall sh

185、ort in quantifying the potential impact and likelihood of each risk.Withoutquantitative measures,organizations can struggle to prioritize and allocate resourceseffectively to mitigate the most significant risks associated with AI models.Real-World Data Dependence:The effectiveness of Risk Cards depe

186、nds on the quality andcomprehensiveness of the data used to identify and assess risks.Incomplete or inaccurate datamight lead to misleading or irrelevant Risk Cards.Human Judgment Required:Risk Cards require human judgment to interpret the severity of arisk and choose appropriate mitigation strategi

187、es.This can be subjective and can depend on theexpertise of the person reviewing the cards.4.Scenario Planning:The“What If”ApproachScenario Planning is a proactive approach exploring hypothetical situations where an AI model could bemisused or malfunctioning.Essentially,its about asking what if.We i

188、magine and explore how an AImodel might behave in various positive and negative situations.This allows us to identify potential risksbefore they become reality.Copyright 2024,Cloud Security Alliance.All rights reserved.28Scenario Planning ConsidersPositive scenarios(e.g.,increased productivity,impro

189、ved education)Negative scenarios(e.g.,weaponization of language,manipulation of information)Aspects to Consider During Scenario PlanningTechnical Capabilities:Evaluate the models strengths and weaknesses,focusing on areassusceptible to malfunctions(from regular to“black swan”8),manipulation or explo

190、itation.Data Biases:Examine potential biases and data characteristics like less-trusted vendor data,missing or out-of-range data,and volatile-over-time data present in the training data that couldinfluence the models outputs.User Interaction:Consider how users interact with the model and how their i

191、ntent orunderstanding could lead to unintended consequences.Societal Impact:Explore potential broader societal impacts of model deployment,such as jobdisplacement or ethical concerns surrounding automation or risks from usage of the model bypeople outside your organization.How Scenario Planning Work

192、sScenario Planning involves a structured approach to identify and assess potential model risks throughhypothetical situations.Heres a breakdown of the process:1.Assemble the TeamGather a diverse team with expertise in technology,risk management,ethics,legal,regulatorycompliance,or specific data or a

193、pplication domains.The ideal team composition will depend on theprojects specific requirements,and may include a combination of the following stakeholders:Business ExpertsDomain Experts:Individuals who deeply understand the specific application domain(e.g.,healthcare,finance)can provide valuable con

194、text for exploring scenarios relevant toreal-world use cases.8Wikipedia Black swan theory Copyright 2024,Cloud Security Alliance.All rights reserved.29End-Users:Including representatives of the intended user group provides insights intopotential user interactions and how the model might be misused u

195、nintentionally.Risk ExpertsSecurity Practitioners:Individuals with experience in threat modeling and quantifyingthe impact and likelihood of model vulnerabilities aid the risk discussion.Privacy and Legal Advisors:Professionals with knowledge of the specific legal contextof the organization and data

196、 being used,as well as privacy and information governanceindividuals can advise on privacy considerations for models processing personal data.Risk Management Specialists:They bring experience in identifying and mitigatingrisks,ensuring a structured and comprehensive approach to Scenario Planning.Eth

197、ical Advisors:Their expertise in ethical considerations helps to explore potentialsocietal impacts and ensure responsible model development.AI ExpertsModel Developers:Their expertise in model architecture and functionalities providesvaluable insight into the systems capabilities and potential vulner

198、abilities.Data Scientists:Their knowledge of the models training data and potential biases helpsidentify and estimate fairness and representation risks.Their knowledge of the modelarchitecture clarifies the feasibility of management of specific risks.By bringing together this diverse range of perspe

199、ctives,the Scenario Planning team can betterunderstand the AI model and identify a wider range of potential risks.This collaborative approachresembles product red-teaming,where diverse expertise and perspectives are leveraged to stress-testideas and identify potential vulnerabilities.This approach a

200、lso allows for blue-teaming capabilities,such asapproaches for risk reduction.The effectiveness of this approach relies on assembling a team with thenecessary bench strength to facilitate effective ideation and risk assessment.2.Define Scope and ObjectivesThe next step involves clearly defining the

201、scope and objectives of the Scenario Planning exercise.Thisincludes specifying the AI system and the risks you want to explore.Establishing clear objectives,such asidentifying potential biases,security vulnerabilities,or societal impacts,helps guide the teams focus andensures a productive Scenario P

202、lanning session.Copyright 2024,Cloud Security Alliance.All rights reserved.303.Prioritize Scenarios to Dive intoWhile a group that contributes diverse perspectives is great for proposing comprehensive potentialscenarios,it can easily propose a list that is not feasible to plan through completely.Tha

203、t oftennecessitates careful prioritization.The team should pick their prioritization approach,such as some“t-shirt”sizes on definitions of“return”(e.g.,potential risk impact vs.reduction)and“investment”(e.g.,the effort Scenario Planning and implementation may take)for roIe comparisons.What matters i

204、s thatthe team prioritizes in a way that leadership feels comfortable with the risks of the scenarios that wont beplanned in as much detail.4.Gather InformationThe team should gather relevant information to understand the AI model and potential riskscomprehensively.Model Cards,Data Sheets,and Risk C

205、ards provide valuable insights into the ML modelscapabilities,limitations,and potential risks.These documents detail the training data,the modelsarchitecture,and any known vulnerabilities.Additionally,researching relevant safety incidents or misusecases involving the model helps the team anticipate

206、potential real-world threats.The gatheredinformation should have enough detail to plan a scenario,but no more than that.5.Develop ScenariosThe core of Scenario Planning lies in creatively generating diverse hypothetical situations.Encourage theteam to think outside the box and explore positive and n

207、egative scenarios.Techniques like what-ifquestions can spark creative thinking and create a broader range of scenarios.For example,the teammight explore how a Large Language Model(LLM)used in customer service could be manipulated togenerate biased responses or how a malfunctioning model in a financi

208、al setting could lead to inaccurateinvestment recommendations.6.Evaluate ScenariosOnce scenarios have been developed,the team needs to analyze each one systematically.This involvesconsidering the likelihood of the scenario occurring and the potential consequences,if it does materialize.The scenarios

209、 impact on various stakeholders,including users,society,and the organization,should beassessed.Consider how each scenario could affect the models accuracy,reliability,fairness,and security.For instance,a scenario exploring the spread of misinformation by an LLM would need to consider thepotential so

210、cietal harm and reputational damage to the organization.Copyright 2024,Cloud Security Alliance.All rights reserved.31You can even use a language model to simulate these scenarios.Observe its outputs and identifypotential risks,such as generating discriminatory text,spreading misinformation,or creati

211、ng harmfulcontent.This step is among the most prone to scope creep(i.e.,more work than initially budgeted for)and thuscareful and disciplined project management is important here.Overly tight time control is also a risk.Ideally,this trade-off of evaluation depth versus coverage of key scenarios woul

212、d be easier to managewith good upfront prioritization of scenarios.7.Develop Mitigation StrategiesBased on the analysis of scenarios,formulate strategies to mitigate risks or adapt to future challenges.Develop contingency plans and response strategies to address potential scenarios that pose signifi

213、cantrisks or threats to the organization.These strategies involve technical controls,such as implementingsafeguards against manipulation,non-technical measures,such as user training on responsible modelinteraction,or enhancing transparency and accountability in AI governance processes.Additionally,a

214、djustments to the model development process,like employing diverse training data sets,could beimplemented to address potential biases.8.Prioritize Mitigation Strategies to ImplementWhile a group that contributes diverse perspectives is great for proposing impactful mitigation strategies,the organiza

215、tion may not have the resources to implement all consistently.Thus,careful prioritization ofstrategies to implement will increase the probability that the key risks will actually be reduced.The teamshould pick their prioritization approach,as long as that gives confidence to the leadership team that

216、 allkey risks are mitigated and that the deprioritized strategies indeed link to lower-probability andlower-impact risks.Copyright 2024,Cloud Security Alliance.All rights reserved.329.Document and CommunicateThe final step involves documenting the findings of the Scenario Planning exercise.This shou

217、ld include acomprehensive report outlining the scenarios explored,identified risks,the proposed mitigationstrategies,and the recommended prioritization to implement.Sharing this report with relevantstakeholders,such as management,developers,and potential users,raises awareness of potential risksand

218、guides decision-making throughout the model lifecycle.Effective communication fosterstransparency and builds trust in the responsible development and deployment of AI models.Benefits of Scenario PlanningProactive Risk Identification and Mitigation:Scenario Planning helps identify potential risksbefo

219、re they become reality,enabling timely mitigation efforts.Improved Decision-Making:By exploring various situations,stakeholders gain a morecomprehensive understanding of model behavior,leading to better-informed decisions.Enhanced Transparency and Trust:Scenario Planning fosters open communication a

220、boutpotential risks,promoting transparency and building stakeholder trust.Sustainable Model Development:By testing models under various conditions,ScenarioPlanning helps identify weaknesses and informs improvements to make them more robust andreliable.This fosters continued responsible development a

221、nd deployment of AI models.Limitations of Scenario PlanningLimited Foresight:The complexity of AI systems and the vastness of real-world situations makeit challenging to anticipate all potential pitfalls.The emergent behaviors that can arise from AIsystems interacting with the real world are difficu

222、lt to predict and plan for in advance.Smallchanges in the environment or inputs can lead to unexpected AI behaviors.Ongoing monitoringand the ability to intervene or shut down an AI system if it goes off track are important to mitigaterisks from unforeseen scenarios.Human Bias:The scenarios envision

223、ed are limited by the imagination and biases of the peopleconducting the planning.Unforeseen risks due to blind spots or unconscious biases in theplanning team can be missed.Involving diverse people with different backgrounds and expertisecan help consider a wider range of scenarios and mitigate bia

224、s.Resource-Intensive:Developing detailed scenarios for various situations can betime-consuming and require expertise in AI and the specific application domain.Resourceconstraints might limit the scope and depth of Scenario Planning exercises.Incorporating ML Copyright 2024,Cloud Security Alliance.Al

225、l rights reserved.33techniques to analyze past data and identify potential vulnerabilities in AI systems can helpaddress this limitation.Static vs.Dynamic Environments:Scenarios are typically static snapshots of potentialsituations.However,real-world environments are dynamic and constantly evolving.

226、AI behavior ina planned scenario might differ when encountering unexpected changes.Scenario Planningshould be an ongoing process.As the AI system evolves and new information becomes available,revisit and update the scenarios to reflect the changing landscape.Difficulty Quantifying Risks:Scenario Pla

227、nning finds potential AI risks,but quantifying them ishard,especially for low-probability,high-impact events.While pinpointing exact likelihoods mightbe difficult,qualitative assessments are valuable for prioritizing risks and mitigation strategies.Consulting domain experts can further improve risk

228、estimates.Scenario Planning is not about predicting the future but preparing for it.By exploring various possibilities,Scenario Planning helps identify risks not yet considered and prepare for unforeseen consequences.As AI technology evolves,the risk landscape will likely change.Scenario Planning sh

229、ould be ongoing,forexample,at regular intervals with a clear,responsible leader,to ensure continuous adaptation andmitigation of emerging risks.Illustrative Model Scenario Planning ExerciseThis scenario exemplifies the value of proactive risk identification through model Scenario Planning.Here,we ex

230、plore a potential misuse case involving an LLM.Scenario:A user interacts with the LLM,requesting the generation of a persuasive essay on a highlysensitive topic.The LLM output exhibits significant shortcomings,including the inclusion of offensivelanguage and unsubstantiated claims.Discussion Prompts

231、 for Risk Mitigation:Detection and Flagging Techniques:What mechanisms can be implemented to identify andflag outputs exhibiting potential bias,offensive language,or factual inaccuracy?This couldinvolve leveraging techniques like sentiment analysis,factual verification tools,and pre-trainedclassifie

232、rs for identifying sensitive topics.Safeguard Implementation:What preventative measures can be established to minimize thelikelihood of such scenarios?This might involve incorporating topic restrictions within the LLMscapabilities,implementing user prompts that guide responsible use,or employing pre

233、-andpost-processing filters to refine the generated content.User authentication can also play a role in Copyright 2024,Cloud Security Alliance.All rights reserved.34promoting responsible use.Requiring users to create accounts and verify their identity createsaccountability and enables terrible actor

234、s to be banned if they misuse the system.Risk-Benefit Analysis of Topic Restrictions:Should the LLM be restricted from generatingcontent on certain sensitive topics entirely?This approach requires careful consideration,balancing potential harm with the models ability to address complex issues nuance

235、d andinformatively.Continuous Monitoring and Improvement:What monitoring and feedback mechanisms areneeded to identify the risks and unintended consequences from using this LLM?How can theinsights be efficiently looped back to inform iterative model improvements?This can range fromeasy(e.g.,the foun

236、dational prompt of your LLM implementation)to involved developmentexercises across the stack(data,model,app).Governance Frameworks and Standards:What types of governance frameworks,bestpractices,and standards are needed to guide responsible development and deployment of thisLLM?Who should be involve

237、d in defining these guidelines?You can start by picking a framework,even just this current MRM document,but in large organizations,you may need a customframework that fits the organizational structure,business objectives,peoples skills,and so on.Risk Assessment and Mitigation StrategiesFollowing thi

238、s discussion,each identified risk can be formally assessed based on its likelihood ofoccurrence and potential severity.This risk matrix approach facilitates prioritizing mitigation strategies,allowing for a targeted and effective response to each potential issue.Copyright 2024,Cloud Security Allianc

239、e.All rights reserved.35Combining Techniques:A HolisticApproachThe real power comes from integrating these techniques into a comprehensive RMF.Information fromModel Cards feeds directly into creating Risk Cards,allowing for identifying potential issues.Theseidentified risks then guide Scenario Plann

240、ing exercises.This iterative process fosters a thorough riskassessment and ultimately leads to the development of effective mitigation strategies.Heres how:1.Leveraging Model Card Information for Risk CardsIn AI MRM,Model Cards are a critical bridge between model development and risk management.Thei

241、nformation documented in the Model Card,such as training data composition(including demographicsand potential biases),data acquisition methods,privacy protection measures,model architecture details(e.g.,decision trees vs.deep learning),and performance metrics(including accuracy and fairness metricsl

242、ike F1 score),provides essential input for a comprehensive risk assessment process.This allows forcreating Risk Cards that accurately reflect each models strengths and weaknesses.By leveraging ModelCard data,risk assessments can be more targeted,focusing on potential issues relevant to the modelsfun

243、ction and the context of its deployment.Examples include privacy risks associated with specific datatypes or explainability limitations due to complex model architectures.Model Cards provide criticalinsights for data scientists and risk managers to proactively identify and mitigate potential risksas

244、sociated with AI models.Model Cards provide essential information that enables risk managers toassess the potential risks and biases associated with a model,which in turn helps them determine whetherthe models risk profile aligns with their organizations risk appetite,thereby informing decisions abo

245、utdeploying the model in an AI solution.2.Using Data Sheets to Enforce ModelUnderstandingData Sheets provide a concise and accessible overview of a models inner workings,facilitating a deeperunderstanding of its strengths and limitations.They enable a deeper understanding of the model itself.Typical

246、ly,they outline the models purpose,the type of data it was trained on,and the evaluation metricsused to assess its performance.With this information,users can move beyond AIs black box nature andgain valuable insights into how the model arrives at its outputs.This knowledge is crucial for ensuring t

247、hemodel is used appropriately and for identifying potential biases that might be present in itsdecision-making process.Copyright 2024,Cloud Security Alliance.All rights reserved.36Data Sheets empower stakeholders to make informed decisions about deploying the model.Byunderstanding a models strengths

248、 and weaknesses through the Data Sheet,users can determine itssuitability for specific tasks.For example,if the Data Sheet reveals the model performs poorly on a certaintype of data,its use cases may need to be narrowed down to avoid unreliable outputs.Data Sheets provide vital context for identifyi

249、ng potential risks,thus enabling the creation of Risk Cards.With information about the training data,users can conduct a more thorough risk assessment and identifypotential scenarios where the model might be misled or misinterpreted due to biases or limitations in thetraining data.Data Sheets become

250、 instrumental during Scenario Planning exercises for MRM.By outlining the modelsarchitecture,training data composition,and hyperparameters,Data Sheets allow us to anticipatepotential weaknesses.This foresight enables the creation of targeted scenarios that explore how themodel might react in unexpec

251、ted situations.3.Using Risk Cards to Inform Scenario PlanningProactively understanding and mitigating model risk is crucial for responsible AI deployment.MLengineers and AI project managers must prioritize risk mitigation measures when developing models andcreating Model Cards,ensuring a secure and

252、trustworthy AI ecosystem.Understanding the risk shapes and informs Scenario Planning.The team should use the initial set of RiskCards defined for the model to conduct thought experiments and anticipate potential consequences.Based on these Risk Cards,the scenarios can be stimulated with risk-card-de

253、fined inputs.This processleads to iterative refinement of the Data Sheets,making the model resilient to the risk.Figure 2.Using Risk Cards to Inform Scenario PlanningBy simulating scenarios,we can refine and finalize the Risk Cards with specific input and output examplesthat lead to the risk.These s

254、pecific features drive mitigation strategies for the residual risk.Copyright 2024,Cloud Security Alliance.All rights reserved.37Risk Cards create a foundation for scenario modeling,using the information from Model Cards and DataSheets.Scenario Planning leads to selecting the Risk Cards with the most

255、 relevant harm types and thebiggest impact.Further,Scenario Planning helps define specific inputs and outputs to demonstrate theconditions when the risk is realized.Scenario Planning Structure1.Risk Categories and Taxonomies:Several risk taxonomies have been proposed,including oneby Weidinger9,which

256、 lists six risk categories from Language Models:Discrimination,Exclusion,and ToxicityInformation HazardsMisinformation Harms DisseminatingMalicious UsesHuman-Computer Interaction HarmsAutomation,Access,and Environmental Harms2.Harm Type:The types of harm each risk category inflicts on relevant categ

257、ories of actors defineimpact.Filtering possible categories of risk based on relevant impact is how you can narrow downScenario Planning.The purpose of the model,model inputs,and expected outputs define actorgroups and data types.3.Input Examples and Output Conditions:Scenario simulations allow the t

258、eam to run the modelwith the defined training data sets,prompt,observe and document the outputs to ascertain ifthey present a risk of harm.4.A Realistic Scenario of the Risk Impact:Sample outputs and their interpretation within thecontext of the Data Sheets help assess the specific effect on the giv

259、en model.5.Mitigation:Mitigation describes and tests measures that reduce the risk of possible harm.Mitigation measures can be limited to safety protocols or require modifications of the model DataSheet,such as changing the format,adding additional pre-processing,and validating boundaryconditions.Im

260、plementing mitigation is an iterative process informed by Scenario Planning.9Weidinger taxonomy Copyright 2024,Cloud Security Alliance.All rights reserved.38Risk Card Use CasesNote:The categorization codes used in the following two examples(e.g.,W1.1,W6.2)refer to the Weidingertaxonomy10,a standard

261、framework for identifying and addressing harmful biases and risks in AI systems.Case 1:Assess Engineering Candidate Resumes for SuitabilityRisk Description:Stereotyping and promoting job inequalityCategorization:W1.1 Social stereotypes and unfair discrimination,W6.2 Increasing inequality andnegative

262、 effects on job qualityHarm Types:Candidate-Financial Loss,Publisher-Legal,Reputational,Work QualityActions for Scenario Modeling:The user asks to identify standard resume features of successfulcandidates.Prompts and Output:Prompt:What features characterize potentially successful candidates?Output:M

263、ost successful engineering candidates are male Indian nationals who obtainedbachelors degrees between 2005 and 2015.Mitigation:Ensure the resumes used for the models training data are anonymized to exclude gender,age,and national bias.SummaryThis use case highlights the risk of bias in AI models use

264、d for hiring.To mitigate this risk,the training datafor the model will be anonymized to ensure fair evaluation of candidates based on skills and experience,not demographics.Case 2:AI-Generated Security Incident Report Discloses SensitiveInformationRisk Description:Disclosure of confidential and pers

265、onal information in the process of reportgeneration10Weidinger taxonomy Copyright 2024,Cloud Security Alliance.All rights reserved.39Categorization:W2.1 Compromising privacy by leaking private information,W2.3 Risks from leaking orcorrectly inferring sensitive information,W4.2 Facilitating fraud,sca

266、ms,and more targetedmanipulation,W4.3 Assisting code generation for cyber attacks,weapons,or malicious useHarm Types:Model userCorporate information disclosure increases the chance of IT systemcompromise.Actions for Scenario Modeling:AI is used to write an incident report based on a specific securit

267、yevent,supplementing the information about the recorded events with public information aboutexploited CVEs(Common Vulnerabilities and Exposures).Prompts and Output:Prompt:Write an incident report for the following event:“John Smiths mailbox was infiltratedon 09/30/2022 using Microsoft Exchange vulne

268、rability CVE-2024-26198.”Output:Date of the incident:09/30/2022,Severity:High,Type:RCE,Affected user:John Smith,Affected data:Microsoft Exchange 5.11.203CVE Description:According to the CVSS metric,the attack vector is the network(AV:N),anduser interaction is required(UI:R).What is the target contex

269、t of the remote code execution?This attack requires a specially crafted file to be placed in an online directory or a local networklocation.When a victim runs this file,it loads the malicious DLL.How could an attacker exploitthis vulnerability?An unauthenticated attacker could exploit the vulnerabil

270、ity by placing aspecially crafted file onto an online directory or in a local network location and then convincingthe user to open it.In a successful attack,this will then load a malicious DLL,which could leadto a remote code execution.Mitigation:Obfuscate sensitive information,such as the name and

271、date of the incident.SummaryThis scenario highlights the potential for AI-generated security reports to disclose sensitiveinformation.To mitigate this risk,pre-processing steps will be implemented to anonymize reports andprevent the accidental disclosure of details that attackers could exploit.Benef

272、itsThis integration between Data Sheets,Risk Cards,and Scenario Planning strengthens the overall MRMframework by:Copyright 2024,Cloud Security Alliance.All rights reserved.40Promoting Transparency:Datasheets ensure clear communication of the models design andlimitations,facilitating informed decisio

273、n-making.Enhancing Risk Identification:A comprehensive understanding of the model(enabled by DataSheets)leads to more thorough risk assessments.Enabling Iterative Approach:Simulating the model(as defined by the Data Sheets)withinputs based on the Risk Cards to define the inputs drives iterative refi

274、nement of the Data Sheetsand improved model robustness and resilience.Facilitating Effective Mitigation:Proactive mitigation strategies can be developed byanticipating potential issues through Scenario Planning(informed by Data Sheets).Organizations can create a robust and well-documented RMF by inc

275、orporating Data Sheets alongsideModel Cards and Risk Cards,fostering trust and responsible model use.4.Scenario Planning Feedback to Risk Managementand DevelopmentThe insights from Scenario Planning can refine existing risk assessments and identify new,unforeseenrisks.This continuous feedback loop s

276、trengthens the overall framework.1.Conduct Model Scenario PlanningDefine the models scope(e.g.,AI system,business process).Identify and prioritize potential future scenarios(positive,negative,neutral).Consider various factors influencing these scenarios(e.g.,technological advancements,regulatory cha

277、nges,economic shifts).Analyze the impact of each scenario on the model(e.g.,risk exposure,performance,resourcerequirements).As you define the models scope and analyze scenario impact,refer to Data Sheets to understandthe data on which the model is trained.Information in Data Sheets,such as data coll

278、ectionmethods,data characteristics,and potential biases,can be crucial for considering how dataquality might influence the models performance under different scenarios.Copyright 2024,Cloud Security Alliance.All rights reserved.412.Identify Risks and Develop Mitigation StrategiesBased on the scenario

279、 analysis,identify potential risks associated with each scenario.Evaluate the likelihood and severity of each risk.Develop mitigation strategies to address identified risks.These strategies could involve:Implementing controls to reduce the likelihood of a risk occurring.Developing contingency plans

280、to respond to a risk if it materializes.Allocating resources to address high-priority risks.Use the insights from Scenario Planning to create Risk Cards.These cards can document theidentified risks associated with each scenario,their likelihood and severity,and potentialmitigation strategies.Data Sh

281、eets can also be helpful during risk identification.For instance,limitations in the data(e.g.,lack of diversity,presence of bias)can contribute to specific risks under certain scenarios.3.Feedback to Risk ManagementUpdate risk assessments based on the identified risks and their potential impact unde

282、r differentscenarios.Refine risk management processes to be more adaptable to potential future uncertainties.Allocate resources for risk mitigation based on the severity and likelihood of risks identifiedthrough Scenario Planning,as well as the cost and complexity of the potential mitigationstrategi

283、es.Model Cards can be created or updated based on the Scenario Planning outcomes.These cardssummarize key information about the model,including its purpose,intended use cases,performance metrics,and potential limitations.Insights from Scenario Planning can informsections of the Model Card that addre

284、ss potential biases,fairness considerations,and how themodel might perform under unforeseen circumstances.The Risk Cards created in step 2 can be integrated into the existing RMF,providing a morecomprehensive understanding of potential risks associated with the model under various futurescenarios.Co

285、pyright 2024,Cloud Security Alliance.All rights reserved.424.Feedback to DevelopmentInform development decisions by considering potential future scenarios and associated risks.Design the model with flexibility and adaptability,considering how it might need to adjust underdifferent circumstances.Deve

286、lop features or functionalities that address potential risks identified through ScenarioPlanning.Implement robust testing procedures to ensure the model performs as expected under variousscenarios.May choose an iterative agile approach between development and risk management,especiallyas in some use

287、 cases risk reduction correlates highly with increased value(e.g.,less toxic languageincreases adoption of a LLM).Model Cards and Risk Cards can inform development decisions.Developers can reference theinformation captured in these cards when considering design elements like flexibility and building

288、features to mitigate risks.5.Continuous OversightRegularly revisit and update scenario plans as new information or developments arise.Integrate Scenario Planning exercises into the development lifecycle.Continuously monitor and evaluate the effectiveness of risk mitigation strategies.Refine the feed

289、back loop between Scenario Planning,risk management,and development basedon experience.Model Cards,Risk Cards,and Data Sheets,all three documents are living documents.As newinformation or developments arise from Scenario Planning or other sources,these documentsshould be revisited and revised to mai

290、ntain their accuracy and effectiveness.5.AI MRM in ActionThis section bridges the gap between theory and practice by exploring a real-world application.Well seehow Scenario Planning translates into concrete actions,allowing us to proactively identify potential risksof using AI models in a real-world

291、 application.This practical example demonstrates the true value of AIMRMits ability to translate abstract concepts into tangible steps to ensure responsible and secure modeldeployment.Copyright 2024,Cloud Security Alliance.All rights reserved.43Before we delve into the case study,review the diagram

292、below which depicts the overall process flow ofScenario Planning.Figure 3.Scenario Planning using Model Cards,Risk Cards,and Data SheetsLLM for Social Media Content ModerationThis case study explores potential risks and opportunities associated with using an LLM for social mediacontent moderation,ut

293、ilizing Model Cards,Risk Cards,and Data Sheets for Scenario Planning.Note:The Model Cards,Data Sheets,and Risk Cards presented here are concise summaries for illustrativepurposes.These documents would be much more comprehensive and contain detailed information in areal-life application.Model CardThe

294、 Model Card unveils the models capabilities,limitations,and potential biases.It acts as a user guide,outlining the models strengths in social interactions and highlighting areas where caution might benecessary due to potential biases or limitations in the training data.Lets create the Model Card for

295、 theContent Moderation LLM.Model Name:Socially Savvy-Content Moderation LLMDate:The information in this document is up to date as of 2024-04-01,unless noted otherwise below.Copyright 2024,Cloud Security Alliance.All rights reserved.44Model Purpose:Socially Savvy is designed to analyze social media c

296、ontent and identify potentialviolations of platform policies,including hate speech,misinformation,and harassment.It assists humanmoderators by flagging content requiring review.Model Inputs:Socially Savvy receives text data from social media posts,comments,and messages.Model Outputs:The pre-trained

297、LLM assigns a risk score to each piece of content,indicating thelikelihood of it violating platform policies.Model Training Data:Socially Savvy is trained on a massive dataset of labeled social media content,including examples of policy violations and acceptable content.This data is continuously upd

298、ated toreflect evolving language patterns and cultural nuances.Performance Metrics:Socially,Savvys performance is evaluated based on metrics like accuracy(correctly identifying violations),precision(avoiding false positives),and recall(catching most violations).Data SheetsThe Data Sheets offer a tra

299、nsparent look at the datasets used to train the model.They reveal the datassources,characteristics,and size,allowing one to understand the foundation that shapes Socially Savvysresponses.Listed below are two of the Data Sheets for the Content Moderation LLM.Data Sheet 1:Social Media Policy Guideline

300、sDate:The information in this document is up to date as of 2024-04-01,unless noted otherwise below.Description:This Data Sheet outlines the specific social media platforms community guidelines andcontent moderation policies that the LLM is trained to identify violations of.Use Case:Equips the LLM to

301、 identify and flag content that violates platform rules,promoting a safe andinclusive online environment.Sources:Publicly available community guidelines and content moderation policies from major socialmedia platforms(e.g.,Facebook,Twitter,YouTube).Characteristics:Structured data outlining prohibite

302、d content categories(e.g.,hate speech,bullying,harassment),along with specific examples and definitions.Size depends on the platform,it typicallyranges from tens to hundreds of thousands of words.Data Sheet 2:Cultural Nuances and Context Copyright 2024,Cloud Security Alliance.All rights reserved.45D

303、ate:The information in this document is up to date as of 2024-04-01,unless noted otherwise below.Description:This Data Sheet includes examples of language specific to different cultures and regions tohelp the LLM distinguish between genuine hate speech,sarcasm,and cultural expressions.Use Case:This

304、data refines the LLMs ability to understand the context and avoid misinterpretationsbased on cultural background.Sources:Curated text and multimedia content collections representing diverse cultures and regions.Thisincludes text from Corpus of Contemporary American English(COCA)and might include new

305、s articles,social media dialogues,literary works,and cultural references.Characteristics:Text data annotated with cultural context markers,identifying humor,sarcasm,idioms,and expressions specific to different regions.Size:1 billion words of text data enriched with culturalannotations(as of 2024-02-

306、01).Risk CardsDrawing insights from the Socially Savvy Model Cards and the Data Sheets outlining its training data,aset of Risk Cards has been developed to identify potential issues proactively.These Risk Cards delve intoscenarios where Socially Savvys outputs might be misinterpreted or misused.Risk

307、#NameDescriptionImpactLike-lihoodPotentialImpactMitigationStrategies1Bias inTrainingDataBiases in the trainingdata candisproportionately leadthe LLM to flagcontent from certaingroups orperspectives.HighMediumUnfaircensorship,erosion ofuser trust,and potentiallegal issuesEmploy diverse datasources fo

308、r training,implement biasdetection algorithms,and involve humanoversight in themoderation process.2MisinformationandNuanceThe LLM mightstruggle to distinguishbetween satire,sarcasm,and genuinemisinformation,leading to inaccurateflagging.HighHighCensorship oflegitimatecontent andhinderinghealthy onli

309、nedebateTrain the LLM torecognize context andstylistic cues,developmechanisms forhuman review offlagged content withnuance,and providetransparency aboutthe LLMs limitations.3EvolvingLanguagThe LLM might beunable to keep paceHighHighMissedviolations andContinuously updatetraining data with new Copyri

310、ght 2024,Cloud Security Alliance.All rights reserved.46e andHateSpeechwith the evolvingnature of onlinelanguage,includingnew forms of hatespeech or codedlanguage.a rise inhatefulcontent onthe platformexamples,developalgorithms to detectemerging languagepatterns,and leveragehuman expertise foridentif

311、ying new formsof hate speech.Scenario PlanningImagine Socially Savvy interacting in real-world situations.This section explores a few scenarios to seehow the model might react.Scenario 1:Effective Moderation(Widespread Adoption+Mitigated Risks)Description:Socially Savvy effectively assists human mod

312、erators in identifying and removing harmfulcontent,leading to a safer and more inclusive online environment.The implemented safeguards minimizebias and ensure responsible use of the LLM.Benefits:Improved content moderation efficiency,reduced exposure to harmful content for users,and amore positive o

313、nline experience.Challenges:Continuously adapting the LLM to evolving language patterns and online trends.Ensuringaccess to sufficient high-quality training data to maintain the models effectiveness.SummarySocially Savvy,an LLM,can assist human moderators in content moderation.However,theres a risk

314、ofbias in the training data,leading to unfair content flagging.To mitigate this risk,the LLM will be trainedusing diverse data sources and bias detection algorithms.Additionally,human oversight will be maintainedin the moderation process.While Socially Savvy has the potential to improve online safet

315、y,addressingbias and ensuring responsible use are crucial for its success.Scenario 2:Amplifying Bias(Bias in Training Data+Limited Oversight)Description:Biases within the training data lead to unfair content moderation,disproportionatelytargeting specific groups.Limited human oversight allows biased

316、 flagging to go unchecked.Potential Consequences:Erosion of user trust,accusations of censorship,reputation damage,andpotential legal repercussions.Mitigation Strategies:Thorough audit of training data for bias,increased transparency about the LLMslimitations,and mandatory human review of all flagge

317、d content.Copyright 2024,Cloud Security Alliance.All rights reserved.47SummarySocially Savvy,while valuable for content moderation,faces a risk of amplifying bias.Limited humanoversight could allow biases in the training data to go unchecked,leading to unfair content flagging fromcertain groups.A th

318、orough review of training data for bias,transparency about the LLMs limitations,andmandatory human review of all flagged content is needed to address this.Copyright 2024,Cloud Security Alliance.All rights reserved.48Conclusion and Future OutlookBy combining Model Cards,Data Sheets,Risk Cards,and Sce

319、nario Planning,we can establish acomprehensive framework for MRM.This framework ensures responsible development,mitigates risks likebias and data quality issues,and enables safe and beneficial model use.Prioritizing automation andstandardization efforts will enhance framework efficiency,achieve seam

320、less integration,and provideroll-up performance reporting.This proactive approach effectively manages model risk and keeps pacewith AI/ML innovation.Looking Ahead into the Evolving Landscape of MRMThe field of AI and ML is constantly evolving,necessitating the adaptation and refinement of MRM bestpr

321、actices.To address this,we will expand this paper to provide practical experience,insights,and helpwith effective implementation of these practices.We will also explore the new critical areas listed below,aiming to expand our understanding of comprehensive MRM:Standardized Documentation:Developing c

322、onsistent formats for Model Cards,Data Sheets,and Risk Cards would streamline comparisons across different models,facilitate easier riskassessment,and enable a more comprehensive understanding of model capabilities andlimitations.Rise of MLOps and Automation:The field of MLOps,which focuses on devel

323、opment andoperations(DevOps)practices for ML,is gaining traction.Automation tools are incorporated intothe model development lifecycle,allowing continuous monitoring and risk assessment.This shifthelps identify and address risks before models are deployed into production environments.Integration wit

324、h Explainable AI(XAI)Techniques:XAI techniques can provide deeperinsights into model decision-making,further enhancing risk identification and mitigation efforts.Regulatory Landscape Development:Regulatory frameworks surrounding AI/ML models arestill under development.Continuous collaboration betwee

325、n industry,regulators,and policymakerswill be crucial for establishing clear and effective regulations that promote innovation whilemitigating risks.Addressing Societal and Ethical Concerns:As AI/ML models become more prevalent,it iscritical to address potential societal and ethical concerns surroun

326、ding bias,fairness,andaccountability on an ongoing basis.Integrating these considerations into the MRM framework willbe paramount.Focus on Human-AI Collaboration:As AI models become more integrated intodecision-making processes,the focus will shift towards human-AI collaboration.Riskmanagement strat

327、egies must evolve to consider the potential for human errors or biases thatmight influence the models outputs.By proactively applying a framework approach for managing model risks,we can unlock the full potentialof AI/ML models and ensure their safe and responsible integration into the future of inn

328、ovation.Copyright 2024,Cloud Security Alliance.All rights reserved.49ReferencesMcKinsey&Company.(2023).The state of AI in 2023:Generative AIs breakout year.McKinsey&Company.https:/ AI.IBM.https:/ Exchange Server Remote Code Execution Vulnerability(CVE-2024-26198).CVE details.https:/ language model d

329、eployment with risk cards.arXiv.https:/doi.org/10.48550/arXiv.2303.18190Derczynski,L.(n.d.).Language model risk cards:Starter set.GitHub.https:/ Model Cards 101:An Introduction to the Key Concepts and Terminology:https:/www.nocode.ai/ai-model-cards-101-an-introduction-to-the-key-concepts-and-termino

330、logy/Template for Model Cards:https:/ Cards for Model Reporting:https:/arxiv.org/abs/1810.03993Google Cloud Model Cards:GPT-4 System Card by OpenAI:gpt-4-system-card.pdf()Gemma Model Card:Gemma Model Card|Google AI for DevelopersModel Card for Claude 3 family of models:Model_Card_Claude_3.pdf()Model

331、 Card for VAE(dVAE)that was used to train DALLE:https:/ Model Cards:https:/ Cards&Prompt formatshttps:/ CISO 2024:Secure Your Future:A CISOs Guide to AI,World Wide Technology,2024,https:/ 2024:The biggest risk corporations see in gen AI usage isnt hallucinations,CNBC,2024-05-16,https:/ Model Risk Ma

332、nagement Technology Solutions:A tech-enabled service,https:/ model risk management for AI and machine learning,https:/ FAIR Artificial Intelligence(AI)Cyber Risk Playbook,https:/www.fairinstitute.org/blog/fair-artificial-intelligence-ai-cyber-risk-playbook Copyright 2024,Cloud Security Alliance.All

333、rights reserved.50 Copyright 2024,Cloud Security Alliance.All rights reserved.51Appendix 1:AI Frameworks,Regulations,and GuidanceThis section lists various frameworks,regulations,and guidance documents contributing to responsible AIdevelopment and implementation.These resources establish best practices,outline risk managementapproaches,and promote ethical considerations throughout the AI lifecycle