關于人工智能的黑色思考-當人工智能遇到安全.pdf

1、關于人工智能的黑色思考-當人工智能遇到安全美國喬治亞大學教授喬治亞網絡安全與隱私學院院長安全智能從安全的角度審視機器學習智能安全安全對抗中的人工智能和挑戰思考一：AI與機器學習的成功Thought 1:The Success of AI人工智能深度學習的成功應用利用機器學習刷帖刷評價“Leverage deep learning language models(Recurrent Neural Networks or RNNs)to automate the generation of fake online reviews for products and serviceshttps:/ 2

2、:The Opposite of AIArtificial Intelligence（人工智能）Natural Stupidity*（天然傻缺）The Opposite of Artificial Intelligence*The term was initially used by Drew McDermott in his 1976 paper“Artificial Intelligence meets Natural Stupidity”,http:/dl.acm.org/citation.cfm?id=1045340Image Recognition:Flagship of AI Ap

3、plicationshttps:/www.cs.toronto.edu/ranzato/publications/taigman_cvpr14.pdf深度學習對手寫識別的應用(MNIST dataset）http:/ dataset）利用模型分類0輸入http:/ dataset）Error Rate:0.4%http:/ IntelligenceNatural Stupidity關于識別結果的問題可能的結果：0 1,2,9 App hangs App gets owned Not sure 利用模型分類輸入FileAPP思考二:AI 敵不過 Natural Stupidity最近發現的相關漏

4、洞（by 360 Team）OpenCVCVE-2017-12597CVE-2017-12598CVE-2017-12599CVE-2017-12600CVE-2017-12601CVE-2017-12602CVE-2017-12603CVE-2017-12604CVE-2017-12605CVE-2017-12606NumpyCVE-2017-12852OpenEXRCVE-2017-12596LibjasperCVE-2017-9782 人工智能應用實現細節 Complexity and Dependency Stupidity lead to Software Vulnerability

5、 詳細漏洞實例“基于深度學習的智能系統中的安全漏洞及影響”，肖奇學The Devil is in the Detail思考三：從安全的角度考慮準確率Thought 3:The Probability GameThe Probability GameMNIST(NYU/Google Labs)Accuracy:99.6%DeepFace(Facebook)Accuracy:97.25%The Probability Game從人工智能的角度看？MNIST(NYU/Google Labs)Accuracy:99.6%DeepFace(Facebook)Accuracy:97.25%Great Ac

6、hievementHuman 97.5%The Probability Game從安全的角度看？MNIST(NYU/Google Labs)Accuracy:99.6%DeepFace(Facebook)Accuracy:97.25%Error rate 0.4%100%Adversarial LearningAdversarial Learning系統化穩定生成對抗樣本 白盒/灰盒對抗手段 對抗神經元網絡與對抗樣本生成 許偉林，“對抗式機器學習中的攻擊與防御”黑盒對抗手段利用漏洞挖掘的方法自動生成對抗樣本 肖奇學,“基于深度學習的智能系統中的安全漏洞及影響”How to turn 0.4%t

7、o 100%?Thought 4：The Risk of Data Poisoning 思考四：數據污染攻擊機器學習對訓練數據的依賴性Garbage In,Garbage Out How purported great ML models can be screwed up by bad databy Hillary Sanders(SOPHOS)Data Poisoninghttps:/ Poisoning Example數據污染的先例？Spam War and SEOData Poisoning數據遷移學習 陳樹華,“基于知識共享的新一代風控體系”學習階段的數據保護思考五：安全對抗中的智能

8、和挑戰Thought 5:Beyond Deep Learning攻防對抗中的工具自動漏洞挖掘自動漏洞利用攻擊重放Beyond Deep Learning攻防對抗中的策略攻擊目標選擇后門注入與排除對手資源與策略預測From Machine Learning to Creating Machine 從 機器自動學習 到 自動構造機器Intelligence in Security安全工具中的智能攻擊/Exploitation:通過輸入 對程序進行巧妙的非功能性組合Creative composition of unintended functions攻擊組件的有機結合實例Composition

9、in Exploitation:glibc-chaininghttps:/ Gadget etc.）3.拼接部分或完整的圖靈機ROP，JOP，DOP關于人工智能與安全的黑暗思考1.深度學習的能力已受到黑產的肯定2.再強大的人工智能也敵不過軟件實現者的疏漏3.自然誤判率雖小，但被攻擊者系統放大，就變成穩定的逃逸手段4.隨著人工智能應用推廣，數據污染攻擊接踵而至更廣義的智能安全1.不僅僅是惡意樣本和行為分類，2.還有安全攻防中的決策與博弈：目標選擇，資源調度，攻擊重放及復用，3.終極的漏洞利用是自動的圖靈機構造和程序生成總結When AI Meets Security 謝謝Thought:The

10、Unknown Features思考？：The Unknown FeaturesHistorical Neural Network Follieshttps:/neil.fraser.name/writing/tank/(written in year 2003)Unconfirmed Story about An Early Effort(1980s)of Classifying Objects Using Neural Network Advantage of Deep LearningData driven network with experience,design by trail

11、and errorNo need to know whyChallengesStatistically impressive but individually unreliableMore uncertain features=more attack pointsConsequence of Rich FeaturesMore feature does not lead to better securityAVPASS:Inferring features and detection rules of AVsMore is Lesshttps:/ Machine Learning&Automatically Evading Classifier”Less is More