7-2 開放環境下圖神經網絡及其應用.pdf

《7-2 開放環境下圖神經網絡及其應用.pdf》由會員分享，可在線閱讀，更多相關《7-2 開放環境下圖神經網絡及其應用.pdf（45頁珍藏版）》請在三個皮匠報告上搜索。

1、石川 教授北京郵電大學開放環境下圖神經開放環境下圖神經網絡與應用網絡與應用 2009BUPT TSEG 2網網絡建模絡建模絡是描述和建模復雜系統的通語2融絡社交絡神經元絡信息絡物絡互聯ABC3網網絡表示學習絡表示學習絡表學習成嵌將節點嵌到低維向量空間中應節點分類鏈路預測社區發現絡演化p 易于并p 可結合經典機器學習法4淺淺層模型層模型淺層模型 基于分解的法 e.g.,Laplacian eigenmaps 基于隨機游的法 e.g.,DeepWalk,node2vec5深深層模型層模型5深層模型應深層神經絡基于動編碼器的法 e.g.,DNGR and SDNE基于圖神經絡的法 聚合鄰居信息，并應

2、神經絡 e.g.,GCN,GraphSage,GAT6圖神經網絡圖神經網絡3.迭代 次:#(%)*#%+*+-/%+*?/#,4.預測:7=softmax(B+*)1.輸圖 和節點屬性 2.初始化:#(E)=E#,7開放開放環境下的圖神經網絡環境下的圖神經網絡異質性多種類型節點和邊共存動態性圖結構和屬性動態演化稀疏性可見的交互和屬性稀疏脆弱性圖結構和屬性易受攻擊簡單靜態圖難以建模開放環境中的復雜系統8報告內容報告內容 開放環境下圖神經絡異質圖神經絡 HAN(HAN2019)動態圖神經絡 MetaDyGNN(WSDM2022)稀疏圖神經絡 HeCo(KDD2021)對抗圖神經絡 RoHe(AAA

3、I2022)l應用9HeterogeneousGraphlHeterogeneous Graph(HG,Heterogeneous Information Network)contain multiple object types and/or multiple link types.Bibliographic dataMovie dataSocial network dataKnowledge graph10Basic Concepts in HGlNetwork schemaMeta-level description of a networklMeta path(Sun VLDB2011

4、)A relation sequences connecting object pairsContain rich semanticsYizhou Sun,Jiawei Han,Xifeng Yan,Philip S.Yu,Tianyi Wu.PathSim:Meta Path-Based Top-k SimilaritySearch in Heterogeneous Information Networks.VLDBpp.992-1003,2011.11Essence of HeterogeneousGraphlA modeling paradigmlA data form=,=,Stati

5、c,Topology,Homogeneous GraphDynamic,Attribute,Heterogeneous GraphGraphSimple,TopologyKnowledge GraphComplex,KnowledgeHeterogeneous GraphControllable complex,Rich semantics12HG RepresentationWhy HG representation Heterogeneity is ubiquitous Information loss Rich semanticsChallenges How to handle hete

6、rogeneity How to fuse information How to capture rich semanticsStatic,TopologyHomogeneous GraphDynamic,AttributedHeterogeneous Graph13Heterogeneous information network Embedding for Recommendation(HERec)Framework of HERecChuan Shi,Binbin Hu,Wayne Xin Zhao,Philip S.Yu.Heterogeneous Information Networ

7、k Embedding for Recommendation.TKDE2018(Google Citation 559)14HAN FrameworkHeterogeneous GraphAttention Network(HAN)Xiao Wang,Houye Ji,Chuan Shi,Bai Wang,Peng Cui,Philip S.Yu,Yanfang Ye.Heterogeneous Graph Attention Network.WWW 2019.(Google Citation 835)15Dynamic graphDynamic graphDynamic graphs are

8、 ubiquitous.Topology and featureevolve.Interactions for new nodes are sparse.ChallengesHow to handle dynamic?How to model evolution?How to alleviate sparsity?16Motivation of MetaDyGNNCheng Yang,Chunchen Wang,Yuanfu Lu,Xumeng Gong,Chuan Shi,Wei Wang,Xu Zhan.Few-shot LinkPrediction in Dynamic Networks

9、.WSDM 2022.Link prediction of newly emerging nodes in dynamic networkslIt is an important task to alleviate the sparsity for new nodes.lNew nodes in dynamic networks have only a few links at their arrivals.lExisted dynamic GNN models are not specialized for few-shot scenarios.How to combine dynamic

10、GNN and Meta-learning?Dynamic NetworkMAMLMeta-learninglMeta-learning can extract general knowledge across different training tasks and quickly adapt it to few-shot testing tasks.17Challenges:C1:How to extract general knowledge of dynamic links via meta-learning?lKnowledge of a specific node formaliz

11、e each task of a single nodelKnowledge shared across nodes hierarchically adaptive meta-learnerC2:How to tailor dynamic GNNs in meta-learning settings?lLightweight dynamic GNN moduleChallenges18MetaDyGNN Framework19Effectiveness ExperimentsMain Experiment/Link PredictionFew-shot Link PredictionMetaD

12、yGNN achieves the best performance both on link prediction andfew-shot link prediction.20Sparse graphSparse GraphSparse graphs are ubiquitous.Edges between nodes are sparse，even noise.Little information is available,including fewlabels.ChallengesHow to handle sparsity?How to complete topology?How to

13、 make better use of structure information?21Motivation of HeCoXiao Wang,Nian Liu,Hui Han,Chuan Shi.Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning.KDD 2021.l Most HGNNs belong to the semi-supervised settings,while its veryexpensive to obtain enough labels.l A promisin

14、g solution:Self-supervised Learningl Spontaneously find supervised signals from the data itself.l Typical supervised learning:Contrastive learningl Push forward positives,push away negatives.l It has been widely used in CV and NLP,but not in HG.How to conduct heterogeneous contrastive learning on a

15、HG?SimCLRMoCo22ChallengeslThree fundamental challengeslC1:How to design a heterogeneous contrastive mechanismcomplex structures,multiple views cross view,view-invariant factors lC2:How to select proper views in a HG cover both of the local and high-order structure lC3:How to set a difficult contrast

16、ive tasktoo similar views too weak signalsinformation diversity or harder negative samplesnetwork schemameta-path23HeCoSelf-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning(HeCo)24ExperimentslNode Classification25ExperimentslNode clusteringlModel extensions26Fragile GraphFr

17、agility of GraphGraph is not stable,easily changing structureand features.Graph topology and node feature are vulnerable to adversarial attacks.Graph may propagate adversarial attacks along topology.ChallengesHow to analyze the fragility of graphs and algorithms?How to overcome the adversarial risk?

18、How to alleviate cascade effect?27Motivation of RoHeUnique Metapath-based AggregationHGNNs are HighlyFragile in ExperimentMengmei Zhang,Xiao Wang,Meiqi Zhu,Chuan Shi,Zhiqiang Zhang,Jun Zhou.Robust Heterogeneous Graph Neural Networks against Adversarial Attacks.AAAI 202228RobustnessAnalysisKey reason

19、s of vulnerabilityl Soft Attention Mechanism166166l Perturbation Enlargement Effect GCN HGNNs131641312EncodeTransitingProbabilityFail to remove unreliable neighborsFail to EncodeTransitingProbability29The Proposed RoHeRobust Heterogeneous GNN Framework(RoHe)TransitingPrior Module(For Perturbation En

20、largement)Neighbor Purifier(For Soft Attention)30Experiments31報告內容報告內容l開放環境下圖神經絡異質圖神經絡 HAN(HAN2019)動態圖神經絡 MetaDyGNN(WSDM2022)稀疏圖神經絡 HeCo(KDD2021)對抗圖神經絡 RoHe(AAAI2022)應Prohibited Item Detection on Heterogeneous Risk Graphs(CIKM 2021)Prohibited Item Detection via Graph Structure Learning(WWW2022)32Gra

21、ph-based Prohibited Item DetectionProhibited Item Detectionpersonal riskinferior goodsplatform riskhuge finessocial riskserious crimesPornographyGunsWildlifeIllegal drugsMore than 1.35 million listings ofwildlife were removed in Taobaoin 2019More than 1 million fake medicine for COVID-19 were remove

22、d in Amazon in 2020Prohibited itemsProhibited Item DetectionChallengesSuffer from adversarial attributes of prohibited itemsNeglect the abundant structure informationLack of enough manual labels33Heterogeneous Risk GraphYugang Ji,Chuan Shi,Xiao Wang.Prohibited Item Detection on Heterogeneous Risk Gr

23、aphsC.CIKM 2021.Heterogeneous Risk GraphIDRelationDescriptionr1Same selleritems belong to the same sellerr2Same visitoritems have been visited by same visitorsr3Relevant selleritems of relevant sellers to overcome the multiple fake identifications of adversarial sellers.Risk RelationsRich structural

24、informationwith few labels34Overall framework of HSPDFeature EngineeringExtract Risk-Relation HRG ConstructionItem LogsHeterogeneous Self-Supervised LearningDirected Pairwise Self-TrainingRank Prohibited ItemsDetection Report35ExperimentsDatasetsBaselinesu LR&GBDTu?GraphSAGE&MTL u?GATNE&HAN&HGTu?Eff

25、ectiveness analysisu Model analysisu Ablation studyu Case studyu Online testingTasks36Experiments37GSL-based Prohibited Item DetectionHeterogeneous Risk Graph ConstructionLimitations of existing GNNsRequire high-quality structures for message passingLabel homophilyFeature smoothness38Framework of RG

26、SLYugang Ji,Guanyi Chu,Xiao Wang,Chuan Shi,et al.Prohibited Item Detection via Risk Graph Structure Learning.TheWeb Conference 2022(WWW2022).Risk Graph ConstructionHeterogeneous Structure LearningIterative Structure Optimization39Heterogeneous Structure Learningheterogeneous similarity measurehetero

27、geneous structure filterheterogeneous message passing!heterogeneous similarity measure()heterogeneousmessage passing#!#$#%#!#$#&#(threshold&structure graph(#!#%#$#&#%#&(#)#*#+#attribute graph!#heterogeneous structure learning module40Iterative Pairwise Metric LearningPairwise Metric Learning Iterati

28、ve Optimizationpairwise labeling!#!$labeled nodes!#!$!%!%pairwise labels(a)risk graphpairwise labeling632517632517!#!heterogeneous similarity measure()heterogeneousmessage passing#!#$#%#!#$#&#(CONCAT()!threshold()!)labeled nodes(Eq.(6)and(7)Eq.(2)-(4)(c)pairwise metric learning$(b)heterogeneous stru

29、cture learninggenerate t+1thembedding-based graph with Eq.(10)attributesstructures)!)$(#!#%#$#&#&(#*#+#,#%)$41ExperimentsDatasetsBaselinesuLR&MLPuGraphSAGE&GAT uGRCN&GAUGuHAN&HGTTasksuPerformance EvaluationuVariantAnalysisuParameter AnalysisuOnline Testing42Experiments43ConclusionslOpen environment

30、introduce critical challengesHeterogeneityDynamicSparsityFragilitylMaking embedding reliableFairness or debias,e.g.,age and genderRobustExplainable,e.g.,disentangled learninglReal-world applicationsSoftware engineeringBiological systemLarge-scale industrial scenarios44More materialslMore materials in my webpage:www.shichuan.org45ThanksQ&Awww.shichuan.org