關于網絡嵌入和圖卷積神經網絡的一些思考（34頁）——基于圖神經網絡認知的智能計算專場.pdf

1、關于網絡嵌入和圖卷積神經網絡的一些思考崔 鵬清華大學Perspectives and Outlook on Network Embedding and GCN網絡/圖數據圖是對于數據的一/通用、全面、復雜的表示形式網絡無處不在社交網絡生物網絡金融網絡物聯網信息網絡物流網絡為什么網絡很重要？我們很少只關心數據本身，而不關心數據之間的關聯Reflected by relational subjects Decided by relational subjects TargetTargetImage CharacterizationSocial Capital網絡數據對機器學習模型不友好G=(V,E

2、)LinksTopologyInapplicability of ML methodsNetwork DataFeature ExtractionPattern DiscoveryPipeline for network analysisNetwork ApplicationsLearnabilityLearning from NetworksNetwork EmbeddingGNNG=(V,E)G=(V)Vector SpacegenerateembedEasy to parallelCan apply classical ML methods網絡嵌入(Network Embedding)網

3、絡嵌入的目標Goal Support network inference in vector spaceReflect network structureMaintain network propertiesBACTransitivitypBasic idea:recursive definition of statespA simple example:PageRank圖神經網絡GNNF.Scarselli,et al.The graph neural network model.IEEE TNN,2009.定義在圖拓撲上的學習框架pMain idea:pass messages betwe

4、en pairs of nodes&agglomeratepStacking multiple layers like standard CNNs:pState-of-the-art results on node classification圖卷積神經網絡GCNT.N.Kipf and M.Welling.Semi-supervised classification with graph convolutional networks.ICLR,2017.圖神經網絡GNN簡史網絡嵌入與圖神經網絡GraphFeatureNetwork EmbeddingGCNInputTask resultsM

5、odelOutputEmbeddingTask resultsFeatureTopology to VectorFusion of Topology and FeaturesUnsupervised vs.(Semi-)Supervised圖卷積網絡 v.網絡嵌入p In some sense,they are different.p Graphs exist in mathematics.(Data Structure)p Mathematical structures used to model pairwise relations between objectsp Networks ex

6、ist in the real world.(Data)p Social networks,logistic networks,biology networks,transactionnetworks,etc.p A network can be represented by a graph.p A dataset that is not a network can also be represented by a graph.圖卷積網絡應用于自然語言處理pMany papers on BERT+GNN.pBERT is for retrieval.pIt creates an initial

7、 graph of relevant entities and the initial evidence.pGNN is for reasoning.pIt collects evidence(i.e.,old messages on the entities)and arrive at new conclusions(i.e.,new messages on the entities),by passing the messages around and aggregating them.Cognitive Graph for Multi-Hop Reading Comprehension

8、at Scale.Ding et al.,ACL 2019.Dynamically Fused Graph Network for Multi-hop Reasoning.Xiao et al.,ACL 2019.圖卷積網絡應用于計算機視覺pA popular trend in CV is to construct a graph during the learning process.pTo process multiple objects or parts in a scene,and to infer their relationships.pExample:Scene graphs.S

9、cene Graph Generation by Iterative Message Passing.Xu et al.,CVPR 2017.Image Generation from Scene Graphs.Johnson et al.,CVPR 2018.圖卷積網絡應用于符號推理pWe can view the process of symbolic reasoning as a directed acyclic graph.pMany recent efforts use GNNs to perform symbolic reasoning.Learning by Abstractio

10、n:The Neural State Machine.Hudson&Manning,2019.Can Graph Neural Networks Help Logic Reasoning?Zhang et al.,2019.Symbolic Graph Reasoning Meets Convolutions.Liang et al.,NeurIPS 2018.pStructural equation modeling,a form of causal modeling,tries to describe the relationships between the variables as a

11、 directed acyclic graph(DAG).pGNN can be used to represent a nonlinear structural equation and help find the DAG,after treating the adjacency matrix as parameters.圖卷積網絡應用于結構方程建模DAG-GNN:DAG Structure Learning with Graph Neural Networks.Yu et al.,ICML 2019.（大多數）圖卷積網絡方法的PipelinepCo-occurrence(neighborh

12、ood)網絡嵌:拓撲向量化pHigh-order proximities網絡嵌:拓撲向量化pCommunities網絡嵌:拓撲向量化pHeterogeneous networks網絡嵌:拓撲向量化（大多數）網絡嵌入方法的PipelineLearning for Networks v.s.Learning via GraphsLearning for networksLearning Via GraphsNetwork EmbeddingGCN網絡嵌入方法解決的核心問題Reducing representation dimensionality while preserving necessar

13、y topological structures and properties.Nodes&LinksNode NeighborhoodCommunityPair-wise ProximityHyper EdgesGlobal StructureNon-transitivityAsymmetric TransitivityDynamicUncertaintyHeterogeneityInterpretabilityTopology-driven圖卷積神經網絡方法解決的核心問題Fusing topology and features in the way of smoothing feature

14、s with the assistance of topology.Feature-driven如果問題是拓撲驅動的？p Since GCN is filtering features,it is inevitably feature-drivenp Structure only provides auxiliary information(e.g.for filtering/smoothing)p When feature plays the key role,GNN performs good p How about the contrary?p Synthesis data:stocha

15、stic block model+random featuresMethodResultsRandom10.0GCN18.31.1DeepWalk99.00.1網絡嵌入 v.圖神經網絡There is no better one,but there is more proper one.反思：圖神經網絡是否真的是深度學習方法？p Recall GNN formulation:!#$=&!(,=*+,$/./0*+,$/.p How about removing the non-linear component:!#$=!(p Stacking multiple layers and add s

16、oftmax classification:12=3456789!:=3456789!($(:,$=3456789:!(Wu,Felix,et al.Simplifying graph convolutional networks.ICML,2019.High-order proximity30p This simplified GNN(SGC)shows remarkable results:Node classification Text Classification反思：圖神經網絡是否真的是深度學習方法？Wu,Felix,et al.Simplifying graph convolutional networks.ICML,2019.總結p Unsupervised vs.(Semi-)Supervisedp Learning for Networks vs.Learning via Graphsp Topology-driven vs.Feature-drivenp Both GCN and NE need to treat the counterpart as the baselinesTHANKS!THANKS!THANKS!