Multi-level Disentangled Contrastive Learning on Heterogeneous Graphs
Authors: 
Tong Lin, Cangqi Zhou, Qianmu Li, Dianming HuAuthors Info & Claims
Tong Lin, Cangqi Zhou, Qianmu Li, Dianming HuAuthors Info & ClaimsPages 628 - 634
Abstract
Self-supervised learning methods, including contrastive learning based on Heterogeneous graph neural networks (HGNNs), have achieved great success in learning the representations of heterogeneous information networks (HINs). However, the existing self-supervised methods usually neglect the entanglement of the latent factors behind HINs, which decreases the performance of downstream tasks. In this paper, we propose Multi-level Disentangled Heterogeneous Graph Contrastive Learning method and learning disentangled HIN node representations in a self-supervised way. Specifically, we first design a tailored encoder to capture the latent factors and semantics of nodes in input HIN and learn their factorized representations. Then we propose a novel contrastive learning discrimination objective designed for disentangled HIN node representation learning. Extensive experiments conducted on various real-world datasets demonstrate the superiority of our method against state-of-the-art baselines.
References
[1]
Mengru Chen, Chao Huang, Lianghao Xia, Wei Wei, Yong Xu, and Ronghua Luo. 2023. Heterogeneous graph contrastive learning for recommendation. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 544–552.
[2]
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems 29 (2016).
[3]
Sanghyeok Chu, Dongwan Kim, and Bohyung Han. 2021. Learning debiased and disentangled representations for semantic segmentation. Advances in Neural Information Processing Systems 34 (2021), 8355–8366.
[4]
Ching-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, and Stefanie Jegelka. 2020. Debiased contrastive learning. Advances in neural information processing systems 33 (2020), 8765–8775.
[5]
Andrea Dittadi, Frederik Träuble, Francesco Locatello, Manuel Wüthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, and Bernhard Schölkopf. 2020. On the transfer of disentangled representations in realistic settings. arXiv preprint arXiv:2010.14407 (2020).
[6]
Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of The Web Conference 2020. 2331–2341.
[7]
Marco Fumero, Luca Cosmo, Simone Melzi, and Emanuele Rodolà. 2021. Learning disentangled representations via product manifold projection. In International conference on machine learning. PMLR, 3530–3540.
[8]
Jun Han, Martin Renqiang Min, Ligong Han, Li Erran Li, and Xuan Zhang. 2021. Disentangled recurrent wasserstein autoencoder. arXiv preprint arXiv:2101.07496 (2021).
[9]
Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In International conference on machine learning. PMLR, 4116–4126.
[10]
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2016. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations.
[11]
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. Advances in neural information processing systems 33 (2020), 18661–18673.
[12]
Thomas N Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016).
[13]
Haoyang Li, Ziwei Zhang, Xin Wang, and Wenwu Zhu. 2022. Disentangled graph contrastive learning with independence promotion. IEEE Transactions on Knowledge and Data Engineering (2022).
[14]
Xiang Li, Danhao Ding, Ben Kao, Yizhou Sun, and Nikos Mamoulis. 2021. Leveraging meta-path contexts for classification in heterogeneous information networks. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 912–923.
[15]
Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. 2019. Disentangled graph convolutional networks. In International conference on machine learning. PMLR, 4212–4221.
[16]
Chanyoung Park, Donghyun Kim, Jiawei Han, and Hwanjo Yu. 2020. Unsupervised attributed multiplex network embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 5371–5378.
[17]
Yizhou Sun and Jiawei Han. 2013. Mining heterogeneous information networks: a structural analysis approach. Acm Sigkdd Explorations Newsletter 14, 2 (2013), 20–28.
[18]
Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S Yu, and Tianyi Wu. 2011. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment 4, 11 (2011), 992–1003.
[19]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
[20]
Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2018. Deep graph infomax. arXiv preprint arXiv:1809.10341 (2018).
[21]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019. Heterogeneous graph attention network. In The world wide web conference. 2022–2032.
[22]
Xiao Wang, Nian Liu, Hui Han, and Chuan Shi. 2021. Self-supervised heterogeneous graph neural network with co-contrastive learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 1726–1736.
[23]
Junkang Wu, Wentao Shi, Xuezhi Cao, Jiawei Chen, Wenqiang Lei, Fuzheng Zhang, Wei Wu, and Xiangnan He. 2021. DisenKGAT: knowledge graph embedding with disentangled graph attention network. In Proceedings of the 30th ACM international conference on information & knowledge management. 2140–2149.
[24]
Dongkuan Xu, Wei Cheng, Dongsheng Luo, Haifeng Chen, and Xiang Zhang. 2021. Infogcl: Information-aware graph contrastive learning. Advances in Neural Information Processing Systems 34 (2021), 30414–30425.
[25]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks?arXiv preprint arXiv:1810.00826 (2018).
[26]
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph contrastive learning with augmentations. Advances in neural information processing systems 33 (2020), 5812–5823.
[27]
Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019. Heterogeneous graph neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 793–803.
[28]
Jianan Zhao, Xiao Wang, Chuan Shi, Zekuan Liu, and Yanfang Ye. 2020. Network schema preserving heterogeneous information network embedding. In International joint conference on artificial intelligence (IJCAI).
[29]
Tianxiang Zhao, Xiang Zhang, and Suhang Wang. 2022. Exploring edge disentanglement for node classification. In Proceedings of the ACM Web Conference 2022. 1028–1036.
[30]
Yanqiao Zhu, Yichen Xu, Hejie Cui, Carl Yang, Qiang Liu, and Shu Wu. 2022. Structure-enhanced heterogeneous graph contrastive learning. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM). SIAM, 82–90.
Index Terms
- Multi-level Disentangled Contrastive Learning on Heterogeneous Graphs
Recommendations
Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data MiningHeterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN). However, most HGNNs follow a semi-supervised learning manner, which notably limits their wide use ...
Disentangled contrastive learning on graphs
NIPS '21: Proceedings of the 35th International Conference on Neural Information Processing SystemsRecently, self-supervised learning for graph neural networks (GNNs) has attracted considerable attention because of their notable successes in learning the representation of graph-structure data. However, the formation of a real-world graph typically ...
Semi-supervised heterogeneous graph contrastive learning with label-guided
AbstractHeterogeneous Graph Neural Networks represent a powerful approach to understand and utilize the intricate structures and semantics within complex graphs. When it comes to semi-supervised learning on graphs, the challenge lies in effectively ...
Comments
Information & Contributors
Information
Published In
February 2024
757 pages
ISBN:9798400709234
DOI:10.1145/3651671
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
ICMLC 2024
ICMLC 2024: 2024 16th International Conference on Machine Learning and Computing
February 2 - 5, 2024
Shenzhen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 52Total Downloads
- Downloads (Last 12 months)52
- Downloads (Last 6 weeks)12
Reflects downloads up to 27 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format