Natural Language Processing
Published October 19, 2018 | Version ainl2018
Presentation Open

Graph Clustering for Natural Language Processing

Creators

  • 1. University of Mannheim

Description

Graph-based representations are proven to be an effective approach for a variety of Natural Language Processing (NLP) tasks. Graph clustering makes it possible to extract useful knowledge by exploiting the implicit structure of the data. In this tutorial, we will present several efficient graph clustering algorithms, show their strengths and weaknesses as well as their implementations and applications. Then, the evaluation methodology in unsupervised NLP tasks will be discussed.

Notes

These materials are published under a CC BY-NC-SA license. Please feel welcome to share them! For viewer convenience, the slides published on Zenodo do not include interactive step-by-step examples.

Files

GraphClustering.pdf

Files (5.9 MB)

Name Size Download all
md5:c096a6daa97810ad4c5faa1c8f5a14e4
5.9 MB Preview Download

Additional details

Cites
Conference paper: 10.18653/v1/P17-1145 (DOI)
Conference paper: 10.18653/v1/P18-2010 (DOI)
Journal article: 10.1162/COLI_a_00354 (DOI)
Is continued by
Presentation: 10.5281/zenodo.3510160 (DOI)
Is identical to
Presentation: urn:nbn:de:bsz:180-madoc-465240 (URN)
Is previous version of
Lesson: 10.5281/zenodo.4291121 (DOI)
Is supplement to
Book: 10.1007/978-3-030-01204-5 (DOI)
References
Book: 10.1007/978-3-642-25923-4 (DOI)
Book: 978-0-521-89613-9 (ISBN)
Software: https://github.com/nlpub/watset-java (URL)

References

  • Azadani, M.N., Ghadiri, N., Davoodijam, E.: Graph-based biomedical text summarization: An itemset mining and sentence clustering approach. Journal of Biomedical Informatics. 84, 42–58 (2018). https://doi.org/10.1016/j.jbi.2018.06.005.
  • Baker, C.F., Fillmore, C.J., Lowe, J.B.: The Berkeley FrameNet Project. In: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics - Volume 1. pp. 86–90. Association for Computational Linguistics, Montréal, QC, Canada (1998). https://doi.org/10.3115/980845.980860.
  • Biemann, C.: Chinese Whispers: An Efficient Graph Clustering Algorithm and Its Application to Natural Language Processing Problems. In: Proceedings of the First Workshop on Graph Based Methods for Natural Language Processing. pp. 73–80. Association for Computational Linguistics, New York, NY, USA (2006). https://doi.org/10.3115/1654758.1654774.
  • Biemann, C.: Structure Discovery in Natural Language. Springer Berlin Heidelberg (2012). https://doi.org/10.1007/978-3-642-25923-4.
  • Buluç, A., Fineman, J.T., Frigo, M., Gilbert, J.R., Leiserson, C.E.: Parallel Sparse Matrix-vector and Matrix-transpose-vector Multiplication Using Compressed Sparse Blocks. In: Proceedings of the Twenty-first Annual Symposium on Parallelism in Algorithms and Architectures. pp. 233–244. ACM, Calgary, AB, Canada (2009). https://doi.org/10.1145/1583991.1584053.
  • van Dongen, S.: Graph Clustering by Flow Simulation, (2000).
  • Dorogovtsev, S.N., Mendes, J.F.F.: Language as an evolving word web. Proceedings of the Royal Society of London B: Biological Sciences. 268, 2603–2606 (2001). https://doi.org/10.1098/rspb.2001.1824.
  • Dorow, B., Widdows, D.: Discovering Corpus-Specific Word Senses. In: Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics - Volume 2. pp. 79–82. Association for Computational Linguistics, Budapest, Hungary (2003). https://doi.org/10.3115/1067737.1067753.
  • Dror, R., Baumer, G., Shlomov, S., Reichart, R.: The Hitchhiker's Guide to Testing Statistical Significance in Natural Language Processing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1383–1392. Association for Computational Linguistics, Melbourne, VIC, Australia (2018). https://doi.org/10.18653/v1/P18-1128.
  • Duff, I.S., Grimes, R.G., Lewis, J.G.: Sparse Matrix Test Problems. ACM Transactions on Mathematical Software. 15, 1–14 (1989). https://doi.org/10.1145/62038.62043.
  • Faralli, S., Panchenko, A., Biemann, C., Ponzetto, S.P.: Linked Disambiguated Distributional Semantic Networks. In: The Semantic Web – ISWC 2016, 15th International Semantic Web Conference, Kobe, Japan, October 17–21, 2016, Proceedings, Part II. pp. 56–64. Springer International Publishing, Cham, Switzerland (2016). https://doi.org/10.1007/978-3-319-46547-0_7.
  • Fellbaum, C.: WordNet: An Electronic Database. MIT Press (1998).
  • Fillmore, C.J.: Frame Semantics. In: Linguistics in the Morning Calm. pp. 111–137. Hanshin Publishing Co., Seoul, South Korea (1982).
  • Fortunato, S.: Community detection in graphs. Physics Reports. 486, 75–174 (2010). https://doi.org/10.1016/j.physrep.2009.11.002.
  • Frey, B.J., Dueck, D.: Clustering by Passing Messages Between Data Points. Science. 315, 972–976 (2007). https://doi.org/10.1126/science.1136800.
  • Goldhahn, D., Eckart, T., Quasthoff, U.: Building Large Monolingual Dictionaries at the Leipzig Corpora Collection: From 100 to 200 Languages. In: Proceedings of the Eight International Conference on Language Resources and Evaluation. pp. 759–765. European Language Resources Association (ELRA), Istanbul, Turkey (2012).
  • Goyal, P., Ferrara, E.: Graph embedding techniques, applications, and performance: A survey. Knowledge-Based Systems. 151, 78–94 (2018). https://doi.org/10.1016/j.knosys.2018.03.022.
  • Hope, D., Keller, B.: MaxMax: A Graph-Based Soft Clustering Algorithm Applied to Word Sense Induction. In: Computational Linguistics and Intelligent Text Processing, 14th International Conference, CICLing 2013, Samos, Greece, March 24-30, 2013, Proceedings, Part I. pp. 368–381. Springer Berlin Heidelberg, Berlin; Heidelberg, Germany (2013). https://doi.org/10.1007/978-3-642-37247-6_30.
  • Hope, D., Keller, B.: UoS: A Graph-Based System for Graded Word Sense Induction. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013). pp. 689–694. Association for Computational Linguistics, Atlanta, GA, USA (2013).
  • Kawahara, D., Peterson, D.W., Palmer, M.: A Step-wise Usage-based Method for Inducing Polysemy-aware Verb Classes. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers. pp. 1030–1040. Association for Computational Linguistics, Baltimore, MD, USA (2014). https://doi.org/10.3115/v1/P14-1097.
  • Krizhanovsky, A.A., Smirnov, A.V.: An approach to automated construction of a general-purpose lexical ontology based on Wiktionary. Journal of Computer and Systems Sciences International. 52, 215–225 (2013). https://doi.org/10.1134/S1064230713020068.
  • Lewis, M., Steedman, M.: Unsupervised Induction of Cross-Lingual Semantic Relations. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. pp. 681–692. Association for Computational Linguistics, Seattle, WA, USA (2013).
  • von Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing. 17, 395–416 (2007). https://doi.org/10.1007/s11222-007-9033-z.
  • Lyzinski, V., Sell, G., Jansen, A.: An Evaluation of Graph Clustering Methods for Unsupervised Term Discovery. In: INTERSPEECH-2015. pp. 3209–3213. International Speech Communication Association, Dresden, Germany (2015).
  • Manandhar, S., Klapaftis, I., Dligach, D., Pradhan, S.: SemEval-2010 Task 14: Word Sense Induction & Disambiguation. In: Proceedings of the 5th International Workshop on Semantic Evaluation. pp. 63–68. Association for Computational Linguistics, Uppsala, Sweden (2010).
  • Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press (2008).
  • Marcheggiani, D., Titov, I.: Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pp. 1506–1515. Association for Computational Linguistics, Copenhagen, Denmark (2017). https://doi.org/10.18653/v1/D17-1159.
  • Mihalcea, R., Radev, D.: Graph-Based Natural Language Processing and Information Retrieval. Cambridge University Press (2011). https://doi.org/10.1017/CBO9780511976247.
  • Navigli, R., Ponzetto, S.P.: BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence. 193, 217–250 (2012). https://doi.org/10.1016/j.artint.2012.07.001.
  • Padó, S.: User's guide to sigf: Significance testing by approximate randomisation. (2006).
  • Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank Citation Ranking: Bringing Order to the Web. Stanford InfoLab (1999).
  • Panchenko, A., Ruppert, E., Faralli, S., Ponzetto, S.P., Biemann, C.: Building a Web-Scale Dependency-Parsed Corpus from Common Crawl. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation. pp. 1816–1823. European Language Resources Association (ELRA), Miyazaki, Japan (2018).
  • Pelevina, M., Arefiev, N., Biemann, C., Panchenko, A.: Making Sense of Word Embeddings. In: Proceedings of the 1st Workshop on Representation Learning for NLP. pp. 174–183. Association for Computational Linguistics, Berlin, Germany (2016). https://doi.org/10.18653/v1/W16-1620.
  • Steyvers, M., Tenenbaum, J.B.: The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth. Cognitive Science. 29, 41–78 (2005). https://doi.org/10.1207/s15516709cog2901_3.
  • Tauer, G., Date, K., Nagi, R., Sudit, M.: An incremental graph-partitioning algorithm for entity resolution. Information Fusion. 46, 171–183 (2019). https://doi.org/10.1016/j.inffus.2018.06.001.
  • Ustalov, D., Panchenko, A., Biemann, C.: Automatic Induction of Synsets from a Graph of Synonyms. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1579–1590. Association for Computational Linguistics, Vancouver, BC, Canada (2017). https://doi.org/10.18653/v1/P17-1145.
  • Ustalov, D., Panchenko, A., Kutuzov, A., Biemann, C., Ponzetto, S.P.: Unsupervised Semantic Frame Induction using Triclustering. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 55–62. Association for Computational Linguistics, Melbourne, VIC, Australia (2018). https://doi.org/10.18653/v1/P18-2010.
  • Viamontes Esquivel, A., Rosvall, M.: Comparing network covers using mutual information, http://arxiv.org/abs/1202.0425.
  • Vlasblom, J., Wodak, S.J.: Markov clustering versus affinity propagation for the partitioning of protein interaction graphs. BMC Bioinformatics. 10, 99 (2009). https://doi.org/10.1186/1471-2105-10-99.
  • Yeh, A.: More accurate tests for the statistical significance of result differences. In: Proceedings of the 18th Conference on Computational Linguistics - Volume 2. pp. 947–953. Association for Computational Linguistics, Saarbrücken, Germany (2000). https://doi.org/10.3115/992730.992783.
  • Zesch, T., Müller, C., Gurevych, I.: Extracting Lexical Semantic Knowledge from Wikipedia and Wiktionary. In: Proceedings of the 6th International Conference on Language Resources and Evaluation. pp. 1646–1652. European Language Resources Association (ELRA), Marrakech, Morocco (2008).