A review on synergizing knowledge graphs and large language models (original) (raw)

References

  1. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan (2020) Language models are few-shot learners. In: Proceedings of the 34th international conference on neural information processing systems. NIPS ’20. Curran Associates Inc., Red Hook
  2. Hadi MU, Al Tashi Q, Shah A, Qureshi R, Muneer A, Irfan M, Zafar A, Shaikh MB, Akhtar N, Wu J et al (2024) Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects. Authorea Preprints
  3. Nickel M, Murphy K, Tresp V, Gabrilovich E (2016) A review of relational machine learning for knowledge graphs. Proc IEEE 104(1):11–33
    Article Google Scholar
  4. Manakul P, Liusie A, Gales MJ (2023) Selfcheckgpt: zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896
  5. Xu Z, Jain S, Kankanhalli M (2024) Hallucination is inevitable: an innate limitation of large language models. arXiv preprint arXiv:2401.11817
  6. Liu Y, Yao Y, Ton J-F, Zhang X, Cheng RGH, Klochkov Y, Taufiq MF, Li H (2023) Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.05374
  7. Ibrahim N, Aboulela S, Ibrahim A, Kashef R (2024) A survey on augmenting knowledge graphs (kgs) with large language models (llms): models, evaluation metrics, benchmarks, and challenges. Discov Artif Intell 4(1):76
    Article Google Scholar
  8. Ji S, Pan S, Cambria E, Marttinen P, Philip SY (2021) A survey on knowledge graphs: representation, acquisition, and applications. IEEE Trans Neural Netw Learn Syst 33(2):494–514
    Article MathSciNet Google Scholar
  9. Yin D, Dong L, Cheng H, Liu X, Chang K-W, Wei F, Gao J (2022) A survey of knowledge-intensive nlp with pre-trained language models. arXiv preprint arXiv:2202.08772
  10. Jiang X, Xu C, Shen Y, Sun X, Tang L, Wang S, Chen Z, Wang Y, Guo J (2023) On the evolution of knowledge graphs: a survey and perspective. arXiv preprint arXiv:2310.04835
  11. Pan S, Luo L, Wang Y, Chen C, Wang J, Wu X (2024) Unifying large language models and knowledge graphs: a roadmap. IEEE Trans Knowl Data Eng
  12. Fichtl A, Vladika J, Groh G (2024) Adapter-based approaches to knowledge-enhanced language models—a survey. arXiv preprint arXiv:2411.16403
  13. Paulheim H (2016) Knowledge graph refinement: a survey of approaches and evaluation methods. Semant Web 8:489–508
    Article Google Scholar
  14. Hogan A, Blomqvist E, Cochez M, D’amato C, Melo GD, Gutierrez C, Kirrane S, Gayo J, Navigli R et al (2021) Knowledge graphs. ACM Comput Surv 54(4):32
    Google Scholar
  15. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: North American chapter of the Association for Computational Linguistics
  16. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners
  17. Raffel C, Shazeer NM, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2019) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21:140–114067
    MathSciNet Google Scholar
  18. Finnie-Ansley J, Denny P, Becker BA, Luxton-Reilly A, Prather J (2022) The robots are coming: Exploring the implications of openai codex on introductory programming. In: Proceedings of the 24th Australasian computing education conference, pp 10–19
  19. Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S (2023) Palm: scaling language modeling with pathways. J Mach Learn Res 24(240):1–113
    Google Scholar
  20. Baktash JA, Dawodi M (2023) Gpt-4: a review on advancements and opportunities in natural language processing. arXiv preprint arXiv:2305.03195
  21. Bi X, Chen D, Chen G, Chen S, Dai D, Deng C, Ding H, Dong K, Du Q, Fu Z et al (2024) Deepseek llm: scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954
  22. Zhang W, Chen J, Li J, Xu Z, Pan JZ, Chen H (2024) Knowledge graph reasoning with logics and embeddings: survey and perspective. In: 2024 IEEE international conference on knowledge graph (ICKG). IEEE, pp 492–499
  23. Pan JZ, Razniewski S, Kalo J-C, Singhania S, Chen J, Dietze S, Jabeen H, Omeliyanenko J, Zhang W, Lissandrini M, Biswas R, Melo G, Bonifati A, Vakaj E, Dragoni M, Graux D (2023) Large language models and knowledge graphs: opportunities and challenges. arXiv arXiv: abs/2308.06374
  24. Xu M, Ning Y, Li Y, Chen J, Wen J, Xiao Y, Zhou S, Pan B, Bao Z, Miao X et al (2025) Reasoning based on symbolic and parametric knowledge bases: a survey. arXiv preprint arXiv:2501.01030
  25. Hofer M, Obraczka D, Saeedi A, Köpcke H, Rahm E (2024) Construction of knowledge graphs: current state and challenges. Information 15(8):509
    Article Google Scholar
  26. Auer S, Oelen A, Haris M, Stocker M, D’Souza J, Farfar KE, Vogt L, Prinz M, Wiens V, Jaradeh MY (2020) Improving access to scientific literature with knowledge graphs. Bibliothek Forschung und Praxis 44(3):516–529
    Article Google Scholar
  27. Mohit B (2014) Named entity recognition. In: NLP of semitic languages
  28. Xiang W, Wang B (2019) A survey of event extraction from text. IEEE Access 7:173111–173137
    Article Google Scholar
  29. Padia A (2017) Cleaning noisy knowledge graphs. In: Proceedings of the doctoral consortium at the 16th international semantic web conference, vol 1962
  30. Oh H, Jones A, Finin T (2024) Employing word-embedding for schema matching in standard lifecycle management. J Ind Inf Integr 38:100547
    Google Scholar
  31. Bao W, Cao Y, Yang Y, Che H, Huang J, Wen S (2024) Data-driven stock forecasting models based on neural networks: a review. Inf Fusion 8:102616
    Google Scholar
  32. Adjali O, Grimal P, Ferret O, Ghannay S, Le Borgne H (2023) Explicit knowledge integration for knowledge-aware visual question answering about named entities. In: Proceedings of the 2023 ACM international conference on multimedia retrieval, pp 29–38
  33. Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, Lewis M, Yih W-T et al (2021) Retrieval-augmented generation for knowledge-intensive NLP tasks
  34. Kaddour J, Harris J, Mozes M, Bradley H, Raileanu R, McHardy R (2023) Challenges and applications of large language models. arXiv preprint arXiv:2307.10169
  35. Liu W, Zhou P, Zhao Z, Wang Z, Ju Q, Deng H, Wang P (2019) K-bert: enabling language representation with knowledge graph. arXiv arXiv: abs/1909.07606
  36. Wang X, Gao T, Zhu Z, Zhang Z, Liu Z, Li J, Tang J (2021) Kepler: a unified model for knowledge embedding and pre-trained language representation. Trans Assoc Comput Linguist 9:176–194
    Article Google Scholar
  37. Seow M-J, Qian L (2024) Knowledge augmented intelligence using large language models for advanced data analytics. In: SPE eastern regional meeting. SPE, p 021-001003
  38. Okeyo G, Chen L, Wang H, Sterritt R (2014) Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive Mob Comput 10:155–172
    Article Google Scholar
  39. Chen X, Jia S, Xiang Y (2020) A review: knowledge reasoning over knowledge graph. Expert Syst Appl 141:112948
    Article Google Scholar
  40. Wang R, Tang D, Duan N, Wei Z, Huang X, Ji J, Cao G, Jiang D, Zhou M (2020) K-adapter: infusing knowledge into pre-trained models with adapters. In: Findings
  41. Sirocchi C, Bogliolo A, Montagna S (2024) Medical-informed machine learning: integrating prior knowledge into medical decision systems. BMC Med Inform Decis Mak 24(Suppl 4):186
    Article Google Scholar
  42. Zeng Z, Cheng Q, Si Y (2023) Logical rule-based knowledge graph reasoning: a comprehensive survey. Mathematics 11(21):4486
    Article Google Scholar
  43. Jiang M, Lin BY, Wang S, Xu Y, Yu W, Zhu C (2024) Knowledge-augmented methods for natural language processing. In: Springer briefs in computer science
  44. Shuster K, Komeili M, Adolphs L, Roller S, Szlam A, Weston J (2022) Language models that seek for knowledge: modular search & generation for dialogue and prompt completion. arXiv preprint arXiv:2203.13224
  45. Hollart C, Benedek L, Tikhomirov P, Novak P, Balint S, Yordanov K (2024) Functional role of dynamic knowledge synchronization in large language models. Authorea Preprints
  46. Li F, Zhang M, Fu G, Ji D (2017) A neural joint model for entity and relation extraction from biomedical text. BMC Bioinform 18:1–11
    Article Google Scholar
  47. Zhou H, Hu C, Yuan Y, Cui Y, Jin Y, Chen C, Wu H, Yuan D, Jiang L, Wu D et al (2024) Large language model (llm) for telecommunications: a comprehensive survey on principles, key techniques, and opportunities. arXiv preprint arXiv:2405.10825
  48. Zhang Y, Yang Q (2021) A survey on multi-task learning. IEEE Trans Knowl Data Eng 34(12):5586–5609
    Article Google Scholar
  49. Duong A-Q, Ho N-H, Pant S, Kim S, Kim S-H, Yang H-J (2024) Residual relation-aware attention deep graph-recurrent model for emotion recognition in conversation. IEEE Access
  50. Ying H, Zhuang F, Zhang F, Liu Y, Xu G, Xie X, Xiong H, Wu J (2018) Sequential recommender system based on hierarchical attention network. In: IJCAI international joint conference on artificial intelligence
  51. Zhu S, Sun S (2024) Exploring knowledge graph-based neural-symbolic system from application perspective. arXiv preprint arXiv:2405.03524
  52. Bai G, Chai Z, Ling C, Wang S, Lu J, Zhang N, Shi T, Yu Z, Zhu M, Zhang Y et al (2024) Beyond efficiency: a systematic survey of resource-efficient large language models. arXiv preprint arXiv:2401.00625
  53. Devalal S, Karthikeyan A (2018) Lora technology-an overview. In: 2018 Second international conference on electronics, communication and aerospace technology (ICECA). IEEE, pp 284–290
  54. Dong W, Yan D, Lin Z, Wang P (2024) Efficient adaptation of large vision transformer via adapter re-composing. Adv Neural Inf Process Syst 36:262
    Google Scholar
  55. Czekalski E, Watson D (2024) Efficiently updating domain knowledge in large language models: techniques for knowledge injection without comprehensive retraining
  56. Fichtl A (2024) Evaluating adapter-based knowledge-enhanced language models in the biomedical domain
  57. Padhi T, Kursuncu U, Kumar Y, Shalin VL, Fronczek LP (2024) Improving contextual congruence across modalities for effective multimodal marketing using knowledge-infused learning. arXiv preprint arXiv:2402.03607
  58. Chen Z, Xu L, Zheng H, Chen L, Tolba A, Zhao L, Yu K, Feng H (2024) Evolution and prospects of foundation models: from large language models to large multimodal models. Comput Mater Contin 80(2):655
    Google Scholar
  59. Huang Y, Xu J, Lai J, Jiang Z, Chen T, Li Z, Yao Y, Ma X, Yang Y et al (2023) Advancing transformer architecture in long-context large language models: a comprehensive survey. arXiv preprint arXiv:2311.12351
  60. Cho HN, Jun TJ, Kim Y-H, Kang H, Ahn I, Gwon H, Kim Y, Seo J, Choi H, Kim M (2024) Task-specific transformer-based language models in health care: scoping review. JMIR Med Inform 12:49724
    Article Google Scholar
  61. Yang M, Chen J, Zhang Y, Liu J, Zhang J, Ma Q, Verma H, Zhang Q, Zhou M, King I et al (2024) Low-rank adaptation for foundation models: a comprehensive review. arXiv preprint arXiv:2501.00365
  62. Lin X, Su T, Huang Z, Xue S, Liu H, Chen E (2024) A knowledge-injected curriculum pretraining framework for question answering. In: Proceedings of the ACM on web conference 2024, pp 1986–1997
  63. Wu A, Kuang K, Zhu M, Wang Y, Zheng Y, Han K, Li B, Chen G, Wu F, Zhang K (2024) Causality for large language models. arXiv preprint arXiv:2410.15319
  64. Radanliev P, Santos O, Brandon-Jones A, Joinson A (2024) Ethics and responsible AI deployment. Front Artif Intell 7:1377011
    Article Google Scholar
  65. Radanliev P (2025) Ai ethics: integrating transparency, fairness, and privacy in ai development. Appl Artif Intell 39(1):2463722
    Article Google Scholar
  66. Du Z, Qian Y, Liu X, Ding M, Qiu J, Yang Z, Tang J (2021) Glm: general language model pretraining with autoregressive blank infilling. In: Annual meeting of the association for computational linguistics
  67. GLM T, Zeng A, Xu B, Wang B, Zhang C, Yin D, Zhang D, Rojas D et al (2024) ChatGLM: a family of large language models from GLM-130B to GLM-4 all tools
  68. Liu X, Lei X, Wang S, Huang Y, Feng Z, Wen B, Cheng J, Ke P, Xu Y, Tam WL, Zhang X, Sun L, Wang H, Zhang J, Huang M, Dong Y, Tang J (2023) AlignBench: benchmarking chinese alignment of large language models
  69. Du Z, Qian Y, Liu X, Ding M, Qiu J, Yang Z, Tang J (2021) All nlp tasks are generation tasks: a general pretraining framework
  70. Schick T, Schütze H (2021) It’s not just size that matters: small language models are also few-shot learners
  71. Patil R, Gudivada V (2024) A review of current trends, techniques, and challenges in large language models (llms). Appl Sci 14(5):2074
    Article Google Scholar
  72. Yang Z, Chen J, Du Z, Yu W, Wang W, Hong W, Jiang Z, Xu B, Dong Y, Tang J (2024) Mathglm-vision: solving mathematical problems with multi-modal large language model. arXiv preprint arXiv:2409.13729

Download references