Neuraltalk+: neural image captioning with visual assistance capabilities (original) (raw)

References

  1. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473
  2. Banerjee S, Lavie A (2005) Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. pp 65–72
  3. Chen Z, Wang J, Ma A, Zhong Y (2022) Typeformer: Multiscale transformer with type controller for remote sensing image caption. IEEE Geosci Remote Sens Lett 19:1–5
    MATH Google Scholar
  4. Cheng Q, Zhou Y, Fu P, Xu Y, Zhang L (2021) A deep semantic alignment network for the cross-modal image-text retrieval in remote sensing. IEEE J Sel Top Appl Earth Obs Remote Sens 14:4284–4297
    MATH Google Scholar
  5. Chu F, Cao J, Shao Z, Pang Y (2022) Illumination-guided transformer-based network for multispectral pedestrian detection. In: CAAI international conference on artificial intelligence. Springer, pp 343–355
  6. Effendi J, Sakti S, Nakamura S (2021) End-to-End Image-to-Speech Generation for Untranscribed Unknown Languages. IEEE Access 9:55144–55154
    Google Scholar
  7. Fang H, Gupta S, Iandola F, Srivastava RK, Deng L, Dollár P, Gao J, He X, Mitchell M, Platt JC, et al. (2015) From captions to visual concepts and back. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1473–1482
  8. Fang Z, Wang J, Hu X, Liang L, Gan Z, Wang L, Yang Y, Liu Z (2022) Injecting semantic concepts into end-to-end image captioning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 18009–18019
  9. Farhadi A, Hejrati M, Sadeghi MA, Young P, Rashtchian C, Hockenmaier J, Forsyth D (2010) Every picture tells a story: Generating sentences from images. In: Computer Vision–ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV 11. Springer, pp 15–29
  10. Gong Y, Wang L, Hodosh M, Hockenmaier J, Lazebnik S (2014) Improving image-sentence embeddings using large weakly annotated photo collections. In: European conference on computer vision. Springer, pp 529–545
  11. Guo L, Liu J, Zhu X, Yao P, Lu S, Lu H (2020) Normalized and Geometry-Aware Self-Attention Network for Image Captioning. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Seattle, WA, USA, pp 10324–10333
    Google Scholar
  12. Haque AUl, Ghani S, Saeed M (2021) Image captioning with positional and geometrical semantics. IEEE Access 9:160917–160925
  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770–778
  14. Herdade S, Kappeler A, Boakye K, Soares J (2019) Image captioning: Transforming objects into words. In: Conference and workshop on neural information processing systems
  15. Hodosh M, Young P, Hockenmaier J (2013) Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics. J Artif Intel Res 47:853–899
    MathSciNet MATH Google Scholar
  16. MZakir Hossain, F Sohel, MF Shiratuddin, H Laga, (2019) A comprehensive survey of deep learning for image captioning. ACM Comput Surv 51(6)
  17. Hossain MZ, Sohel F, Shiratuddin MF, Laga H, Bennamoun M (2021) Text to image synthesis for improved image captioning. IEEE Access 9:64918–64928
    Google Scholar
  18. Jiang W, Li X, Hu H, Lu Q, Liu B (2021) Multi-Gate Attention Network for Image Captioning. IEEE Access 9:69700–69709
    MATH Google Scholar
  19. Jin J, Fu K, R Cui, F Sha, C Zhang (2015) Aligning where to see and what to tell: image caption with region-based attention and scene factorization. arXiv:1506.06272
  20. Karpathy A, Fei-Fei L (2014) Deep visual-semantic alignments for generating image descriptions. IEEE Trans Pattern Anal Mach Intell 39:664–676
    MATH Google Scholar
  21. Kiros R, Salakhutdinov R, Zemel RS (2014) Multimodal neural language models. In: International Conference on Machine Learning
  22. Kulkarni G, Premraj V, Ordonez V, Dhar S, Li S, Choi Y, Berg AC, Berg TL (2013) Babytalk: Understanding and generating simple image descriptions. IEEE Trans Pattern Anal Mach Intell 35(12):2891–2903
    Google Scholar
  23. Li J, Yao P, Guo L, Zhang W (2019) Boosted Transformer for Image Captioning. Appl Sci 9(16):3260
    MATH Google Scholar
  24. Li W, Qu Z, Song H, Wang P, Xue B (2020) The traffic scene understanding and prediction based on image captioning. IEEE Access 9:1420–1427
    MATH Google Scholar
  25. Lian Z, Zhang Y, Li H, Wang R, Hu X (2023) Cross modification attention-based deliberation model for image captioning. Appl Intell 53(5):5910–5933
    MATH Google Scholar
  26. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision. Springer, pp 740–755
  27. Ordonez V, Kulkarni G, Berg T (2011) Im2text: Describing images using 1 million captioned photographs. Adv Neural Inf Process Sys 24
  28. Papineni K, Roukos S, Ward T, Zhu W-J (2002) Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. pp 311–318
  29. Shao Z, Han J, Debattista K, Pang Y (2023) Textual context-aware dense captioning with diverse words. IEEE Trans Multimed
  30. Shao Z, Han J, Marnerides D, Debattista K (2022) Region-object relation-aware dense captioning via transformer. IEEE Trans Neural Netw Learn Sys
  31. Sharma H, Padha D (2023) A comprehensive survey on image captioning: from handcrafted to deep learning-based techniques, a taxonomy and open research issues. Artif Intel Rev 1–43
  32. Vedantam R, Zitnick CL, Parikh D (2015) CIDEr: Consensus-based Image Description Evaluation. arXiv:1411.5726
  33. Vinyals O, Toshev A, Bengio S, Erhan D (2015) Show and tell: A neural image caption generator. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Boston, MA, USA, pp 3156–3164
    Google Scholar
  34. Wang C, Gu X (2022) Image captioning with adaptive incremental global context attention. Appl Intel 1–23
  35. Wang C, Yang H, Bartz C, Meinel C (2016) Image captioning with deep bidirectional lstms. In: Proceedings of the 24th ACM international conference on Multimedia. pp 988–997
  36. Wang Y, Xu J, Sun Y (2022) End-to-end transformer based model for image captioning. Proc AAAI Conf Artif Intel 36:2585–2594
    MATH Google Scholar
  37. Wei J, Li Z, Zhu J, Ma H (2023) Enhance understanding and reasoning ability for image captioning. Appl Intell 53(3):2706–2722
    MATH Google Scholar
  38. Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y (2015) Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning. PMLR, pp 2048–2057
  39. Yang Y, Teo C, Daumé III H, Aloimonos Y (2011) Corpus-guided sentence generation of natural images. In: Proceedings of the 2011 conference on empirical methods in natural language processing. pp 444–454
  40. Yu J, Li J, Yu Z, Huang Q (2019) Multimodal transformer with multi-view visual representation for image captioning. IEEE Trans Circuits Syst Video Technol 30(12):4467–4480
    MATH Google Scholar
  41. Zhang J, Fang Z, Wang Z (2022) Multi-feature fusion enhanced transformer with multi-layer fused decoding for image captioning. Appl Intel 1–17
  42. Zhao W, Wu X, Luo J (2020) Cross-domain image captioning via cross-modal retrieval and model adaptation. IEEE Trans Image Process 30:1180–1192
    MathSciNet MATH Google Scholar
  43. Zhou D, Yang J, Bao R (2021) Collaborative strategy network for spatial attention image captioning. Appl Intel 1–16:100
    MATH Google Scholar

Download references