Automatic Caption Generation for Chest X-Ray Using CNN Algorithm
Authors/Creators
- 1. Student, Department of Information Technology, Shri Ramswaroop Memorial College of Engineering and Management, Lucknow, Uttar Pradesh
- 2. Professor, Department of Information Technology, Shri Ramswaroop Memorial College of Engineering and Management, Lucknow, Uttar Pradesh
Description
The automatic caption generation of chest X-ray report is a hot research topic at present. Image captioning aims to automatically describe the relationship of an image with a sentence, and this work has attracted research from both computer vision and natural language processing research communities. This research paper proposes a novel approach to automatically generating captions for medical images using Convolutional Neural Network (CNN) algorithm. The system was trained on a large dataset of medical images and their corresponding captions, and was evaluated using a variety of metrics including BLEU score and human evaluation. The results indicate that the proposed approach outperforms existing captioning systems in terms of caption accuracy and fluency. The proposed system has potential applications in the healthcare domain, where accurate and timely description of medical images can be critical for diagnosis and treatment.
Files
AUTOMATIC CAPTION GENERATION -Formatted Paper.pdf
Files
(406.2 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:88a5e539ead21df9a9957b85276af587
|
406.2 kB | Preview Download |
Additional details
References
- 1. Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015). Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3156-3164).
- 2. Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., ... & Bengio, Y. (2015, June). Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning (pp. 2048-2057). PMLR.
- 3. Aker, A., & Gaizauskas, R. (2010, July). Generating image descriptions using dependency relational patterns. In Proceedings of the 48th annual meeting of the association for computational linguistics (pp. 1250-1258).
- 4. Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. (2010). Every picture tells a story: Generating sentences from images. In Computer Vision–ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV 11 (pp. 15-29). Springer Berlin Heidelberg.
- 5. Yin, C., Qian, B., Wei, J., Li, X., Zhang, X., Li, Y., & Zheng, Q. (2019, November). Automatic generation of medical imaging diagnostic report with hierarchical recurrent neural network. In 2019 IEEE international conference on data mining (ICDM) (pp. 728-737). IEEE.
- 6. Shin, H. C., Roberts, K., Lu, L., Demner-Fushman, D., Yao, J., & Summers, R. M. (2016). Learning to read chest x-rays: Recurrent neural cascade model for automated image annotation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2497-2506).
- 7. Huang, X., Zhong, B., Cao, Y., Yi, Y., & Gu, M. (2020, December). Chest X-ray lung Chinese description generation based on semantic labels and hierarchical LSTM. In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 1020-1023). IEEE.
- 8. Avni, U., Greenspan, H., Konen, E., Sharon, M., & Goldberger, J. (2010). X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words. IEEE Transactions on Medical Imaging, 30(3), 733-746.
- 9. Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc.".
- 10. Eickhoff, C., Schwall, I., Garcia Seco De Herrera, A., & Müller, H. (2017). Overview of ImageCLEFcaption 2017–image caption prediction and concept detection for biomedical images. CLEF 2017 working Notes, 1866(Workin).