Abstract This project will explore the connection between vision and language from different directions in which we will integrate computer vision and natural language processing methods. By using these two fields together, automatic systems that transcribe the visual content of images with vivid descriptions which are very alike to human language will be obtained. Similarly, in the context of this project, retrieval systems that describe the sentence or paragraph-based textual queries visually via related images or image sets will be constructed.

Sponsors: The Scientific and Technological Research Council of Turkey (TUBITAK) The Support Program for Scientific and Technological Research Projects (Award# 113E116) and European Union under European Cooperation in Science and Technology (COST) Programme (ICT COST IC1037 Action)

Related Publications Data-Driven Image Captioning via Salient Region Discovery
IET Computer Vision
Mert Kilickaya, Burak Kerim Akkus, Ruket Cakici, Aykut Erdem, Erkut Erdem, Nazli Ikizler-CinbisAutomatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
International Joint Conference on Artificial Intelligence (IJCAI) 2017 - Journal Track
Adrian Muscat, Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erkut Erdem, Nazli Ikizler Cinbis, Frank Keller, Barbara PlankRe-evaluating Automatic Metrics for Image Captioning
The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2017)
Mert Kilickaya, Aykut Erdem, Nazli Ikizler Cinbis, Erkut ErdemLeveraging Captions in the Wild to Improve Object Detection
The 5th Workshop on Vision and Language (VL'16) – in conjuction with ACL 2016
Mert Kilickaya, Nazli Ikizler-Cinbis, Erkut Erdem and Aykut ErdemTasvirEt: Görüntülerden Otomatik Türkçe Açıklama Oluşturma İçin Bir Denektaşı Veri Kümesi (TasvirEt: A Benchmark Dataset for Automatic Turkish Description Generation from Images)
24. IEEE Sinyal İşleme ve İletişim Uygulamaları Kurultayı (SIU 2016), Zonguldak, Mayis 2016
Mesut Erhan Unal, Begum Citamak, Semih Yagcioglu, Aykut Erdem, Erkut Erdem, Nazli Ikizler-Cinbis, Ruket CakiciAutomatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
Journal of Artificial Intelligence Research, Vol. 55, pp. 409-442, February 2016
Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erkut Erdem, Nazli Ikizler-Cinbis, Frank Keller, Adrian Muscat, Barbara Plank