Journal article Open Access

Implicit and Explicit Concept Relations in Deep Neural Networks for Multi-Label Video/Image Annotation

Markatopoulou, Foteini; Mezaris, Vasileios; Patras, Ioannis


JSON Export

{
  "files": [
    {
      "links": {
        "self": "https://zenodo.org/api/files/4c693bc0-b73e-4f72-9034-53346c9b1ca6/csvt18_preprint.pdf"
      }, 
      "checksum": "md5:c1f73ff41efa0635d450541c3e1cd7f1", 
      "bucket": "4c693bc0-b73e-4f72-9034-53346c9b1ca6", 
      "key": "csvt18_preprint.pdf", 
      "type": "pdf", 
      "size": 1299480
    }
  ], 
  "owners": [
    26037
  ], 
  "doi": "10.1109/TCSVT.2018.2848458", 
  "stats": {
    "version_unique_downloads": 34.0, 
    "unique_views": 50.0, 
    "views": 51.0, 
    "downloads": 36.0, 
    "unique_downloads": 34.0, 
    "version_unique_views": 50.0, 
    "volume": 46781280.0, 
    "version_downloads": 36.0, 
    "version_views": 51.0, 
    "version_volume": 46781280.0
  }, 
  "links": {
    "doi": "https://doi.org/10.1109/TCSVT.2018.2848458", 
    "latest_html": "https://zenodo.org/record/1308778", 
    "bucket": "https://zenodo.org/api/files/4c693bc0-b73e-4f72-9034-53346c9b1ca6", 
    "badge": "https://zenodo.org/badge/doi/10.1109/TCSVT.2018.2848458.svg", 
    "html": "https://zenodo.org/record/1308778", 
    "latest": "https://zenodo.org/api/records/1308778"
  }, 
  "created": "2018-07-10T12:21:10.463443+00:00", 
  "updated": "2019-04-09T14:03:03.993279+00:00", 
  "conceptrecid": "1308777", 
  "revision": 4, 
  "id": 1308778, 
  "metadata": {
    "access_right_category": "success", 
    "doi": "10.1109/TCSVT.2018.2848458", 
    "description": "<p>In this work we propose a DCNN (Deep Convolutional Neural Network) architecture that addresses the problem of video/image concept annotation by exploiting concept relations at two different levels. At the first level, we build on ideas from multi-task learning, and propose an approach to learn conceptspecific representations that are sparse, linear combinations of representations of latent concepts. By enforcing the sharing of the latent concept representations, we exploit the implicit relations between the target concepts. At a second level, we build on ideas from structured output learning, and propose the introduction, at training time, of a new cost term that explicitly models the correlations between the concepts. By doing so, we explicitly model the structure in the output space (i.e., the concept labels). Both of the above are implemented using standard convolutional layers and are incorporated in a single DCNN architecture that can then be trained end-to-end with standard back-propagation. Experiments on four large-scale video and image datasets show that the proposed DCNN improves concept annotation accuracy and outperforms the related state of-the-art methods.</p>", 
    "license": {
      "id": "CC-BY-4.0"
    }, 
    "title": "Implicit and Explicit Concept Relations in Deep Neural Networks for Multi-Label Video/Image Annotation", 
    "relations": {
      "version": [
        {
          "count": 1, 
          "index": 0, 
          "parent": {
            "pid_type": "recid", 
            "pid_value": "1308777"
          }, 
          "is_last": true, 
          "last_child": {
            "pid_type": "recid", 
            "pid_value": "1308778"
          }
        }
      ]
    }, 
    "communities": [
      {
        "id": "invid-h2020"
      }
    ], 
    "grants": [
      {
        "code": "687786", 
        "links": {
          "self": "https://zenodo.org/api/grants/10.13039/501100000780::687786"
        }, 
        "title": "In Video Veritas \u2013 Verification of Social Media Video Content for the News Industry", 
        "acronym": "InVID", 
        "program": "H2020", 
        "funder": {
          "doi": "10.13039/501100000780", 
          "acronyms": [
            "EC"
          ], 
          "name": "European Commission", 
          "links": {
            "self": "https://zenodo.org/api/funders/10.13039/501100000780"
          }
        }
      }
    ], 
    "keywords": [
      "video/image concept annotation", 
      "deep learning", 
      "multi-task learning", 
      "structured outputs", 
      "multi-label learning", 
      "concept correlations", 
      "video analysis"
    ], 
    "publication_date": "2018-06-18", 
    "creators": [
      {
        "affiliation": "CERTH-ITI", 
        "name": "Markatopoulou, Foteini"
      }, 
      {
        "affiliation": "CERTH-ITI", 
        "name": "Mezaris, Vasileios"
      }, 
      {
        "affiliation": "Queen Mary University of London", 
        "name": "Patras, Ioannis"
      }
    ], 
    "access_right": "open", 
    "resource_type": {
      "subtype": "article", 
      "type": "publication", 
      "title": "Journal article"
    }
  }
}
51
36
views
downloads
Views 51
Downloads 36
Data volume 46.8 MB
Unique views 50
Unique downloads 34

Share

Cite as