SNOR v1: Structured and Normalized OpenReview Reviews and Comments

38,262 linked records from OpenReview submissions to the ICLR (2017-2025) and Neurips (2021-2025) Machine Learning conferences. Paper submissions are linked to Semantic Scholar, a dynamic academic graph. Also provided are 462,995 structured comments from reviewers. Semantic Scholar includes information about authors, citations, and a variety of other metadata. By default, linked rows contain citation counts, venue information, Specter embeddings and author ids - but other information is easily retrievable using the SemanticScholar API.

Example Record (loaded as a Pandas dataframe row)

      id                                                               BkbY4psgg
      semantic_scholar_id               6b024162f81e8ff7aa34c3a43d601a912d012c78
      raw_decision                                                ICLR 2017 Oral
      normalized_decision                                                   Oral
      title                    Making Neural Programming Architectures Genera...
      abstract                 Empirically, neural networks that attempt to l...
      keywords                                                   [Deep learning]
      accepted                                                              True
      publication_venue        International Conference on Learning Represent...
      publication_venue_id                  939c6e1d-0d17-4d6e-8a82-66d960df0e40
      url                      https://www.semanticscholar.org/paper/6b024162...
      citation_count                                                         146
      embedding                [-0.0735881552, 0.3261716962, -0.3699628115, -...
      authors                              [Jonathon Cai, Richard Shin, D. Song]
      authorIds                                   [2350111, 39428234, 143711382]
      conference_year                                                       2017
      conference_name                                                       iclr
      conf_id                                                           iclr2017
      review_scores                                              [8.0, 9.0, 8.0]
      review_score_avg                                                  8.333333
      review_confidences                                         [8.0, 9.0, 8.0]
      review_confidence_avg                                                  4.0

In addition to the paper information, there are also 462,995 structured comments from reviewers. These comments include references to papers, anonymous author signatures, and arbitrary content (typically in the form of title:content blocks which render in OpenReview). Reviews are distingushed from other comments by the 'is_review' field, which is set to True for reviews. These comments will also have a numeric rating and confidence score. Finally, all comments have a reply_to_id field, which links to the id of the paper that the comment is replying to. Review comments will have a reply_to_id that links to the id of the paper they are reviewing. 

Example comment:

        {'conference_id': 'iclr2017',
        'paper_id': 'B1jnyXXJx',
        'comment_id': 'BJPZL-vmx',
        'signature': 'ICLR.cc/2017/conference/paper4/AnonReviewer1',
        'content': {
          'title': 'hyperparameter optimization and momentum vs CPN',
          'question': "The hyperparameters of gradient descent seem to be chosen once and fixed. 
                       Would optimizing the gradient descent hyperparameters lead to equivalent 
                       performance as the CPN method?\n\nFollowing up on another reviewer's 
                       question: CPN seems closely related to momentum. Can you provide a clear 
                       example to show how CPN is qualitatively distinct from momentum? (I believe
                       it is, but this could be clarified further in the paper)"
          },
        'reply_to_id': 'B1jnyXXJx',
        'is_review': False,
        'rating': None,
        'numeric_rating': None,
        'confidence': None,
        'numeric_confidence': None}