Introduction

The RIMES-database (Reconnaissance et Indexation de données Manuscrites et de fac similÉS / Recognition and Indexing of handwritten documents and faxes) comprises handwritten correspondence letters, in French, “sent” by individuals to companies or administrations; all correspondence is fictitious and there is no PII in the records. 

The database was collected by asking volunteers to write handwritten letters in exchange for gift vouchers. Volunteers were given a fictitious identity (same sex as the real one) and up to 5 scenarios. Each scenario was chosen from among 9 realistic topics: change of personal data (address, bank account), request for information, opening and closing (customer account), change of contract or order, complaint (e.g. poor quality of service), payment difficulties (request for delay, tax exemption, etc.), reminder, claim with other circumstances and a target (administrations or service providers such as telephone, electricity, bank, insurance companies). The volunteers wrote a letter with this information in their own words. The layout was free and the only request was to use white paper and to write legibly in black ink.

Contents

There are several files in this record; they correpond to the whole dataset and also some subsets that were created during the project. 

The Communications part is where the full document images and their annotations are stored. We give some detail of what the annotations comprise and briefly describe the other subsets; NB there was a subset comprising images from the cropped logos, which are not distributed here, due to some issues with the annotations.

Communications  -- Images_Courriers.zip

There are in total 5605 communications, each containing from 2 to 3 pages:

• One correspondence (mandatory)
• One questionnaire (mandatory)
• One fax (optional)

Filenames are constituted as follows:

• Communication number [1, 5605]
• Underscore “_”
• One letter [F, L, Q]
  • L for correspondence/Letter (Lettre, in French)
  • Questionnaire
  • Fax

The images and the corresponding annotations are split into 3 folders:

• DVD1: images from 1 to 1799
• DVD2: images from 1800 to 3699
• DVD3: images from 3700 to 5605

There are in total 12610 images.

The annotation files are in xml format and support different tasks:

• Document Structure Identification
• Handwritten text recognition
• Writer recognition
• Information Extraction

Document Structure Identification

For the document structure, there are 8 "types" to be identified in the Letters for the different blocks of text in the image:

1. Sender address
2. Recipient address
3. Date/Place (each is annotated with it's own tag)
4. Subject
5. Introduction (Ouverture in French)
6. Text body
7. Signature
8. PS / annex

Types 1 and 2 have further details, if it is a person or a corporate entity and address can also contain telephone/fax number.

For example:

Handwritten text recognition

There are annotations for the paragraphs; line breaks are indicated with "\n". The transcriptions are verbatim and contain the same spelling and grammar errors that could be seen in the pages. When there could be more than one possible spelling (j’essaie/j’essaye, événement/évènement, ultrason/ultra son, and writing errors), the options are in the ground truth following a special construction: 

Writer recognition

There is an identity for each writer in the database, so the usual tasks of identification and verification can be realized. The writer is identified in the usual French form, with family name, in all caps, first followed by the given name, for instance:

Paragraphs -- images_blocs_de_texte.zip

The main body of the letters was cropped from the full page images and stored as grayscale JPEG images and split into 3 folders:

• DVD1: images from 1 to 1799 (1796 images)
• DVD2: images from 1800 to 3699 (1899 images)
• DVD3: images from 3700 to 5605 (1905 images)

The transcriptions can be obtained from the corresponding Communications transcriptions; the numbers in the filenames correspond to the communications.

Cursive words -- imagettes_mots_cursif.zip

The paragraphs were split into lines and each line was further split into words.

The images were split into 57 blocks (lot in French) organized in folders named:

• lot_N_rimes_version_definitive, where N is the block number [1, 57]

Each folder has data from 100 letters, further organized into sub-folders following the convention:

• <Communication number>_L

Then each sub-folder has one image per word, with naming:

• <Communication number>_L_<Line Number>_<Word position>.tiff

Where Line Number and Word position start from 0. The transcription should be inferred from the corresponding Communications transcriptions.

Character snippets -- imagettes_caracteres.zip

Words were split to characters (A to Z) and digits (0 to 9), totalling 95269 images. They are distributed in 3 folders:

• characters_rimes_DVD1
• characters_rimes_DVD2
• characters_rimes_DVD3

Each is further divided into 4 blocks (lot in French):

• lot_1
• lot_2
• lot_3
• lot_4

The naming of the image files follows the following:

• <Class>_<Correspondance number>_<character position in the image>.png

Where Class is in [A-Z0-9].

Acknowledgments 

This dataset was originally collected and prepared in 2007 by the following partners: DGA/CTA/DT/GIP - CEP Arcueil; TSP – ARTEMIS Télécom SudParis; and A2iA SA, as part of the Techno-Vision project. This project was funded by the French ministries for Research and Defense (Ministère de la Recherche and Ministère de la Défense).

After the acquisition of A2iA SA in September 2018,  Mitek Systems, Inc. became a legal owner of the dataset, and decided to release it publicly – which was one of the objectives of the project after its conclusion – under a permissive license in 2024, to encourage open science.