August 23, 2018 Software Open Access

Amrith Krishna; Bishal Santra; Sasi Prasanth Bandaru; Gaurav Sahu; Vishnu Dutt Sharma; Pavankumar Satuluri; Pawan Goyal

Citation Style Language JSON Export

{
  "publisher": "Zenodo", 
  "DOI": "10.5281/zenodo.1035413", 
  "title": "Word Segmentation in Sanskrit Using Energy Based Models", 
  "issued": {
    "date-parts": [
      [
        2018, 
        8, 
        23
      ]
    ]
  }, 
  "abstract": "<p>This is the repository for word segmentation in sanskrit using energy based models.</p>\n\n<p>&nbsp;</p>\n\n<p># Word Segmentation in Sanskrit Using Energy Based Models<br>\n<br>\n&nbsp;<br>\n## Getting Started<br>\n&nbsp;<br>\nPlease download the 2 compressed files &#39;dir.zip&#39; and &#39;wordsegmentation.rar&#39; to your working directory and extract them into folders named &#39;dir&#39; and &#39;wordsegmentation&#39; respectively.<br>\n&nbsp;<br>\nYour working directory should be as follows<br>\n* Working Directory<br>\n&nbsp; * wordsegmentation<br>\n&nbsp;&nbsp;&nbsp; * skt_dcs_DS.bz2_4K_bigram_mir_10K<br>\n&nbsp;&nbsp;&nbsp; * skt_dcs_DS.bz2_4K_bigram_mir_heldout<br>\n&nbsp; * dir<br>\n&nbsp;<br>\n## Prerequisites<br>\n* Python3<br>\n&nbsp; * scipy<br>\n&nbsp; * numpy<br>\n&nbsp; * csv<br>\n&nbsp; * pickle<br>\n&nbsp; * multiprocessing<br>\n&nbsp; * bz2<br>\n## Instructions for Training<br>\nChange your current directory to &#39;dir&#39;<br>\n&nbsp;<br>\nRun the file Train_clique.py by using the following command<br>\n&nbsp;<br>\n* python Train_clique.py<br>\n&nbsp;<br>\nTo train on different input features like BM2,BM3,BR2,BR3,PM2,PM3,PR,PR3 please modify the bz2_input_folder value in the main function before beginning the training.<br>\n&nbsp;<br>\nFeature&nbsp; | bz2_input_folder<br>\n------------- | -------------<br>\nBM2 | wordsegmentation/skt_dcs_DS.bz2_4K_bigram_mir_10K/<br>\nBM3 | wordsegmentation/skt_dcs_DS.bz2_1L_bigram_mir_10K<br>\nBR2 | wordsegmentation/skt_dcs_DS.bz2_4K_bigram_rfe_10K/<br>\nBR3 | wordsegmentation/skt_dcs_DS.bz2_1L_bigram_rfe_10K/<br>\nPM2 | wordsegmentation/skt_dcs_DS.bz2_4K_pmi_mir_10K/<br>\nPM3 | wordsegmentation/skt_dcs_DS.bz2_1L_pmi_mir_10K2/<br>\nPR2 | wordsegmentation/skt_dcs_DS.bz2_4K_pmi_rfe_10K/<br>\nPR3 | wordsegmentation/skt_dcs_DS.bz2_1L_pmi_rfe_10K/<br>\n&nbsp;<br>\n## Instructions for Testing<br>\n&nbsp;<br>\nAfter training, please modify the &#39;modelList&#39; dictionary&nbsp; in &#39;test_clique.py&#39; with the name of the neural network that has been saved during training. While testing for a feature, please provide the name of the neural net which was trained for the same feature.<br>\n&nbsp;<br>\nWe only provide the trained model for the feature BM2 which was our best performing feature. If the name of the neural net is not changed, then the testing will be performed on the pre-trained model for BM2 provided in outputs/train_t7978754709018<br>\n&nbsp;<br>\nTo test with a particular feature vector use the tag of the feature while execution<br>\n&nbsp;<br>\n* python test_clique.py -t &lt;tag&gt;<br>\n&nbsp;<br>\nFor example: &nbsp;<br>\n&nbsp; * python test_clique.py -t BM2<br>\n&nbsp;<br>\nAfter finishing the testing please run the following command to see the precision and recall values for both the word and word++ prediction tasks<br>\n&nbsp;<br>\n* python evaluate.py &lt;tag&gt;<br>\n&nbsp;<br>\nFor example: &nbsp;<br>\n&nbsp; * python evaluate.py BM2</p>", 
  "author": [
    {
      "family": "Amrith Krishna"
    }, 
    {
      "family": "Bishal Santra"
    }, 
    {
      "family": "Sasi Prasanth Bandaru"
    }, 
    {
      "family": "Gaurav Sahu"
    }, 
    {
      "family": "Vishnu Dutt Sharma"
    }, 
    {
      "family": "Pavankumar Satuluri"
    }, 
    {
      "family": "Pawan Goyal"
    }
  ], 
  "id": "1035413", 
  "event-place": "Brussels, Belgium", 
  "type": "article", 
  "event": "Conference on Empirical Methods in Natural Language Processing (EMNLP`)"
}