# ARPIP: Ancestral Sequence Reconstruction with insertions and deletions under the Poisson Indel Process

ARPIP is an Ancestral Sequence Reconstruction software - a method for Ancestral Sequence Reconstruction that uses 
Poisson Indel Process (Bouchard-Côté & Jordan, PNAS, 2013) to model insertions and deletions on a phylogenetic tree 
assuming independence among sites. ARPIP consists of two main algorithms. The first algorithm (IndelPoints) infers for
each site the insertion and deletion points on the tree topology by highest probability. The second algorithm reconstructs 
ancestral characters on the pruned subtrees in a manner similar to FastML (Pupko et al. , Mol. Biol. Evol, 2000). 
---
#### Data used in this study

This folder and its sub-folders contain the data used in ARPIP paper. 


The supplemental data contains 4 folders namely:
#### Javapip simulated data -100 replication with int1 el100 tr2
The simulated data set contains 100 MSA/tree replicates.  Each replicate was simulated using an 8 taxa tree with a topology sampled from the uniform distribution 
	and branch lengths sampled from an exponential distribution with the rate 2. The process intesity is 1 and the expected sequence length is 100. The naming mechanism of files is as follow: sim-[0...99]+
	
	-input:
	* .newick: The original phylogenetic tree produced by JavaPIP.
	* .fasta: The orginal MSA produced by simulator.
	
	-true:
	* _events_javaPIP: The log of true events happend during the tree/msa production. Please not some events happend before the root of the tree and were excluded in this file.  
		For more information about the process please read the PIP orginal paper (Bouchard-Côté & Jordan, PNAS, 2013). In recap: there are some events that they happened before 
		the tree root and the got extinct before root, therefore, there are nothing observable. In this file the event are like this: "node: type of the event: number of event blongs to this event".
		For example in sim-0_no_stem_events_javaPIP: 'internal_1:I:7' means it would be insertion (denoted by 'I') at 'internal_1' in the next 7 events are followed by this event.
		Next event (a member of that set) internal_1:S:L means at 'internal_1' there was a Substitution (denoted by 'S') and the char state was 'L'.
		Next event (still a member of that set, it would last till the next insertion denote by 'I') leaf_1:S:L means the char was inserted before under process substitution ('S') at node was observed with
		char state 'L'. Please note that this file is extracted from ''_events" file which could be provided on request.
	* _msa_internal.fasta: The true ancestral states produce by JavaPIP. Please note that this file is build based on the events and other files which is excluded from this section. These data could be provided on request.
	
	-ouput:
	* _infered.fasta: The reconstructed ancestor seqeunces using ARPIP algorithm.  
	* _mlindel: The inferred indel events using ARPIP alogrithm. In the mentioned file there are two types of files: a single insertion (12:I;) which defines 
		insertion at node 12 and  one insertion, multiple deletions(1:X;3:X;12:X;14:I;) which the deletion events indecated by 'X'.

	* _new.newick: The original phylogenetic tree produced by JavaPIP but the internal nodes (ancestors) are renamed with digits. 
			(mainly because some code cannot handle internal nodes with char name. Ps: word root is not there but the tree is stil rooted.)

#### INDELible simulated data - 200 replication with indel are 0.01 and 0.05
The data simulated by INDELible contains two sets of 100 MSA/tree replicas. Each replica was simulated using an 8 taxa tree from PIP simulations. 
We used the Zipfian (power law) distribution for the indel model with $a=1.7$, to generate the samples where $a>1$ is the exponent characterizing the distribution.
The maximum indel length was set to 5 to avoid MSAs with excessively long gaps. Two different indel rates of $0.01$ and $0.05$ were used for the simulation with INDELible.

	-input:
	*.txt: The phylogenetic tree used by INDELible with the internal nodes name been added.
	* .newick: The original phylogenetic tree produced by JavaPIP and used by simulator.
	*_1.fas: The raw sequences generate by simulator.
	*_TRUE_1_new_msa.fas: The orginal MSA produced by simulator.
	*_ANCESTRAL_1_new.fas: True ancestral values generate by simulator.
	
	
	-output:
	* _anc.fasta: The reconstructed ancestor seqeunces using ARPIP algorithm.  
	* _node_rel.txt: The reconstructed phylogeny in ancestral format in which the parent node and the children nodes of each node of the tree are specified.
	* _new.newick: The original phylogenetic tree produced by JavaPIP but the internal nodes (ancestors) are renamed with digits. 	
	
	
#### Data used for figures Covid ARPIP and FastML:
This folder contains the data we used in comparison of ARPIP with FastML on Covid dataset. The two sub-folders here are input and output.
	
	-input:
	-Covid PRANK: Contains config file, MSA from PRANK aligner and the tree reconstructed using PhyML 3.0. on the mentioned MSA.
	-Covid ProPIP: Contains config file, MSA from ProPIP aligner and the tree reconstructed using PhyML 3.0. on the mentioned MSA
	
	-output:
	-ARPIP Covid PRANK/ProPIP: 
		*_anc.fasta: The reconstructed ancestor seqeunces using ARPIP algorithm.  
		*_mlindelpoints: The inferred indel events using ARPIP alogrithm. In the mentioned file there are two types of files: a single insertion (12:I;) which defines 
		insertion at node 12 and  one insertion, multiple deletions(1:X;3:X;12:X;14:I;) which the deletion events indecated by 'X'.
		*_node_rel.txt: This file is defining the internal nodes name and their relation to other nodes.
		*_tree.nwk: New rooted tree based on reconstructed tree from PhyML 3.0.

	-FastML Covid PRANK/ProPIP: Contains output of FastML webserver for ancestral reconstrution combined with the result of FastML stand alone code for indel reconstruction,
	on the MSA from PRANK aligner and the tree reconstructed using PhyML 3.0. on the mentioned MSA. 
		*_just_ancestors.fasta: The reconstructed ancestor seqeunces using FastML algorithm. 
		*_prank.fasta/_propip.fasta: ancstors+leaves 
		*_tree.newick: The original tree with optimized branch lengths.
		* For FastML's output please check: http://fastml.tau.ac.il/sourceNew.php


#### Data used for figures simulated comparison ARPIP FasML PAML wgap-wogap:
This folder contains two subfolders: input and output:
	Input: this folder is contained two folders: input_wgap and input_wogap sim-0 data with and without gappy sites.
	-input_wgap: 
		*sim-0_tree.newick: The phylogenetic tree produced by JavaPIP simulator.
		*sim-0_msa.fasta: The MSA produced by JavaPIP simulator.
		*sim-0_msa_internal.fasta: The true internal states of mentioned MSA extracted from simulator.
	-input_wgap: 
		*sim-0_tree.newick: The phylogenetic tree produced by JavaPIP simulator.
		*sim-0_msa_no_gap.fasta: The MSA produced by JavaPIP simulator which gappy columns (sites) were eliminated. 
		*sim-0_msa_internal_no_gap.fasta: The true internal states of mentioned MSA.
	-output:
	- output-arpip-wogap:
		*sim-0_inferred.fasta: The reconstructed ancestor seqeunces using ARPIP algorithm. 
		*sim-0_mlindels: The inferred indel events using ARPIP alogrithm. In this file the 'I' indicates insertion event which is written after the node's name and 'X' indicates deletion with the same pattern.
	- output-arpip-wgap:
		*sim-0_inferred.fasta: The reconstructed ancestor seqeunces using ARPIP algorithm on sequences which the columns containing gap were excluded. 
		*sim-0_mlindels: The inferred indel events using ARPIP alogrithm. In this file the 'I' indicates insertion event which is written after the node's name and 'X' indicates deletion with the same pattern.
	- output-fastml-wgap:
		*FastML_simulator_just_ancestors.fasta:  The reconstructed ancestor seqeunces using FastML algorithm.
		*FastML_tree_ancestors.txt: The file is defining the internal nodes name and their relation to other nodes.
	- output-fastml-wogap:
		*FastML_simulator_just_ancestors.fasta:  The reconstructed ancestor seqeunces using FastML algorithm which the columns containing gap were excluded. 
		*FastML_tree_ancestors.txt: The file defining the internal nodes name and their relation to other nodes.
	- output-paml-wogap: 
		*paml_sim-0_msa_internal.fasta: The reconstructed ancestor seqeunces using PAML algorithm. Please not that PAML works on nongappy sequences. 

---
#### Citation
Please cite our paper:

Gholamhossein Jowkar, Julia Pecerska, Manuel Gil, and Maria Anisimova  
ARPIP: Ancestral sequence Reconstruction with insertions and deletions under the Poisson Indel Process. 
Systematic Biology (2022)

---
##### Refference
[1] Alexandre Bouchard-Côté and Michael I. Jordan. Evolutionary inference via the Poisson Indel Process.Proc. Natl. Acad. Sci. U.S.A., 110(4):1160, 2013.
[2] Tal Pupko and et al. A fast algorithm for joint reconstruction of ancestral amino acid sequences Molecular biology and evolution. 2000. 17, 6. 890–896.

---
#### Support
In case of bugs or improvement suggestions feel free to:
    
- Write to [Gholam-Hossein Jowkar](mailto:jowk@zhaw.ch)
