Various 
=================

``bnlearn`` contains several examples within the library that can be used to practice with the functionalities of :func:`bnlearn.structure_learning`, :func:`bnlearn.parameter_learning` and :func:`bnlearn.inference`.


DataFrames
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In ``bnlearn``, there is one example dataset that can be imported; the **sprinkler** dataset. Note this dataset is readily one-hot, without missing values, and as such does not require any further pre-processing steps. The DAG example models (see Example DAG section) can however be converted from the model to a dataframe.


.. code-block:: python

   import bnlearn as bn
   # Import dataset
   df = bn.import_example()
   # Structure learning
   model = bn.structure_learning.fit(df)
   # Plot
   G = bn.plot(model)


DAG
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

``bnlearn`` contains several example Directed Acyclic Graphs:
	* 'sprinkler' (default)
	* 'alarm'
	* 'andes'
	* 'asia'
	* 'pathfinder'
	* 'sachs'
	* 'miserables'

Each DAG can be loaded using the :func:`bnlearn.bnlearn.import_DAG` function. With the :func:`bnlearn.bnlearn.sampling` function a ``DataFrame`` can be created for *n* samples.
The sprinkler DAG is a special case because it is not loaded from a *bif* file but created manually. Therefore, the **sprinkler** model can be generated with(out) a CPD by: ``CPD=False``.


.. code-block:: python
   
   import bnlearn as bn
   # Import dataset
   DAG = bn.import_DAG('sachs', CPD=True)
   # plot the keys of the DAG
   DAG.keys()
   # dict_keys(['model', 'adjmat'])

   # The model contains the BayesianNetwork with the CPDs.
   # The adjmat contains the adjacency matrix with the relationships between the nodes.

   # plot ground truth
   G = bn.plot(DAG)

   # Sampling
   df = bn.sampling(DAG, n=1000)




Import from BIF
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Each Bayesian DAG model that is loaded with :func:`bnlearn.bnlearn.import_DAG` is derived from a *bif* file. The *bif* file is a common format for Bayesian networks that can be used for the exchange of knowledge and experimental results in the community. More information can be found (here)[http://www.cs.washington.edu/dm/vfml/appendixes/bif.htm].


.. code-block:: python

	import bnlearn as bn

	bif_file= 'sprinkler'
	bif_file= 'alarm'
	bif_file= 'andes'
	bif_file= 'asia'
	bif_file= 'pathfinder'
	bif_file= 'sachs'
	bif_file= 'miserables'
	bif_file= 'filepath/to/model.bif'

	# Loading example dataset
	model = bn.import_DAG(bif_file)


Start with RAW data
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


Lets demonstrate by example how to process your own dataset containing mixed variables. I will demonstrate this by the titanic case. This dataset contains both continues as well as categorical variables and can easily imported using :func:`bnlearn.bnlearn.import_example`.
With the function :func:`bnlearn.bnlearn.df2onehot` it can help to convert the mixed dataset towards a one-hot matrix. The settings are adjustable, but by default the unique non-zero values must be above 80% per variable, and the minimal number of samples must be at least 10 per variable.


.. code-block:: python

   import bnlearn as bn
   # Load titanic dataset containing mixed variables
   df_raw = bn.import_example(data='titanic')
   # Pre-processing of the input dataset
   dfhot, dfnum = bn.df2onehot(df_raw)
   # Structure learning
   DAG = bn.structure_learning.fit(dfnum)
   # Plot
   G = bn.plot(DAG)

.. _fig-titanic:

.. figure:: ../figs/fig_titanic.png


From this point we can learn the parameters using the DAG and input dataframe.

.. code-block:: python

   # Parameter learning
   model = bn.parameter_learning.fit(DAG, dfnum)

Finally, we can start making inferences. Note that the variable and evidence names should exactly match the input data (case sensitive).

.. code-block:: python

   # Print CPDs
   bn.print_CPD(model)
   # Make inference
   q = bn.inference.fit(model, variables=['Survived'], evidence={'Sex':0, 'Pclass':1})
   
   print(q.df)
   print(q._str())
   

.. table::

     +-------------+-----------------+
     | Survived    |   phi(Survived) |
     +=============+=================+
     | Survived(0) |          0.3312 |
     +-------------+-----------------+
     | Survived(1) |          0.6688 |
     +-------------+-----------------+



Structure learning
=========================

A different, but quite straightforward approach to build a DAG from data is to identify independencies in the data set using hypothesis tests, such as chi2 test statistic. The p_value of the test, and a heuristic flag that indicates if the sample size was sufficient. The p_value is the probability of observing the computed chi2 statistic (or an even higher chi2 value), given the null hypothesis that X and Y are independent given Zs. This can be used to make independence judgements, at a given level of significance.


Example (1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: python
   
   import bnlearn as bn
   # Load dataframe
   df = bn.import_example()
   # Learn structure
   model = bn.structure_learning.fit(df)
   # adjacency matrix:
   model['adjmat']

   # print
   print(model['adjmat'])


Reading the table from left to right we see that Cloudy is connected to Sprinkler and also to Rain in a directed manner.
Sprinkler is connect to Wet_grass.
Rain is connected to Wet_grass.
Wet_grass is connected to nothing.


.. table::
  
  +-----------+--------+-----------+-------+-----------+
  |           | Cloudy | Sprinkler | Rain  | Wet_Grass |
  +===========+========+===========+=======+===========+
  | Cloudy    | False  | True      | True  | False     |
  +-----------+--------+-----------+-------+-----------+
  | Sprinkler | False  | False     | False | True      |
  +-----------+--------+-----------+-------+-----------+
  | Rain      | False  | False     | False | True      |
  +-----------+--------+-----------+-------+-----------+
  | Wet_Grass | False  | False     | False | False     |
  +-----------+--------+-----------+-------+-----------+



Example (2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For this example, we will be investigating the sprinkler data set. This is a very simple data set with 4 variables and each variable can contain value [1] or [0]. The question we can ask: What are the relationships and dependencies across the variables? Note that his data set is already pre-processed and no missing values are present.


Let's bring in our dataset.

.. code-block:: python

  import bnlearn as bn
  df = bn.import_example()
  df.head()


.. table::

  +--------+-----------+------+-------------+
  |Cloudy  | Sprinkler | Rain |  Wet_Grass  |
  +========+===========+======+=============+
  |    0   |      1    |  0   |      1      |
  +--------+-----------+------+-------------+
  |    1   |      1    |  1   |      1      |
  +--------+-----------+------+-------------+
  |    1   |      0    |  1   |      1      |
  +--------+-----------+------+-------------+
  |    ... |      ...  | ...  |     ...     |
  +--------+-----------+------+-------------+
  |    0   |      0    |  0   |      0      |
  +--------+-----------+------+-------------+
  |    1   |      0    |  0   |      0      |
  +--------+-----------+------+-------------+
  |    1   |      0    |  1   |      1      |
  +--------+-----------+------+-------------+

From the ``bnlearn`` library, we'll need the :class:`~bnlearn.structure_learning.fit` for this exercise:

.. code-block:: python

   import bnlearn as bn
    model = bn.structure_learning.fit(df)
    G = bn.plot(model)


.. |logo3| image:: ../figs/fig_sprinkler_sl.png
    :scale: 60%

.. table:: Learned structure on the Sprinkler data set.
   :align: center

   +---------+
   | |logo3| |
   +---------+
   
      

We can specificy the method and scoring type. As described previously, some methods are more expensive to run then others. Make the decision on the number of variables, hardware in your machine, time you are willing to wait etc

**Method types:**

* hillclimbsearch or hc (greedy local search if many more nodes are involved)
* exhaustivesearch or ex (exhaustive search for very small networks)
* constraintsearch or cs (Constraint-based Structure Learning by first identifing independencies in the data set using hypothesis test, chi2)

**Scoring types:**

* bic
* k2
* bdeu


.. code-block:: python

    import bnlearn as bn
    model_hc_bic  = bn.structure_learning.fit(df, methodtype='hc', scoretype='bic')
    model_hc_k2   = bn.structure_learning.fit(df, methodtype='hc', scoretype='k2')
    model_hc_bdeu = bn.structure_learning.fit(df, methodtype='hc', scoretype='bdeu')
    model_ex_bic  = bn.structure_learning.fit(df, methodtype='ex', scoretype='bic')
    model_ex_k2   = bn.structure_learning.fit(df, methodtype='ex', scoretype='k2')
    model_ex_bdeu = bn.structure_learning.fit(df, methodtype='ex', scoretype='bdeu')



Example (3)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Lets learn the structure of a more complex data set and compare it to another one.

.. code-block:: python

    import bnlearn as bn
    # Load asia DAG
    model_true = bn.import_DAG('asia')
    # plot ground truth
    G = bn.plot(model_true)

.. _fig2a_asia_groundtruth:

.. figure:: ../figs/fig2a_asia_groundtruth.png

  True DAG of the Asia data set.

  
.. code-block:: python

    # Sampling
    df = bn.sampling(model_true, n=10000)
    # Structure learning of sampled dataset
    model_learned = bn.structure_learning.fit(df, methodtype='hc', scoretype='bic')

.. _fig2b_asia_structurelearning:

.. figure:: ../figs/fig2b_asia_structurelearning.png

  Learned DAG based on data set.


.. code-block:: python

    # Plot based on structure learning of sampled data
    bn.plot(model_learned, pos=G['pos'])
    # Compare networks and make plot
    bn.compare_networks(model_true, model_learned, pos=G['pos'])

.. _fig2c_asia_comparion:

.. figure:: ../figs/fig2c_asia_comparion.png
.. figure:: ../figs/fig2d_confmatrix.png

  Comparison True vs. learned DAG.


Parameter learning
=========================

Extracting adjacency matrix after Parameter learning:

.. code-block:: python
   
    import bnlearn as bn
    # Load dataframe
    df = bnlearn.import_example()
    # Import DAG
    DAG = bnlearn.import_DAG('sprinkler', CPD=False)
    # Learn parameters
    model = bnlearn.parameter_learning.fit(DAG, df)
    # adjacency matrix:
    model['adjmat']

    # print
    print(model['adjmat'])

.. table::
  
  +-----------+--------+-----------+-------+-----------+
  |           | Cloudy | Sprinkler | Rain  | Wet_Grass |
  +===========+========+===========+=======+===========+
  | Cloudy    | False  | True      | True  | False     |
  +-----------+--------+-----------+-------+-----------+
  | Sprinkler | False  | False     | False | True      |
  +-----------+--------+-----------+-------+-----------+
  | Rain      | False  | False     | False | True      |
  +-----------+--------+-----------+-------+-----------+
  | Wet_Grass | False  | False     | False | False     |
  +-----------+--------+-----------+-------+-----------+




Create a Bayesian Network, learn its parameters from data and perform the inference
====================================================================================================

Lets make an example were we have data with many measurements, and we have expert information of the relations between nodes. 
Our goal is to create DAG on the expert knowledge and learn the CPDs. To showcase this, I will use the sprinkler example.

Import example dataset of the sprinkler dataset.

.. code-block:: python

    import bnlearn as bn
    df = bn.import_example('sprinkler')
    print(tabulate(df.head(), tablefmt="grid", headers="keys"))

.. table::

     +----+----------+-------------+--------+-------------+
     |    |   Cloudy |   Sprinkler |   Rain |   Wet_Grass |
     +====+==========+=============+========+=============+
     |  0 |        0 |           0 |      0 |           0 |
     +----+----------+-------------+--------+-------------+
     |  1 |        1 |           0 |      1 |           1 |
     +----+----------+-------------+--------+-------------+
     |  2 |        0 |           1 |      0 |           1 |
     +----+----------+-------------+--------+-------------+
     |  3 |        1 |           1 |      1 |           1 |
     +----+----------+-------------+--------+-------------+
     |  4 |        1 |           1 |      1 |           1 |
     +----+----------+-------------+--------+-------------+
     | .. |      ... |         ... |    ... |         ... |
     +----+----------+-------------+--------+-------------+


Define the network structure. This can be based on expert knowledge.

.. code-block:: python

    edges = [('Cloudy', 'Sprinkler'),
             ('Cloudy', 'Rain'),
             ('Sprinkler', 'Wet_Grass'),
             ('Rain', 'Wet_Grass')]

Make the actual Bayesian DAG

.. code-block:: python
    
    import bnlearn as bn
    DAG = bn.make_DAG(edges)
    # [BNLEARN] Bayesian DAG created.

    # Print the CPDs
    bn.print_CPD(DAG)
    # [BNLEARN.print_CPD] No CPDs to print. Use bnlearn.plot(DAG) to make a plot.

Plot the DAG

.. code-block:: python

    bnlearn.plot(DAG)


.. _fig-DAG-sprinkler:

.. figure:: ../figs/DAG_sprinkler.png

Parameter learning on the user-defined DAG and input data using maximumlikelihood.

.. code-block:: python
    
    DAG = bn.parameter_learning.fit(DAG, df, methodtype='maximumlikelihood')


Lets print the learned CPDs:

.. code-block:: python

    bn.print_CPD(DAG)
    
    # [BNLEARN.print_CPD] Independencies:
    # (Cloudy _|_ Wet_Grass | Rain, Sprinkler)
    # (Sprinkler _|_ Rain | Cloudy)
    # (Rain _|_ Sprinkler | Cloudy)
    # (Wet_Grass _|_ Cloudy | Rain, Sprinkler)
    # [BNLEARN.print_CPD] Nodes: ['Cloudy', 'Sprinkler', 'Rain', 'Wet_Grass']
    # [BNLEARN.print_CPD] Edges: [('Cloudy', 'Sprinkler'), ('Cloudy', 'Rain'), ('Sprinkler', 'Wet_Grass'), ('Rain', 'Wet_Grass')]

CPD of Cloudy:
    +-----------+-------+
    | Cloudy(0) | 0.494 |
    +-----------+-------+
    | Cloudy(1) | 0.506 |
    +-----------+-------+

CPD of Sprinkler:
    +--------------+--------------------+--------------------+
    | Cloudy       | Cloudy(0)          | Cloudy(1)          |
    +--------------+--------------------+--------------------+
    | Sprinkler(0) | 0.4807692307692308 | 0.7075098814229249 |
    +--------------+--------------------+--------------------+
    | Sprinkler(1) | 0.5192307692307693 | 0.2924901185770751 |
    +--------------+--------------------+--------------------+

CPD of Rain:
    +---------+--------------------+---------------------+
    | Cloudy  | Cloudy(0)          | Cloudy(1)           |
    +---------+--------------------+---------------------+
    | Rain(0) | 0.6518218623481782 | 0.33695652173913043 |
    +---------+--------------------+---------------------+
    | Rain(1) | 0.3481781376518219 | 0.6630434782608695  |
    +---------+--------------------+---------------------+

CPD of Wet_Grass:
    +--------------+--------------------+---------------------+---------------------+---------------------+
    | Rain         | Rain(0)            | Rain(0)             | Rain(1)             | Rain(1)             |
    +--------------+--------------------+---------------------+---------------------+---------------------+
    | Sprinkler    | Sprinkler(0)       | Sprinkler(1)        | Sprinkler(0)        | Sprinkler(1)        |
    +--------------+--------------------+---------------------+---------------------+---------------------+
    | Wet_Grass(0) | 0.7553816046966731 | 0.33755274261603374 | 0.25588235294117645 | 0.37910447761194027 |
    +--------------+--------------------+---------------------+---------------------+---------------------+
    | Wet_Grass(1) | 0.2446183953033268 | 0.6624472573839663  | 0.7441176470588236  | 0.6208955223880597  |
    +--------------+--------------------+---------------------+---------------------+---------------------+

Lets make an inference:

.. code-block:: python
    
    q1 = bn.inference.fit(DAG, variables=['Wet_Grass'], evidence={'Rain':1, 'Sprinkler':0, 'Cloudy':1})

    +--------------+------------------+
    | Wet_Grass    |   phi(Wet_Grass) |
    +==============+==================+
    | Wet_Grass(0) |           0.2559 |
    +--------------+------------------+
    | Wet_Grass(1) |           0.7441 |
    +--------------+------------------+

Print the values:

.. code-block:: python
    
    print(q1.df)
    # array([0.25588235, 0.74411765])


.. raw:: html

	<hr>
	<center>
		<script async type="text/javascript" src="//cdn.carbonads.com/carbon.js?serve=CEADP27U&placement=erdogantgithubio" id="_carbonads_js"></script>
	</center>
	<hr>

