Software Open Access

Paradoxes of probabilistic programming, and how to condition on events of measure zero with infinitesimal probabilities

Jules Jacobs

DataCite XML Export

<?xml version='1.0' encoding='utf-8'?>
<resource xmlns:xsi="" xmlns="" xsi:schemaLocation="">
  <identifier identifierType="DOI">10.5281/zenodo.4075076</identifier>
      <creatorName>Jules Jacobs</creatorName>
      <affiliation>TU Delft</affiliation>
    <title>Paradoxes of probabilistic programming, and how to condition on events of measure zero with infinitesimal probabilities</title>
    <date dateType="Issued">2020-10-09</date>
  <resourceType resourceTypeGeneral="Software"/>
    <alternateIdentifier alternateIdentifierType="url"></alternateIdentifier>
    <relatedIdentifier relatedIdentifierType="DOI" relationType="IsVersionOf">10.5281/zenodo.4075075</relatedIdentifier>
    <rights rightsURI="">Creative Commons Attribution 4.0 International</rights>
    <rights rightsURI="info:eu-repo/semantics/openAccess">Open Access</rights>
    <description descriptionType="Abstract">&lt;p&gt;Abstract Probabilistic programming languages allow programmers to write down conditional probability distributions that represent statistical and machine learning models as programs that use observe statements. These programs are run by accumulating likelihood at each observe statement, and using the likelihood to steer random choices and weigh results with inference algorithms such as importance sampling or MCMC. We argue that naive likelihood accumulation does not give desirable semantics and leads to paradoxes when an observe statement is used to condition on a measure-zero event, particularly when the observe statement is executed conditionally on random data. We show that the paradoxes disappear if we explicitly model measure-zero events as a limit of positive measure events, and that we can execute these type of probabilistic programs by accumulating infinitesimal probabilities rather than probability densities. Our extension improves probabilistic programming languages as an executable notation for probability distributions by making it more well-behaved and more expressive, by allowing the programmer to be explicit about which limit is intended when conditioning on an event of measure zero.&lt;/p&gt;</description>
    <description descriptionType="Other">Since it is a single file of code I did not make a virtual machine. If you think it is better to make a virtual machine I can still do so. I will remove this notice eventually.</description>
All versions This version
Views 130130
Downloads 3232
Data volume 379.5 kB379.5 kB
Unique views 112112
Unique downloads 3232


Cite as