ZTF Source Classification Project

There is a newer version of the record available.

Published April 2024 | Version 0.1.0
Dataset Open

The data for "The ZTF Source Classification Project: III. A Catalog of Variable Sources"

Authors/Creators

  • 1. University of Minnesota
  • 2. ROR icon California Institute of Technology
  • 3. ROR icon University of Amsterdam
  • 4. ROR icon University of Washington
  • 5. ROR icon Pacific Lutheran University
  • 6. ROR icon Indian Institute of Technology Gandhinagar
  • 7. ROR icon University of California, Berkeley
  • 8. ROR icon Lawrence Berkeley National Laboratory
  • 9. ROR icon Carnegie Mellon University
  • 10. ROR icon Jet Propulsion Laboratory

Description

The classification of variable objects provides insight into a wide variety of astrophysics ranging from stellar interiors to galactic nuclei. The Zwicky Transient Facility (ZTF) provides time series observations that record the variability of more than a billion sources. The scale of these data necessitates automated approaches to make a thorough analysis. Building on previous work, this paper reports the results of the ZTF Source Classification Project (SCoPe), which trains neural network and XGBoost machine learning (ML) algorithms to perform dichotomous classification of variable ZTF sources using a manually constructed training set containing 170,632 light curves. We find that several classifiers achieve high precision and recall scores, suggesting the reliability of their predictions for 236,607,991 light curves across 92 ZTF fields. We also identify the most important features for XGB classification and compare the performance of the two ML algorithms, finding a pattern of higher precision among XGB classifiers. The resulting classification catalog is available to the public, and the software developed for SCoPe is open-source and adaptable to future time-domain surveys.

Notes

 

Notes for 0.1.0

  1. Fields are now seperated by hour in Right Ascension (RA). Fields  with centers between N-1 and N hours RA are located in N_prediction_xgb_dnn_fields.zip.  For example a field with a center of 3.7 hours RA would be located in 4_prediction_xgb_dnn_fields.zip
  2. The seperation is done by the location of the field center so there may be parts of fields that are not in the within the RA indicated. Field centers and corners can be found on this GitHub page.
  3. fields.json now denotes which fields with classifications are contained in each folder.

 

------------------------------------------------------------------------------------

Notes for 0.0.4

  1. All fields are now contained (in CSV format) within a single predictions_dnn_xgb_92_fields.zip file.

------------------------------------------------------------------------------------

Notes for 0.0.3

  1. All fields are now contained (in CSV format) within a single predictions_dnn_xgb_77_fields.zip file.

 

------------------------------------------------------------------------------------

Notes for 0.0.2

  1.  From Version 0.0.1 to 0.0.2, classification predictions have been updated by the inclusion of the Gaia parallax error in the set of ontological features. Any fields downloaded from Version 0.0.1 should thus be replaced by those from Version 0.0.2.
  2. The structure of the catalog has been updated - all fields are now contained (in CSV format) within a single predictions_dnn_xgb_70_fields.zip file.
  3. A demo file containing 100 rows of predictions along with a Jupyter notebook to explain the columns are included in this release. See the scope-ml repository for details on setting up the environment needed to run this notebook.
  4. The new fields.json file contains a list of all fields having classification predictions.

Files

field_296_100rows.csv

Files (15.6 GB)

Name Size
md5:e750ddc1e1436842f427bdb2e76e03b8
36.3 MB Preview Download
md5:43576f67fe020d3802f03c5afaf0534c
206.7 MB Preview Download
md5:28de54edb42ac9db2c0ce5faa9fe1793
170.5 MB Preview Download
md5:8b3192f65d0ae74ef75537aed1a95965
340.0 MB Preview Download
md5:e3e45177dc415c3c72c20f184d8faa51
591.6 MB Preview Download
md5:8bf19d40ad380b5f16e29d36c151ac5d
221.3 MB Preview Download
md5:0219ca49668d14add35fb31aa75f9fe6
172.1 MB Preview Download
md5:1b4d460d4baadb1315b46dfa80f4f111
466.7 MB Preview Download
md5:852a914d8fb905774110192cd31974d4
786.6 MB Preview Download
md5:5e0d00feb40e373f8349ab30f6a7f1a7
5.2 GB Preview Download
md5:319c4d0b3db0ab4476bb223e5cec25cc
179.7 MB Preview Download
md5:85628b22a69befed2065197a76a069c5
1.5 GB Preview Download
md5:eba546c414061ca33db56e523bf83a01
208 Bytes Preview Download
md5:386c4ac01d6ade88ab71ef4ed55c18ca
179.3 MB Preview Download
md5:85e4ef935e0886844da0ef02caed4fbb
464.9 MB Preview Download
md5:276f120c7970bd8237e16d5e52eaf39f
601.0 MB Preview Download
md5:36a42c05219c91dfc460bb21f8428658
141.4 MB Preview Download
md5:692a79819bf44dc86928586e7a34f28e
545.8 MB Preview Download
md5:29780252952fb4ac7593fe6361a3adbb
1.1 GB Preview Download
md5:8927442d17b073b9c94175b251691de7
709.1 MB Preview Download
md5:142208c47fed3434ac47c1ad2e41234a
424.7 MB Preview Download
md5:0605363c5e06f2f56f5178e38c9cbea5
840.0 MB Preview Download
md5:633f358f64e09cfbbe0a2b9d3e4827a6
206 Bytes Preview Download
md5:e2a40d6c665d5fbef22f8032ce2ccbfe
125.8 MB Preview Download
md5:9d72a8e3168d1822472540d483a0a34a
50.4 kB Preview Download
md5:58085cdd50befbdf158ac818c5881386
1.3 kB Preview Download
md5:5cc82c40ae58ca2a41c5c269ed711492
91.7 kB Preview Download
md5:7b7beedb23df5c215ab6ff1cd87d2efc
51.3 MB Preview Download
md5:ac58c38129d7e42ecafcf117f3c9c38d
41.9 MB Preview Download
md5:d310d280134cd3f09e4ceab9799bf3a9
436.2 MB Download

Additional details

Software

Repository URL
https://github.com/ZwickyTransientFacility/scope-ml