Published April 2022 | Version 1.1.2
Dataset Open

The data for "The ZTF Source Classification Project: III. A Catalog of Variable Sources"

  • 1. University of Minnesota
  • 2. ROR icon California Institute of Technology
  • 3. ROR icon University of Amsterdam
  • 4. ROR icon University of Washington
  • 5. ROR icon Pacific Lutheran University
  • 6. ROR icon Indian Institute of Technology Gandhinagar
  • 7. ROR icon University of California, Berkeley
  • 8. ROR icon Lawrence Berkeley National Laboratory
  • 9. ROR icon Carnegie Mellon University
  • 10. ROR icon Jet Propulsion Laboratory

Description

The classification of variable objects provides insight into a wide variety of astrophysics ranging from stellar interiors to galactic nuclei. The Zwicky Transient Facility (ZTF) provides time series observations that record the variability of more than a billion sources. The scale of these data necessitates automated approaches to make a thorough analysis. Building on previous work, this paper reports the results of the ZTF Source Classification Project (SCoPe), which trains neural network and XGBoost machine learning (ML) algorithms to perform dichotomous classification of variable ZTF sources using a manually constructed training set containing 170,632 light curves. We find that several classifiers achieve high precision and recall scores, suggesting the reliability of their predictions for 1,648,948,910 light curves across 636 ZTF fields. We also identify the most important features for XGB classification and compare the performance of the two ML algorithms, finding a pattern of higher precision among XGB classifiers. The resulting classification catalog is available to the public, and the software developed for SCoPe is open-source and adaptable to future time-domain surveys.

Notes

Notes for 1.2.2

  1. Changed all files into the parquet format to reduce data volume when unzipped. File names changed from  N-1_N_prediction_xgb_dnn_fields.zip to N-1_N_fields.zip
  2. DNN models updated to work with tf 2.15+
  3. Minor gaps in coverage filled

-----------------------------------------------------------------------------------

Notes for 1.2.1

  1. Fixed issues with 17_16_prediction_xgb_dnn_fields.zip and 18_17_prediction_xgb_dnn_fields.zip

-----------------------------------------------------------------------------------

Notes for 1.2.0

  1. The catalog is complete. Please contact warsh029@umn.edu for questions or help.

-----------------------------------------------------------------------------------

Notes for 1.1.8

  1. In SCoPe_Sky_Coverage.png The darkest blue regions indicate parts of the sky that are included in the original data release. The lightest blue region show the area added in this data release. The medium blue region show what was avalible in the previous version.

-----------------------------------------------------------------------------------

Notes for 1.1.3

  1. In SCoPe_Sky_Coverage.png dark regions indicate parts of the sky that are included in the current catalog and light regions show the total coverage of ZTF.

---------------------------------------------------------------------------------

Notes for 0.1.1

  1. Fields are now seperated by hour in Right Ascension (RA). Fields  with centers between N-1 and N hours RA are located inN_N-1_prediction_xgb_dnn_fields.zip.  For example a field with a center of 3.7 hours RA would be located in 04_03_prediction_xgb_dnn_fields.zip
  2. The seperation is done by the location of the field center so there may be parts of fields that are not in the within the RA indicated. Field centers and corners can be found on this GitHub page.
  3. 01_00_prediction_xgb_dnn_fields.zip is fully completed

------------------------------------------------------------------------------------

Notes for 0.1.0

  1. Fields are now seperated by hour in Right Ascension (RA). Fields  with centers between N-1 and N hours RA are located in N_prediction_xgb_dnn_fields.zip.  For example a field with a center of 3.7 hours RA would be located in 4_prediction_xgb_dnn_fields.zip
  2. The seperation is done by the location of the field center so there may be parts of fields that are not in the within the RA indicated. Field centers and corners can be found on this GitHub page.
  3. fields.json now denotes which fields with classifications are contained in each folder.

 

------------------------------------------------------------------------------------

Notes for 0.0.4

  1. All fields are now contained (in CSV format) within a single predictions_dnn_xgb_92_fields.zip file.

------------------------------------------------------------------------------------

Notes for 0.0.3

  1. All fields are now contained (in CSV format) within a single predictions_dnn_xgb_77_fields.zip file.

 

------------------------------------------------------------------------------------

Notes for 0.0.2

  1.  From Version 0.0.1 to 0.0.2, classification predictions have been updated by the inclusion of the Gaia parallax error in the set of ontological features. Any fields downloaded from Version 0.0.1 should thus be replaced by those from Version 0.0.2.
  2. The structure of the catalog has been updated - all fields are now contained (in CSV format) within a single predictions_dnn_xgb_70_fields.zip file.
  3. A demo file containing 100 rows of predictions along with a Jupyter notebook to explain the columns are included in this release. See the scope-ml repository for details on setting up the environment needed to run this notebook.
  4. The new fields.json file contains a list of all fields having classification predictions.

Files

01_00_fields.zip

Files (68.9 GB)

Name Size Download all
md5:6606020c5f9f50024cf20909dcd36990
1.9 GB Preview Download
md5:8ab31bf6e64fbdf997476ac24422709c
1.8 GB Preview Download
md5:3e092d17452a22f76907abdff4ea8129
1.5 GB Preview Download
md5:5be31795223273dbf7497d1550f60b6f
1.4 GB Preview Download
md5:9938f88701a9a1fdbb4977f7d0075287
1.6 GB Preview Download
md5:f86765d8ec1fc54e51b19d349779c988
2.1 GB Preview Download
md5:24d2d1c0a59d74a8d4c5d7812e8041ce
2.7 GB Preview Download
md5:78a8f6989023d88bcbfe1dd2bfd18e70
2.9 GB Preview Download
md5:a7bcacf17c0308c81c429641ebf54c19
1.5 GB Preview Download
md5:631b576d0ff13e24bedd86adb967f834
1.0 GB Preview Download
md5:0399a9f44cf8199a6709463711a5f469
802.4 MB Preview Download
md5:7021ec95482f8d5e9d798c067be01245
889.3 MB Preview Download
md5:fbdf25694e3dc9c86fbd7365cd8fa0e8
980.9 MB Preview Download
md5:74f15b5ce0b34d3407ed2f554f862046
1.1 GB Preview Download
md5:c012ce07eb1f87464d06003178b0e718
1.2 GB Preview Download
md5:6983594ed055aeb5b6d7686164538797
1.5 GB Preview Download
md5:371719f276ca308d5f57756807e617b7
2.4 GB Preview Download
md5:2511394cec4ab8a052318d3c4a7e90e4
5.7 GB Preview Download
md5:c10b328a8cf105b85e43b81b72ff7c8f
9.9 GB Preview Download
md5:4848f6f077abcccbf4923dd091c835e8
10.8 GB Preview Download
md5:4ddb497abb4a834e0835924735d8ff17
6.8 GB Preview Download
md5:1e77afa173f7ee43e7081d5fceb0fc38
3.5 GB Preview Download
md5:29955f4b3222777f143fb43fbf3fe420
2.4 GB Preview Download
md5:4d815436aa2bd0717926b97dc536f5e8
2.0 GB Preview Download
md5:9d72a8e3168d1822472540d483a0a34a
50.4 kB Preview Download
md5:3c3bcb31b0d81ebd5d3eea401d85ac16
4.1 kB Preview Download
md5:5cc82c40ae58ca2a41c5c269ed711492
91.7 kB Preview Download
md5:c452179a21aa077b6d13e62a2d382981
58.7 kB Preview Download
md5:e4c5dd2392f6ad9a1eea60a42a06a8fd
17.5 MB Preview Download
md5:ac58c38129d7e42ecafcf117f3c9c38d
41.9 MB Preview Download
md5:d310d280134cd3f09e4ceab9799bf3a9
436.2 MB Download

Additional details