Published February 7, 2024 | Version v1

Classifying HSC sources using machine learing

Authors/Creators

Description

I used 73.6 million photometric sources from the Hyper Suprime-Cam (HSC). I used 201,490 matched sources (stars: 8,500, galaxies: 164,905, quasars: 28,085) with spectroscopically labelled sources from the Sloan Digital Sky Survey (SDSS) to train an optimised random forest classifier. The performance metric (F1 score) scores relatively high across all classifications (stars: 0.935, galaxies: 0.991, quasars: 0.937). I applied the trained model to previously unlabelled sources from the HSC photometric catalogue. This resulted in individual classification probabilities for each source, with 59% of galaxies, 10% of quasars, and 56% of stars having classification probabilities greater than 0.9. Finally I used a non-linear dimension reduction technique, Uniform Manifold Approximation and Projection (UMAP), in fully-supervised schemes to visualise the separation of galaxies, quasars, and stars in a two-dimensional space.
 
 

File descriptions:

All files are Pandas DataFrames. `SDSS_spec_xmwise_all.pkl` contains the spectroscopically observed sources from the SDSS which is used in the reference. The compressed csv files with the extension `csv.gz` contains photometrically observed sources from the HSC. They are named using the following convention `<order>_<min RA>_ra_<max RA>`. Please place these files into the `HSC_sources` directory. `misclassified_sources.pkl` contains sources that were misclassified by the model.

Stars : S     Galaxies : G    Quasars : Q
(Missed S as Q:310,  as G:48       Missed G as S:98, as Q:382      Missed Q as G:961, as S:81)

Files

Files (11.1 GB)

Name Size
md5:b255eab236954463b850de26f992b070
717.8 MB Download
md5:a61aff2d077a07f605eb62159213c29a
643.6 MB Download
md5:d1d1362fa05eda643a51598b2b2947e0
543.3 MB Download
md5:76683bad4688d622e935a3688ea4dc3c
133.5 MB Download
md5:14099e9b790eb6c43d56cae08b368a25
4.3 MB Download
md5:d13026e47937e837e417c33dce8da307
1.2 GB Download
md5:d1e946ece1ab7ac7110dad75938f384d
911.7 MB Download
md5:cfc5dff29b63f8f23a0fb467c2f68851
519.8 MB Download
md5:33d1284fe21c287e5fe9a518fabb1c2f
979.2 MB Download
md5:9117601756aae2306f108d4053e310e7
2.3 MB Download
md5:dbd63c8ae3cdbedf4276fba446e8c0e0
236 Bytes Download
md5:7564c4fbcaac8f2951f82acf56b76950
381.1 MB Download
md5:fe9619de8b32574a4db203aa610a127a
1.1 GB Download
md5:79066589eca3d1f9f84c2f0805432e68
1.2 GB Download
md5:0041e2afc36126ebe4306dcce823f3ed
1.2 GB Download
md5:0d6f8fd7abbb282f7b9b08c7989d1797
600.7 kB Download
md5:768756f8d2ebddea7400999c6edec1e7
1.6 GB Download

Additional details

Additional titles

Alternative title
機械学習を使ったすばる天体の種族分類

References

  • Identifying galaxies, quasars, and stars with machine learning: A new catalog of classifications for 111 million SDSS sources without spectra (Clarke et al. 2020, A&A, 639, A84) 'https://www.aanda.org/articles/aa/full_html/2020/07/aa36770-19/aa36770-19.html'