Transmission electron microscopy

There is a newer version of the record available.

Published August 13, 2020 | Version 0.5.1
Software Open

LiberTEM/LiberTEM: 0.5.1

Creators

  • 1. Jülich Research Centre, Ernst Ruska Centre
  • 2. RWTH Aachen University, Jülich Research Centre, Ernst Ruska Centre
  • 3. APJ Abdul Kalam Technological University
  • 4. APJ Abdul Kalam Technical University
  • 5. Chandigarh University
  • 6. Siliguri Institute of Technology
  • 7. Jadavpur University
  • 8. Direct Electron
  • 9. University of Antwerp
  • 10. Lawrence Berkeley National Laboratory
  • 11. Monash University eResearch Centre
  • 12. ETH Zürich
  • 13. Columbia University

Contributors

Others:

  • 1. FZ Jülich
  • 2. X-Spectrum
  • 3. MPSD
  • 4. HZM
  • 5. Quantum Detectors
  • 6. pyXem
  • 7. HZDR
  • 8. HU Berlin
  • 9. Berkeley Lab
  • 10. Microscopy Australia
  • 11. DESY
  • 12. Monash
  • 13. U Antwerp
  • 14. Weizmann
  • 15. PNDetector
  • 16. NeXus project
  • 17. Gatan
  • 18. U Glasgow
  • 19. U Bremen
  • 20. Nion
  • 21. Point Electronic
  • 22. Python Software Foundation
  • 23. Lille
  • 24. U Limerick
  • 25. Iambic Innovation
  • 26. U Wisconsin
  • 27. PSI
  • 28. U Queensland
  • 29. HZB

Description

Homepage: https://libertem.github.io/LiberTEM/
GitHub repository: https://github.com/LiberTEM/LiberTEM/
PyPI: https://pypi.org/project/libertem/

LiberTEM is an open source platform for high-throughput distributed processing of large-scale binary data sets using a simplified MapReduce programming model. The current focus is pixelated scanning transmission electron microscopy (STEM) and scanning electron beam diffraction data.

It is designed for high throughput and scalability on PCs, single server nodes, clusters and cloud services. On clusters it can use fast distributed local storage on high-performance SSDs. That way it achieves very high aggregate IO performance on a compact and cost-efficient system built from stock components.

LiberTEM is supported on Linux, Mac OS X and Windows. Other platforms that allow installation of Python 3 and the required packages will likely work as well. The GUI is running in a web browser.

Installation

The short version:

$ virtualenv -p python3 ~/libertem-venv/
$ source ~/libertem-venv/bin/activate
(libertem) $ pip install "libertem[torch]"

Please see our documentation for details!

Deployment as a single-node system for a local user is thoroughly tested and can be considered stable. Deployment on a cluster is experimental and still requires some additional work, see Issue #105.

Applications

Please see the applications section of our documentation for details!

The Python API and user-defined functions (UDFs) can be used for more complex operations with arbitrary masks and other features like data export. There are example Jupyter notebooks available in the examples directory. If you are having trouble running the examples, please let us know, either by filing an issue or by joining our Gitter chat.

LiberTEM is suitable as a high-performance processing backend for other applications, including live data streams. Contact us if you are interested!

LiberTEM is evolving rapidly and prioritizes features following user demand and contributions. In the future we'd like to implement live acquisition, and more analysis methods for all applications of pixelated STEM and other large-scale detector data. If you like to influence the direction this project is taking, or if you'd like to contribute, please join our gitter chat and our general mailing list.

File formats

LiberTEM currently opens most file formats used for pixelated STEM. See our general information on loading data and format-specific documentation for more information!

  • Raw binary files
  • Thermo Fisher EMPAD detector files
  • Quantum Detectors MIB format
  • Nanomegas .blo block files
  • Direct Electron DE5 files (HDF5-based) for DE-Series detectors
  • Gatan K2 IS raw format
  • Stacks of Gatan DM3 and DM4 files (via openNCEM)
  • FRMS6 from PNDetector pnCCD cameras (currently alpha, gain correction still needs UI changes)
  • FEI SER files (via openNCEM)
  • HDF5-based formats such as Hyperspy files, NeXus and EMD
  • Please contact us if you are interested in support for an additional format!

License

LiberTEM is licensed under GPLv3. The I/O parts are also available under the MIT license, please see LICENSE files in the subdirectories for details.

Notes

This project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 856538).

We gratefully acknowledge funding from the Initiative and Network Fund of the Helmholtz Association within the Helmholtz Young Investigator Group moreSTEM under Contract No. VH-NG-1317 at Forschungszentrum Jülich in Germany.

We gratefully acknowledge funding from the Information & Data Science Pilot Project 'Ptychography 4.0' of the Helmholtz Association.

We kindly acknowledge funding from Google Summer of Code 2019 under the umbrella of the Python software foundation.

STEMx equipment and software for 4D STEM data acquisition with K2 IS camera courtesy of Gatan Inc.

Forschungszentrum Jülich is supporting LiberTEM with funding for personnel, access to its infrastructure and administrative support.

Files

Files (31.4 MB)

Name Size Download all
md5:65a98d66990f7602d903763da9803bfe
2.6 MB Download
md5:0e419391ceccb10f385fb1f825973dcd
2.8 MB Download
md5:ec650a3cb083890e915c646eec4ee270
26.0 MB Download

Additional details

Cites
10.5281/zenodo.2450624 (DOI)
10.1364/ao.51.005698 (DOI)
https://web.archive.org/web/20180809021832/http://www.gatan.com/products/tem-imaging-spectroscopy/k2-camera (URL)
10.5281/zenodo.1477847 (DOI)
10.1107/s0907444913000863 (DOI)
10.1107/s1600576714027575 (DOI)
10.18429/jacow-icalepcs2017-tucpa01 (DOI)
10.1038/srep26348 (DOI)
10.1186/s40679-015-0006-6 (DOI)
10.1016/j.ultramic.2016.03.006 (DOI)
10.1088/1742-6596/644/1/012032 (DOI)
10.1017/s1431927617000940 (DOI)
10.1002/9783527808465.EMC2016.6284 (DOI)
10.1017/s1431927616009430 (DOI)
10.1016/s0304-3991(79)80021-2 (DOI)
10.1107/s0567739469001045 (DOI)
10.1002/bbpc.19700741112 (DOI)
10.1038/374630a0 (DOI)
10.1364/josaa.10.000231 (DOI)
10.1088/0022-3727/25/2/003 (DOI)
10.1016/0304-3991(88)90371-3 (DOI)
10.1007/978-1-4757-5581-7_15 (DOI)
10.1017/s1431927615015664 (DOI)
10.1038/s41586-018-0298-5 (DOI)
10.1017/s1431927614002037 (DOI)
10.1017/s1431927615011836 (DOI)
10.1016/j.ultramic.2014.09.013 (DOI)
10.1016/j.ultramic.2014.10.013 (DOI)
http://quantumdetectors.com/wp-content/uploads/2017/01/1532-Merlin-for-EM-Technical-Datasheet-v2.pdf (URL)
10.1017/s1431927617010637 (DOI)
10.1038/nmeth.2472 (DOI)
http://www.x-spectrum.de/index_htm_files/X-Spectrum_datasheet_2M.pdf (URL)
arXiv:1805.04957 (arXiv)
https://www.ebook.de/de/product/21119819/jakob_nielsen_usability_engineering.html (URL)
10.1109/jstars.2011.2162643 (DOI)
10.1007/bf00350249 (DOI)
10.1016/s0167-8191(02)00094-7 (DOI)
10.1007/978-3-319-22988-1_5 (DOI)
10.1145/356635.356640 (DOI)
https://hackernoon.com/the-rules-of-optimization-why-so-many-performance-efforts-fail-cf06aad89099 (URL)
https://software.intel.com/en-us/articles/de-mystifying-software-performance-optimization (URL)
10.1016/j.jss.2016.11.037 (DOI)
10.1145/1327452.1327492 (DOI)
10.5445/ksp/1000060221 (DOI)
10.3389/fninf.2014.00030 (DOI)
10.1145/2814204.2814207 (DOI)
https://www.amazon.com/Designing-User-Interface-Human-Computer-Interaction/dp/0321537351?SubscriptionId=AKIAIOBINVZYXZQZ2U3A&tag=chimbori05-20&linkCode=xm2&camp=2025&creative=165953&creativeASIN=0321537351 (URL)
10.1093/jmicro/dfu070 (DOI)
https://www.toptal.com/react/react-redux-and-immutablejs (URL)
https://medium.com/@lachlanmiller_52885/redux-driven-ui-design-8586a84d808f (URL)
10.1109/32.508313 (DOI)
https://pyfai.readthedocs.io/en/latest/pyFAI.html (URL)
10.1109/mcse.2010.51 (DOI)
10.5281/zenodo.1478763 (DOI)
https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/architecting-lustre-storage-white-paper.pdf (URL)
10.1109/nuicone.2012.6493198 (DOI)
10.1145/358438.349320 (DOI)
10.1109/mcse.2011.37 (DOI)
https://www.linuxjournal.com/article/6345?page=0,0 (URL)
https://www.doc.ic.ac.uk/~phjk/AdvancedCompArchitecture/2003-04/Lectures/Ch03/ACA-CH03-CurrentVersion.pdf (URL)
10.1109/msst.2010.5496972 (DOI)
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html (URL)
10.1090/s0025-5718-1965-0178586-1 (DOI)
10.1145/2934664 (DOI)
10.25080/majora-7b98e3ed-013 (DOI)
http://www.netlib.org/blas/ (URL)
https://pytorch.org/ (URL)
https://software.intel.com/en-us/mkl (URL)
https://godbolt.org/ (URL)
http://numba.pydata.org/ (URL)
https://reactjs.org/ (URL)
https://redux.js.org/ (URL)
https://www.linuxjournal.com/content/docker-lightweight-linux-containers-consistent-development-and-deployment (URL)
10.1109/mcc.2014.51 (DOI)
10.1109/52.60586 (DOI)
10.1038/nmeth.2019 (DOI)
10.5210/fm.v10i8.1265 (DOI)
10.29379/jedem.v4i2.138 (DOI)
https://support.hdfgroup.org/HDF5/Tutor/HDF5Intro.pdf (URL)
10.2139/ssrn.2568185 (DOI)
https://www.apache.org/ (URL)
https://www.linuxfoundation.org/ (URL)
10.1007/978-1-4757-2519-3_1 (DOI)
10.1145/1356052.1356053 (DOI)
http://docs.dask.org/en/latest/spark.html (URL)
10.1002/spe.2203 (DOI)
http://perso.ens-lyon.fr/christophe.alias/impact2011/impact-07.pdf (URL)
10.1103/physrevlett.120.167204 (DOI)
http://arxiv.org/abs/1612.02485v1 (URL)
10.1103/physrevlett.121.243902 (DOI)
https://distributed.dask.org/en/latest/journey.html (URL)
https://medium.com/swlh/the-case-for-flux-379b7d1982c6 (URL)
https://redux.js.org/introduction/core-concepts (URL)
https://redux.js.org/basics/usage-with-react (URL)
https://github.com/axboe/fio (URL)
2128/19740 (Handle)
10.1063/1.5096245 (DOI)
10.1038/s41563-019-0387-3 (DOI)
10.1103/physrevlett.78.1074 (DOI)
10.1017/s1431927619000497 (DOI)
10.1109/FSKD.2014.6980942 (DOI)
10.1088/0034-4885/71/1/016102 (DOI)
10.1145/3221269.3223036 (DOI)
10.5281/zenodo.3592520 (DOI)
10.5281/zenodo.2566137 (DOI)
10.1142/9789811204579_0005 (DOI)
10.1016/j.ultramic.2019.112890 (DOI)

Funding

VIDEO – Versatile and Innovative Detector for Electron Optics 780487
European Commission
CritCat – Towards Replacement of Critical Catalyst Materials by Improved Nanoparticle Control and Rational Design 686053
European Commission
ESTEEM3 – Enabling Science and Technology through European Electron Microscopy 823717
European Commission

References

  • [Weber2018] Weber, Dieter (2018): Development of {IT} system and {TEM} camera performance. Zenodo. 10.5281/zenodo.2450624
  • [Bon2012] Pierre Bon and Serge Monneret and Benoit Wattellier (2012): Noniterative boundary-artifact-free wavefront reconstruction from its derivatives. The Optical Society. 10.1364/ao.51.005698
  • [K2IS2018] {{Gatan Inc.}} (2018): {K2} {IS} camera. https://web.archive.org/web/20180809021832/http://www.gatan.com/products/tem-imaging-spectroscopy/k2-camera
  • [Clausen2018] Clausen, Alexander and Weber, Dieter and {Probonopd} and Caron, Jan and Nord, Magnus and Müller-Caspary, Knut and Ophus, Colin and Dunin-Borkowski, Rafal (2018): Libertem/Libertem: 0.1.0. Zenodo. 10.5281/zenodo.1477847
  • [Sauter2013] Nicholas K. Sauter and Johan Hattne and Ralf W. Grosse-Kunstleve and Nathaniel Echols (2013): New Python-based methods for data processing. International Union of Crystallography ({IUCr}). 10.1107/s0907444913000863
  • [Koennecke2015] Mark Könnecke and Frederick A. Akeroyd and Herbert J. Bernstein and Aaron S. Brewster and Stuart I. Campbell and Björn Clausen and Stephen Cottrell and Jens Uwe Hoffmann and Pete R. Jemian and David Männicke and Raymond Osborn and Peter F. Peterson and Tobias Richter and Jiro Suzuki and Benjamin Watts and Eugen Wintersberger and Joachim Wuttke (2015): The {NeXus} data format. International Union of Crystallography ({IUCr}). 10.1107/s1600576714027575
  • [Fangohr2018] Fangohr, Hans and Aplin, Steve and Barty, Anton and Beg, Marijan and Bondar, Valerii and Boukhelef, Djelloul and Brockhauser, Sandor and Danilevski, Cyril and Ehsan, Wajid and Esenov, Sergey and Flucke, Gero and Giovanetti, Gabriele and Goeries, Dennis and Hauf, Steffen and Heisen, Burkhard and Hickin, David and Khakhulin, Dmitry and Klimovskaia, Anna and Kluyver, Thomas and Kuhn, Manuela and Kuster, Markus and Lang, Philipp-Michael and Maia, Luis and Mariani, Valerio and Mekinda, Leonce and Michelat, Thomas and Parenti, Andrea and Previtali, Gianpietro and Santos, Hugo and Silenzi, Alessandro and Sztuk-Dambietz, Jolanta and Szuba, Janusz and Teichmann, Martin and Weger, Kerstin and Wiggins, John and Wrona, Krzysztof and Xu, Chen (2018): Data Analysis Support in Karabo at European XFEL. JACoW, Geneva, Switzerland. 10.18429/jacow-icalepcs2017-tucpa01
  • [Jesse2016] S. Jesse and M. Chi and A. Belianinov and C. Beekman and S. V. Kalinin and A. Y. Borisevich and A. R. Lupini (2016): Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography. Springer Nature. 10.1038/srep26348
  • [Belianinov2015] Alex Belianinov and Rama Vasudevan and Evgheni Strelcov and Chad Steed and Sang Mo Yang and Alexander Tselev and Stephen Jesse and Michael Biegalski and Galen Shipman and Christopher Symons and Albina Borisevich and Rick Archibald and Sergei Kalinin (2015): Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets. Springer Nature. 10.1186/s40679-015-0006-6
  • [Krajnak2016] Matus Krajnak and Damien McGrouther and Dzmitry Maneuski and Val O{\textquotesingle} Shea and Stephen McVitie (2016): Pixelated detectors and improved efficiency for magnetic imaging in {STEM} differential phase contrast. Elsevier {BV}. 10.1016/j.ultramic.2016.03.006
  • [Yang2015] H Yang and L Jones and H Ryll and M Simson and H Soltau and Y Kondo and R Sagawa and H Banba and I MacLaren and P D Nellist (2015): 4D {STEM}: High efficiency phase contrast imaging using a fast pixelated detector. {IOP} Publishing. 10.1088/1742-6596/644/1/012032
  • [Sagawa2017] Ryusuke Sagawa and Hao Yang and Lewys Jones and Martin Simson and Martin Huth and Heike Soltau and Peter D. Nellist and Yukihito Kondo (2017): Development of Fast Pixelated {STEM} Detector and its Applications using 4-Dimensional Dataset. Cambridge University Press ({CUP}). 10.1017/s1431927617000940
  • [doi:10.1002/9783527808465.EMC2016.6284] MacLaren, Ian and Nord, Magnus and Ross, Andrew and Krajnak, Matus and Hart, Martin and Doye, Alastair and McGrouther, Damien and Bali, Rantej and Banerjee, Archan and Hadfield, Robert (2016): Pixelated STEM detectors: opportunities and challenges. American Cancer Society. 10.1002/9783527808465.EMC2016.6284
  • [Nguyen2016] Kayla X. Nguyen and Prafull Purohit and Robert Hovden and Emrah Turgut and Mark W. Tate and Lena F. Kourkoutis and Gregory D. Fuchs and Sol M. Gruner and David A. Muller (2016): 4D-{STEM} for Quantitative Imaging of Magnetic Materials with Enhanced Contrast and Resolution. Cambridge University Press ({CUP}). 10.1017/s1431927616009430
  • [Cowley1979] J.M. Cowley (1979): Coherent interference in convergent-beam electron diffraction and shadow imaging. Elsevier {BV}. 10.1016/s0304-3991(79)80021-2
  • [Hoppe1969] W. Hoppe (1969): Beugung im inhomogenen Primärstrahlwellenfeld. I. Prinzip einer Phasenmessung von Elektronenbeungungsinterferenzen. International Union of Crystallography ({IUCr}). 10.1107/s0567739469001045
  • [Hegerl1970] R. Hegerl and W. Hoppe (1970): Dynamische Theorie der Kristallstrukturanalyse durch Elektronenbeugung im inhomogenen Primärstrahlwellenfeld. Wiley. 10.1002/bbpc.19700741112
  • [Nellist1995] P. D. Nellist and B. C. McCallum and J. M. Rodenburg (1995): Resolution beyond the {\textquotesingle}information limit{\textquotesingle} in transmission electron microscopy. Springer Nature. 10.1038/374630a0
  • [McCallum1993] B. C. McCallum and J. M. Rodenburg (1993): Simultaneous reconstruction of object and aperture functions from multiple far-field intensity measurements. The Optical Society. 10.1364/josaa.10.000231
  • [Friedman1992] S L Friedman and J M Rodenburg (1992): Optical demonstration of a new principle of far-field microscopy. {IOP} Publishing. 10.1088/0022-3727/25/2/003
  • [Humphreys1988] C.J. Humphreys and D.J. Eaglesham and D.M. Maher and H.L. Fraser (1988): {CBED} and {CBIM} from semiconductors and superconductors. Elsevier {BV}. 10.1016/0304-3991(88)90371-3
  • [Steeds1979] J. W. Steeds (1979): Convergent Beam Electron Diffraction. Springer {US}. 10.1007/978-1-4757-5581-7_15
  • [Tate2016] Mark W. Tate and Prafull Purohit and Darol Chamberlain and Kayla X. Nguyen and Robert Hovden and Celesta S. Chang and Pratiti Deb and Emrah Turgut and John T. Heron and Darrell G. Schlom and Daniel C. Ralph and Gregory D. Fuchs and Katherine S. Shanks and Hugh T. Philipp and David A. Muller and Sol M. Gruner (2016): High Dynamic Range Pixel Array Detector for Scanning Transmission Electron Microscopy. Cambridge University Press ({CUP}). 10.1017/s1431927615015664
  • [Jiang2018] Yi Jiang and Zhen Chen and Yimo Han and Pratiti Deb and Hui Gao and Saien Xie and Prafull Purohit and Mark W. Tate and Jiwoong Park and Sol M. Gruner and Veit Elser and David A. Muller (2018): Electron ptychography of 2D materials to deep sub-{\aa}ngström resolution. Springer Nature. 10.1038/s41586-018-0298-5
  • [Ophus2014] Colin Ophus and Peter Ercius and Michael Sarahan and Cory Czarnik and Jim Ciston (2014): Recording and Using 4D-{STEM} Datasets in Materials Science. Cambridge University Press ({CUP}). 10.1017/s1431927614002037
  • [Simson2015] M. Simson and H. Ryll and H. Banba and R. Hartmann and M. Huth and S. Ihle and L. Jones and Y. Kondo and K. Muller and P.D. Nellist and R. Sagawa and J. Schmidt and H. Soltau and L. Striider and H. Yang (2015): 4D-{STEM} Imaging With the {pnCCD} (S){TEM}-Camera. Cambridge University Press ({CUP}). 10.1017/s1431927615011836
  • [Pennycook2015] Timothy J. Pennycook and Andrew R. Lupini and Hao Yang and Matthew F. Murfitt and Lewys Jones and Peter D. Nellist (2015): Efficient phase contrast imaging in {STEM} using a pixelated detector. Part 1: Experimental demonstration at atomic resolution. Elsevier {BV}. 10.1016/j.ultramic.2014.09.013
  • [Yang2015a] Hao Yang and Timothy J. Pennycook and Peter D. Nellist (2015): Efficient phase contrast imaging in {STEM} using a pixelated detector. Part {II}: Optimisation of imaging conditions. Elsevier {BV}. 10.1016/j.ultramic.2014.10.013
  • [QuantumMerlin2017] {Quantum Detectors} (2017): Merlin for {EM} technical datasheet. http://quantumdetectors.com/wp-content/uploads/2017/01/1532-Merlin-for-EM-Technical-Datasheet-v2.pdf
  • [Lawrence2017] Ethan L. Lawrence and Shery L. Y. Chang and Peter A. Crozier (2017): In situ {TEM} observations of Oxygen Surface Dynamics in {CeO}2 Cubes. Cambridge University Press ({CUP}). 10.1017/s1431927617010637
  • [Li2013] Xueming Li and Paul Mooney and Shawn Zheng and Christopher R Booth and Michael B Braunfeld and Sander Gubbens and David A Agard and Yifan Cheng (2013): Electron counting and beam-induced motion correction enable near-atomic-resolution single-particle cryo-{EM}. Springer Nature. 10.1038/nmeth.2472
  • [XSpectrum-LambdaM2] X-Spectrum: {Lambda2M} large area {Medipix3} based detector array. http://www.x-spectrum.de/index_htm_files/X-Spectrum_datasheet_2M.pdf
  • [Li:2018ngp] Li, Xin and Dyck, Ondrej and Kalinin, Sergei V. and Jesse, Stephen (2018): Compressed Sensing of Scanning Transmission Electron Microscopy {(STEM)} on Non-Rectangular Scans. 1805.04957
  • [Nielsen1994] Nielsen, Jakob (1994): Usability Engineering. Elsevier Science. https://www.ebook.de/de/product/21119819/jakob_nielsen_usability_engineering.html
  • [Lee2011] Craig A. Lee and Samuel D. Gasster and Antonio Plaza and Chein-I Chang and Bormin Huang (2011): Recent Developments in High Performance Computing for Remote Sensing: A Review. Institute of Electrical and Electronics Engineers ({IEEE}). 10.1109/jstars.2011.2162643
  • [Tezduyar1996] T. Tezduyar and S. Aliabadi and M. Behr and A. Johnson and V. Kalro and M. Litke (1996): Flow simulation and high performance computing. Springer Nature America, Inc. 10.1007/bf00350249
  • [Allcock2002] Bill Allcock and Joe Bester and John Bresnahan and Ann L. Chervenak and Ian Foster and Carl Kesselman and Sam Meder and Veronika Nefedova and Darcy Quesnel and Steven Tuecke (2002): Data management and transfer in high-performance computational grid environments. Elsevier {BV}. 10.1016/s0167-8191(02)00094-7
  • [Pan2016] Ming Pan and Cory Czarnik (2016): Image Detectors for Environmental Transmission Electron Microscopy ({ETEM}). Springer International Publishing. 10.1007/978-3-319-22988-1_5
  • [Knuth1974] Donald E. Knuth (1974): Structured Programming with go to Statements. Association for Computing Machinery ({ACM}). 10.1145/356635.356640
  • [Busch2016] Busch, Kat (2016): The Rules of Optimization: Why So Many Performance Efforts Fail. https://hackernoon.com/the-rules-of-optimization-why-so-many-performance-efforts-fail-cf06aad89099
  • [Delvecchio2011] Delvecchio, Paul (2011): De-Mystifying Software Performance Optimization. https://software.intel.com/en-us/articles/de-mystifying-software-performance-optimization
  • [Mavridis2017] Ilias Mavridis and Helen Karatza (2017): Performance evaluation of cloud-based log file analysis with Apache Hadoop and Apache Spark. Elsevier {BV}. 10.1016/j.jss.2016.11.037
  • [Dean2008] Jeffrey Dean and Sanjay Ghemawat (2008): {MapReduce}. Association for Computing Machinery ({ACM}). 10.1145/1327452.1327492
  • [Stegmaier2017] Stegmaier, Johannes (2017): New Methods to Improve Large-Scale Microscopy Image Analysis with Prior Knowledge and Uncertainty. KIT Scientific Publishing. 10.5445/ksp/1000060221
  • [Goscinski2014] Wojtek J. Goscinski and Paul McIntosh and Ulrich Felzmann and Anton Maksimenko and Christopher J. Hall and Timur Gureyev and Darren Thompson and Andrew Janke and Graham Galloway and Neil E. B. Killeen and Parnesh Raniga and Owen Kaluza and Amanda Ng and Govinda Poudel and David G. Barnes and Toan Nguyen and Paul Bonnington and Gary F. Egan (2014): The multi-modal Australian {ScienceS} Imaging and Visualization Environment ({MASSIVE}) high performance computing infrastructure: applications in neuroscience and neuroinformatics research. Frontiers Media {SA}. 10.3389/fninf.2014.00030
  • [Foust2015] Gabriel Foust and Jaakko Järvi and Sean Parent (2015): Generating reactive programs for graphical user interfaces from multi-way dataflow constraint systems. {ACM} Press. 10.1145/2814204.2814207
  • [Shneiderman2009] Ben Shneiderman and Catherine Plaisant and Maxine Cohen and Steven Jacobs (2009): Designing the User Interface: Strategies for Effective Human-Computer Interaction (5th Edition). Pearson. https://www.amazon.com/Designing-User-Interface-Human-Computer-Interaction/dp/0321537351?SubscriptionId=AKIAIOBINVZYXZQZ2U3A&tag=chimbori05-20&linkCode=xm2&camp=2025&creative=165953&creativeASIN=0321537351
  • [Tsukamoto2014] Takafumi Tsukamoto and Takuo Yasunaga (2014): Development of a user-friendly system for image processing of electron microscopy by integrating a web browser and {PIONE} with Eos. Oxford University Press ({OUP}). 10.1093/jmicro/dfu070
  • [Rogic] Ivan Rogic: {React}, {Redux} and {Immutable.js}: Ingredients for Efficient Web Applications. https://www.toptal.com/react/react-redux-and-immutablejs
  • [Lachlan] Lachlan Miller (2017): Redux-driven {UI} design. https://medium.com/@lachlanmiller_52885/redux-driven-ui-design-8586a84d808f
  • [Taylor1996] R.N. Taylor and N. Medvidovic and K.M. Anderson and E.J. Whitehead and J.E. Robbins and K.A. Nies and P. Oreizy and D.L. Dubrow (1996): A component- and message-based architectural style for {GUI} software. Institute of Electrical and Electronics Engineers ({IEEE}). 10.1109/32.508313
  • [pyFAI] {Data analysis unit, European Synchrotron Radiation Facility, Grenoble} (2018): General introduction to {PyFAI}. https://pyfai.readthedocs.io/en/latest/pyFAI.html
  • [Alted2010] Francesc Alted (2010): Why Modern {CPUs} Are Starving and What Can Be Done about It. Institute of Electrical and Electronics Engineers ({IEEE}). 10.1109/mcse.2010.51
  • [Clausen2018a] Clausen, Alexander and Weber, Dieter and {Probonopd} and Caron, Jan and Nord, Magnus and Müller-Caspary, Knut and Ophus, Colin and Dunin-Borkowski, Rafal (2018): Libertem/Libertem: 0.1.0. Zenodo. 10.5281/zenodo.1478763
  • [IntelLustre] {Intel High Performance Data Division} (2014): Architecting a High Performance Storage System. https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/architecting-lustre-storage-white-paper.pdf
  • [Patel2012] Aditya B. Patel and Manashvi Birla and Ushma Nair (2012): Addressing big data problem using {Hadoop} and {Map Reduce}. {IEEE}. 10.1109/nuicone.2012.6493198
  • [Larsen2000] Samuel Larsen and Saman Amarasinghe (2000): Exploiting superword level parallelism with multimedia instruction sets. Association for Computing Machinery ({ACM}). 10.1145/358438.349320
  • [Walt2011] St{\'{e}}fan van der Walt and S Chris Colbert and Gaël Varoquaux (2011): The {NumPy} Array: A Structure for Efficient Numerical Computation. Institute of Electrical and Electronics Engineers ({IEEE}). 10.1109/mcse.2011.37
  • [Thadani19951AE] Moti Thadani and Yousef Y. A. Khalidi (1995): An Efficient Zero-Copy {I / O} Framework for {UNIX} ®.
  • [Stancevic2003] Dragan Stancevic (2003): Zero Copy I: User-Mode Perspective. https://www.linuxjournal.com/article/6345?page=0,0
  • [Kelly2003] Paul H J Kelly (2003): Advanced Computer Architecture: Caches and Memory Systems. https://www.doc.ic.ac.uk/~phjk/AdvancedCompArchitecture/2003-04/Lectures/Ch03/ACA-CH03-CurrentVersion.pdf
  • [Shvachko2010] Konstantin Shvachko and Hairong Kuang and Sanjay Radia and Robert Chansler (2010): The Hadoop Distributed File System. {IEEE}. 10.1109/msst.2010.5496972
  • [HDFS-Short] {Apache Software Foundation} (2018): {HDFS} Short-Circuit Local Reads. https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
  • [Cooley1965] James W. Cooley and John W. Tukey (1965): An algorithm for the machine calculation of complex Fourier series. American Mathematical Society ({AMS}). 10.1090/s0025-5718-1965-0178586-1
  • [Zaharia2016] Matei Zaharia and Michael J. Franklin and Ali Ghodsi and Joseph Gonzalez and Scott Shenker and Ion Stoica and Reynold S. Xin and Patrick Wendell and Tathagata Das and Michael Armbrust and Ankur Dave and Xiangrui Meng and Josh Rosen and Shivaram Venkataraman (2016): Apache Spark. Association for Computing Machinery ({ACM}). 10.1145/2934664
  • [Rocklin2015] Matthew Rocklin (2015): Dask: Parallel Computation with Blocked algorithms and Task Scheduling. {SciPy}. 10.25080/majora-7b98e3ed-013
  • [BLAS] (2017): {BLAS} ({Basic Linear Algebra Subprograms}). http://www.netlib.org/blas/
  • [PyTorch] (2018): PyTorch. https://pytorch.org/
  • [MKL] Intel (2018): Intel® Math Kernel Library. https://software.intel.com/en-us/mkl
  • [Godbolt] Matt Godbolt: Compiler Explorer. https://godbolt.org/
  • [Numba] (2018): Numba: A high-performance {Python} compiler. http://numba.pydata.org/
  • [ReactJS] {Facebook Inc.} (2018): React: A {JavaScript} library for building user interfaces. https://reactjs.org/
  • [ReduxJS] Dan Abramov (2018): Redux: A predictable state container for {JavaScript} apps.. https://redux.js.org/
  • [MerkelDocker] Dirk Merkel (2014): Docker: Lightweight Linux Containers for Consistent Development and Deployment. https://www.linuxjournal.com/content/docker-lightweight-linux-containers-consistent-development-and-deployment
  • [Bernstein2014] David Bernstein (2014): Containers and Cloud: From {LXC} to Docker to Kubernetes. Institute of Electrical and Electronics Engineers ({IEEE}). 10.1109/mcc.2014.51
  • [Shaw1990] Mary Shaw (1990): Prospects for an engineering discipline of software. Institute of Electrical and Electronics Engineers ({IEEE}). 10.1109/52.60586
  • [Schindelin2012] Johannes Schindelin and Ignacio Arganda-Carreras and Erwin Frise and Verena Kaynig and Mark Longair and Tobias Pietzsch and Stephan Preibisch and Curtis Rueden and Stephan Saalfeld and Benjamin Schmid and Jean-Yves Tinevez and Daniel James White and Volker Hartenstein and Kevin Eliceiri and Pavel Tomancak and Albert Cardona (2012): Fiji: an open-source platform for biological-image analysis. Springer Nature. 10.1038/nmeth.2019
  • [Willinsky2005] John Willinsky (2005): The unacknowledged convergence of open source, open access, and open science. University of Illinois Libraries. 10.5210/fm.v10i8.1265
  • [Zuiderwijk2012] Anneke Zuiderwijk and Keith Jeffery and Marijn Janssen (2012): The Potential of Metadata for Linked Open Data and its Value for Users and Publishers. {JEDEM} Journal of e-Democracy and Open Government. 10.29379/jedem.v4i2.138
  • [HDF5] (2016): High Level Introduction to {HDF5}. https://support.hdfgroup.org/HDF5/Tutor/HDF5Intro.pdf
  • [Okoli2015] Chitu Okoli and Johannes Nguyen (2015): Business Models for Free and Open Source Software. Elsevier {BV}. 10.2139/ssrn.2568185
  • [ApacheFoundation] {{The APACHE Software Foundation}} (2018): The {APACHE} Software Foundation. https://www.apache.org/
  • [Linuxfoundation] {The Linux Foundation} (2018): The Linux Foundation. https://www.linuxfoundation.org/
  • [Williams1996] David B. Williams and C. Barry Carter (1996): The Transmission Electron Microscope. Springer {US}. 10.1007/978-1-4757-2519-3_1
  • [Goto2008] Kazushige Goto and Robert A. van de Geijn (2008): Anatomy of high-performance matrix multiplication. Association for Computing Machinery ({ACM}). 10.1145/1356052.1356053
  • [DaskSpark] {Anaconda Inc.} (2018): Comparison to Spark. http://docs.dask.org/en/latest/spark.html
  • [Lemire2012] Daniel Lemire and Leonid Boytsov: Decoding billions of integers per second through vectorization. 10.1002/spe.2203
  • [grosser2011polly] Grosser, Tobias and Zheng, Hongbin and Aloor, Raghesh and Simb{\"u}rger, Andreas and Gr{\"o}{\ss}linger, Armin and Pouchet, Louis-No{\"e}l: Polly-Polyhedral optimization in {LLVM}. http://perso.ens-lyon.fr/christophe.alias/impact2011/impact-07.pdf
  • [Song2018] Dongsheng Song and Zi-An Li and Jan Caron and Andr{\'{a}}s Kov{\'{a}}cs and Huanfang Tian and Chiming Jin and Haifeng Du and Mingliang Tian and Jianqi Li and Jing Zhu and Rafal E. Dunin-Borkowski (2018): Quantification of Magnetic Surface and Edge States in an {FeGe} Nanostripe by Off-Axis Electron Holography. American Physical Society ({APS}). 10.1103/physrevlett.120.167204
  • [Mehta2016] Parmita Mehta and Sven Dorkenwald and Dongfang Zhao and Tomer Kaftan and Alvin Cheung and Magdalena Balazinska and Ariel Rokem and Andrew Connolly and Jacob Vanderplas and Yusra AlSayyad: Comparative Evaluation of Big-Data Systems on Scientific Image Analytics Workloads. http://arxiv.org/abs/1612.02485v1
  • [Goy2018] Alexandre Goy and Kwabena Arthur and Shuai Li and George Barbastathis (2018): Low Photon Count Phase Retrieval Using Deep Learning. American Physical Society ({APS}). 10.1103/physrevlett.121.243902
  • [DistributedJourney] {{Anaconda, Inc}} (2018): Journey of a Task. https://distributed.dask.org/en/latest/journey.html
  • [AbramovFlux] Dan Abramov (2015): The Case for Flux. https://medium.com/swlh/the-case-for-flux-379b7d1982c6
  • [ReduxCore] Dan Abramov (2018): Redux: Core Concepts. https://redux.js.org/introduction/core-concepts
  • [ReduxReact] Dan Abramov (2018): {Redux}: Usage with {React}. https://redux.js.org/basics/usage-with-react
  • [FIO] Jens Axboe: Flexible I/O Tester. https://github.com/axboe/fio
  • [Caron2018] Caron, Jan (2018): Model-based reconstruction of magnetisation distributions in nanostructures from electron optical phase images [E-Book]. Forschungszentrum, Zentralbibliothek. http://hdl.handle.net/2128/19740
  • [Guzzinati2019] Giulio Guzzinati and Wannes Ghielens and Christoph Mahr and Armand Béché and Andreas Rosenauer and Toon Calders and Jo Verbeeck (2019): Electron Bessel beam diffraction for precise and accurate nanoscale strain mapping. 10.1063/1.5096245
  • [Panova2019] Ouliana Panova and Colin Ophus and Christopher J. Takacs and Karen C. Bustillo and Luke Balhorn and Alberto Salleo and Nitash Balsara and Andrew M. Minor (2019): Diffraction imaging of nanocrystalline structures in organic semiconductor molecular thin films. Springer Science and Business Media {LLC}. 10.1038/s41563-019-0387-3
  • [Gibson1997] J. M. Gibson and M. M. J. Treacy (1997): Diminished Medium-Range Order Observed in Annealed Amorphous Germanium. American Physical Society ({APS}). 10.1103/physrevlett.78.1074
  • [Ophus_2019] Colin Ophus (2019): Four-Dimensional Scanning Transmission Electron Microscopy (4D-{STEM}): From Scanning Nanodiffraction to Ptychography and Beyond. Cambridge University Press ({CUP}). 10.1017/s1431927619000497
  • [6980942] S. C. F. {Lin} and C. Y. {Wong} and G. {Jiang} and M. A. {Rahman} and N. M. {Kwok} (2014): Radial fourier analysis ({{RFA}}) image descriptor. 10.1109/FSKD.2014.6980942
  • [Lichte2008] Lichte, H. and Lehmann, M. (2008): Electron holography–basics and applications. 10.1088/0034-4885/71/1/016102
  • [Schubert2018] Erich Schubert and Michael Gertz (2018): Numerically stable parallel computation of (co-)variance. {ACM} Press. 10.1145/3221269.3223036
  • [ophus_colin_2019_3592520] Ophus, Colin and Savitzky, Benjamin (2019): {Simulated calibration dataset for 4D scanning transmission electron microscopy}. Zenodo. 10.5281/zenodo.3592520
  • [giulio_guzzinati_2019_2566137] Giulio Guzzinati and Wannes Ghielens and Christoph Mahr and Armand Béché and Andreas Rosenauer and Toon Calders and Jo Verbeeck (2019): {Electron Bessel beam diffraction patterns, line scan of Si/SiGe multilayer}. Zenodo. 10.5281/zenodo.2566137
  • [Weber2020] Dieter Weber and Alexander Clausen and Rafal E. Dunin-Borkowski (2020): Next-Generation Information Technology Systems for Fast Detectors in Electron Microscopy. World Scientific. 10.1142/9789811204579_0005
  • [Zeltmann2019] Steven E Zeltmann and Alexander Müller and Karen C Bustillo and Benjamin Savitzky and Lauren Hughes and Andrew M Minor and Colin Ophus (2019): Patterned Probes for High Precision {4D-STEM} Bragg Measurements. 10.1016/j.ultramic.2019.112890