Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018;4(1):11.
doi: 10.1186/s40679-018-0060-y. Epub 2018 Sep 22.

An open-source software ecosystem for the interactive exploration of ultrafast electron scattering data

Laurent P René de Cotret  1 , Martin R Otto  1 , Mark J Stern  1 , Bradley J Siwick  1   2
Affiliations

An open-source software ecosystem for the interactive exploration of ultrafast electron scattering data

Laurent P René de Cotret et al. Adv Struct Chem Imaging. 2018.

Abstract

This paper details a software ecosystem comprising three free and open-source Python packages for processing raw ultrafast electron scattering (UES) data and interactively exploring the processed data. The first package, iris, is graphical user-interface program and library for interactive exploration of UES data. Under the hood, iris makes use of npstreams, an extensions of numpy to streaming array-processing, for high-throughput parallel data reduction. Finally, we present scikit-ued, a library of reusable routines and data structures for analysis of UES data, including specialized image processing algorithms, simulation routines, and crystal structure manipulation operations. In this paper, some of the features or all three packages are highlighted, such as parallel data reduction, image registration, interactive exploration. The packages are fully tested and documented and are released under permissive licenses.

Keywords: Data processing; Open-source; Ultrafast electron scattering; Visualization.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Ultrafast electron scattering data exploration workflow. To maximize signals, ultrafast electron scattering experiments consist in many identical sub-experiments (or scans), as shown on the top right. Data reduction consists in combining these sub-experiments, which usually involve image-alignment, intensity normalization, and other corrections. Each individual time-delay can be reduced in parallel. The resulting reduced data are a stack of scattering patterns, similar to a video. Scattering intensity time-series can be extracted from either a single pixel or integrated over a groups of pixels (lower right)
Fig. 2
Fig. 2
Overview of the GUI component of iris. Two GUI instances show typical datasets. On the top left, Bragg peak dynamics for photoexcited single-crystal data is shown. Diffracted intensity is integrated in the red square and its time dependence is shown in the bottom panel. On the bottom right, azimuthally averaged baseline-corrected polycrystalline scattering data are presented. The pre-photoexcitation scattering patterns have been subtracted so that dynamics are more evident. Diffraction patterns are color-coded based on their time-delay, shown below. Diffracted intensity is integrated inside the blue zone and its time dependence is again shown on the bottom panel. Both integration regions can be interactively dragged, updating the time-series in real-time
Fig. 3
Fig. 3
Performance characterization of the npstreams package in comparison with numpy. The task was to average sequences of 2D arrays (representing scattering patterns). a Wall time of averaging for a sequence of 10 arrays (solid) superimposed with maximum memory usage (dashed). The vertical-dashed line marks the array size of 2048×2048, equivalent to a scattering pattern of 4 megapixels. b Speed-up of using npstreams vs. numpy for averaging a sequence of arrays of size 512×512 elements
Fig. 4
Fig. 4
How to run npstreams benchmarks suite from the interactive Python interpreter. By default, a pre-selected set of functions from both the numpy and npstreams packages are compared
Fig. 5
Fig. 5
Removing image hotspots using an iterative baseline-removal algorithm based on the discrete wavelet transform. a Original scattering pattern shows hotspots due to laser light hitting the detector. b Baseline of a. The hotspots are treated as the foreground by the iterative algorithm
Fig. 6
Fig. 6
Scattering pattern alignment based on the masked normalized cross-correlation algorithm. a Reference scattering patterns of polycrystalline chromium. b Misaligned scattering pattern. c Difference between the patterns in a and b shows structure indicative of a sideways shift. Note that the beam block has not moved. d Pixel mask of the beam block. Black pixels represent zones to be ignored by the alignment procedure. e Aligned image, registered without the mask in d. f Difference between a and e shows residual misalignment. g Aligned image, registered with the mask in d. f Difference between a and g shows successful alignment
Fig. 7
Fig. 7
Example of symmetrization of single-crystal scattering patterns. a Raw single-crystal scattering pattern. Inset shows the pixel mask indicating where the beam-block is located. Pixels under the mask are ignored during the symmetrization routine. c Sixfold rotational symmetrization of scattering pattern in a with beam-block masked reduces noise and enhances dynamics
Fig. 8
Fig. 8
Generating a Crystal instance for vanadium dioxide from the Crystallography Open Database entry 9009089 [34], inside the interactive Python interpreter
Fig. 9
Fig. 9
Simulated electrostatic potential of orthorhombic barium titanate (BaTiO3) projected onto the z=0 plane Structure file was taken from [35]
Fig. 10
Fig. 10
Simulated polycrystalline electron diffraction patterns for gold, graphite, and monoclinic M1 VO2
See this image and copyright information in PMC

Similar articles

  • Array programming with NumPy.
    Harris CR, Millman KJ, van der Walt SJ, Gommers R, Virtanen P, Cournapeau D, Wieser E, Taylor J, Berg S, Smith NJ, Kern R, Picus M, Hoyer S, van Kerkwijk MH, Brett M, Haldane A, Del Río JF, Wiebe M, Peterson P, Gérard-Marchant P, Sheppard K, Reddy T, Weckesser W, Abbasi H, Gohlke C, Oliphant TE. Harris CR, et al. Nature. 2020 Sep;585(7825):357-362. doi: 10.1038/s41586-020-2649-2. Epub 2020 Sep 16. Nature. 2020. PMID: 32939066 Free PMC article. Review.
  • Analyzing microtomography data with Python and the scikit-image library.
    Gouillart E, Nunez-Iglesias J, van der Walt S. Gouillart E, et al. Adv Struct Chem Imaging. 2017;2(1):18. doi: 10.1186/s40679-016-0031-0. Epub 2016 Dec 7. Adv Struct Chem Imaging. 2017. PMID: 29142808 Free PMC article.
  • Survey of Non-Rigid Registration Tools in Medicine.
    Keszei AP, Berkels B, Deserno TM. Keszei AP, et al. J Digit Imaging. 2017 Feb;30(1):102-116. doi: 10.1007/s10278-016-9915-8. J Digit Imaging. 2017. PMID: 27730414 Free PMC article. Review.
  • scikit-image: image processing in Python.
    van der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, Gouillart E, Yu T; scikit-image contributors. van der Walt S, et al. PeerJ. 2014 Jun 19;2:e453. doi: 10.7717/peerj.453. eCollection 2014. PeerJ. 2014. PMID: 25024921 Free PMC article.
  • PyMS: a Python toolkit for processing of gas chromatography-mass spectrometry (GC-MS) data. Application and comparative study of selected tools.
    O'Callaghan S, De Souza DP, Isaac A, Wang Q, Hodkinson L, Olshansky M, Erwin T, Appelbe B, Tull DL, Roessner U, Bacic A, McConville MJ, Likić VA. O'Callaghan S, et al. BMC Bioinformatics. 2012 May 30;13:115. doi: 10.1186/1471-2105-13-115. BMC Bioinformatics. 2012. PMID: 22647087 Free PMC article.
See all similar articles

Cited by

References

    1. Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, Weissig H, Shindyalov IN, Bourne PE. The protein data bank. Nucleic Acids Res. 2000;28(1):235–242. doi: 10.1093/nar/28.1.235. - DOI - PMC - PubMed
    1. Bjrkman T. Cif2cell: generating geometries for electronic structure programs. Comput. Phys. Commun. 2011;182(5):1183–1186. doi: 10.1016/j.cpc.2011.01.013. - DOI
    1. Collette A. Python and HDF5. Sebastopol: O’Reilly; 2013.
    1. Cowley JM. Chapter 11—multi-slice approaches. In: Cowley JM, editor. Diffraction physics (Third Edition), North-Holland Personal Library. 3. Amsterdam: North-Holland; 1995. pp. 231–254.
    1. Dalcin L, Bradshaw R, Smith K, Citro C, Behnel S, Seljebotn DS. Cython: the best of both worlds. Comput. Sci. Eng. 2010;13:31–39.

LinkOut - more resources

-