Practical Electron Microscopy and Database

An Online Book, Second Edition by Dr. Yougui Liao (2006)

Practical Electron Microscopy and Database - An Online Book

Chapter/Index: Introduction | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | Appendix

k-Means Clustering for Sorting Diffraction Patterns

In the study of nanoscale crystals using scanning electron nanodiffraction patterns, e.g. with 4D STEM-Diffraction, k-means clustering has been applied for sorting electron diffraction patterns based on their structural characteristics. This clustering approach helps analyze variations in the lattice orientations across different regions of peptide crystals (e.g. page0008), which might otherwise remain undetected. By categorizing diffraction patterns into clusters, k-means enables the identification of regions with similar structural properties, thereby revealing shifts in lattice orientation and the presence of nanoscale subdomains within the crystals.

To execute k-means clustering, each diffraction pattern's Euclidean distance from the cluster's average pattern was calculated (see ICs/page4254), iteratively assigning patterns to the closest cluster center. This process was refined by masking the primary beam to avoid its influence, ensuring that the clustering reflects true structural variations rather than artifacts of intense central diffraction signals. The classification method used here was critical for mapping structural heterogeneity across crystals, facilitating deeper insights into the internal mosaicity of these microcrystals. Through this unsupervised classification of machine learning, the study effectively highlighted subtle orientation changes that are difficult to observe through standard imaging or diffraction methods.

The procedure of k-means clustering for sorting diffraction patterns can be:

  • Data Collection and Preparation: The crystals are analyzed using 4D Scanning Transmission Electron Microscopy (4D-STEM). This involves capturing diffraction patterns at various points across the crystal to create a comprehensive dataset. Diffraction patterns are then pre-processed to correct for background noise and beam shifts.
    • Hybrid Electron Counting: The diffraction data undergoes hybrid electron counting to improve signal quality. This process differentiates individual electron events and helps retain dynamic range by addressing electron coincidence losses, which is essential for accurate clustering.
  • Clustering with k-means:
    • The diffraction patterns are processed to identify regions with similar lattice orientations using k-means clustering.
    • Initial cluster centers are chosen, and each diffraction pattern is assigned to the closest cluster based on Euclidean distance, with iterative reclassification until convergence.
    • To reduce noise impact, the primary beam region is masked, ensuring clustering is driven by relevant diffraction data rather than background or direct beam effects.
    • The final clustering result reveals areas within the crystal with distinct orientations, mapped to show orientation variations across the crystal.
    For instance, Figure 8 illustrates the workflow for unsupervised clustering to define regions with similar diffraction patterns within peptide crystals. The clustering process starts with the initialization of cluster centers within the 4DSTEM dataset, followed by masking the primary beam to reduce its impact on classification. Diffraction patterns are sequentially compared to each cluster center, using Euclidean distance to assign them to the nearest cluster. This iterative approach continues with recalculating average patterns for each cluster and reassigning individual patterns until convergence. The resulting clustering map shows the spatial distribution of distinct diffraction regions across the crystal, allowing visualization of nanoscale lattice variations with sub-100 nm resolution. This approach enables detailed analysis of lattice reorientation and structural differences within each peptide crystal.
  • Indexing and Interpretation: Cluster averages are compared against a library of simulated diffraction patterns to index lattice orientations and quantify reorientation angles, enabling visualization of subtle orientation changes within the crystal structure.
  • HRTEM Comparison: Additional analysis using High-Resolution Transmission Electron Microscopy (HRTEM) provides visual context, although 4DSTEM data yields finer structural insights due to its sensitivity to subtle lattice changes.

Workflow of unsupervised clustering to identify regions with similar diffraction patterns in peptide crystals. (Step 1) Initial cluster centers are selected within the 4DSTEM dataset (shown as white boxes on the left), and all diffraction patterns are grouped accordingly. (Step 2) The primary beam is masked to prevent it from influencing the clustering process. (Step 3) Each diffraction pattern is sequentially compared to each cluster center using Euclidean distance and assigned to the closest cluster. (Step 4) Average diffraction patterns are calculated for each cluster, and (step 5) these averages are used to reassign individual patterns until the clustering stabilizes (Step 6), resulting in a final map showing the spatial distribution of regions with similar diffraction patterns
Figure 8. Workflow of unsupervised clustering to identify regions with similar diffraction patterns in peptide crystals. (Step 1) Initial cluster centers are selected within the 4DSTEM dataset (shown as white boxes on the left), and all diffraction patterns are grouped accordingly. (Step 2) The primary beam is masked to prevent it from influencing the clustering process. (Step 3) Each diffraction pattern is sequentially compared to each cluster center using Euclidean distance and assigned to the closest cluster. (Step 4) Average diffraction patterns are calculated for each cluster, and (step 5) these averages are used to reassign individual patterns until the clustering stabilizes (Step 6), resulting in a final map showing the spatial distribution of regions with similar diffraction patterns. [1]

 

 

 

 

 

 

 

 

 

[1] Gallagher-Jones M, Ophus C, Bustillo KC, Boyer DR, Panova O, Glynn C, Zee C-T, Ciston J, Mancia KC, Minor AM & Rodriguez JA (2019). Nanoscale mosaicity revealed in peptide microcrystals by scanning electron nanodiffraction. Commun Biol 2, 26.