Localizing and Characterizing Colloidal Particles Scattering Using Lens-free Holographic Microscopy

Xia Hua(华夏)$^{\dag}$, Cheng Yang(杨程)$^{\dag}$, Ye Huang(黄烨), Feng Yan(闫锋), Xun Cao(曹汛)$^{\hyperlink{s}{**}}$

doi:10.1088/0256-307X/36/1/014204

⇦Chinese Physics Letters, 2019, Vol. 36, No. 1, Article code 014204⇨ Localizing and Characterizing Colloidal Particles Scattering Using Lens-free Holographic Microscopy * Xia Hua (华夏)†, Cheng Yang (杨程)†, Ye Huang (黄烨), Feng Yan (闫锋), Xun Cao (曹汛)** Affiliations School of Electronic Science and Engineering, Nanjing University, Nanjing 210093 Received 24 September 2018, online 25 December 2018 ^*Supported by the National Key R&D Program of China under Grant No 2016YFA0202102, the Fundamental Research Funds for the Central Universities under Grant No 14380078, and the Scientific Research Foundation of Graduate School of Nanjing University under Grant No 2017CL02.
^†Xia Hua and Cheng Yang contributed equally to this work.
^**Corresponding author. Email: caoxun@nju.edu.cn Citation Text: Hua X, Yang C, Huang Y, Yan F and Cao X et al 2019 Chin. Phys. Lett. 36 014204 Abstract Lens-free holographic microscopy could achieve both improved resolution and field of view (FOV), which has huge potential applications in biomedicine, fluid mechanics and soft matter physics. Unfortunately, due to the limited sensor pixel size, target objects could not be located to a satisfactory level. Recent studies have shown that electromagnetic scattering can be fitted to digital holograms to obtain the 3D positions of isolated colloidal spheres with nanometer precision and millisecond temporal resolution. Here, we describe a lens-free holographic imaging technique that fits multi-sphere superposition scattering to digital holograms to obtain in situ particle position and model parameters: size and refractive index of colloidal spheres. We show that the proposed method can be utilized to analyze the location and character of colloidal particles under large FOV with high density. DOI:10.1088/0256-307X/36/1/014204 PACS:42.40.-i, 42.30.Va, 42.40.Lx © 2019 Chinese Physics Society Article Text A great number of existing and emerging applications would benefit from tracking colloidal particles' motions in three dimensions. There are other applications that require accurate characterizing of colloidal particles, especially when this information could be obtained simultaneously. The technology of in-line holographic microscopy combined with the theory of light scattering has been a powerful tool in the research of soft materials, especially in characterizing and tracking colloids or other soft materials. The particles are dispersed in a fluid and the diameters vary from a few hundreds of nanometers to a few micrometers. Lee et al. interpreted holographic snapshots of individual colloidal particles with the Lorenz–Mie scattering theory to obtain the precise information about the location, as well as the size and refractive index of the particles.[1] This technological scheme has been proven to yield similarly precise and meaningful results for imperfect spheres, such as individual dimpled colloidal spheres, and has been proved to be efficient for not only isolated particles but also multiple colloidal particles spaced several wavelengths or more from each other.[2] The Lorenz–Mie theory of light scattering has then been extended to work for both colloidal fractal aggregates[3] and protein aggregates[4] and has been utilized in detection or characterization colloidal particles dispersed in turbid media,[5] like waste-water pollution source monitoring.[6] Previous work was built upon a standard inverted optical microscopy, replacing the conventional incandescent illuminator and condenser with collimated and attenuated HeNe laser. A conventional objective is utilized to magnify the interference pattern and then the hologram is recorded with a grey scale video camera. However, this technique suffers from the inherent trade-off between the large field-of-view (FOV) and high imaging resolution. The high magnification of those holographic microscopy prevents the usage of this technique in the scenario that needs to localize and character multiple colloidal particles under large FOV. To obtain an image with both high resolution and large FOV, mechanical scanning and stitching are required to expand the limited FOV, which not only complicates the imaging procedure but also significantly increases the overall cost of these systems. Even so, the time resolution will still be compromised. Lens-free holographic microscopy has recently emerged as a new imaging technique in various applications.[7-9] In contrast to lens-based holographic microscopy, lens-free holographic microscopy directly samples the light transmitted through the object without the use of any imaging lenses between the object and the sensor planes, hence the space-bandwidth product is decoupled from the spatial resolution. Holography[10,11] encodes the three-dimensional (3D) information of the sample through interference of the object's scattered light with the reference wave, and the depth of a sample and its original image are reconstructed via auto-focusing algorithm and phase retrieval. Using the unit-magnification configuration, lens-free holography microscopy has the native FOV of the imaging sensor, without requiring any lenses and other intermediate optical components. This further allows to significantly simplify the hardware setup and effectively circumvent the optical aberrations and chromaticity that are unavoidable in conventional lens-based imaging systems. Moreover, the whole system is compact and cost efficiency, providing a potential solution for tacking colloidal particles' motion and character in the resource-limited environment. This work shows that images captured by a simple lens-free holographic microscopy could be combined with the Lorenz–Mie theory to obtain accurate 3D position and optical character of colloidal spheres under full FOV of the imaging sensor. This method works over the entire range of particle sizes and compositions for which the Lorenz–Mie theory applies. We believe that the present approach can be broadly applicable to the study of measuring interactions in colloidal systems and particle image velocimetry, where large FOV is required. Our lens-free holographic microscopy is built with a commercial gray-scale video camera (Imaging Source DMK 27AUJ003), pixel size (1.67 µm). This system provides a total $6.17\times 4.55$ mm$^{2}$ field of view. We illuminate the sample with a coherent beam (uniphase, 20 mV, $\lambda=532$ nm). A diagram of our system is given in Fig. 1.

Fig. 1. The schematic diagram of the experimental configure.

The light scattered by a particle then interferences with un-diffracted light, which is considered as the reference beam, a two-dimensional interference pattern that encodes information about the 3D positions and characters of particles is formed at the sensor plane and captured by the imaging sensor. The 3D location and characters of colloidal particles is achieved by fitting the pattern, pixel by pixel, to the Lorenz–Mie equation. The whole view of a lens-free image obtained by our experimental setup with FOV of more than 24 mm$^2$ is shown in Fig. 2. Here, we describe the forward model for hologram formation from a single spherical colloidal particle and the least-squares method used to fit this model to the data. We follow the work of Lee et al.[1] closely. In their model, a hologram $H$ is computed as functions of the 3D position ${x,y,z}$, its radius $r_{\rm p}$, its refractive index $n_{\rm p}$. The scattered field generally a large 2D pattern at the focal plane is $$\begin{align} I(\rho)=|E_{\rm s}(r)+E_{\rm r}(r)|^{2}.~~ \tag {1} \end{align} $$ A colloidal particle scatters a portion $E_{\rm s}(r)$ of a coherent laser beam $E_{\rm r}(r)$. The scattered beam interferes with the unscattered portion of the beam at the imaging sensor, thereby forming an in-line hologram, $I(\rho)$. The normalized image at $z_{\rm p}$ is related to the calculated Mie scattering pattern $f_{\rm s}(r)$, in the plane $z=0$ by $$\begin{align} B(\rho)=\,&\frac{I(\rho)}{|u_{0}(\rho)|^{2}}=1+2\alpha R\{ f_{\rm s}(r-r_{\rm p})\\ &\cdot\hat{\varepsilon }e^{-ikz_{\rm p}} \}+|f_{\rm s}(r-r_{\rm p})|^{2},~~ \tag {2} \end{align} $$ which can be fitted to the Lorenz–Mie equation regarding to the particle's 3D position, radius and its refractive index as free parameters.

Fig. 2. Lens-free image obtained by our experimental setup with FOV of more than 24 mm$^{2}$.

Fig. 3. Fitting single colloidal particle to normalized holograms. (a) Normalized hologram for a 9.5 µm silica sphere in water at 893 µm, numerical fit to Eq. (1). (b) Data for a 4 µm diameter PS sphere in water at 534 µm. (c) Data for a 2 µm diameter silica sphere in water at 529 µm.

Figure 3 shows a normalized hologram for a single silica sphere dispersed in water at three different heights above the imaging sensor. This sphere was obtained from a commercial sample with a nominal diameter of $9.51\pm 0.751$ µm (Bangs Labs, PS07N/12201). Fitting the Lorenz–Mie solution to the hologram could acquire the position of sphere object with nanometer precision, hence the spatial resolution is much higher than the commonly used angular spectrum method in holographic reconstruction. Least-square fitting method requires good initial guess and constraint on parameters. We observed that estimating the initial guess directly from the raw hologram is difficult. Alternatively, we forth-propagated the raw hologram back to the object plane with the angular spectrum method and we estimate the initial guess from the reconstruction result other than the raw hologram. To achieve a robust fitting result, we fit only a subset of the total pixels once a time, then multiple results are performed by a RANSAC algorithm[12] to eliminate outliers, which may refer to the saddle point. Firstly, we use auto focusing algorithm and angular spectrum method to forth-propagate the hologram from the sensor plane to object plane, then a rough guess of 3D positions and the radius is achieved. To have a rough guess of the refractive index, we draw 100 samples form uniform distribution ranging from 1.0 to 2.0 as candidate for the initial guess of refractive index. Then we use these candidates to calculate the forward guess, that is to say, to generate the hologram with these candidates, then we calculate the mean square error between the raw hologram date with the generated data. We choose the one that minimizes the MSE among all the candidate as our rough initial guess of refractive index. Numerical fits to digitized and normalized holographic images were performed with the Levenberg–Marquardt nonlinear least-squares minimization algorithm.

Fig. 4. Fitting superposed colloidal particles to normalized holograms. (a) Normalized hologram for two 9.5 µm silica spheres in water at 1107.4 µm, 1123.0 µm, numerical fitted to Eq. (1). (b) Data for a 4 µm diameter silica sphere in water at 728.2 µm, 764.3 µm. (c) Data for a 4 µm diameter PS sphere in water at 677.2 µm, 603.3 µm. (d) Data for a 2 µm diameter silica sphere in water at 533.0 µm, 591.7 µm.

Figure 4 shows the normalized hologram for superposed silica sphere and polystyrene sulfate sphere with different diameters dispersed in water above the imaging sensor. These spheres were obtained from a commercial sample with nominal diameters of 9.51$\pm $0.751 µm (Bangs Labs, PS07 N/12201); 4.07 µm (Bangs Labs, SS05N/12602); 4.19$\pm $0.27 µm (Bangs Labs, PS05N/6910); and 2.14 µm (Bangs Labs, SS04N/8258). When resolving multiple colloidal particles, the accuracy of initial guess is even more deterministic.[13] To search for a more accurate initial guess, a gradually refined strategy is proposed. The gradually refine strategy searching from a border range with a large variance to a narrow scope with a small variance ensures a better initial guess. Without loss of generality, we start with randomly choosing the candidate of the initial guess on a border range. We sample every candidate from a Gaussian distribution, for which the mean value is the initial guess. For each parameter, we choose the value calculated from the holographic reconstruction as the mean value of our sampling function. The random subset technique[14] is also applied to speed up the process. We used those initial guess candidates to compute the forward model, the Lorenz–Mie theory, and choose the one that minimizes the MSE between the forward model and normalized hologram.

Fig. 5. Pseudocode for the proposed method.

The candidates of initial guess for the next stage are sampled from another Gaussian distribution, taking the result from the previous stage as the new mean value. To achieve a robust result, the volume of the random subset is gradually increased. In each stage, we fit the hologram using the Levenberg–Marquardt nonlinear least-squares minimization algorithm with respect to the current initial guess. After computing all the candidates, we choose the one that achieves the best fitting result as the input of next stage. The algorithm is terminated according to a given tolerance. Algorithms can be included using the commands, as shown in Fig. 5. The proposed method is under a diffraction-limited imaging system, thus the radial resolution is diffraction limit. Derived from the Rayleigh criterion, as shown in Fig. 6, two particles are regarded as just resolved when the principal diffraction maximum of one image coincides with the first minimum of the other.

Fig. 6. Particles far apart (a) or meeting the Rayleigh criterion (b) can be distinguished. Particles closer than the Rayleigh criterion (c) are difficult to distinguish.

Fig. 7. (a) The raw data obtained by our setup. (b) Fitting multiple superposed colloidal particles to normalized holograms. (c) The 3D plot of multiple colloidal particles, where different colors represent different refractive indexes.

The 3D plot of multiple colloidal particles is shown in Fig. 7. This demonstrates the ability of the proposed method to localize and characterize multiple particles under large field of view, where different colors represent different refractive indexes. In conclusion, we have demonstrated that a lens-free holographic microscopy with only a monochromatic imaging sensor and a coherent illumination can be used to measure a colloidal sphere's position and size with nanometer-scale resolution, and its refractive index with high precision under full FOV at one time. Unlike model-based analytical methods, fitting to the exact Lorenz–Mie scattering theory is robust and reliable over a far wider range of particle sizes. Four different kinds of colloidal particles ranging from 2 µm to 9 µm are used to demonstrate the reliability of numerical result. Some issues still deserve further consideration. For example, one crucial step is the choice of the initial guess of auto-focusing procedure, where the accuracy is vital to the quality of the localization and characterization result. However, when superposed multiple colloidal particles that stay at different depths, auto-focusing tends to give out an average depth. Hence, the accuracy of the initial guess is compromised. The fundamental limit of lens-free holographic microscopy is the effective pixel size. Due to the spatial under-sampling, the imaging sensor will fail to record diffraction rings corresponding to high spatial frequency information of the samples. Using a sensor with smaller pixel size can directly alleviate the spatial under sampling problem. In the future, we aim at applying pixel-super-resolution techniques to further improve the accuracy of colloidal particle tracking and characterizing. References Characterizing and tracking single colloidal particles with video holographic microscopyImaging multiple colloidal particles by fitting electromagnetic scattering solutions to digital hologramsHolographic characterization of colloidal fractal aggregatesHolographic Characterization of Protein AggregatesHolographic characterization of colloidal particles in turbid mediaHolographic characterization of contaminants in water: Differentiation of suspended particles in heterogeneous dispersionsImaging without lenses: achievements and remaining challenges of wide-field on-chip microscopyLensless high-resolution on-chip optofluidic microscopes for Caenorhabditis elegans and cell imagingLensfree holographic imaging for on-chip cytometry and diagnosticsDigital in-line holography for biological applicationsA pattern matching approach to selection of particles from low-contrast electron micrographsBayesian approach to analyzing holograms of colloidal particlesNon-linear least-squares fitting in IDL with MPFIT

[1]	Lee S H , Roichman Y, Yi G R , Kim S H , Yang S M , Blaaderen A, Oostrum P and Grier D G 2007 Opt. Express 15 18275
[2]	Fung J, Perry R W, Dimiduk T G and Manoharan V N 2012 J. Quant. Spectrosc. Radiat. Transfer 113 2482
[3]	Wang C, Cheong F C, Ruffner D B, Zhong X, Ward M D and Grier D G 2016 Soft Matter 12 8774
[4]	Wang C, Zhong X, Ruffner D B, Stutt A, Philips L A, Ward M D and Grier D G 2016 J. Pharm. Sci. 105 1074
[5]	Cheong F C, Kasimbeg P, Ruffner D B, Hlaing E H, Blusewicz J M, Philips L A and Grier D G 2017 Appl. Phys. Lett. 111 153702
[6]	Philips L A, Ruffner D B, Cheong F C, Blusewicz J M, Kasimbeg P, Waisi B, McCutcheon J R and Grier D G 2017 Water Resour. Res. 122 431
[7]	Greenbaum A, Luo W, Su T W , Göröcs Z, Xue L, Isikman S O, Coskun A F, Mudanyali O and Ozcan A 2012 Nat. Methods 9 889
[8]	Cui X, Lee L M , Heng X, Zhong W, Sternberg P W, Psaltis D and Yang C 2008 Proc. Natl. Acad. Sci. USA 105 10670
[9]	Seo S, Su T W , Tseng D K, Erlinger A and Ozcan A 2009 Lab Chip 9 777
[10]	Goodman J W 2005 Introduction to Fourier Optics (Englewood: Roberts and Company Publishers)
[11]	Xu W, Jericho M, Meinertzhagen I and Kreuzer H 2001 Proc. Natl. Acad. Sci. USA 98 11301
[12]	Vargas J, Alvarez-Cabrera A L , Marabini R, Carazo J M and Sorzano C O S 2014 Bioinformatics 30 2891
[13]	Dimiduk T G and Manoharan V N 2016 Opt. Express 24 24045
[14]	Dimiduk T G, Perry R W, Fung J and Manoharan V N 2014 Appl. Opt. 53 G177