⇦Chinese Physics Letters, 2016, Vol. 33, No. 2, Article code 024204⇨ Numerical Evaluation of Effect of Motion of Samples on Ptychographic Imaging and Solution with a Random Phase Modulator * Zhao-Hui Li(李昭慧)1**, Jian-Qi Zhang(张建奇)1,2, De-Lian Liu(刘德连)1, Xiao-Rui Wang(王晓蕊)1 Affiliations 1School of Physics and Optoelectronic Engineering, Xidian University, Xi'an 710071 2Collaborative Innovation Center of Information Sensing and Understanding, Xidian University, Xi'an 710071 Received 24 September 2015 ^*Supported by the National Natural Science Foundation of China under Grant Nos 61301290 and 61377007, and the Fundamental Research Funds for the Central Universities under Grant Nos NSIY151410 and NSIZ011401.
^**Corresponding author. Email: lzh791001@163.com Citation Text: Li Z H, Zhang J Q, Liu D L and Wang X R 2016 Chin. Phys. Lett. 33 024204 Abstract The retrieval of ptychographical imaging encounters a degradation produced by motion of samples. The mechanism of capturing the diffractive patterns of a sample which is motile is simulated numerically. The relation between the retrieval degradation and the amplitude of motion is evaluated quantitatively. Experiments indicate that the reliability and resolution of the complex amplitude retrieval of a sample is inversely proportional to its vibratory amplitude. A random phase modulated aperture is employed to cure the degradation produced by the motion of samples to a certain extent. DOI:10.1088/0256-307X/33/2/024204 PACS:42.30.-d, 42.25.Fx, 42.30.Rx, 42.30.Kq © 2016 Chinese Physics Society Article Text Ptychographical imaging engine (PIE) is a novel method for coherent diffractive imaging (CDI). PIE searches the unique complex solution of a sample which is consistent with its diffractive intensity measurements under overlap illuminations by phase retrieval iteration.[1-3] Ptychography has uniquely advantages: (i) It records the far-field diffractive intensity patterns, hence it alleviates focusing devices, breaking through the physical diffraction limitation and achieving high-resolution detections. (ii) It uses a light probe (or an illumination aperture) to illuminate regions of a sample successively and overlapping partly, which can synthesize an arbitrary field of view (FOV) and can solve the inherent ambiguity of conventional CDI. In the past five years, PIE has been applied in many fields such as biomedical research,[4-6] materials science,[7,8] precision measurement[9] and computer holography.[10] In Refs. [11–13], Liu et al. carried out innovative works about the effects on ptychography from the coherency of illumination, the characters of CCD, and the phase retrial algorithms. In Refs. [14,15], Fu et al. studied the performance of PIE and proposed a novel algorithm to promote its adaption. In Ref. [16], Shi et al. presented an outstanding approach to cancel the machine scanning time and to keep diffractive patterns captured correctly. In Ref. [17], Chris et al. creatively proposed a continuous motion scan ptychography to increase the imaging speed. They performed scans in a linear relative motion with constant velocity between the sample and the camera, rather than in random motion. Using aperture coding and wave front modulating in imaging systems is a convenient and cheap strategy to enhance the imaging quality, which is widely accepted.[18-21] Recently, a random phase mask was introduced in ptychography to reduce the dynamic range of diffraction patterns,[20] or to resist the imaging worsening caused by transverse shift errors of the probe.[21] However, a far-field diffractive pattern cannot be recorded clearly unless through a sufficiently long exposure. In addition, capturing the entire sample by mechanical scanning will consume a substantially long time. Therefore, hitherto ptychography focused on the imaging of static or slowly changing samples. In fact, motions commonly exist in nature, for example, the self-swimming of microorganisms, the migration of motile cells, the deformation of biological tissues in vivo. Nevertheless, to the best of our knowledge, there has been no report on a sample that is in speedy motion measured by ptychography. To expand practical applications of ptychography, it is necessary to examine the influence of motions of samples on PIE. In this Letter, we investigate the influence from motion of a sample on ptychographical imaging, and reveal the fact that the motion makes the retrieval quality decline and aggravates the degradation with the increasing vibratory amplitude. We find that a random modulator has an ability of reducing the degradation caused by motion of samples to a certain extent. The model of ptychographical imaging setup used in the present study is shown in Fig. 1(a). An incident beam passes through an aperture, which is a light probe, illuminates and transmits a region of a sample. The normal incidence light is a coherent plane wave with wavelength $\lambda=638$ nm and unit amplitude. A rectangular aperture with width of $W$ is applied. The probe shifts in the sample plane of $M\times N$ times with step pitch of $p$, as illustrated in Fig. 1(b). In Ref. [22], Shi et al. studied the effect of overlap illumination ratio on reconstruction quality of PIE and found that 50% ratio at least was a guarantee of a unique solution of iteration, and a ratio higher than 50% brought a faster convergence, simultaneity, more scanning and record times. Since a rapid recording procedure is necessary in the ptychographical imaging of a motile sample, we choose 50% overlap ratio, which means that $p$ is set as $W/2$. The probe function $P_{m,n}$ at the $(m,n)$th shift is expressed as $$\begin{align} P_{m,n}=\,&P(x-x_m,y-y_n)\\ =\,&\begin{cases} 1, & {\rm rect}\Big(\frac{x-x_m}{W},\frac{y-y_n}{W}\Big),\\ 0, & {\rm otherwise}, \end{cases}~~ \tag {1} \end{align} $$ where $(x_m,y_n)$ is the central position coordinates of $P_{m,n}$.

Fig. 1. Models of ptychographical imaging and motion of sample. (a) The scheme of ptychography setting with an RPM, (b) the scanning model, and (c) a part of the vibratory-vector map of a motile sample.

A pseudo-random phase modulator clinging to the empty aperture is employed in this system. The transmission function of the random phase modulator (RPM) is expressed as $S(x,y)=\exp[j\pi R]$, where $R$ is a random fraction distributed in (0,1) uniformly. The probe function with RPM is $$\begin{align} P^{\rm s}_{m,n}=P_{m,n}\cdot S(x,y).~~ \tag {2} \end{align} $$ A sample in a static state has a complex amplitude transmission of $O(x,y)=|O(x,y)|\exp{[j\phi_0(x,y)]}$. Now we deduce the transmission when the sample is moving. When the sample is thin and the diffraction is far-field type, in comparison with the diffractive distance, the thickness of the sample is too small to be ignored. We concentrate on the motion that appears in the interior of the thin sample, so the displacement on the $Z$-axis can be ignored. Due to the fact that any complicated motion can be decomposed into a set of vibrations with different frequencies, we should only consider the vibrations on the plane perpendicular to the $Z$ axis. Suppose that the parts (some vivo tissues, or living microorganisms) in the sample are vibrating independently. At the time $t$, the displacement of a part is varying in sinusoidal format in its vibratory direction along a horizontal angle $\varphi_{\rm a}$, $$\begin{align} d(t)=A_m\cdot W\cdot \sin{(2\pi f_vt+\varphi_{\rm v})},~~ \tag {3} \end{align} $$ where $A_m$ is a positive fraction which denotes the ratio of vibratory amplitude to the width of aperture, $f_{\rm v}$ is the vibratory frequency, and $\varphi_{\rm v}$ is the initial phase offset. Both $\varphi_{\rm a}$ and $\varphi_{\rm v}$ take random values in $(0,2\pi)$, this guarantees an unpredictable internal random motion. We assume that parts of the sample vibrate in the same frequency and amplitude respectively, while in different directions and at different initial positions, they are defined by random $\varphi_{\rm a}$ and random $\varphi_{\rm v}$. The vibration can be designated by a vector graphically. A region of the vibratory-vector map of the sample is depicted in Fig. 1.(c). The position of a part in the sample at time $t$ is calculated by $$\begin{alignat}{1} \!\!\!\!\![x(t),y(t)]=[x-d(t)\cos(\varphi_{\rm a}),y-d(t)\sin(\varphi_{\rm a})].~~ \tag {4} \end{alignat} $$ The transmission function of the sample involving variable $t$ is expressed as $$\begin{align} O(x,y,t)=O[x(t),y(t)].~~ \tag {5} \end{align} $$ The exiting wave front after the oscillating sample illuminated by $P_{m,n}$ or $P^{\rm s}_{m,n}$ reads $$\begin{align} E_{m,n}(x,y,t)=\,&O(x,y,t)\cdot P_{m,n},~~ \tag {6} \end{align} $$ $$\begin{align} E^{\rm s}_{m,n}(x,y,t)=\,&O(x,y,t)\cdot P^{\rm s}_{m,n}.~~ \tag {7} \end{align} $$ A diffractive pattern of the exiting wave front occurs at the far-field plane and can be recorded by a CCD. The captured image at one exposure is an accumulation of the intensity of diffraction during the exposure time of $T$, $$\begin{align} I_{m,n}=\,&\int_0^T|\mathscr{F}\{E_{m,n}(x,y,t)\}|^2dt,~~ \tag {8} \end{align} $$ $$\begin{align} I^{\rm s}_{m,n}=\,&\int_0^T|\mathscr {F}\{E^{\rm s}_{m,n}(x,y,t)\}|^2dt,~~ \tag {9} \end{align} $$ where $\mathscr{F}$ represents a Fraunhofer propagation in optics, and a Fourier transformation in calculation.

Fig. 2. The diffractive patterns of a static sample taken from a rectangular aperture (a) without RPM and (b) with RPM.

Shifting and settling the probe in sample plane $M\times N$ times, we obtain a series of diffractive patterns. Some of $I_{m,n}$ and $I^{\rm s}_{m,n}$ are shown in Figs. 2(a) and 2(b), respectively. Compared with $I_{m,n}$ in which the diffractive information is centripetal, $I^{\rm s}_{m,n}$ in which the diffractive information distributes evenly in the Fourier space is less sensitive to the phase shifts arising from the motion in sample space. The procedure of reconstructing the complex amplitude of a sample by PIE is explained as follows: (i) Create a random complex matrix as an initial guess of the transmission of the sample: $O_0(x,y)=\exp{[j\varphi(x,y)]}$, where $\varphi(x,y)$ is a random phase function. (ii) Calculate the exiting wave front of a region of the sample illuminated by the probe at the $(m,n)$th translation: $E^0_{m,n}=O_0(x,y)P_{m,n}$. A Fourier transform is employed to obtain its far-field diffraction distribution: $D^0_{m,n}=\mathscr{F}\{E^0_{m,n}\}$. (iii) Replace the modulus of $D^0_{m,n}$ with the square-root of the recorded intensity pattern $I_{m,n}$, and keep the phase, called $D^1_{m,n}=\sqrt{I_{m,n}}D^0_{m,n}/|D^0_{m,n}|$. (iv) Inversely propagate $D^1_{m,n}$ to the space of the sample, an inverse fast Fourier transform being applied, $E^1_{m,n}=\mathscr{F}^{-1}\{D^1_{m,n}\}$. Update $O_0(x,y)$ to $O_0^1(x,y)$ by the formula $$\begin{alignat}{1} O_0^1=&O_0+\alpha\frac{P_{m,n}^*|P_{m,n}|\times[E^1_{m,n}-E^0_{m,n}]} {(|P_{m,n}|^2+\beta)\max(|P_{m,n}|)},~~ \tag {10} \end{alignat} $$ where $\alpha$ and $\beta$ are fractional constants. (v) Translate the probe to the next location $(m+1,n)$ with pitch of $p$, repeat steps 1–4 until the entire sample is accessed, at which point a single iteration is completed, the sample is refreshed totally to $O_0^{M\times N}(x,y)$. Set $O_1(x,y)=O_0^{M\times N}(x,y)$ as the new guess of the sample for the next iteration, then repeat steps 2–5. (vi) The iterative cycle is continued until the predefined times of iterations or an error threshold is achieved.[3] The final complex amplitude $O_k(x,y)$ is obtained where the index $k$ is the iteration times. Note that when an RPM is added, $P_{m,n}$ should be replaced by $P^{\rm s}_{m,n}$, and $I_{m,n}$ should be replaced by $I^{\rm s}_{m,n}$. An experiment is performed to estimate the effect of motion of sample on ptychography and the restoration is brought by an RPM. A sample and its subregion are shown in Fig. 3(a). A resolution target is used as its modulus, and a continuous-gray-level image is treated as phase distribution with brightness representing range of $(0,\pi)$. The sample is cut off into many pieces, each one represents a part of the sample. These parts are oscillating with $f_{\rm v}$ of 10 Hz with $A_m$ increasing from 0 to 0.05 linearly. Since the width of aperture is approximately 100 pixels in this experiment, $A_m=0.01$ corresponds to a case that the displacement of the sample exceeds the minimum distinguishable scale, or resolution. Scan the sample by an aperture with or without an RPM and capture the diffractive patterns. Diffractive patterns are numerically achieved by the discrete Eqs. (8) and (9). The exposure time is 0.5 s, which is shorter than that in the previous reports.[3,7,20] The temporal interval of the discrete integral is 10 ms, which is a typical value of CCD response time. Thus each diffractive pattern is the sum of diffractive intensities at every 10 ms during 0.5 s. Then reconstruct the sample. Some reconstructed results with or without RPM at the same locations under different $A_m$ are illustrated in Figs. 3(b)–3(d).

Fig. 3. Experimental results: (a) original transmission amplitude of a static sample and several reconstructions of the sample in motion with or without an RPM at (b) $A_m=0.01$, (c) $A_m=0.03$, and (d) $A_m=0.05$.

It can be observed from these experimental results that the degradation is exacerbated as the vibratory amplitude increases. When an RPM is introduced, it cures the degradation partly. In Fig. 3(b), one can observe a serious distortion of lines in the recovered modulus image without RPM when $A_m=0.01$, while the lines in the modulus image with RPM still keep clear shapes. In Fig. 3(c), at $A_m=0.03$, the lines in the modulus image without RPM twine around each other, while the lines in the modulus image with RPM are separated to be discriminated. In Fig. 3(d), at $A_m=0.05$, the lines in the modulus image without RPM are chaotic and the ones in the modulus image with RPM are defocused and accompanied with ghost images so that they cannot be recognized. The advantages of RPM are lost when the vibratory amplitude is substantially high. To evaluate ground noise and similarity quantitatively, mean square error (MSE) and correlation coefficient (CC) between the original static sample and the reconstructed sample are calculated. The CC and MSE for modulus and phase should be evaluated. The MSE between a reconstructed image $\hat{f}(x,y)$ and its original image $f(x,y)$ is defined as $$\begin{alignat}{1} {\rm MSE}=\frac{1}{XY}\sum^X_{x=1}\sum^Y_{y=1}[f(x,y)-\hat{f}(x,y)]^2.~~ \tag {11} \end{alignat} $$ The correlation coefficient (CC) is defined as $$\begin{align} {\rm CC}=\frac{{\rm conv}[f(x,y)-\hat{f}(x,y)]}{\sigma\hat{\sigma}},~~ \tag {12} \end{align} $$ where the operator ${\rm conv}$ represents a cross covariance, $\sigma$ and $\hat{\sigma}$ represent the standard deviations of $f(x,y)$ and $\hat{f}(x,y)$, respectively.[15]

Fig. 4. CC and MSE curves of the reconstructed results at (a) $A_m=0.01$, (b) $A_m=0.03$, and (c) $A_m=0.05$.

Figures 4(a)–4(c) are the CC and MSE curves of the reconstructed results from diffractive patterns taken by an aperture with or without RPM varying with iterative times at $A_m=0.01$, $A_m=0.03$, and $A_m=0.05$, respectively. The highest values these CC curves can reach finally present a decline trend with the vibratory amplitude increasing. The CC curves of phase are at the top of those of the modulus all the time, further proving that characters of samples, which have fine structures such as modulus, are more easily affected by motion than those which have smooth distribution, as phase. The CC curves with RPM jump to the highest values immediately, while the ones without RPM slowly climb, and the CC curves with RPM surpass those without RPM. The comparison indicates that an accelerated convergence and higher reliability are brought about by the RPM. The MSE curves with RPM lie in the bottom of the graphs behind the ones without RPM, denoting the background noise reduced by the RPM. In conclusion, we have analyzed the degradation in ptychography caused by the motion of samples. Experiments indicate that the resolution and reliability of the complex amplitude retrieval of a sample are inversely proportional to its vibratory amplitude. A random phase modulated aperture can accelerate the convergence speed and can restore the retrieval with high quality to a limited extent. This work further suggests that at high amplitude vibration, ptychography using an RPM encounters an artifact problem. In future work, new wave-coding approach should be studied. High speed cameras and exposure coding approach should be introduced. Algorithms for motion detection, motion compensation, motion blur reduction and image restoration should also be employed in post processes. References Movable Aperture Lensless Transmission Microscopy: A Novel Phase Retrieval AlgorithmHigh-Resolution Scanning X-ray Diffraction MicroscopyKeyhole coherent diffractive imagingPtychographic X-ray computed tomography at the nanoscalePtychography – a label free, high-contrast imaging technique for live cells using quantitative phase informationMapping biological composition through quantitative phase and absorption X-ray ptychographyCoherent Diffractive Imaging Using Phase Front ModificationsWide-field, high-resolution Fourier ptychographic microscopyMeasurement of the complex transmittance of large optical elements with Ptychographical Iterative EngineHigh resolution integral holography using Fourier ptychographic approachInfluence of the partial coherence to the PIE imaging methodInfluence of Charge Coupled Device Saturation on PIE ImagingHigh-contrast imaging for weakly diffracting specimens in coherent diffraction imagingA general phase retrieval algorithm based on a ptychographical iterative engine for coherent diffractive imagingGeneralized Ptychography with Diverse ProbesContinuous motion scan ptychography: characterization for increased speed in coherent x-ray imagingNear-field ptychography: phase retrieval for inline holography using a structured illuminationSoft X-ray spectromicroscopy using ptychography with randomly phased illuminationPtychographical Imaging Algorithm with a Single Random Phase Encoding

[1]	Faulkner H and Rodenburg J 2004 Phys. Rev. Lett. 93 023903
[2]	Thibault P, Dierolf M, Menzel A, Bunk O et al 2008 Science 321 379
[3]	Abbey B, Nugent K A, Williams G J, Clark J N et al 2008 Nat. Photon. 4 394
[4]	Dierolf M, Menzel A, Thibault P, Schneider P et al 2010 Nature 467 436
[5]	Marrison J, Raty L, Marriott P and O'Toole P 2013 Sci. Rep. 3 2369
[6]	Jones M W, Elgass K, Junker M D, Luu M B et al 2014 Sci. Rep. 4 6796
[7]	Johnson I, Jefimovs K, Bunk O, David C et al 2008 Phys. Rev. Lett. 100 155503
[8]	Zheng G A, Horstmeyer R and Yang C 2013 Nat. Photon. 7 739
[9]	Wang H Y, Liu C, Veetil S P, Pan X C et al 2014 Opt. Express 22 2159
[10]	Li Z H, Zhang J Q, Wang X R and Liu D 2014 Opt. Express 22 31935
[11]	Pan X C, Lin Q, Liu C and Zhu J Q 2012 Sci. Sin.-Phys. Mech. & Astron. 42 889 (in Chinese)
[12]	Wang B S, Gao S M, Wang J C, Zhu J Q et al 2013 Acta Opt. Sin. 33 0611001 (in Chinese)
[13]	Pan X C, Veetil S P, Liu C, Lin Q et al 2013 Chin. Opt. Lett. 11 021103
[14]	Fu J, Li P, Tan R and Chen L Y 2012 {The 7th International Conference on System of Systems Engineering} (Genova, Italy 16–19 July 2012) p 111
[15]	Fu J and Li P 2013 Chin. Phys. B 22 014204
[16]	Shi Y S, Wang Y L and Zhang S G 2013 Chin. Phys. Lett. 30 054203
[17]	Deng J, Nashed Y S G, Chen S, Phillips N W et al 2015 Opt. Express 23 5438
[18]	Wang Z H, Wang Y L, Li T and Shi Y S 2014 Acta Phys. Sin. 63 164204 (in Chinese)
[19]	Stockmar M, Cloetens P, Zanette I, Enders B et al 2013 Sci. Rep. 3 01927
[20]	Maiden A, Morrison G, Kaulich B, Gianoncelli A et al 2013 Nat. Commun. 4 1669
[21]	Shi Y S, Wang Y L, Li T, Gao Q K et al 2013 Chin. Phys. Lett. 30 074203
[22]	Shi Y S, Wang Y L, Li T, Gao Q K et al 2013 Acta Phys. Sin. 62 064206 (in Chinese)