⇦Chinese Physics Letters, 2016, Vol. 33, No. 4, Article code 044206⇨ Optical Transfer Function Reconstruction in Incoherent Fourier Ptychography * Zong-Liang Xie(谢宗良)1,2,3, Bo Qi(亓波)1,2**, Hao-Tong Ma(马浩统)1,2,4, Ge Ren(任戈)1,2, Yu-Feng Tan(谭玉凤)1,2,3, Bi He(贺璧)1,2,3, Heng-Liang Zeng(曾恒亮)1,2,3, Chuan Jiang(江川)1,2,3 Affiliations 1Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209 2Key Laboratory of Optical Engineering, Chinese Academy of Sciences, Chengdu 610209 3University of Chinese Academy of Sciences, Beijing 100049 4College of Opto-electric Science and Engineering, National University of Defense Technology, Changsha 410073 Received 18 December 2015 ^*Supported by the National Natural Science Foundation of China under Grant No 61205144, the Research Project of National University of Defense Technology under Grant No JC13-07-01, and the Key Laboratory of High Power Laser and Physics of Chinese Academy of Sciences.
^**Corresponding author. Email: qibo@ioe.ac.cn Citation Text: Xie Z L, Qi B, Ma H T, Ren G and Tan Y F et al 2016 Chin. Phys. Lett. 33 044206 Abstract An optical transfer function (OTF) reconstruction model is first embedded into incoherent Fourier ptychography (IFP). The leading result is a proposed algorithm that can recover both the super-resolution image and the OTF of an imaging system with unknown aberrations simultaneously. This model overcomes the difficult problem of OTF estimation that the previous IFP faces. The effectiveness of this algorithm is demonstrated by numerical simulations, and the superior reconstruction is presented. We believe that the reported algorithm can extend the original IFP for more complex conditions and may provide a solution by using structured light for characterization of optical systems' aberrations. DOI:10.1088/0256-307X/33/4/044206 PACS:42.30.-d, 42.30.Kq, 42.30.Lr, 42.30.Wb © 2016 Chinese Physics Society Article Text Fourier ptychography (FP) is a newly developed ptychographic imaging method for microscopy with illumination diversity.[1] It expands the Fourier passband and recovers the complex field by utilizing a sequence of low-resolution images created by diversely angular illumination of a light-emitting-diode (LED) array to stitch in the Fourier domain. However, despite the usage of LEDs with partially spatial coherence, the recovery algorithm based on coherent phase retrieval still limits FP in incoherent applications. To develop FP for incoherent imaging, a state-multiplexed recovery scheme[2] adopting the mixed-state concept in ptychography[3] has been reported, which succeeds in handling incoherent illumination consisting of multiple coherent light sources in FP setting.[2] However, the prior knowledge of limited decomposition modes and the LED-array-based configuration still make it impossible for FP to be implemented in applications of incoherent macroscopic imaging. Recently, incoherent Fourier ptychography (IFP) has been proposed to recover super-resolution images with incoherently structured illumination.[4,5] It also works by iteratively stitching between the Fourier and spatial domain. In the spatial domain, the recorded images are utilized as the constraint, while the optical transform function (OTF) of the imaging system performs the support constraint in the Fourier domain. Unlike conventional FP, IFP projects a scanning illumination pattern onto the object instead of an LED array positioned behind the sample, which is compatible with many existing illumination-based imaging schemes. Moreover, the reconstruction algorithm of IFP based on the incoherent imaging method is absolutely improved for incoherent conditions in comparison with generalized FP by using coherent phase retrieval. With the above advantages, IFP may find wide applications in incoherent macroscopic practice, such as remote sensing, active-illumination night vision systems and imaging radar. Despite the successful demonstration of IFP in the laboratory environment, it is still limited to the usage of the known OTF in iteration. In fact, the OTF is usually unknown or is difficult to estimate. On the one hand, an estimated OTF, whose modulus known as modulation transfer function (MTF) is shown as an example in Fig. 1(a), can be obtained with the prior knowledge of pupil based on incoherent imaging theory.[6,7] However, if the imaging system is influenced by unknown aberrations shown in Fig. 1(b), its real OTF will become greatly distinguished from the estimated one, whose modulus is shown in Fig. 1(c). Therefore, the recovered image would be degraded by the estimated errors of the OTF. On the other hand, even if we estimate the OTF of an imaging system using a point source test, the space-variant aberrations will also cause the similar reconstruction errors. In this Letter, we embed an OTF reconstruction model into the algorithm of IFP. Such a model allows us to circumvent the difficult OTF estimation noted above. In this case, only an initial guess of the OTF is needed to start the reconstruction procedure. Then a resulting super-resolution image and a reconstructed OTF can be acquired by updating the OTF with iteration proceeding. The technique may not only fit IFP for further applications, but also provide a possible solution for characterization of aberration behavior.

Fig. 1. (a) Estimated MTF, (b) loaded aberrations, and (c) real MTF.

Fig. 2. Schematic diagram of the incoherent Fourier ptychographical imaging with OTF recovery encoding.

The schematic diagram of the incoherent Fourier ptychographical imaging with OTF recovery encoding is shown in Fig. 2. An incoherent illumination unit, consisting of a diffuser inserted in front of a diffused LED and an imaging lens, projects the image of the diffuser onto the object. In this way, the frequency between the object and the illumination pattern is mixed so that some high-frequency component can be shifted to the passband of the collection optics. Then the active illumination unit is scanned to illuminate the object, and the corresponding intensity images containing information beyond the cutoff frequency are captured. Based on these diversely pattern-modulated images, the super-resolution image and the OTF can be extracted by employing our proposed algorithm. Figure 3 shows the flowchart of our proposed IFP algorithm with OTF recovery, which is described in detail as follows: (1) start with initial guesses of the object image $I_{{\rm obj},0}$, the unknown projection pattern $P_0$, and the optical transfer function ${\rm OTF}_0$. (2) For the $n$th iteration, generate a target image as $$\begin{align} I_{{\rm t}n} =I_{{\rm obj},n-1} \cdot P_{n-1} (x-x_n),~~ \tag {1} \end{align} $$ where $x_n$ ($n=1,2,3 {\ldots}$) denotes the shifting position of the projection pattern. (3) Update the target image in the Fourier domain by using the captured image $I_n$ modulated by the speckle pattern in the $n$th position as follows: $$\begin{alignat}{1} {\cal F}(I_{{\rm t}n}^{\rm update})=\,&{\cal F}(I_{{\rm t}n})+{\rm OTF}_{n-1}\\ &\cdot [{\cal F}(I_n)-{\rm OTF}_{n-1} \cdot {\cal F}(I_{{\rm t}n})],~~ \tag {2} \end{alignat} $$ where ${\cal F}(\cdot)$ presents the Fourier transform. (4) Acquire the renewed object image in the spatial domain with the updated target image $I_{{\rm t}n}^{\rm update}$ and projection pattern $P_{n-1}$ according to $$\begin{align} I_{{\rm obj},n}^{\rm update}=\,&I_{{\rm obj},n-1} +\frac{P_{n-1}}{(\max (P_{n-1}))^2}\\ &\cdot ({I_{{\rm t}n}^{\rm update} -I_{{\rm obj},n-1} \cdot P_{n-1}}).~~ \tag {3} \end{align} $$ (5) Also, the unknown illumination pattern can be updated with $$\begin{alignat}{1} P_n =P_{n-1} +\frac{I_{{\rm obj},n}^{\rm update}}{({\max (I_{{\rm obj},n}^{\rm update})})^2} \cdot ({I_{{\rm t}n}^{\rm update} -I_{{\rm t}n}}).~~ \tag {4} \end{alignat} $$ (6) Perform the important updating process of OTF by using the mathematic model as follows: $$\begin{align} {\rm OTF}_n =\,&{\rm OTF}_{n-1} +\frac{{\rm conj}({\cal F}(I_{{\rm t}n}))}{({\max ({{\cal F}( {I_{{\rm t}n}})})})^2}\\ &\cdot ({\cal F}({I_n})-{\rm OTF}_{n-1} \cdot {\cal F}( {I_{{\rm t}n}})),~~ \tag {5} \end{align} $$ where ${\rm conj}(\cdot)$ denotes the conjugate operator. This model is motivated by the similar form of the extraction function in improved ptychography[8] and spectrum updating function in FP.[1] In the Fourier domain, the correction of the OTF is extracted from the difference between the spectra of the measured intensity and producing intensity divided by the current spectrum of the target image. Then the updating process is finished by adding this correction to the current OTF guess. (7) Complete an entire iteration by repeating the above steps 2–6 for all intensity measurements and calculate the correlation coefficients (Co) to evaluate the reconstruction,[9-11] which is defined as $$\begin{alignat}{1} Co(I_{{\rm obj},n},I)={\rm cov}(I_{{\rm obj},n},I)(\sigma _{I_{\rm obj},n} \cdot \sigma _I)^{-1}),~~ \tag {6} \end{alignat} $$ where $I_{{\rm obj},n}$ and $I$ denote the recovered and the original image, respectively, ${\rm cov}(I_{{\rm obj},n},I)$ is the cross-covariance between them, and $\sigma$ denotes the standard deviation. The larger the metric in the range of [0, 1], the better the quality of recovery. (8) Repeat steps 2–7 until the Co value meets the presupposed threshold, then the reconstructed super-resolution image and the OTF can be obtained. To validate our proposed algorithm, we conduct a series of simulations. Simulations are performed with monochromatic green light of 532 nm wavelength and an imaging system with aberrations shown in Fig. 1(b) loaded. Specifically, the pupil size of the simulated system is 10 mm and the focal length is 300 mm. The corresponding MTF of the system is shown in Fig. 1(c). To clearly show the resolution enhancement, a star resolution image containing $256\times256$ pixels with pixel size 1.95 μm shown in Fig. 4(a) is used as the object. The sample assumed to be far enough is imaged on the focal plane of the system. A random gray image with $268\times268$ pixels shown in Fig. 4(b) presents the intensity distribution of the projection pattern, which is scanned to 13 different positions for illumination modulation along each of two dimensions with a shift step of 1 pixel. Based on the incoherent imaging theory,[6,7] a total of 169 images modulated by the projected pattern in different positions are simulated to be captured. Then to have a clear comparison, we run both the original IFP algorithm and our proposed IFP algorithm with OTF reconstruction to deal with the raw data set. Here in both the algorithms, the initial guess of the object image is one of the raw images, and the initial illumination pattern is set to one. In the original algorithm, we use the estimated OTF in the iteration, whose modulus is shown in Fig. 1(a), while in the reported algorithm it is only used as the initial guess of the OTF to start the reconstruction.

Fig. 3. Flowchart of our proposed IFP algorithm with OTF recovery.

Fig. 4. (a) Object intensity, and (b) the random projected pattern.

Figure 5 shows the superior reconstructions of IFP with OTF recovery. As a reference, the near-diffraction-limited image formed by the imaging system without structure illumination is presented in Fig. 5(a1) with its Fourier spectrum shown in Fig. 5(a2). Compared with the case without IFP, the recovered image by using IFP shown in Fig. 5(b1) obviously offers resolution enhancement, and Fig. 5(b2) shows the corresponding spectrum, which is beyond the cutoff frequency shown in Fig. 5(a2). This super-resolution image reconstruction has been demonstrated by original IFP.[4,5] However, it is also evident that the image recovered by IFP is terribly degraded by the estimated errors of the OTF and many details are lost. Luckily, the problem can be solved well by using our proposed algorithm. Figure 5(c1) shows the reconstructed image by using IFP with OTF recovery. With our proposed algorithm, it is easy to confirm that the sharpness of the recovered image is greatly improved and more details can be acquired, compared with the results previously presented. Its corresponding Co value is 0.96 while that for original IFP is 0.53. The corresponding spectrum is shown in Fig. 5(c2), which also has wider passbands than the one in Fig. 5(a2) and provides more clear frequency details than the one in Fig. 5(b2). The results demonstrate superior reconstruction of our reported algorithm with respect to the previous technique, when the OTF of the employed optical system is uncertain.

Fig. 5. Demonstration of the superior reconstructions of IFP with OTF recovery. (a1) The formed image without IFP and (a2) its corresponding spectrum. (b1) The image recovered by IFP without OTF recovery and (b2) its corresponding spectrum. (c1) The image recovered by IFP with OTF recovery and (c2) its corresponding spectrum.

Fig. 6. Reconstructed modulation transfer function.

It is also necessary to directly evaluate the performance of the OTF reconstruction. The modulus of the OTF known as the MTF describes attenuation of the sinusoidal image irradiance components as a function of the spatial frequency, while the phase part decides the translation of the corresponding sinusoidal spatial component in the image plane.[6,7] In the reconstruction, we concentrate on the image intensity, caring slightly about the image translation, thus the recovered MTF shown in Fig. 6 is used here to evaluate the quality of the OTF reconstruction. It can be found that the recovered MTF is almost the same as the real one shown in Fig. 1(c). The corresponding value of Co is very close to 1. Motivated by the good quality of the MTF reconstruction, we infer that the aberrations may be extracted with the Zernike decomposition by a nonlinear optimization according to the recovered MTF. Therefore, the proposed algorithm may provide a possible solution for characterization of systems' distortions. In summary, we have proposed and demonstrated an optical transfer function reconstruction model embedded into IFP. Our proposed algorithm produces superior reconstruction compared with the previously reported algorithm when the OTF is unknown. Therefore, by using our reported OTF recovery model, IFP can be suitable for more complicated situations and may really contribute to incoherent macroscopic applications, such as remote sensing, imaging radar, and synthetic aperture imaging. We also note that, with good capacity of reconstructing the OTF, the proposed technique may realize wavefront sensing by using the structure light, which would be a large leap in active-illumination imaging schemes without holographic detection. References Wide-field, high-resolution Fourier ptychographic microscopySpectral multiplexing and coherent-state decomposition in Fourier ptychographic imagingReconstructing state mixtures from diffraction measurementsHigh-resolution fluorescence imaging via pattern-illuminated Fourier ptychographyIncoherent Fourier ptychographic photography using structured lightAn improved ptychographical phase retrieval algorithm for diffractive imagingPtychographical Imaging Algorithm with a Single Random Phase EncodingGeneralized Ptychography with Diverse ProbesAperture-Scanning Fourier Ptychographic Encoding with Phase Modulation

[1]	Zheng G A, Horstmeyer R and Yang C H 2013 Nat. Photon. 7 739
[2]	Dong S Y, Shiradkar R, Nanda P and Zheng G A 2014 Biomed. Opt. Express 5 1757
[3]	Thibault P and Menzel A 2013 Nature 494 68
[4]	Dong S Y, Nanda P, Shiradkar R, Guo K K and Zheng G A 2014 Opt. Express 22 20856
[5]	Dong S Y, Nanda P, Guo K K, Liao J and Zheng G A 2015 Photon. Res. 3 19
[6]	Goodman J W 2004 Introduction to Fourier Optics 3rd edn (Austin TX: Roberts and Company)
[7]	Schmidt J D 2010 Numerical Simulation of Optical Wave Propagation (Bellingham: SPIE)
[8]	Maiden A M and Rodenburg J M 2009 Ultramicroscopy 109 1256
[9]	Shi Y S, Wang Y L, Li T, Gao Q K, Wan H, Zhang S G and Wu Z B 2013 Chin. Phys. Lett. 30 074203
[10]	Shi Y S, Wang Y L and Zhang S G 2013 Chin. Phys. Lett. 30 054203
[11]	Xie Z L, Ma H T, Qi B, Ren G, Tan Y F, He B, Zeng H L and Jiang C 2015 Chin. Phys. Lett. 32 124203