Intelligent image-activated cell sorting 2.0†

Akihiro Isozaki ^ab, Hideharu Mikami ^a, Hiroshi Tezuka ^c, Hiroki Matsumura ^a, Kangrui Huang ^a, Marino Akamine ^a, Kotaro Hiramatsu ^a, Takanori Iino ^d, Takuro Ito ^ae, Hiroshi Karakawa ^a, Yusuke Kasai ^f, Yan Li ^g, Yuta Nakagawa ^a, Shinsuke Ohnuki ^h, Tadataka Ota ^a, Yong Qian ^g, Shinya Sakuma ^f, Takeichiro Sekiya ^d, Yoshitaka Shirasaki ⁱ, Nobutake Suzuki ⁱ, Ehsen Tayyabi ^j, Tsubasa Wakamiya ^a, Muzhen Xu ^a, Mai Yamagishi ⁱ, Haochen Yan ^a, Qiang Yu ^d, Sheng Yan ^a, Dan Yuan ^a, Wei Zhang ^g, Yaqi Zhao ^a, Fumihito Arai ^f, Robert E. Campbell ^ag, Christophe Danelon ^k, Dino Di Carlo ^almn, Kei Hiraki ^a, Yu Hoshino ^o, Yoichiroh Hosokawa ^p, Mary Inaba ^c, Atsuhiro Nakagawa ^q, Yoshikazu Ohya ^hr, Minoru Oikawa ^s, Sotaro Uemura ⁱ, Yasuyuki Ozeki ^d, Takeaki Sugimura ^aet, Nao Nitta ^aet and Keisuke Goda *^aelu
aDepartment of Chemistry, The University of Tokyo, Tokyo 113-0033, Japan. E-mail: goda@chem.s.u-tokyo.ac.jp
^bKanagawa Institute of Industrial Science and Technology, 705-1 Shimoimaizumi, Ebina, Kanagawa 243-0435, Japan
^cDepartment of Creative Informatics, The University of Tokyo, Tokyo 113-0033, Japan
^dDepartment of Electrical Engineering and Information Systems, The University of Tokyo, Tokyo 113-8656, Japan
^eJapan Science and Technology Agency, Kawaguchi 332-0012, Japan
^fDepartment of Micro-Nano Mechanical Science and Engineering, Nagoya University, Nagoya 464-8601, Japan
^gDepartment of Chemistry, University of Alberta, Edmonton T6G 2G2, Canada
^hDepartment of Integrated Biosciences, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8562, Japan
ⁱDepartment of Biological Sciences, The University of Tokyo, Tokyo 113-0033, Japan
^jDepartment of Chemistry, University of Toronto, Ontario M5S 3H6, Canada
^kDepartment of Bionanoscience, Delft University of Technology, van der Maasweg 9, Delft 2629 HZ, The Netherlands
^lDepartment of Bioengineering, University of California, Los Angeles, California 90095, USA
^mDepartment of Mechanical Engineering, University of California, Los Angeles, California 90095, USA
ⁿCalifornia NanoSystems Institute, University of California, Los Angeles, California 90095, USA
^oDepartment of Chemical Engineering, Kyushu University, Fukuoka 819-0395, Japan
^pDivision of Materials Science, Nara Institute of Science and Technology, Ikoma 630-0192, Japan
^qDepartment of Neurosurgery, Tohoku University, Sendai 980-8577, Japan
^rAIST-UTokyo Advanced Operando-Measurement Technology Open Innovation Laboratory (OPERANDO-OIL), National Institute of Advanced Industrial Science and Technology (AIST), 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8589, Japan
^sNatural Sciences Cluster, Sciences Unit, Kochi University, Kochi 780-8520, Japan
^tCYBO, Tokyo 101-0022, Japan
^uInstitute of Technological Sciences, Wuhan University, Hubei 430072, China

First published on 18th May 2020

---

---

Introduction

The advent of intelligent image-activated cell sorting (iIACS)^1,2 has enabled high-throughput intelligent image-based sorting of single live cells from heterogeneous populations. iIACS is an on-chip microfluidic technology that builds on a seamless integration of a high-throughput fluorescence microscope, cell focuser, cell sorter, and deep neural network on a hybrid software–hardware data management architecture, thereby providing the combined merits of optical microscopy, fluorescence-activated cell sorting (FACS),^3–6 and deep learning.^7–9 Specifically, iIACS conducts high-throughput sorting of cells or cell clusters with unique morphochemical features that are difficult to discern when compressing these spatial data into intensity signals in FACS. Therefore, iIACS serves as an essential part of holistic single-cell analysis by providing direct connections between population-level analysis (flow cytometry), cell-level analysis (microscopy), and gene-level analysis (sequencing) of sorted cells.² In other words, iIACS has the potential to uncover and exploit the vastly heterogeneous relations between cellular genotype and phenotype (e.g., cellular size, cellular shape, nuclear shape, nucleus-to-cytoplasm ratio, cytoskeletal organization, RNA localization, lipid droplet distribution, chromosome number, cellular aggregation).^1,2

The range of applications and discoveries enabled by an iIACS machine is directly dictated by its system performance, which is characterized by its specifications such as throughput and sensitivity. The throughput is the number of detectable or sortable events (e.g., single cells, cell clusters, cell debris) that flow through the iIACS machine per unit of time, whereas the sensitivity is a detection threshold or an ability to resolve event populations, typically expressed in the number of molecules of equivalent soluble fluorophores (MESFs).^6,10 It is important to note that throughout the decades-long history of FACS,^3–6,11 multiple stages of system improvement and the installation of advanced components (e.g., lasers, optics, flow cells, photodetectors, electronics) have helped researchers evolve their research methodology, thereby leading to groundbreaking discoveries in immunology, hematology, and pathology. For example, the improved throughput of FACS has allowed detection of rare cells in blood such as circulating tumor cells, fetal cells in maternal blood, and antigen-specific T cells with high statistical accuracy.^12–14 Also, the installation of more powerful lasers, more sensitive photomultiplier tubes, and higher-NA objective lenses has enabled researchers to resolve various CD markers for immunophenotyping and even to detect extracellular vesicles. Likewise, iIACS' evolution in the next decade is expected to bring us a new class of applications and discoveries.

In this article, we report an iIACS machine that far surpasses the state-of-the-art iIACS machine in throughput and sensitivity. Specifically, it provides a high throughput of ∼2000 events per second (eps) and a high sensitivity of ∼50 MESFs, both of which are 20 times superior to those achieved in previous reports.^1,2 This is made possible by employing (i) an image-sensor-based optomechanical flow imaging method known as virtual-freezing fluorescence imaging (VIFFI)¹⁵ and (ii) a real-time intelligent image processor on an 8-PC server equipped with 8 multi-core central processing units (CPUs) and graphics processing units (GPUs) for intelligent decision-making, in order to significantly boost the imaging performance and computational power of the iIACS machine. We characterize the iIACS machine with fluorescent particles and various cell types and show that the performance of the iIACS machine is close to its achievable design specification. Equipped with the improved capabilities, this new generation of the iIACS technology holds promise for diverse applications in immunology, microbiology, stem cell biology, cancer biology, pathology, and synthetic biology.

Results

Overall schematic and functionality of the iIACS machine

The iIACS machine is schematically shown in Fig. 1 (see Movie S1† to understand the functionality of the iIACS machine). Employing the architecture of the iIACS machine we have previously reported^1,2 as a basis, but with significant enhancements, upgrades, and optimizations, the iIACS machine seamlessly integrates (1) a two-step three-dimensional (3D) on-chip hydrodynamic cell focuser for flowing cells at the center of the microchannel, (2) an image-sensor-based optomechanical microscope based on VIFFI¹⁵ for acquiring high-quality fluorescence images of cells flowing at a high flow speed of 1 m s⁻¹, (3) a speed meter for measuring the flow speed of cells at the VIFFI microscope and predicting their arrival time at the sort point, (4) a real-time intelligent image processor composed of a field-programmable gate array (FPGA), two CPUs, eight sets of CPUs and GPUs, and a network switch, all on a 10-Gbps all-IP network for high-speed digital image processing and decision-making, (5) an on-chip dual-membrane push–pull cell sorter for rapidly and physically isolating target cells from the cell stream.¹⁶ A 3D acoustic focuser can optionally be installed at the downstream of the optical interrogation point to maintain flowing cells at the center of the microchannel. The basic operation of the iIACS machine is as follows: suspended cells in a sample tube are injected into the microfluidic chip of the iIACS machine, focused by the hydrodynamic cell focuser into a single stream, imaged by the VIFFI microscope, analyzed by the real-time image processor, and sorted by the dual-membrane push–pull cell sorter triggered by decisions made by the image processor. The entire process is operated in a fully automated and real-time manner. The details of each critical element (optomechanical microscope, real-time image processor, and on-chip dual-membrane push–pull cell sorter) are described below.

Principles of the optomechanical microscope

A key factor for the high performance of the iIACS machine is the employment of VIFFI for sensitive, blur-free image acquisition of flowing cells. VIFFI is different from frequency-division multiplexed (FDM) microscopy^17–19 used in the original iIACS machine^1,2 in that cell images are obtained with an image sensor, offering higher imaging sensitivity and lower computational cost for image construction than FDM microscopy. In VIFFI, fluorescence images of flowing cells are acquired by a high-speed complementary metal oxide semiconductor (CMOS) image sensor through an optical imaging system with an image scanner. The image scanner cancels the flow motion of the fluorescence images of cells formed on the image sensor, enabling the extension of each cell's exposure time without motion blur. In addition, the exposure time is further extended by suppressing the motion blur using a spatiotemporal control of excitation light irradiation, allowing for fluorescence image acquisition of flowing cells with exceptionally high imaging sensitivity. Furthermore, since cell images are directly obtained from raw image frames of the image sensor by a simple cropping operation, VIFFI requires much simpler pre-processing of raw data compared with FDM microscopy, which contributes to the high-speed data processing and hence to the high-throughput operation. Specifically, we implemented VIFFI with an excitation laser (Coherent Genesis CX 488/2000 STM, wavelength = 488 nm), an acousto-optic deflector (ISOMET OAD948) for the spatiotemporal control of the excitation beam, an objective lens (Leica HC PL APO CS2, 20×, numerical aperture = 0.75) for fluorescence collection, a scientific CMOS (sCMOS) camera (PCO edge 5.5) for image acquisition, which has a high sensitivity or low readout noise of 1.7 electrons (root mean square) and a high data transfer rate (572 Mpixels s⁻¹), and a polygon mirror (Lincoln Laser RTA-B) for image scanning. Fluorescence images of two wavelength bands (504–543 nm and 591–800 nm) were simultaneously obtained by removing the excitation beam with a dichroic mirror (Semrock Di01-R488/561-25x36, edge wavelengths: 500 nm and 575.5 nm) followed by splitting their combined image with a dichroic mirror (Semrock FF580-FDi01-25x36, edge wavelength: 580 nm). More details about the principles and experimental setup of VIFFI are described in Mikami et al.'s paper.¹⁵

Architecture of the real-time intelligent image processor

As schematically shown in Fig. 1 and 2, the real-time intelligent image processor mainly consists of the following four parts: (i) the camera control (CC) node which controls the sCMOS camera and transfers raw image data through the all-IP network under the user datagram protocol (UDP), (ii) the image analysis (IA) cluster which analyzes acquired images, extract morphochemical features, and make a sort/unsort decision for each detected event, (iii) the timing management (TM) node which manages the timing of cell sorting, (iv) the network switch (NETGEAR XS716T-100AJS) which connects all the nodes via an Ethernet at 10 Gbps or 1 Gbps. These four components cooperate with each other and enable real-time data processing and decision-making. Details of each component are as follows. The CC node is equipped with a multi-core CPU (Intel Core i7-7700K), a counter board (CONTEC CNT-3204MT-LPE), a 10-Gbps Ethernet port, and a pair of camera link ports (10 taps, 85 MHz). This node receives digitized image data from the sCMOS camera through the camera link ports, encapsulates the image data together with count numbers provided by the counter board in UDP packets, and sends the packets to the IA cluster through the all-IP network. The IA cluster consists of two parts: master node and slave nodes. The master node is equipped with a multi-core CPU (Intel Core i7-8700K), solid state drive (SSD) storages, and a 10-Gbps Ethernet port. It makes sort/unsort decisions based on the results of the classical image analysis. Meanwhile, it also stores all processed cell images and a summarized datasheet of the analysis on the SSD storages. The slave nodes are composed of an 8-PC server, which is equipped with 8 multi-core CPUs (Intel Core i7-8700K) and 8 GPUs (NVIDIA GeForce GTX 1080 Ti), and is used to make sort/unsort decisions based on the results of a convolutional neural network (CNN). The 8 PC-server is deployed, monitored, and managed through an interface provided by the Landscape management tool (not shown in Fig. 1 and 2). The online image analysis at both the master and slave nodes is implemented as software (coded on C++ and Python), allowing us to employ various customized algorithms for specific applications. The TM node is on an FPGA board (Xilinx KC705) equipped with a 1-Gbps Ethernet port. It manages the timing of the entire process, which is essential for accurate cell sorting.

Signal processing in the real-time intelligent image processor

In the real-time intelligent image processor (Fig. 1 and 2), the CC node, TM node, and IA cluster conduct various types of signal processing including time management, image analysis, and sort-decision making. Details of the signal processing flow are described as follows. When a cell passes through the two laser beams for timing detection, the corresponding photodetectors send a pair of electrical pulses to the TM node, which encapsulates these signals together with sCMOS camera trigger signals into a packet and assigns a time stamp to the packet. The packet signal with the time stamp is sent to the IA cluster. Consecutive image frames acquired by the VIFFI microscope, each of which consists of 2560 × 88 pixels, are sent to the IA cluster via the CC node. The IA cluster crops two-color images of cells from the raw image frames using the time stamps sent by the TM node and analyzes the images using either an image processing algorithm coded with OpenCV on C++ or the CNN coded with Keras and TensorFlow on Python to create binary decision signals. The TM node receives the decision signals and triggers the dual-membrane push–pull cell sorter.

On-chip dual-membrane push-pull cell sorter

The dual-membrane push–pull cell sorter is based on high-speed control of local flow at the sort point and isolation of target cells from the central stream with piezoelectrically actuated dual membrane pumps. The configuration, principles, and fabrication of the cell sorter have been reported previously.^1,2,16 Each piezoelectric actuator was placed on the corresponding glass membrane fabricated as a part of the microfluidic chip. The piezoelectric actuators deform the glass membranes in a push–pull manner and produce local flow in the direction perpendicular to the cell flow at the sort point. The local flow isolates target cells from the central stream of cells. When the local flow is not produced, cells flow into the central microchannel at the three-way junction, whereas when the local flow is produced, cells flow into either the upper or lower channel. Sorted cells are collected in one outlet regardless of the direction of the local flow since the upper and lower channels are connected to one outlet at the downstream. The dual membrane push–pull sorter is advantageous for achieving high-throughput sorting since it is unnecessary to initialize the actuators. For sorting, a ramp voltage signal with an amplitude of 80 V and rising time of 200 μs is applied to each actuator.

Imaging performance

As shown in Fig. 3a, the imaging sensitivity of the iIACS machine was evaluated with standard 8-peak fluorescent particles. The upper panel shows results from a channel that detects green light whose wavelength ranges from 504 nm to 543 nm while the lower panel shows results from a channel that detects red light whose wavelength ranges from 591 nm to 800 nm. In the figure, each histogram shows the distribution of the signal intensity values of the particles with different fluorescence intensity values (left axis), the blue dots indicate the average signal intensity values and the MESF values of the populations (right axis), and the regression line and fit function are displayed in blue. The black x markers indicate the detection limit, which corresponds to the imaging sensitivity of the iIACS machine in the units of the MESF. As indicated in the figure, both channels show similar image sensitivity values, which is effective for multicolor imaging. Moreover, the detection limit of the iIACS machine is as low as 50 MESFs, which is 20 times superior to that achieved in previous reports.^1,2 This high sensitivity enables the iIACS machine to resolve and distinguish fluorescent particles whose fluorescence intensity differences are small and intracellular molecules with weak fluorescence signals such as expressed fluorescent proteins.

We also validated the imaging performance of the iIACS machine using various cell types. By employing the VIFFI microscope, the iIACS machine has higher imaging performance (i.e., images with higher sensitivity) than what was previously reported.^1,2Fig. 3b shows fluorescence images of Jurkat cells via fluorescence in situ hybridization (FISH) imaging, human embryonic kidney cells expressing fibril-forming protein, budding yeast (Saccharomyces cerevisiae) cells, and liposome-based synthetic cells acquired by the iIACS machine with the VIFFI microscope. Notably, the FISH images of the Jurkat cells provide information about the number and position of the multiple localized fluorescent spots showing specific gene sequences on the chromosome, which are useful for medical applications.^20,21 The images of the human embryonic kidney cells expressing fibril-forming protein clearly show a unique morphological feature, the needle-shaped green fluorescent fibrils, which could serve as an additional cell-labeling tool for biological applications.²² The images of the budding yeast cells provide information about their morphology, which is important for food science and metabolic engineering.^23,24 The images of liposome-based synthetic cells contain information about their intracellular protein localization, which allows us to identify and select liposomes that exhibit high-order internal organization that will aid the creation of artificial cells.²⁵ The high quality of these images indicates that the iIACS machine can identify the intracellular molecular distribution and morphological features of various cell types and use them as biomarkers to sort cells. Among these cell types, we used the budding yeast cells to demonstrate high-content sorting using the iIACS machine, which is impossible with FACS^3–6 and time-consuming with image-based cell-picking.^26–28 Furthermore, high-content cell sorting is much more efficient and effective than the previously used FDM-microscope-based iIACS machine because of its significantly improved throughput and sensitivity.

Sorting performance

We evaluated the sorting performance of the iIACS machine with fluorescent particles. Specifically, a mixture of 3-μm and 6-μm particles was tested as a sample to demonstrate high-throughput (>1000 eps) sorting of 3-μm particles based on the classical algorithm and CNN, during which both sorted and unsorted particles were simultaneously collected. To evaluate the sorting performance, we gathered the sorted and unsorted particles at the bottom of a well plate via centrifugation, subsequently scanned the entire bottom area under a commercial fluorescence microscope, and judged whether each particle was a 3-μm or 6-μm particle using fluorescence microscope images. Fig. 4a and b show the processing time of the data transfer and image analysis for each event based on the classical algorithm and CNN, respectively. The measured events are rank-ordered in the total processing time. As expected, the processing time based on the CNN was much longer than that based on the classical algorithm (4.28 times longer on average). However, owing to the 8-PC server on the real-time intelligent image processor, 98.8% of the events during the CNN-based sorting with a high throughput of 1133 eps were processed within 32 ms. The microscopic enumeration of the sorted and unsorted particles clearly indicates that the smaller particles were enriched with a high purity of 74.8% and a high throughput of 1023 eps based on the classical algorithm (Fig. 4c) and 81.9% and 1133 eps based on the CNN (Fig. 4d), respectively. These results are indicative of the iIACS machine's ability to perform high-content sorting based on both the classical algorithm and CNN with the high throughout and high purity.

Next, we characterized the sorting performance of the iIACS machine with fluorescent particles with various throughput values ranging from ∼500 eps to ∼2000 eps. Fig. 5a summarizes all the results we obtained for the particle sorting experiments with theoretically estimated purity and yield values as functions of the throughput. These theoretical estimates were derived based on Poisson statistics of events,^2,29,30 in which the time interval T between successive events is given by a cumulative probability distribution function, P_p(T < t) = 1 – e^−λt, where λ is the throughput (see ESI Note S1, ESI Fig. S1, and ESI† Table S1 for the theoretical estimation of the sort purity and yield). Note that the theoretical estimations vary, depending on the target-particle concentration, r, given by the ratio of the number of target particles to the sum of the numbers of target and non-target particles. Both the purity and yield of the theoretical estimates decrease with increasing throughput because the interval between successive events can be shorter than half of the sort window (∼425 μs) in a high-throughput sorting process, which leads to sorting error. The experimental data using both the classical algorithm and CNN are consistent with those of the theoretical estimates, indicating that the overall performance of the iIACS machine is close to its achievable design specification. In these experiments, we optimized the settings of the iIACS machine to obtain high purity at the expense of yield, given a constant throughout value. A further optimization can be performed to obtain a higher purity value by further sacrificing the yield. In contrast, high yield can also be obtained by tuning the settings at the expense of purity. Here, one of the most essential setting parameters is the sort window because the TM node's prediction of the sort timing is not perfect. Specifically, the shorter/longer the sort window, the higher/lower the purity and the lower/higher the yield.

Finally, we performed high-content sorting of budding yeast cells with the iIACS machine. Specifically, we prepared a cell population that consists of two strains of budding yeast, evaluated the aspect ratio of the mother part of budding cells based on obtained images, and then sorted out the budding cells with an aspect ratio of <0.7. The iIACS machine detected 44928 events and sorted out 679 cells (about 1.5% of the total population). We then evaluated the sorting performance by quantifying the aspect ratio of the sorted cells based on fluorescence microscope images obtained by the centrifugation-based method. As shown in Fig. 5b, the population of the sorted cells shifted to the lower aspect-ratio region compared with that of the pre-sorted cells. Moreover, the cumulative frequencies of the sorted cells were 67.7%, 87.1%, and 96.3% at the aspect ratio of 0.7, 0.8, and 0.9, respectively, which are larger than those of the pre-sorted cells, validating the sorting performance based on the cell aspect ratio. These results indicate that it is feasible to perform a large set of multiple sorting experiments with various target morphological features (e.g., circularity, perimeter, etc.). The sorted cells can be further analyzed for genotyping to provide the links between their morphological phenotype and genotype.

Discussion

Although the throughput and sensitivity of the iIACS machine have been significantly enhanced in this work, its capabilities can be further improved in several directions. First, multidimensional fluorescence imaging methods (e.g., multichannel fluorescence imaging³¹ and light-sheet illumination-based or confocal image acquisition-based volumetric image acquisition³²) can be implemented on this iIACS machine for increased biochemical specificity and spatial information without the need for making significant modifications to the iIACS machine. Second, super-resolution imaging techniques can be employed on the VIFFI platform. Third, the iIACS machine can be integrated with multichannel microfluidic devices and multiple sorting modules for multitarget sorting. Fourth, the design specifications of the iIACS machine can be tailored by developing and implementing a high purity/yield mode at the expense of yield/purity; by optimizing the sort window (e.g., shortening/lengthening the sort window) for high purity/yield; by implementing a speed meter with higher accuracy that can be used to predict the optimum sort timing from the measured speed and acceleration of flowing cells. Finally, implementation of tools for single-cell genomics, metabolomics, lipidomics, and proteomics is expected to interrogate the molecular content of the sorted cells and correlate it with their single-cell images. With these additional capabilities, iIACS is expected to serve as an indispensable tool for diverse applications in immunology, microbiology, stem cell biology, cancer biology, pathology, and synthetic biology.

Beyond the sorting application demonstrated in this work, there remain a number of unexploited applications including those that are currently challenging with other technologies such as FACS. For example, real-time sorting of rare cells such as circulating tumor cells, circulating fetal cells, and hematopoietic stem cells in blood based on their morphological features (e.g., nuclear shape, nucleus-to-cytoplasm ratio, cell clustering) is important for research in cytology, oncology, genetics, hematology, and stem cell biology. Also, given the increased sensitivity of the iIACS machine, the technology is now compatible with FISH, a widely used technique for detecting chromosomal abnormalities in patients, such that FISH sorting becomes a feasible application. Moreover, sorting of budding yeast cells with aberrant morphological traits is significant for studying functional genomic analysis and producing tasty sake and wine. Finally, it is meaningful to sort X and Y chromosome bearing sperm from each other as well as to sperm with a less condensed nucleus for sex sorting or sperm selection for assisted reproductive technologies.

Artificial intelligence (AI) technology, which has been dramatically developing in the recent decade, has enormous potential to revolutionize medical diagnostics⁷ and drug discovery.^33,34 Large data sets drive AI-based decision-making in biomedicine, which imaging flow cytometry techniques have helped to generate, but the heterogeneity in even clonal populations of cells necessitates sorting to uncover links between high-content images and underlying cellular processes.^35–37 The upgraded iIACS technology has the potential to greatly accelerate this trend because it provides high-quality single-cell images, compatible with AI-based image analysis, and allows linking image-based information with molecular underpinnings of a particular trait once sorted. Given the 20-fold improvement in sorting throughput, it now becomes reasonable to sort through millions of cells from disease samples or samples modified with knockout/knockdown libraries (e.g. CRISPR, RNAi) and iteratively classify and assay sub-populations. In this way, unique morphochemical features can be mapped to unique gene expression and behavioral differences. Ultimately, this can help identify new functionally distinct cell populations playing a role in health and disease and new drug targets associated with structural changes in cells.

Materials and methods

Preparation of Jurkat cells

Jurkat cells (TIB-152) (ATCC, USA) were used for FISH imaging. These cells were prepared according to the protocol described in a previous paper,³¹ followed by staining with 1/1000 diluted 7-AAD Viability Dye [0.005% (w/v) as the original solution, Beckman Coulter, A07704] for counterstaining.

Preparation of cells expressing fibril-forming protein

Cells expressing green fluorescent fibril-forming protein GFP-1POK were prepared in two steps: construction of GFP-1POK plasmid and transfection of the GFP-1POK plasmid into mammalian cells. First, to construct GFP-1POK plasmid, we follow the convention of Garcia-Seisdedos et al.³⁸ and refer to the fibril-forming protein by its corresponding Protein Data Bank (PDB) ID (https://www.rcsb.org). 1POK is isoaspartyl dipeptidase from Escherichia coli.³⁹ The gene encoding 1POK was synthesized (Integrated DNA Technologies, USA) with mammalian codon optimization. The gene of enhanced GFP (EGFP) was genetically fused to the C-terminus of the 1POK with a 10-amino acid flexible peptide linker (GGGGSGGGGS) by overlap extension polymerase chain reaction (PCR). The whole gene was then inserted between restriction sites NheI and XhoI of mammalian expression vector pcDNA3.1(+) to provide the pcDNA3.1-GFP-1POK plasmid. The pcDNA3.1-GFP-1POK plasmid was purified using GeneJET Plasmid Miniprep kit (ThermoFisher, USA). Second, the GFP-1POK plasmid was used to transfect human embryonic kidney cells (HEK293FT) (ThermoFisher, USA) as follows. The HEK293FT cells were maintained in Dulbecco's modified Eagle medium (DMEM) (FUJIFILM Wako Pure Chemical Corporation, Japan) supplemented with 10% fetal bovine serum (FBS) (ThermoFisher, USA), 2 mM GlutaMax (ThermoFisher, USA), and penicillin–streptomycin at 37 °C and 5% CO₂. 24 hours prior to transfection, ∼5 × 10⁵ cells were seeded into a 60-mm dish (Corning, USA) and allowed to reach ∼30% confluency. For transient transfection, for each 60-mm dish of cells, a well-mixed solution of 6 μg of plasmid DNA in serum-free DMEM (600 μL) was added to a well-mixed solution of 12 μL polyethylenimine (PEI) (1 mg mL⁻¹ pH 7.0 stock solution)⁴⁰ in serum-free DMEM (600 μL). The combined solution was well-mixed and incubated at room temperature for 15 minutes. This solution was then added to the cells, for which the medium had been changed to fresh complete DMEM, and left in the 60-mm dish. At 48 hours post-transfection, the cells were detached by treatment with trypsin (Nacalai Tesque, Japan) at 37 °C for 1–2 minutes, and then resuspended in 2 mL PBS buffer. See ESI† Table S2 for gene and primer sequences.

Preparation of budding yeast cells

Two strains of budding yeast Saccharomyces cerevisiae were used: K2 (ref. 41) for sorting and CLIB157 (ref. 42) for imaging and sorting. K2 cells were obtained from Dr. Takeshi Akao (National Research Institute of Brewing). CLIB157 cells were obtained from Prof. Joseph Schacherer (University of Strasbourg). Both strains were cultured, fixed, and stained according to the protocol described in a previous paper⁴³ with some modifications. Specifically, these cells were stocked at −80 °C in 16.7% glycerol stock solution. For sorting experiments, cells were activated on an agar plate of yeast-extract–peptone–dextrose (YPD) medium containing 1% (w/v) Bacto yeast extract (BD Biosciences, San Jose, CA), 2% (w/v) Bacto peptone (BD Biosciences), and 2% (w/v) glucose. Then, the cells on the YPD agar plate were incubated at 25 °C for three days for colony formation. The colonies were inoculated into 2 mL of the YPD liquid medium and pre-cultured at 25 °C for 5 hours. The pre-cultured cells were inoculated into 20 mL of the YPD medium and were incubated at 25 °C for 16 hours. After the incubation, these cells at the logarithmic phase of 6.4 × 10⁶ to 1.0 × 10⁷ cells mL⁻¹ were fixed at 25 °C for 30 minutes in YPD medium supplemented with formaldehyde (3.7% final concentration) and 100 mM potassium phosphate buffer (pH 6.5). The cell-surface mannoproteins of the fixed cells were stained by 1 mg mL⁻¹ fluorescein isothiocyanate (FITC)-conjugated concanavalin A (Sigma-Aldrich, St. Louis, MO) in P buffer (10 mM sodium phosphate and 150 mM NaCl, pH 7.2) for 10 minutes, followed by washing twice with P buffer. The washed cells were suspended in synthetic complete medium without leucine at a final concentration of 2 × 10⁶ cells mL⁻¹.

Preparation of liposome-based synthetic cells

Synthetic cells based on liposomes encapsulating the protein synthesis using recombinant elements (PURE) system were used. The liposomes were prepared according to the protocol described in a previous paper,⁴⁴ with the exception that no DNA was used and purified bacterial tubulin was supplemented in the swelling solution. Briefly, a 20-μL reaction mixture was assembled on ice by adding purified bacterial tubulin BtubAB labelled with Atto488 (3.75 μM final concentration) and unlabelled purified BtubAB (1.24 μM final concentration) into PUREfrex2.0 (GeneFrontier Corporation, Japan). About 20 mg of phospholipid-coated beads⁴⁴ was poured into the solution and incubated on ice for 2 hours. A small fraction of TexasRed-conjugated phospholipids (Thermo Fisher Scientific, USA) was included in the lipid film. The sample containing liposomes was subjected to four freeze–thaw cycles. 8 μL of the supernatant was transferred into 10 μL of a solution containing PUREfrex2.0 Solution I, deionized water, and Proteinase K (83 mg L⁻¹ final concentration; Promega) at a 28:11:1 volume ratio. The solution was incubated at 37 °C for about 2 hours to enable the formation of bacterial protein filaments in the vesicle lumen. The sample was then diluted ∼20000 times in PBS before imaging with the iIACS machine.

CNN for sorting particles

We used the six-layer CNN that was developed in Nitta et al.¹ as a basis, but used image datasets taken by the VIFFI microscope for training and validating the CNN. Specifically, we prepared 10000 images of 3-μm particles and 10000 images of 6-μm particles, 80% of which were used for training while the rest were used for validation. The CNN provides the probability of the image being classified as either of the classes (i.e., 3-μm-particle class or 6-μm-particle class) to make a sort decision based on the probabilities. We sorted events with the probability of >99% for the 3-μm-particle class.

Image processing algorithms for sorting budding yeast cells

For each event, we created a binary image mask from a fluorescence image, followed by the calculation of the aspect ratio of the mother cell from the budding yeast images. Specifically, we performed distance transformation on the binary image mask to find local peaks. Then, we obtained a segmentation contour by a watershed algorithm with the local peaks as markers. Finally, we fitted a part of the contour that surrounds the largest convex area (regarded as a mother cell) with an ellipse and calculated its aspect ratio as the ratio of the minor axis to the major axis.

Author contributions

K. G. conceived the concept of iIACS. H. Mikami, T. Iino, K. Hiramatsu, and Y. Ozeki designed and constructed the optics including the VIFFI microscope and speed meter and designed and implemented the hardware. N. N., A. I., S. S., F. A., Y. Kasai, D. D. C., and T. Sugimura designed, fabricated, and evaluated the microfluidic chip, fluidic system, hydrodynamic focusers, sorter, and optics-microfluidics integration unit. H. T., M. O., K. Hiraki, M. I., T. O., Q. Y., T. Sekiya, T. Sugimura, and N. N. designed and implemented the hardware and software of the speed meter, 10 Gbps all-IP network, CC node, IA cluster, TM node, and sort driver. H. Mikami, T. Sugimura, and K. Huang implemented the CNN. Y. S., N. S., and M. Yamagishi developed the centrifugation-based cell counting device. A. I., H. Mikami, T. Iino, S. S., H. T., M. O., K. Hiraki, M. I., Y. Kasai, Y. Hosokawa, T. Sugimura, Y. Ozeki, and N. N. integrated the iIACS machine. A. I., S. Y., H. Matsumura, K. Huang, D. Y., Y. Z., M. X., Y. N., H. Y., T. W., M. A., E. T., T. Iino, H. T., H. K., T. Sugimura, and N. N. conducted the sorting experiments and data analysis. A. I., H. Mikami, S. Y., H. Matsumura, Y. Ohya, and S. O. planned experiments. T. Sugimura, K. Huang, Y. Z., and H. Mikami designed and implemented the data analysis software. Y. Ohya and S. O. prepared the budding yeast cells and analyzed the results. D. Y. and C. D. prepared the liposome-based synthetic cells and analyzed the results. M. X., Y. L., Y. Q., W. Z., and R. E. C. prepared the human embryonic kidney cells and analyzed the results. M. Yamagishi prepared the Jurkat cells and analyzed the results. A. I., H. Mikami, S. Y., Y. Hoshino, T. Ito, A. N., Y. Hosokawa, S. U., T. Sugimura, Y. Ozeki, N. N., and K. G. supervised the work. K. G. led the iIACS development program. A. I., H. Mikami, S. Y., H. Matsumura, K. Huang, D. Y., Y. Ohya, S. O., C. D., R. E. C., Y. L., M. Yamagishi, D. D. C., and K. G. wrote the paper. A. I., H. Mikami, H. Matsumura, K. Huang, D. Y., and K. G. provided assistance with the figures. All authors participated in editing it.

Conflicts of interest

H. Mikami, Y. Ozeki, and K. G. are inventors on a patent covering the VIFFI microscope. S. S., and F. A. are inventors on a patent application covering the dual-membrane push-pull cell sorter. N. N., T. Sugimura, and K. G. are inventors on a patent covering the data analysis and display method. N. N. is the president of K. K. CYBO. N. N., T. Sugimura, and K. G. are shareholders of K. K. CYBO.

Acknowledgements

This work was supported by the ImPACT program (CSTI, Cabinet Office, Government of Japan), JSPS Core-to-Core Program, JSPS KAKENHI grant number 19H05633, White Rock Foundation, KISTEC, Nakatani Foundation, and Precise Measurement Technology Promotion Foundation. N. N. is an ISAC Marylou Ingram Scholar. We thank members of the Marileen Dogterom lab for discussions about the Btub proteins, Anne Doerr for purifying and labeling the Btub proteins, Johannes Kattan for performing preliminary experiments, and members of the Danelon lab for preparing the lipid-coated beads.

References

1. N. Nitta, T. Sugimura, A. Isozaki, H. Mikami, K. Hiraki, S. Sakuma, T. Iino, F. Arai, T. Endo, Y. Fujiwaki, H. Fukuzawa, M. Hase, T. Hayakawa, K. Hiramatsu, Y. Hoshino, M. Inaba, T. Ito, H. Karakawa, Y. Kasai, K. Koizumi, S. Lee, C. Lei, M. Li, T. Maeno, S. Matsusaka, D. Murakami, A. Nakagawa, Y. Oguchi, M. Oikawa, T. Ota, K. Shiba, H. Shintaku, Y. Shirasaki, K. Suga, Y. Suzuki, N. Suzuki, Y. Tanaka, H. Tezuka, C. Toyokawa, Y. Yalikun, M. Yamada, M. Yamagishi, T. Yamano, A. Yasumoto, Y. Yatomi, M. Yazawa, D. Di Carlo, Y. Hosokawa, S. Uemura, Y. Ozeki and K. Goda, Cell, 2018, 175, 266–276 CrossRef CAS PubMed .
2. A. Isozaki, H. Mikami, K. Hiramatsu, S. Sakuma, Y. Kasai, T. Iino, T. Yamano, A. Yasumoto, Y. Oguchi, N. Suzuki, Y. Shirasaki, T. Endo, T. Ito, K. Hiraki, M. Yamada, S. Matsusaka, T. Hayakawa, H. Fukuzawa, Y. Yatomi, F. Arai, D. Di Carlo, A. Nakagawa, Y. Hoshino, Y. Hosokawa, S. Uemura, T. Sugimura, Y. Ozeki, N. Nitta and K. Goda, Nat. Protoc., 2019, 14, 2370–2415 CrossRef CAS PubMed .
3. L. A. Herzenberg, C. Gottlinger, W. Muller, A. Radbruch and D. Recktenwald, Flow cytometry and cell sorting, Springer, Berlin Heidelberg, 2nd edn, 1992 Search PubMed .
4. L. A. Herzenberg, D. Parks, B. Sahaf, O. Perez, M. Roederer and L. A. Herzenberg, Clin. Chem., 2002, 48, 1819–1827 CrossRef CAS .
5. J. W. Tung, K. Heydari, R. Tirouvanziam, B. Sahaf, D. R. Parks, L. A. Herzenberg and L. A. Herzenberg, Clin. Lab. Med., 2007, 27, 453–468 CrossRef PubMed .
6. H. M. Shapiro, Practical flow cytometry, John Wiley & Sons, Hoboken, NJ, 4th edn, 2005 Search PubMed .
7. E. J. Topol, Nat. Med., 2019, 25, 44–56 CrossRef CAS .
8. E. Moen, D. Bannon, T. Kudo, W. Graf, M. Covert and D. Van Valen, Nat. Methods, 2019, 16, 1233–1246 CrossRef CAS .
9. M. Doan and A. E. Carpenter, Nat. Mater., 2019, 18, 414–418 CrossRef CAS .
10. A. Schwartz and E. Fernández-Repollet, Ann. N. Y. Acad. Sci., 1993, 677, 28–39 CrossRef CAS .
11. A. Cossarizza, H. Chang, A. Radbruch, A. Acs, D. Adam, S. Adam-Klages, W. W. Agace, M. Akdis, M. Allez, L. Nogueira Almeida, G. Alvisi, G. Anderson and E. Al, Eur. J. Immunol., 2019, 49, 1457–1973 CrossRef CAS .
12. A. Sekizawa, A. Farina, D. Kai, Z. Ji, Y. Wang, V. M. Falco, S. Elmes and D. W. Bianchi, Fetal. Diagn. Ther., 1999, 14, 229–233 CrossRef CAS .
13. M. J. M. Magbanua, E. V. Sosa, R. Roy, L. E. Eisenbud, J. H. Scott, A. Olshen, D. Pinkel, H. S. Rugo and J. W. Park, Cancer Res., 2013, 73, 30–40 CrossRef CAS PubMed .
14. V. Rubio, T. B. Stuge, N. Singh, M. R. Betts, J. S. Weber, M. Roederer and P. P. Lee, Nat. Med., 2003, 9, 1377–1382 CrossRef CAS PubMed .
15. H. Mikami, M. Kawaguchi, C. J. Huang, H. Matsumura, T. Sugimura, K. Huang, C. Lei, S. Ueno, T. Miura, T. Ito, K. Nagasawa, T. Maeno, H. Watarai, M. Yamagishi, S. Uemura, S. Ohnuki, Y. Ohya, H. Kurokawa, S. Matsusaka, C. W. Sun, Y. Ozeki and K. Goda, Nat. Commun., 2020, 11, 1162 CrossRef CAS PubMed .
16. S. Sakuma, Y. Kasai, T. Hayakawa and F. Arai, Lab Chip, 2017, 17, 2760–2767 RSC .
17. H. Mikami, J. Harmon, H. Kobayashi, S. Hamad, Y. Wang, O. Iwata, K. Suzuki, T. Ito, Y. Aisaka, N. Kutsuna, K. Nagasawa, H. Watarai, Y. Ozeki and K. Goda, Optica, 2018, 5, 117–126 CrossRef CAS .
18. H. Mikami, C. Lei, N. Nitta, T. Sugimura, T. Ito, Y. Ozeki and K. Goda, Chem, 2018, 4, 2278–2300 CAS .
19. H. Kanno, H. Mikami, Y. Kaya, Y. Ozeki and K. Goda, Opt. Lett., 2019, 44, 467–470 CrossRef .
20. C. Cui, W. Shu and P. Li, Front. Cell Dev. Biol., 2016, 4, 89 Search PubMed .
21. G. M. Baerlocher, I. Vulto, G. de Jong and P. M. Lansdorp, Nat. Protoc., 2006, 1, 2365–2376 CrossRef CAS .
22. E. A. Rodriguez, R. E. Campbell, J. Y. Lin, M. Z. Lin, A. Miyawaki, A. E. Palmer, X. Shu, J. Zhang and R. Y. Tsien, Trends Biochem. Sci., 2017, 42, 111–129 CrossRef CAS PubMed .
23. A. Krizhevesky, I. Sutskever and G. E. Hinton, in Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS 2012), ed. F. Pereira, C. J. C. Burges, L. Bottou and K. Q. Weinberger, Curran Associates, Inc., 2012, pp. 1097–1105 Search PubMed .
24. T. Goshima, R. Nakamura, K. Kume, H. Okada, E. Ichikawa, H. Tamura, H. Hasuda, M. Inahashi, N. Okazaki, T. Akao, H. Shimoi, M. Mizunuma, Y. Ohya and D. Hirata, Biosci., Biotechnol., Biochem., 2016, 80, 1657–1662 CrossRef CAS PubMed .
25. E. Godino, J. N. López, D. Foschepoth, C. Cleij, A. Doerr, C. F. Castellà and C. Danelon, Nat. Commun., 2019, 10, 4969 CrossRef .
26. A. O. Ogunniyi, C. M. Story, E. Papa, E. Guillen and J. C. Love, Nat. Protoc., 2009, 4, 767–782 CrossRef CAS .
27. K. D. Piatkevich, E. E. Jung, C. Straub, C. Linghu, D. Park, H.-J. Suk, D. R. Hochbaum, D. Goodwin, E. Pnevmatikakis, N. Pak, T. Kawashima, C.-T. Yang, J. L. Rhoades, O. Shemesh, S. Asano, Y.-G. Yoon, L. Freifeld, J. L. Saulnier, C. Riegler, F. Engert, T. Hughes, M. Drobizhev, B. Szabo, M. B. Ahrens, S. W. Flavell, B. L. Sabatini and E. S. Boyden, Nat. Chem. Biol., 2018, 14, 352–360 CrossRef CAS .
28. C. Brasko, K. Smith, C. Molnar, N. Farago, L. Hegedus, A. Balind, T. Balassa, A. Szkalisity, F. Sukosd, K. Kocsis, B. Balint, L. Paavolainen, M. Z. Enyedi, I. Nagy, L. G. Puskas, L. Haracska, G. Tamas and P. Horvath, Nat. Commun., 2018, 9, 4969 CrossRef .
29. T. Iino, K. Okano, S. W. Lee, T. Yamakawa, H. Hagihara, Z.-Y. Hong, T. Maeno, Y. Kasai, S. Sakuma, T. Hayakawa, F. Arai, Y. Ozeki, K. Goda and Y. Hosokawa, Lab Chip, 2019, 19, 2669–2677 RSC .
30. T. Lindmo, D. C. Peters and R. G. Sweet, Flow Cytometry and Sorting, Wiley-Liss, Inc., 2nd edn, 1990 Search PubMed .
31. D. A. Basiji, W. E. Ortyn, L. Liang, V. Venkatachalam and P. Morrissey, Clin. Lab. Med., 2007, 27, 653–670 CrossRef PubMed .
32. Y. Han, R. Tang, Y. Gu, A. C. Zhang, W. Cai, V. Castor, S. H. Cho, W. Alaynick and Y.-H. Lo, Optica, 2019, 6, 1297–1304 CrossRef .
33. M. A. Bray, S. Singh, H. Han, C. T. Davis, B. Borgeson, C. Hartland, M. Kost-Alimova, S. M. Gustafsdottir, C. C. Gibson and A. E. Carpenter, Nat. Protoc., 2016, 11, 1757–1774 CrossRef CAS .
34. P. Schneider, W. P. Walters, A. T. Plowright, N. Sieroka, J. Listgarten, R. A. Goodnow, J. Fisher, J. M. Jansen, J. S. Duca, T. S. Rush, M. Zentgraf, J. E. Hill, E. Krutoholow, M. Kohler, J. Blaney, K. Funatsu, C. Luebkemann and G. Schneider, Nat. Rev. Drug Discovery, 2020, 19, 353–364 CrossRef PubMed .
35. S. E. Leggett, J. Y. Sim, J. E. Rubins, Z. J. Neronha, E. K. Williams and I. Y. Wong, Integr. Biol., 2016, 8, 1133–1144 CrossRef CAS .
36. D. Di Carlo, H. T. K. Tse and D. R. Gossett, in Single-Cell Analysis: Methods and Protocols, ed. S. Lindström and H. Andersson-Svahn, Humana Press, Totowa, NJ, 2012, pp. 1–10 Search PubMed .
37. A. Gough, A. M. Stern, J. Maier, T. Lezon, T. Y. Shun, C. Chennubhotla, M. E. Schurdak, S. A. Haney and D. Lansing Taylor, SLAS Discovery, 2017, 22, 213–237 CAS .
38. H. Garcia-Seisdedos, C. Empereur-Mot, N. Elad and E. D. Levy, Nature, 2017, 548, 244–247 CrossRef CAS .
39. D. Jozic, J. T. Kaiser, R. Huber, W. Bode and K. Maskos, J. Mol. Biol., 2003, 332, 243–256 CrossRef CAS .
40. S. Zhang, L. He, Y. Zhou and Y. Wang, in Methods in Molecular Biology, ed. A. Penna and B. Constantin, Humana Press, New York, NY, 1st edn, 2018, vol. 1843, pp. 17–39 Search PubMed .
41. S. Ohnuki, H. Okada, A. Friedrich, Y. Kanno, T. Goshima, H. Hasuda, M. Inahashi, N. Okazaki, H. Tamura, R. Nakamura, D. Hirata, H. Fukuda, H. Shimoi, K. Kitamoto, D. Watanabe, J. Schacherer, T. Akao and Y. Ohya, G3: Genes, Genomes, Genet., 2017, 7, 2807–2820 CrossRef CAS .
42. J. Schacherer, J. A. Shapiro, D. M. Ruderfer and L. Kruglyak, Nature, 2009, 458, 342–345 CrossRef CAS .
43. Y. Ohya, J. Sese, M. Yukawa, F. Sano, Y. Nakatani, T. L. Saito, A. Saka, T. Fukuda, S. Ishihara, S. Oka, G. Suzuki, M. Watanabe, A. Hirata, M. Ohtani, H. Sawai, N. Fraysse, J. P. Latgé, J. M. François, M. Aebi, S. Tanaka, S. Muramatsu, H. Araki, K. Sonoike, S. Nogami and S. Morishita, Proc. Natl. Acad. Sci. U. S. A., 2005, 102, 19015–19020 CrossRef CAS PubMed .
44. D. Blanken, P. Van Nies and C. Danelon, Phys. Biol., 2019, 16, 045002 CrossRef CAS PubMed .

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: 10.1039/d0lc00080a

This journal is © The Royal Society of Chemistry 2020