UC Berkeley BIDS ImageXD 2019

———————————————————————–

(1) Matthew Akamatsu - UCB

Postdoc - matt.akamatsu@berkeley.edu

3D image analysis and machine learning to complement 3D stochastic simulations and cryo-electron microscopy

Force generation by actin assembly shapes cellular membranes. An experimentally constrained multiscale model shows that a minimal actin network is sufficient to internalize endocytic pits against membrane tension. We developed a fluorescence-based molecule-counting method in live mammalian cells and found that ~200 Arp2/3 complexes assemble at sites of clathrin-mediated endocytosis in human cells. The advent of lattice light-sheet microscopy and 3D particle tracking methods promises to revolutionize the field of quantitative cell biology. We hypothesize that our molecule counting method, in combination with these live-cell 3D microscopy and particle tracking methods and machine learning based identification of cellular processes, will allow for the systematic, quantitative spatiotemporal understanding of dynamical events such as actin assembly at every cellular organelle and subcellular process.

———————————————————————–

(2) Adam Anderson - UCB

Post-doc - admndrsn@berkeley.edu

Recognizing the Impossible Image: How 3D Imaging Can Lead to Optical Character Recognition for Cuneiform Tablets

Archaeology in the Middle East and ancient Near East has a long and illustrious history, with more than 150 years of western academic scholarship in England, France, Germany, and the US. From the 1840s onward western archaeologists, like Sir Austen Henry Layard and Gertrude Bell, made early discoveries of textual artifacts in the heart of Mesopotamia, and awoke a deep curiosity in deciphering the beginnings of human history. Since these early discoveries, more than 500,000 cuneiform tablets have found their way to museums around the world, such as the Phoebe A. Hearst Museum, with a collection of ca. 1,000 tablets and artifacts with cuneiform writing.

By working with cutting edge tools and methods, including 3D digitization and Machine Learning (ML), our proposed 3D imaging project will make these ancient data publicly available, and provide repositories of big data for training the Optical Character Recognition (OCR) of cuneiform artifacts. With 4,000 years of history, the hand-written cuneiform signs include glyph variations that have posed a significant obstacle to OCR. If we are ever to build an OCR for cuneiform tablets, we have to train the machine on the wide variety of forms and contexts each glyph can take. Despite the fact that 2D images of tablets are readily available, the majority of the photographs (e.g. these 100k from the CDLI) do not have enough detail to allow for accurate rasterization and vectorization of the glyphs. In order to accommodate for the cuneiform regional dialects from these archives, 3D scanning of a wide variety of cuneiform tablets will be necessary.

Using ML methods, our project will delineate the variation within each glyph for each period represented in the collection, no doubt leading to countless new insights into these ancient archives. Partnerships with participating museums, will remain central to this project as well, in order to build up a sufficiently large training corpus of 3D signs in pair-wise context. Shown in the figure included, the proposed method records the depth of each point-cloud, which makes it possible to distinguish between different scribal hands. By building a large dataset of 3D models, this project will be able to encode greater dimensionality for ca. 500 unicode signs, and will train OCR software with the glyph variation needed to cross all time periods of cuneiform writing. The results can be used to identify unprovenanced artifacts with cuneiform around the world.

———————————————————————–

(3) Zhe Bai - LBNL

Postdoctoral Scholar - zhebai@lbl.gov

CT Template Creation using Nonlinear Image Registration for TBI Analysis

Understanding the impact of a brain injury and its possible treatments is a valuable challenge to social goods. We create computed tomography (CT) templates, leveraging diffeomorphic mapping algorithms, to analyze and compare 3D brain scans across a large population. The created templates help register and segment CT scans based on well known brain atlases, allowing more effective analysis of CT scans of traumatic brain injury (TBI) patients. Image registration is performed by warping each TBI patient’s CT scan to the template in a common human anatomical coordinate system. We build a pipeline to segment each subject’s scans based on our newly built template, by inversely mapping the hand-segmented brain atlas with histogram matching. The subregions of a brain are then statistically correlated to the patient’s Glasgow Coma/Outcome Score using a machine learning model. The study facilitates the diagnostic process in radiology and guides personalized treatment for TBI patients.

———————————————————————–

(4) Alexandra Bruefach - UCB

Graduate Student - abruefach@berkeley.edu

Deconvolution of electron energy loss spectral maps for atomic resolution property mapping

Energy Loss Spectroscopy (EELS) allows for the measurement of energy losses due to inelastic scattering of the electron probe upon interacting with the specimen, which provides information relating to the chemical identity, valence state, coordination number, bandgap, and plasmonic modes of the sample. A pixel-by-pixel energy loss map can be created by combining STEM and EELS (STEM-EELS), which allows for an energy loss spectrum to be taken at each probed point. However, EELS signals are highly convoluted due to the fact that inelastic scattering events mask peaks associated with plasmons of electronic transitions. Additionally, the zero-loss peak tends to drift unpredictably during the scanning process, making a direct comparison of signals impossible. To accurately quantify the features hidden within the spectra, it is necessary to implement individual spectral shifts and fits, as well as apply denoising algorithms to mask the non-Gaussian noise distribution. Furthermore, EELS signals lack information relating the crystallinity of a sample. Since structure ultimately governs the physical phenomena of nanomaterials, it is desirable to combine crystallographic STEM imaging with STEM-EELS to discover structural features that lead to enhancement or dampening of material properties. In this work, we highlight our recent advances towards processing STEM-EELS/HAADF STEM data for atomic resolution 2D images, with the goal of extending this to 3 dimensional tomographic reconstructions.

———————————————————————–

(5) Xingye Chen - UCSF/Tsinghua University

Ph.D. student - Xingye.Chen@ucsf.edu

Super-resolution Imaging Fluorescent Dipole Assembly with Polarized Structured Illumination Microscopy

Fluorescence polarization microscopy images both the intensity and orientation of fluorescent dipoles and plays a vital role in studying the molecular structures and dynamics of biocomplexes. However, it is difficult to resolve the dipole assemblies of subcellular structures and their dynamics in living cells at superresolution levels. Here, we report polarized structured illumination microscopy (pSIM), which decouples the entangled spatial and angular structural illumination by interpreting the dipoles in spatioangular hyperspace. We demonstrate the application of pSIM on a series of biological filamentous systems, such as cytoskeleton networks. pSIM can be directly applied to a large variety of commercial and home-built SIM systems.

———————————————————————–

(6) Fernando Constantino - Accenture

Senior Manager - f.constantino@accenture.com

Video Analytics applications in the Enterprise

Computer Vision is one area in the Artificial Intelligence domain where we already see a lot of applications nowadays. Ranging from autonomous vehicles, law enforcement and public safety, smart cities all the way to popular consumer apps such as Snap and the viral FaceApp.

With the focus on Enterprise use-cases, Accenture developed a Video Analytics Platform to help accelerate deployment of Computer Vision applications that can drive a tangible business outcome.

Together with our clients, Accenture is working in areas like Quality Inspection & Control, Workforce Productivity, Retail store optimization as well as Safety and Security.

Our approach combines hardware elements, in special where the requirements lead to the Edge processing, as well as a robust and flexible platform capable of deploying a variety of models.

———————————————————————–

(7) Olivia Creasey - UCSF & UCB

PhD Candidate - olivia.creasey@ucsf.edu

Quantification of pancreatic islet tissue structure through local cell-cell and cell-extracellular matrix interactions

Biological tissue function is intimately linked with biological tissue structure, defined as the 3D spatial arrangement of cells and extracellular matrix within a tissue. Changes in tissue structure are also intimately linked to tissue dysfunction during disease progression. In insulin-producing pancreatic islets of Langerhans, the relationships between tissue structure, tissue function, and diseases like diabetes remain poorly understood. Here, we developed an image analysis pipeline that provides quantitative measures of islet tissue structure in terms of the areas of cell-cell and cell-extracellular matrix interactions between multiple endocrine cell types, capillary basement membrane (BM), and peri-islet BM. We first use a machine learning-based classifier to automatically segment endocrine cells, capillaries, and peri-islet BM from high-resolution, high-depth 3D confocal images of human or mouse islets. Second, we use a custom Imaris Xtension to semi-automatically compile quantitative metrics of cell morphology and measure the number and surface area of individual cell-cell, cell-capillary BM, and cell-peri-islet BM interfaces from the segmented cells. The resulting quantitative description of each cell’s local structural microenvironment reveals how cells interact with each other and other tissue components. This allows us to more deeply classify cell types based on their structural niche, in contrast to only considering traditional marker expression. We propose that these measurements provide an additional layer of functional information that augments traditional cell atlas data. Further, this analysis pipeline can also be used to understand how tissue structure varies between species, over the course of disease progression, during development, or in response to other stimuli.

———————————————————————–

(8) Caroline Cypranowska - UCB

Graduate Student Researcher - casegura@berkeley.edu

Protein quantification

Motor neurons, across many chordate species, generally have far-reaching processes. For example, the presynaptic connections to a motor neuron in the adult human sciatic nerve is typically over 1 m from its downstream target muscle. These long processes pose unique challenges for quantifying protein expression by conventional fluorescence microscopy, even in smaller model organisms such as the Drosophila melanogaster larva in which some classes of motor neuron synapse with several muscles that are up to 1 mm apart. The goal of my project is to develop an image processing pipeline to compare expression of Complexin, a key regulator of neurotransmission, in type Ib and Is motor neurons in the larval fruit fly. The computational pipeline for this project includes registration of multiple 3D confocal microscopy stacks, the creation of synaptic volumes for each class of motor neuron by modeling a surface, creating masks from the surface, and integrating the fluorescence of the masked voxels. Implementation of this pipeline on fluorescence microscopy images of dissected larval preparations has shown that expression of Complexin between type Ib and Is motor neurons globally is fairly similar, but exists at greater local concentration in the termini of type Ib motor neurons.

———————————————————————–

(9) Alexandre de Siqueira - UCB

Postdoctoral researcher - alex.desiqueira@berkeley.edu

skimage: 3D Image Processing

Our tutorial (together with Dr. Daniela Ushizima) will present how to analyze three dimensional stacked and volumetric images at scale in Python, mainly using scikit-image. We start the tutorial presenting a brief overview of scikit-image and some related packages in the scientific Python ecosystem, such as NumPy, SciPy and matplotlib. Then, we discuss how to process two and three dimensional data through several steps. First, we pre-process the data using filtering, binarization and segmentation techniques. After that, we cover how to inspect, count and measure attributes of objects and regions of interest in the data, and also how to analyze their shape, color and texture features. At the end, we present the visualization of large 3D data and introduce just-in-time compilers and parallel processing to accelerate large data analysis. Real-world examples are given from domains such as materials science and biomedicine, and the material used in the tutorial is freely available.

———————————————————————–

(10) Brenda Delgadillo Moreno - CSU East Bay & LBNL

Graduate student Intern - bdelgadillo@lbl.gov

LID and BMP effect on Groundwater-Surface Water Interaction and Recharge in the East Bay

In September 2014, Jerry Brown signed the Sustainable Groundwater Management Act (SGMA) to require “management and use of groundwater in a manner that can be maintained during the planning and implementation horizon without causing undesirable results” (CDWR, 2016). As a result, East Bay MUD and the City of Hayward formed a Groundwater Sustainability Agency (GSA) to develop a Groundwater Sustainability Plan (GSP). Relative to the most contentious, high priority basins in California, the urbanized EB Plain, where nearly all of the water supply is imported surface water, has medium priority status according to CDWR. In contrast to Central Valley issues related to severe overdraft or land subsidence, the EB Plain GSP will set basic groundwork, quantifying water budget components and water fluxes. In particular, little is known about how the altered, engineered creeks of the East Bay interact with groundwater system. And, while new construction has typically included green infrastructure such as bioswales, permeable pavement, and stormwater detention, little is known about the ultimate fate of the retained water and their overall effect on recharge. My project will help to quantify the effectiveness of LID features in the East Bay, by constraining the timing of recharge and the locations of discharge for the water that infiltrates. According Newcomer et. al. (2014) there have been few field-based studies done to directly measure recharge rates below a LID BMPs and more are required to evaluate their effectiveness in LID BMPs recharge capability vs. other sources urban recharge, for example an irrigated lawn. Success of the GSP and implementation of SGMA objectives will likely serve as a way to diversify water sources for the EB urban area and may require enhanced groundwater recharge. This project will provide the GSA and other water managers with information about stream-groundwater exchange and the fate of water that enters the groundwater system via distribution system leakage and one or more existing LID features. My goal is to contribute information to multiple organizations with a conceptual model that can be used locally to help increase the diversification of water sources and that includes the potential effectiveness of enhanced recharge.I hope to develop a better understanding of GIS applications, isotropic tracers, contaminants in our local groundwater, modeling, data extrapolation, and the east bay’s groundwater movement.

———————————————————————–

(11) Cedric Espenel - Stanford university

R&D Scientist Engineer - espenel@stanford.edu

microscopy image analysis

Faithful inheritance of genetic information through sexual reproduction relies on the formation of crossovers (CO) between homologous chromosomes during meiosis, which, in turn, relies on the formation and repair of numerous double-strand breaks (DSBs). As DSBs pose a potential threat to the genome, mechanisms that ensure timely and error-free DSB repair are crucial for successful meiosis. During C. elegans meiosis, DSBs form in 4-7 fold excess of the eventual number of COs, most of them loading the recombinase RAD-51 and other pro-CO factors appearing as discrete foci along the chromosome. Following the number and distribution of such proteins is of paramount importance to understand the process happening during the course of CO formation. Current approaches require many hours of manual curation and depend on approaches that are difficult to reproduce. Here, we show that linear Support Vector Machine (SVM) can solve this challenge. We adapted a method previously developed by Dalal and Triggs. We train a linear SVM on a Histograms of Oriented Gradient (HOG) of image windows containing nucleus in the Pachytene stage of meiosis. We can then use our train classifier to recognize nucleus in relatively large microscopy 3D stacks images of whole-mount gonads (60 to 120 stacks, 15 to 30 stitched 512x512 images). We demonstrate that this approach can robustly identify the position of nuclei along the gonad and allow counting the number of DSBs per nucleus. A direct comparison with the manual approach showed no significant differences in the number of DSBs per nucleus found but lead to an important reduction in curation time. We conclude that this approach is accurate, reproducible and require less curation time. Moreover, it could be expanded to identify nucleus in different meiotic stage within the gonad and thus accurately monitoring the formation of DSBs during meiosis.

———————————————————————–

(12) Kira Evans - Chan Zuckerberg Initiative

Software Engineer - kira.evans@chanzuckerberg.com

napari: multi-dimensional image visualization in python

Recent advances in imaging hardware and probes have made it routine for biologists to generate large volumes of 3D, 4D, and 5D data, which is challenging to visualize, let alone analyze. Simultaneously, the Scientific Python ecosystem has grown dramatically in the past few years, particularly in machine learning, with the potential to accelerate and automate workflows for the hundreds of thousands of biologists working with microscopy images data every day. However, there are few options for visualizing and working with large multi-dimensional image datasets from within Python, making these machine learning tools out of reach for most biologists whose analysis is often coupled to interactive visualization and annotation.

To address these needs we are developing napari: a fast, interactive, multi-dimensional image viewer for Python. We’re developing napari in the open on GitHub with an open source license to facilitate transparency, reuse, and extensibility (https://github.com/napari/napari). Napari is designed for browsing, annotating, and analyzing large multi-dimensional images. It is built on Qt (for the GUI), vispy (for performant GPU-based rendering), and the scientific Python stack (numpy, scipy, and scikit-image). It includes critical viewer features out-of-the-box, such as support for large multi-dimensional data, and layering and annotation. By integrating closely with the Python ecosystem, napari can be easily coupled to leading machine learning tools (e.g. TensorFlow, PyTorch), enabling more user-friendly automated analysis. We are planning a version-controlled plugin ecosystem for napari to help users share trained models and custom analysis routines, thereby providing biologists easy access to advanced machine learning through a performant image viewer.

———————————————————————–

(13) Xiomara Gonzalez - UCB, LBNL

Undergraduate Student - xiomarag@berkeley.edu

Automating fiber detection in ceramic-matrix composites using a mixed-scale dense deep convolutional neural network

Ceramic-matrix composites (CMCs) have the potential to become the leading material of choice for aviation structures due to their ability to withstand high tensile and compressive loads at temperatures above 1700oC. CMCs are hierarchical materials composed of SiC fibers, each with a BN coating, woven into custom bundle patterns and embedded in a SiC matrix. Using synchrotron-based x-ray micro-computed tomography, CMCs can be imaged in three dimensions at micrometer-scale resolution. Deformations captured from three-dimensional analysis of the CMCs microstructure are essential to understanding the properties of the material. This proves to be a challenge when datasets contain thousands of images, each 2000 x 2000 pixels and voxel edge length of 1um. Here, we utilize a mixed-scale dense deep convolutional neural network (MS-D DCNN) to automate the detection of CMCs fiber centers in multiple datasets. The network architecture uses dilated convolutions to capture features at different scales and densely connects feature maps with each other. This allows us to obtain accurate detections with only a few image training pairs and parameters. Preliminary results show the neural network obtaining an average F1 score of 0.986 on test sets. With further testing and parameter tuning, our goal is to use the center detections to track fibers throughout the 3D volume and locate regions where fiber tears occur as the material is exposed to dire environments.

———————————————————————–

(14) Kate Groschner - UCB

PhD Candidate - cgroschner@berkeley.edu

Segmentation of Micrographs for High Throughput HRTEM

Deep learning methods have increasingly been applied to all types of microscopy, but only recently have they been used for analysis in electron microscopy (EM). Several networks have been shown for segmentation of biological samples in cryo-EM data but to date no work has been done on segmentation of high resolution transmission electron microscopy (HRTEM). HRTEM is used for imaging the atomic structure of materials, usually in nanomaterials. Without the ability to automatically identify and classify regions of interest in HRTEM images, it is impossible to incorporate HRTEM into a high throughput analysis workflow and therefore impossible to obtain statistical representations of nanoparticle populations. However, segmentation of HRTEM images is challenging because of their significant differences from real or even cryo-EM images because images are on such an extremely small length scale that the separation between the object and background can be challenging even for human observers. Unlike in other micrographs, objects are not usually defined by clear edges or change in grayscale value but instead by the internal structure of object compared to that of the background. Clearly this makes for an interesting segmentation challenge. In our work we have explored the parameters which make segmentation of HRTEM images possible. Using the well known U-Net architecture, we have trained a network with sensitivity to atomic lattice structures which achieves 94% pixel-wise accuracy on the segmentation of nanoparticle regions. This network shows the flexibility of deep learning methods across image types and the ability to segment even at the smallest length scales.

———————————————————————–

(15) Michael Kellman - UCB

Graduate Student Researcher - kellman@berkeley.edu

Data driven design for computational microscopy

Computational imaging marries the design of hardware and computation to create a new generation of systems to image beyond what is normally possible. Generally, a computational imaging system’s performance is governed by how well information is encoded in the measurements (experimental design) and decoded from the measurements (computational reconstruction). In settings where the encoding is non-linear and/or the decoding is iterative and non-linear (e.g. compressed sensing, phase retrieval), traditionally methods to analyze the system become difficult to use and do not necessarily result in improved performance. In this talk, I will overview our recent work to jointly learn aspects of the experimental design and computational reconstruction to optimize the performance of a computational imaging system. We will consider unrolling the iterations of a traditional model-based image reconstruction (e.g. compressed sensing, phase retrieval) to form a network made up of linear layers (gradient steps) and non-linear layers (proximal steps) and then optimizing the system end-to-end. In a case study, I will show how a standard microscope can be transformed to image at a much higher resolution (super-resolution), the short comings of this approach, and how learning the design of the microscope is able to overcome these short comings.

———————————————————————–

(16) Aniket Kesari - UCB

Graduate Student - akesari@berkeley.edu

Improving Traffic Safety Through Video Analysis in Jakarta, Indonesia

This project presents the results of a partnership with Jakarta Smart City (JSC) and United Nations Global Pulse Jakarta (PLJ) to create a video analysis pipeline for the purpose of improving traffic safety in Jakarta. The pipeline transforms raw traffic video footage into databases that are ready to be used for traffic analysis. By analyzing these patterns, the city of Jakarta will better understand how human behavior and built infrastructure contribute to traffic challenges and safety risks. The results of this work should also be broadly applicable to smart city initiatives around the globe as they improve urban planning and sustainability through data science approaches.

———————————————————————–

(17) Carolyn Larabel - UCB, LBNL

Professor - calarabell@lbl.gov

Imaging complex biological machines in action

This talk will describe plans and recent accomplishments in integrating machine-learning algorithms for segmentation into the pipeline for processing soft X-ray tomography (SXT) cell imaging data. SXT rapidly visualizes mesoscale functional structures (e.g. organelles, molecular machines) of intact, unperturbed cells better than any other imaging technique. However, segmenting and annotating the tomographic reconstruction remains a labor-intensive manual process. Using supervised machine learning to train computers to automatically identify and isolate each of the major organelles from other cell contents will overcome this bottleneck. In addition, giving the task of segmentation over to computers will ensure objectivity and consistency of data deposited in the Human Cell Atlas.

———————————————————————–

(18) Dima Lituiev - UCSF

Postdoc - d.lituiev@gmail.com

Slideslicer: a package for manipulation of whole slide imaging and annotations

Recent advances in computer vision (CV) and machine learning together with a broader acceptance of digital pathology workflows both in research and clinic call for flexible open source tools to manipulate not only the WSI but also annotations. Here we present an open source Python package for jointly handling WSI and annotations for purposes of machine learning. Our library leverages a range of widely accepted open-source tools and formats such as Openslide, shapely, MS-COCO API, and lxml. We describe implementation and features, and demonstrate its deployment in several computer vision pipelines.

———————————————————————–

(19) Xinghua Lou - Vicarious

R&D - xinghua.lou@gmail.com

A Rapid and Efficient 2D/3D Nuclear Segmentation Method for Analysis of Early Mouse Embryo and Stem Cell Image Data

Segmentation is a fundamental problem that dominates the success of microscopic image analysis. In almost 25 years of cell detection software development, there is still no single piece of commercial software that works well in practice when applied to early mouse embryo or stem cell image data. To address this need, we developed MINS (modular interactive nuclear segmentation) as a MATLAB/C++-based segmentation tool tailored for counting cells and fluorescent intensity measurements of 2D and 3D image data. Our aim was to develop a tool that is accurate and efficient yet straightforward and user friendly. The MINS pipeline comprises three major cascaded modules: detection, segmentation, and cell position classification. An extensive evaluation of MINS on both 2D and 3D images, and comparison to related tools, reveals improvements in segmentation accuracy and usability. Thus, its accuracy and ease of use will allow MINS to be implemented for routine single-cell-level image analyses.

———————————————————————–

(20) Jeremy Maitin-Shepard - Google

Senior Software Engineer - jbms@google.com

Tools for large-scale data visualization and storage

I will present two related open source tools:

Neuroglancer is an open-source web-based 3-d volumetric data viewer that integrates arbitrary (oblique) cross-sectional views of image and segmentation data with 3-d rendering of object surfaces and annotations. With native support for multi-resolution data, it scales from small in-memory volumes to petabyte-scale volumes. It has become widely adopted within the electron microscopy connectomics community but is also widely applicable to other domains. As a purely client-side web-based tool, it supports a variety of existing data formats and protocols and can be easily adapted to new ones. The Python integration library allows it to be used as a lightweight interactive visualization and annotation tool, combining both external data sources and in-memory NumPy array-like volumes.

TensorStore is an open-source library under development for efficiently reading, writing and sharing large multi-dimensional datasets, designed to be compatible with a wide variety of existing array formats and storage mechanism, and scaling from multi-megabyte datasets stored on a single machine to multi-petabyte datasets read and written concurrently by hundreds of thousands of machines in parallel. It provides a uniform API for reading and writing any supported underlying array representation, and thereby decouples tools and pipelines from the actual stored representation.

———————————————————————–

(21) Silvia Miramontes - LBNL

Graduate Student Researcher Assistant - mirasilvia@lbl.gov

Convolutional Neural Networks of Fiber Detection models

To provide with a scientific image analysis of experimental data acquired at high resolution, we have recently developed a set of Python-based pipelines that enable data management in decision-making processes. To illustrate our pipelines, we show the investigation of resistant materials that can withstand high temperatures, with their quality control being dependent on the microtomography scanners at LBNL ALS beamline 8.3.2. For example, our software enables the spatial analysis of fibers, matrix crack detection, among others. While there are several approaches for fiber detection from microtomography data, materials scientists lack computational schemes to validate the accuracy of different fiber detection models. Thus, we are eager to present a set of statistical methods to analyze microtomography images in 3D and visualize respective fiber beds, including the introduction of a lossless data reduction algorithm based on maximum projection to detect the specimen bulk. The main contribution is our method based on a convolutional neural network that enables the evaluation of results from automated fiber detection models, particularly when compared with human curated datasets. All algorithms are built using free tools from the Scientific Python Software stack to allow full reproducibility of the experiments, and illustrate our results using algorithms designed to probe sample content from gigabyte-size image volumes with non-specific computational infrastructure.

———————————————————————–

(22) April Myers - UCB

Graduate Student - april_myers@berkeley.edu

Videographic data of facial activity reveals variance in neural populations associated with fear conditioning

Ensembles of neurons in the visual cortex have highly structured activity thought to represent visual stimuli, yet has been shown to strongly correlate to events unrelated to vision such as motor behavior. Earlier this year, Stringer et al (2019) used shared variance component analysis (SVCA) to illustrate that significant portions of the variance in spontaneous neuronal ensemble activity can be reliably explained by high-dimensional facial and eye movement signals related to overall arousal as well as other factors. The extent to which these results hold true for learned behavior remains to be seen. Our group seeks to reveal how changes in neural populations in the visual cortex underlie perceptual learning. However, obscuring of learning signals by variance introduced by latent signals may hinder this goal. SVCA may be a useful tool towards this end. We plan to analyze videographic data of the facial activity of mice during fear conditioning and correlate events observed in these videos with ensemble activity. We will use SVCA to denoise our data by identifying and removing variance not relevant to fear conditioning in order to establish a pipeline to evaluate important physiological correlates of fear. We hope that the topics in advanced image processing discussed at ImageXD will assist in the detailed and rigorous analysis of our data.

———————————————————————–

(23) Tamas Nagy - UCSF

PhD Student - iam@tamasnagy.com

Working with multidimensional microscopy images in Julia

Julia is a fast general-purpose language that is particularly well-suited for working with high-dimensional arrays as found in image analysis. It already features a robust imaging library in the JuliaImages project, but it really starts to stand out when metadata is associated with each axis of the image. Towards this goal, I have developed a standards-compliant, well-tested OME-TIFF reader for working with multidimensional microscopy images. It exposes the advanced metadata embedded in each image so that it is trivial to implement dimension-order independent indexing, unitful indexing, per-plane metadata extraction, etc. I will also show how to leverage this metadata to craft more general, more resilient, and faster image analysis algorithms using the wider JuliaImages ecosystem.

———————————————————————–

(24) Debora Nasser Diniz - Federal University of Ouro Preto

Cell Nuclei Detection from Pap Smear Images - debnasser@gmail.com

An Iterated Local Search Algorithm for Cell Nuclei Detection from Pap Smear Images

we propose an Iterated Local Search approach to detect cervical cell nuclei from digitized Pap smear slides. The problem consists in finding the best values for the parameters to identify where the cell nuclei are located in the image. This is an important step in building a computational tool to help pathologists to identify cell alterations from Pap tests. Our approach is evaluated by using the ISBI Overlapping Cervical Cytology Image Segmentation Challenge (2014) database, which has 945 synthetic images and their respective ground truth. The precision achieved by the proposed heuristic approach is among the best ones in the literature; however, the recall still needs improvement.

———————————————————————–

(25) Henry Pinkard - UCB

Graduate Student - henry.pinkard@gmail.com

Neural network-controlled microscopy and robust registration of noisy data using optimization

During the past two decades we have been able to answer many fundamental questions about the organizing principles of the immune system using in vivo 2-photon microscopy of mouse lymph nodes. However, one long-standing limitation of this approach has been the relatively small volumes of living tissues that can be imaged in a given experiment. Imaging a large volumes requires dynamic adjustment of the power of a scanning laser used to generate contrast to compensate for the scattering and absorption of living tissue, and doing so incorrectly destroys the sample that is being imaged. We developed a way to solve this problem by using a neural network, which has been trained to compensate for sample-induced scattering and absorption, to control the laser intensity of a two-photon microscope as it images.

This technique has allowed the in vivo imaging of immune responses on a whole lymph node level for the first time, but it also presents several challenging image registration tasks. A lymph node during an immune response is a dynamic environment, and deep-tissue fluorescence imaging has inherently low signal to noise (SNR). Traditional image registration techniques compute point estimates of optimal registrations by approximating a correlation coefficient between two images with a cross-correlation. Cross-correlations have the advantage that they can be quickly computed with Fourier transforms, but fail on the noisy and heterogenous data used here. To address this, we developed a novel image registration algorithm in which we optimize a model of image movement using cross-correlation as an objective function. This enables image registration that is robust to variations in image brightness and SNR.

———————————————————————–

(26) Esther Rolf - UCB

PhD Student - esther_rolf@berkeley.edu

Global scale observation using satellite imagery and machine learning

Abundant and temporally recurrent data collection of satellite imagery has the potential to transform our understanding of the world, from monitoring deforestation, to tracking animal migration, to assessing economic development. While globally comprehensive imagery can illuminate remote regions of the world, the abundance and richness of these data also generate challenges. Computing with and even storing large quantities of satellite imagery, which can be collected on the order of terabytes per day, is often beyond the skills and resources of a single researcher. To address this challenge, we use advanced mathematical techniques from the theory of random convolutional kitchen sinks to develop task-agnostic features that capture a remarkable amount of predictive signal for a variety of task domains. Our work makes satellite imagery and machine learning accessible to researchers of nearly all skill sets.

Specifically, we develop a methodology that transforms raw imagery into succinct vector representations of images, which can be paired with simple linear regressions to predict virtually any variable of interest that is visible from space. Users of the system need only to download these geo-referenced feature vectors, merge their own geo-referenced labels, and solve a linear regression. After computing and storing the vectors, we test their predictive performance on seven diverse tasks at US scale, and four globally (due to limited data availability of the other three). Our method predicts unseen labels will high skill across all seven tasks, with performance on par with or exceeding that of a fine-tuned convolution neural net (CNN) designed as a comparison for these tasks, with speed that is orders of magnitude faster than the CNN. The training and prediction step is computationally feasible on a standard laptop. In designing a simple and scalable architecture for satellite imagery and machine learning, we also gain insight into what computational tools and operations are most efficient for satellite imagery predictions, and perhaps structured imagery predictions more broadly.

This class of approach has the potential to impact all areas of imaging science, where rapid development and evaluation of task-specific models on task agnostic features enables researchers from very different backgrounds to build powerful models across the entire dataset. This is the result of a collaboration between students and faculty in Computer Science and the Global Policy Lab at UCB.

———————————————————————–

(27) Loic Royer - Chan Zuckerberg Biohub

Group Leader - loic.royer@czbiohub.org

Pushing the Limits of Fluorescence Microscopy with adaptive imaging and machine learning

Fluorescence microscopy lets biologist see and understand the intricate machinery at the heart of living systems and has led to numerous discoveries. Any technological progress towards improving image quality would extend the range of possible observations and would consequently open up the path to new findings. I will show how modern machine learning and smart robotic microscopes can push the boundaries of observability. One fundamental obstacle in microscopy takes the form of a trade-of between imaging speed, spatial resolution, light exposure, and imaging depth. We have shown that deep learning can circumvent these physical limitations: microscopy images can be restored even if 60-fold fewer photons are used during acquisition, isotropic resolution can be achieved even with a 10-fold under-sampling along the axial direction, and diffraction-limited structures can be resolved at 20-times higher frame-rates compared to state-of-the-art methods. Moreover, I will demonstrate how smart microscopy techniques can achieve the full optical resolution of light-sheet microscopes - instruments capable of capturing the entire developmental arch of an embryo from a single cell to a fully formed motile organism. Our instrument improves spatial resolution and signal strength two to five-fold, recovers cellular and sub-cellular structures in many regions otherwise not resolved, adapts to spatiotemporal dynamics of genetically encoded fluorescent markers and robustly optimises imaging performance during large-scale morphogenetic changes in living organisms.

———————————————————————–

(28) Xiongtao Ruan - UCB

Postdoctoral Research Fellow - ruan.xiongtao@gmail.com

CellOrganizer: tools for learning and using generative models of 3D cell shape structure

Cell shape plays a crucial role in many cellular processes. However, most analysis of cell shape uses a handful of descriptive features and does not adequately capture cell shape variance and dynamics, especially in 3D. Constructing generative models can provide a better basis for analyzing, comparing and interpreting differences in cell shape. Here we implemented the SPHARM-RPDM method for 3D cell shape modeling in the open-source software CellOrganizer. The method allows robust and efficient modeling of 3D cell shapes, especially for complicated ones. We also implemented several capabilities for the use of shape models such as shape reconstruction, shape space visualization, sampling from shape spaces, and simulation of shape dynamics. To enable use by diverse users, we have made CellOrganizer available as Matlab source code and precompiled binaries that do not require a MATLAB license. We also provide versions usable through Galaxy, Docker container, Singularity and Jupyter notebook for users with different programming backgrounds and computational resources.

———————————————————————–

(29) Robert Sare - Stanford University

Ph.D. student (and Descartes Labs data science intern) - rmsare@stanford.edu

Edge detection of fault scarps

scarplet is a Python package for applying image processing techniques to digital elevation data, in particular for detecting and measuring the maturity of fault scarps and other landforms. It is intended for earth scientists who want to apply diffusion dating methods to or extract other landforms from large datasets using user-defined landform template functions. Our group has used the scarplet API to measure landform attributes in tiled raster data at scale, in particular within the major fault zones of northern California, including the San Andreas fault.

———————————————————————–

(30) Carl Schwendinger-Schreck - UCB

Postdoc - carlschreck213@gmail.com

Inferring phenotype from population structure

One of the most time-consuming steps involved in performing a genetic screen is selecting for the desired phenotype. Here, I describe a fast and inexpensive machine learning protocol for genetic screening in which phenotype is inferred from spatial population structure.

Even in a model organism as simple as yeast, scanning through thousands of colonies can require massive manpower, sophisticated image analysis tools, and a suite of automation. No matter how the phenotype manifests, whether in an easily-screened assay like expression/repression of a fluorescent reporter or in a more natural change in growth dynamics that may leave behind no readily-visible signature, we want to distinguish the phenotypes using as little data as possible. To highlight the value of machine learning in genetic screening, I use here bud-site selection S.cerevisiae (budding yeast) mutants-where the final cell morphology is normal (i.e. round), but the dynamics of growth differ (whether buds form axially or bipolarly or neither, see Fig. 1). In this case, I’ve chosen single timepoint images of each colony so that timelapse movies are not required.

To test our protocol, I created thousands of budding cell colonies using physics-based computational simulations. These simulated colonies are inoculated with a single cell and expand outward as cells grow and push their neighbors. In order to mimic how S. cerevisiae colonies grow in the lab, I constrain growth to 2D (as if colonies are growing on flat plate) and restrict cellular growth to a region near the colony edge (as if nutrients are diffusing in from outside the colony). My goal is, from these images alone, to develop an algorithm to distinguish which budding type corresponds to which colony.

By training neural networks using scikit-learn in Python, I can achieve an accuracy up to model 87%. We can achieve this high degree of accuracy even though the colonies have no readily-visible distinguishing features to the naked eye. Further, by selecting colonies based upon prediction probabilities, I can push this accuracy even higher - for instance by breaking the test set in half and focusing the subset with the largest probabilities I achieve an accuracy of 98%. This method may provide both tremendous cost savings as it circumvents additional time consuming and expensive phenotypic screens.

———————————————————————–

(31) Xiaoyu Shi - UCSF

postdoc - Xiaoyu.Shi@ucsf.edu

Machine learning for live-cell super-resolution microscopy

It is very challenging to achieve both super spatial and temporal resolution of biological processes in live cells. Most super-resolution fluorescence microscopy methods take much longer time to acquire images, compared to conventional fluorescence microscopy. We propose to use machining learning algorithms to reconstruct the super-resolution images from the live cell images taken with fast conventional fluorescence microscopes.

———————————————————————–

(32) Nicholas Sofroniew - Chan Zuckerberg Initiative

Computational Biologist - nsofroniew@chanzuckerberg.com

napari: multi-dimensional image visualization in python

napari: multi-dimensional image visualization in python

Kira Evans1, Kevin Yamauchi2, Ahmet Can Solak2, Bryant Chhun2, Shannon Axelrod1, Jeremy Freeman1, Loic Royer2, Juan Nunez-Iglesias3, Nicholas James Sofroniew1,

1. Chan Zuckerberg Initiative
2. Chan Zuckerberg Biohub
3. Monash University

Recent advances in imaging hardware and probes have made it routine for biologists to generate large volumes of 3D, 4D, and 5D data, which is challenging to visualize, let alone analyze. Simultaneously, the Scientific Python ecosystem has grown dramatically in the past few years, particularly in machine learning, with the potential to accelerate and automate workflows for the hundreds of thousands of biologists working with microscopy images data every day. However, there are few options for visualizing and working with large multi-dimensional image datasets from within Python, making these machine learning tools out of reach for most biologists whose analysis is often coupled to interactive visualization and annotation.

To address these needs we are developing napari: a fast, interactive, multi-dimensional image viewer for Python. We’re developing napari in the open on GitHub with an open source license to facilitate transparency, reuse, and extensibility (https://github.com/napari/napari). Napari is designed for browsing, annotating, and analyzing large multi-dimensional images. It is built on Qt (for the GUI), vispy (for performant GPU-based rendering), and the scientific Python stack (numpy, scipy, and scikit-image). It includes critical viewer features out-of-the-box, such as support for large multi-dimensional data, and layering and annotation. By integrating closely with the Python ecosystem, napari can be easily coupled to leading machine learning tools (e.g. TensorFlow, PyTorch), enabling more user-friendly automated analysis. We are planning a version-controlled plugin ecosystem for napari to help users share trained models and custom analysis routines, thereby providing biologists easy access to advanced machine learning through a performant image viewer.

———————————————————————–

(33) Jennifer Sun - UCSF

Postdoc Scholar - J.suninchina@gmail.com

Semi-Automated Puncta Detetion with ImageJ

To study the anatomical changes in the synaptic markers during visual plasticity, we applied two-photon imaging to repeatedly image GFP-labeled synapses every 2-3 days in mouse visual cortex. Images of a labeled neuron and its collateral spines and neurite structures were acquired by serial scanning a region and processed offline for 3D stitching (Fig A). To track the multiday changes of synaptic markers in the same neuron, images from different time points were aligned to the baseline images with both rigid and non-rigid registration in the persistent red channel, which can be matched perfectly for comparison (Fig B). The same transformation matrix is then applied to green channel for the GFP-labeled synapses, and the spherical clusters of spines could be detected and analyzed with customized and semi-automated ImageJ package (Fig C, refer to github repository), followed by manual inspection to verify the gain/loss of each single puncta. For control animals during normal sensory experience, alignment and puncta detection of synaptic markers are quit comparable even when imaged three days apart.

———————————————————————–

(34) Anton Tremsin - UCB

Full Research - ast@ssl.berkeley.edu

Energy-resolved neutron imaging for non destructive studies in materials science, cultural heritage and energy research

Recently developed neutron detection technology enables novel studies where neutrons used as probes of internal structure of various objects. Detectors, which were initially developed for NASA satellite-based imaging, were adopted to applications, where we use neutrons instead of photons. This somewhat exotic imaging technology enables unique studies in fields as diverse as materials science, energy research, cultural heritage, aerospace engineering and many others. In these imaging experiments, one of the challenges is a large amount of imaging data combined with neutron transmission spectra, which need to be analyzed in order to extract the properties of investigated samples. Various data analysis tools and fast fitting methods need to be developed in near future to take full advantage of these novel imaging methods.

———————————————————————–

(35) Michael Viacheslavov - BioAge Labs

Histology & In Vivo Associate II - michael@bioagelabs.com

Novel Automated Methods for Scoring Regenerating Myofibers

When receiving an injury, the skeletal muscle will do everything in its power to recruit immune cells to clean up all the necrotic fibers and activate satellite cells that ultimately regenerate newly formed muscle fibers. Healthy myofibers typically are multinucleated, with nuclei positioned on the periphery of the cell. Regenerative myofibers on the other hand, are not multinucleated and can be identified by their singly and centrally positioned nucleus. To measure the regenerative index of a specimen the microscopist would typically identify the injured area as a region of interest (ROI), manually count the amount of centrally nucleated myofibers (CNM), count the total amount of cells, and calculate the CNM per total cell density. Although the capacity to biologically assay regeneration within the skeletal muscle has been very well established, the ability to analyze the microscope images have been quite limiting by human vision. Not only is such a traditional counting methods time consuming, but also brings noise when comparing the cell counts of different scorers. To counteract such time and labor intensive work, BioAge Labs has explored and adopted several automated methods on ImageJ to cut down on the amount of time it would typically take to count the cells manually.

———————————————————————–

(36) Panod Viseshchitra - UCB

Graduate student - panodvis@berkeley.edu

Determination of the 3D nanostructure of Calcium Silicate Hydrate and Calcium (alumino)silicate hydrate by using Scanning Transmission Electron Microscope (STEM) and soft x-ray tomography

Calcium Silicate Hydrate (C-S-H) and Calcium (alumino)silicate hydrate (C-A-S-H) are the key binding phases and primary contributors to the mechanical properties of most hydrated cements and concretes. The porosity of the hydration products plays a key role in the durability-based performance of concrete. The size distribution and connectivity of the pores will determine the ability of fluids and ions to flow through the network, potentially degrading the material. To date, there have been many studies on the nanostructure of portland-cement-based systems. The results of these techniques are distilled only into two-dimensional plots. Due to the complex nature of cement hydration and the sensitivity of the products formed, the majority of three-dimensional(3D) information available is not at the nanoscale. This study will aim to perform 3D STEM tomography to characterize the porous networks of C-S-H and C-A-S-H in cements. The successful methodology leads the way to precise determination of the nanostructure of calcium silicate hydrates. This project aims to empirically investigate this missing information and to integrate the results with the ones obtained spectromicroscopy at ALS. The resulting information is critical of the development of a new generation of optimized green cements.

———————————————————————–

(37) Yina Wang - UCSF

Postdoc - yina.wang@ucsf.edu

STORM Correlation analysis

Correlation analysis is one of the most widely used image-processing methods. In the quantitative analysis of localization-based superresolution images, there still lacks a generalized coordinate-based correlation analysis framework to take full advantage of the superresolution information. We propose a coordinate-based correlation analysis framework for localization-based superresolution microscopy. We mathematically prove that point-point distance distribution is equivalent to pixel-based correlation function. This framework can be easily extended to model the effect of localization uncertainty, to the time domain and other distance definitions. We demonstrated the versatility and advantages of our framework in three applications of superresolution microscopy: model-free image alignment and averaging for structural analysis, spatiotemporal correlation analysis for mapping molecule diffusion, and quantifying spatial relationships between complex structures.

———————————————————————–

(38) Wanpeng Wang - UCSF

Postdoc - georgewwp@gmail.com

Investigating the mechanism of cell-cell contact using microscopy

Cell-cell contact underlies the complex biological process of multicellular organisms. Tissue establishment, neuron connectivity and cancer pathology impinge on cell-cell contact and communication. Emerging evidence demonstrated that signaling protein dispersion can be mediated by direct contact of specialized filopodia of cells. These contacts depend on synapse proteins and the relying of signal depends on the membrane potential. Thus, neurons, developing organ and the cancer cells seem to employ a shared toolkit for cell-cell contact and communication. The new paradigm challenges our understanding of morphogen gradient formation and suggests direct contact between signal producing and receiving cells as a mechanism to confer communication specificity.

We investigate the mechanism of cell-cell contact Drosophila trachea developmental model. Using multiple microscopy methods, including structural illuminated microscopy, light-sheet microscopy and confocal microscopy, we explore the biology of cell-cell contact. We’re also implementing machine-learning techniques to do label free imaging.

———————————————————————–

(39) Ke Xu - UCB

Graduate Student - ke_xu@berkeley.edu

Tomography image segmentation

Phase segmentation for 3D tomography image of Ancient Roman concrete is of great interest as part of research in designing next generation of concrete. In-situ synchrotron X-ray microtomography (uCT) tests of ancient Roman concrete samples under uniaxial compressive loading were carried out to study the complicated 3D crack propagation and evaluate the fracture pattern. Supervised and unsupervised machine learning (ML) methods were used to segment different phases (matrix, aggregate and porosity phases) of the tomograms. Digital volume correlation (DVC) mapping of displacement fields and major principal strain fields were used to visualize the crack propagation and characterize the fracture pattern. Findings indicate that the ancient Roman concrete samples presented a stable and ductile fracture pattern in the in-situ compression test. Multiple micro and macro cracks propagation, diffraction and furcation of cracks traces were observed based on the 3D reconstructed results using computer vision and statistical methods for analysis.

———————————————————————–

(40) Iris Young - UCSF

Postdoc - irisdyoung@gmail.com

Modeling flexibility in macromolecular structures obtained by cryo-electron microscopy

For determination of the 3-dimensional structures of macromolecules there are presently two favored techniques, X-ray crystallography and electron microscopy, which are suitable for partly-overlapping ranges in this size scale (roughly 1-100 nm). Both techniques require boosting signal-to-noise by averaging signal over a large number of individual particles. In crystallography this is accomplished experimentally, but in electron microscopy, many views of many particles are recorded individually and averaged computationally, providing an opportunity for improved image analysis and reconstruction tools to improve on current methods. Namely, internal motions of particles are currently treated as a source of error in most reconstruction algorithms, or at best handled only partially, placing various restrictions on the types of motions that can be modeled. Our project seeks to incorporate motion into the reconstructed volumes at all stages of map reconstruction to preserve as much of this information as possible, allowing for faithful recovery of continuous deformations.

Our approach builds upon and expands the cisTEM cryo-electron microscopy data processing software. In a first pass we will obtain a static volume by existing image alignment and reconstruction algorithms and refine a flexible atomic model into this volume. In a second pass, images may be aligned to any of the reference volumes along a trajectory matching the motions of the atomic model, and both the reference models and reconstruction will be iteratively refined. The route by which the model informs the reference volume currently involves combining the high frequency information from the model with the low frequency information from the current reference volume, which is reconstructed after each round of image alignment from the raw images directly. The step that merges information from the atomic model and reference volume is one possible place the image analysis community might have valuable input. The other step is the classification of noisy images during alignment to a flexible reference volume. I would be highly interested in exploring ways of handling these steps that might have been approached differently in other disciplines.

———————————————————————–

(41) Patrick Zager - Weill Cornell Medical College

Research Technician - zagerpatrick@gmail.com

Applications of Computer Vision in Vison Science

Imaging of the eye, in live animals or through extracted tissues, is a costly procedure which bears results that are largely still analyzed qualitatively. Image analysis offers a means of better leveraging this data, allowing us to quantitatively determine measures of phenotype or visual function. This work summarizes numerous applications of computer vision in tandem with experimental approaches, cell biology and physiology, in vision science research. First, flat mounts, individual tissue layers of the eye which are surgically extracted and laid flat for microscopy, are analyzed to calculate parameters such as: individual retinal pigment epithelium morphology, immune response, or angiogenesis. Next, cross sectional images of the eye, captured through optical coherence tomography, are analyzed to determine the thickness of various tissue layers of the eye. Finally, images captured through video-oculography are analyzed to quantify the visual acuity of mice. This work demonstrates the broad applicability of computational image analysis in the field of vision science.