New AI Tool for Image Analysis

When Silicon Valley VC firm Andreesen Horowitz funded a couple of Stanford students in the cell analysis space I took notice.?As a fan of Marc Andreessen (founder of Netscape) and Ben Horowitz (author of “The Hard Thing about Hard Things”), I was curious what technology could attract this Silicon Valley heavyweight.?Nurlybek Mursaliyev and Michael Lee founded Biodock in 2020 to apply AI to customer image analysis pipelines.?At Nurlybek’s urging, I ran a demo on an image set I had on hand.?The results were enlightening and I thought I’d share my experience.

Background- Thresholding and Segmentation

I’m generally familiar with cell image analysis approaches.?The most common is setting a simple brightness threshold and enclosing the areas above this as “objects” or cells in this case.?Sometimes one uses a nuclear stain to indicate a cell and then draws an estimated cell boundary that is equidistant between any two nuclei.?

In most cases, a number of simple measurements result from identifying each cell.?Intensity (average, peak, total), area, perimeter, length, width, and circularity or aspect ratio can be calculated for each.?One can use these measurements, and their distribution, to further discriminate those of interest.?For instance, cells are generally not less than 5 microns in diameter although debris is and often bright. ?By filtering on size, cell identification is improved.?Furthermore, one may only consider cells that are also within a brightness range.?This is analogous to a flow cytometry scatter plot showing gating filters around double positives, double negatives, etc.? An assay may simply be a normalized count of the filtered cells in the image.?Alternatively, a measurement such as average intensity, within the population of cells, can be used to quantify an assay result.??

Pixel Classification

The next advancement that I’ve encountered is referred to as Pixel Classification systems.?The new thing here is that one can manually trace or select the objects of interest from training images and the software will “learn” the measurement filters that select for the desired objects; very handy and pretty understandable.

AI and Deep Learning

Artificial Intelligence has an approach called Deep Learning that is not that easily understood.?This is what Biodock does. When I’ve asked experts what Deep Learning is, I usually get some version of “it works like your neurons” and that isn’t very satisfying.

“The description problem (or "representation") I think is fundamental to Neural Networks. What does the firing of a neuron "mean"? A key point of parallel distributed processing is that information is distributed across the network. That leads to robustness, generalization, ... but also that we can't attribute or explain in a localized or algorithmic manner what the system is doing.”??-B. Rasnow

In Deep Learning, as in Pixel Classification, we circle the objects we want and it learns to recognize them in new images and circles them for us. ?Once it finds the objects, you get all the measurements provided by the previous techniques. The difference is in how it learns.

I found this video that used handwritten numeral recognition as an example. But what is a neural network?

In the video, when trying to recognize the written digits 0 through 9, one might look for loops or curls and long or short horizontal, diagonal or vertical lines.?Each digit would have a different profile of these primitive shapes, i.e. an 8 has two loops, no lines and a 9 has one loop and one line, and thus could be discriminated.?One could imagine a secondary interaction layer where a curl could be the unclosed loop of a 0 or 9 so would have some non-zero weighting for those digits to account for bad handwriting.?

In the case of microscopy images, an object may have a contiguous edge of bright pixels with a radius of curvature within a range.?It may also have a radial intensity profile that has the characteristic perimeter halo of a cell under phase contrast.?One can imagine many more primitive features that could be evaluated against an image leading to a confident discrimination of the objects of interest.

If I had to describe it to a layman, I'd say it’s like a multi-level game of “20 Questions” where the?answers from each level are used in the next level with weighting to get the discrimination you want.??

So what does this do for you??Well first, it gives you time to get coffee.?Training takes longer than simple Pixel Classification; hours not seconds.?Assuming you will re-use the model you are training, this is not a big deal.?For your wait, you get much more robust results for systems with any complexity.???

Test Drive

Biodock announced its coming out of beta recently and Nurlybek encouraged me to sign up for an account.?I had a series of time lapse images acquired in phase contrast.?They are of macrophages grown on a SiMPore membrane (https://simpore.com/membrane-chips/) and the cells on top exhibit a bright phase image when in focus while the cells underneath the membrane exhibit a dark phase image at that same focus.?This is a result of how phase contrast works.?Here is a typical image from the series.?There is a confluent feeder layer of other cells on the membrane.?In the time lapse video the white cells move around until penetrating the membrane and turning black.

Images courtesy of James McGrath, University of Rochester

After registering for an account, it’s very straightforward to load your images into the application.

I have to say that just having the large image data set in a cloud account was convenient.??

Next I defined the two populations to teach, WhitePhase and BlackPhase cells.?I trained 11 images by tracing each cell type; this was more work than the Pixel Classification required but obviously the better the training, the better the results.?I think Biodock also offers this as a service.

Results

After 5 hours, the Model was finished and I could Analyze the unknown images with the Model.?I liked watching a virtual machine spin up on AWS to do my compute.?The results for this assay are simply the ratio of WhitePhaseCell to BlackPhaseCell which tells me what percentage of the cells migrated across the membrane.

Comparison

I had analyzed these images in a Pixel Classification system a few years ago and present a comparison of that result vs Biodock’s deep learning.?As you can see, the additional information in the neural network appears to be more rigorous as indicated by the two double white cells that were missed by the Pixel Classification approach.

Pixel Classification?????????????????????????????????????????????????Deep Learning

Training your data will definitely get you up close and personal with it as you decide where exactly your object’s boundary is.?I had some suggestions for bio vernacular and simplifying the UX but I’d encourage anyone looking for this kind of solution to check out the free demo and engage with Nurlybek and Michael for feedback on your experience.

Sign up here: