Crowd-sourced BioGames: managing the big data problem for next-generation lab-on-a-chip platforms - PubMed (original) (raw)

Crowd-sourced BioGames: managing the big data problem for next-generation lab-on-a-chip platforms

Sam Mavandadi et al. Lab Chip. 2012.

Abstract

We describe a crowd-sourcing based solution for handling large quantities of data that are created by e.g., emerging digital imaging and sensing devices, including next generation lab-on-a-chip platforms. We show that in cases where the diagnosis is a binary decision (e.g., positive vs. negative, or infected vs. uninfected), it is possible to make accurate diagnosis by crowd-sourcing the raw data (e.g., microscopic images of specimens/cells) using entertaining digital games (i.e., ) that are played on PCs, tablets or mobile phones. We report the results and the analysis of a large-scale public experiment toward diagnosis of malaria infected human red blood cells (RBCs), where binary responses from approximately 1000 untrained individuals from more than 60 different countries are combined together (corresponding to more than 1 million cell diagnoses), resulting in an accuracy level that is comparable to those of expert medical professionals. This platform holds promise toward cost-effective and accurate tele-pathology, improved training of medical personnel, and can also be used to manage the "Big Data" problem that is emerging through next generation digital lab-on-a-chip devices.

PubMed Disclaimer

Figures

Fig. 1

Fig. 1

Left: The designed game can be played on multiple platforms. The user is asked to kill or bank infected and healthy cells, respectively. Right: Geographic locations of gamers that have generated the diagnoses so far; each balloon indicates an individual gamer. A blue balloon indicates a gamer with more than 100 submitted cell diagnoses, and a red balloon indicates a gamer with less than 100 submitted cell diagnoses. Since its public launch in May 2012, we have had more than 2,150 gamers from 77 countries, who registered on our servers generating more than 1.5 million individual cell diagnoses.

Fig. 2

Fig. 2

Histogram of the overall accuracy levels of the 989 individual gamers who managed to label more than 100 cell images.

Fig. 3

Fig. 3

Distribution of the deviation from the true accuracies of those estimated using the control images; the mean is at −0.014 and the variance is 0.026. The green curve is that of a Normal distribution with the same variance as the deviations and a mean of 0.

Fig. 4

Fig. 4

The performance results obtained through combining responses of 989 gamers. The “effective crowd size” is the minimum number of times that every cell in the dataset has been diagnosed. Accuracy ≜ (TP + TN)/(TP + TN + FP + FN), Sensitivity ≜ TP/(TP + FN), Specificity ≜ TN/(TN + FP), PPV ≜ TP/(TP + FP), NPV ≜ TN/(TN + FN), FPR ≜ FP/(TN + FP), where TP, TN, FP, and FN correspond to the number of true positive, true negative, false positive, and false negative labels respectively.

References

    1. Nokia. Nokia 808 Pureview. 2012 http://europe.nokia.com/find-products/devices/nokia-808-pureview.
    1. Breslauer DN, Maamari RN, Switz NA, Lam WA, Fletcher DA. PLoS One. 2009;4:e6320. -PMC -PubMed
    1. Tseng D, Mudanyali O, Oztoprak C, Isikman SO, Sencan I, Yaglidere O, Ozcan A. Lab Chip. 2010;10:1787–1792. -PMC -PubMed
    1. Bishara W, Su T-W, Coskun AF, Ozcan A. Opt Express. 2010;18:11181–11191. -PMC -PubMed
    1. Miller AR, Davis GL, Oden ZM, Razavi MR, Fateh A, Ghazanfari M, Abdolrahimi F, Poorazar S, Sakhaie F, Olsen RJ, et al. PLoS One. 2010;5:e11890. -PMC -PubMed

Publication types

MeSH terms

LinkOut - more resources