Prior Knowledge Improves Decoding of Finger Flexion from Electrocorticographic Signals (original) (raw)

Abstract

Brain-computer interfaces (BCIs) use brain signals to convey a user's intent. Some BCI approaches begin by decoding kinematic parameters of movements from brain signals, and then proceed to using these signals, in absence of movements, to allow a user to control an output. Recent results have shown that electrocorticographic (ECoG) recordings from the surface of the brain in humans can give information about kinematic parameters (e.g., hand velocity or finger flexion).The decoding approaches in these studies usually employed classical classification/regression algorithms that derive a linear mapping between brain signals and outputs. However, they typically only incorporate little prior information about the target movement parameter. In this paper, we incorporate prior knowledge using a Bayesian decoding method, and use it to decode finger flexion from ECoG signals. Specifically, we exploit the constraints that govern finger flexion and incorporate these constraints in the construction, structure, and the probabilistic functions of the prior model of a switched non-parametric dynamic system (SNDS). Given a measurement model resulting from a traditional linear regression method, we decoded finger flexion using posterior estimation that combined the prior and measurement models. Our results show that the application of the Bayesian decoding model, which incorporates prior knowledge, improves decoding performance compared to the application of a linear regression model, which does not incorporate prior knowledge. Thus, the results presented in this paper may ultimately lead to neurally controlled hand prostheses with full fine-grained finger articulation.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (60)

  1. Acharya, S., Fifer, M. S., Benz, H. L., Crone, N. E., and Thakor, N. V. (2010). Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand. J. Neural Eng. 7, 046002.
  2. Azzouzi, M., and Nabney, I. T. (1999). "Modelling financial time series with switching state space models," in Proceedings on IEEE/IAFE 1999 Con- ference on Computational Intelligence for Financial Engineering, Port Jef- ferson, NY: Institute of Electrical and Electronics Engineers (IEEE), 240C249.
  3. Bashashati, A., Fatourechi, M., Ward, R. K., and Birch, G. E. (2007). A sur- vey of signal processing algorithms in brain-computer interfaces based on electrical brain signals. J. Neural Eng. 4, R32-R57.
  4. Bradberry, T. J., Gentili, R. J., and Contreras-Vidal, J. L. (2010). Recon- structing three-dimensional hand movements from noninvasive elec- troencephalographic signals. J. Neu- rosci. 30, 3432-3437.
  5. Bradberry, T. J., Gentili, R. J., and Contreras-Vidal, J. L. (2011). Fast attainment of computer cursor con- trol with noninvasively acquired brain signals. J. Neural Eng. 8, 036010.
  6. Carmena, J. M., Lebedev, M. A., Henriquez, C. S., and Nicolelis, M. A. L. (2005). Stable ensem- ble performance with single-neuron variability during reaching move- ments in primates. J. Neurosci. 25, 10712-10716.
  7. Chao, Z. C., Nagasaka, Y., and Fujii, N. (2010). Long-term asynchro- nous decoding of arm motion using electrocorticographic signals in monkeys. Front. Neuroeng. 3:3. doi:10.3389/fneng.2010.00003
  8. Chapin, J. K., Moxon, K. A., Markowitz, R. S., and Nicolelis, M. A. (1999). Real-time control of a robot arm using simultaneously recorded neu- rons in the motor cortex. Nat. Neu- rosci. 2, 664-670.
  9. Cowell, R. G., Dawid, P. A., Lauritzen, S. L., and Spiegelhalter, D. J. (2003). Probabilistic Networks and Expert Systems (Information Science and Statistics). New York: Springer.
  10. Droppo, J., and Acero, A. (2004). "Noise robust speech recognition with a switching linear dynamic model," in Acoustics, Speech, and Signal Process- ing, 2004. Proceedings. (ICASSP'04). IEEE International Conference on, Vol. 1, Montreal, I-953-6.
  11. Ferguson, J. (1980). "Variable dura- tion models for speech," in Proc. of the Symposium on the Application of Hidden Markov Models to Text and Speech, Princeton, NJ, 143-179.
  12. Frank, E., Trigg, L., Holmes, G., Witten, I. H., and Aha, W. (2000). "Naive Bayes for regression," in Machine Learning, Boston: Kluwer Academic Publishers, 5-26.
  13. Friedman, N., Goldszmidt, M., and Lee, T. J. (1998). "Bayesian net- work classification with continuous attributes: getting the best of both discretization and parametric fit- ting," in Proceedings of the Interna- tional Conference on Machine Learn- ing (ICML) (San Francisco, CA: Morgan Kaufmann), 179-187.
  14. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. (2009). The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11, 10-18.
  15. Hatsopoulos, N., Joshi, J., and O'Leary, J. G. (2004). Decoding continuous and discrete motor behaviors using motor and premotor cortical ensem- bles. J. Neurophysiol. 92, 1165-1174.
  16. Isard, M., and Blake, A. (1998). Conden- sation -conditional density prop- agation for visual tracking. Int. J. Comput. Vis. 29, 5-28.
  17. Kim, K. H., Kim, S. S., and Kim, S. J. (2006). Superiority of nonlin- ear mapping in decoding multiple single-unit neuronal spike trains: a simulation study. J. Neurosci. Meth- ods 150, 202-211.
  18. Kim, S.-P., Rao, Y. N., Erdogmus, D., Sanchez, J. C., Nicolelis, M. A. L., and Principe, J. C. (2005). Deter- mining patterns in neural activ- ity for reaching movements using nonnegative matrix factorization. EURASIP J. Appl. Signal Process. 2005, 3113-3121.
  19. Koyama, S., Chase, S. M., Whitford, A. S., Velliste, M., Schwartz, A. B., and Kass, R. E. (2010). Comparison of brain-computer interface decoding algorithms in open-loop and closed- loop control. J. Comput. Neurosci. 29, 73-87.
  20. Kubánek, J., Miller, K. J., Ojemann, J. G., Wolpaw, J. R., and Schalk, G. (2009). Decoding flexion of individual fin- gers using electrocorticographic sig- nals in humans. J. Neural Eng. 6, 066001.
  21. Lauritzen, S. L. (1992). Propagation of probabilities, means, and variances in mixed graphical association mod- els. J. Am. Stat. Assoc. 87, 1098-1108.
  22. Lebedev, M. A., Carmena, J. M., O'Doherty, J. E., Zacksenhouse, M., Henriquez, C. S., Principe, J. C., and Nicolelis, M. A. L. (2005). Corti- cal ensemble adaptation to represent velocity of an artificial actuator con- trolled by a brain-machine interface. J. Neurosci. 25, 4681-4693.
  23. Levinson, S. (1986). "Continuously variable duration hidden Markov models for speech analysis," in Acoustics, Speech, and Signal Processing, IEEE International Con- ference on ICASSP'86, Vol. 11, Tokyo, 1241-1244.
  24. Loader, C. R. (1999). Bandwidth selec- tion: classical or plug-in? Ann. Stat. 27, 415-438.
  25. Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., and Arnaldi, B. (2007). A review of classification algorithms for EEG-based brain- computer interfaces. J. Neural Eng. 4, 1-1.
  26. Marple, S. L. (1986). Digital Spectral Analysis: With Applications. Upper Saddle River, NJ: Prentice-Hall, Inc.
  27. Marron, J. S., and Wand, M. P. (1992). Exact mean integrated squared error. Ann. Stat. 20, 712-736.
  28. Maskell, S., and Gordon, N. (2001). A tutorial on particle filters for on-line nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50, 174-188.
  29. McFarland, D. J., Krusienski, D. J., Sarnacki, W. A., and Wolpaw, J. R. (2008). Emulation of computer mouse control with a noninvasive brain-computer interface. J. Neural Eng. 5, 101-110.
  30. McFarland, D. J., Sarnacki, W. A., and Wolpaw, J. R. (2010). Elec- troencephalographic EEG control of three-dimensional movement. J. Neural Eng. 7, 036007.
  31. Muller, K.-R., Anderson, C., and Birch, G. (2003). Linear and nonlinear methods for brain-computer inter- faces. IEEE Trans. Neural Syst. Reha- bil. Eng. 11, 165-169.
  32. Mulliken, G. H., Musallam, S., and Andersen, R. A. (2008). Decoding trajectories from posterior parietal cortex ensembles. J. Neurosci. 28, 12913-12926.
  33. Oh, S. M., Rehg, J., Balch, T., and Del- laert, F. (2005). "Learning and infer- ence in parametric switching lin- ear dynamic systems," in Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, Vol. 2, Beijing, 1161-1168.
  34. Oh, S. M., Rehg, J. M., Balch, T., and Del- laert, F. (2008). Learning and infer- ring motion patterns using para- metric segmental switching linear dynamic systems. Int. J. Comput. Vis. 77, 103-124.
  35. Ostendorf, M., Digalakis, V., and Kim- ball, O. (1996). From HMM's to segment models: a unified view of stochastic modeling for speech recognition. IEEE Trans. Speech Audio Process. 4, 360-378.
  36. Pavlovic, V., Rehg, J. M., and Mac- Cormick, J. (2001). Learning switch- ing linear models of human motion. Adv. Neural Inf. Process Syst. 2000, 981-987.
  37. Presacco, A., Goodman, R., Forrester, L. W., and Contreras-Vidal, J. L. (2011). Neural decoding of treadmill walking from non- invasive, electroencephalographic (EEG) signals. J. Neurophysiol. doi: 10.1152/jn.00104.2011. [Epub ahead of print].
  38. Rasmussen, C. E. (1996). Evaluation of Gaussian Processes and Other Meth- ods for Non-Linear Regression. Tech- nical Report, University of Toronto, Toronto.
  39. Rosti, A. V. I., and Gales, M. (2004). "Rao-blackwellised Gibbs sampling for switching linear dynamical sys- tems," in IEEE International Confer- ence on Acoustics, Speech, and Signal Processing (ICASSP 2004), Montreal, 809-812.
  40. Russell, M., and Moore, R. (1985). "Explicit modelling of state occu- pancy in hidden Markov models for automatic speech recognition," in Acoustics, Speech, and Signal Process- ing, IEEE International Conference on ICASSP'85, Vol. 10, Tampa, FL, 5-8.
  41. Sanchez, J., Erdogmus, D., Rao, Y., Principe, J., Nicolelis, M., and Wess- berg, J. (2003). "Learning the con- tributions of the motor, premo- tor, and posterior parietal cortices for hand trajectory reconstruction in a brain machine interface," in Neural Engineering, 2003. Confer- ence Proceedings. First International IEEE EMBS Conference on, Cancun, 59-62.
  42. Sanchez, J. C., Erdogmus, D., and Principe, J. C. (2002). "Comparison between nonlinear mappings and linear state estimation to model the relation from motor cortical neu- ronal firing to hand movements," in Proceedings of SAB Workshop on Motor Control in Humans and Robots: On the Interplay of Real Brains and Artificial Devices, Edin- burgh, 59-65.
  43. Schalk, G. (2010). Can electrocor- ticography (ECoG) support robust and powerful brain-computer interfaces? Front. Neuroeng. 3:9. doi:10.3389/fneng.2010.00009
  44. Schalk, G., Kubánek, J., Miller, K. J., Anderson, N. R., Leuthardt, E. C., Ojemann, J. G., Limbrick, D., Moran, D., Gerhardt, L. A., and Wolpaw, J. R. (2007). Decoding two-dimensional movement trajectories using electro- corticographic signals in humans. J. Neural Eng. 4, 264-275.
  45. Schalk, G., McFarland, D. J., Hinter- berger, T., Birbaumer, N., and Wol- paw, J. R. (2004). BCI2000: a general- purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51, 1034-1043.
  46. Schalk, G., and Mellinger, J. (2010). A Practical Guide to Brain-Computer Interfacing with BCI2000. New York, NY: Springer-Verlag.
  47. Schwartz, A. B., Taylor, D. M., and Tillery, S. I. (2001). Extraction algo- rithms for cortical control of arm prosthetics. Curr. Opin. Neurobiol. 11, 701-707.
  48. Snelson, E., and Ghahramani, Z. (2006). Sparse Gaussian processes using pseudo-inputs. Adv. Neural Inf. Process. Syst. 18, 1257-1264.
  49. Taylor, D. M., Tillery, S. I., and Schwartz, A. B. (2002). Direct cortical con- trol of 3D neuroprosthetic devices. Science 296, 1829-1832.
  50. Taylor, G., Sigal, L., Fleet, D., and Hin- ton, G. (2010). "Dynamical binary latent variable models for 3d human pose tracking," in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, San Fran- cisco, CA, 631-638.
  51. Taylor, G. W., Hinton, G. E., and Roweis, S. (2006). Modeling human motion using binary latent vari- ables. Adv. Neural Inf. Process. Syst. 19, 1345.
  52. Tong, Y., and Ji, Q. (2008). "Learning Bayesian networks with qualitative constraints," in CVPR 2008. IEEE Conference on, Anchorage, 1-8.
  53. Tong, Y., Liao, W., and Ji, Q. (2007). Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1683-1699.
  54. Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., and Schwartz, A. B. (2008). Cortical control of a pros- thetic arm for self-feeding. Nature 453, 1098-1101.
  55. Viola, P., and Jones, M. (2004). Robust real-time face detection. Int. J. Com- put. Vis. 57, 137-154.
  56. Wang, Z., Ji, Q., Miller, K., and Schalk, G. (2010). "Decoding finger flexion from electrocorticographic signals using a sparse Gaussian process," in Proceedings of the 2010 20th International Conference on Pat- tern Recognition, ICPR'10 (Washing- ton, DC: IEEE Computer Society), 3756-3759.
  57. Wolpaw, J. R. (2007). "Brain- computer interfaces (BCIs) for communication and control," in Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, Assets'07 (New York, NY: ACM), 1-2.
  58. Wolpaw, J. R., and McFarland, D. J. (2004). Control of a two- dimensional movement signal by a noninvasive brain-computer inter- face in humans. Proc. Natl. Acad. Sci. U.S.A. 101, 17849-17854.
  59. Yao, B.,Walther, D., Beck, D., and Fei-Fei, L. (2009). "Hierarchical mixture of classification experts uncovers inter- actions between brain regions," in Annual Conference on Neural Infor- mation Processing Systems (NIPS) (Vancouver: NIPS).
  60. Zoeter, O., and Heskes, T. (2003). Hierarchical visualization of time- series data using switching lin- ear dynamical systems.IEEE Trans. Pattern Anal. Machine Intell. 25, 1202-1214.