Automated Expert Modeling for Automated Student Evaluation (original) (raw)

Abstract

This paper presents automated expert modeling for automated student evaluation, or AEMASE (pronounced “amaze”). This technique grades students by comparing their actions to a model of expert behavior. The expert model is constructed with machine learning techniques, avoiding the costly and time-consuming process of manual knowledge elicitation and expert system implementation. A brief summary of after action review (AAR) and intelligent tutoring systems (ITS) provides background for a prototype AAR application with a learning expert model. A validation experiment confirms that the prototype accurately grades student behavior on a tactical aircraft maneuver application. Finally, several topics for further research are proposed.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Centre, C.A.L.L.: The after action review process, learning more from our training. Army Lessons Learned Centre Dispatches 6(3) (1999)
    Google Scholar
  2. DeGrosky, M.T.: Improving after action review (aar) practice. In: Eighth International Wildland Fire Safety Summit. (2005)
    Google Scholar
  3. Meliza, L.: After action review: Methods and tools (2004) (retrieved on December 2, 2005), http://www.hqda.army.mil/ari/news/factsheets.shtml
  4. Jensen, R., Nolan, M., Chen, D.Y.: Automatic causal explanation analysis for combined arms training aar. In: Proceedings of the Industry/Interservice, Training, Simulation and Education Conference (I/ITSEC) (2005)
    Google Scholar
  5. Cohen, P.A., Kulik, J.A., Kulik, C.C.: Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal 19, 237–248 (1982)
    Google Scholar
  6. Corbett, A.T.: Cognitive computer tutors: Solving the two- sigma problem. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS, vol. 2109, pp. 137–147. Springer, Heidelberg (2001)
    Chapter Google Scholar
  7. Burns, H., Capps, C.: Foundations of intelligent tutoring systems: An introduction. In: Polson, M.C., Richardson, J.J. (eds.) Foundations of Intelligent Tutoring Systems, pp. 1–18. Lawrence Erlbaum Associates Publishers, Mahwah (1988)
    Google Scholar
  8. Murray, T.: Authoring intelligent tutoring systems: An analysis of the state of the art. International Journal of Artificial Intelligence in Education 10, 98–129 (1999)
    Google Scholar
  9. Welch, G., Foxlin, E.: Motion tracking: no silver bullet, but a respectable arsenal. IEEE Computer Graphics and Applications 22(6), 24–38 (2002)
    Article Google Scholar
  10. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1992)
    Google Scholar
  11. Srinivasa, N., Ahuja, N.: A topological and temporal correlator network for spatiotemporal pattern learning, recognition and recall. IEEE Transactions on Neural Networks 10(2), 356–371 (1999)
    Article Google Scholar
  12. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001)
    MATH Google Scholar
  13. Sutton, R., Barto, A.: 3.5 The Markov Property. In: Reinforcement Learning, pp. 61–65. MIT Press, Cambridge (1998)
    Google Scholar
  14. Kornecki, A., Hilburn, T., Diefenbach, T., Towhidnajad, M.: Intelligent tutoring issues for atc training system. IEEE Transactions on Control Systems Technology 1(3), 204–211 (1993)
    Article Google Scholar
  15. Hess, S.M., MacMillan, J., Sefarty, D., Elliot, L.: From cognitive task analysis to simulation: Developing a synthetic team task for awacs weapons directors, Air Force Research Laboratory (1999)
    Google Scholar
  16. Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, pp. 1137–1145 (1995)
    Google Scholar
  17. Zweig, C.: Receiver-operating characteristic (roc) plots: a fundamental evaluation tool in clinical medicine. Clin. Chem. 39(4), 561–577 (1993)
    Google Scholar
  18. Ali, K., Langley, P., Maloof, M., Sage, S., Binford, T.: Improving rooftop detection with interactive visual learning. In: Proceedings of the Image Understanding Workshop, pp. 479–492. Morgan Kaufmann, San Francisco (1998)
    Google Scholar

Download references

Author information

Authors and Affiliations

  1. Sandia National Labs MS 1188, PO Box 5800, Albuquerque, NM, 87185, USA
    Robert G. Abbott

Editor information

Editors and Affiliations

  1. JAIST, 1-1, Asahi-dai, Nomi, 923-1292, Ishikawa, Japan
    Mitsuru Ikeda
  2. Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA, USA
    Kevin D. Ashley
  3. Graduate Institute of Network Learning Technology, National Central University, 300, Jhongda Rd., 32001, Jhongli City,Taoyuan County, Taiwan
    Tak-Wai Chan

Rights and permissions

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Abbott, R.G. (2006). Automated Expert Modeling for Automated Student Evaluation. In: Ikeda, M., Ashley, K.D., Chan, TW. (eds) Intelligent Tutoring Systems. ITS 2006. Lecture Notes in Computer Science, vol 4053. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11774303\_1

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us