Code Obfuscation against Static and Dynamic Reverse Engineering (original) (raw)

Abstract

The process of reverse engineering allows attackers to understand the behavior of software and extract proprietary algorithms and data structures (e.g. cryptographic keys) from it. Code obfuscation is frequently employed to mitigate this risk. However, while most of today’s obfuscation methods are targeted against static reverse engineering, where the attacker analyzes the code without actually executing it, they are still insecure against dynamic analysis techniques, where the behavior of the software is inspected at runtime. In this paper, we introduce a novel code obfuscation scheme that applies the concept of software diversification to the control flow graph of the software to enhance its complexity. Our approach aims at making dynamic reverse engineering considerably harder as the information an attacker can retrieve from the analysis of a single run of the program with a certain input, is useless for understanding the program behavior on other inputs. Based on a prototype implementation we show that our approach improves resistance against both static disassembling tools and dynamic reverse engineering at a reasonable performance penalty.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Anckaert, B., De Bosschere, K.: Diversity for Software Protection
    Google Scholar
  2. Anckaert, B., De Sutter, B., De Bosschere, K.: Software piracy prevention through diversity. In: Proceedings of the 4th ACM Workshop on Digital Rights Management, DRM 2004, pp. 63–71. ACM, New York (2004)
    Google Scholar
  3. Barak, B., Goldreich, O., Impagliazzo, R., Rudich, S., Sahai, A., Vadhan, S.P., Yang, K.: On the (Im)possibility of obfuscating programs. In: Kilian, J. (ed.) CRYPTO 2001. LNCS, vol. 2139, pp. 1–18. Springer, Heidelberg (2001)
    Chapter Google Scholar
  4. Billet, O., Gilbert, H., Ech-Chatbi, C.: Cryptanalysis of a white box AES implementation. In: Handschuh, H., Hasan, M.A. (eds.) SAC 2004. LNCS, vol. 3357, pp. 227–240. Springer, Heidelberg (2005)
    Chapter Google Scholar
  5. Cappaert, J., Preneel, B.: A general model for hiding control flow. In: Proceedings of the Tenth Annual ACM Workshop on Digital Rights Management. ACM, New York (2010)
    Google Scholar
  6. Chidamber, S., Kemerer, C.: A metrics suite for object oriented design. IEEE Transactions on Software Engineering 20(6) (2002)
    Google Scholar
  7. Chow, S., Eisen, P., Johnson, H., van Oorschot, P.: A white-box DES implementation for DRM applications. In: Digital Rights Management, pp. 1–15 (2003)
    Google Scholar
  8. Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations (1997)
    Google Scholar
  9. De Sutter, B., Anckaert, B., Geiregat, J., Chanet, D., De Bosschere, K.: Instruction set limitation in support of software diversity. In: Lee, P.J., Cheon, J.H. (eds.) ICISC 2008. LNCS, vol. 5461, pp. 152–165. Springer, Heidelberg (2009)
    Chapter Google Scholar
  10. Franz, M.: E unibus pluram: massive-scale software diversity as a defense mechanism. In: Proceedings of the 2010 Workshop on New Security Paradigms. ACM, New York (2010)
    Google Scholar
  11. Halstead, M.: Elements of software science. Elsevier, New York (1977)
    MATH Google Scholar
  12. Harrison, W., Magel, K.: A complexity measure based on nesting level. ACM Sigplan Notices 16(3) (1981)
    Google Scholar
  13. Henry, S., Kafura, D.: Software Structure Metrics Based on Information Flow. IEEE Transactions on Software Engineering 7(5), 510–518 (1981)
    Article Google Scholar
  14. Jacob, M., Boneh, D., Felten, E.: Attacking an obfuscated cipher by injecting faults. In: Digital Rights Management, pp. 16–31 (2003)
    Google Scholar
  15. Kinder, J., Veith, H.: Jakstab: A static analysis platform for binaries. In: Gupta, A., Malik, S. (eds.) CAV 2008. LNCS, vol. 5123, pp. 423–427. Springer, Heidelberg (2008)
    Chapter Google Scholar
  16. Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: Proceedings of the 10th ACM Conference on Computer and Communications Security. ACM, New York (2003)
    Google Scholar
  17. Lynn, B., Prabhakaran, M., Sahai, A.: Positive results and techniques for obfuscation. In: Cachin, C., Camenisch, J.L. (eds.) EUROCRYPT 2004. LNCS, vol. 3027, pp. 20–39. Springer, Heidelberg (2004)
    Chapter Google Scholar
  18. Madou, M., Anckaert, B., De Sutter, B., De Bosschere, K.: Hybrid static-dynamic attacks against software protection mechanisms. In: Proceedings of the 5th ACM Workshop on Digital Rights Management, pp. 75–82. ACM, New York (2005)
    Chapter Google Scholar
  19. McCabe, T.: A complexity measure. IEEE Transactions on Software Engineering (1976)
    Google Scholar
  20. Michiels, W., Gorissen, P.: Mechanism for software tamper resistance: an application of white-box cryptography. In: Proceedings of the 2007 ACM Workshop on Digital Rights Management, pp. 82–89. ACM, New York (2007)
    Chapter Google Scholar
  21. Munson Taghi, M., John, C.: Measurement of data structure complexity. Journal of Systems and Software 20(3), 217–225 (1993)
    Article Google Scholar
  22. Oviedo, E.: Control flow, data flow and program complexity. McGraw-Hill, Inc., New York (1993)
    Google Scholar
  23. Wee, H.: On obfuscating point functions. In: Proceedings of the Thirty-Seventh Annual ACM Symposium on Theory of Computing. ACM, New York (2005)
    Google Scholar
  24. Wyseur, B., Michiels, W., Gorissen, P., Preneel, B.: Cryptanalysis of white-box DES implementations with arbitrary external encodings. In: Adams, C., Miri, A., Wiener, M. (eds.) SAC 2007. LNCS, vol. 4876, pp. 264–277. Springer, Heidelberg (2007)
    Chapter Google Scholar

Download references

Author information

Authors and Affiliations

  1. Vienna University of Technology, Austria
    Sebastian Schrittwieser
  2. Darmstadt University of Technology, Germany
    Stefan Katzenbeisser

Authors

  1. Sebastian Schrittwieser
    You can also search for this author inPubMed Google Scholar
  2. Stefan Katzenbeisser
    You can also search for this author inPubMed Google Scholar

Editor information

Editors and Affiliations

  1. Digimarc Corporation, 9405 Gemini Drive, 97008, Beaverton, OR, USA
    Tomáš Filler
  2. Faculty of Electrical Engineering, Department of Cybernetics, Czech Technical University, Karlovo namesti 13, 121 35, Prague 2, Czech Republic
    Tomáš Pevný
  3. Department of Electrical and Computer Engineering, T. J. Watson School, SUNY Binghamton, 13902, Binghamton, NY, USA
    Scott Craver
  4. Department of Computer Science, University of Oxford, Wolfson Building, Parks Road, OX1 3QD, Oxford, UK
    Andrew Ker

Rights and permissions

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Schrittwieser, S., Katzenbeisser, S. (2011). Code Obfuscation against Static and Dynamic Reverse Engineering. In: Filler, T., Pevný, T., Craver, S., Ker, A. (eds) Information Hiding. IH 2011. Lecture Notes in Computer Science, vol 6958. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24178-9\_19

Download citation

Publish with us