Digital reverse_engineering_of_executable_files.pdf

(3585 KB) Pobierz
´
UNIVERSITE LIBRE DE BRUXELLES
Facult´ des Sciences
e
D´partement d’Informatique
e
Digital reverse engineering
of executable files.
Obfuscation techniques
against patching.
Nikita Veshchikov
Promoteur :
Prof. Olivier Markowitch
M´moire pr´sent´ en vue de
e
e
e
l’obtention du grade de
Master en Sciences Informatiques
Ann´e acad´mique 2010 - 2011
e
e
Acknowledgments
First of all, I would like to thank my family for their patience.
I would like to thank my advisor - Olivier Markowitch for his advices and support.
I am grateful to everyone who helped me editing this paper, especially Tony Osborne
and Julia Zavyalova.
I would also like to thank persons who suggested interesting ideas and new sections
for this work - Liran Lerman and Markus Lindstr¨m.
o
A very special thanks goes to everyone who listened to my explanations about reverse
engineering, code obfuscation and error correction on numerous occasions. Thank you
for your patience!
Contents
1 Introduction
1.1 Goal and context
1.2 Organization . .
1.3 Contributions . .
1.4 Notations . . . .
1
1
1
2
2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
I
Understanding reverse engineering
3
4
4
5
5
8
8
9
10
11
12
12
12
2 History
2.1 Reasons for reverse engineering . . . . . . . . . . . . . . . . . . . . . . .
2.2 Reverse engineering in military . . . . . . . . . . . . . . . . . . . . . . .
2.3 Digital reverse engineering . . . . . . . . . . . . . . . . . . . . . . . . . .
3 Definitions of reverse engineering
3.1 Intuition behind reverse engineering . . . . . . . . . . . . . . . . . . . .
3.2 Definition of reverse engineering . . . . . . . . . . . . . . . . . . . . . . .
3.3 Definition of digital reverse engineering . . . . . . . . . . . . . . . . . . .
4 Legal aspects of digital reverse engineering
4.1 Intellectual property protection . . . . . . . . . . . . . . . . . . . . . . .
4.1.1 Copyright . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.2 Patent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
II Minimum knowledge required to perform digital reverse engi-
neering
13
5 Theoretical knowledge
5.1 Programming languages . . . . . . . . . . . .
5.1.1 Determining language used . . . . . .
5.2 Compilers . . . . . . . . . . . . . . . . . . . .
5.2.1 General changes in the code structure
5.2.2 Changes due to optimization . . . . .
5.3 Operating systems . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
14
14
15
15
16
17
20
22
22
24
6 Reversing tools
6.1 Hex editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Sandboxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
i
6.2.1 Virtual environments . . . . .
Disassemblers . . . . . . . . . . . . .
6.3.1 Decompilers . . . . . . . . . .
Debuggers . . . . . . . . . . . . . . .
Monitoring tools . . . . . . . . . . .
Dumping tools . . . . . . . . . . . .
Visual representations . . . . . . . .
Automated deobfuscators . . . . . .
Miscellaneous useful tools . . . . . .
6.9.1 File type recognition . . . . .
6.9.2 Strings and pattern searching
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
24
25
26
26
28
29
29
30
32
33
33
39
39
39
40
40
41
47
52
58
60
62
63
64
65
65
65
66
67
68
69
69
6.3
6.4
6.5
6.6
6.7
6.8
6.9
7 Code obfuscation
7.1 The definition . . . . . . . . . . . . . . . . . . . . . . . .
7.2 The problem . . . . . . . . . . . . . . . . . . . . . . . .
7.2.1 Why obfuscate? . . . . . . . . . . . . . . . . . . .
7.3 Anti-reversing techniques . . . . . . . . . . . . . . . . .
7.3.1 Packing techniques . . . . . . . . . . . . . . . . .
7.3.2 Control flow obfuscation . . . . . . . . . . . . . .
7.3.3 Detection of digital reverse engineering . . . . . .
7.3.4 Crashing and confusing reversing tools . . . . . .
7.3.5 Data transformations . . . . . . . . . . . . . . .
7.3.6 Hiding data . . . . . . . . . . . . . . . . . . . . .
7.3.7 Eliminating symbolic information . . . . . . . . .
7.3.8 Human reversers versus automated deobfuscators
7.4 Pushing the reversing problem out of the software world
7.4.1 Program as a service . . . . . . . . . . . . . . . .
7.4.2 Cryptoprocessors . . . . . . . . . . . . . . . . . .
7.4.3 Dongles . . . . . . . . . . . . . . . . . . . . . . .
7.4.4 Trusted computing . . . . . . . . . . . . . . . . .
7.4.5 Hardware protections summary . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
8 Applied reversing
8.1 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
III
Contribution
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
71
72
72
72
73
73
73
74
74
74
75
77
9 Anti-patching
9.1 A known problem . . . . . . . . . . . . . . . . . . . . .
9.2 Existing solutions . . . . . . . . . . . . . . . . . . . . .
9.2.1 Manual checking . . . . . . . . . . . . . . . . .
9.2.2 Automatic error detection . . . . . . . . . . . .
9.2.3 Check results of computations . . . . . . . . . .
9.2.4 Algorithm TPCA: Checker Network . . . . . .
9.3 Error detecting and error correcting codes . . . . . . .
9.3.1 The idea behind error detection and correction
9.3.2 Error detecting codes . . . . . . . . . . . . . .
9.3.3 Error correcting codes . . . . . . . . . . . . . .

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin