Perhaps about Base64 Encoding??? XOR Encoding? Not that either?
Congratulations🥳, you are about to learn about 3 core ways malware use data encoding to their advantage!
In the previous article https://8ksec.io/dissecting-windows-malware-series-process-injections-part-2/, we introduced the mechanism of Process Injection that malware use to achieve Stealth and Evasion. We saw direct implementation of:
Process Injection
Process Hollowing
And the use of other interesting Stealth associated mechanism.
What's In It For Me❓
Attackers do this to make it harder for Malware Analysts to fully understand what the malware does and how it operates.
An interesting footnote – in many cases, attackers prefer using simpler encryption methods like Base64 Encoding or XOR Encoding because they are:
Space-efficient – they can be used in space-constrained environments and even in various Shellcodes.
Less noticeable to the eye – they do not require calling functions from the cryptographic API of a specific encryption, thus are more “quiet” in the eyes of the Analyst.
Efficient – because these are relatively compact encryption methods that don’t require a large number of code lines and/or calling external API functions, they less affect the performance of the malware during execution.
A Brief Overview of Encryption Methods📚
XOR Encoding
In its simplest form, it uses a character (byte) that serves as a key, for example: 0x3C.
If we perform a XOR operation with the key on each character in our string, we will get a completely different representation of the existing string, meaningless in human eyes.
For example, the address: http[:]//badsite[.]com with the XOR operation, using the key – 0x2d on each character will translate to – VJJN…_ZMWJ[.]QS.
Overall, it doesn’t sound so terrible, right?
The issue becomes complicated when playing around with this method😈
For instance, given an ASCII string representing an IP address, a domain, or even names of functions themselves used by the malware – we could use the content of the string itself for encryption.
We could start the XOR operation at the beginning of the string, at the end or literally at any position we choose – On a randomly chosen character.
The output character we get from the XOR operation will serve us as a “key” to perform the XOR operation with the next character in the string and so on…
Base64 Encoding
This method essentially converts binary data representation into a set of 64 characters.
There are several schemes for the different types of Base64 Encoding out there, but most contain the characters: [0-9,a-z,A-Z] and two additional characters, usually + and =, used for padding the string length to fit the format.
The conversion is done as follows: every 3 bytes of binary data are converted to 4 bytes of Base64 data.
Just to tickle your ears, the string ATT is represented by the following 3 bytes in hexadecimal: 0x41 0x54 0x54 (each character in ASCII is 8 bits which is one byte which is two characters in hex).
Look for the indexes – 16, 21, 17, and 20, we will get the characters: QVRT accordingly.
Also here, the attacker has a wide range of possibilities for implementing Base64 Encoding.
He could, for instance, change the indexing table to have a different order, which will affect all the indexing and how the conversion is done.
Use of Cryptography API Functions
Most encryption algorithms under this category are based on the fact that:
Performing brute-forcing in an attempt to decrypt the source, given the complexity of the key and the encryption, simply isn’t possible (it would take decades – yes, decades…).
The way the algorithm works is publicly accessible for inspection, but without the key, it’s impossible to decrypt the source.
As mentioned earlier, using encryption algorithms in this category is usually easier to identify during the analysis of malware. This is reflected in the use of informative constants, aka ‘Consts’, importing libraries and functions, and others…
Let’s briefly go over the symmetric encryption algorithm – AES:
It is a symmetric algorithm meaning the same key is used for both encryption and decryption – we’ll remember this when analyzing the malware, aiming to find the key for performing the decoding.
The algorithm essentially takes 128 bits of content and turns it into 128 bits of cipher text (encrypted and unreadable content).
The key used for encryption can be in sizes of 128 bits, 192 bits, or 256 bits. Each size indicates the number of rounds that will be performed during the algorithm on the content we wish to encrypt (10 rounds, 12 rounds, or 14 rounds accordingly).
The algorithm essentially takes 16 bytes (128 bits), arranges them in a 4×4 grid, and performs operations of:
XOR -> Byte Substitution -> Shifting Rows -> Mixing Columns -> Adding Round Key
In addition, a mechanism called ‘Key scheduling’ is also used in order not to use the same key in every round, but we won’t get into that right now.
Generally speaking, most modern encryption algorithms are based on Substitution Box and Permutation Box, where the previously mentioned Substitution Bytes stage is actually a Substitution Box.
For the Math Lovers Among Us➕
As a footnote for the math enthusiasts, an S-Box is a non-linear transformation of the form:
fi in the field, receives n bits as input and returns fi for each xi in the following manner (meaning m bits).
f1,f2,…,fm such: fi
In our case, since we talked about AES, the n parameter represents 128 bits. The whole mapping process we presented is actually one round (as mentioned earlier) and according to the algorithm, a number of such rounds will be performed.
Here comes the importance of the secret key mentioned earlier.
The key is divided into several “chunks” where each chunk essentially undergoes XOR with the output we received after each round we did – if we hadn’t done that, we could simply decrypt the algorithm by performing the operations we did in reverse order (given access to the sequence of operations themselves and without needing the key) – this stage is called key scheduling.
The Data Decoding Checklist📈
We’ll see that it is actually based on a custom encoding approach – it uses XOR encryption, Base64, and also AES encryption in the unique way it chose to implement.
In analyzing encrypted malware, we need to remember that we must identify the following three key principles:
Identify the type of encoding the malware uses (if at all), among them: AES, Base64, XOR Encoding, and others…
In case there is use of encoding, what are the functions aimed at performing the encoding.
Let's Start Reversing😎
CDEFGHIJKLMNOPQRSTUVWXYZABcdefghijklmnopqrstuvwxyzab0123456789+/
This is a Base64 index table (note the change in the order of characters in the index table as mentioned earlier) used for some sort of encoding, the malware will perform.
In addition, to a number of interesting functions: WriteFile, CreateThread, we can see several error message strings: Data not multiple of block size, Incorrect key length, and Empty key.
We can guess that it refers to some kind of encryption where the size of the block and the length of the key are important.
Subsequently, we see a cmd.exe window opening, simply waiting – we can infer that the malware might be trying to execute a reverse shell, where commands are actually sent to the malware on the previously seen port, for local execution at the endpoint.
A Little Side Note
We can see that the malware tries to run commands like dir and we can see a lot of encrypted data received through the created connection.
Using PEiD, specifically with the KANAL Plugin, we can see memory addresses identified with constants and high entropy levels (in a word, entropy indicates memory addresses where the data is more random than usual, pointing to some form of implemented encryption).
Continue to analyze the addresses in memory characterized by high entropy, we notice that at address 0x004120A4, we find the Base64 index table we saw before.
We can conclude that the malware uses Base64 encoding with a custom index table and AES encryption.
Running a search for all the XOR
xor eax,eax.
We’ll perform cross-reference to the addresses remaining after the filtering with the addresses we found before in the entropy scan and discover that there are 6 main functions containing these commands.
We’ll need to understand which function calls, whether they call one another (spoiler: they do) and generally understand the big picture of what is happening here.
Based on the fact that we expect to see some reference to AES encryption, we will try to understand how the XOR functions we found are related to this.
After some cross-referencing to the XOR functions, we discover that the main function is the one that calls the function whose name we changed to be xor_func1
Since we anticipate references to AES encryption, the observant will notice that the offset of the string, which the function receives as a parameter, not only resembles a key but also comprises 16 characters.
These 16 characters, corresponding to 16 bytes (with each ASCII character representing one byte, totaling 128 bits), immediately suggest that we are dealing with AES-128 encryption, which is based on a 16-byte-long key.
Return to the Bigger Picture🌄
xor_func1 which receives a number of parameters, among them a string that looks like an AES key.
After that, we saw API functions calls to WSAStartup and WSASocketA whose purpose is to perform initialization for a network connection.
Several additional calls are made whose purposes are:
1. Performing DNS resolving for the domain passed as a parameter
2. Converting the port (displayed as 8910) from little endian to big endian for the network convention during communication
3. Performing connect to start the communication
4. And calling another function *sub_4015B7*.
Starting to analyze the function *sub_4015B7*, we discover several interesting things:
5. The malware creates a process of *cmd.exe*:
We will discover that it is referenced in the function sub_40103F which is called from sub_401082 and this in turn contains a nested loop.
Finally, we discover that the function sub_401082
Importance of Understanding the Chronological Order🔎
There is a function that performs the decoding stage based on the Base64 index table called right after the call to CreateProcess that creates cmd.exe. Then, a socket connection is created with the calls (WSAStartup, WSASocketA, and additional functions from the library Ws2_32.dll) to create communication to the malicious domain.
A pipe is created that will pass the input/output from the created socket, to cmd.exe.
Analysis of the calls related to AES encryption reveals that the function responsible for performing the AES encryption is passed as a parameter to the second thread we saw, and this happens after the discussed process – cmd.exe has already started running.
Finally, we witness a call to the API function WriteFile
When dealing with data encoding mechanisms, understanding the chronological order of the various encoding and decoding components is crucial. That’s the key to successfully decoding the encoded data!
What Have We Seen😎
The malware we saw created a reverse shell, where the data it sends back to the C2 server undergoes encryption.
In terms of the chronological order and the big picture, all the commands sent from the C2 to the reverse shell are sent encoded using Base64 and undergo decoding using the custom index table we found.
Diving deeper, performing analysis and cross-referencing, we found out that all the responses sent back to the C2 server undergo AES encryption using the key:
ijklmnopqrstuvwx
right before being sent back.
We reviewed the way AES encryption works: how does it work, what is it based on, a bit of mathematical explanation, and how it is displayed in the assembly code of the malware sample.
Most importantly , we saw what the workflow we want to implement when investigating malware that use encryptions of various kinds:
We want to identify what kind of encryption/encoding the malware is using.
Discover which functions perform the encryption/encoding and which additional memory addresses are related to the process (for example, strings in the data segment).
What Haven't We Seen (And Probably Will)👀
Last but not least, we did not manage to present here the *Decoding stage itself*.
The decoding can be performed in several ways, the main ones are:
1. Manual Decoding – in this approach, we identify the functions that perform the decoding, run the malware in a debugger, and see the strings decoded in real-time. If we fail to identify these functions, or if the malware does not perform decoding at any stage during the run – we’re out of luck…
2. Decoding Using Coding – we can write a script that will perform the decoding for us.
With that being said, we’ll need to give it all the necessary information for performing the decoding.
For example, if it’s Base64, we will need to give it the index table, if it’s AES or DES, we will need to give it the password (which we will have to extract from the assembly code) and the mode of encryption (for instance, AES contains several modes: CBC or EBC, etc…) – in other words, there’s quite a bit of data that we first need to extract from the assembly code, so that our code can run and perform what we command it to.
3. Instrumentation – Once we identify the functions responsible for performing the decoding and encoding and the parameters they receive, we can change the flow of the program’s execution so that it will perform the decoding for us – with the data we provide it.
We achieve this using plugins we write for OllyDbg, ImmDbg,x32/64Dbg that will perform modifications to the values of the registers and to memory addresses in order to perform the decoding for us.
This requires a fairly deep understanding of what happens behind the scenes, to allocate buffers from which we will write and read, changing the value of the EIP register (and other registers accordingly), and more…
It’s not the easiest to implement but once you understand it deeply, it can be applied to a significant number of malware families out there – definitely a skill worth acquiring for the long run.
A Side Note📃
We can implement this partially in the debugger itself.
In cases the encryption used by the malware is reversible (the same algorithm for encryption and decoding) we can simply give the function that performs the encoding, the encoded data and receive it decoded at the end of its run…
What's Next😏
Malware have different purposes, but all of them are made to infiltrate data in one way or another.
Thus, we need to be able to research and analyze network traffic.
BUT, before we dive to the world of Network Traffic Analysis, we’ll do a short detour.
As you sure have noticed, reverse engineering malware requires some understanding of Low-Level knowledge and OS Internals.
Since this aspect is so important, we’ll review the difference between the main CPU architecture types – RISC and CISC. How does they differ from one another, and how does it affect the way malware are written on each architecture.
References
Practical Malware Analysis By Michael Sikorski and Andrew Honig
The Malware Sample
The graph chart with the XOR and AES functions call image
The Base64 Index Table example image
The XOR initial explanatory image
The unsuccessful vs successful base64 decoding’ image
