Volshell for Overlay Offsets

Using Volshell to Determine Offsets for Volatility Overlays

For the last year I experienced an issue where all established TCP connections that show up using the netscan plugin in Volatility would have the owning process PID and name missing. I struggled to find documentation on the problem, but was only able to trace it back to likely relating to an offset issue. It was only after I attended the Volatility training class (which is highly recommended) and learned how to use the volshell plugin that it seemed feasible to attempt to fix this.

Please keep in mind that the vast majority of Volatility users shouldn't ever need to change the overlay offsets.  If your systems are abnormal like mine, though, I believe you can use this process for most offset calculations.  The information below is likely not the best way to find the correct offsets, but it is only way I know of and I hope it may help. 


The problem
Missing owner processes for established TCP connections:

Offset(P)   Proto Local Address    Foreign Address  State       Pid      Owner Created
0x1bb392890 TCPv4 <redacted>:49224 <redacted>:10123 ESTABLISHED -------- --------------


The solution
Volatility uses overlays to apply structure to the data in memory to facilitate the parsing of data by plugins.  The overlays are applied according to the profile you specify when running Volatility (e.g. Win7SP1x64).   When your systems are just plain odd, like mine, they may not have everything in the same places as "normal" systems.  In my case the problem appears specifically for established TCP connections.    I had to use a known "established" TCP connection and process combination to help identify what the actual overlay offsets were so that I could change the "normal" offset over to my abnormal offset. 

Each overlay is a data structure that acts as a table of contents for that specific type of data structure.  This situation requires we examine the "_TCP_ENDPOINT" data structure to see what the offset is set to for the "Owner" field.  We know that a pointer to the owning "_EPROCESS" structure should reside in the "_TCP_ENDPOINT" owner field.

This figure shows how everything should work for my test scenario.  We get the virtual address for the "_EPROCESS" structure of our owning process from changing context to it in volshell.  Then we get the physical address for the "_TCP_ENDPOINT" structure from the netscan plugin.  We can then look for the little endian version of the "_EPROCESS" virutal address in the space assigned to the "_TCP_ENDPOINT" structure to see what the actual offset is.  We can then update the overlay with the correct offset and all should be good.  Hopefully. :)

The steps
1.) Identify a process in a memory sample that in which you know the name and/or PID for an established TCP connection.  You will use this as your template to identify the owning process offset to modify the overlay. My example was obtained with a netstat on the machine and the port identified (TCP 10123) was tied to the "CcmExec.exe" process.

Netstat output:
TCP <redacted>:64986 <redacted>:10123 ESTABLISHED 3820
[CcmExec.exe]

2.) Run the volshell plugin against your sample memory image.

python vol.py -f memory.img --profile=Win7SP1x64 volshell

3.) You will start out in the system context, but you want to change to the context of the process you identified in step 1. Make note of the address volshell returns to you.

>>> cc(name="CcmExec.exe")
Current context: CcmExec.exe @ 0xfffffa8009ef55e0, pid=4916, ppid=736 DTB=0x1562f3000

4.) Your goal in future steps will be to identify the pointer to that _EPROCESS structure address for your sample process. The pointer is listed in little endian format, though, so you will need to translate it.

0xfffffa8009ef55e0 => e0 55 ef 09 80 fa ff ff

5.) You can use the dt() function in volshell to display the information for the "_TCP_ENDPOINT" type. Here you can see that it believes the process owner for TCP endpoint connections can be found at the offset of 0x238. We know that is wrong in my case, but we now know where to start looking.

>>> dt("_TCP_ENDPOINT")
'_TCP_ENDPOINT' (None bytes)
0x0 : CreateTime ['WinTimeStamp', {'is_utc': True, 'value': 0}]
0x18 : InetAF ['pointer', ['_INETAF']]
0x20 : AddrInfo ['pointer', ['_ADDRINFO']]
0x28 : ListEntry ['_LIST_ENTRY']
0x68 : State ['Enumeration', {'target': 'long', 'choices': {0: 'CLOSED', 1: 'LISTENING', 2: 'SYN_SENT', 3: 'SYN_RCVD', 4: 'ESTABLISHED', 5: 'FIN_WAIT1', 6: 'FIN_WAIT2', 7: 'CLOSE_WAIT', 8: 'CLOSING', 9: 'L: 'DELETE_TCB'}}]
0x6c : LocalPort ['unsigned be short']
0x6e : RemotePort ['unsigned be short']
0x238 : Owner ['pointer', ['_EPROCESS']]

If you want to overlay this structure on the data you can run the dt(<structure>,<address>) command. The physical address for the _TCP_ENDPOINT structure can be found with the netscan plugin and is the first field for the line that you are trying to track down. Note that "space=addrspace().base" needs to be added if the address is a physical offset.

>>> dt("_TCP_ENDPOINT", address=0x1bb392890, space=addrspace().base)
[_TCP_ENDPOINT _TCP_ENDPOINT] @ 0x1BB392890
0x0 : CreateTime 1970-01-01 00:00:00 UTC+0000
0x18 : InetAF 18755738026519642192
0x20 : AddrInfo 18755638046702079660
0x28 : ListEntry 7436052664
0x68 : State ESTABLISHED
0x6c : LocalPort 49224
0x6e : RemotePort 10123
0x238 : Owner 8900317189

6.) Next you want to find the instance of the _EPROCESS pointer address ("e0 55 ef 09 80 fa ff ff") in the _TCP_ENDPOINT structure. You can do this by opening the memory dump in a hex editor and searching for the string of hex characters or you can use the db() function in volshell to display the bytes starting at the _TCP_ENDPOINT base address and going forward until it is found. I searched with a length of 1024, pasted the output in notepad++, and searched for the hex character string.


Printing out the _TCP_ENDPOINT data:
>>> db(0x1bb392890,space=addrspace().base, length=1024)
0x1bb392890 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 ................
0x1bb3928a0 30 28 fb 09 80 fa ff ff 00 7a 64 07 80 fa ff ff 0(.......zd.....
0x1bb3928b0 80 a4 4e 0c 80 fa ff ff 10 c0 64 07 80 fa ff ff ..N.......d.....
<snip>

After pasting the data into notepad++ I searched for the string and found that it was at 0x1bb392ad8.


Verify the pointer address is really at that address by displaying the bytes at that address.
>>> db(0x1bb392ad8, space=addrspace().base, length=8)
0x1bb392ad8 e0 55 ef 09 80 fa ff ff .U......

7.) Calculate the corrected offset using the offset of the real owning process pointer (0x1bb392ad8) and subtracting the _TCP_ENDPOINT base address (0x1bb392890).

0x1bb392ad8 - 0x1bb392890 = 0x248

8.) Open the overlay file (e.g. volatility\plugins\overlays\windows\tcpip_vtypes.py), locate the existing offset (0x238) for your data type/profile and change it to the new offset. In my case I needed to change 0x238 to 0x248. Save and close the file.  Note: if you have any reservations, make a backup of the file first so you can easily backout your change.


9.) Run the netscan plugin to validate that the owning process information is now populated correctly.

Offset(P)   Proto Local Address    Foreign Address  State       Pid  Owner Created
0x1bb392890 TCPv4 <redacted>:49224 <redacted>:10123 ESTABLISHED 4916 CcmExec.exe

Hope it helps!