(dramatic entrance)
(audience applaud)
- What I wanna talk about today was twofold.
First thing we wanna chat about a little bit
is why live forensics?
Why are we doing live response?
And the second thing I wanna talk about
is a tool that we released at DFLabs,
a company I currently work for now,
that will help you kind of script
and automate some of your live response collection.
I really appreciate the two keynotes
because they teed things up for me very well,
talking about kind of the crossover
between instant response and threat hunting,
as well as what is evil, right?
We talked about this morning,
how do you find evil?
What is evil?
What needs to be looked into
versus what may be normal?
So why do we do live forensics, right?
Well, as he talked about in the introduction,
Dead Box Forensics,
we just really don't do it anymore.
There's no time for it.
The amount of data that we're collecting
in those kind of situations is just kind of
unnecessary.
So we do live forensics when we want
to preserve things like running processes,
network connections, open files,
things like that.
When we know those are gonna be lost even
if we did pull the drive
or shut the system down, right.
And obviously there's situations
where a system can't be shut down
so we have to do live forensics, right.
Try telling somebody that you're gonna shut
down their domain controller
or you need to take a critical application offline
because you think it might be compromised.
So we're forced to do live forensics at that point.
Now, of course,
we've heard other talks about some great EDR tools,
some great network collection tools,
things like that that are great for grabbing things
like running processes and network connections.
So why, do we even need to do live forensics?
Well, you know, sure.
Some environments may not have
those EDR tools in place.
Obviously in the smaller environments
it may not be affordable and there's still gonna
be times where as great as the EDR solutions are,
you're gonna need some more in depth information.
You're looking for hooks or evidence
of advanced threats.
Those may be things that your EDR tools
aren't gonna pick up.
So as Ben talked about, right,
threat hunting vs. IR we've heard a bunch
of different people mention,
what is the definition?
Where do those things intersect?
And I think we can all agree
that there's a lot of crossover.
We may not agree on exactly where that is,
what the definition of one versus the other is,
but there's a lot of crossover.
So your IR obviously starts with a pivot point, right.
When you're doing an IR investigation
and you're gonna do live forensics as part of that,
you've got that pivot point.
Maybe it's an AV alert.
Maybe you've been,
something alerted on a data breach.
Maybe you've got a IDS alert, right,
you've got that pivot point to work off of
and you can work from there.
In threat hunting we're looking
for that pivot point, right.
So we don't know that there is an alert.
We don't know that there's an incident.
We're looking for that sign of something
that's abnormal so that we can go from there
and then the IR starts.
So there's a lot of crossover between the two.
Now, this morning,
another great tee up,
we talked about what are signs of evil?
What are we looking for when we're doing
these live forensics, right?
Well, we're looking for things that I can't say
or I shouldn't say are always bad
or bad no matter what
but are more normally suspicious, right.
Things like your misnamed processes,
connections to known bad hosts, right.
You've got something connecting to something
that's on your threat intelligence list,
that's probably bad.
Abnormal process owners, right,
I think we all got a copy of the Find Evil Poster.
Everybody knows that.
Everybody should be using that,
it's a fantastic resource.
So I don't need to go up here
and regurgitate all that to you
but that's what you're looking for.
Now, some things are gonna be more subtle, right.
Those sort of things that Rick talked
about this morning.
Maybe they're normal,
maybe they're not normal.
Maybe there's a good reason for this happening
but it could be completely malicious
or completely benign.
So odd ports,
code injection,
things like that may be normal for your AV
but it's probably not normal
for Adobe, right, hopefully.
And like they talked about this morning too,
establish your baselines.
Know what's normal.
If you're gonna be doing good quality threat hunting
and you're gonna be doing live forensics
and acquiring this data off of these hosts, right,
that's what we're gonna talk about
is how to actually acquire that data to examine,
you need to know what's normal.
You need to be able to compare that to something,
so baselines are hugely critical.
So when we talk about doing live forensics, right,
it's a great tool.
We all know there's a dozen reasons
why we should be doing it.
But there's certain challenges that you run
into in live forensics that obviously you don't run
into with the traditional disk space forensics, right.
It's very, very important that
that process be documented
because when you're doing live forensics
you're changing something, right.
I mean, there's no way around that.
You are altering the data and that used to be,
10 years ago,
that was a no, no.
No, we can never change the data
or we can never use it.
And that's just not the case anymore
but it's important to document that
because you need to be able to prove
how you changed the evidence.
Are these artifacts on this host a result
of something I did or is it a result
of something the attacker did?
And if your process isn't documented,
if you just have this ad hoc process
where you're gonna plug in a drive,
go run some tools and get the output,
and then six months later you go to court
or there's some sort of a legal proceeding
and somebody asks you,
well, what did you do?
You know, I plugged something in,
I ran some tools and this is what I got.
Well okay, you're gonna have a bad time.
It needs to be something that's repeatable,
that goes to the same point.
You need to be able to say that,
look, this is a process that we've tested.
It's documented and we do this the same
every single time, right.
That's hugely important and it needs to be secure.
You're going, you're running something
on a live system that maybe your domain controller
or it may just be a user workstation,
but it could also be a critical application system.
So it needs to be something that's secure.
You need to know what you're running
and you need to make sure that those tools
that you're running only do
what you expect them to do, right.
I mean, hopefully,
best, worst case scenario,
it just does something unexpected.
Worst, worst case scenario,
right, somehow your USB drive
or whatever you're running the tools off of
had gotten infected with malware, okay.
And now you're installing a rootkit
on your domain controller
and you're gonna have a bad week.
So it needs to be a secure process.
So how do we achieve that?
Normally it's through batch files, right.
Normally we've got two,
three dozen different little individual
live response tools.
Maybe one goes out and grab network connections,
one's gonna grab open files,
things like that.
We get all these little tools that are great
at doing the individual thing that they do
and we put them all on a USB drive
and then we script them via batch file
because we want it to be a repeatable process.
We wanna be able to take that batch file
and say this is exactly what we ran.
Okay, well that works but there's some problems
with batch files when you start looking
at some of those requirements
like being documented and secure.
There's no native logging or audit trails
in a batch file, right.
You run a batch file, it runs,
that's it, that's all you get.
So if you wanna have that documentation
that's something you have to build
into your batch file.
Now I've got to not only run the,
you know, put commands in my batch file
to run the tools,
I've gotta put commands in there to
pipe the output out.
Now I need to put extra commands in there
to actually make sure that I'm documenting
what processes we'll run.
Some tools are OS or CPU-architecture-specific, right.
So maybe I have a great tool that I love
but it only runs on Windows 7 or newer,
or maybe it's only for 64 bit systems.
Okay, now I've gotta put those statements
in my batch file.
It's easy to modify batch files, right.
Again, you talk about the best worst case scenario
and the worst worst case scenario.
Best worst, maybe somebody just opened
up the batch file,
accidentally edited and made some changes
and now something's gonna fail to run.
Worst worst case,
somebody does something malicious
and now you're running commands with your batch file
that you don't mean to.
And like we talked about earlier,
tools can easily be deleted or over placed,
so it's not terribly secure.
When I was doing IR the last batch file
that I was using to run all of my tools,
I think was something like 1,700 lines
or something stupid like that.
Now granted, that had some pretty output
and do you wanna do this and that sort of thing.
But then you go try to add another tool to that
and you've gotta parse through 1,700 lines
to figure out exactly where you want it
and still there's some issues with batch files.
So what I started writing and finally finished
when I took this new position was something
we're calling the No-Script Automation Tool.
And the idea is it's free,
you can download it.
It's supposed to kind of help overcome some
of those limitations of batch files, right.
So obviously, right,
no scripting required.
That should be kind of obvious with the name.
What we wanted to do was make it easy
to configure for different CPU architectures,
different OSs, wanted to take care
of those things that you normally have
to spend all kinds of time scripting
like logging and hashing output.
And we wanted to take care of some
of the security issues of batch files.
So making sure that the commands and the tools
are verified before you run them,
they're known and they're documented so
that you can go back to that afterwards.
So and as a teaser to the end,
I didn't put the link to actually download
it until the end so everybody has to stick around.
So we're gonna jump right
into actually configuring it
and then I'll tell you at the end
how to download it.
So the idea behind that is that you have
this one executable that will basically,
automatically take care of executing all
of those tools that you would normally
have to put in a batch file.
And so the way we do that and we'll see,
in lieu of having being the only person up here
who had to have a laptop to do a live demo,
I put some videos in here so there'll be a couple
of videos actually going over this process
and executing the tool.
Hopefully I can talk at the same speed as the video.
It's gonna be a little bit awkward
but we'll try it.
So to configure,
to set up your tools,
you just create a tools directory, right.
So you got your USB drive,
you've got NAT sitting on your USB drive.
You create this tools directory and then you start
to organize your tools into different categories.
Now, you don't have to,
you can just throw them all in the tools directory
and run them all at once,
but the nice thing about organizing them
into this tools directory that's over here
is that you can create an .ini file
or one or more .ini files that will allow you
to specify which directories you want to run, right.
So there's a default .ini file,
you can set it up to run everything.
But if you only wanna run maybe your file system
and your OS tools,
you can have an .ini file that just specifies
those two directories,
call that from the command line and run it that way.
So it's a good idea to be able
to split up your tools like that.
So the other thing you can do
within these tools directories
is set up individual folders.
So we can set up folders here
for operating system version numbers
as well as CPU architectures and nest them inside.
So you can have your file system directory
and then you can create a 6-10.0 folder.
So when you're running tools
from the file system directory
it's only going to run on Windows version 6-10.
You can create an x64 directory
that will only run tools on a x64 bit system.
And that will take care of all this automatically.
When it executes,
it will automatically detect the OS
and the CPU architecture
and just run the proper tools so you don't have
to script all that stuff.
As you can see,
you can nest them here so you can have a 6-10
and then an x64.
You can do it in reverse
but you can't double down on that.
So you can't have a 6-10 x64 and then 8-10.
It only goes two levels.
So how do we specify command line arguments, right?
That's important, we don't just run tools.
We normally run them via the command line.
We're gonna put them into a text file
that's just named with .cmd at the end.
So if you wanna run ping.exe
you just create a new text file,
put it in that same directory with ping.exe
and call it ping.exe.cmd.
And you just put the command line arguments
right in that text file.
And you can have more than one,
so if you wanna run the same tool
with three different command line arguments,
you can certainly do that.
You just put one on each line,
it will execute it once per command line.
All right, so this is the first video.
Okay, so we're gonna create that tools directory.
And that's gonna be where we're gonna store
all of our tools by default.
Now, it's gonna be in the same directory
as the NAT executable itself
because that's where NATs gonna look for it.
So within the tools directory I talked about
we're gonna create those individual,
sort of category folders, right.
So we'll create a folder for our network tools,
our OS tools,
things like that.
So we'll make our network directory
and within that we can start adding
our different network tools.
So in this case I'm just throwing one tool
in here for a demo purpose.
Now you'll see,
when I run this I tend to prepend everything
with a name.
In this case I've used NAT_.
It's not a requirement for the NAT tool
but it is good best practice.
Again, we talked about differentiating
what you did versus what maybe an attacker
or somebody else did.
So the easiest way or one of the easiest ways
to do that is just prepend the file name
in front of all of your tools so you can see,
hey, that was obviously associated
with something I did.
Now you can see here,
I'm gonna create that x64 folder
so we can jump inside that and put a tool
inside that that will only be run inside
of a 64 bit OS.
We can also create that file system
to the operating system version folders.
So we'll create that 6-10 folder there
and that will only be run on Windows Vista
through Windows 10 Operating Systems.
And then within that,
obviously, we can nest another directory
there as well.
So we threw that 6-10 directory
inside the 64 bit folder.
So now it'll only run on 64 bit operating systems.
Okay, so now I'm gonna throw that .cmd file in there.
It will open that up and we'll take a look at it.
You can see its named the exact same
as the executable,
that's how NAT ties those two together.
But we just prepend a .cmd at the end
and you can see in this case all I'm doing
is just accepting the EULA, right.
It's one of those system internals tools
that's gonna pop up a EULA so we can make sure
that we accept the EULA and we don't get a popup
when we execute that.
But you can put any commands you want
in that .cmd file.
So there's a couple of special commands
that you can put inside the .cmd file
that are specific to NAT.
They're variables that allow you
to do some special things and NAT
will place these variables when you actually
run the tool at runtime.
So the first one is the %NOOUT% variable.
And basically what that's gonna do
is tell NAT that it doesn't need to take care
of the output, right.
So some tools you can do a -l
when specify the output directory.
You don't need NAT to pipe that output
for you into a file.
So if you specify the %NOOUT% command
in the command file NAT will not pipe
that output out for you automatically,
you can specify that in the command file.
Now obviously, where we do that is going
to be different for each execution, right.
You don't know what drive letter,
what folder you're gonna wanna pipe the output
into when you're running NAT at runtime.
So that's what that second variable is for,
the %OUTDIR% directory.
When you run NAT it's going to ask you
where you wanna output the files to?
And that's gonna change so you can use
that %OUTDIR% variable within your command file
to specify whatever output directory
is chosen at runtime.
And the last one is just the %SYSROOT% variable, right,
see Windows system 32.
But on some systems that can change
so that's just a variable that you can use.
So how do we actually run this?
What does it actually look like when we execute?
So this is the command line interface.
Pretty simple, you've got a help menu obviously.
We'll talk about integrity files.
We also have that .ini file so like I said,
you can create the default .ini file.
That's what will be used
if you don't specify anything.
But you can specify that .ini file
on the command line if you wanna call
a different one that maybe only runs
a certain subset of your tools.
So while we're waiting for me in the video here,
the integrity check file is something
that can be used to verify the tools
that you're running, right.
So we talked about tools
getting accidentally replaced, maliciously replaced,
we talked about the same thing in your commands.
So an integrity check file basically allows you
to kind of create a gold standard,
if you will,
or a known set of tools that you create that.
And then you can create this integrity
check file and every time you're executing
that after that,
it's gonna check and make sure that all
of your tools are the way they were
when you created that integrity file, right.
So nothing's been changed.
No MD5s are different.
No command line arguments are different.
So we'll go ahead and we're gonna run NAT
and you can see you get a security warning.
We haven't created that integrity check file yet
so it's gonna pop up and say,
hey, you know,
you're not verifying your tools,
this can be dangerous.
Are you sure you wanna continue.
Now, there's nothing but you can if you want.
I'm not here to tell you how to do your business
but it will at least prompt you to say,
hey something, you should be creating
this integrity check file.
So we're gonna select no.
And then we're gonna run NAT
and we're gonna run it with the -c command
to create that integrity check file.
And you can see what it's done
is it's gone through and it's hashed every single
executable in all of our paths.
It's hashed all of the .cmd files.
And you can see for the text files
it's actually also displaying the content
of those text files.
So it's actually gonna show us
what commands were running.
So it's gonna go through.
It's gonna do all that.
It's gonna display it to you.
And we're gonna say,
are these commands and these executables trusted?
If you say yes,
it'll create that integrity check file.
And you can see it's gonna ask you for a password.
So we enter the password and it's going to
take that information and encrypt it
and put it into a file on the USB drive itself
so that will always be present there,
and it will always check that from now on
every time you execute NAT.
I should've done this in like six
or seven chunks, I apologize.
All right, so now we're gonna go ahead
and run NAT again.
Now you can see it's gonna prompt us
for the password.
So we'll go ahead
and if we enter an incorrect password
it's just gonna pop up,
tell us the password's wrong.
If we run it again,
this time we'll enter the correct password.
Possibly,
might have clicked the right password.
There we go,
okay so you can see what I've done here
is I've modified that respond.bat file.
So this is what happens.
We entered the correct password
but it's gonna pop up and say,
hey, something doesn't match here.
Now, do you wanna run this?
Are you okay with this not matching?
Is this a acceptable change
or is this maybe malicious?
We can see in this case it's gonna show us
what the new file was.
If we want we can use V and it will actually display
the contents of the integrity check file
that currently exists so we can compare
what it used to say with what it does say.
Now, in this case I just changed the text to
this is another sample batch file
versus a sample batch file.
And we can choose then after we view the contents
of that whether that's an acceptable change or not.
Now, there's some,
there in the security warning you'll see
it notes some tools do change things like config files
every time they execute.
So it's not always malicious when you see a change.
But it's something that you wanna look at
and so that's why we can display both versions
and give you the option to run that tool or not.
So you can see we've entered the correct password,
no change has been made.
We can see it's detected a 64 bit
Windows 10 operating system.
And it tells us by default the data's gonna be written
to the D drive in that particular folder.
Now, the way that that's chosen by default
is the root of the drive
that you're running the tool from
and the name of the Windows host.
So you can change that if you want
but that's the way that the default is chosen.
If you're running it from,
say, a Windows Share or something like that,
obviously you can change it.
But in this case we're gonna say no,
we don't wanna change it.
And it's gonna go right into the execution.
We can see it's gonna tell us as we're executing
what tools we're executing.
It's gonna hash the output at the very end
and then it's gonna tell us whether
it completed successfully or not.
And that's how simple it is.
There was no other scripting required.
It's just gonna run all those tools for us.
So as soon as I click any key to continue here
we're gonna jump and just look
at the output real quick so we can see
what we're actually collecting.
So we can see this is the directory
that we've created up here.
Now, if we open that up we're gonna see
there's a couple of special files in there.
One of them is
the NAT.log which is our log file.
The other one is MD5.txt.
And then you can see we've got the same directories
that we saw in our .ini file,
in this case I only ran it with three.
So the NAT.log file,
which I believe is the first one
we're gonna look at here,
that's gonna be a log of all of the activity
that NAT took.
So here we go.
So we can see it's timestamped.
We can see exactly what commands were executed,
what tools they were run from,
what the MD5 hash values of the tool,
all that stuff is?
So when we talk about the process needing
to be documented and repeatable,
here we can see we can document exactly
the date and time we ran the tool,
what the hash value was,
where it was located?
Everything is right there
and that comes prepackaged right along
with all of your output.
Dramatic pause for the next file.
This is the MD5.txt file
and this is a hash of all of your output.
So the very last thing we do before we exit NAT
is hash all that output.
So as soon as those were created,
we have a hash value of the output
of each one of the tools you ran,
so you can prove that, you know,
right after I ran it this was the hash value.
This hasn't been modified.
These are legitimate files.
And the very last thing we'll look at
is just some of the sample outputs
so you can see we've jumped into the network folder.
We can see the output is gonna be named
for the tool.
So tcpvcon.txt I think is the one I opened.
That was the tool I ran.
You can see we've got openports
and then openports_1.
That's what it'll look like if you run a tool
with more than one command line option.
You'll have a one, a two, a three appended to it.
So you've got the output in separate files.
So if we open tcpvcon.txt we'll see it's basically
just gonna be the output of the text
of the tool that we ran.
But the only thing that will be appended
up at the top is gonna be the date and time
that it was executed as well as
the command line arguments that were executed.
So if you're running the tool multiple times,
multiple command line arguments,
it will tell you what command line tool,
what command line you executed that tool with.
So final notes on the tool before I wrap up.
Free, use it at your own risk.
If you blow up your domain controller with it,
don't send me hate mail on Twitter or something.
Just do your thing.
It's available at dflabs.com/NAT.
It's a marketing thing.
I tried to get it without the put in your name
and your email address and all that.
They do ask for that information
before you download it.
We won't spam you or anything like that.
We'll give you a little tip.
All that's gonna do once you register
is send you to our GitHub page.
So if you came across our GitHub page
you could probably download it right from there
without having to put in your information,
just throwing that out there.
So questions, comments,
suggestions, any other entertaining things
that you want to send me,
that's my email address right there.
If you don't write that down
and you wanna contact me later,
I'm at LinkedIn, Twitter
and obviously you can contact me right
through the company on our website.
All right, thank you guys very much.
The download link.
Appreciate the time.
(applause)
(dramatic music)