1
00:00:01,101 --> 00:00:05,038
(angelic synthesizer
pad fades in)

2
00:00:05,038 --> 00:00:09,743
(low-pitched hum crashes
and then fades out)

3
00:00:09,743 --> 00:00:12,746
(audience applauds)

4
00:00:14,014 --> 00:00:15,514
- So I'm Josh Bryant,
like you said,

5
00:00:15,515 --> 00:00:18,451
I'm a Director of Technical
Account Management at Tanium.

6
00:00:18,451 --> 00:00:20,420
- And I'm Robert Falcone,
I'm a Threat Researcher

7
00:00:20,420 --> 00:00:24,024
over at Palo Alto
networks on Unit 42.

8
00:00:24,024 --> 00:00:26,726
- And today we're gonna
talk about web shells.

9
00:00:26,726 --> 00:00:28,028
Why web shells?

10
00:00:28,028 --> 00:00:29,828
Well, this is what U.S.
Cert had to say about it

11
00:00:29,829 --> 00:00:31,197
a few years ago.

12
00:00:31,197 --> 00:00:33,033
They've been around
a really long time,

13
00:00:33,033 --> 00:00:35,702
probably since the
creation of the web,

14
00:00:35,702 --> 00:00:39,305
and the tactics are just
getting better and better

15
00:00:39,305 --> 00:00:42,042
and we're finding
more and more of 'em.

16
00:00:42,042 --> 00:00:45,278
- Yeah, so, this is what we're
gonna be talking about today,

17
00:00:45,278 --> 00:00:46,613
is these web shells, okay?

18
00:00:48,014 --> 00:00:50,917
There's so many out
there, it's unbelievable.

19
00:00:50,917 --> 00:00:53,920
So you can go to
Github repositories

20
00:00:53,920 --> 00:00:57,323
and find hundreds of these
just easily accessible.

21
00:00:57,323 --> 00:01:00,894
And they range dramatically
in capabilities.

22
00:01:00,894 --> 00:01:02,262
But the reason why
they're really important

23
00:01:02,262 --> 00:01:05,230
is because of the
access that they provide

24
00:01:05,230 --> 00:01:06,499
to the web server.

25
00:01:06,499 --> 00:01:08,768
Whether it be running
commands specifically,

26
00:01:08,768 --> 00:01:11,938
uploading, downloading
files, that kind of thing.

27
00:01:11,938 --> 00:01:15,308
And like I said, they're
very easy to obtain,

28
00:01:15,308 --> 00:01:18,378
so in a lot of cases, actors
don't have to build their own.

29
00:01:20,346 --> 00:01:22,916
So, the one web shell
that we're gonna

30
00:01:22,916 --> 00:01:27,152
be talking about today is
not one of those web shells

31
00:01:27,153 --> 00:01:29,923
that you can freely
grab off the Internet.

32
00:01:29,923 --> 00:01:34,327
This is a custom-built web shell

33
00:01:34,327 --> 00:01:36,529
that an adversary was
using, they developed it

34
00:01:36,529 --> 00:01:41,167
to be used in their
campaigns in the Middle East.

35
00:01:41,167 --> 00:01:45,905
And me and my colleague
named it TwoFace,

36
00:01:45,905 --> 00:01:47,774
because of the
unique design which

37
00:01:47,774 --> 00:01:50,477
they used to develop it.

38
00:01:50,477 --> 00:01:53,580
- And as Rob was
mentioning, this is the one

39
00:01:53,580 --> 00:01:56,950
that I showed at this
very conference last year,

40
00:01:56,950 --> 00:02:00,320
however, at the time, I
couldn't find a name for it.

41
00:02:00,320 --> 00:02:02,821
I could've sworn, I can't
possibly be the first one

42
00:02:02,822 --> 00:02:05,091
to come across this,
still probably wasn't,

43
00:02:05,091 --> 00:02:07,092
we just happened to be
doing the same research

44
00:02:07,093 --> 00:02:09,429
from a couple different
angles at the same time.

45
00:02:09,429 --> 00:02:11,598
And we're able to come
together to kind of update this

46
00:02:11,598 --> 00:02:12,899
and give you guys
some new information.

47
00:02:12,899 --> 00:02:14,701
- Yep, so I named it.

48
00:02:14,701 --> 00:02:18,704
It's a silly name,
but it was named

49
00:02:18,705 --> 00:02:21,241
because there's two
layers to it, okay?

50
00:02:21,241 --> 00:02:23,476
The first layer is a loader.

51
00:02:23,476 --> 00:02:27,013
It does, its whole
big responsibility

52
00:02:27,013 --> 00:02:28,348
is to install the second layer,

53
00:02:28,348 --> 00:02:30,116
which is the TwoFace payload.

54
00:02:31,317 --> 00:02:33,219
And we're gonna talk
about this in detail

55
00:02:33,219 --> 00:02:35,021
going through this talk.

56
00:02:35,021 --> 00:02:36,890
So, what's the loader?

57
00:02:36,890 --> 00:02:40,226
Like I said, its purpose is
to install the second layer,

58
00:02:40,226 --> 00:02:41,628
which is the payload.

59
00:02:41,628 --> 00:02:45,498
But, what they did was,
the actors actually took

60
00:02:45,498 --> 00:02:47,466
their loader code
and appended it

61
00:02:47,467 --> 00:02:51,905
to legitimate content
on that web server.

62
00:02:51,905 --> 00:02:54,474
In this case, I chose one
that was a news article

63
00:02:54,474 --> 00:02:56,976
on an IIS server,
but what Josh and I

64
00:02:56,976 --> 00:03:00,747
are gonna talk about
throughout this presentation,

65
00:03:00,747 --> 00:03:05,718
they also appended it to error
messages on Exchange servers.

66
00:03:07,086 --> 00:03:09,122
So they're really focused
on Exchange servers.

67
00:03:09,122 --> 00:03:13,993
Okay, so the loader, the
actor, when they wanna start

68
00:03:13,993 --> 00:03:15,495
interacting with the web shell,

69
00:03:15,495 --> 00:03:19,432
they send a specific POST
request to the loader,

70
00:03:19,432 --> 00:03:21,733
which installs the
payload, that second layer

71
00:03:21,734 --> 00:03:22,902
I just mentioned.

72
00:03:22,902 --> 00:03:25,171
As you can see on this slide,

73
00:03:25,171 --> 00:03:29,475
this is where all the capability
exists for this payload,

74
00:03:29,475 --> 00:03:30,710
for this web shell.

75
00:03:30,710 --> 00:03:34,948
It's pretty capable,
upload, download files,

76
00:03:34,948 --> 00:03:38,017
execute commands,
timestomp files.

77
00:03:38,017 --> 00:03:41,853
You can even run SQL queries
on remote SQL servers.

78
00:03:41,854 --> 00:03:43,723
But one thing I
should focus on here

79
00:03:43,723 --> 00:03:47,160
is that you have to
authenticate to this web shell

80
00:03:47,160 --> 00:03:49,028
to actually be able
to use this thing.

81
00:03:49,028 --> 00:03:52,832
So you can't just
find this web shell

82
00:03:52,832 --> 00:03:55,101
and start interacting with
it without that password,

83
00:03:55,101 --> 00:03:56,369
which is really important.

84
00:03:57,570 --> 00:03:59,939
So, how does the actor
actually interact

85
00:03:59,939 --> 00:04:01,474
with this web shell?

86
00:04:01,474 --> 00:04:05,911
How did they construct the
POST request to the loader

87
00:04:06,813 --> 00:04:08,648
to install the payload?

88
00:04:08,648 --> 00:04:12,018
Well, the actor will
provide a 64-byte key,

89
00:04:12,018 --> 00:04:14,687
which is fairly massive, right?

90
00:04:14,687 --> 00:04:17,323
The loader will actually
manipulate that key

91
00:04:17,322 --> 00:04:19,892
with a key salt, so
that's pretty clever,

92
00:04:19,892 --> 00:04:21,761
and then the result of
that is gonna be used

93
00:04:21,761 --> 00:04:24,831
to decrypt the embedded
payload web shell,

94
00:04:24,831 --> 00:04:28,401
and then that's saved
to the web server.

95
00:04:28,401 --> 00:04:31,037
And, so that's
the whole process.

96
00:04:31,037 --> 00:04:34,207
And I felt like that,
you really need to see it

97
00:04:34,207 --> 00:04:37,343
to believe what that process is.

98
00:04:37,343 --> 00:04:40,046
So I made a demo video, okay?

99
00:04:40,046 --> 00:04:43,616
Like I said, we're gonna talk
about Exchange primarily,

100
00:04:43,616 --> 00:04:46,252
but they also had
one sample that

101
00:04:46,252 --> 00:04:49,455
I'm gonna show you right now
that ran specifically on IIS.

102
00:04:49,455 --> 00:04:51,957
So for simplicity, I
just chose that sample,

103
00:04:51,958 --> 00:04:53,226
and I set up an IIS server

104
00:04:53,226 --> 00:04:54,694
just to show you how it works.

105
00:04:54,694 --> 00:04:56,929
- Yeah, it's important to
note that IIS is a dependency

106
00:04:56,929 --> 00:04:59,666
of Exchange, so there's not
too much of a difference here.

107
00:04:59,666 --> 00:05:03,169
- Yep, so here I'm just
showing you in this video,

108
00:05:03,169 --> 00:05:05,270
I just set up a
simple web server,

109
00:05:05,271 --> 00:05:07,073
and I'm showing you
that this web server,

110
00:05:07,073 --> 00:05:08,741
the only thing
that's hosted on it,

111
00:05:08,741 --> 00:05:12,645
is the TwoFace loader
that I named TwoFace.aspx,

112
00:05:12,645 --> 00:05:14,347
right there, okay?

113
00:05:14,347 --> 00:05:16,049
We're gonna hop over
into the browser

114
00:05:16,049 --> 00:05:18,150
and just browse to it, okay?

115
00:05:18,151 --> 00:05:19,852
And what you're gonna see
is something very similar

116
00:05:19,852 --> 00:05:21,921
to what you saw on
the slides there,

117
00:05:21,921 --> 00:05:23,623
you're gonna see a news article

118
00:05:23,623 --> 00:05:25,792
that was meant to be
on that web server.

119
00:05:27,260 --> 00:05:28,795
There it is.

120
00:05:28,795 --> 00:05:31,564
But, if the actor wanted to
actually use the web shell,

121
00:05:31,564 --> 00:05:34,100
they're gonna have to
instruct the loader

122
00:05:34,100 --> 00:05:35,601
to install the payload,

123
00:05:35,601 --> 00:05:39,105
so I wrote a script to be
able to do that myself.

124
00:05:39,105 --> 00:05:41,774
So here's the key
that's required

125
00:05:41,774 --> 00:05:44,877
to install this
particular payload.

126
00:05:44,877 --> 00:05:47,180
So I'm gonna run my script,
which is gonna craft

127
00:05:47,180 --> 00:05:50,516
the correct HTTP request,
which is gonna send

128
00:05:50,516 --> 00:05:55,221
the key over with the
filename to drop it to.

129
00:05:55,221 --> 00:05:56,189
There it goes.

130
00:05:57,357 --> 00:06:00,026
The TwoFace loader
responded back,

131
00:06:00,026 --> 00:06:02,628
saying, yep, I
installed the payload.

132
00:06:02,628 --> 00:06:05,832
And we're gonna hop back
over to the IIS server.

133
00:06:05,832 --> 00:06:09,268
I'm gonna show you that this
new file called drop.aspx

134
00:06:09,268 --> 00:06:11,804
is gonna be on the
IIS server as well.

135
00:06:11,804 --> 00:06:13,239
There it is right there.

136
00:06:13,239 --> 00:06:15,073
So if we navigate to
that in the browser,

137
00:06:15,074 --> 00:06:18,077
that's where you're gonna
see the functional web shell

138
00:06:18,077 --> 00:06:19,579
for TwoFace right here.

139
00:06:19,579 --> 00:06:21,080
It's a little bit
different than the one

140
00:06:21,080 --> 00:06:23,916
you saw in the slides, but
very similar variation.

141
00:06:23,916 --> 00:06:26,185
So right now I'm trying to
just run a command, whoami,

142
00:06:26,185 --> 00:06:27,887
in this web shell.

143
00:06:27,887 --> 00:06:30,289
It didn't work, 'cause
I had to authenticate.

144
00:06:30,289 --> 00:06:33,025
You can see that that
bar at the top is red,

145
00:06:33,025 --> 00:06:35,594
and when I authenticated
it turned green.

146
00:06:35,595 --> 00:06:38,431
And now I run the whoami
command and it worked perfect.

147
00:06:39,932 --> 00:06:44,804
So we're gonna talk about
TwoFace in-depth coming up,

148
00:06:44,804 --> 00:06:48,340
and exactly what the actor
was running command-wise.

149
00:06:48,341 --> 00:06:51,711
So when you see us talking
about running commands

150
00:06:51,711 --> 00:06:54,914
and doing things using
TwoFace, think of the actor

151
00:06:54,914 --> 00:06:57,517
actually interacting with
something like this, okay?

152
00:06:58,484 --> 00:07:00,820
So, from a hunting perspective,

153
00:07:00,820 --> 00:07:03,656
how did we go about
hunting TwoFace

154
00:07:03,656 --> 00:07:05,358
and the web shells themselves?

155
00:07:05,358 --> 00:07:09,061
As a threat researcher, I
rely a lot on YARA rules

156
00:07:09,061 --> 00:07:10,763
and other kind of
detection signatures,

157
00:07:10,763 --> 00:07:13,232
so we had to write them, right?

158
00:07:13,232 --> 00:07:16,803
But the whole purpose of
the two layers for TwoFace

159
00:07:16,803 --> 00:07:19,070
was that the loader
layer's very obfuscated.

160
00:07:19,071 --> 00:07:20,706
It's difficult to detect.

161
00:07:20,706 --> 00:07:22,575
So, what did we have to do?

162
00:07:22,575 --> 00:07:25,111
We had to figure out how
to detect the payload

163
00:07:25,111 --> 00:07:27,246
that's embedded
within the loader.

164
00:07:27,246 --> 00:07:28,981
So, how did we do that?

165
00:07:28,981 --> 00:07:32,718
Well, we took a look at the
actual decryption mechanism

166
00:07:32,718 --> 00:07:34,987
within the loader, and we found,

167
00:07:34,987 --> 00:07:36,923
this is how they
applied the key salt

168
00:07:36,923 --> 00:07:41,928
to the 64-byte key, and then
they used that resulting value

169
00:07:43,129 --> 00:07:46,732
as the key to decrypt
the embedded payload.

170
00:07:46,732 --> 00:07:49,402
And I don't know if
there's anyone in here

171
00:07:49,402 --> 00:07:51,804
that studies ciphers,
but this is probably

172
00:07:51,804 --> 00:07:56,174
the weakest cipher in the
history of cryptography.

173
00:07:56,175 --> 00:07:58,110
They're using addition
and subtraction,

174
00:07:58,110 --> 00:08:00,179
so we can easily
just reverse that,

175
00:08:00,179 --> 00:08:03,482
and we wrote a simple script
to do a crypto attack on it,

176
00:08:03,483 --> 00:08:06,219
and we were able to
find the 64-byte key,

177
00:08:06,219 --> 00:08:08,754
and then we were able
to write some YARA rules

178
00:08:08,754 --> 00:08:12,525
for the payload, because we
now have the cleartext, okay?

179
00:08:12,525 --> 00:08:16,128
So, from a threat researcher,
that's how I went about

180
00:08:16,128 --> 00:08:19,265
digging through our content,
by writing YARA signatures,

181
00:08:19,265 --> 00:08:22,068
or YARA rules, to hunt
for these payloads.

182
00:08:24,470 --> 00:08:26,272
- Alright, now that we
have seen a little bit

183
00:08:26,272 --> 00:08:28,307
about what web shells
are, how they work,

184
00:08:28,307 --> 00:08:32,912
and how this attacker was using
it, it's time to go hunting.

185
00:08:32,912 --> 00:08:35,648
And if you've maybe
been living under a rock

186
00:08:35,648 --> 00:08:37,183
for the past few
years, or haven't gone

187
00:08:37,183 --> 00:08:39,018
to a few other conferences,
you may have missed

188
00:08:39,018 --> 00:08:42,088
the fact that attackers
think in graphs.

189
00:08:42,087 --> 00:08:45,123
When I go hunting, I wanna
think like a bad guy,

190
00:08:45,124 --> 00:08:48,594
so I need to think in graphs
to be successful in my hunt.

191
00:08:48,594 --> 00:08:53,599
So let's look at how an
attacker might get the web shell

192
00:08:54,734 --> 00:08:57,203
on to the target, on
to the Exchange server.

193
00:08:57,203 --> 00:08:58,771
So here I've created
a couple different

194
00:08:58,771 --> 00:09:00,272
attack graph scenarios.

195
00:09:00,273 --> 00:09:02,875
The first one I call here
is the Attacker's Hope.

196
00:09:02,875 --> 00:09:05,545
This is how the attacker
wishes it worked.

197
00:09:05,545 --> 00:09:08,548
It's plausible but not likely,

198
00:09:08,548 --> 00:09:10,616
it may also be how
your management

199
00:09:10,616 --> 00:09:13,185
might think this
type of attack works.

200
00:09:13,185 --> 00:09:17,089
So here our admin uses
Domain Admin everywhere,

201
00:09:17,089 --> 00:09:19,559
he's never heard of
credential hygiene.

202
00:09:19,559 --> 00:09:21,294
He just logs in with
his Domain Admin cred,

203
00:09:21,294 --> 00:09:22,728
so everybody gets in, in fact

204
00:09:22,728 --> 00:09:24,664
if you all look under
your seats right now,

205
00:09:24,664 --> 00:09:26,332
you've all got Domain Admins.

206
00:09:28,634 --> 00:09:30,937
So now our attacker
is going to attempt

207
00:09:30,937 --> 00:09:33,773
to directly compromise
the Exchange server.

208
00:09:33,773 --> 00:09:35,308
He's gonna try a few
different techniques.

209
00:09:35,308 --> 00:09:37,877
This is standard for implanting
any kind of web shell

210
00:09:37,877 --> 00:09:40,378
on any server, however,
these techniques tend

211
00:09:40,379 --> 00:09:42,315
not to work on Exchange servers.

212
00:09:42,315 --> 00:09:44,016
There's no SQL in Exchange,

213
00:09:44,016 --> 00:09:45,985
so SQL injection's
not gonna work.

214
00:09:45,985 --> 00:09:47,520
There might be a
remote file inclusion,

215
00:09:47,520 --> 00:09:49,589
that's probably not gonna work.

216
00:09:49,589 --> 00:09:52,158
You might have some
cross-site scripting.

217
00:09:52,158 --> 00:09:55,227
Again, plausible, probably
not gonna do anything.

218
00:09:55,227 --> 00:09:57,730
Might have local file
inclusion, they're gonna try it,

219
00:09:57,730 --> 00:09:59,699
probably not gonna
work in this instance.

220
00:09:59,699 --> 00:10:03,102
Our most likely culprit is
gonna be remote code execution.

221
00:10:03,102 --> 00:10:07,540
I could not find any remote
code execution vulnerabilities

222
00:10:07,540 --> 00:10:09,608
in Exchange itself,
there have been some

223
00:10:09,609 --> 00:10:12,612
in the underlying
IIS in the past,

224
00:10:12,612 --> 00:10:16,983
but if you have a fairly
up-to-date system,

225
00:10:16,983 --> 00:10:18,684
unlikely here as well.

226
00:10:18,684 --> 00:10:21,621
But here they're gonna use
the remote code execution,

227
00:10:21,621 --> 00:10:23,656
get access to the
Exchange server,

228
00:10:23,656 --> 00:10:26,525
and because our Domain
Admin uses creds everywhere,

229
00:10:26,525 --> 00:10:30,529
he's gonna go ahead and
run Mimikatz, not a virus,

230
00:10:30,529 --> 00:10:33,733
and dump creds and then use that

231
00:10:33,733 --> 00:10:36,535
to gain access to the
rest of the environment.

232
00:10:38,270 --> 00:10:41,540
So now let's look at a
more likely scenario.

233
00:10:41,540 --> 00:10:44,843
Here our attacker starts
by sending an email

234
00:10:44,844 --> 00:10:46,812
to their target.

235
00:10:46,812 --> 00:10:50,883
Our user here loves
email, loves attachments,

236
00:10:50,883 --> 00:10:53,119
needs to enable macros,
and because they

237
00:10:53,119 --> 00:10:55,955
just love to run macros,
they go ahead and run it.

238
00:10:55,955 --> 00:10:58,791
Now our attacker has
gained a foothold

239
00:10:58,791 --> 00:11:02,862
into the environment, but they
need elevated permissions.

240
00:11:02,862 --> 00:11:05,464
This scenario, we're
a little bit better

241
00:11:05,464 --> 00:11:07,466
at our credential hygienes,
we haven't exposed

242
00:11:07,466 --> 00:11:10,069
our Domain Admin creds
to this endpoint.

243
00:11:10,069 --> 00:11:14,540
So it's gonna cause a little
havoc, induce a Help Desk call.

244
00:11:14,540 --> 00:11:16,742
Our Help Desk just
wanting to do good here

245
00:11:16,742 --> 00:11:20,445
logs in with their admin
credentials on to this box,

246
00:11:20,446 --> 00:11:22,314
allowing our attacker
to gain access

247
00:11:22,314 --> 00:11:24,884
to a file server
that the Help Desk

248
00:11:24,884 --> 00:11:27,019
just happened to be
admin of as well.

249
00:11:27,019 --> 00:11:28,688
However, he was not
an Exchange admin,

250
00:11:28,688 --> 00:11:31,157
so we're kinda going in
a roundabout path here

251
00:11:31,157 --> 00:11:33,426
to make our way to
the Exchange server.

252
00:11:33,426 --> 00:11:36,462
Here our attacker finds a
misconfiguration in Exchange,

253
00:11:36,462 --> 00:11:38,664
he didn't have to exploit
any vulnerabilities here.

254
00:11:38,664 --> 00:11:41,167
He's able to directly
access the file system,

255
00:11:41,167 --> 00:11:44,570
and drop the web shell payload
on to the Exchange server.

256
00:11:50,543 --> 00:11:55,514
So from there, because Exchange
is 99% of a Domain Admin,

257
00:11:56,916 --> 00:11:58,384
he doesn't even need to bother
trying going after that.

258
00:11:58,384 --> 00:12:00,386
Remember, Domain Admin is
never the goal of the attacker,

259
00:12:00,386 --> 00:12:01,654
he's after the data.

260
00:12:01,654 --> 00:12:03,189
Exchange is a high-value target,

261
00:12:03,189 --> 00:12:05,790
it stores a lot of
information in it,

262
00:12:05,791 --> 00:12:08,094
and from there he can
modify other groups

263
00:12:08,094 --> 00:12:11,697
and objects in AD to
give himself access

264
00:12:11,697 --> 00:12:14,500
to these other sources of
data that they're truly after.

265
00:12:16,035 --> 00:12:18,603
So some time later,
our security team

266
00:12:18,604 --> 00:12:21,540
gets a virus alert
because one of the tools

267
00:12:21,540 --> 00:12:23,742
got uploaded to
VirusTotal and we finally

268
00:12:23,743 --> 00:12:25,211
got some signatures on it.

269
00:12:25,211 --> 00:12:27,379
So our security team comes in,

270
00:12:27,379 --> 00:12:30,015
and he just cleans up
everything that he got

271
00:12:30,015 --> 00:12:31,917
a virus alert on, because
they're not really doing

272
00:12:31,917 --> 00:12:34,320
any true threat hunting
here, they're just doing that

273
00:12:34,320 --> 00:12:37,723
reactive stuff where they're
just responding to alerts.

274
00:12:37,723 --> 00:12:39,992
Now you'll notice
that there's one thing

275
00:12:39,992 --> 00:12:43,728
on this slide that our
security guy didn't find,

276
00:12:43,729 --> 00:12:46,432
and it's that web shell
hiding on the Exchange server.

277
00:12:47,900 --> 00:12:49,635
Which brings us to our
last scenario here,

278
00:12:49,635 --> 00:12:51,470
which I call Return
of the Attacker.

279
00:12:51,470 --> 00:12:55,274
Because we missed the web
shell on the Exchange server,

280
00:12:55,274 --> 00:12:58,010
our attacker can
immediately come back in,

281
00:12:58,010 --> 00:13:01,313
and now he's a little mad
and wants better access

282
00:13:01,313 --> 00:13:03,649
than he had before, so he goes
after the Domain Admin creds

283
00:13:03,649 --> 00:13:07,153
this time and regains complete
control of the environment.

284
00:13:09,088 --> 00:13:12,925
- So, Josh gave some
really good scenarios there

285
00:13:12,925 --> 00:13:14,993
with the attack graphs.

286
00:13:14,994 --> 00:13:17,329
I'm gonna actually talk about
a little bit of the commands

287
00:13:17,329 --> 00:13:22,200
that we saw during my team's
research efforts on this,

288
00:13:22,201 --> 00:13:25,237
in regards to what
the adversary did

289
00:13:25,237 --> 00:13:28,274
with their access
to that network.

290
00:13:28,274 --> 00:13:32,044
So, I really, really wanted
to make a pew-pew map,

291
00:13:32,044 --> 00:13:36,382
so we're gonna start talking
about, geographically,

292
00:13:36,382 --> 00:13:40,218
where the actors were carrying
out these activities from.

293
00:13:40,219 --> 00:13:42,521
And it just gives a
little bit of a high-level

294
00:13:43,689 --> 00:13:46,158
insight into what
they were doing.

295
00:13:46,158 --> 00:13:50,162
So, going back to June 16, 2016-

296
00:13:50,162 --> 00:13:52,164
- Pew!
- I know this is,

297
00:13:52,164 --> 00:13:55,334
I know this is timely and
pertinent information,

298
00:13:55,334 --> 00:13:59,704
but back on June 16, we
saw the first activity

299
00:13:59,705 --> 00:14:02,641
from this specific
adversary on this network.

300
00:14:02,641 --> 00:14:06,846
And what they did was, you can
see that little orange server

301
00:14:06,846 --> 00:14:09,181
down in the bottom-right
corner here.

302
00:14:09,181 --> 00:14:11,584
That is our Exchange
server, okay?

303
00:14:11,584 --> 00:14:14,954
And what the adversary
did on June 16 was,

304
00:14:14,954 --> 00:14:17,256
they actually used
legitimate credentials

305
00:14:17,256 --> 00:14:21,260
to log in to Outlook,
specifically OWA.

306
00:14:21,260 --> 00:14:26,165
And they actually used
an IP address from Iran,

307
00:14:27,433 --> 00:14:28,701
from all places.

308
00:14:28,701 --> 00:14:29,869
- So this kinda lines
up with the scenarios

309
00:14:29,869 --> 00:14:31,303
that we're just going to.

310
00:14:31,303 --> 00:14:33,272
Obviously they had already
obtained credentials

311
00:14:33,272 --> 00:14:35,641
into the environment,
they weren't exploiting

312
00:14:35,641 --> 00:14:37,075
something on the
Exchange server first,

313
00:14:37,076 --> 00:14:39,311
they're using what
they got to get in.

314
00:14:39,311 --> 00:14:40,613
- Yup.

315
00:14:40,613 --> 00:14:43,415
The next day, they also
logged into the Exchange

316
00:14:43,415 --> 00:14:45,016
server from the same IP address,

317
00:14:46,185 --> 00:14:48,319
and then the next
day, on June 18,

318
00:14:48,320 --> 00:14:50,522
that's when we start
seeing this adversary

319
00:14:50,522 --> 00:14:53,425
using the TwoFace
web shell to interact

320
00:14:53,425 --> 00:14:55,895
with the Exchange
server specifically.

321
00:14:57,963 --> 00:15:00,266
Fast-forward a couple
months later in September,

322
00:15:00,266 --> 00:15:03,868
we saw this adversary
log in, I'm sorry,

323
00:15:03,869 --> 00:15:06,538
use TwoFace to run six commands

324
00:15:06,538 --> 00:15:08,741
from an IP address in France.

325
00:15:08,741 --> 00:15:13,746
And then again, from France,
running 16 commands in March.

326
00:15:14,914 --> 00:15:17,082
And then, I really wish
we had pew-pew noises.

327
00:15:17,082 --> 00:15:19,718
- Pew-pew!
- 61 commands they ran

328
00:15:19,718 --> 00:15:23,088
in April, and then
finally in May 2017,

329
00:15:23,088 --> 00:15:26,425
we saw the last activity,
which was 10 commands

330
00:15:26,425 --> 00:15:28,193
run from an IP
address in Germany.

331
00:15:28,193 --> 00:15:29,962
So you can see that
this threat group

332
00:15:29,962 --> 00:15:33,598
has at least the
foresight to know that

333
00:15:33,599 --> 00:15:35,401
some sort of forensic
analyst is gonna see

334
00:15:35,401 --> 00:15:38,170
where all these activities
are coming from,

335
00:15:38,170 --> 00:15:41,940
and to try to cloak where
they're originating from.

336
00:15:41,941 --> 00:15:43,943
So, let's kinda
dig into each one

337
00:15:43,943 --> 00:15:45,644
of those dotted lines there,

338
00:15:45,644 --> 00:15:48,213
what they did with
their access, okay?

339
00:15:48,213 --> 00:15:51,250
So, like I said, on
June 16, they used

340
00:15:51,250 --> 00:15:54,119
legitimate credentials
to log into OWA.

341
00:15:54,119 --> 00:15:55,554
So what did they do?

342
00:15:55,554 --> 00:15:57,856
So they immediately opened
up the address book,

343
00:15:57,856 --> 00:16:01,961
and they looked through 15
different distribution lists,

344
00:16:01,961 --> 00:16:04,096
and over the course
of a couple hours

345
00:16:04,096 --> 00:16:07,266
they looked through 81
different user accounts, okay?

346
00:16:07,266 --> 00:16:09,968
So that's pretty interesting,
why did they do that?

347
00:16:09,969 --> 00:16:13,038
Well, what they
did was, they used

348
00:16:13,038 --> 00:16:14,974
one of those
distribution lists that

349
00:16:14,974 --> 00:16:17,910
at this organization was
called Network Support,

350
00:16:17,910 --> 00:16:20,179
and they sent an email to them.

351
00:16:20,179 --> 00:16:23,182
The subject was
Network Team Support.

352
00:16:23,182 --> 00:16:25,017
Now, we don't know what
the attachment was,

353
00:16:25,017 --> 00:16:27,686
and we don't know what the
content of the message was,

354
00:16:27,686 --> 00:16:30,089
but if you think about
that distribution list

355
00:16:30,089 --> 00:16:32,758
and the people that are probably
on that distribution list,

356
00:16:32,758 --> 00:16:34,727
they probably have some
pretty decent access

357
00:16:34,727 --> 00:16:36,228
to the rest of the network.

358
00:16:36,228 --> 00:16:38,363
- [Josh] What better way
than to phish your target

359
00:16:38,364 --> 00:16:41,100
by making the email come
from with inside the domain?

360
00:16:41,100 --> 00:16:43,202
- [Robert] Yep, pretty clever.

361
00:16:43,202 --> 00:16:45,204
So unfortunately, we don't
know how they did it,

362
00:16:45,204 --> 00:16:47,506
but that is likely what
they did right here.

363
00:16:48,907 --> 00:16:52,311
So, I said again they logged
into the Exchange server

364
00:16:52,311 --> 00:16:55,781
the next day, but this time
they didn't do anything

365
00:16:55,781 --> 00:16:57,215
with emails or
anything like that.

366
00:16:57,216 --> 00:16:59,585
They actually went
and downloaded

367
00:16:59,585 --> 00:17:01,854
what they called the offline
address book from Outlook.

368
00:17:01,854 --> 00:17:04,256
So they used Outlook to
just download this file

369
00:17:05,424 --> 00:17:06,992
that contains the address book,

370
00:17:06,992 --> 00:17:11,730
probably for some offline
processing or data retention,

371
00:17:11,730 --> 00:17:13,098
so they don't have
to keep interacting

372
00:17:13,098 --> 00:17:15,800
with the Exchange server
to look through these-

373
00:17:15,800 --> 00:17:17,569
- [Josh] Lot of
valuable information

374
00:17:17,569 --> 00:17:20,772
they can get out of here,
so it's gonna have not only

375
00:17:20,772 --> 00:17:23,574
the email addresses of
everybody in the organization,

376
00:17:23,575 --> 00:17:25,310
it's also gonna have
any external contacts

377
00:17:25,310 --> 00:17:26,677
they had stored in there.

378
00:17:26,678 --> 00:17:31,183
But phone numbers,
addresses, office locations,

379
00:17:31,183 --> 00:17:33,585
and possibly some of their
organization structure.

380
00:17:33,585 --> 00:17:35,087
They can see who's
a manager of who

381
00:17:35,087 --> 00:17:37,856
depending on what
attributes they've populated

382
00:17:37,856 --> 00:17:39,324
on these accounts.

383
00:17:39,324 --> 00:17:40,993
- [Robert] Yeah, and imagine
doing data exfiltration

384
00:17:40,993 --> 00:17:43,227
as an actor using
something like Outlook,

385
00:17:43,228 --> 00:17:44,730
that's pretty awesome.

386
00:17:46,165 --> 00:17:48,834
So, the next day is when
we saw the first activity

387
00:17:48,834 --> 00:17:50,702
using this web shell, okay?

388
00:17:50,702 --> 00:17:53,038
So, remember I said
that there was a loader

389
00:17:53,038 --> 00:17:55,641
and a payload layer to TwoFace.

390
00:17:55,641 --> 00:17:58,143
In the top-left
corner of this slide

391
00:17:58,143 --> 00:18:02,114
you can see this
URL, error2.aspx

392
00:18:02,114 --> 00:18:04,216
is the file that they're
interacting with.

393
00:18:04,216 --> 00:18:06,752
- [Josh] That's a
legitimate page in Exchange,

394
00:18:06,752 --> 00:18:10,254
that's the actual error page
that Exchange 2010 uses.

395
00:18:10,255 --> 00:18:13,625
So they're appending the
code to a legitimate file

396
00:18:13,625 --> 00:18:15,928
on the server to
act as their loader.

397
00:18:15,928 --> 00:18:18,397
- [Robert] Yep, and then what
they're doing with that loader

398
00:18:18,397 --> 00:18:21,300
is they're using it to install
the payload at error3.aspx,

399
00:18:22,701 --> 00:18:25,938
and, I mean, if there's an
error2.aspx that's legitimate,

400
00:18:25,938 --> 00:18:30,943
there's probably an error3.aspx
legitimate file as well.

401
00:18:32,377 --> 00:18:33,611
- [Josh] Right, if you're
not paying close attention,

402
00:18:33,612 --> 00:18:34,780
this is something
that you might miss.

403
00:18:34,780 --> 00:18:36,248
Especially if
you're not familiar

404
00:18:36,248 --> 00:18:37,749
with the application itself and
what's supposed to be there.

405
00:18:37,749 --> 00:18:39,318
- [Robert] Error3
doesn't exist, does it?

406
00:18:39,318 --> 00:18:40,786
- It does not exist.
- Okay, good.

407
00:18:40,786 --> 00:18:44,223
So, with this access,
they run one command,

408
00:18:44,223 --> 00:18:47,693
and one command alone, and
it's this whoami command.

409
00:18:47,693 --> 00:18:49,561
So we actually think
that they just did this

410
00:18:49,561 --> 00:18:52,997
to see what account
they had access to,

411
00:18:52,998 --> 00:18:56,602
or if the TwoFace payload-
- Fun fact, that account

412
00:18:56,602 --> 00:18:58,070
would be system.
- Yes, it is

413
00:18:58,070 --> 00:18:58,904
definitely system.
- 'Cause this runs

414
00:18:58,904 --> 00:19:00,405
in the system contacts.

415
00:19:00,405 --> 00:19:04,776
- [Robert] So, now, they
know that their web shell

416
00:19:04,776 --> 00:19:08,280
is successfully deployed, and
they don't do anything else,

417
00:19:08,280 --> 00:19:11,049
they just kind of,
they don't do anything.

418
00:19:11,049 --> 00:19:14,052
So fast forward about two
months, two and a half months,

419
00:19:14,052 --> 00:19:17,356
we start seeing this
adversary use the web shell

420
00:19:17,356 --> 00:19:19,525
to do some more
interesting stuff, okay?

421
00:19:19,525 --> 00:19:21,593
So the first that
they do is they upload

422
00:19:21,593 --> 00:19:25,898
this executable, m64.exe,
using the web shell,

423
00:19:25,898 --> 00:19:29,101
and then they run it with
very specific parameters.

424
00:19:29,101 --> 00:19:31,136
Now, if you look at
those parameters,

425
00:19:31,136 --> 00:19:32,538
you can probably
tell what this is.

426
00:19:32,538 --> 00:19:35,174
This is Mimikatz.

427
00:19:35,174 --> 00:19:39,711
- [Josh] Yep, and if you've
never seen somebody run Mimikatz

428
00:19:39,711 --> 00:19:42,047
on an Exchange
server specifically,

429
00:19:42,047 --> 00:19:44,950
if run and running
OWA, it's pretty much

430
00:19:44,950 --> 00:19:47,586
like cracking open
a credential pinata.

431
00:19:47,586 --> 00:19:50,222
There's credentials spilling
out all over the place.

432
00:19:50,222 --> 00:19:52,357
- [Robert] Yep, so,
and what they did was

433
00:19:52,357 --> 00:19:55,961
they outputted this information
into a text file, 01.txt,

434
00:19:55,961 --> 00:19:59,398
and then they exfiltrate
that data using the web shell

435
00:19:59,398 --> 00:20:02,033
by using the type command.

436
00:20:02,034 --> 00:20:05,070
And in Windows, that just
displays the contents of a file,

437
00:20:05,070 --> 00:20:06,705
they copy-and-paste it
from their web shell

438
00:20:06,705 --> 00:20:08,639
to their own system,
and they've exfiltrated

439
00:20:08,640 --> 00:20:12,211
the credentials, so
it's pretty clever.

440
00:20:12,211 --> 00:20:14,112
Then they clean up after
themselves by just deleting

441
00:20:14,112 --> 00:20:17,316
the text file, the executable,
and the TwoFace payload

442
00:20:17,316 --> 00:20:19,384
from the server to
clean up their tracks.

443
00:20:20,819 --> 00:20:25,224
Okay, jump six months into
the future to March 3, 2017.

444
00:20:26,391 --> 00:20:27,793
This is where we
start seeing them

445
00:20:27,793 --> 00:20:29,895
doing some pivoting activity
using the web shell.

446
00:20:29,895 --> 00:20:31,230
So the first thing that they do

447
00:20:31,230 --> 00:20:32,831
is they run this
net group command

448
00:20:32,831 --> 00:20:35,667
looking for Exchange
Trusted Subsystem.

449
00:20:35,667 --> 00:20:37,735
- [Josh] And that's a
really powerful group.

450
00:20:37,736 --> 00:20:41,173
That is what makes an Exchange
server pretty much 99%

451
00:20:41,173 --> 00:20:43,709
of a Domain Admin,
that group has rights

452
00:20:43,709 --> 00:20:47,246
to do almost anything
in Active Directory.

453
00:20:47,246 --> 00:20:50,415
It typically is only gonna
have the Exchange server

454
00:20:50,415 --> 00:20:52,251
computer objects as a member.

455
00:20:52,251 --> 00:20:54,485
If you see anything else besides

456
00:20:54,486 --> 00:20:57,389
an Exchange server computer
object in that group,

457
00:20:57,389 --> 00:20:58,889
that means something's wrong.

458
00:20:58,890 --> 00:21:03,629
Sometimes organizations will
put people in there by mistake,

459
00:21:04,763 --> 00:21:05,297
but definitely
worth investigate.

460
00:21:06,265 --> 00:21:08,967
- So what that command does,

461
00:21:08,967 --> 00:21:11,937
is it gives you a list
of all of the hostnames

462
00:21:11,937 --> 00:21:14,305
for Exchange servers
on the network, okay?

463
00:21:14,306 --> 00:21:15,641
So what the actor did after that

464
00:21:15,641 --> 00:21:18,477
was they just started
trying to find

465
00:21:18,477 --> 00:21:21,480
where Outlook's
content is being hosted

466
00:21:21,480 --> 00:21:23,415
for OWA and those
kind of services

467
00:21:23,415 --> 00:21:25,684
on these remote
Exchange servers.

468
00:21:25,684 --> 00:21:27,386
- You can kind of
tell that they knew

469
00:21:27,386 --> 00:21:29,388
exactly what they
were looking for here,

470
00:21:29,388 --> 00:21:33,158
because they're enumerating
a very specific path.

471
00:21:33,158 --> 00:21:36,128
This indicates that their
target was Exchange 2010.

472
00:21:36,128 --> 00:21:37,763
- Yep, and so what they did was,

473
00:21:37,763 --> 00:21:41,399
when they found that path,
they uploaded another web shell

474
00:21:41,400 --> 00:21:44,536
to the current web shell
called Exchange.aspx.

475
00:21:44,536 --> 00:21:46,538
Then what they did was,
they just timestomped it

476
00:21:46,538 --> 00:21:48,740
to match the same timestamps

477
00:21:48,740 --> 00:21:51,910
as a legitimate
file in Exchange.

478
00:21:51,910 --> 00:21:54,713
And this one specifically
was Exchange.asmx.

479
00:21:54,713 --> 00:21:56,715
- Yeah, asmx is a legit file,

480
00:21:56,715 --> 00:21:58,149
that's where the
Exchange web service is,

481
00:21:58,150 --> 00:22:01,086
kind of like an API
interface for Exchange.

482
00:22:01,086 --> 00:22:03,689
And, pretty clever,
it's one letter off

483
00:22:03,689 --> 00:22:06,425
in the extension, so again,
something you might miss,

484
00:22:06,425 --> 00:22:09,293
especially if you're just
looking through some log files.

485
00:22:09,294 --> 00:22:12,331
- Yep, so, after
they timestomped it,

486
00:22:12,331 --> 00:22:13,932
what they did was that's
when they started doing

487
00:22:13,932 --> 00:22:15,334
their lateral movement.

488
00:22:15,334 --> 00:22:18,403
They used the copy command,
that's all they did,

489
00:22:18,403 --> 00:22:20,339
they just copied
that new web shell

490
00:22:20,339 --> 00:22:23,308
over to the other
Exchange servers

491
00:22:23,308 --> 00:22:26,378
that were, which
were discovered using

492
00:22:26,378 --> 00:22:28,447
that net group exchange
Trusted Subsystem.

493
00:22:28,447 --> 00:22:30,748
- Yep, and that Exchange
Trusted Subsystem

494
00:22:30,749 --> 00:22:34,519
has to be a local administrator
on every Exchange server

495
00:22:34,519 --> 00:22:35,921
in order for
Exchange to function,

496
00:22:35,921 --> 00:22:37,456
so that's how
they're very easily

497
00:22:37,456 --> 00:22:39,390
able to laterally move between

498
00:22:39,391 --> 00:22:41,026
all the Exchange servers
in the environment.

499
00:22:41,026 --> 00:22:42,494
- Yep.

500
00:22:42,494 --> 00:22:44,730
Finally, before they ended
their activity here in March,

501
00:22:44,730 --> 00:22:46,765
they just delete the
payload, the TwoFace payload,

502
00:22:46,765 --> 00:22:47,632
and they move on.

503
00:22:49,468 --> 00:22:51,903
They come back a couple
months later in April,

504
00:22:51,903 --> 00:22:53,605
and what we think happened was,

505
00:22:53,605 --> 00:22:56,775
they lost access to
one of the web shells

506
00:22:56,775 --> 00:22:58,844
that's hosted on
the Exchange server.

507
00:22:58,844 --> 00:23:01,546
And what they did
was, they came back,

508
00:23:01,546 --> 00:23:04,950
they ran that same net group
Exchange Trusted Subsystem

509
00:23:04,950 --> 00:23:08,387
command to find all of the
Exchange servers on the network,

510
00:23:08,387 --> 00:23:10,121
and then they run
the hostname command

511
00:23:10,122 --> 00:23:11,757
to figure out where they're at.

512
00:23:11,757 --> 00:23:13,225
Which one of those
Exchange servers

513
00:23:13,225 --> 00:23:15,092
that they're currently
interacting with.

514
00:23:15,093 --> 00:23:16,595
- [Josh] Yeah,
this could've been

515
00:23:16,595 --> 00:23:18,397
because of like we showed
in the scenario earlier,

516
00:23:18,397 --> 00:23:22,533
that maybe our security
personnel at the
target organization

517
00:23:22,534 --> 00:23:26,505
found something and cleaned
up the original web shell,

518
00:23:26,505 --> 00:23:28,573
but more than likely
it could have been just

519
00:23:28,573 --> 00:23:30,375
that they had to
replace the server

520
00:23:30,375 --> 00:23:32,244
because they had a
hardware malfunction

521
00:23:32,244 --> 00:23:33,712
or something else
that caused them

522
00:23:33,712 --> 00:23:35,247
to replace the Exchange server.

523
00:23:35,247 --> 00:23:37,682
- [Robert] Yep, so, after
that what they start doing

524
00:23:37,682 --> 00:23:40,752
is they start trying
to list the contents

525
00:23:40,752 --> 00:23:44,089
of that specific
path within Exchange

526
00:23:44,089 --> 00:23:47,692
on the other four Exchange
servers on the network.

527
00:23:47,692 --> 00:23:50,529
And it's telling that
they probably lost access

528
00:23:50,529 --> 00:23:53,732
to one of those
with these commands.

529
00:23:53,732 --> 00:23:57,536
They immediately try to
ping server number three,

530
00:23:57,536 --> 00:24:00,505
and we don't see the
output of this obviously,

531
00:24:00,505 --> 00:24:03,542
but it's highly likely
that that didn't succeed.

532
00:24:03,542 --> 00:24:05,110
'Cause then, the
next thing they do,

533
00:24:05,110 --> 00:24:07,178
is they try to list the contents

534
00:24:07,179 --> 00:24:10,582
or try to access the C: drive
and D: drive on that server,

535
00:24:10,582 --> 00:24:13,318
and they weren't
successful, okay?

536
00:24:13,318 --> 00:24:17,856
So, what did the actor
do in response to this?

537
00:24:17,856 --> 00:24:21,393
They uploaded yet another
web shell to this web shell,

538
00:24:21,393 --> 00:24:23,895
and this global.aspx
that they uploaded

539
00:24:23,895 --> 00:24:26,498
was just another
variant of TwoFace.

540
00:24:26,498 --> 00:24:31,503
And they used the attrib
command to hide it,

541
00:24:32,370 --> 00:24:33,839
and then they copied it over

542
00:24:33,839 --> 00:24:35,373
to all of the Exchange servers
they still had access to.

543
00:24:35,373 --> 00:24:37,676
- [Josh] That global.aspx
is not unique to Exchange,

544
00:24:37,676 --> 00:24:39,878
it's actually a part of IIS,

545
00:24:39,878 --> 00:24:42,581
it's typically a
hidden file that holds

546
00:24:42,581 --> 00:24:43,747
configuration information.

547
00:24:43,748 --> 00:24:45,650
It's not always present,
and it's typically

548
00:24:45,650 --> 00:24:48,320
not accessible, it's not
served by the web servers.

549
00:24:48,320 --> 00:24:50,522
You can't access it
through a browser,

550
00:24:50,522 --> 00:24:52,724
generally only
through a file system.

551
00:24:52,724 --> 00:24:54,359
- [Robert] So while
they're on the web server,

552
00:24:54,359 --> 00:24:58,129
they uploaded another
executable, mom.64,

553
00:24:58,129 --> 00:25:00,599
which, again, is Mimikatz.

554
00:25:00,599 --> 00:25:03,534
They exfiltrated the output
by using the type command,

555
00:25:03,535 --> 00:25:05,003
and then deleted
a bunch of files

556
00:25:05,003 --> 00:25:06,504
to clean up after themselves,

557
00:25:06,505 --> 00:25:07,973
including the text file,
including the executable,

558
00:25:07,973 --> 00:25:11,209
and the TwoFace payload
web shell itself.

559
00:25:12,611 --> 00:25:14,513
Okay, so this is the last
activity that we saw,

560
00:25:14,513 --> 00:25:17,148
was in May 2017.

561
00:25:17,148 --> 00:25:21,086
And we believe we lost
visibility on to this

562
00:25:21,086 --> 00:25:24,022
because the organization
that was impacted

563
00:25:24,022 --> 00:25:26,191
started doing remediation,
and they actually

564
00:25:26,191 --> 00:25:28,159
caught these guys and was able

565
00:25:28,159 --> 00:25:30,228
to successfully kick
them off their network.

566
00:25:30,228 --> 00:25:33,498
But what we saw them do
before that happened was,

567
00:25:33,498 --> 00:25:36,801
they immediately tried to
ping an external IP address,

568
00:25:36,801 --> 00:25:39,303
4.2.2.4, and we think
that they were doing that

569
00:25:39,304 --> 00:25:43,408
to see if they could establish
outbound network connections.

570
00:25:43,408 --> 00:25:45,443
And immediately after
that, they tried to find

571
00:25:45,443 --> 00:25:47,245
Domain Admins on the
network by running

572
00:25:47,245 --> 00:25:49,113
this net group command.

573
00:25:49,114 --> 00:25:52,017
And again, they upload
another executable,

574
00:25:52,017 --> 00:25:53,852
and this one's
MicrosoftUpdate.exe.

575
00:25:53,852 --> 00:25:55,887
- [Josh] Yeah, obviously,
these are good guys now.

576
00:25:55,887 --> 00:25:57,489
They're trying to patch
this server for them,

577
00:25:57,489 --> 00:25:59,858
because they feel
bad for their target.

578
00:25:59,858 --> 00:26:01,259
- Yeah, and they're also dumping

579
00:26:01,259 --> 00:26:03,695
the credentials while
they're doing that.

580
00:26:03,695 --> 00:26:05,263
One other thing that I
should mention, though,

581
00:26:05,263 --> 00:26:09,266
and it may have been
kind of lost over,

582
00:26:09,267 --> 00:26:11,570
is the names of
the Mimikatz tools

583
00:26:11,570 --> 00:26:12,904
that they're uploading
kind of blend in

584
00:26:12,904 --> 00:26:14,873
to other Microsoft
executables as well.

585
00:26:14,873 --> 00:26:18,109
So they're trying
their best to hide

586
00:26:18,109 --> 00:26:19,578
while they're doing all this.

587
00:26:19,578 --> 00:26:21,980
So they run that executable,

588
00:26:21,980 --> 00:26:24,516
and you can see that the
parameters change a little bit,

589
00:26:24,516 --> 00:26:26,184
but again, this is Mimikatz.

590
00:26:26,184 --> 00:26:28,787
They're outputting the
credentials to mic.txt,

591
00:26:28,787 --> 00:26:31,389
and then exfiltrating it
with the type command.

592
00:26:31,389 --> 00:26:33,491
So you can start seeing
that there's a clear trend

593
00:26:33,491 --> 00:26:34,693
happening here.

594
00:26:34,693 --> 00:26:36,661
Every time they
access the web shell

595
00:26:36,661 --> 00:26:37,996
they're dumping
credentials to see

596
00:26:37,996 --> 00:26:39,364
if they got anything new,

597
00:26:39,364 --> 00:26:42,734
and to expand their
presence on the network.

598
00:26:44,569 --> 00:26:46,870
- So now that we've
had our bad guy hat on,

599
00:26:46,871 --> 00:26:48,239
let's put our good guy hat on,

600
00:26:48,239 --> 00:26:50,875
and take a look at
how we would examine

601
00:26:50,875 --> 00:26:54,913
this type of stuff on the
Exchange server itself.

602
00:26:54,913 --> 00:26:57,215
The first thing I
need to do is find out

603
00:26:57,215 --> 00:26:58,882
where the logs are at.

604
00:26:58,883 --> 00:27:01,219
If I'm coming in
and I'm not familiar

605
00:27:01,219 --> 00:27:04,589
with the Exchange environment
where I'm doing my hunt,

606
00:27:04,589 --> 00:27:08,158
I can whip up the Exchange
management shell here,

607
00:27:08,159 --> 00:27:09,728
just a PowerShell module,

608
00:27:09,728 --> 00:27:11,963
and run the command here
in the first example.

609
00:27:11,963 --> 00:27:15,200
That's gonna tell me where all
of the Exchange servers are,

610
00:27:15,200 --> 00:27:17,836
specifically the ones that have
the Client Access role on it

611
00:27:17,836 --> 00:27:19,971
which is gonna be what runs OWA.

612
00:27:19,971 --> 00:27:22,574
So I know how to focus my hunt.

613
00:27:22,574 --> 00:27:25,877
The second command
here is gonna show me

614
00:27:25,877 --> 00:27:28,780
where the IIS logs are stored.

615
00:27:28,780 --> 00:27:31,649
Now, they're generally gonna
be in the default location,

616
00:27:31,650 --> 00:27:33,652
but this will tell
you where they're at

617
00:27:33,652 --> 00:27:38,657
if they've been moved, just
quick to run and find out.

618
00:27:39,524 --> 00:27:40,958
So once I've gathered my logs,

619
00:27:40,959 --> 00:27:44,829
I need to look for some
indicators within those logs.

620
00:27:44,829 --> 00:27:48,733
And our first big giveaway
that there's something

621
00:27:48,733 --> 00:27:50,301
fishy going on here,

622
00:27:50,301 --> 00:27:54,406
is POST operations that
have a low RequestCount.

623
00:27:55,573 --> 00:27:58,309
There are very few pages
on an Exchange server

624
00:27:58,309 --> 00:28:00,278
that are gonna
have POST, period,

625
00:28:00,278 --> 00:28:03,648
and those are gonna
have a very high count

626
00:28:03,648 --> 00:28:06,551
because you're gonna have
normal day-to-day users

627
00:28:06,551 --> 00:28:09,854
accessing it on a regular
basis, issuing that POST to it.

628
00:28:09,854 --> 00:28:12,257
- [Robert] And if you think
about from my pew-pew map,

629
00:28:12,257 --> 00:28:14,559
there weren't a lot of
commands that were being run

630
00:28:14,559 --> 00:28:16,428
each time they
visited the web shell,

631
00:28:16,428 --> 00:28:19,531
so it's likely those
would be kind of exposed.

632
00:28:19,531 --> 00:28:22,900
I mean, 61 commands is gonna
be 61 POST requests, right?

633
00:28:22,901 --> 00:28:24,335
And that's probably
gonna stand out

634
00:28:24,335 --> 00:28:27,839
when you're looking at
other legitimate files

635
00:28:27,839 --> 00:28:30,375
that are handling POST
requests in Exchange,

636
00:28:30,375 --> 00:28:31,743
that are gonna have
lots and lots and lots

637
00:28:31,743 --> 00:28:33,278
and thousands of requests.

638
00:28:33,278 --> 00:28:35,747
- [Josh] Now, you
might run into a few

639
00:28:35,747 --> 00:28:37,415
false positives here and there,

640
00:28:37,415 --> 00:28:40,085
because it's case-sensitive
when it logs,

641
00:28:40,085 --> 00:28:42,520
so if you have somebody
that types the URL in

642
00:28:42,520 --> 00:28:47,525
in a weird way, they might
trigger a POST count for that,

643
00:28:48,660 --> 00:28:50,895
but those will be very
easy to filter out

644
00:28:50,895 --> 00:28:52,530
when you're looking
at this here.

645
00:28:52,530 --> 00:28:54,999
So the next thing that
we want to look at,

646
00:28:54,999 --> 00:28:57,068
will help us focus our hunt,

647
00:28:57,068 --> 00:29:00,605
is to pay close
attention to URIs

648
00:29:00,605 --> 00:29:02,340
that don't require
authentication.

649
00:29:02,340 --> 00:29:04,641
If I'm attacker
putting a web shell

650
00:29:04,642 --> 00:29:07,912
on any kind of web server here,

651
00:29:07,912 --> 00:29:12,917
I don't want to be blocked
by having to authenticate

652
00:29:12,917 --> 00:29:14,619
before I can load the page.

653
00:29:14,619 --> 00:29:17,655
I wanna be able to
access it with anonymous

654
00:29:17,655 --> 00:29:19,591
so that it comes
right up every time.

655
00:29:21,092 --> 00:29:23,962
And then finally, you
wanna look at GET requests

656
00:29:23,962 --> 00:29:26,164
that fail, 404 errors.

657
00:29:26,164 --> 00:29:29,234
Now, there's bound to
be a few here and there,

658
00:29:29,234 --> 00:29:33,905
typically very low on static
applications like Exchange.

659
00:29:33,905 --> 00:29:35,572
But the reason I
have this up here is,

660
00:29:35,573 --> 00:29:40,512
last year I was investigating
another web shell,

661
00:29:40,512 --> 00:29:44,482
and the attackers
had actually modified

662
00:29:44,482 --> 00:29:47,585
the logging configuration
on the Exchange server

663
00:29:47,585 --> 00:29:50,588
so that it filtered
access to the web shell

664
00:29:50,588 --> 00:29:52,123
out of the logs.

665
00:29:52,123 --> 00:29:54,159
Now they hid it in
that same error2.aspx,

666
00:29:55,293 --> 00:29:57,395
which I would expect
somebody would run into

667
00:29:57,395 --> 00:29:59,631
an error eventually, right?

668
00:29:59,631 --> 00:30:01,699
You'd have a legitimate
entry in the log

669
00:30:01,699 --> 00:30:03,001
for the error page.

670
00:30:03,001 --> 00:30:05,703
So what stood out is
there were no entries

671
00:30:05,703 --> 00:30:07,906
for the error page at all.

672
00:30:07,906 --> 00:30:11,309
So I flipped over and
looked at the 404 errors,

673
00:30:11,309 --> 00:30:14,445
and now, because attackers
are humans just like us

674
00:30:14,445 --> 00:30:17,715
and make mistakes,
I saw the attacker

675
00:30:17,715 --> 00:30:21,319
making typos trying to
access their web shell.

676
00:30:21,319 --> 00:30:23,053
That gave 'em away.

677
00:30:23,054 --> 00:30:25,456
So once I've gathered
this information,

678
00:30:25,456 --> 00:30:29,561
the next thing I wanna look
at is the User Agent String.

679
00:30:29,561 --> 00:30:31,361
Now, even in very
large environments

680
00:30:31,362 --> 00:30:33,097
where you have hundreds
of thousands of users

681
00:30:33,097 --> 00:30:36,100
accessing this
all day and night,

682
00:30:36,100 --> 00:30:39,971
the attacker's User Agent
String tends to stand out.

683
00:30:39,971 --> 00:30:43,408
This is because
they often use it

684
00:30:43,408 --> 00:30:47,078
similar to the 64-byte key
that Robert was talking out,

685
00:30:47,078 --> 00:30:50,447
it's a access method
to the web shell.

686
00:30:50,448 --> 00:30:53,918
If I connect and my
User Agent String

687
00:30:53,918 --> 00:30:56,787
doesn't match the one that
I programmed into the shell,

688
00:30:56,788 --> 00:30:58,289
I don't see the shell.

689
00:30:58,289 --> 00:31:00,925
This prevents hunters,
legitimate users,

690
00:31:00,925 --> 00:31:03,861
from accidentally stumbling
upon the web shell.

691
00:31:03,862 --> 00:31:06,364
- [Robert] Yeah, and I can
speak to the User Agent

692
00:31:06,364 --> 00:31:08,199
specifically to
this threat group.

693
00:31:08,199 --> 00:31:11,302
They love really old
versions of Firefox.

694
00:31:11,302 --> 00:31:14,205
Like, many many years
old versions of Firefox.

695
00:31:14,205 --> 00:31:16,608
So they kinda stand
out when you're doing

696
00:31:16,608 --> 00:31:18,543
this kind of hunting.
- And remember,

697
00:31:18,543 --> 00:31:20,812
User Agent Strings
are very easily faked.

698
00:31:21,913 --> 00:31:23,648
So, pay close
attention to those.

699
00:31:23,648 --> 00:31:26,016
Once you've gathered
your User Agent String,

700
00:31:26,017 --> 00:31:27,986
you can use that
to start tracking

701
00:31:27,986 --> 00:31:29,687
some of the attacker's activity.

702
00:31:29,687 --> 00:31:32,223
So here I'm showing, we
have the User Agent String

703
00:31:32,223 --> 00:31:33,758
that connected to the web shell,

704
00:31:33,758 --> 00:31:37,729
and then immediately after,
we see the User Agent String

705
00:31:37,729 --> 00:31:41,299
accessing OWA itself
and the user account

706
00:31:41,299 --> 00:31:43,668
that they used to log into OWA.

707
00:31:43,668 --> 00:31:46,404
So now I know my attacker
is using the credentials

708
00:31:46,404 --> 00:31:50,375
that they stole to access
this mailbox specifically.

709
00:31:50,375 --> 00:31:53,745
Now, there may be a
chance that you could

710
00:31:53,745 --> 00:31:55,479
mix up some legitimate use,

711
00:31:55,480 --> 00:31:58,650
but to make yourself
extra certain

712
00:31:58,650 --> 00:32:01,119
that you are tracking
the attacker activity,

713
00:32:01,119 --> 00:32:04,422
you can add in looking
at the ClientId field.

714
00:32:04,422 --> 00:32:06,090
This is in Exchange
2013 and later.

715
00:32:06,090 --> 00:32:08,426
In earlier versions,
it was session ID.

716
00:32:08,426 --> 00:32:11,995
This ID is a unique GUID
that is a server-side

717
00:32:11,996 --> 00:32:15,199
representation of the cookie
that the client generates,

718
00:32:15,199 --> 00:32:17,769
so it's persistent and unique

719
00:32:17,769 --> 00:32:20,505
to the session
that initiated it.

720
00:32:20,505 --> 00:32:23,341
So using that, we'll
go in and dig through

721
00:32:23,341 --> 00:32:27,645
and find out, okay, what
accounts did my attacker access?

722
00:32:27,645 --> 00:32:30,148
I know that I need to put
those on my remediation list

723
00:32:30,148 --> 00:32:31,515
and do something about it.

724
00:32:31,516 --> 00:32:34,719
Might also help us track
what type of information

725
00:32:34,719 --> 00:32:37,288
that they may have taken
out of these mailboxes.

726
00:32:41,326 --> 00:32:44,028
One other method, I actually
introduced this last year,

727
00:32:44,028 --> 00:32:46,898
it's a PowerShell script
that's basically a wrapper

728
00:32:46,898 --> 00:32:49,600
around Jared Atkinson's
PowerForensics,

729
00:32:49,600 --> 00:32:53,170
comparing the filetime
in the master file table.

730
00:32:53,171 --> 00:32:56,607
So what this does is it
uses the install time

731
00:32:56,607 --> 00:32:58,176
on the Active
Directory attribute

732
00:32:58,176 --> 00:33:00,712
of the Exchange
server as a baseline.

733
00:33:00,712 --> 00:33:02,580
This is the time
that all the files

734
00:33:02,580 --> 00:33:04,314
in the install should match.

735
00:33:04,315 --> 00:33:06,918
If they don't match,
throw me an alert,

736
00:33:06,918 --> 00:33:08,119
as you see here, saying that,

737
00:33:08,119 --> 00:33:10,521
hey, this file looks
like it doesn't belong.

738
00:33:10,521 --> 00:33:14,225
It works in 2013 and
later only because

739
00:33:14,225 --> 00:33:16,327
every update is a full install,

740
00:33:16,327 --> 00:33:18,930
so every file will
have the same file time

741
00:33:18,930 --> 00:33:21,199
except for anything
that doesn't belong.

742
00:33:24,502 --> 00:33:27,405
And with that, I have
one last graph for you.

743
00:33:27,405 --> 00:33:31,509
So we use everything that
we learned here today.

744
00:33:31,509 --> 00:33:34,679
Here our environment is
already completely compromised.

745
00:33:34,679 --> 00:33:37,882
Our security guy asked
for our DFIR help,

746
00:33:37,882 --> 00:33:41,352
our DFIR expert comes in and
they start their investigation

747
00:33:41,352 --> 00:33:42,820
with the Exchange server.

748
00:33:42,820 --> 00:33:45,822
Sometimes you go in for a hunt
and it might be proactive,

749
00:33:45,823 --> 00:33:47,959
you don't know where to start.

750
00:33:47,959 --> 00:33:49,460
Exchange is a high-value target,

751
00:33:49,460 --> 00:33:51,162
it's a really easy
place to start,

752
00:33:51,162 --> 00:33:53,765
it's a really easy place
to knock off your list.

753
00:33:53,765 --> 00:33:56,567
As you can see, you can just
filter through some logs,

754
00:33:56,567 --> 00:33:59,237
run this script, easily find
what you're looking for.

755
00:34:00,104 --> 00:34:02,607
So now our DFIR hero comes in,

756
00:34:02,607 --> 00:34:05,510
finds those unusual
POST operations we
were talking about,

757
00:34:05,510 --> 00:34:07,779
knows that this looks
like a web shell.

758
00:34:08,913 --> 00:34:10,048
They go ahead and
clean the web shell up

759
00:34:10,047 --> 00:34:11,982
and start tracing the activity

760
00:34:11,983 --> 00:34:14,252
of the attacker through
the environment,

761
00:34:14,252 --> 00:34:16,087
cleaning up things and
getting the attacker out

762
00:34:16,087 --> 00:34:17,021
as we go along.

763
00:34:20,257 --> 00:34:22,125
So with that, thank you all.

764
00:34:22,126 --> 00:34:23,594
- [Robert] Okay, are we all god?

765
00:34:23,594 --> 00:34:27,565
Alright, thanks everyone!
(audience applauds)

766
00:34:27,565 --> 00:34:30,967
(mysterious piano music)

767
00:34:40,445 --> 00:34:44,482
(sibilant white noise
crashes and fades out)