Upcoming Blue Team Series on Incident Response, Threat Hunting, and Purple Teaming

At Cincinnati B-Sides yesterday, I had some conversations about

• Incident Response
• Threat Hunting
• Purple Teaming

These are "new" concepts in the Cyber Security space. I'm going to be publishing some posts over the next few days to publish definitions of these terms, how the enable the business, and how to stand up these functions in your organization.

Please put your notes in the comments!

Using the NSRL on a Modern Machine

Like anyone else that works IR, I found myself with a machine and I wasn't sure if I had malware on it. I figured I'd use the RDS from NSRL to "subtract" out known good files. On a "modern" machine (defined as SSD storage with multiple cores), just "subtracting" out the known good is still time consuming. So, I did the following to make it much faster.

A simple "grep -v" would've taken >25 hours. With my process it was 15 seconds.

My system:

• Intel i7-4930K (6c, 12t) 3.4GHz
• ASUS X79 Deluxe Motherboard
• 64GB RAM
• Corsair 240GB ssd (nothing too fancy)
• Fedora 25

How did I accomplish this magic?

Get the NSRL

I won't go into details, but something like this:

1. Download the "Modern" and "Legacy" RDS ISO's from the NSRL
2. Mount them (mount -o loop RDS_257_legacy.iso /mnt/legacy/ ; mount -o loop RDS_257_modern.iso /mnt/modern/) 
3. Uncompress the big files
1. cd ; mkdir nsrl ; cd nsrl
2. mkdir modern ; cd modern
3. unzip /mnt/modern/NSRLFile.txt.zip
4. mv NSRLFile.txt NSRLFile-modern.txt
5. cut -f2 -d, NSRLFile-modern.txt | cut -f2 -d\" | sort -u > nsrl-modern-su.md5
6. cd ..
7. mkdir legacy ; cd legacy
8. unzip /mnt/legacy/NSRLFile.txt.zip
9. mv NSRLFile.txt NSRLFile-legacy.txt
10. cut -f2 -d, NSRLFile-legacy.txt | cut -f2 -d\" | sort -u > nsrl-legacy-su.md5
11. cd ..
4. Combine legacy & modern
1. cd ~/nsrl
2. cat {legacy,modern}/nsrl*-su.md5 | sort | uniq -c > nsrl-modern_legacy.md5

It's just that easy! ;)

Source Image

To "prepare" my source image, I had run the "file" command on everything in the image. I then grepped through that output file for "executable". That gave me executables formatted as: 

• MS-DOS 
• PE32
• DLL's
• All kinds of other stuff

I then ran md5sum on every file in that list. I saved those in "executables.md5s" There were 6800 unique executables listed in this file.

Obviously, I skipped some steps here. Hit me up in the comments if you want details.

Subtracting the NSRL from Source

My first attempt was:

 grep -vi -f nsrl-modern_legacy.md5 executables.md5s

That ran out of memory and crashed. 

I've done A BUNCH of work with "grep -v -f A B". I've learned that it can still be VERY fast if B is HUGE. What slows down grep is when A gets big. So, let's keep A small and get this done!

Normally, subtracting a bunch of things out of one input file must be done sequentially. This is slow. Boo!

Second Attempt

My second attempt taught me that I'm going to have to do this backwards. I want to find the intersection of these two files. Once I have that (very small) list, I can quickly subtract it from executables.md5s.

Third Attempt

This attempt took me down a fascinating path that was totally fruitless. I'll write a different blog on that later. :)

Fourth Attempt

The "executables.md5s" was the output of the "md5sum" program. So, I pulled out just the md5sums. Also, NSRL uses all uppercase, while the md5sum is all lowecase by default. This took care of both:

 awk '{print toupper($1);}' executables.md5s > executables.md5sonly

I tried:

 pv nsrl-modern_legacy.md5 | grep -f executables.md5sonly -i > tacos

If you don't know pv, go check it out! It's like "cat" with a "done-o-meter"!

This was going to take 23 hours!

Final Attempt

I have 6 cores, 12 threads of execution. I wanted to go with 4 x Cores for my number of runs. This command splits the already (comparatively) small executables.md5sonly into 24 files:

 cd ~/nsrl ; mkdir exesplit ; cd exesplit

 split -d -n l/24 ../executables.md5sonly exesplit.

 time ls exesplit.* | parallel --jobs 24 grep -f {} ../nsrl-modern_legacy.md5 > all.out

This kicked off 24 parallel jobs of grep all searching through the NSRL for md5's from my suspect machine. There were 3700 md5sums in all.out

It took 7.8 seconds.

But... I'm not quite done yet. That just shows the files that are in my suspect machine AND in the NSRL. So, what I need is the files that are in the suspect machine that AREN'T in the NSRL. That's simple:

 cd ~/nsrl

 time grep -v -f exesplit/all.out executables.md5sonly > executables-nonsrl.md5

That took 2.2 seconds.

I'm now down to 3100 md5sums listed in executables-nonsrl.md5. I eliminated over half my files to check for malice.

Virus Total

My co-worker has a script to run these against the Virus Total. Running all 6800 files would've taken >24 hours. Only running 3100 takes < 12 hours. And we can be pretty confident that we won't be querying for things that are known good.

Significant Security Incidents

When getting into Information Security, it is helpful to know about certain key attacks. Here are some that come to mind for me:

This is not a complete list, by far. I welcome people to add links or notes to things that they think should be on the list, too.

1997: Eligible Receiver
https://en.wikipedia.org/wiki/Eligible_Receiver_97

2003 attack. Discovered in 2005. Titan Rain
https://en.wikipedia.org/wiki/Titan_Rain
http://content.time.com/time/printout/0,8816,1098371,00.html
http://courses.cs.washington.edu/courses/csep590/05au/readings/titan.rain.htm
https://www.theguardian.com/technology/2007/sep/04/news.internet

2005 attack. Discovered in 2010. Stuxnet
https://en.wikipedia.org/wiki/Stuxnet
https://www.youtube.com/watch?v=rOwMW6agpTI
https://www.symantec.com/content/en/us/enterprise/media/security_response/whitepapers/w32_stuxnet_dossier.pdf
http://isis-online.org/isis-reports/detail/stuxnet-malware-and-natanz-update-of-isis-december-22-2010-reportsupa-href1/8
http://isis-online.org/uploads/isis-reports/documents/stuxnet_update_15Feb2011.pdf

2006 attack. Discovered in 2011. Night Dragon Operation
https://en.wikipedia.org/wiki/Night_Dragon_Operation
https://www.mcafee.com/jp/resources/white-papers/wp-global-energy-cyberattacks-night-dragon.pdf

2008. Australian ISC Water Services
http://csrc.nist.gov/groups/SMA/fisma/ics/documents/Maroochy-Water-Services-Case-Study_briefing.pdf

2009 attack. Discovered in 2010. Operation Aurora
https://en.wikipedia.org/wiki/Operation_Aurora

2014 Heartbleed Vulnerability
Let's someone image your RAM. Nothing logged on host OS.
https://en.wikipedia.org/wiki/Heartbleed

2014 Shellshock Vulnerability
I'm over-simplifying, but this allowed Remote Code Execution (RCE) in bash. So, you could go to a website and just execute code on the web server. Trivial to exploit. even I wrote an exploit for it while on a conference call.

https://en.wikipedia.org/wiki/Shellshock_(software_bug)

June 2017 Cybersecurity Reading List

Here are the books on my bookshelf that I'm either excited to read or have recently read:

• Windows Internals, Part 1, version 7 by Yosifovich, et all from Microsoft Press
• Troubleshooting with the Windows Sysinternals Tools by Russinovich, et all from Microsoft Press
• Learn PowerShell in a Month of Lunches, 3rd Edition by Jones & Hicks
• Practical Forensic Imaging from No Starch Press
• Metasploit, the Penetration Tester's Guide by Kennedy et al from No Starch Press
• Metasploit Toolkit from Syngress
• Linux Firewalls from No Starch Press
• Practical Malware Analysis from No Starch Press
• Practical Packet Analysis 3rd Edition by Chris Sanders from No Starch Press
• Defensive Security Handbook from O'Reilly

And some major breaches you should be aware of:

• Operation Aurora
• Titan Rain
• Target Breach

Things I've already read that you may be interested in:

• Ten Strategies of a World-Class Cybersecurity Operations Center by Carson Zimmerman from MITRE
• I can't recommend this enough. For those that want to "get into security" or want to really take their career to the next level, this is unbeatable.
• The Tao of Network Security Monitoring by Richard Bejtlich
• Real Digital Forensics by Jones, Bejtlich, Rose
• Extrusion Detection by Richard Bejtlich
• The Practice of Network Security Monitoring by Richard Bejtlich from No Starch Press
• The Linux Command Line from No Starch Press

What are you reading? Did you find any of these helpful? Are there other books I should be aware of that are better than these?

Splitting MP3's

Sometimes you need to split long MP3's into smaller chunks. I deal with this with audio books and listening to my SANS classes. (Not my SANS classes. I'm not good enough yet. But, everyone needs something to strive for!)

These are all Linux commands. They should work in most SANS Linux VM's (like the SEC504, SEC511, FOR500, or FOR508 ones). They'll also work under Windows Subsystem for Linux

Anyway, those MP3's from SANS have long, vague filenames. So, I use this script to convert a filename like "SEC511_1A_B01.mp3" to something simple like "1A.mp3":

ls *3 | while read fn ; do nfn=$(echo $fn | cut -f2 -d_) ; mv -vi $fn $nfn.mp3 ; done

First, cd into the directory containing your mp3's. Then, create a directory called split to store the split files (md split). You'll need to install mp3splt by running:
sudo apt-get install mp3splt

Finally(!), this command will split at 5 minute intervals (-t 5.00), store the output in a directory called split (-d split/), and operate on all files in the current directory named *mp3.

mp3splt -f -t 5.00 -d split/ *mp3

That should do it! Any questions?

EDIT: June 4, 2017: Added "-f" option to mp3splt command.

Windows 10 RAID for Photographers

These techniques should work under Windows Vista, 7, 8, 8.1, or 10. I'm just using 10 as it is the most up to date as of the time I'm writing this (May 2017). Specifically, Windows 10 1703 / Creators Update.

This is a draft, if you'd like to see me continue, just let me know in the comments!

Introduction

A photographer has unique storage needs. They tend to need to maintain an archive of data that can be in the terabytes. They also are working with an active data set (from a current shoot) that's in the 10's of gigabytes.

This article will walk through how a photograph can use RAID 5 to meet those needs.

What is RAID?

First, What is RAID? Go check out Wikipedia at https://en.wikipedia.org/wiki/RAID for a nice article. The basic idea is that hard drives fail. So, spread your data across more than one so that you don't lose everything if one dies. different levels of RAID provide different levels of protection. In this article I'll focus on level 5 (called RAID 5) which provides a great balance of cost, performance, and value.

RAID 5 take three (or more) drives and puts them into a pool called an array. Some fancy math is done so that your data and "parity information" is stored across all three drives. Then, if any one drive fails, the computer can use the parity information to recreate the missing data. This happens on the fly; you may not even notice that a drive has failed. You replace that drive, and the computer rebuilds the data that was on it. Once the array is healthy again, any drive could fail without impacting your work.

I support desktop machines at a business about an hour from my house. I've got RAID 1 (a cheaper option) in those desktops. When a drive fails, the end user doesn't notice. During my monthly visits, I detect and fix the failed drives. They don't even know that they averted disaster.

What is required?

For RAID 5, you need 3 hard drives. Most desktop computers can easily handle that. If you are lucky enough to live by a Frye's, Micro Center, or similar, they'll be happy to help you out. You may need things like this:

• Amazon Link to Drive
• Amazon Link to SATA Cables
• Amazon Link to SATA Power Splitter

Expect to spend about $300 for "value" drives.

How?

Physically connect the new drives to power and SATA ports in your computer. (I'm using a Virtual Machine with small disks. Your large drives and physical machine will be similar)

This has a great writeup on configuring RAID under the new Windows 10 Storage Spaces:

https://www.pcsteps.com/738-software-raid-windows-storage-pools/

Ransomware

TBD

Conclusion

TBD

Standing up a Security Operations Center

I see people struggle with starting a Security Operations Center. It is a daunting task that is frequently bestowed by management with a high priority, lots of money for blinkenlights but almost none for training.

Here's the first thing to know:

• People > Process > Technology

People are more important that Process which is more important that Technology (or Tools). Yes, all three are required. Yes, tools make us more efficient, but you've gotta have great people to start. Remember that when you're budgeting.

So, you're about to spend a bunch of money on software, appliances, and other tools to get you better controls. So, remember this:

• Prevention -> Detection -> Response

What's that mean? Of course you want to prevent as much "evil" in your environment as possible. But, prevention fails. Get your management on board with that as early as possible. Otherwise, the question becomes "How many times can I fail before you fire me?"

Once prevention fails, you have to rely on Detection. So your IDS tells you that someone is pwned. Then what? Respond! That's where your Incident Response kicks in.

Standing up all this is very difficult and you probably don't have the man-power already to do it right. You'll need Engineers to get everything installed and functional. Not easy work. But, then you'll quickly need Analysts to go through all the data generated by these systems; understand & prioritize it. Then, those Analysts get to do the really fun stuff: Incident Response and Forensics.

Training

Formal Training is so incredibly important. I suggest:

• SEC401: Security Essentials Bootcamp Style
• SEC511: Continuous Monitoring and Security Operations
• SEC504: Hacker Tools, Techniques, Exploits, and Incident Handling
• FOR500: Windows Forensic Analysis
• FOR508: Advanced Digital Forensics, Incident Response, and Threat Hunting

Books

Before you get too into this, you need your framework. What are your guiding principles. I highly suggest the "MITRE SOC Book", Ten Strategies of a World-Class Cybersecurity Operations Center by Carson Zimmerman. This is a wonderful, free resource. This helped jump start the SOC building process at one place I worked.

Cisco press has their Security Operations Center: Building, Operating, and Maintaining your SOC book. It was helpful. A good compliment to the MITRE one.

Syngress has Designing and Building Security Operations Center from David Nathans. Something you want on your shelf, too!

I've had the pleasure of hearing both Carson and David speak. These guys know their stuff and have distilled much of it into paper so you can learn from their mistakes.