OSX Incident Response

Getting Started With Incident Response

When I'm dealing with a Mac that's known to be compromised, the first step is to consider the client's situation and the potential nature of the breach. For example, if the device may have been used in a crime or could become part of a criminal investigation, I would recommend the client use a digital forensics lab that can image the device and recover artifacts from memory without polluting the evidence. Cleanroom forensics is a different process from what we will cover here, which is more akin to a SOC team investigation to determine what an intruder or a malware infection may have done that has not already been logged by detection software. For example, has there been lateral movement, has data been exfiltrated, and system manipulation? Are there other indicators of attack or compromise that we haven't yet discovered? These are the questions that we want to set out to answer as quickly as possible to protect the business.

Let's assume for the purposes of our scenario, then, that an employee has brought us a machine after discovering and removing a malware infection. The machine is still powered on, and we have the necessary credentials (and authority) to examine the machine fully. 

Say Hello to Sysdiagnose

With that out of the way, let's set about collecting some initial information. Typically, investigators will want to list things like the system version, currently running processes, network configuration, Bluetooth set up, mounted volumes, install history, system log and much more besides. You could invest quite some time writing your own custom scripts to collect that and other information (we'll do a bit of custom script writing later in this series), but if you have direct access to the machine you can save yourself a lot of work by leveraging the built-in sysdiagnose tool provided by Apple. 

The sysdiagnose tool was not designed for security or incident response purposes. Apple wrote it for macOS beta testers as a means of collecting just about everything they could ever want to know about a Mac when investigating OS bug reports. But for that reason, it's ideal for our purposes, too. Here's an image showing just some of the data that it collects.

What's even nicer, from the point of convenience, is that if you have physical access to the machine you can kickoff the report simply by pressing this keychord:

Control-Option-Command-Shift-Period

If you get it right, you'll see the display briefly flash indicating that the process has begun. 

If using the keychord is a problem for any reason, head over to the Terminal app. There are a bunch of command line options you can specify (see man sysdiagnose page), but for our purposes we will just run it in vanilla mode. Enter the following, type the admin password, and confirm that you want to proceed when prompted:

$ sudo sysdiagnose

In this case you will not see the display flash. 

Regardless of which way you invoke the tool, it'll take a few minutes to complete, so you might want to take the opportunity to make a coffee, walk the dog, feed the cat, or while away the time as you see fit until sysdiagnose has done its thing. 

Exploring Files Collected by Sysdiagnose

When sysdiagnose has finished, it'll pop a Finder window showing you the compressed result. Copy it off to your local machine, then double-click it to unpack it and have a quick scroll through what's been collected. Yes, there's a lot of juicy stuff in there: everything from a full ps to netstat, kextstat, system_profiler, top, Wifi scans and much, much more.  

When working with large amounts of text files I like to use BBEdit, which offers many useful functions for quickly searching and manipulating multiple files. The features I'll use are all available in the free version, so if you don't already have a copy of BBEdit just go ahead and download the free demo. Of course, if you have your own way of working with large sets of files, that's fine, too. 

If you have BBEdit in the Dock, grab the sysdiagnose parent folder in the Finder and drag and drop it on top of the BBEdit Dock icon. When the project view opens up, scroll down to the logs folder in BBEdit's Sidebar, click the disclosure triangle and scroll down again. You should see useful things like Install.log and InstallHistory.plist among many other goodies. 

Still in the logs subfolder, find the folder SystemExp, descend into that and open up the folder named "Dock" (followed by a date and timestamp). In here, you'll find useful stuff such as CachedWindows.txt, which might tell you a little about the user's recent activity (although much more to come on that in the following post!). 

Also, take a look at dockextras.txt file, which may include info on things like the last time the user connected to Facetime, Messages and a bunch of other apps. 

Interlude – A Note About Timestamps

Before we move on, a note about the timestamps you see here, as you'll encounter these elsewhere in macOS logs. Timestamps like this

587381138.016775

may look like Unix epoch timestamps (that is, seconds since 1/1/1970), but if you try to convert them using Unix epoch time you'll get nonsense dates. These are actually Cocoa timestamps, which are similar but the seconds are counted since 1/1/2001. To convert them, add the difference between Unix and Cocoa start dates in seconds (that's a fixed integer of 978307200) and use the date command line utility with the -r switch. We remove the fraction of a second and just deal with the whole integer, like so:

$ date -r $((587381137 + 978307200))

That returns the more human-friendly date of Tue 13 Aug 2019 16:25:37 +07 from the Cocoa timestamp.

Finding Traces of Malicious Activity

Just below the logs folder you should see a file lsappinfo.txt. Click on it to load it into the main editor window. This file contains a lot of useful data about currently running applications, but even more useful for incident response – when we're likely faced with a situation where malware has been and gone – is to look in the two files below, the admin (501) and root (0) dumps of lsregister. These are dumps of the databases held by Launch Services and contain detailed information about every application that has been available to the user.

Let's walk through a practical example of how we might use this information to learn more about an infection.

If you scroll through lsregister-0.txt, you'll notice each record has a path field and many have a CFBundleIdentifier field. To make a cursory examination of this file, I'll use BBEdit's 'Process Lines Containing' function (from the Text menu) and copy all lines containing CFBundleIdentifier to a new document. 

In the resulting text window, I'll use the same function only this time I'll delete all lines containing "com.apple" to narrow down my search (as we've pointed out before, some malware likes to disguise itself by using the "com.apple" label, so bear that in mind).

On my suspect device, this gives the following results. The highlighted ones will stand out to anyone familiar with macOS malware. There's a bunch of commodity adware/PUP programs, but the ones in red are particularly interesting.

Let's see what more we can find out about them. We'll start with the bottom one, since that kind of bundle identifier is a non-standard pattern rarely used by legitimate software. Using BBEdit's Multi-File Search function (Shift-Command-F), we can rapidly search through all the files collected by sysdiagnose for this identifier and see what else is known about it. 

Add the identifier to the "Find" field and choose "Frontmost project" from the "Search in:" panel below. Then click 'Find All'.

Our search results have revealed the Path, full App Name and team ID (aka "Developer Signature"). But further investigation on the machine shows no evidence the application still exists. After trying searches on VirusTotal and other public search engines, the teamID led us to a Russian-language stackoverflow post.

It turns out that the developer signature was used to sign an "app" that was in fact a Bash script bundled in an Application wrapper. It looks very much like a variant of OSX.Shlayer. There's a high probability that the item found on our machine was a variant of the same malware, given that they were both signed by the same developer.

Returning to our list of labels, note that the second item, com.lights.Oblivion, is a bundle identifier associated with OSX.CrescentCore.

And what about the other highlighted item, com.ableton.live? Ableton Live is a legitimate commercial program, but there's also cracked versions on the internet that are used for cryptojacking. Again, using the Multi-File search, we can find more info in the sysdiagnose folder. This time a result in the install.log reveals that the app was delivered in an unsigned .pkg. Since there is no chance that a company like Ableton would be distributing their software without proper code signing, there's a strong likelihood that this package is malware. 

It seems our user's machine has seen quite a lot of action!

While the above example isn't particularly methodical, it does hopefully give you an idea of what you can do with such a vast amount of data and a few multi-file searches. 

One Log To Rule Them All

Among the many other files worth exploring in the sysdiagnose folder, there is one other that deserves special mention. Scroll down (either in BBEdit or Finder) to a file called system_logs.logarchive.

As the name suggests, this is a collection of macOS system logs, the sort that are typically viewed in the Console.app. The file is actually a directory, but its contents are unreadable in BBEdit; however, double-clicking it in BBEdit will open it in the Console.app. You can also read this format with the log command in the Terminal. The latter is a far more powerful and effective tool for investigative work, but it does take a little practice to master. As there are many good guides on the log command, such as here and here, as well as the man page itself, we won't go into details here. However, there are a couple of oddities about the "unified logging" system that I haven't seen covered elsewhere and which are worth being aware of. 

First, note that the system_logs.logarchive file collected by sysdiagnose only contains a subset of the logs available. You can see the range of information collected by using the stats command. For example, 

$ log stats --archive <path to logarchive file> --overview

In this case, we see logs collected from August 15th to 20th. Now let's run the same command on the machine without specifying the name of the logarchive file in the sysdisagnose folder.

$ log stats --overview

With no logarchive file specified, the command returns the stats for the main system log datastore held on the device.

That's quite a lot more (and also quite a lot larger!) and covers around 30 days worth of logs, from July 22nd to August 21st. To collect all the log info, run a separate collection command.  Be sure to specify a destination that is safe to write to (such as a connected device or quarantined folder) as by default the collect verb will save to the current working directory.

$ sudo log collect --output <path to dst>

The other oddity of this tool is that if you run the stats command on your newly collected log file, you may find it contains logs reaching even further back in time than the previous output of --overview indicated. In this case, the collect command appears to have reached back an additional 4 days, to 18th July.

The cause of these oddities is unknown (at least to me) – whether it's a bug or intended behavior – but the vagaries of the log command are worth bearing in mind. 

Exploring fs_usage for File Activity

One other file we'll mention in the sysdiagnose folder before moving on is fs_usage.txt. This gives you a capture of file activity when you ran the sysdiagnose utility. It is useful to see what was occurring at the time of collection. You can quickly parse fs_usage.txt to get a list of every process that was involved in file activity. Try to cd into the sysdiagnose parent directory, then use something like the following to uniquely list processes that were interacting with the file system:

$ awk '{print $NF}' fs_usage.txt | cut -d. -f1 | sort -u

AirPlayXPCHelper

CoreServicesUIAg

Electron

Finder

Opera

Slack

...snip...

Telegram

UserEventAgent

WireGuardNetworkExt

WireGuardNetworkExtension

We can do something similar to quickly get a list of all file paths that were accessed. Note we're grepping out files accessed by sysdiagnose itself to ignore our own activity:

$ awk '{print $0}' fs_usage.txt | grep '/' | sort -u | grep -v -i sysdiagnose

However, as fs_usage only records file activity at the time we ran the utility, we need something better to provide historical records of file events. 

FSEvents – Old, Not Obsolete

Fortunately, such records of file system events are created in a hidden folder at the root of each volume or disk image. 

/.fseventsd

You can easily toggle visibility of this and other useful hidden folders in the Finder by using the keychord:

Command-Shift-Period 

As we see in the image above, this folder is protected, so we will need to drop down to root on the command line to inspect it.

The .fseventsd folder contains data files compressed with gzip. Although we could manually unzip each file, hexdump it or extract the printable characters with strings, that all requires a lot of labor and the results are likely to lose context. A better solution is to use the free tool FSEventsParser. This has the ability to create both SQL database and spreadsheet output, giving us access to much more powerful queries and analysis. 

Running the tool in its most basic form requires specifying the source and destination folders (more recent versions also require the -t switch and either folder or image for a value). Depending on the number of records, this may take some time.

$ python FSEParser_V3.3.py -s -t folder /.fseventsd -o /Users/lab/Desktop/FSEvents_Out

The output, however, is well worth it. With FSEvents, we can conduct queries such as which files were sent to the Trash, what devices were mounted, which files were accessed or what websites were visited on a particular date. 

Like the unified logs, .fseventsd will only reach back a limited timespan as the records are continually churned to save space. How far back depends on a number of factors, including how active the system is, but if your suspicious events occurred close enough to the collection time, you may well have some extremely rich data that you can mine for evidence of malicious activity. 

Be aware that activities like updating the OS will wipe out existing logs in the .fseventsd folder (you can use the install.log in the sysdiagnose folder to determine when the most recent update occurred), and it's also not unheard of for some events to fail to be recorded at all, such as during especially heavy I/O activity.

Another issue to bear in mind is that users can deliberately prevent the system from recording FSEvent activity by creating a touch file inside the .fseventsd folder.

$ sudo touch /.fseventsd/no_log

What all that means is that you can't assume something didn't happen just because you didn't find a record of it in .fseventsd. However, what you do find can often prove extremely illuminating.

Why Investigate User Behavior?

There's a few reasons why we might be interested in user behavior. First, there's the possibility of either unintentional or malicious insider threats. What has the user been using the device for, what have they accessed and who have they communicated with?

Second, 'user behavior' isn't necessarily restricted to the authorized or designated user (or users), but also covers unauthorized users including remote and local attackers. Who has accounts on the device, when have they been accessed, and do those access times correlate with the pattern of behaviour we would expect to see from the authorized users? These are all questions that we would want to be able to answer.

Third, a lot of confidential and personal user data is stored away in hidden or obscure databases on macOS. While Apple have made some efforts recently to lock these down, many are still scrapable by processes running with the user's privileges, but not necessarily their knowledge. By looking at these databases, what they contain, and when they were accessed, we can get a sense of what data the company might have lost in an attack, from everything from personal communications, to contacts, web history, notes, notifications and more.

A Quick Review of SQLite

Although some data we will come across is in Apple's property plist format and less occassionally plain text files, most of the data we're interested in is saved in sqlite databases. I am certainly no expert with SQL, but we can very quickly extract interesting data with a few simple commands and utilities. You can use the free DB Browser for SQLite if you want a GUI front end, or you can use the command line. I tend to use the command line for quick, broad-brush looks at what a database contains and turn to the SQLite Browser if I really want to dig deep and run fine-grained queries. Here are some very basic commands that serve me well.

sqlite3 /path to db/ .dump

This is my go-to command, which just pumps out everything in one (potentially huge) flood of data. It's a great way to quickly look at what kind of info the database might contain. You can grep that output, too, if you're looking for specific kinds of things like filepaths, email or URL addresses, and piping the output to a plain text file can make it easy to save and review if you don't want to work directly on the command line all the time.

sqlite3 /path to db/ .tables

Tables gives you a sense of the different kinds of data stored and which might be most interesting to look at in detail.

sqlite3 /path to db/ 'select * from [tablename]'

Another one of my go-to commands, this is equivalent of doing a "dump" on a specific table.

sqlite3 /path to db/ .schema

This command is essential to understand the structure of the tables in the database. The .schema command allows you to understand what columns each table contains and what kind of data they hold. We'll look at an example of doing this below.

Finding Interesting Data on macOS

There's a few challenges when investigating user activity on the Mac, and the first is actually finding the databases of interest. Aside from the fact that they are littered all over the user and system folders, they can also move around from one version of macOS to another and have also been known to change structure from time to time.

In the last post, when we played with sysdiagnose, you may recall that one location the utility scraped logs from was /var/db. There's user data in there, too. For example, in CoreDuet, you may find the Knowledge/knowledgeC.db and the People/interactionC.db.



SANS macOS forensics instructor Sarah Edwards did a great post on mining the knowledgeC.db which I highly recommend. From it, you will be able to discern a great deal of information about the user's Application usage and activity, browser habits and more. Some of this information we'll also gather from other sources below, but the more corroborating evidence you can gather to base your interpretations on the better.

The interactionC.db may give you insight into the user's email activity, something we will return to later in this post. In the meantime, let's use this database for a simple example of how we can interpret the SQL databases in general. Drop into a root shell (or use sudo), then change

$ cd /private/var/db/CoreDuet/People

and list the contents with ls -al. You should see interactionC.db in the listing.

If we run .tables on this database, we can see it contains some interesting looking items.

Let's dump everything from the ZCONTACTS table and have a look at the data.

$ sqlite3 interactionC.db 'select * from ZCONTACTS'

Each line has a form like this:

3|2|31|0|0|17|0|0|2|583116432.847281||583091780||0|588622185|||[email protected]

|[email protected]|

Sure, we can see the email address in plain text, but what does the rest of the data mean? This is where .schema helps us out. After running the .schema command, look for the CREATE TABLE ZCONTACTS schema in the output.

$ sqlite3 interactionC.db .schema

The comma-separated fields tell us the table column name and the kind of data the ZCONTACTS table accepts (eg. Z_OPT column takes integers). There are 20 possible colums in this table, and we can match those up with each column from data extracted from the table earlier, where each column in that output is separated by a |. Here we also used the method of converting Cocoa timestamps to human-readable dates that we discussed in Part One .

The data indicates that between 25 June and 28 August the recipient received 17 messages from the email address identified in fields 18 and 19.

However, a word of caution about interpretation. Until you are very familiar with how a given database is populated (and depopulated) over time, do not jump to conclusions about what you think it's telling you. Could there have been more or less than 17 messages during that time? Unless you know what criteria the underlying process uses for including or removing a record in its database, that's very difficult to say for sure. In similar vein, note that the timestamps may not always be reliable either. You cannot assume that a single database is sufficient to establish a particular conclusion. That's why corroborating evidence from other databases and other activity is essential. What we are looking at with these sources of data are indications of particular activity rather than cast-iron proof of it.

Databases in the User Library

A great deal of user data is held in various directories within the ~/Library folder. The following code will pump out an exhaustive list of .db files that can be accessed as the current user (try with sudo to see what extras you can get).

$ cd ~/Library/Application Support; find . -type f -name "*.db" 2> /dev/null

However, here's another difficulty if you're working on Mojave or later. Since Apple brought in enhanced user protections, you may find some files off limits even with sudo. To get around that, you could try taking a trip to the System Preferences.app and adding the Terminal to Full Disk Access. That's assuming, of course, that there are no concerns about 'contaminating' a device with your own activity.

Dumping a list of all the possible databases might look daunting, but here’s just a few of the more interesting Apple ones you might want to look at on top of those associated with 3rd party software, email clients, browsers and so on.

./Application Support/Knowledge/knowledgeC.db

./Application Support/com.apple.TCC/TCC.db

./Containers/com.apple.Notes/Data/Library/Caches/com.apple.Notes/Cache.db

./Mail/../

./Messages/chat.db

./Safari/History.db

./Suggestions/snippets.db

Let's look at a few examples. Surprisingly, the Messages' chat.db is entirely unprotected, so you can dump messages in plain text. You might be surprised to find just how unguarded people can be on informal chat platforms like this.

$ sqlite3 chat.db .tables

This user has basically left themselves open to compromise from any process running under their own user name.

$ sqlite3 chat.db 'select * from message'

Mail is also completely readable once you dig down through the hierarchy of folders. Here the messages are not stored in a sqlite database, but use the .emlx format. These encode the email content in base64, which can easily be extracted and decoded.

You can save yourself a lot of time with emails by reading the snippets.db in the Suggestions folder. This contains databases that are meant to speed up predictive suggestions by the OS in application searches (Contacts, Mail, etc), as well as Spotlight and the browser address bar. The snippets.db contains snippets of email conversations and contact information.

Sometimes you'll get silent' permission denied' issues on these databases, even when using root and Terminal has Full Disk Access. For example, in the image below, the file size of the queue.db clearly indicates that there's more data in there than I seem to be getting from sqlite.

When these kind of things happen, a 'quick and dirty' solution can be to turn to either the strings command or the xxd utility to quickly dump out the raw ASCII text and see if the contents are worthy of interest.

Mining the Darwin_User_Dir for Data

Apple hide some databases in an obscure folder in /var/folders/. When logged in as a given user, you can leverage the DARWIN_USER_DIR environment variable to get there.

$ cd $(getconf DARWIN_USER_DIR)

Again, you may find even with Terminal added to Full Disk Access, some directories will remain off limits, even for the root user, like the SafariFamily folder appears to be.

In this case, we can't even dump the strings because we cannot even get permission to list the file.

The only way to get access to these kinds of protected places is to turn off System Integrity Protection, which may or may not be something you are able to do, depending on the case.

Reading User Notifications, Blobs & Plists

One of the databases you'll find in the folder that the variable DARWIN_USER_DIR takes you to is the database that stores data from Notifications – messages sent from Applications like Mail, Slack and so on to the Notification Center and which appear as alerts and banners in the top right of the screen. Fortunately or unfortunately, depending on how you look at it, we don't need special permissions to read this database.

If you're logged in as the user whose Notifications you want to look at, the following command will take you to the directory where the sqlite database is located.

$ cd $(getconf DARWIN_USER_DIR)/com.apple.notificationcenter/

The Notifications database has changed at some point in time, and if you list the contents of the directory you may see both a db and a db2 folder. Change directory into each in turn and run the .tables and .schema commands to compare the different structures. Both use blobs for the data, so you will need a couple of tricks to learn how to read these.

One way is to open the database in DB Browser for SQLite, click on the blob data and view the source in binary format. You can export that source as blob.bin and then use plutil -p blob.bin to output it in nice human-readable text.

I'm usually in too much of a hurry to do all that. Instead, I'll do something like

$ sqlite3 db 'select * from app_info'

to browse through the list of apps that have sent Notifications, then run a few greps on the entire database. For example, if I want to read Slack notifications, I can use something like this:

$ strings $(getconf DARWIN_USER_DIR)/com.apple.notificationcenter/db2/db | grep -i -A4 slack

And of course I can just change 'slack' for 'mail' or whatever else looks interesting. I might then use the previous method with the DB Browser and plutil mentioned above to dig deeper.

Reading Data from Notes, More Blob Tricks

There's plenty of application databases that we haven't touched on, but one that I want to cover in this overview is Apple's Notes. Not only might this be a good source of information about user activity, it's also trickier to deal with than the other databases we've looked at.

We can find the Notes database in the ~/Library/Group Containers folder. Let's quickly review the tables:

$ sqlite3 ~/Library/Group Containers/group.com.apple.notes/NoteStore.sqlite .tables

This somewhat byzantine-looking one-liner will dump all the user's iCloud notes to stdout.

$ for i in $(sqlite3 ~/Library/Group Containers/group.com.apple.notes/NoteStore.sqlite "select Z_PK from ZICNOTEDATA;"); do sqlite3 ~/Library/Group Containers/group.com.apple.notes/NoteStore.sqlite "select writefile('body1.gz.z', ZDATA) from ZICNOTEDATA where Z_PK = '$i';"; zcat body1.gz.Z ; done

Let's take a look at how it works. The first part selects Z_PK column – the primary keys or unique identifiers of the notes in the database – and then iterates over each one. The second part takes the primary key and for each note in the ZICNOTEDATA table, it extracts the ZDATA blob containing the note's content. Next, writefile writes the blob to a compressed file body1.gz.z in the current working directory and finally zcat decompresses it into plain text!

Finding Other Data Stores

If you are interested in a particular application or process but do not know what it uses for a backing store, if anything, there's a couple of investigative methods you can try. First, see if the process is running in Activity Monitor. If it is, click the Info button and select the 'Open Files and Ports' tab and see where it's writing to. You could also do the same thing with lsof on the command line.

If that doesn't work, try running strings on the executable file and grepping for / to search for paths that the program might write to. If you're still out of luck you may have to do a little more macOS reverse engineering to understand what the program is up to and find where it hides its data.

Usurping the Sudoers File

One of the first places I want to look for system manipulation is in the /etc/sudoers file. This file can be used to allow users to run processes with elevated privileges without being challenged for a password. To check whether the sudoers file has been modified, we will use the visudo command rather than opening the file directly in vi or another editor. Using visudo is safer as it prevents the file being saved in an invalid format.

$ sudo visudo

Modifications to the sudoers file will typically be seen at the end of the file. In part, that's because the easiest way for a process to write to it is by simply appending to it, but also the commands in the file take precedence in reverse order, with the later commands overriding earlier ones. For that reason, it's important for attackers that their commands override any others that may target the same users, groups or hosts. In this example, we can see that a malicious process has added a line to allow the user 'lab' – or more importantly any process running as that user – to run the command at the path shown on any host (ALL) without authenticating.

Cuckoos in the PATH

The $PATH environment variable lists the paths where the shell will search for programs to execute that correspond to a given command name. We can see the user's path list with

$ echo $PATH

In this example, the user's path contains the following locations:

We can use a short Bash script to iterate over the paths, and list their contents, sorted by date modified in descending order.

#! /bin/bash

while IFS=: read -d: -r path; do

    cd $path

    echo $path

    ls -altR

done <<< "${PATH:+"${PATH}:"}"  

From the results, we can quickly see which files were modified most recently. Pay particular attention to what is at the top of the path, as /usr/local/bin is in the above example. This location will be searched first when a command is issued on the command line, ahead of system paths. A "cuckoo" script named, say, sudo or any other commonly used system utility, inserted at the top of the path would get called before – in other words, instead of – the real utility. A malicious actor could write a fake sudo script which first called the actor's own routines before passing on the user's intended actions to the real sudo utility. Done properly, this would be completely transparent to the user, and of course the attacker would have gained elevated privileges along the way.

Bash, Zsh and Other Shells

In a similar way, an attacker could modify one of several files that determine things like shell aliases. An alias in say the .bashrc file could replace every call to sudo with a call to an attacker's script. To search for this possibility, be sure to check the contents of the following for such manipulations:

~/.bash_profile        # if it exists, read once when you log in to the shell

~/.bash_login          # if it exists, read once if .bash_profile doesn't exist

~/.profile             # if it exists, read once if the two above don't exist 

/etc/profile           # only read if none of the above exist

~/.bashrc              # if it exists, read every time you start a new shell

~/.bash_logout         # if it exists, read when the login shell exits

And look for the same for other shell environments the user might have like .zshrc for Zsh.

Etc, Hosts and Friends

It's also worth running a time-sorted ls on the etc folder.

$ cd /etc; ls -altR

On this compromised system, it's very clear what's been modified recently.

The hosts file is a leftover from the past and the way computers used to resolve domain names to IP addresses, a primitive form of DNS. These days the only use of the hosts file is to loopback certain domain names to the localhost, 127.0.0.1, which effectively prevents the system from reaching out to these domains. The hosts file is often manipulated by malware to stop the system checking in with certain remote services, such as Apple or other software vendors. A healthy hosts file will typically have very few entries, like so:

Networking and Sharing Prefs

While we're discussing network communications, let's check on several other areas that can be manipulated. In System Preferences' Network pane, click Advanced… and look at the Proxies tab. Some malware will use an autoproxy to redirect user's traffic in order to achieve a man-in-the-middle attack. We can also pull this information from the data we collected from sysdiagnose by searching on "autoproxy". Here we see the good news that no autoproxy is set.

We can utilise the networksetup utility here to output similar information to what you can see in the System Preferences UI regarding each network service.

#! /bin/bash

n=$(networksetup -listallnetworkservices | grep -v asterisk)

for nt in $n; do

    printf "n$ntn--------n";

    networksetup -getinfo $nt;

done

We can also find this information in the sysdiagnose report in the output of SystemProfiler's SPNetworkLocationDataType.spx file.

Finding Local and Remote Logins

Let's start with the obvious. Does the device have any of its sharing preferences enabled? In the User Interface, these are listed in the Sharing Pane:

As explained in Malware Hunting on macOS | A Practical Guide, we can get the same information from netstat by grepping for particular ports associated with sharing services. For convenience, we can run a one-liner on the command line or via script that will succinctly output the Sharing preferences:

$ rmMgmt=`netstat -na | grep LISTEN | grep tcp46 | grep "*.3283" | wc -l`; scrShrng=`netstat -na | grep LISTEN | egrep 'tcp4|tcp6' | grep "*.5900" | wc -l`; flShrng=`netstat -na | grep LISTEN | egrep 'tcp4|tcp6' | egrep "*.88|*.445|*.548" | wc -l`;rLgn=`netstat -na | grep LISTEN | egrep 'tcp4|tcp6' | grep "*.22" | wc -l`; rAE=`netstat -na | grep LISTEN | egrep 'tcp4|tcp6' | grep "*.3031" | wc -l`; bmM=`netstat -na | grep LISTEN | egrep 'tcp4|tcp6' | grep "*.4488" | wc -l`;printf "nThe following services are OFF if '0', or ON otherwise:nScreen Sharing: %snFile Sharing: %snRemote Login: %snRemote Mgmt: %snRemote Apple Events: %snBack to My Mac: %snn" "$scrShrng" "$flShrng" "$rLgn" "$rmMgmt" "$rAE" "$bmM";

This lengthy pipeline of commands should return something like this.

In the sysdiagnose/network-info folder, the netstat.txt file will also list, among other things, active internet connections. Alternatively, you can collect much of the same relevant information with the following commands:

Active Internet connections (including servers):
$ netstat -A -a -l -n -v

Routing tables:
$ netstat -n -r -a -l

Also, check the user's home folder for the invisible .ssh directory and the addition of any attacker public keys. Here, an unwanted process has secretly written a known_hosts file into the ssh folder so that the process can ensure it's connecting to its own C2 server before exfiltrating user data or downloading further components.

Either on the system itself or from the sysdiagnose folder, look for the existence of the kcpassword file. This file only exists if the system has Auto login set up, which allows a user to login to the Mac without providing a user name or password. Although it's unlikely a remote attacker would chose to set this up, a local one might (such as a co-worker), if they had hopes of physical access in the future. Perhaps more importantly, the file contains the user's actual login password in encoded but not encrypted form. It's a simple thing to decode it, but it does require having already achieved elevated privileges to do so.

The /usr/sbin/sysadminctl utility has a few useful options for checking on Guest account and other settings. This one-liner will output some useful status information:

$ state=("automaticTime" "afpGuestAccess" "filesystem" "guestAccount" "smbGuestAccess"); for i in "${state[@]}"; do sysadminctl -"${i}" status; done;

Achieving Persistence Through Application Bundles

We have already covered macOS persistence techniques, and I encourage you to refer to that for an in-depth treatment. However, it's worth mentioning here one of the upshots of Apple's recent change to requiring developers to use Application bundles for things like kexts and login items, which is that it can now be much harder to track these down. In the past, all 3rd party extensions would have been in /Library/Extensions and all login items could be tracked through the loginitems.plist file. Recent changes mean these can now be anywhere that an application can be, and that is pretty much everywhere!

In the first post in this series, we looked at an example of using LSRegister to hunt for unusual or unwanted applications. We can also leverage the Spotlight backend to search for the location of apps once we have a target bundle identifier to hand. For example:

$ mdfind "kMDItemCFBundleIdentifier == 'com.cnaa4c4d'"

Be careful with the syntax: the whole search statement is encapsulated in double quotes and the value to search for is within single quotes. More information about using mdfind can be found in the utility's man page. A list of possible predicate search terms can be printed out with

$ mdimport -X

Manipulating Users Through Their Browsers

For the vast majority of attacks, the gateway to compromise comes through interaction with the user, so it's important to check on applications that are used for communications. Have these applications' default settings been manipulated to make further exploitation and compromise easier for the attacker?

We already took a look at this in general in Part 2, but specifically some of the items we would want to look at are the addition of browser extensions, default home page and search criteria, security settings, additional or privileged users and password use. We should also check the default download location and iterate over that folder for recent activity.

I've explained here how we can examine recent downloads that have been tagged with Apple's LSQuarantine bit, but this bit is easily removed and the records in the LSQuarantine file are not all that reliable. A full listing of the user's browser history is better scraped from the relevant folders and databases belonging to each browser app. Although browser history does not tell us directly about system manipulation, by tracking the urls of malicious sites visited we can build a picture not only of where malware may have come from, but where it might be sending our user to for further compromises. We can also use any malicious URLs found in browser history as search terms across our collected data.

Although there are many browsers, I will only deal with the major ones here. It should be possible to apply the same principles in these examples to other browsers. Safari, Firefox, Chrome and Opera all have slightly different ways of storing history. Here's a few examples.

Browser History

To retrieve Safari history (Terminal will require Full Disk Access in Mojave and later):

$ sqlite3 ~/Library/Safari/History.db "SELECT h.visit_time, i.url FROM history_visits h INNER JOIN history_items i ON h.history_item = i.id"

To retrieve a list of sites that have acquired Push Notifications permissions in Safari:

$ plutil -p ~/Library/Safari/UserNotificationPermissions.plist | grep -a3 '"Permission" => 1'

To retrieve the last session data from Safari:

$ plutil -p ~/Library/Safari/LastSession.plist | grep -iv sessionstate .

Chrome history can be gathered with the following command:

$ sqlite3 ~/Library/Application Support/Google/Chrome/Default/History "SELECT datetime(((v.visit_time/1000000)-11644473600), 'unixepoch'), u.url FROM visits v INNER JOIN urls u ON u.id = v.url;"

Similar will work for Vivaldi and other Chromium based browsers once you substitute the appropriate path to the browser’s database. For example:

$ sqlite3 ~/Library/Application Support/Vivaldi/Default/History "SELECT datetime(((v.visit_time/1000000)-11644473600), 'unixepoch'), u.url FROM visits v INNER JOIN urls u ON u.id = v.url;"

Firefox History is slightly different.

$ sqlite3 ~/Library/Application Support/Firefox/Profiles/*/places.sqlite "SELECT datetime(last_visit_date/1000000,'unixepoch'), url, title from moz_places"

Browser Extensions

I've previously described how the Safari Extensions format has changed recently and how this can be leveraged by bad actors. To retrieve an old-style list of Safari browser extensions:

$ plutil -p ~/Library/Safari/Extensions/Extensions.plist| grep "Bundle Directory Name" | sort --ignore-case

The new .appex style which require an Application bundle can be enumerated via the pluginkit utility.

$ pluginkit -mDvvv -p com.apple.Safari.extension

Extensions, particularly in Chrome have long been problematic and an easy way for scammers to control user's browsing. Extensions can be enumerated in Chromium browsers from the Extensions folder:

$ ~/Library/Application Support/Google/Chrome/Default/Extensions; ls -al

Unfortunately, the randomized names and lack of human-readable identifiers is not helpful.

Suffice to say it is worth going over the contents of each directory thoroughly.

Like Safari, Firefox uses a similar, though reversed, bundleIdentifier format for Extension names, which is far more user-friendly:

$ cd ~/Library/Application Support/Firefox/Profiles/*/extensions; ls -al

Browser Security Settings

Some adware and malware attempt to turn off the browser's built-in anti-phishing settings, which is surprisingly easy to do. We can check this setting for various browsers with a few simple one-liners.

For Safari:

$ defaults read com.apple.Safari WarnAboutFraudulentWebsites

The reply should be 1 to indicate the setting is active.

Chrome and Chromium browsers typically use a "safebrowsing" key in the Preferences file located in the Defaults folder. You can simply grep for "safebrowsing" and look for {"enabled: true,"} in the result to indicate anti-phishing and malware protection is on.

$ grep 'safebrowsing' ~/Library/Application Support/Google/Chrome/Default/Preferences

Opera is slightly different, using the key "fraud_protection_enabled" rather than 'safebrowsing'.

$ grep 'fraud_protection_enabled' ~/Library/Application Support/com.operasoftware.Opera/Preferences

In Firefox, preferences are held in the prefs.js file. The following command

$ grep 'browser.safebrowsing' ~/Library/Application Support/Firefox/Profiles/*/prefs.js

will return "safebrowsing.malware.enabled" and "phishing.enabled" as false if the safe search settings have been disabled, as shown in the following images:

If the settings are on, those keys will not be present.

There are many other settings that can be mined from the browser's support folders aside from history, preferences and extensions using the same techniques as above. These locations should also be searched for manipulation of user settings and preferences such as default home page and search engines.