So as a Tenable Partner we have a Lab-License for the Tenable Product Suites which we often use to test new Products, Features, Updates and recreate issues in the Lab for further analysis. For this reason I was again setting up a Nessus Network Monitor in my Home Network with Mirrored Traffic from a Switch Uplink.
If you want to perform cheap and rasy Port-Mirroring at home you don’t have to rob a Cisco Dealer! Netgear offers cheap and functional switches that worked pretty good in my HomeNetwork so far.
With the Netgear GS108E you can grab a 40€ 8-Port Gigabit switch – which is Manageable and lets you just specify the port mirror in a Webinterface – on Amazon.
And with the Netgear GS110EMX you even get 2x 10G-Ports for 217€ – again with a Simple Webinterface for Portmirror Setup – on Amazon.
Please Note that I don’t want/get any incentives if you go to those Amazon links! They are just for your reference and you can buy elsewhere and chose other manageable Switches as well!
Also of course those are consumer Grade switches not intended for Enterprise Usage! I however find the value exceptional for Home Lab Port-Mirror Setups!
Just for reference – I set up the Mirror for the Switches Uplink Port which goes to another story of my house where the Internet Router is located. As there is other Stuff Connected on that level as well the Port-Mirror was not perfect, but I was not looking for a perfect setup, just some traffic to test on and play arround with.
The Perfect placement for a Nessus Network Monitor would be directly on the Internet Breakout (internal Side of the Router or Firewall to reduce Noise) and additional Sensors/Nessus Network Monitors in front of Sensitive Systems or VLAN choke Points – for example Production VLAN Uplinks.
The Evil Outdated Chrome User-Agent
When I got the Nessus Network Monitor set up and traffic mirrored I was greeted with a couple of interesting Vulnerabilities:
I know the System! Its my *cough* Windows HomeServer 2011 installed on a neat HP MicroServer N36L (wildly outdated, but new versions are available). As Windows HomeServer 2011 is long out of support this is a horrible system to have running! And I thought I probably now have the kick in the ass to do it quickly!
But let me back up – at first glance I was recognizing outdated Browser and Chrome Vulnerabilities captured by Nessus Network Monitor. A couple of years ago I had an old Android tablet in my gym for playing music. All separated away in an IoT WiFi Network that is limited by ACL’s to only access the Internet.
This tablet was running old vulnerable Versions of Chrome as well, but as I only used it for Spotify and not browsing the web at all I let it slide…
The thing is: I replace that Android tablet with an (old and yet again vulnerable) iPad i had left over. So I looked into it and as you can see above identified the Chrome coming from my Windows Home Server 2011 *cough-again*
Before hunting for the old Chrome Installation (and btw – what is it doing accessing the Internet without me using it?!?!) I took a Peak at a specific Plugin Output to identify the exact Chrome Version:
A specific Chrome Version Number in the User-Agent is pretty good indicator its Chrome. But Version 47 is really old…
One of the Chrome Vulnerability Plugins (9083) that you can see on the Right was reporting CVE IDs from 2016 so it was probably even Older!
Googling it you will find references pointing to a release in Dec 2015.
This got me a bit worried: I dont run old Browser Versions on Computers that are quite capeable to Updateing and Running the newest Browser Versions.
Also I have all Browsers on Auto-Update – but then Again, a forgotten Browser that was never Used could lurk in an old Version on my Homeserver….
The Part where I got worried
Mhhh… Never beeing used equals old version.
User-Agent beaconing out to the Internet equals usage…
I got really worried when I connected to the system and:
THERE WAS NO CHROME! 🙀 :shockedcatfaceemoji:
This got me thinking that when I would let Malware beacon out I would use a commonly used User-Agent to mask my C&C traffic to the Internet! Overdoing this by choosing far to old or non-existing User-Agents is stuff that happens! Also as this system has been running for years this could be an indicator of a really old compromise.
Now I was at a loss! I feared Malware compromise but I did not know what process called out to the Internet…
Ofcourse there is plenty of options with windows to investigate this but out of the box if you have not set up anything prior I found out it is pretty hard to investigate this!
This is when I hat to learn again that Incident Response works far better if you plan ahead and deploy tools and logging prior to an incident!
Let the Hunt begin
But as I had a Nessus Network Monitor deployed I had at least some logging capabilities now!
Sadly out of the Box you only get the vulnerability Plugins that triggered in the GUI but it is possible to Log a detailed Realtime feed of all Plugins Triggering with some more Information than beeing displayed in the Webinterface.
You can set up this realtime-logs.txt file in the Webinterface Settings:
This will create a plain Logfile on the Disk of the NNM system.
Please note that I created a lot of the screenshots while writing this Blogpost so Timestamps will not be in Sync with the “Incident Timeline” but show the points they are created for.
I grepped this file for “Chrome/47” to get an Idea how, when and where the Strange old Chrome User-Agent was beeing used:
This was weird – Internal and External Traffic!
Also you can see specific beaconing patterns – short requests with hours of delay in between.
I was again freaked out by Google Hits to the external IP Adress and the Random GET URI Parameter!
If you google for the HTTP GET URI you will also find some Inconclusive Malware Analysis matching at least parts of the string….
Needing a Break to think
This is when I against all Forensic Best-Practices took offline the Server and Checked it offline with an Anti Virus scanner via c’t desinfect 2019.
I am in no way saying that AV is the solution for this nor did I have any hope that AV would give me a positive signal that the system was not compromised. However as I needed a break I decided that AV could at least give me a signal if the system got compromised 3 ways sideways and filled with bad stuff.
So AV will not give me piece of mind, but if it finds stuff i know I have a problem.
And I did not necessarily needed the break because I was exhausted but because I had registered for a Blackhills Infosec Webcast about Sysmon and Applocker that was beginning to start and could be watched by an Hour of Crosstrainer in my Gym!
You know there is even a name for the phenomena when you see stuff exactly in the right moment. Like when you buy a new car and start seeing them on the road everywhere all of a sudden.
Its called: the Baader-Meinhof phenomenon
I am telling you this because Sysmon probably is the answer to finding out which process is beaconing out at the timestamp when it is beeing logged by Nessus Network Monitor!
Check out the Blackhills Infosec Webcasts:
The AV came back clean and more Logging was required – as well as a good nights sleep!
The Webcast about Sysmon and Applocker was very informative and Syslog was exactly the answer I needed! I was thinking about another Tool from the Sysinternals Suite – tcpview – however I know how big these logs become and Syslog seems pretty awesome and something I have to deploy on all my Windows Machines anyhow soon!
So I set my trap – if I was compromised for years I could probably sleep another night over it – and Installed Sysmon and tailed my NNM realtime log.
Not everything goes according to plan!
This morning (August 16th) I woke up and ofcourse again there was an Old Chrome beaconing out (which was already shown above):
So I checked the Sysmon log and was greeted with no matching log:
Little sidestory: When I set up Sysmon I verified that date/time stamps where close enough to each other on the NNM host and the Windows Server to be able to correlate Events properly based on Time.
Ofcourse I found out that the ESXi in my Lab was not able to ntp out and time was off for 15 Minutes…
Let that be a lesson to never skip on proper basic Setup in productive environments – it will always bite you in the ass in case of an incident!
Luckily this was only my lab which I don’t even run continuously so I have somewhat of a lame excuse and I promise that I always check the basics in productive Setups that I perform!
All of this however doesn’t change the fact that I don’t have a matching event in Sysmon… I also searched for the destination IP but got nothing….
Finally the Conclusion
What got me to the conclusion was that the beacon was internal and the destination IP of the becaon (and a couple of inbetween) was my LG Smart TV which is ofcourse joined to my Network to Stream Netflix!
Thats when i remembered that Windows Home Server does have some UPNP Streaming stuff going on and also that I have Plex Server running on the Windows System and a Plex App on the TV.
After getting no Google Results to Windows HomeServer 2011 and the User-Agent in question I stumbled over Plex in some Google results.
Low and behold – The Villain:
Not Everything is resolved but I dont think I got hacked anymore….
So I should still get to the bottom of why the fuck Plex is beaconing to the Internet and my Smart TV with 4 year old Chrome User-Agent strings – probably because they are embedding horrible old code for some reason – but I am now certain that I was not hacked….
At least …. 99% …. somewhat…. lets not talk about paranoia….