2010 Botnet Incident

In 2010 I was working with a Software Developement startup company, as a Security Analyst on a development project, we encountered a significant #botnet attack. It began in June during an #application-security scan when I detected malicious traffic targeting my project web server. Using #Wireshark, I found the traffic originating from multiple sources across the network. This caused a brief network outage. After reporting the incident to the IT staff, they took action, and the network stabilized until September.

On September 22, 2010, the network went down around 11 am. Since I wasn't dependent on the local infrastructure, I decided to work from home. At 9:30 pm, the IT manager called for help. Given my previous experience with the IT team, I knew the network setup well.

When I reached the office, the network was completely down, with no communication between the servers and the floor area. Despite having internet connectivity, machines couldn't access the internet. I started with basic tests, ping to the AD, DNS servers and swithches from the floor, but nothing worked. I moved to the server room and found the same issue. It was 11:30 pm when I began a Wireshark capture on my laptop. I discovered a massive amount of Spanning Tree Protocol (#STP) traffic on the server room L3 Switch and significant broadcast traffic on the floor switches. It became clear that the L2 switches had fallen into hub mode, and attacker was trying to infect more machines.

The IT team had already tried restarting the entire setup including servers and switches, but the network would go down again within 30 minutes. We needed to identify the attacking machines, but logging into the L2 switches failed.

We devised a new #strategy: shut down the entire network, including desktops and servers. The L2 switches supported 10/100/1000 Mbps speeds. We reconfigured all L2 switches with access ports to 10 Mbps and uplink ports to 1000 Mbps. After restarting the network and desktops, we began monitoring for high port usage. Within minutes, we identified six machines consuming the entire 10 Mbps port bandwidth. Disconnecting these machines restored the network.

Running at 10 Mbps, the network stabilized. We decided to keep this configuration until all infected machines were identified. The next day, the network was slow but functional. I began #forensic analysis on the six infected machines. Despite having licensed Windows XP and Win7 with paid antivirus on branded Dell desktops, the attacker had successfully infiltrated the network. Continuous monitoring revealed more infected machines connecting to a Command and Control (#C&C) center, confirming a botnet attack.

Botnets typically utilize resources for profit rather than bringing down networks, indicating involvement of multiple #hacker groups. Means the two hacker groups are fighting for capturing the nodes from same network, and their fight bring down the network. #Antivirus solutions were ineffective due to #rootkits installed on many machines. We resorted to formatting all machines, ensuring the attack wouldn't reoccur.

After sleepless nights #analyzing the data, we traced the source to an #open-source application installed by developers for a project. The botnet, known as #ZeroAccess. Initially I was not sure, but 2011 I did researcher again on the disk stored from 2010 incident, it was Zeroaccess, exploited a zero-day #vulnerability. Despite the extensive cleanup, it was a lesson in the importance of strict security protocols and vigilant monitoring to prevent such incidents in the future.

Dispite such a big incident we are able to restart the develeopement activity within 3 to 4 days with new security controls in place.

Handling such IT security incidents requires a strong foundation of knowledge, an excellent strategy and keen presence of mind.