Is this IP attacking my server?

Like many others, I maintain a server that runs 24/7 and it’s connected to the internet. Yesterday I came home to a most unusual sight on my server’s status LEDs – unexplained network activity. It’s unusual because for me, my server is almost always idle. It doesn’t serve web pages, and in fact it doesn’t really do much for the internet except for accepting SSH connections so I can access my home resources while away. So I thought I might want to investigate the cause of the suspicious network activity.

It was a good thing that username and password authentication have been disabled. My investigations after the break!

So I remotely logged into my server via SSH, and poked around to see what was going on. I checked to see if an active process is generating traffic, but found something a little more interesting through top:

top - 17:30:42 up 3 days,  9:33,  2 users,  load average: 2.37, 2.06, 1.63
Tasks:  67 total,   6 running,  58 sleeping,   0 stopped,   3 zombie

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4173 root       9   0  2528 2528 2412 S  4.9  4.1   0:00.15 sshd
 4171 root      13   0  2776 2776 2572 S  4.2  4.5   0:00.18 sshd
 4172 root       9   0  2528 2528 2412 S  4.2  4.1   0:00.13 sshd
 4178 root      12   0  2524 2524 2408 R  3.6  4.1   0:00.11 sshd
 4174 sshd      10   0     0    0    0 Z  3.2  0.0   0:00.10 sshd
 4176 sshd      10   0  2584 2584 2464 R  3.2  4.2   0:00.10 sshd
 4177 sshd      10   0  2584 2584 2464 S  3.2  4.2   0:00.10 sshd
 4182 root      14   0  2496 2496 2396 S  3.2  4.0   0:00.10 sshd
 4175 sshd       9   0  2624 2624 2456 S  2.9  4.2   0:00.09 sshd
 4179 root      12   0  2496 2496 2388 S  2.9  4.0   0:00.09 sshd
 4180 sshd      16   0  2544 2544 2424 R  2.9  4.1   0:00.09 sshd
 4181 root      13   0  2496 2496 2388 S  2.9  4.0   0:00.09 sshd

 

Why would there be so many sshd processes? Could the attacker have already compromised my server and used it to make outgoing SSH connections? I wanted to check if there are in fact so many connections being made, and who the other party is. Here’s where netstat comes into play.

Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52839 SYN_RECV
tcp        0      0 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52860 SYN_RECV
tcp        1      1 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:51682 LAST_ACK
tcp        1      1 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:51667 LAST_ACK
tcp        0    464 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52025 ESTABLISHED
tcp        0      0 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52092 ESTABLISHED
tcp        0      0 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52101 ESTABLISHED
tcp        0      0 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52168 ESTABLISHED
tcp        0    268 192-168-1-2.tpgi.co:ssh 192-168-1-100.tpg:55205 ESTABLISHED
tcp        0      0 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52769 ESTABLISHED
tcp        0      0 192-168-1-2.tpgi.co:ssh mala.ccert.edu.cn:52778 ESTABLISHED
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  3      [ ]         DGRAM                    2809165  /dev/log
unix  3      [ ]         STREAM     CONNECTED     5970226
unix  3      [ ]         STREAM     CONNECTED     5970225
unix  3      [ ]         STREAM     CONNECTED     5970221
unix  3      [ ]         STREAM     CONNECTED     5970220
unix  3      [ ]         STREAM     CONNECTED     5970204
unix  3      [ ]         STREAM     CONNECTED     5970203
unix  3      [ ]         STREAM     CONNECTED     5970189
unix  3      [ ]         STREAM     CONNECTED     5970188
unix  2      [ ]         DGRAM                    5963244
unix  3      [ ]         STREAM     CONNECTED     5963243
unix  3      [ ]         STREAM     CONNECTED     5963242
unix  3      [ ]         STREAM     CONNECTED     2752
unix  3      [ ]         STREAM     CONNECTED     2751

 

Uh-oh. Someone from China has made several connections to my machine. It doesn’t look good. Strangely enough it’s not sending a lot of data even with so many TCP connections established. At this point in time, I powered off the server, which would definitely stop the attacker immediately.

Friends from uni suggested that I should capture the network activity via wireshark to take a closer look at what’s going on. Unfortunately at this stage I’ve already interrupted it. Bad luck – I guess I’ll switch my server back on, install wireshark and see if the attacker comes back. This time, I’ll be ready.

So who is this guy, and why is it attacking me? The attacker appeared to be from an educational institute (.edu.cn domain). Funnily enough, the attacker IP address hosts a web page that names the group as “CERNET Computer Emergency Response Team”. Either there’s someone in there doing some unscrupulous deeds, or even the emergency response team was compromised and became an bot controlled by someone else.

In the meantime, I decided to inspect the state of my server and see the extent of the damage. First thing I checked was the auth log messages. It appears that recently there have been a number of hosts that tried to gain access to my server via bruteforce dictionary attack on username and password.

Aug 31 09:09:06 NAS sshd[24889]: Invalid user oracle from 59.52.25.251
Aug 31 09:09:09 NAS sshd[24904]: Invalid user test from 59.52.25.251
...

Aug 31 09:24:49 NAS sshd[25590]: Did not receive identification string
from 66.147.164.211
Aug 31 09:36:33 NAS sshd[26018]: reverse mapping checking getaddrinfo
for 66-147-164-211.focaldata.net failed - POSSIBLE BREAK-IN ATTEMPT!
Aug 31 09:36:35 NAS sshd[26020]: Invalid user admin from 66.147.164.211
Aug 31 09:36:35 NAS sshd[26020]: reverse mapping checking getaddrinfo
for 66-147-164-211.focaldata.net failed - POSSIBLE BREAK-IN ATTEMPT!
Aug 31 09:36:38 NAS sshd[26024]: Invalid user test from 66.147.164.211
Aug 31 09:36:38 NAS sshd[26024]: reverse mapping checking getaddrinfo
for 66-147-164-211.focaldata.net failed - POSSIBLE BREAK-IN ATTEMPT!
...

Aug 31 22:15:03 NAS sshd[7457]: Address 61.178.241.16 maps to
16.241.178.61.dail.ts.gs.dynamic.163data.com.cn, but this does not
map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Aug 31 22:15:06 NAS sshd[7459]: Address 61.178.241.16 maps to
16.241.178.61.dail.ts.gs.dynamic.163data.com.cn, but this does not
map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Aug 31 22:15:09 NAS sshd[7463]: Address 61.178.241.16 maps to
16.241.178.61.dail.ts.gs.dynamic.163data.com.cn, but this does not
map back to the address - POSSIBLE BREAK-IN ATTEMPT!
...

Sep  1 07:24:57 NAS sshd[23180]: Invalid user oracle from 203.240.203.108
Sep  1 07:25:00 NAS sshd[23182]: Invalid user test from 203.240.203.108
Sep  1 07:30:25 NAS sshd[23203]: reverse mapping checking getaddrinfo
for 243.67.85.222.broad.zz.ha.dynamic.163data.com.cn failed - POSSIBLE
BREAK-IN ATTEMPT!
Sep  1 07:30:29 NAS sshd[23216]: reverse mapping checking getaddrinfo
for 243.67.85.222.broad.zz.ha.dynamic.163data.com.cn failed - POSSIBLE
BREAK-IN ATTEMPT!
Sep  1 07:30:33 NAS sshd[23218]: reverse mapping checking getaddrinfo
for 243.67.85.222.broad.zz.ha.dynamic.163data.com.cn failed - POSSIBLE
BREAK-IN ATTEMPT!
...

Sep  1 20:29:37 NAS sshd[26809]: Invalid user oracle from 110.12.64.193
Sep  1 20:29:40 NAS sshd[26811]: Invalid user test from 110.12.64.193
Sep  1 20:30:16 NAS sshd[26839]: Invalid user squid from 110.12.64.193
Sep  1 20:30:19 NAS sshd[26841]: Invalid user tomcat from 110.12.64.193
Sep  1 20:30:27 NAS sshd[26847]: Invalid user nagios from 110.12.64.193
Sep  1 20:30:30 NAS sshd[26849]: Invalid user jboss from 110.12.64.193
...

Sep  2 14:47:03 NAS sshd[32043]: Did not receive identification string from
218.249.193.156
Sep  2 15:01:09 NAS sshd[32103]: Invalid user sgknoc from 218.249.193.156
Sep  2 15:01:21 NAS sshd[32109]: Invalid user adminftp from 218.249.193.156
Sep  2 15:02:01 NAS sshd[32127]: Invalid user ftpd from 218.249.193.156
Sep  2 15:02:17 NAS sshd[32146]: Invalid user idctest from 218.249.193.156
Sep  2 15:02:23 NAS sshd[32148]: Invalid user oracle from 218.249.193.156
...

Sep  2 17:13:06 NAS sshd[717]: Invalid user plant from 202.112.50.28
Sep  2 17:13:10 NAS sshd[719]: Invalid user ueda from 202.112.50.28
Sep  2 17:13:14 NAS sshd[732]: Invalid user ajith from 202.112.50.28
Sep  2 17:13:18 NAS sshd[734]: Invalid user akihisa from 202.112.50.28
Sep  2 17:13:21 NAS sshd[736]: Invalid user akino from 202.112.50.28
Sep  2 17:13:25 NAS sshd[738]: Invalid user akira from 202.112.50.28
Sep  2 17:13:28 NAS sshd[740]: Invalid user atsumi from 202.112.50.28
...

It’s pretty much the same pattern of bruteforce attacks from a number of different IPs. I wonder why my firewall didn’t pick up on this obvious pattern of incessant connection attempts.

But, after looking at a quick nmap of that last IP reveals that it does some dodgy things too – filesharing BitTorrent tracker and eDonkey service:

Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2009-09-02 19:53 EST
Machine 202.112.50.28 MIGHT actually be listening on probe port 80
DNS resolution of 1 IPs took 0.02s.
Initiating Connect() Scan against mala.ccert.edu.cn (202.112.50.28)
[1680 ports] at 19:53
Discovered open port 21/tcp on 202.112.50.28
Discovered open port 80/tcp on 202.112.50.28
Discovered open port 1723/tcp on 202.112.50.28
Discovered open port 22/tcp on 202.112.50.28
Connect() Scan Timing: About 15.24% done; ETC: 19:57 (0:02:47 remaining)
Increasing send delay for 202.112.50.28 from 0 to 5 due to 12 out of 39 dropped
probes since last increase.
Increasing send delay for 202.112.50.28 from 5 to 10 due to 11 out of 11 dropped
probes since last increase.
Increasing send delay for 202.112.50.28 from 10 to 20 due to 11 out of 11 dropped
probes since last increase.
Connect() Scan Timing: About 25.71% done; ETC: 20:06 (0:09:35 remaining)
Increasing send delay for 202.112.50.28 from 20 to 40 due to 11 out of 11 dropped
probes since last increase.
Increasing send delay for 202.112.50.28 from 40 to 80 due to 11 out of 11 dropped
probes since last increase.
Increasing send delay for 202.112.50.28 from 80 to 160 due to max_successful_tryno
increase to 4
Increasing send delay for 202.112.50.28 from 160 to 320 due to max_successful_tryno
increase to 5
Discovered open port 995/tcp on 202.112.50.28
Discovered open port 543/tcp on 202.112.50.28
Discovered open port 3306/tcp on 202.112.50.28
Connect() Scan Timing: About 92.72% done; ETC: 20:07 (0:01:00 remaining)
Discovered open port 544/tcp on 202.112.50.28
The Connect() Scan took 866.71s to scan 1680 total ports.
Initiating service scan against 8 services on mala.ccert.edu.cn (202.112.50.28) at 20:08
The service scan took 108.58s to scan 8 services on 1 host.
Host mala.ccert.edu.cn (202.112.50.28) appears to be up ... good.
Interesting ports on mala.ccert.edu.cn (202.112.50.28):
Not shown: 1659 closed ports
PORT     STATE    SERVICE           VERSION
21/tcp   open     ftp               vsftpd 1.1.3
22/tcp   open     ssh               OpenSSH 3.5p1 (protocol 1.99)
80/tcp   open     http              Apache httpd 2.2.4 ((Unix) mod_ssl/2.2.4
OpenSSL/0.9.7a DAV/2 PHP/5.2.3)
92/tcp   filtered npp
113/tcp  filtered auth
135/tcp  filtered msrpc
137/tcp  filtered netbios-ns
138/tcp  filtered netbios-dgm
139/tcp  filtered netbios-ssn
445/tcp  filtered microsoft-ds
543/tcp  open     klogin?
544/tcp  open     kshell?
593/tcp  filtered http-rpc-epmap
995/tcp  open     ssl/pop3          UW Imap pop3d 2001.78rh
1025/tcp filtered NFS-or-IIS
1433/tcp filtered ms-sql-s
1723/tcp open     pptp?
3306/tcp open     mysql             MySQL 4.0.21-log
4662/tcp filtered edonkey
4899/tcp filtered radmin
6881/tcp filtered bittorent-tracker
Service Info: OS: Unix

Anyway, I hope this means the attacker has not been successful in penetrating my authentication mechanism. I’m safe for now I guess, or am I?