Sunday, July 6, 2014

ntopng on Ubuntu 14.04 LTS Server

Following directions are deprecated, updated instructions are available at ntopng 1.2.2 on Ubuntu 14.04, revisited.


ntopng is the next generation version of the original ntop, a network traffic probe that shows the network usage, similar to what the popular top Unix command does. ntopng is based on libpcap and it has been written in a portable way in order to virtually run on every Unix platform, MacOSX and on Win32 as well. — http://www.ntop.org/products/ntop/

ntopng is different from the original ntop package found in the default Ubuntu repositories. Perhaps the biggest different between the two is that ntopng has an updated and modern HTML5 web interface which has data visualizations that update in real-time, whereas ntop provides a static and very generic looking web interface.

Installing ntopng

Note that packages from the repository below are x64 only. If you are using i386, then ntopng must be built from source.

Following the download pages on ntop.org and looking for stable Ubuntu packages will eventually lead you to this website repository. The directions below follow those in the listed website.

Add the http://www.nmon.net/apt-stable/ repository and its accompanying public key:

ubuntu@trustytahr:~$ sudo -i
[sudo] password for ubuntu: 

root@trustytahr:~# /bin/echo -e "deb http://www.nmon.net/apt-stable/ x64/\ndeb http://www.nmon.net/apt-stable/ all/" > /etc/apt/sources.list.d/ntop.list

root@trustytahr:~# wget -qO - http://www.nmon.net/apt-stable/ntop.key | sudo apt-key add -
OK
  • Switch completely to the root user, rather than using sudo to run the above commands.

Update apt repositories with apt-get update and ignore the conflicting messages at the end:

root@trustytahr:~# apt-get update

Reading package lists... Done
W: Conflicting distribution: http://www.nmon.net x64/ Release (expected x64 but got )
W: Conflicting distribution: http://www.nmon.net all/ Release (expected all but got )

Install the ntopng and ntopng-data packages:

root@trustytahr:~# apt-get -y install ntopng ntopng-data

The installation creates startup script /etc/init.d/ntopng, and the ntopng binary itself as /usr/local/bin/ntopng. The rest of the relevant files can be all found under the /usr/local/share/ntopng directory. See output of dpkg -L ntopng for a full listing.

Running ntopng

Starting ntopng is as simple as executing the /usr/local/bin/ntopng binary:

root@trustytahr:/usr/local/bin# ./ntopng 
06/Jul/2014 01:15:34 [Ntop.cpp:555] Setting local networks to 192.168.1.0/24,0.0.0.0/32,224.0.0.0/8,239.0.0.0/8,255.255.255.255/32,127.0.0.0/8
06/Jul/2014 01:15:34 [Redis.cpp:50] Successfully connected to Redis 127.0.0.1:6379
06/Jul/2014 01:15:34 [PcapInterface.cpp:81] Reading packets from interface eth0...
06/Jul/2014 01:15:34 [Ntop.cpp:662] Registered interface eth0 [id: 0]
06/Jul/2014 01:15:34 [PcapInterface.cpp:81] Reading packets from interface lo...
06/Jul/2014 01:15:34 [Ntop.cpp:662] Registered interface lo [id: 1]
06/Jul/2014 01:15:34 [Utils.cpp:251] User changed to nobody
06/Jul/2014 01:15:34 [main.cpp:152] PID stored in file /var/tmp/ntopng.pid
06/Jul/2014 01:15:34 [HTTPserver.cpp:351] HTTPS Disabled: missing SSL certificate /usr/local/share/ntopng/httpdocs/ssl/ntopng-cert.pem
06/Jul/2014 01:15:34 [HTTPserver.cpp:352] Please read README.SSL if you want to enable SSL
06/Jul/2014 01:15:34 [HTTPserver.cpp:389] Web server dirs [/usr/local/share/ntopng/httpdocs][/usr/local/share/ntopng/scripts]
06/Jul/2014 01:15:34 [HTTPserver.cpp:392] HTTP server listening on port 3000
06/Jul/2014 01:15:34 [main.cpp:186] Using RRD version 1.4.7
06/Jul/2014 01:15:34 [main.cpp:202] Working directory: /var/tmp/ntopng
06/Jul/2014 01:15:34 [main.cpp:204] Scripts/HTML pages directory: /usr/local/share/ntopng
06/Jul/2014 01:15:34 [Ntop.cpp:181] Welcome to ntopng x86_64 v.1.1.4 (r7806) - (C) 1998-14 ntop.org
06/Jul/2014 01:15:34 [PeriodicActivities.cpp:53] Started periodic activities loop...
06/Jul/2014 01:15:34 [NetworkInterface.cpp:770] Started packet polling on interface eth0...
06/Jul/2014 01:15:34 [NetworkInterface.cpp:770] Started packet polling on interface lo...
  • Above output shows ./ntopng run as a foreground process, optionally it could have been ran as a background process with ./ntopng &.
  • By default the working directory is /var/tmp and the listening port for the web interface is at port 3000.

While the process is running, the web interface should now be accessible via browser at http://<SERVER IP>:3000 — http://localhost:3000 if viewing the web page locally.

The default username is admin and password is admin.

Running ntopng as a service

Running ntopng as a service, rather than a foreground or background script requires additional steps.

Note that ntopng has several options that can be passed at either the command-line or via configuration file. All the options are listed in the man page.

A configuration file should be created at /etc/ntopng/ntopng.conf, below is a configuration file I used (the commented descriptions are excerpts directly taken from man ntopng):

# /etc/ntopng/ntopng.conf
#
#        The  configuration  file is similar to the command line, with the exception that an equal
#        sign '=' must be used between key and value. Example:  -i=p1p2  or  --interface=p1p2  For
#        options with no value (e.g. -v) the equal is also necessary. Example: "-v=" must be used.
#
#
#       -G|--pid-path
#        Specifies the path where the PID (process ID) is saved.
#
-G=/var/tmp/ntopng.pid
#
#       -e|--daemon
#        This  parameter  causes ntop to become a daemon, i.e. a task which runs in the background
#        without connection to a specific terminal. To use ntop other than as a casual  monitoring
#        tool, you probably will want to use this option.
#
-e=
#
#       -i|--interface
#        Specifies  the  network  interface or collector endpoint to be used by ntopng for network
#        monitoring. On Unix you can specify both the interface name  (e.g.  lo)  or  the  numeric
#        interface id as shown by ntopng -h. On Windows you must use the interface number instead.
#        Note that you can specify -i multiple times in order to instruct ntopng to create  multi‐
#        ple interfaces.
#
-i=eth0
#
#       -w|--http-port
#        Sets the HTTP port of the embedded web server.
#
-w=3000
#
#       -m|--local-networks
#        ntopng determines the ip addresses and netmasks for each active interface. Any traffic on
#        those  networks  is considered local. This parameter allows the user to define additional
#        networks and subnetworks whose traffic is also considered local in  ntopng  reports.  All
#        other hosts are considered remote. If not specified the default is set to 192.168.1.0/24.
#
#        Commas  separate  multiple  network  values.  Both netmask and CIDR notation may be used,
#        even mixed together, for instance "131.114.21.0/24,10.0.0.0/255.0.0.0".
#
-m=192.168.1.0/24
#
#       -n|--dns-mode
#        Sets the DNS address resolution mode: 0 - Decode DNS responses  and  resolve  only  local
#        (-m)  numeric  IPs  1  -  Decode DNS responses and resolve all numeric IPs 2 - Decode DNS
#        responses and don't resolve numeric IPs 3 - Don't decode DNS responses and don't  resolve
#
-n=1
#
#       -S|--sticky-hosts
#        ntopng  periodically purges idle hosts. With this option you can modify this behaviour by
#        telling ntopng not to purge the hosts specified by -S. This parameter requires  an  argu‐
#        ment  that  can  be  "all"  (Keep  all hosts in memory), "local" (Keep only local hosts),
#        "remote" (Keep only remote hosts), "none" (Flush hosts when idle).
#
-S=
#
#       -d|--data-dir
#        Specifies the data directory (it must be writable). Default directory is ./data
#
-d=/var/tmp/ntopng
#
#       -q|--disable-autologout
#        Disable web interface logout for inactivity.
#
-q=

Create the /etc/ntopng/ntopng.start as an empty file:

root@trustytahr:~# touch /etc/ntopng/ntopng.start

root@trustytahr:~# ls -l /etc/ntopng/ntopng.start
-rw-r--r-- 1 root root 0 Jul  6 01:41 /etc/ntopng/ntopng.start
  • For some reason the start_ntopng() function in the /etc/init.d/ntopng startup script requires this file in order for the service to start.

Start the ntopng service:

root@trustytahr:~# service ntopng start
Starting ntopng

The ntopng process can be verified with the service ntopng status or in ps aux output:

root@trustytahr:~# ps aux | grep [n]topng
nobody    2343  0.6  1.3 764432 27900 ?        Ssl  01:45   0:00 /usr/local/bin/ntopng /etc/ntopng/ntopng.conf

root@trustytahr:~# service ntopng status
ntopng running as 2343

Changing default password for web interface

Command-line Method

Verify connectivity to redis-server with redis-cli ping command:

root@trustytahr:~# redis-cli ping
PONG

Generate an md5 hash of our new password with md5sum:

root@trustytahr:~# echo -n "NewPassword_4ntopng" | md5sum
a13a83f9a23740e9ba453b2df3c41f9a  -

Use this hash in the redis-cli SET user.admin.password command:

root@trustytahr:~# redis-cli SET user.admin.password a13a83f9a23740e9ba453b2df3c41f9a
OK

Web Interface Method

After logging into the web interface:

Settings (Gear Icon) > Manage Users > Set Password

Troubleshooting

  • If the service fails to start, run /usr/local/bin/ntopng /etc/ntopng/ntopng.conf directly to see all messages on stdout.
  • The log file can be found at: /var/tmp/ntopng/ntopng.log
  • ntopng requires the redis-server service to be running:
  • root@trustytahr:~# service redis-server status
    redis-server is running
    
    root@trustytahr:~# netstat -tunlp | grep -i redis-server
    tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      2139/redis-server 1
    

Bug Fixes

The /etc/init.d/ntopng startup script has a few minor bugs that could use a couple fixes:

  • /usr/local/bin/ntopng can actually be run without the root user; however, the ntopng process will not be able to enter the listening interface into promiscuous mode and will still return a successful exit status.
  • The startup script parses for PID_FILE using grep to find the -G switch, but doesn't alternatively search for its synonymous long switch --pid-file.
  • The service ntopng status command will return nothing if ntopng hasn't started.

diff -u output for the above fixes:

root@trustytahr:~# diff -u /etc/init.d/ntopng.orig /etc/init.d/ntopng
--- /etc/init.d/ntopng.orig 2014-07-06 02:30:21.133552653 -1000
+++ /etc/init.d/ntopng 2014-07-06 02:44:55.057578340 -1000
@@ -13,6 +13,11 @@
 # Short-Description: Start/stop ntopng web
 ### END INIT INFO
 
+if [ $(id -u) -ne 0 ]; then
+    echo "ntopng requires root."
+    exit 1
+fi
+
 start_ntopng() {
     FORCE=$1
 
@@ -34,7 +39,7 @@
     if [ -f /etc/ntopng/ntopng.start ] || [ $FORCE -eq 1 ]; then
  echo "Starting ntopng"
 
- PID_FILE=$(cat /etc/ntopng/ntopng.conf | grep '\-G='|cut -d '=' -f 2)
+ PID_FILE=$(grep '^-G\|^--pid-file' /etc/ntopng/ntopng.conf | cut -d '=' -f 2)
         if [ -f $PID_FILE ]; then
      PID=$(cat $PID_FILE)
             if [ $PID -gt 0 ]; then
@@ -60,7 +65,7 @@
     fi
 
     if [ -f /etc/ntopng/ntopng.conf ]; then
- PID_FILE=$(cat /etc/ntopng/ntopng.conf | grep '\-G='|cut -d '=' -f 2)
+ PID_FILE=$(grep '^-G\|^--pid-file' /etc/ntopng/ntopng.conf | cut -d '=' -f 2)
  if [ -f "$PID_FILE" ]; then
      PID=$(cat $PID_FILE)
      if [ $PID -gt 0 ]; then
@@ -86,7 +91,7 @@
  return 0
     fi
 
-    PID_FILE=$(cat /etc/ntopng/ntopng.conf | grep '\-G='|cut -d '=' -f 2)
+    PID_FILE=$(grep '^-G\|^--pid-file' /etc/ntopng/ntopng.conf | cut -d '=' -f 2)
     if [ -f $PID_FILE ]; then
  PID=$(cat $PID_FILE)
  if [ $PID -gt 0 ]; then
@@ -94,6 +99,8 @@
  else
      echo "No running ntopng pid [$PID] in [$PID_FILE]"
  fi
+    else
+        echo "ntopng not running: process id file $PID_FILE not found."
     fi
     
     return 0

References

http://xmodulo.com/2013/10/set-web-based-network-traffic-monitoring-linux.html
http://blog.redbranch.net/2013/12/12/reset-ntopng-admin-password/

2 comments:

  1. Thanks for the instructions. I have ntopng up and running on Ubuntu 14.10. I can manually start by running "sudo service ntopng start", but ntopng does not automatically start at boot. I did copy your script into /etc/init.d. I checked /var/log/ntopng/ntopng.log and see the following messages:

    bruce@HomeServer:/var/log/ntopng$ more ntopng.log
    06/Feb/2015 22:25:09 [Ntop.cpp:461] Setting local networks to 192.168.1.0/24,0.0
    .0.0/32,224.0.0.0/8,239.0.0.0/8,255.255.255.255/32,127.0.0.0/8
    06/Feb/2015 22:25:09 [PcapInterface.cpp:54] Reading packets from interface eth0.
    ..
    06/Feb/2015 22:25:09 [Ntop.cpp:568] Registered interface eth0 [id: 0]
    06/Feb/2015 22:25:09 [Utils.cpp:235] User changed to nobody
    06/Feb/2015 22:25:09 [Ntop.cpp:435] Parent process is exiting (this is normal)
    06/Feb/2015 22:25:09 [main.cpp:147] PID stored in file /var/tmp/ntopng.pid
    06/Feb/2015 22:25:09 [HTTPserver.cpp:342] HTTP server listening on port 3000 [/u
    sr/share/ntopng/httpdocs][/usr/share/ntopng/scripts]
    06/Feb/2015 22:25:09 [main.cpp:179] Using RRD version 1.4.8
    06/Feb/2015 22:25:09 [main.cpp:188] Working directory: /var/lib/ntopng/ntopng
    06/Feb/2015 22:25:09 [main.cpp:190] Scripts/HTML pages directory: /usr/share/nto
    png
    06/Feb/2015 22:25:09 [Ntop.cpp:165] Welcome to ntopng x86_64 v.1.1 (r) - (C) 199
    8-13 ntop.org
    06/Feb/2015 22:25:09 [Redis.cpp:34] ERROR: Unable to connect to redis 127.0.0.1:
    6379
    06/Feb/2015 22:25:09 [Redis.cpp:37] ERROR: ntong requires redis server to be up
    and running
    06/Feb/2015 22:25:09 [Redis.cpp:38] ERROR: Please start it and try again or use
    -r
    06/Feb/2015 22:25:09 [Redis.cpp:39] ERROR: to specify a redis server other than
    the default

    When I enter "service redis-server status" I see redis-server running. I'm not sure if ntopng is trying to start before redis-server is completely up. What should I check next?

    ReplyDelete
  2. Hey Bruce, I wrote a new tutorial for installing ntopng. There was recent changes over the last half year, and the installation and configuration is much easier.

    See my new post http://rsabalburo.blogspot.com/2015/02/ntopng-122-on-ubuntu-1404-revisited.html

    ReplyDelete