PLAnet VisiON's IPTV Probe (Open Source)

open source software for IPTV streams monitoring
License: GPLv2, GPLv3

Source code - iptvprobe_v0.4.tbz2

Questions concerning IPTVProbe can be discussed with Planet Vision Technical Staff

Changelog

Version 0.4

• Periodic database cleanup.

• Multiple multicast streams monitoring per probe.

• Memory leak in iptvprobe_server fixed. Note! The database must be fully recreated when upgrading from previous versions.

Version 0.3

• IGMP requests processing and display of temporal IGMP parameters added.

• Code cleanup, minor bug fixes.

• Protocol of data exchange with the probes updated to the new version.

Version 0.2

• Aminet 110 (ppc) support added.

Version 0.1

• Primary release.

• Lost packets control added.

• Display of UDP packets reception speed added.

Network Scheme

Probe (iptvprobe) and Collector Operation Layout

Building and Installing the Software

The software consists of the following parts:

• Collector - collects statistics from probes. Collected statistics is stored into the database (mysql).

• Probe - is an application working remotely. Statistics on received IP packets is collected by the probe and sent to the collector. The probe consists of a Linux kernel module (Planet Vision_netprobe.ko) and a user level software (iptvprobe).

• Reports subsystem - is a set of web scripts to visualize collected statistics.

Building and Starting the Collector

Building the collector:

cd iptvprobe/userver/
cmake --debug-output . -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
make

Creating the database:

mysqladmin create iptvprobe
mysql iptvprobe < db.sql

Starting the collector:

./iptvprobe_server -l root -s 127.0.0.1

in this example the collector writes statistics into 'iptvprobe' database on localhost (127.0.0.1)

Building iptvprobe

Building the application for x86 platform:

cd iptvprobe/udaemon/
cmake --debug-output . -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
make

Building the Linux kernel module for x86 platform:

cd iptvprobe/kmodule/
make

Building the Linux kernel module for sh4 platform (IP STB AmiNET 130):

cd iptvprobe/kmodule/
export CROSS_COMPILE=sh4-unknown-linux-gnu-
make ARCH=sh amino130

The aminet130_bin/ directory contains an already-built Linux kernel module and a user level application for AmiNET 130 IP STB.

Starting the Probe

Loading the Linux kernel module:

cd iptvprobe/kmodule/
insmod Planet Vision_netprobe.ko hook_position=0

The hook_position parameter specifies IP packets hook position of netfilter:

0 - process all incoming IP packets (PREROUTING policy) 1 - process all outgoing IP packets (POSTROUTING policy)

Value "0" must be set when running the module on a client's device (for example, IP STB AmiNET 130). In case of running the module on a server sending multicast traffic, value "1" must be set.

It is required to make a node in /dev filesystem and to start the iptvpobe application when NetuP_netprobe.ko module is successfully loaded:

mknod /dev/iptvprobe c 61 0
cd iptvprobe/udaemon/
./iptvprobe -i 224.117.117.10 -s 10.1.4.242 -r 5 -p 7700

Command line options:

-i Sets the multicast address for monitoring
-s Specifies the IP address of the collector
-p Specifies the port for the collector to accept connections from probes
-r Specifies a run identifier

Fully assembled command can be found in the reports subsystem.

Reports Subsystem

In order to run the reports subsystem it is required to copy all the web scripts from iptvprobe/report_sys/ to cgi-bin folder of your web server:

cd iptvprobe/report_sys/
cp *.pl /var/www/localhost/cgi-bin/

perl and GD must be installed on the system for proper operation.

Point the web browser to the following URL:

http://address/cgi-bin/iptvprobe_runs.pl

where "address" is an address of your web server.

Start page contains the list of probe runs:

"Command to run on probe" column contains fully assembled command which should be run on the probe.

Graphic representation of multicast flow can be viewed using "UDP timeline diagram" or "PCR Arrival Interval" links. Approximately last 30 seconds are shown on the diagram.

Two example runs were done. The first was done on AmiNET 130 IP STB connected via a simple SOHO switch without IGMP snooping support. The second run was done on AmiNET 130 IP STB connected via a Cisco catalyst switch with IGMP snooping. In the first case there is a lot of IP packet losses. The following diagrams visualize these two cases:

Diagram Explanation

Approximately last 15-30 seconds are shown on the x axis.

The following parameters are shown on the y axis:

• Count of IP packets grouped within 100 msec.

• Number of bytes grouped within 100 msec.

• Red line indicates lost IP packets. A peak is shown on the diagram if discontinuity in "IP packet ID" has been detected. Value 0 means that no packet loss has been detected.

Database Description

SQL table	Description
runs	Runs list. Every run is identified by a unique number
data_ip	IP packets detected by the probe. IP packet arrival time (timestamp) is stored in nanoseconds. Also packet ID (header_id) is stored
data_ts	MPEG Transport Stream (TS) packets detected by the probe. The following values are stored: PID, PTS/DTS/PCR, counter values (cont_counter)
stat_bandwidth	Grouped statistics

IGMP monitoring

Since iptvprobe v0.3, the IGMP requests sent and received by an IP STB may be tracked by the probe. The timestamps of the last group packet before leaving a group and the first packet after joining one are also recorded. These data may be used to evaluate the performance of networking equipment that processes the IGMP requests and IGMP snooping.

Below is an example of this functionality with Aminet 130 IP STB for probe and Planet Vision Middleware for collector. Cisco Catalyst 3560 was used as a commutator with IGMP Snooping support, and Cisco Catalyst C3550-12T served as an IGMP querier.

TV channels were switched on the IP STB. At that moment the STB sends an IGMP request to leave the multicast group 224.121.0.4 and then another one to join the group at 224.121.0.3. These requests are recorded by the probe and displayed in the web interface together with the timestamps:

The items of our interest are embraced in blue frame. It is clear that after the IGMP leave request to the multicast group at 224.121.0.4, the STB continued to receive packets sent to the said group for another 4996.2 ms. Time delay between the IGMP join request to the 224.121.0.3 group and the first received packet for that group amounted to 80 ms. The delay between the IGMP requests (mainly determined by the STB software) was 12.0 ms.

Altogether, the total time of TV channel switching on the level of IGMP requests sums up to 92 milliseconds. The actual delay until the newly selected channel appears on the TV screen depends on many parameters, including MPEG buffering, audio/video sync, etc., and may comprise up to 1 second or even more.

Besides that, the STB has received the packets from both multicast groups for roughly 5 seconds, thus wasting extra bandwidth. To avoid such a loss, the commutator must be configured to instantaneous leave. This is done as follows:

c3560(config)#ip igmp snooping vlan 1 immediate-leave

When configured in this way, the commutator will switch off the data stream for the abandoned group in an instant. The results are shown in the red frame. It may be seen that after the leave request "IGMPv2: Leave Group 224.121.0.11" the STB has received no packets for the 224.121.0.11 group. Therefore, at any given moment the STB receives packets of only one channel.

For correct recording of temporal parameters of IGMP requests, the network interface must be set to promisc regime before starting the probe on the STB:

# ifconfig eth0 promisc

TODO List

• Diagram improvements (scale, shifting, etc.)

• "PCR jitter" diagram, other diagrams

• Probe binary builds for various types of IP STB's

• Probe availability monitor for network management