NetTraffic - HTTP(S) parser
Introduction
Module allows live bandwidth usage monitoring of remote / local devices (routers, modems, hosts with any OS) providing statistics page (counters values) over HTTP(S). Optional authentication: Basic, Digest and form based (POST / GET).
Data from parsing are used by live data rate chart, statistics and quota counter (if enabled).
Counters values may come from device statistics page or especially prepared page.
Configuration
.webp)
Counters address (URL)
Copy URL of statistics page from your browser's address bar.
Some devices uses framesets. In this case, provide address of appropriate statistics frame (inspect using F12 in browser).
Authentication type Basic / Digest
Fill fields: User, Password (one or both, depending on your device requirement). Type is detected automatically.
Authentication through form
Form based authentication requires two URLs (both absolute). First URL to page (frame) with statistics counters - described above Counters address (URL). Second URL - address of form action, where data from login (sign in) form are sent, called here Form destination (action URL).
Source of login (sign in) page may be similar to following.
<form action="here_is_second_url" ... method="...
<input type="text" name="field1" ...
<input type="password" name="field2" ...
<input type="hidden" name="field3" value="defined_value_3" ...
...
<input type="..." name="fieldN" ...
</form>
Field Form destination (action URL) must be filled with absolute address, even if form action address (here_is_second_url) is relative.
Method POST
POST parameters (field Form input fields) have to be written in such scheme, one per line.
field1="here_value_of_field_1" field2="here_likely_password" field3="defined_value_3" ... fieldN="value_of_field_N"
Timeout for counters page is 2s, for authentication page 5s.
Regular expressions
Rules:
- separate expression for sent bytes, separate expression for received bytes,
- each expression must contain single catching group,
- each expression must catch integer.
When option Replace \n to \r\n (LF to CRLF) is checked, before matching expressions, Linux style end of line is replaced with Windows style end of line.
Options
If option Add detected difference resulting from reset or overflow of source counters is checked and any of current session opening values (sent, received) is less than corresponding closing value in previous session, then opening values of current session becomes first data point.
If option Add difference from previous session is checked, difference between opening values in current session and closing values of previous session becomes current session first data point. Else case, or when difference (sent or received) is negative (and first described option is not checked), first data point is zero.
Notice
Avoid using HTTP instead of HTTPS on public networks.
Username and password (stored in configuration file) are encoded, not encrypted.
When you use statistics page of device, if possible login as read-only (unprivileged) user.
Prepared counters page
Below is exemplary PHP script, for hosts with Linux (ifconfig based), that creates page with counters values.
Before use, configure it according to your needs.
Script
<?php
/**
* You might use and modify this script as you like.
*
* THIS SCRIPT IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SCRIPT
* OR THE USE OR OTHER DEALINGS IN THE SCRIPT.
*/
/** Configuration. */
class Cfg {
/** User name. */
const AUTH_USER = 'set_username_here';
/** Password. */
const AUTH_PASS = 'set_password_here';
/** Realm, not necessary. */
const AUTH_REALM = '';
/**
* If array empty, all IPs allowed.
* Empty it or insert your own IP list.
*/
const IP_FILTER = [
'127.0.0.1',
'192.168.0.1',
'10.0.0.1'
];
/** Interface name. */
const READ_IFACE = 'eno1';
/** Minimize output. */
const WRITE_MIN = TRUE;
/** Append timestamp to output. */
const WRITE_TS = TRUE;
}
/** Worker. */
class Cnt {
public function Page() {
$this->IPFilter();
$this->AuthBasic();
$this->PlainWrite();
}
private function IPFilter() {
if (empty(Cfg::IP_FILTER)) {
return;
}
$ip = filter_input(INPUT_SERVER, 'REMOTE_ADDR', FILTER_VALIDATE_IP);
if (!in_array($ip, Cfg::IP_FILTER)) {
header("HTTP/1.1 403 Forbidden");
exit();
}
}
private function AuthBasic() {
$user = isset($_SERVER['PHP_AUTH_USER']) ? $_SERVER['PHP_AUTH_USER'] : NULL;
$pass = isset($_SERVER['PHP_AUTH_PW']) ? $_SERVER['PHP_AUTH_PW'] : NULL;
$valid = Cfg::AUTH_USER === $user && Cfg::AUTH_PASS === $pass;
if (!$valid) {
header('HTTP/1.1 401 Unauthorized');
$header = 'WWW-Authenticate: Basic';
if (!empty(Cfg::AUTH_REALM)) {
$header .= ' realm="' . Cfg::AUTH_REALM . '"';
}
header($header);
exit();
}
}
/** Plain text output. */
private function PlainWrite() {
$textIn = $this->Ifconfig();
$textOut = Cfg::WRITE_MIN ? $this->Reduce($textIn) : $textIn;
if (Cfg::WRITE_TS) {
$textOut .= $this->Timestamp();
}
header('Content-type: text/plain; charset="utf-8"');
echo $textOut;
exit();
}
private function Ifconfig() {
$cmd = 'ifconfig ' . Cfg::READ_IFACE;
$text = shell_exec($cmd);
return $text === NULL ? '' : $text;
}
/** Timestamp line. */
private function Timestamp() {
$time = time();
return "DT seconds $time (" . date('c', $time) . ')' . PHP_EOL;
}
/** Return only lines matching pattern. */
private function Reduce($in) {
$text = preg_replace('#\h+#', ' ', $in);
$out = '';
$groups = [];
$count = preg_match_all('#^\s*[RT]X.+bytes.+$#m', $text, $groups);
if ($count !== FALSE && $count > 0) {
$group = $groups[0];
foreach ($group as $line) {
$out .= trim($line) . PHP_EOL;
}
}
return $out;
}
}
$cnt = new Cnt();
$cnt->Page();
Launch
For testing purposes you might use built-in PHP web server.