USBIP on Debian 12 – USB Device Sharing over IP network

Udgivet den november 28, 2023 af mike

In a world were virtualization is slowly creeping in everywhere the demand for virtual USB cables are growing, luckily the USBIP protocol are implemented in the Linux kernel – and the userland tools are partly implemented in Debian, at least enough for us to mount it manually. If you want to automate it I made another post for that.

On both client and server:

Install the package:

root@system:~# apt-get install usbip

Load the main module and make it persistent:

root@system:~# modprobe usbip_core
root@system:~# echo usbip_core >> /etc/modules

Setting up the server:

The server needs an aditional kernel module:

root@server:~# modprobe usbip_host
root@server:~# echo usbip_host >> /etc/modules

Now you can share the usb device, I plugged in a USB stick into my server:

root@server:~# usbip list -l
 - busid 3-3 (054c:05b9)
   Sony Corp. : unknown product (054c:05b9)

This server only have one usb device attached making the output pretty simple, in this case the busid is “3-3”, but on your system it may have a different format depending on your hardware.

Bind this device to usbip:

root@server:~# usbip bind --busid=3-3
usbip: info: bind device on busid 3-3: complete

And finally start the deamon:

root@server:~# usbipd
usbipd: info: starting usbipd (usbip-utils 2.0)
usbipd: info: listening on 0.0.0.0:3240
usbipd: info: listening on :::3240

This is still a very manual process, later I will create an additional post with an example on how to automate this on Debian 12.

Setting up the client

The client also need a module:

root@client:~# modprobe vhci-hcd
root@client:~# echo vhci-hcd >> /etc/modules

And now you can connect to the remote server and list available devices:

root@client:~# usbip list -r server
Exportable USB devices
======================
 - backup1
        3-3: Sony Corp. : unknown product (054c:05b9)
           : /sys/devices/pci0000:00/0000:00:10.0/usb3/3-3
           : (Defined at Interface level) (00/00/00)

… and attach it:

root@client:~# usbip attach -r server -b 3-3

Which will cause it pop up in dmesg:

root@client:~# dmesg | tail -n10
[3425534.758312] usb-storage 5-1:1.0: USB Mass Storage device detected
[3425534.758672] scsi host9: usb-storage 5-1:1.0
[3425535.790312] scsi 9:0:0:0: Direct-Access     Sony     Storage Media    PMAP PQ: 0 ANSI: 6
[3425535.790979] sd 9:0:0:0: Attached scsi generic sg5 type 0
[3425535.792699] sd 9:0:0:0: [sde] 30375936 512-byte logical blocks: (15.6 GB/14.5 GiB)
[3425535.793509] sd 9:0:0:0: [sde] Write Protect is off
[3425535.793521] sd 9:0:0:0: [sde] Mode Sense: 45 00 00 00
[3425535.794265] sd 9:0:0:0: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[3425535.801659]  sde:
[3425535.801757] sd 9:0:0:0: [sde] Attached SCSI removable disk

Now you can mount the device:

root@client:~# mount /dev/sde /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
root@backup2:~# ls /mnt
boot  client_install  EFI  EULA  install.img  LICENSES  Packages  repodata  RPM-GPG-KEY-CH-8  RPM-GPG-KEY-CH-8-LCM  RPM-GPG-KEY-Platform-V1

And now your Debian Install USB stick from “server” is mounted on “client”, I have not yet done any kind of methodical testing so I don’t really know how the performance is, but right now I was more concerned with getting it to work, than what kind of performance it would deliver, it is probably “good enough” for anything except for maybe high latency gaming and huge data transfers.

But I still made a quick benchmark using:

root@system:~# hdparm -t /dev/sde

On both systems where i got around 200MB/sec on the local system and 85MB/sec on the remote client. But my current use-case is to connect printers and scanners, and the occasional USB Stick to transfer small amounts of files.

For any kind of serious ingestion of video materials I would not recommend this solution.

Detaching USB Device

root@client:~# umount /mnt
root@client:~# usbip port
Imported USB devices
====================
Port 15: <Port in Use> at Super Speed(5000Mbps)
       Sony Corp. : unknown product (054c:05b9)
       5-1 -> usbip://backup1:3240/3-3
           -> remote bus/dev 003/002
root@client:~# usbip detach -p 15
usbip: info: Port 15 is now detached!

Next task is to automate this.

Udgivet i Linux, Uncategorized | Skriv en kommentar

Logging output of cronjobs

Udgivet den oktober 25, 2023 af mike

If you wish to send output of cronjobs to syslog instead of email use the following:

20 10 * * * /opt/script/dostuff.py 2>&1 | /usr/bin/logger -t stuffdoer

If you want live-updates in the log file it can be necessary to unbuffer it, install the “expect” package and modify the command like so:

20 10 * * * /usr/bin/unbuffer /opt/script/dostuff.py 2>&1 | /usr/bin/logger -t stuffdoer

If it’s a python script you can accomplish the same thing by adding “-u” to the shebang.

Udgivet i Linux | Skriv en kommentar

PHP Script to parse nagios.dat

Udgivet den september 5, 2023 af mike

Please add your own error-handling:

#!/usr/bin/php
<?php

function NagiosParser($file)
{
        $status = file_get_contents($file);

        $current = null;
        $parsed = array();

        foreach (explode("\n", $status) as $line)
        {
                if (preg_match('/^([a-z]+) {$/', $line, $matches))              // Block start
                {
                        $block = array();
                        $current = $matches[1];
                }
                else if (preg_match('/}$/', $line, $matches))                   // Block stop
                {
                        $parsed[$current][] = $block;
                        $current = null;
                        unset($block);
                }
                else if (preg_match('/([a-z_]+)=(.+)/', $line, $matches))       // Not null value
                        if ($current == null)
                                die("Parse error: Value outside of block!");
                        else
                                $block[$matches[1]] = $matches[2];
        }
        return $parsed;
}
print_r(NagiosParser("status.dat"));
Udgivet i Nagios, PHP | Skriv en kommentar

Nagios module for monitoring smsdlog for modem-connectivity

Udgivet den marts 31, 2023 af mike 
#!/usr/bin/php
<?php

function secondsToTime($inputSeconds)
{
        $secondsInAMinute = 60;
        $secondsInAnHour  = 60 * $secondsInAMinute;
        $secondsInADay    = 24 * $secondsInAnHour;

        // extract days
        $days = floor($inputSeconds / $secondsInADay);

        // extract hours
        $hourSeconds = $inputSeconds % $secondsInADay;
        $hours = floor($hourSeconds / $secondsInAnHour);

        // extract minutes
        $minuteSeconds = $hourSeconds % $secondsInAnHour;
        $minutes = floor($minuteSeconds / $secondsInAMinute);

        // extract the remaining seconds
        $remainingSeconds = $minuteSeconds % $secondsInAMinute;
        $seconds = ceil($remainingSeconds);

        $res = array();

        if ($days > 0)
                $ret[] = "$days days";

        if ($hours > 0)
                $ret[] = "$hours hours";

        if ($minutes > 0)
                $ret[] = "$minutes minutes";

        if ($seconds > 0)
                $ret[] = "$seconds seconds";

        // return the final array
        $obj = array(
                'd' => (int) $days,
                'h' => (int) $hours,
                'm' => (int) $minutes,
                's' => (int) $seconds,
        );
        if (empty($ret))
                return "0 seconds";
        else
                return implode(", ",$ret);
}

date_default_timezone_set('UTC');
$cmd = exec("grep 'Signal Strength' /var/log/smstools/smsd.log | /usr/bin/tail -n 1");
if (!$cmd)
{
        print("CRITICAL: No lifesign from modem in current logfile\n");
        die(2);
}

list ($year, $month, $day, $hour, $min, $sec) = preg_split("/:|-| /", explode(",", $cmd)[0] );
$ts = mktime($hour, $min, $sec, $month, $day, $year);
$diff = time() - $ts;

if ($diff < 30)
{
        print ("OK: Last lifesign from modem ".secondsToTime($diff)." ago\n");
        die(0);
}
else
{
        print ("CRITICAL: Last lifesign from modem ".secondsToTime($diff)." ago\n");
        die(2);
}
Udgivet i Nagios, Uncategorized | Skriv en kommentar

Function to parse contacts from nagios config

Udgivet den marts 30, 2023 af mike 
#!/usr/bin/php
<?php
$contacts = file_get_contents("/etc/nagios4/objects/auto-contacts.cfg");


function parse_contacts($cfg)
{
        preg_match_all("/define contact{(.*?)}/mis", file_get_contents($cfg), $matches);
        foreach ($matches[1] as $match)
        {
                preg_match_all("/\s+(.+?)\s+(.+?)\n/mis", $match, $m);
                for ($i=0; $i&lt;count($m[1]); $i++)
                        $contact[$m[1][$i]] = $m[2][$i];
                $contacts[] = $contact;
        }
        return $contacts;
}


print_r(parse_contacts("/etc/nagios4/objects/contacts.cfg"));
Udgivet i Uncategorized | Skriv en kommentar

Tunneling HP ILO5 through SSH

Udgivet den oktober 27, 2022 af mike

Creates a tunnel to the ilo port, which in this example is 10.0.0.201 through the jumphost, after this you can connect to ILO by typing in https://locahost and accept the security warning, refreshing and logging in with your credentials.

 ssh jumphost.yourcompany.com -l root -L 443:10.0.0.201:443 -L 17990:10.0.0.201:17990 -L 80:10.0.0.201:80
Udgivet i Linux, SSH | Skriv en kommentar

Unmounting an NFS Share that has gone away

Udgivet den oktober 24, 2022 af mike

On Linux when an NFS share disappears it is not handled very well, to say it nicely, and in order to get a functioning machine again you need to drop that mount, but a simple umount command states that the device is busy.

So you need to both use the lazy and the force flag:

# umount -l -f /mnt/nfsshare
Udgivet i Linux | Skriv en kommentar

Fail2ban crash course

Udgivet den september 14, 2022 af mike

Just install fail2ban and the default settings will keep you protected just fine, after five failed login attempts the ip address of the offending login will be blocked for 10 minutes using iptables.

# apt-get install fail2ban

If you wish to keep tap on what is happening you can always tail the logfile:

# tail -f /var/log/fail2ban.log
... fail2ban.filter         [2374]: INFO    [sshd] Found 10.0.0.2 - 2022-09-14 22:08:01
... fail2ban.filter         [2374]: INFO    [sshd] Found 10.0.0.2 - 2022-09-14 22:08:05
... fail2ban.filter         [2374]: INFO    [sshd] Found 10.0.0.2 - 2022-09-14 22:08:08
... fail2ban.filter         [2374]: INFO    [sshd] Found 10.0.0.2 - 2022-09-14 22:08:16
... fail2ban.filter         [2374]: INFO    [sshd] Found 10.0.0.2 - 2022-09-14 22:08:20
... fail2ban.actions        [2374]: NOTICE  [sshd] Ban 10.0.0.2

Status of fail2ban can be shown with the following command:

# fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 2
|  |- Total failed:     13
|  `- File list:        /var/log/auth.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     2
   `- Banned IP list:   200.111.119.58

Unbanning an ip can be done with this command:

# fail2ban-client set sshd unbanip 196.216.253.24
1

If you wish to ignore one or more addresses create a file /etc/fail2ban/jail.d/whitelist.conf with the following command:

# echo -e '[DEFAULT]\nignoreip = 62.69.153.125' > /etc/fail2ban/jail.d/whitelist.conf

And restart fail2ban:

# systemctl restart fail2ban.service
Udgivet i Linux, Security | Skriv en kommentar

Check TLS Certificate with openssl

Udgivet den august 1, 2022 af mike 
echo | openssl s_client -servername customersite.com -connect servername.com:443 2> /dev/null | openssl x509 -noout -dates
notBefore=May  3 06:49:29 2022 GMT
notAfter=Aug  1 06:49:28 2022 GMT
Udgivet i Linux | Skriv en kommentar

Running and monitoring rsync with Python

Udgivet den juli 30, 2022 af mike 
#!/usr/bin/python3

import subprocess,sys,time,re
import pprint
import string

def execute(cmd):
        process = subprocess.Popen(cmd, shell=True,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
                universal_newlines=True)
        for stdout_line in iter(process.stdout.readline, ""):
                yield stdout_line
        process.stdout.close()
        return_code = process.wait()

        if return_code:
                raise subprocess.CalledProcessError(return_code, cmd)

for line2 in execute("rsync --info=progress2 -rl /usr/ ./mc"):
        line = ''.join(c for c in line2 if c.isprintable())
        result = re.search(r"([0-9\,]+)\s+([0-9]+)%\s+([0-9a-zA-Z\.\/]+)", line)
        if result:
                transfered = result.group(1)
                percentage = result.group(2)
                speed = result.group(3)
                print(f"Rsync process Transfered: {transfered} Percentage: {percentage} Speed: {speed}")
~
Udgivet i Python | Skriv en kommentar