oVirt node-ng update fix

Having problems updating some of your oVirt nodes? Chances are you might have a var_crash volume you need to manually remove and/or ran out of space on your PV/VG if you are running rather small disks on your node.

For a quick fix, SSH into your node and run those:

1
2
3
4
lvremove /dev/<mapper>/<erroneous install>.0
lvremove /dev/<mapper>/<erroneous install>.0+1
lvremove /dev/<mapper>/var_crash
fstrim -av

To upgrade your image, then run

1
yum update ovirt-node-ng-image-update

Reboot your node and run an upgrade via the engine webinterface once again to make sure you’re now on the latest release.

Noisy Focusrite Scarlett and how to fix it

Having noise issues on your external USB audio interface when connected to a desktop PC and powered studio monitors? Chances are you’re having issues with your grounding… Make sure you are using balanced cables, meaning three wires (two phases + ground) to connect your speakers to the output of your interface.

I recently spent quite a few hours chasing down an extremely annoying crackling hum on my audio setup because of that. No ground loop, all devices connected to the same circuit, I even hooked everything up to an online UPS so I get perfect sine voltage but to no avail. When connected to my main workstation there was a noticeable and extremely annoying background noise on my monitors that wasn’t there when I plugged the Focusrite Scarlett 2i2 into my notebook or mobile phone via an USB OTG adapter. Heck, it wasn’t even there when I plugged it into an HP Z600 workstation that is sitting right beside my main computer, all connected to the same UPS, ethernet switches and whatnot…

Turns out I used unbalanced cables from the Scarlett to the monitors and apparently the MSI X99S mainboard (or maybe the power supply?) in my primary desktop computer has for some reason quite a bit of noise on the ground of the USB bus. Or something like that – I didn’t bother to get an oscilloscope to verify.

Swapping the cables from TS to proper TRS (tip-sleeve “mono” to tip-ring-sleeve “stereo” aka balanced) fixed it and everything sounds perfect, no matter what computer the interface is connected to.

Dockerized Ubuntu mirror via nginx proxy

Recently one of my older servers died and I decided to move its data and services to other, newer systems and get rid of the old power hungry hardware. One those services was my local Ubuntu mirror for all the other servers in that colo. Accelerating package updates is nice, but storing hundreds of GB data that is rarely if ever used isn’t. Better replace it with a small reverse proxy, basically mirroring those mirrors I regularly use and save only the stuff that’s actually requested…

Squid seems a bit much and way too complex for this, Varnish works but would need another small webserver besides it if I want to serve some local files (for example misc. ISO images) as well, so… let’s go with nginx. A very lightweight and fast http server and proxy in one package. And let’s run it as a docker service so I can quickly deploy it wherever I want in the future by copying just two files:

docker-compose.yml

1
2
3
4
5
6
7
8
9
10
mirror:
  image: nginx
  volumes:
   - ./mirror.conf:/etc/nginx/conf.d/default.conf
   - ./index.html:/var/www/index.html
   - ./repo-ubunu:/var/repo_mirror/
   - ./cdimages:/var/www/cdimages/
  ports:
   - "80:80"
  command: /bin/bash -c "nginx -g 'daemon off;'"

And the mirror.conf with the nginx configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
upstream ubuntu {
# choose your nearest mirror  
  server de.archive.ubuntu.com;
  server uk.archive.ubuntu.com;
  server us.archive.ubuntu.com;
  server archive.ubuntu.com backup;
}
 
    tcp_nopush on;
    tcp_nodelay on;
    types_hash_max_size 2048;
 
 
    # where the cache is located on disk
    # to keep the data persistent, make it a docker volume
    proxy_cache_path /var/repo_mirror # defines where the cache is stashed
 
    # defines cache path heirarchy
    levels=1:2
 
    # defines name and size of zone where all cache keys and cache metadata are stashed.
    keys_zone=repository_cache:50m
 
    # data access timeout - don't cache packages for more than two weeks
    inactive=14d
 
    # Cache size limit
    max_size=10g;
 
server {
 
  listen 80;
 
  root /var/www/;
 
  # some additional ISO files on the mirror, added via docker volume
  location /cdimages/ {
  autoindex on;
  }
 
  # don't log in production mode, way too much info
  access_log off;
 
  # Location directive for the /ubuntu path
  location /ubuntu {
    
    # cache root, see above
    root /var/repo_mirror/index_data;
 
    # look for files in the following order
    try_files $uri @ubuntu;
  }
 
  # directive for the location defined above
  location @ubuntu {
 
    # map to upstream
    proxy_pass http://ubuntu;
 
    # two weeks of caching for http code 200 response content
    # 15 minutes for 301 and 302
    # one minute for everything else
 
    proxy_cache_valid 200 14d;
    proxy_cache_valid 301 302 15m;
    proxy_cache_valid any 1m;
 
    # set "repository_cache" zone defined above
    proxy_cache repository_cache;
 
    # Use stale data in those error events
    proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
 
    # go to backup server those error events
    proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
 
    # lock parallel requests and fetch from backend only once
    proxy_cache_lock on;
 
    # set some debug headers, just in case
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded_For $proxy_add_x_forwarded_for;
    add_header X-Mirror-Upstream-Status $upstream_status;
    add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
    add_header X-Mirror-Status $upstream_cache_status;
  }
}

Adjust to your liking – this currently works for me.

HP ProLiant: 601 – diskette controller error, what it really means

I recently got this puzzling error message on an HP ProLiant Server: 601 – Diskette Controller Error with the system just stopping at the BIOS, no way to enter the maintenance menu or continue booting. Well that’s odd, I though… being a BL460c blade server. All the documentation I could find online suggested it’s a defective floppy drive, or loose cabling… which a blade server doesn’t have.

Ok, so it’s a legacy error message. Maybe the BBWC? Nope. Backplane module then? Nope. The disks? Nah. The fucking SD card module? Not even that… Well, let’s try a system reset them using the DIP switches. Nope. Switching to backup ROM and booting with that one… Doesn’t work either. Well, fuck. Remove all the mezzanine cards – basically the system stripped down to the bare minimum. Still the same error.

About three hours later… Being the only removable part left, I figured I remove the CMOS battery and check it while I’ve it remove. Turns out it’s dead. Replace it with a new CR2032 button cell and voila – system boots again as if nothing happened.

So there you go… 601 – diskette controller error apparently really means: your CMOS battery is dead, better replace it. Even though the documentation says otherwise.

Dear HP: Fuck you. I want those hours of my life back.

phpMyAdmin root login vs. MariaDB

So I just installed a new Ubuntu 16.04 Xenial box with MariaDB 10.0 and phpMyAdmin but for some reason just can’t login as root user via phpMyAdmin, although it works via the shell. Turn’s out MariaDB enabled the unix_socket plugin-in for the root user, preventing phpMyAdmin from working out of the box like it does with native MySQL. So much for drop-in replacement – figuring that one out was quite frustrating…

Here’s how you disable it:

1
echo "update user set plugin='' where User='root'; flush privileges;" | mysql --defaults-file=/etc/mysql/debian.cnf mysql

Enable pci passthrough for QEMU/KVM

I have a couple older HP ProLiant DL360 Gen 6 and Gen 7 Server I want to use as virtual machine hosts with some PCIe devices mapped through to the guests running there, let’s say so they can directly access the Fibre Channel ports on the host or whatnot. Here’s how to do it (running Ubuntu 14.04 LTS that is, YMMV with other distributions):

  1. Edit your /etc/default/grub and add the intel_iommu=on boot option, it should look something like this (serial console for ILO enabled as well)
    1
    GRUB_CMDLINE_LINUX="console=tty0 console=ttyS1,115200n8 intel_iommu=on"
  2. Add the vfio module to the list in /etc/modules
    1
    2
    3
    4
    5
    6
    7
    8
    # /etc/modules: kernel modules to load at boot time.
    #
    # This file contains the names of kernel modules that should be loaded
    # at boot time, one per line. Lines beginning with "#" are ignored.
    # Parameters can be specified after the module name.
    lp
    rtc
    vfio
  3. Create a file like /etc/modprobe.d/vfio.conf setting the allow_unsafe_interrupts option for the vfio module.
    1
    options vfio_iommu_type1 allow_unsafe_interrupts=1

    Otherwise you’ll get an error when trying to start a VM with a PCI device passed through: “failed to set iommu for container: Operation not permitted”
    Apparently the IOMMU interrupt remapping is kinda broken on this platform.
  4. Reboot your server and create a VM with one of your PCIe adapters passed through – it should show up just like if was a native device on a real, physical server.

Updating a Cisco WIP 310

I recently got a used Cisco WIP 310 VoIP phone for fairly cheap and figured it would be nice to have a wireless handset as well. Or so I thought… because the thing still had it’s original firmware installed and turned out to be fucking useless. Couldn’t even get it to find my wireless network, let alone login or do anything but crash randomly.

Let’s see if there’s a new firmware available… great, it is! 5.0.13 as compared to 5.0.11 that was installed. But of course the firmware update via the supplied updater program didn’t work either. Manually copied the file to the handset and rebooted, came up with 5.0.11 again. Great…

So, long story short – here’s the way I finally managed to update it. Hope it helps someone else:

  1. Set up an open wireless network. For some reason the handset just wouldn’t work with my WPA2 secured network. You’ll need this only for the update procedure.
  2. In the wireless settings, change the IP configuration from DHCP to some static address on your network. Because for some reason DHCP didn’t work either…
  3. Now try to connect your WIP 310 to the network. It may take a couple of tries and/or reboots. Don’t ask me why, but eventually it did connect and I could ping the device.
  4. Open up the IP-Address of your phone in a web browser. Make sure the webserver is enabled: Settings -> Misc Settings -> Enable Web Server
  5. Go to ‘Admin Login’ on the upper right corner. On the ‘Ext 1’ tab set ‘Line Enable’ to ‘no’ and on the ‘Provisioning’ make sure ‘Upgrade Enable’ is set to yes
  6. Now for the magic… Extract the firmware image from the ZIP file and upload it to some Webserver. Make sure you can reach that server from your open wireless network so your phone can access that file.
  7. Schedule a firmware update via your browser: Go to http://your-phone-ip-address/admin/upgrade?http://your-server/path-to/wip310-5.0.13-04-26-2010.img
    You’ll get a message that the phone will update the firmware when idle. Wait a minute or two, maybe watch the Logfile on your webserver in case you have access. Eventually you’ll see a request by a client identifying as “Cisco/WIP310-5.0.11(10301355) (YOUR-PHONES-SERIAL)” or something like that.
  8. Wait for your phone to reboot. Reload the Admin Page in your browser to make sure it’s updated to 5.0.13

After that, you’ll notice the GUI on your phone looks a little different as well. And most importantly… it finally works. It also works with WPA2, hidden SSID networks and whatnot. It still has some bugs registering on some SIP servers. Even though it sometimes works, every now and then you’ll get a not registered error message. To fix that, take a look at the supplied PDF with the release notes. It tells you to change a couple settings:

  1. On the ‘SIP’ tab look for SIP Timer Values, the SIP T1 timer need to be changed from 0.5 to 2
  2. Also on the ‘SIP’ tab look for Response Status Code Handling, the Retry Reg RSC value needs to be set to 401
  3. Now got to the ‘Ext 1’ tab, set ‘Line Enable’ to yes again and enter your SIP settings to login to your SIP service.

After all that is done, the phone finally appears to be working the way it’s supposed to.