Miscellaneous notes from converting Ubuntu 16.04 from NetworkManager/resolveconf to systemd-{networkd,resolved}.

In 14.04, system had the following /etc/network/interfaces:

auto br0
iface br0 inet dhcp
bridge_ports be0

and the following /etc/udev/rules/70-net-persistent.rules:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="XX:XX:XX:XX:XX:XX", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="be0"

But now that we've upgraded to 16.04, we have choices: stay with the old ifupdown, or migrate to NM or systemd-networkd. ifupdown still works just fine, though it may go away some day, doesn't have a convenient UI, etc. So let's try NetworkManager, which has a convenient CLI:

# nmcli con add type con-name br0 ifname br0
# nmcli con modify br0 bridge.stp no
# nmcli con modify br0 ipv6.addr-gen-mode stable-privacy
# nmcli con add type
# nmcli con add type bridge-slave con-name enp1s0f0 ifname enp1s0f0 master br0
# nmcli con up enp1s0f0

Unfortunately, it kills dhclient when tearing down the bridge prior to going to sleep and never restarts it after resume. The end result is that when the IPv4 lease expires there's nothing to renew the DHCP and IPv4 stops working an hour or so after resume.

So maybe let's try networkd. First, we set up the NIC in /etc/systemd/network/enp1s0f0.network:

[Match]
Name=enp1s0f0

[Network]
Bridge=br0

Next, we establish a bridge device in /etc/systemd/network/br0.netdev:

[NetDev]
Name=br0
Kind=bridge
MACAddress=XX:XX:XX:XX:XX:XX

Note that we're cloning the NIC's MAC address to the bridge for convenience. Finally, we need to actually configure the bridge (/etc/systemd/network/br0.network):

[Match]
Name=br0

[Network]
DHCP=ipv4

[DHCP]
UseDomains=true

This sets up the bridge to request IPv4 information via DHCP, and to use the domain provided by the server as a default search domain. Next, we disable NM and resolvconf in favor of the systemd replacements:

# systemctl stop NetworkManager
# systemctl disable NetworkManager
# systemctl stop resolvconf
# systemctl disable resolvconf
# systemctl enable systemd-networkd
# systemctl start systemd-networkd
# systemctl enable systemd-resolved
# systemctl start systemd-resolved
# ln -sf ../run/systemd/resolve/resolv.conf /etc/resolv.conf

Reboot for changes to take effect.

Silly recipe for creating a SSD cache with LVM without wasting space on the SSD device:


# pvcreate /dev/slowdisk
# pvcreate /dev/fastdisk
# vgcreate plum_disk_cache /dev/slowdisk /dev/fastdisk
# lvcreate -n origin plum_disk_cache -l $size_of_slowdisk /dev/slowdisk
# lvcreate -n cache plum_disk_cache -l $size_of_fastdisk /dev/fastdisk
# lvconvert --type cache --cachemode writeback --cachepool plum_disk_cache/cache plum_disk_cache/origin --chunksize 512k -v --poolmetadataspare n
# lvreduce plum_disk_cache/cache -l -6 (or however many it claims to need)
# lvconvert --type cache --cachemode writeback --cachepool plum_disk_cache/cache plum_disk_cache/origin --chunksize 512k -v --poolmetadataspare n


A somewhat clumsy alternative method is this:

(Create VG, allocate "origin" LV across slow disks, add fastdisk to VG...)

# vgs plum_disk_cache -o vg_free_count
Free
67236
# lvcreate --type cache -n cache plum_disk_cache/origin /dev/fastdisk -l 67236 --cachemode writeback --chunksize 512k --poolmetadataspare n; done
# lvchange --cachepolicy smq plum_disk_cache/origin


(Replace 67236 with successive values until it succeeds, being careful to remove any LVs it erroneously creates while trying to set up the cache.)

Hi all,

Here's how you install Ubuntu 14 with a LUKS-encrypted / and /boot partition when using Apple's "quirky and fun" EFI:

1. Insert a USB stick, and boot the installer into "Try Ubuntu" mode (you need to be able to start an xterm before rebooting).


2. Install Ubuntu. On the hard disk, you'll want to create a 64M empty partition, a 256M /boot (unencrypted), and a LUKS container with LVM for /. Format the USB stick as an EFI system partition and let the installer put files there.


3. BEFORE rebooting the installer, change the HDD's empty 64M partition's type code (call it /dev/sda1) to Apple Boot (gdisk type code AB00) and format it HFS+.


4. Mount the new / to /mnt, and the new /boot to /mnt/boot. (The installer might just leave these two filesystems mounted at the end of the installation; I don't remember.


5. Mount the new Apple Boot partition (on the HDD). We'll say you mounted it to /mnt/boot/efi.
   
6. touch /mnt/boot/efi/mach_kernel


7. mkdir -p /mnt/boot/efi/EFI/ubuntu/ /mnt/boot/efi/System/Library/CoreServices/


8. cd /mnt/boot/efi/EFI/ubuntu/


9. ln -s ../../System


10. touch mach_kernel


11. chroot /mnt/ grub-install -v (in theory grub2 knows how to generate Mac-compatible files, though it puts them in the wrong location)


12. Reboot to Mac OSX.


13. diskutil mount /dev/disk0s1


14. Open Preferences, Startup Disk, select the Ubuntu install.


15. Reboot the system to prove that Linux boots correctly.


16. apt-get purge grub-efi-amd64-signed (the signed grub image does not have LUKS/cryptodisk support)


17. Copy the contents of /boot somewhere, and note the device (say /dev/sda2 for this example).


18. cryptsetup luksFormat /dev/sda2 -c aes-xts-plain64
    DO NOT install LVM here; grub2 refuses to allow FS writes to LVM volumes, which is needed for recordfail.


19. Create an entry in /etc/crypttab for the new LUKS container. We'll assume you called the device-mapper node "boot_crypt".


20. /etc/init.d/cryptdisks-early start (prove that crypttab works)


21. mkfs.ext4 /dev/mapper/boot_crypt -L boot (or whatever filesystem you want here)


22. Edit /etc/fstab to point /boot to /dev/mapper/boot_crypt.


23. Copy the saved files from step 17 into the new /boot.


24. echo 'GRUB_ENABLE_CRYPTODISK=y' >> /etc/default/grub


25. grub-install -v (verify that grub-mkimage is called with luks/cryptodisk modules included)


26. update-grub


27. Reboot to Mac OSX.


28. diskutil mount /dev/disk0s1


29. Open Preferences, Startup Disk, select the Ubuntu install. Again.


30. Reboot. Grub should now prompt to unlock the disk before showing the boot menu. For extra credit, set up LUKS keys for automount, since the only thing unencrypted on your HDD is grub, and (in theory) you could stash the rootfs LUKS keys in the initramfs. Macs don't support SecureBoot, at least not the 2012 models.

N.B. The Apple firmware will try to boot whatever the HFS+ startup file points to; using efibootmgr does no good here. If you rewrite the boot.efi file the safe way (write boot.efi.new, mv boot.efi.new boot.efi) you'll have to set the startup disk in OSX again, because the "startup file" is really an extent map in the HFS+ superblock. Quite possibly OSX simply copies the contents of boot.efi into the blocks pointed to by the startup file when a Startup Disk is set.

Hi all,

Here's how you install Ubuntu 14 with a LUKS-encrypted / and /boot partition when using legacy BIOS:

1. Install Ubuntu. You'll want to create a 256M /boot (unencrypted), a LUKS container with LVM for /.


2. Boot the system to prove that it boots correctly.


3. Copy the contents of /boot somewhere, and note the device (say /dev/sda1 for this example).


4. cryptsetup luksFormat /dev/sda1 -c aes-xts-plain64
   DO NOT install LVM here; grub2 refuses to allow FS writes to LVM volumes, which is needed for recordfail.


5. Create an entry in /etc/crypttab for the new LUKS container. We'll assume you called the device-mapper node "boot_crypt".


6. /etc/init.d/cryptdisks-early start (prove that crypttab works)


7. mkfs.ext4 /dev/mapper/boot_crypt -L boot (or whatever filesystem you want here)


8. Edit /etc/fstab to point /boot to /dev/mapper/boot_crypt.


9. Copy the saved files from step 3 into the new /boot.


10. echo 'GRUB_ENABLE_CRYPTODISK=y' >> /etc/default/grub


11. grub-install -v (verify that grub-mkimage is called with luks/cryptodisk modules included)


12. update-grub


13. Reboot. Grub should now prompt to unlock the disk before showing the boot menu. For extra credit, set up LUKS keys for automount, since the only thing unencrypted on your HDD is grub, so in theory you could stash the key for the rootfs LUKS keys in the initramfs.

Let's face it folks, Apple's firmware isn't EFI1 and it isn't UEFI2 either. It's a weird "hybrid", i.e. mess. I posted quite some time ago about how to make AAPLEFI find grub2-efi, but it's slow because the firmware scans all boot devices looking for bootx64.efi files, and it's not easily recoverable if your machine forgets one of the pieces it needs. What you really want is fastboot mode where the EFIVARS retains the location of the bootloader. This is /fairly/ simple if you're already booted with BIOS/CSM.

(Hopefully you left enough space to create a EFI boot partition somewhere. It can be quite small.)

1. Format disk (say /dev/sdX) with GPT label. Use gdisk.
2. Create smallish partition (64MB) with type AB00 and name 'Recovery HD'.
3. Format partition as HFS+.
4. Run blkid -o udev /dev/sdXY to ensure that you got a HFS+ with a UUID. The partition must have a UUID for the AAPLEFI to recognize it.
4. mkdir /efiboot and make an fstab entry to automount the small partition.
5. Mount partition.
6. mkdir -p /efiboot/EFI/grub/ /efiboot/EFI/boot/ /efiboot/System/Library/CoreServices/
7. apt-get install grub-efi-amd64
8. grub-install /dev/sdX
9. ln /efiboot/EFI/grub/grub.efi /efiboot/EFI/boot/bootx64.efi
10. update-grub
11. touch /efiboot/mach_kernel
12. ln /efiboot/EFI/grub/grub.efi /efiboot/System/Library/CoreServices/boot.efi
13. Dump the following into /efiboot/System/Library/CoreServices/SystemVersion.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>ProductBuildVersion</key>
<string></string>
<key>ProductCopyright</key>
<string></string>
<key>ProductName</key>
<string>Ubuntu</string>
<key>ProductUserVisibleVersion</key>
<string>12.04</string>
<key>ProductVersion</key>
<string>12.04</string>
</dict>
</plist>

14. Reboot to MacOS.
15. diskutil mount /dev/diskXsY
16. bless --folder=/Volumes/some_volume --file=/Volumes/some_volume/efi/grub/grub.efi --setBoot. Or you can use the Startup Disk Preference panel to select the disk.
17. Reboot to Linux. Hopefully it comes up super-fast; if not, it'll eventually find the bootx64.efi.
18. When you're satisfied, go back to gdisk and tell it to recreate protective MBR. This erases your legacy grub2 installation.

You are going to create a chroot on a VM host, then set up a PXE environment to net-boot the system. Let %CHROOT% be the chroot location on the host; %VM_NET_IP% be the IP address of the local VM network; %VM_NET_CIDR% be the CIDR of the local VM network; and %VM_NET_BRIDGE% be the name of the bridge that ties together the local VM network.

1. Use debootstrap to set up a chroot at %CHROOT%.
2. Add full set of Ubuntu repos to %CHROOT%.
3. Use djwong repo install.sh to install djwong package sources.
4. chroot %CHROOT% su - root
5. apt-get install djwong-mtr djwong-kernel-support
6. Install some sort of kernel.
7. Jump out to host.
8. virsh net-destroy default
9. virsh net-edit default
10. Add "<tftp root="%CHROOT%/boot/" />" as a child of the <ip> element.
11. Add "<bootp file="pxelinux.0" />" as a child of the <dhcp> element.
12. virsh net-start default
13. apt-get install nfs-kernel-server
14. Add "%CHROOT%/ %VM_NET_CIDR%(ro,sync,no_root_squash,no_subtree_check)" to /etc/exports. Do not be tempted to add "-H %VM_NET_IP%" to /etc/default/nfs-kernel-server; this will result in NFS3 being disabled due to braindead nfs init script!
15. /etc/init.d/nfs-kernel-server restart
16. ufw enable
17. ufw allow ssh
18. ufw allow cups
19. ufw allow in on %VM_NET_BRIDGE%
20. Create a VM with e1000 (not virtio-net) NIC, set to boot PXE.
21. Start VM.

It's tinfoil hat time! Let's say you run Debian or Ubuntu, maintain your own package archive, and want to make it harder for someone to tamper with that archive and trick you into installing malicious packages. If you want to sign your Debian packages and apt archive, here's a quick guide to how you set this up. ( Read more... )

One of my basement computers has one of these Intel High-Definition Audio sound cards. There are 6 analog mini jacks on the back, 5 of which are outputs and 1 which is an input. The outputs are: stereo line out, stereo rear out, stereo side out, center, and subwoofer (LFE). Given that I have two ultra-quiet speaker zones in my house (rooms where I want to be able to listen to computer music without the sound pollution of a computer), wouldn't it be neat to route two *different* sound sources to the two stereophonic outputs that are connected to the two zones?

( Read more )

Tonight I head into the horrible world of EFI GPT vs. MS-DOS MBR and all the insanity that is dealing with A**** and I** firmware design choices. All this to see if I could enable the emergency text console on my iMac, and also to test overriding bad BIOS choices to enable high-performance disk controllers.

( Read more )