linux · gentoo · 2017-01-04 · yuex

Introduction

I have been fiddling with MacBook Pro in the past few days. The Quartz Desktop is really awesome. MacOS has got some software supports as pretty as Windows. Graphical applications like Steam and BattleNet are available. But still I didn't get used to BSD toolchains.

At first thought, I told myself not to be so close-minded. So I tried Gentoo Prefix, then MacPorts and HomeBrew. Gentoo Prefix was bootstrapped successfully after manually patched several packages. But the support is not excellent (as stated on Gentoo Prefix's Gentoo wiki page). Some packages' compilations are broken on MacOS because the environment is so different to Linux. Let alone some Linux-specific packages like procps are no way to compile. It seems that there is also a lack of maintainers for Gentoo Prefix on Mac. After all, it's the amd64 Linux where Gentoo shines.

So I turned to MacPorts and HomeBrew.

HomeBrew is awesome. The development is active. The community is large. But the problem is it doesn't handle dependencies well. To me, after the revelation to emerge, this is a huge defect. But it doesn't mean the HomeBrew finds no place in my toolbox. Still I am very happy to use HomeBrew Cask to install GUI apps which has no dependencies. Besides, HomeBrew doesn't provide packages to install pip or cabal modules yet.

Like Portage, MacPorts derives from Unix Port too. But its toolkits are somehow primitive to Portage. You have no way to search files with packages like yum whatprovides unless you have installed it. For Portage, this function is provided by e-file command of pfl which search in an external indexing database. And there are also some broken dependencies. For example, VirtualBox Guestion Additions requires gcc-42 to build. But this package is not provided for MacOS beyond El Capitan. Also the ghc package has been broken for quite a while and doesn't show any indication to change (the maintainer explained the obstacles in the mailing list in Q3 or Q4 of 2016).

The conclusion is that to my knowledge Gentoo Portage on amd64 is still the best choice. Let alone there is also layman by which you can introduce an extra layer to the Portage tree where you can customize your our own versions of some packages. This feature is not provided by HomeBrew or MacPorts yet.

Now the problem is how to get myself a Linux environment on MacOS. First, I tried Docker. But with no Linux kernel, Docker for Mac also has to use a VM to map Linux API to OSX. Well, this just give it no pros. The cons comes from ctrl-p. I have no idea why I have to type ctrl-p twice in the Docker tty to go last command. The ctrl-n and up arrow are good. Also by using Docker, we have no way to customize the kernel.

So finally, I decided to go to virtual machine. One of the cons is that you have to go through the installation again. Like I said, it's fun but tedious. The first thought was that I had to use vagrant. The second thought is that if someone had already prepared some Gentoo images. I checked out the most downloaded Gentoo image from vagrant box list. But later I found it was using OpenRC instead of systemd. and disk was not using LVM. That was why I decided to DIY a vagrant box.

I also tried chroot. But I can not do it natively on MacOS. Because I want a pure GNU/Linux environment. I tried to use docker or virtualbox to start a Linux kernel and then chroot to shared folder to install Gentoo Portage. But the problem is that both vboxfs and osxfs have some problems. vboxfs doesn't allow permission change within the guest OS which will cause you a permission deny hell even if you switched to root. osxfs has some performance issue just like vboxfs. But the most severe problem is that osxfs will just hang. I have to reset Docker for Mac to kill it. But this way works on Linux. So I guess it's a bug related to Moby VM or osxfs.

Anyway, to get yourself a vagrant box, there are just two steps

  • First, you install a Gentoo to a VM
  • Second, you packge the VM and put it online

Install Gentoo to VirtualBox

The process is very similar to install it to a PC but somehow different to my previous post. First, we have to prepare the disk using parted. Second, we also need to configure VirtualBox a little bit to be able to ssh to.

Still, we use [SystemRescueCD][]. Insert it into the Optical Drive of the VM. By default, VirtualBox's network uses NAT. To make ssh easier, we change it to Bridged Adapter to get the VM a dedicated IP address so that we needn't to configure port forwarding. Configure CPU cores and memory close to host machine so that the compilation will be quicker. 20 or 30 Gib should be enough for Gentoo system. We don't taken into user data here because I am sure you want to mount it as a shared folder. Because the disk image is always configured as dynamically allocated, I recomment to double it to 60GiB just in case. I use the vmdk format. It's more popularly supported than vdi. Remember to enable EFI in the VM setting.

Now boot the VM, enter until you see the prompt.

First, we start sshd

/etc/ssh/sshd_config
PasswordAuthentication yes
PermitRootLogin yes

/etc/init.d/sshd restart
passwd

Check out the IP address and ssh to the VM.

Now we prepare the disk using parted. Still, I don't use a swap here. Because I think that sleep or hibernate a VM is pretty rare. In case the memory is used up, you can just allocate more or even over-allocate. But this time we need a EFI partition to install Grub. Again I use LVM here. Just in case you ran out of memory, you can still add a second disk image and extend your LVM VG and still present a single disk to OS. But Luks is not needed anymore. Entrypted a VM? We should encrypt the host OS.

parted /dev/sda
mklabel gpt
mkpart efi fat32 # hundreds of MB should be enough
mkpart root ext4 # all remaining
set 1 boot on
set 2 lvm on

Quit the parted. Since the EFI partition has to be FAT32, we format it as needed

mkfs.vfat -F32 /dev/sda1

Then the LVM partition

pvcreate /dev/sda2
vgcreate vg /dev/sda2
lvcreate -l 100%FREE -n root vg
mkfs.ext4 /dev/vg/root

Next, mount the filesystems and download stage3 and portage

mount /dev/vg/root /mnt
cd /mnt
wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64-systemd/<latest>
tar xvf stage3*
wget http://distfiles.gentoo.org/releases/snapshots/current/portage-latest.tar.xz
tar xvf portage* -C usr

Now, prepare to chroot

mount -t proc /proc proc/
mount -o bind /sys sys/
mount -o bind /dev dev/
cp -L /etc/resolv.conf etc/
chroot . /bin/bash
env-update
source /etc/profile

After that, you may want to install vim and configure /etc/portage/make.conf. The most imoprtant thing is to add MAKEOPTS and CPU_FLAGS_X86. The later one can be automatically generated by cpuid2cpuflags. But first you need to installed.

Before compiling kernel, we modify /etc/fstab to reflect our disk scheme.

vim /etc/fstab
/dev/sda1      /boot/efi    vfat    defaults,noauto,noatime,discard    1 2
/dev/vg/root   /            ext4    defaults,noauto,noatime,discard    0 1

Now, let's compile the kernel. Because we are using a VM, we don't need to worry about drivers. linux-firmware is not need. Just

emerge gentoo-sources genkernel-next

Edit /etc/genkernel.conf

INSTALL="yes"
OLDCONFIG="yes"
MENUCONFIG="yes"
NCONFIG="no"
CLEAN="yes"
MRPROPER="no"
MOUNTBOOT="no"
SAVE_CONFIG="yes"
USECOLOR="yes"
MAKEOPTS="-j5"
LVM="yes"
UDEV="yes"
BOOTLOADER="grub"

Here, we use kernel's default configurations. Thus exit the menuconfig

genkernel all

After a while, the kernel will be compiled and installed. Let's turn to bootloader. First, mount the EFI partition

mkdir -p /boot/efi
mount /dev/sda1 boot/efi

Then install grub

echo "sys-boot/grub device-mapper" >> /etc/portage/package.use/grub
emerge grub

Next, install it. But before that, configure and restart LVM

vim /etc/lvm/lvm.conf
use_lvmetad = 0
/etc/init.d/lvm restart
grub-install --efi-directory=/boot/efi

If you forget to enable EFI for the VM, grub-install will tell not able to embed to BIOS. Just reboot after enable it.

Finally, configure grub and generate the cfg file

vim /etc/default/grub
GRUB_CMDLINE_LINUX="init=/usr/lib/systemd/systemd root=/dev/vg/root dolvm rootfstype=ext4"
grub-mkconfig -o /boot/grub/grub.cfg

You can now verify EFI boot configuration by

efibootmgr -v

efibootmgr -o # change order
efibootmgr --create --label Gentoo --loader "\EFI\gentoo\grubx64.efi" # add new entry
efibootmgr -Bb <hex> # remove entry

Before reboot, remember to check profile and update world

eselect profile set default/linux/amd64/13.0/systemd
emerge --ask --update --deep --newuse @world

There shouldn't be too much to update if you are using the latest stage3 and portage.

Also remember to install NetworkManager and passwd root so that you can login

emerge networkmanager
passwd
reboot

If it goes well, you should be prompt to login. But it seems VirtualBox will lost the EFI boot configuration when devices change like remove install image from optical drive. In that case, you will be prompted a EFI shell. Then

Shell>fs0:
FS0:\>edit startup.nsh

FS0:\EFI\gentoo\grubx64.efi
<ctrl-s><CR><CR> to save
<ctrl-q><CR> to quit
<reboot>

And afterwards, when the EFI have no way to boot you can just type <CR> several times to activate this default script to boot grub.

You can also quickly type <F12> when you see the splash after booting the VM to enter VirtualBox's EFI Manager Program to change the order.

Package It as A Vagrant Box

Before packaging it as a vagrant box, we have to finish some configurations.

First, we finalize systemd's configuration

systemd-machine-id-setup
systemctl enable NetworkManager
systemctl start NetworkManager

You can also setup hostname or timezone here. After that, go check the requirements of a vagrant box, if you like

The things are

  • set up a normal user named vagrant
  • add an insecure pubkey for vagrant
  • add it to sudo
  • passwd root
  • setup and enable sshd
  • install VirtualBox Guest Additions and enable it as startup services

We'll go through this list one by one. Create user vagrant is easy

useradd -m vagrant
su vagrant
passwd
<vagrant>

Then, we add an insecure pubkey to enable private key ssh. This key will be replaced with a randomly generated one upon the first time login of vagrant automatically.

cd ~/.ssh
wget https://raw.githubusercontent.com/mitchellh/vagrant/master/keys/vagrant.pub
mv vagrant.pub authorized_keys
chmod 600 authorized_keys
chmod 700 ~/.ssh

Now we add vagrant to sudoer. But first you may need to install sudo

emerge sys-admin/sudo
vim /etc/sudoers
vagrant ALL=(ALL) NOPASSWD:ALL

Next passwd root

passwd
<vagrant>

Then enable sshd

systemctl enable sshd

And finally, we need to install VirtualBox Guest Additions. You may need to download it with the same version of the VirtualBox from their mirror http://download.virtualbox.org/virtualbox/. Insert into the optical drive of the VM and reboot. After that, you need to mount it

lsblk # to check out the right device to mount
mount /dev/sr0 /mnt
sh /mnt/VBoxLinuxAdditions.run

It will install some service scripts to /etc/init.d/. But since we are using systemd here, we have to write some unit files and enable them. Riaan's Post has already provided some examples. You can take them from there. Just remember to enable them

systemctl enable vboxadd
systemctl enable vboxadd-service
systemctl enable vboxadd-x11

Before packaging, we can do two things to reduce the image size. First, we can remove the gentoo-sources. Second, we can remove the portage tree. These can save nearly 2GiB.

emerge -ac gentoo-sources
emerge --depclean
rm -rf /usr/portage

Now poweroff the VM and change the network back to NAT and run

vagrant package --base <name-of-the-VM-as-showed-in-VirtualBox>

This will generate a package.box file. Before uploading it to vagrant's boxes list, we can test it locally

vagrant box add gentoo /path/to/the/package.box
vagrant init gentoo
vagrant up
vagrant ssh

Acutally, your can skip the steps to add unit files for vboxadd. But then you need to install a vagrant plugin

vagrant plugin install vagrant-vbguest

This automatically detect if the vboxadd is installed. If it is installed but not enabled, it will automatically start it.

Now, you can register an accound on Vagrant's website and upload the box. Next time, you add your box from the box list before start a VM

vagrant box add yuex/gentoo
vagrant init yuex/gentoo
vagrant up
vagrant ssh

On MacOS, you can also install vagrant-manager. It will add an system tray to the menubar to help to monitor and manage your vagrant VMs.