How to make an unattended Ubuntu Server Installation ISO with cloud-init

Posted on Last updated on

Updated on 16 Jan, 2022. I worte a post for Ubuntu 20.04 Server.

It will be convenient if the installation is fully automated when configuring multiple nodes at the same time. However, I believe Ubuntu doesn't care about unattended installation via the CD image. Its developers seem to be unwilling to write a detailed tutorial about that, even some changes they made make trouble unintentionally. Note that installing via PXE Network Boot is a good alternative.

Before getting hands dirty

An unnoticeable fact is that Ubuntu Server replaced its installer silently, which means some tutorials are actually not applicable for specific versions of Ubuntu. According to my observation, we have two installers so far,

  • Before Ubuntu 18.04.3: Debian-Installer, a non-live installer
  • Ubuntu 18.04.4~18.04.5: Old version of subiquity, a live installer, without unattended installation support
    • There are two varients, (thanks for jack's comment)
      • one is called non-live version with Debian-Installer
      • another is called live version with old version of subiquity, which doesn't have unattended installation support
  • Ubuntu 20.04: subiquity, a live installer, with unattended installation support

Consider that Ubuntu 18.04 has excellent compatibility, this tutorial aims at making Ubuntu 18.04.5 Installation ISO. As for Ubuntu 20.04 users, please refer to this tutorial https://gist.github.com/s3rj1k/55b10cd20f31542046018fcce32f103e or my newer blog post for your information, and you can still read this tutorial because some approaches mentioned in this article also work.

Download ISO and Tools

Since Ubuntu 18.04.4~18.04.5 doesn't support unattended installation, we have to use Ubuntu 18.04.3 as the original version. Note that this version is deprecated. Thus it can only be downloaded from old-releases site. Make sure what you download is a non-live version, such as,

1
http://old-releases.ubuntu.com/releases/18.04.3/ubuntu-18.04.3-server-amd64.iso

There is an excellent tool called cubic, which helps us decompress and repack the ISO file and rootfs image. Also, it will automatically set up the virtual environment (chroot), where modifying the root filesystem is quite easy, just like operating a regular Ubuntu.

1
2
3
sudo apt-add-repository ppa:cubic-wizard/release
sudo apt update
sudo apt install cubic

Run cubic command in the terminal to launch this program.

Modify Root Filesystem

Generally speaking, all the files existing in the virtual environment will be directly copied to the machine where this system is installed, except several files generated during installation. Thus, you can preinstall some software in this step.

upload successful
upload successful

Install and Upgrade Software

Consider that Ubuntu 18.04.3 is a bit outdated, so some build-in software should be upgraded.

1
apt update && apt upgrade -y

Note:

  • The Linux Kernel has been installed in this stage, and it will be installed by Debian-installer during installation. Therefore, you still need to perform apt upgrade after installation.
  • You can change an APT repository in the virtual environment, but the file /etc/apt/source.list will be replaced by Debian-installer or cloud-init eventually.
  • SSH server could be installed in advance to save some time in installing OS, and cloud-init will regenerate SSH-key during the first boot.

Write Post-Installation Script

cloud-init is widely used in the cloud industry to initialize the system during the first boot. In fact, we could utilize it to do post-installation configuration. It is recommended to refer to these examples to customize your script: https://cloudinit.readthedocs.io/en/latest/topics/examples.html.

I attach my script located at /etc/cloud/cloud.cfg.d/20-asc-init.cfg here for your reference.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#cloud-config

network:
  config: disabled
bootcmd:
  - /usr/bin/asc_init_net
ntp:
  enabled: true
apt:
  primary:
    - arches: [default]
      uri: http://mirrors.sustech.edu.cn/ubuntu/

package_update: true
package_upgrade: true
power_state:
  mode: reboot

Optional: Configure DHCP for all network interfaces after installation

Debian-installer has a known bug: netcfg/choose_interface=auto fails to pick the functional network interface from many ones. Meanwhile, cloud-init also only configure one interface since most of the cloud instances only have one. If your machine happens to have one interface, you can skip this part. Otherwise, you need a workaround.

I wrote a script /usr/bin/asc_init_net to automatically generate configuration file for Netplan during post-installation. All interfaces will contact the DHCP server to get available IP addresses simultaneously. Of course, this script should be executed by cloud-init as bootcmd.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#!/bin/bash

find /sys/class/net -type l -not -lname '*virtual*' -printf '%f\n' | python3 -c "
import sys
import yaml
c = {
    'network': {
        'version': 2,
        'ethernets': {}
    }
}
for i in sys.stdin.readlines():
    c['network']['ethernets'][i.strip()] = {'dhcp4': True, 'optional': True}
print(yaml.dump(c))
" > /etc/netplan/20-asc-init.yaml && netplan apply

Don't forget to set the correct permission.

1
chmod +x /usr/bin/asc_init_net

Clean up

/etc/machine-id might be generated when you install some software or do something else. However, this ID should vary from machine to machine. If two machines share the same ID, they may get the same IP address from the DHCP server. Therefore, this file should keep empty.

1
echo -n > /etc/machine-id

Note: This file could be regenerated by running systemd-machine-id-setup or rebooting the computer.

Moreover, cloud-init will be executed on the next boot only if it is clean, then it will write something to disk to prevent itself from running again. To make sure it will work as expected, we can clean it manually.

1
cloud-init clean

Write Preseed File

Preseed files store all the answers to the questions that the installer asks. Move to the directory custom-disk, which is created by cubic and contains the supporting files of the CD, including bootloaders and some configuration. Create a file custom-disk/preseed/auto-inst.seed, and write the following content to that file.

upload successful
upload successful
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
### Automatic Installation
d-i auto-install/enable boolean true
d-i debconf/priority select critical

### Localization
d-i debian-installer/locale string en_US.UTF-8
d-i localechooser/supported-locales multiselect en_US.UTF-8
d-i console-setup/ask_detect boolean false
d-i keyboard-configuration/xkb-keymap select us

### Network config
d-i netcfg/enable boolean false
#d-i netcfg/choose_interface select auto
#d-i netcfg/get_hostname string node0
#d-i netcfg/get_domain string unassigned-domain
#d-i netcfg/hostname string node0
d-i hw-detect/load_firmware boolean true

### Mirror settings
d-i apt-setup/use_mirror boolean false
#d-i mirror/http/mirror select CC.archive.ubuntu.com
#d-i mirror/country string manual
#d-i mirror/http/hostname string mirrors.sustech.edu.cn
#d-i mirror/http/directory string /ubuntu
#d-i mirror/http/proxy string

### Account setup
# This is just for testing!!!
d-i passwd/make-user boolean false
d-i passwd/user-fullname string Ubuntu User
d-i passwd/username string ubuntu
d-i passwd/user-password password ubuntu
d-i passwd/user-password-again password ubuntu
#d-i passwd/user-password-crypted password [crypt(3) hash]
d-i user-setup/allow-password-weak boolean true

# Set to true if you want to encrypt the first user's home directory.
d-i user-setup/encrypt-home boolean false

### Clock and time zone setup
d-i clock-setup/utc boolean true

# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string Asia/Shanghai

# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean true

### Partitioning
# !!!DANGER don't use this without knowing what you are doing!!!
# comment out this block it you want the installer to ask about the 
# partitioning, which is much safer!

# The following will partition disk /dev/sda with an EFI partition, a root partition
# and a swap file. AND WONT ASK TO CONFIRM ANYTHING i.e. it will overwrite existing partitions
d-i preseed/early_command string umount /media || true
d-i partman/unmount_active boolean true
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/choose_recipe select atomic
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

# The kernel image (meta) package to be installed;
d-i base-installer/kernel/image string linux-generic
#d-i base-installer/kernel/altmeta string hwe-18.04

### Package selection
# Install the Ubuntu Server seed.
#tasksel tasksel/force-tasks string server
#tasksel tasksel/first multiselect openssh-server
d-i tasksel/first multiselect none
d-i pkgsel/language-packs multiselect en, zh
d-i pkgsel/update-policy select none

# Verbose output and no boot splash screen.
d-i debian-installer/quiet  boolean false
d-i debian-installer/splash boolean false

d-i cdrom-detect/eject boolean true

# Avoid that last message about the install being complete.
# This will just finish and reboot
d-i finish-install/reboot_in_progress note
#d-i debian-installer/exit/poweroff boolean true

Note: The network is disabled during the installation due the problem we talked about earlier. As a result, debian-install will

  • Fail to use self-defined APT repository
  • Skip setting the host name
  • Skip configuring netplan

But, all the problems above can be fixed by cloud-init.

Add Boot Arguments

Automatic installation requires us to pass extra arguments to the kernel. Luckily, cubic will help us to edit the configuration of bootloaders.

Note: There are two bootloaders in the CD image. grub is used to support the EFI firmware, and isolinux is used to support the traditional BIOS.

Write the content below to the corresponding file.

boot/grub/grub.cfg

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
if loadfont /boot/grub/font.pf2 ; then
	set gfxmode=auto
	insmod efi_gop
	insmod efi_uga
	insmod gfxterm
	terminal_output gfxterm
fi

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray

set timeout=3
menuentry "Auto Install Ubuntu Server" {
	set gfxpayload=keep
	linux /install/vmlinuz boot=casper file=/cdrom/preseed/auto-inst.seed auto=true priority=critical locale=en_US quiet ---
	initrd  /install/initrd.gz
}
menuentry "Rescue a broken system" {
	set gfxpayload=keep
	linux /install/vmlinuz boot=casper rescue/enable=true ---
	initrd /install/initrd.gz
}

isolinux/txt.cfg

1
2
3
4
5
default autoinstall
label  auto-install
  menu label ^Auto Install Ubuntu Server
  kernel /install/vmlinuz
  append boot=casper file=/cdrom/preseed/auto-inst.seed vga=788 initrd=/install/initrd.gz auto=true priority=critical locale=en_US quiet ---

Note: isolinux has a build-in option Rescue a broken system.

By now, everything is ready to go. Let us enjoy the automatic installation.

Reference