9627 No longer need 32-bit boot_archive 9628 UFS boot archives are too large 9721 cmd/boot: support cpio boot archive

Review Request #1189 — Created Aug. 29, 2018 and submitted — Latest diff uploaded

citrus
illumos-gate
master
9627, 9628, 9721
1156
5e9492b...
general
9627 No longer need 32-bit boot_archive 9628 UFS boot archives are too large 9721 cmd/boot: support cpio boot archive

This has been tested on OmniOS, creating HSFS and UFS archives with both
old and new code and comparing contents, and test booting. Also testing with CPIO.
For the size calculation problem, there is more testing information in the issue -
1024x too much space was being reserved for directories.

With this change I'm adding a new property on the system/boot-archive:default service to specify the desired archive format; this defaults to cpio. This property is honoured by both the C bootadm code, which is responsible for building the HSFS format archives, and by the script that handles UFS and CPIO.
If the property is not present then the default is HSFS if /usr/bin/mkisofs is present, and CPIO otherwise. If the property is set to an unknown value then CPIO will be used.

The man page for bootadm has been updated to include information on the new option and also to show the -f option for bootadm update-archive (force archive creation) which was previously undocumented.

Testing

On my test system a boot archive contains 1056 files, 92 directories and 216 hardlinks (mostly CPU firmware).

During system boot, 2953 file opens are performed of which 2274 result in ENOENT; the total data read from the archive is 46MiB.

Testing with each of the four archive formats below shows that processing time during boot is in the range 3-9 seconds for all formats, with different boots giving different results. On average archive processing takes around 5 seconds regardless of format.

bloody% svcprop -p config/format boot-archive
cpio

bloody% time pfexec bootadm update-archive -f
updating platform/i86pc/amd64/boot_archive (CPIO)
pfexec bootadm update-archive -f  5.56s user 0.15s system 98% cpu 5.826 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     gzip compressed data - deflate method , original file name
bloody% gzip -dc /platform/i86pc/amd64/boot_archive > /tmp/l
bloody% file /tmp/l
/tmp/l:         ASCII cpio archive - CHR (-c) header
bloody% ls -lh /platform/i86pc/amd64/boot_archive /tmp/l
-rw-r--r--   1 root     root       31.6M Jul  5 15:45 /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 af       other      83.1M Jul  5 15:45 /tmp/l

bloody% svccfg -s boot-archive setprop config/format = ufs
bloody% time pfexec bootadm update-archive -f
updating platform/i86pc/amd64/boot_archive (UFS)
pfexec bootadm update-archive -f  3.86s user 0.92s system 6% cpu 1:15.96 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     English text
bloody% ls -l /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 root     root       45.8M Jul  5 15:48 /platform/i86pc/amd64/boot_archive

bloody% svccfg -s boot-archive setprop config/format = hsfs
bloody% time pfexec bootadm update-archive -f
updating //platform/i86pc/amd64/boot_archive (HSFS)
pfexec bootadm update-archive -f  0.17s user 0.12s system 1% cpu 21.938 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     ISO 9660 filesystem image
bloody% ls -lh /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 root     root       36.4M Jul  5 15:46 /platform/i86pc/amd64/boot_archive

bloody% svccfg -s boot-archive setprop config/format = ufs-nocompress
bloody% time pfexec bootadm update-archive -f
updating platform/i86pc/amd64/boot_archive (UFS-nocompress)
pfexec bootadm update-archive -f  5.32s user 1.54s system 5% cpu 1:56.01 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     gzip compressed data - deflate method , original file name
bloody% gzip -dc /platform/i86pc/amd64/boot_archive >| /tmp/l
bloody% file /tmp/l
/tmp/l:         English text
bloody% ls -lh /platform/i86pc/amd64/boot_archive /tmp/l
-rw-r--r--   1 root     root       27.5M Jul  5 15:43 /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 af       other      91.7M Jul  5 15:44 /tmp/l

Also, testing with multiple mounted boot environments with differing properties, including unset (i.e. an old BE)

omni# beadm list | awk '$3 ~ /\// {print $3}' | while read mp; do
echo "$mp: \\c"
SVCCFG_REPOSITORY=$mp/etc/svc/repository.db svccfg -s boot-archive listprop config/format
done

/: config/format  astring  cpio
/hsfs: config/format  astring  hsfs
/ufs: config/format  astring  ufs
/ufs-nocompress: config/format  astring  ufs-nocompress
/cpio: config/format  astring  cpio
/unset: zsh: exit 1


omni# bootadm -a update_all -f
updating /platform/i86pc/amd64/boot_archive (CPIO)
Creating boot_archive for /hsfs
updating /hsfs//platform/i86pc/amd64/boot_archive (HSFS)
Creating boot_archive for /ufs
updating /ufs/platform/i86pc/amd64/boot_archive (UFS)
Creating boot_archive for /ufs-nocompress
updating /ufs-nocompress/platform/i86pc/amd64/boot_archive (UFS-nocompress)
Creating boot_archive for /cpio
updating /cpio/platform/i86pc/amd64/boot_archive (CPIO)
Creating boot_archive for /unset
updating /unset/platform/i86pc/amd64/boot_archive (UFS)
Loading...