Old code called  dircache_updatefile() in default case. Should the new code do the same?

better switch the return type to bool - we only return 0 or 1 values there.

I'd like to have verbose log there to track what is going on, please add:
if (bam_verbose)                                bam_print("mkisofs not found\n");

also for verbose tracking:

if (bam_verbose) {
  bam_print("%s does not have %s/%s property, using "
       "mkisofs\n", BOOT_ARCHIVE_FMRI, SCF_PG_CONFIG,
       SCF_PROPERTY_FORMAT);
}

and here:

if (bam_verbose) {
    bam_print("creating %s boot archive\n",
        format != NULL? format : "hsfs");
}

this printout can now be removed as we just did announce in use_mkisofs(), also the message is wrong anyhow.

"on x86 get rid of transient boot configuration"
both grub and loader use transient config.

missing:
       -f)     shift
               FORMAT="$1"
               ;;

add error printout here?

actually for sparc we should default to compress=yes, because it will compress individual files in the archive (as opposed to compress the entire boot archive). loader and grub can gunzip the boot_archive  while loading it, but sparc can not, which is why we gzip individual files there.
I am not entirely sure about whole archive versus files compression on x86 - whole archive compression will save us a bit more loading bandwith, individual compressed files will save us memory needed for boot archive.
Also there is a question, should we still need uncompressed kernel on sparc, because x86 seems to be able to cope with compressed kernel just fine...

BOOT_ARCHIVE is set as relative path (without starting '/'), should we set archive with "/$ALT_ROOT" here? The message below is confusing..

There we do have one small issue - we always use mkfile below, therefore we always will create new archive there.

here we need $GZIP_CMD

boot after pkg update, 32-bit archive is gone:
tsoome@uefi-oi:~$ ls -l /platform/i86pc/
total 10
drwxr-xr-x   4 root     sys            6 aug 30 12:10 amd64
drwxr-xr-x  10 root     sys           10 sept 26  2017 kernel
drwxr-xr-x   4 root     sys            8 jaan 19  2018 ucode
drwxr-xr-x   2 root     root           2 aug 22 11:10 updates
tsoome@uefi-oi:~$

tsoome@uefi-oi:~$ sudo -s
Password: 
root@uefi-oi:/home/tsoome# svcprop -p config/format boot-archive
cpio
root@uefi-oi:/home/tsoome# svccfg -s boot-archive setprop config/format = ufs
root@uefi-oi:/home/tsoome# svccfg -s boot-archive:default refresh
root@uefi-oi:/home/tsoome# svcprop -p config/format boot-archive
ufs
root@uefi-oi:/home/tsoome# bootadm update-archive -fv
forced update of archive requested
cannot find: /etc/cluster/nodeid: No such file or directory
cannot find: /etc/devices/mdi_ib_cache: No such file or directory
cannot find: /etc/devices/retire_store: No such file or directory
creating ufs boot archive
updating platform/i86pc/amd64/boot_archive (UFS)
root@uefi-oi:/home/tsoome# 
root@uefi-oi:/home/tsoome# reboot


system booted OK with cpio/ufs/ufs-nocompress/hsfs, however, there is one change missing:
diff --git a/usr/src/cmd/halt/halt.c b/usr/src/cmd/halt/halt.c
index 166e5a163b..657f207a85 100644
--- a/usr/src/cmd/halt/halt.c
+++ b/usr/src/cmd/halt/halt.c
@@ -1584,12 +1584,12 @@ main(int argc, char *argv[])
        if (cmd != A_DUMP) {
                int     start, end, delta;

-               (void) kill(-1, SIGTERM);
-               start = time(NULL);
-
                if (zoneid == GLOBAL_ZONEID && !nosync)
                        do_archives_update(fast_reboot);

+               (void) kill(-1, SIGTERM);
+               start = time(NULL);
+
                end = time(NULL);
                delta = end - start;
                if (delta < 5)

it is needed to make sure we can get format from SMF, otherwise we will have format ping-pong...

update comment too:)

gcc7:
bootadm.c:3332:5: error: suggest explicit braces to avoid ambiguous 'else' [-Werror=dangling-else]

Ship It!

Should that not be "Unsupported archive format"?

Should ufs-nocompress be shown in the usage output?

Is the logic correct in the following situation: An updated system running this code running bootadm -R against an OS instance that doesn't have cpio support in the kernel?
Will the system still generate a cpio archive in this case, because that's the default? Then the system won't boot.
This is one case of a general issue, of what happens when it can't contact SMF? If SMF isn't running, then I believe the error path is safe in the simplest case of that. But with an alternate root, using SMF seems entirely unsafe.
Adding an SMF dependency seems undesirable in any case.
How about setting a property on the zfs root filesystem for the BE instead?
I think you have to be absolutely sure that the target instance you're generating the archive for understands the format you're generating. So you can only use cpio if you can confirm it's valid, else the old hsfs/ufs has to be the default.

First, shouldn't that be $FORMAT, not $1, in the error message?
Second, what if the admin sets the archive property to mkisofs, but that doesn't work because mkisofs is missing (for whatever reason)? We then fall through to running this script. In the old implementation, that was fine because we then tried ufs. But in this case we error out.
I believe we shouldn't error out on invalid input, but choose a sensible format here. Whether we do or we don't, we need to produce a more helpful error or warning: both for completely invalid formats, and separately for the case where mkisofs is asked for but not available.
(And yes, there's the possibility that an a;ternate BE specifies mkisofs but the current system doesn't have it.)

Ship It!

Ship It!