Important Info About Apple Silicon Configuration via LaunchDaemon

[Pico Mitchell]

Hi, after getting into using MDS and being impressed and inspired by its functionality, I’ve ended up wanting something that was more focused on exactly what we need for our refurbishment usage. So, over the past couple weeks, I’ve been working on my own bash script to run in Recovery OS to be able to wipe internal drives and then install macOS via startosinstall. (As an aside, if you are interested in bash code which lists internal drives and allows the user to choose one to wipe before starting an MDS workflow, let me know and I would be happy to share my code for that.)

Wanting to create a script that would hopefully be future-proof, I’ve also built in functionality to be able to use startosinstall’s eraseinstall argument to re-install macOS when logged into existing installations, which I thought would be useful into the future when we will get Apple Silicon Macs donated for refurbishment (even though that will likely be in a few or more years).

Then, I saw your fantastic work on using a LaunchDaemon to install the customization packages on top of a clean install for Apple Silicon Macs and was inspired to add this functionality as well! Hopefully you don’t mind me basically stealing your idea on this. Worth pointing out thought that this isn’t for a competing product or anything and is just something that we will use in-house in at our non-profit for Mac refurbishment.

Since I don’t currently have an Apple Silicon Mac to test on myself, I have been testing this functionality on T2 and non-T2 Intel Macs on macOS 10.13 High Sierra through macOS 11 Big Sur.

I’ve ended up running into a very important issue with user creation that you may be running into as well (but potentially may not even be noticing yet). I also (hopefully) have a possible solution to this problem.

First off, it’s important to note that I am not using pycreateuserpkg to create users, I’m just using sysadminctl instead since it does everything I need without an extra dependency. So, there is a chance that pycreateuserpkg may workaround this possible issue itself and it may not be an issue for you.

Anyway, getting to the point, I have seen occasional problems when the LaunchDaemon runs very early in the boot process and my script starts creating users before all the relevant system services have been started (making some reasonably educated assumptions about the cause there though).

At one point, my admin and standard users did not get created at all, and then another time I ran into a much more odd and confusing issue where my users were created, but my admin user WAS NOT added in the sudoers file even though they were in the admin group and had a Secure Token. This meant that the user could not actually authenticate anything even though they were an admin. The latter is the kind of weird issue that you may have run into but may not have noticed unless you tried do some stuff in the OS rather than re-installing for another test (since I may have run into this issue more than once without noticing it). Worth noting that I have never noticed any user creation issues with my code when it was invoked by macOS itself when using installpackage via startosinstall. I’m assuming that is because macOS will only initiate the custom package installations when the system is in a known ready state for that kind of thing.

Now, for a possible solution… simply having the LaunchDaemon script wait longer before starting the package installation process. I have not tested this tons of times, but I’ve tested it some and feel pretty confident that it should be a reasonable workaround. Also worth pointing out that since I am testing across multiple versions of macOS, I may be running into things that would not happen at all or as often on macOS 11 Big Sur, but if I am right about the cause being trying to create users too early in the boot process, I don’t believe it would be specific to any version of macOS.

I went about finding a process that is started late in the boot process by the time users are ready to be logged in. I’ve currently landed on waiting until “coreauthd” is running before starting my package installations. After my first few test, this seems to work properly, but needs more testing to be certain. I just thought it would be valuable to share this info with you sooner rather than later if you are running into the same issue. And, by sharing this info, we could also spread the testing across more installations if you are working on the same kind of thing and implement this same solution.

A very important note: I am CREATING “.AppleSetupDone” when I install my LaunchDaemon to install my configuration packages. I initially did this because I simply didn’t want Setup Assistant to start talking while my packages were being installed and so that there would be no temptation to start clicking through Setup Assistant screens while the important work was invisibly happening in the background. But, this turned out to also be very important to making sure that “coreauthd” is started when boot process is finished.

If Setup Assistant starts AND the Language Chooser comes up, “coreauthd” does not appear to get launched until the user clicks past the Language Chooser screen, which is not convenient for an automated solution. There may be another relevant process to wait for, but bypassing Setup Assistant and waiting for “coreauthd” is perfect for my usage.

Related to this, I noticed once that when I did let Setup Assistant start and then ran my configuration scripts and rebooted (without clicking though Setup Assistant), the “_mbsetupuser” user was unintentionally left in the admin user group (which can be checked with “dscl . -read /Groups/admin GroupMembership”), which is not the normal behavior when Setup Assistant is normally clicked through by a user. This is another reason that bypassing Setup Assistant while the configuration scripts run may be an important thing to do.

Finally, something that I decided to do for fun but ended up being a crucial helper in debugging all this stuff was having the computer announce when my configuration scripts start and also when they finish and the computer is about to reboot. I tried to simply use the “say”command to do this, but for some reason it seemed to not work before login on Big Sur but did work on older versions of macOS. As a workaround, I saved the text to speech output of my desired phrases via the “Add to Music as Spoken Track” Service available in TextEdit when the text that you want to speak is highlighted. Also, Samantha is a better sounding voice than Alex.

It was because of this announcement that I first noticed how early my LaunchDaemon script was starting (while the boot process progress bar was only about half way though) which quickly led me to thinking that maybe my users were simply getting created too early in the boot process when I started to see these problems with user creation. Relatedly, I also noticed that playing audio early in the boot process can fail and I initially had my first “afplay” command in a loop to keep trying for 10 seconds if it exited with an error. This worked, but I believe this is rendered unnecessary when waiting for “coreauthd” to be running before even starting to do anything.

I think that’s all I have to share right now. I also want to point out again that this info is all very fresh and I’m still testing all this stuff. This may not be a perfect solution and there may be other adjustments that need to be made.

[Pico Mitchell]

Well, it appears this all needs more research and testing. I just had another test case where my admin was created but NOT added to the sudoers file on Big Sur even after waiting for “coreauthd” to start the package installations.

I will investigate if maybe bypassing Setup Assistant when the accounts are getting created is causing an issue? But I’m not sure about that. We’ll see.

Also, it appears the “_mbsetupuser” account gets left as an admin if the computer boots to Setup Assistant and then is shutdown and configured with this technique anyways. So, bypassing Setup Assistant during configuration doesn’t avoid that issue altogether. I’m thinking my LaunchDaemon script will just use dseditgroup to remove “_mbsetupuser” from the admin group after my packages are finished installing.

Would definitely be interested in knowing if you are not running into this same issue when using pycreateuserpkg.

PS. Did another test on another clean install where I DID NOT bypass Setup Assistant and once again got an admin user who is not in the sudoers file. So, it seems like that is not part of the issue.

[Pico Mitchell]

I didn't mean to bury the lede, but this turns out to be a very important factor:

This is interesting… my configuration creates an admin and then a standard user and then sets up auto-login for the standard user.

This issue of the admin not being in the sudoers file is coming up before the admin has ever been fully logged in, but when trying to provide admin authentication in the System Prefernces, run commands as the admin user via AppleScript, and “su my-admin; sudo some_command” from the standard users account. The latter is where I get the message about the account not being in the sudoers file. Trying to provide admin authentication in a GUI prompt just fails as if the password was wrong.

I just found that if I manually log into the admin account, I see a progress spinner for longer than a normal login while I guess some more account setup happens. After a reboot I can run sudo commands etc from the standard account. Very odd.

I don’t believe I have ever run into this issue when creating these exact same users via startosinstall’s installpackage. This all definitely need a bit more investigation.

[Pico Mitchell]

Welp, it appears that using “pycreateuserpkg” DOES NOT have this same problem.

I thought using “sysadminctl” got the job done for me, and it appear did work fine when run in a script via “startosinstall --installpackage”, but clearly it is not up to snuff when trying to create users on boot via LaunchDaemon.

I guess I will switch to using “pycreateuserpkg” like MDS does.

My gut says it still may be valuable to wait for the “coreauthd” process to be sure that the configuration packages don't conflict with or run into any issues because of the computer’s boot process, but I am not certain it is 100% necessary anymore. I did see that one case where the users did not get created at all, but “pycreateuserpkg” may avoid that as well. But, since MDS will allow for users to provide scripts that do any number of crazy things, waiting for a fully booted and ready OS may be worthwhile just as a precaution.

I hope the info about removing the “_mbsetupuser” from the Admin Group and the ideas to bypass Setup Assistant when configuring via LaunchDaemon as well as using text to voice recordings for debugging/user feedback may be useful to you in MDS.

[timothy perfitt]

I have not seen these issues (yet), but have noticed that when running on the m1, occasionally personalization fails and the m1 requires you to boot into recovery and authenticate as a user. Give that you have done a ton of research on this, can we do a zoom call and discuss. I’d like use to learn from each others experience since this is relatively new space. My calendar is here:

https://twocanoes.com/meet

Let me know if you are open to that.

tim

[Pico Mitchell]

Yeah, I’m open to that. I will hopefully have some time next week.

Want to point out though there while there is some overlap with enterprise deployments to what I do to refurbishment, there are a lot of differences as well. Namely, I have no experience using Apples MDM capabilities since they are not built for our needs and would not serve us well.

Most of my knowledge comes from years of trying to bend macOS to our will when there is no clear documentation or path made exactly for what we need. Over the years, I have learned that enterprise deployment info/blogs do tend to have the most useful nuggets for our needs.

Also, as I mentioned, I do not have an M1 to test this stuff on so I am not familiar with the Personalization issue. Apple seems to say a re-install is in order when that comes up: https://support.apple.com/en-us/HT211983 (Interestingly, the bottom of that documentation seems to be the source of some of what https://macos.it-profs.de/macos/m1_erase is doing).

Regardless, sometimes it’s just good to talk through issues with folks with differing perspectives and knowledge

[Pico Mitchell]

Since I try to avoid adding dependencies unless absolutely necessary, and the fact that I really don’t know leaving a weird mystery left unsolved, I returned back to trying to use “sysadminctl” when setting up accounts on boot via LaunchDaemon.

I didn’t fully figure out WHY this bug was happening, but I did determine that it only affects macOS 10.15 Catalina and macOS 11 Big Sur, and I found a way to avoid the bug altogether.

To save myself re-writing it, here are the comments and the “fix” that I wrote in my code for my own future reference:

sysadminctl "${sysadminctl_args[@]}" &> /dev/null

if $set_admin && (( DARWIN_MAJOR_VERSION >= 19 )); then
    # There is an odd bug on macOS 10.15 Catalina and newer when using "sysadminctl -addUser -admin" on boot via LaunchDaemon where the admin user does not get properly setup as an admin, until AFTER they have been logged in.
    # This issue does NOT happen when using "sysadminctl -addUser -admin" when run via "startosinstall --installpackage" and does not ever happen when creating the account with "dscl" and "dseditgroup".
    # When this happens, the admin account will NOT be able to perform "sudo" or AppleScript "with administrator privileges" commands until they have been logged in, which is inconvenient since we don't normally log into the admin account at all.
    # I discovered that when booted into macOS and running as the "fg-demo" user that the "fg-admin" would be properly listed in "dscl . -read /Groups/admin", but the "admin" group would NOT be listed when checked with "id -Gn fg-admin".
    # I then THOUGHT that I fixed this issue by manually adding "fg-admin" to the admin group using "dseditgroup" in this script (right after running "sysadminctl -addUser -admin"). This made some sense to me since the issue does not happen when creating the account with "dscl" and "dseditgroup".
    # To make things a bit weirder, upon further investigation, I found that the issue was actually being fixed before ever running "dseditgroup" simply because I was first checking that "admin" was not listed in "id -Gn" before attempting to run "dseditgroup".
    # Because an expected log entry was not getting written, I found that "dseditgroup" was actually NOT getting run at all because the "admin" group WAS being listed in "id -Gn" when run as root here in the script.
    # So it seems that simply running "id -Gn" as root here in the script after running "sysadminctl -addUser -admin" will seemingly kick things into gear and get the admin account properly setup as a functional admin.
    # Because this didn't quite make sense, I double-checked and reproduced the issue by not running "id -Gn" here in the script at all, and even tried running "dseditgroup" without first checking "id -Gn" and the issue happened again. Interesting that running "dseditgroup" is not even a fix at all!

    id -Gn "${short_name}" &> /dev/null
fi

Feels like some weird quantum Schrodinger's admin situation that the actual reliable fix is to simply check if the account is in the admin group using the “id” command.

Figured I would post back here in case anyone stumbles on this issue from Googling or if you ever decide to move away from “pycreateuserpkg” in MDS and want to be be able to reliably use “sysadminctl”.

I’ve also this wrapped into a bash function with an alternate version of account creation using “dscl” when I want to prevent Secure Tokens on Big Sur using theAuthenticationAuthority ';DisabledTags;SecureToken' tag (https://support.apple.com/guide/deployment-reference-macos/using-secure-and-bootstrap-tokens-apdff2cf769b/web), which can’t be done with “sysadminctl” since it always sets a password before being able to add this AuthenticationAuthority tag. If you are interested in moving away from “pycreateuserpkg” in MDS in the near future, I would be happy to share this function that does some simple argument sanitization and can do “dscl” or “sysadminctl” based on the same passed arguments (including adding “JPEGPhoto” which is a bit more tricky when using “dscl”).

The one downside to using “sysadminctl”/”dscl” is that passwords would need to be passed in plain text, but I’m sure there are some ways they could be stored and passed with reasonable enough security so that they at least can’t be easily extracted from MDS drives.