BridgeOS Kernel Panics – Part 1

About this Mac

In early October I replaced my aging MacBook Pro, a base model from 2007, with a Mac Mini. The outgoing machine had served me admirably but was now starting to show its age. I hummed and hawed over which model to buy, but ended up with a Mac mini. With no prospect of travel in the foreseeable future, a desktop made sense. I also splurged on a 32GB RAM upgrade kit from OWC.

My first week with this new machine was blissful. “Wow!” I exclaimed, “look at the speed at which this Excel file opens! Marvel as I keep 25 Chrome tab opens!”

But one morning a dialog box informed me that my computer had restarted because of a problem. Interesting, I thought, as my computer restarted again. And again. And a third time before finally getting ahold of itself. The machine would run fine for a few hours before succumbing to a series of reboots.

Unexpected Restart Warning

My first reaction was to blame the RAM, a component notorious for causing kernel panics when it fails. I also installed it myself, and maybe I screwed something up. On the other hand, the panic medic boot dialog seemed confident that kernel extensions were at fault.

Panic Medic Boot Dialog box

So what’s going on?

Hypotheses

Before falling too far down a proverbial rabbit hole, I ran through Apple’s recommendations, which you can read here: If your Mac Restarted Because of a Problem

Quick Fixes:

QuickfixResult
Reset SMC/NVRAM/PRAMThe machine continued to kernel panic
Disconnect all Peripherals The machine continued kernel panicked with only a keyboard, mouse, and display connected
Try safe mode
I could boot into Safe Mode, but since the issue is intermittent this didn’t tell me very much
Run hardware diagnosticsHardware diagnostics passed
Basic Kernel Panic Troubleshooting Steps

With those out of the way, it was time to develop some hypothesis.

Hypothesis 1: A kernel extension is responsible for the crashes. Testing this should be straightforward: remove the extensions and see if things stabilize.

Hypothesis 2: There’s something wrong with the RAM. This could be as simple as a faulty stick or as serious as a damaged pin on the logic board. The easiest way to test this hypothesis is by reinstalling the original RAM. But this entails removing the logic board. Each time you disassemble a Mac mini you increases the risk of breaking a delicate part.

Testing Hypothesis 1: Kernel Extensions

When I setup my Mac mini, I started fresh. I reinstalled some apps and restored by data from iCloud Drive and OneDrive. Two apps asked for permission to install kernel extensions: Parallels Desktop 16 and Logitech Options.

Removing Logitech Options is straightforward: run the Logi Uninstaller. I assumed that Parallels would also need an uninstaller, but according to their support page all you need to do is drag the application to the recycle bin.

Would this simple act stop the kernel panics? I rebooted the machine and decided to dig into the logs while I waited to find out.

BridgeOS .. are you to blame?

All the panic logs started by referencing Bridge OS 4.6. A quick search shows that BridgeOS is related to the T2 chip. That same search will also lead you to Apple Discussions threads – some with hundreds of responses – of loosely related T2/BridgeOS problems.

{
  "caused_by": "macos",
  "macos_version": "Mac OS X 10.15.7 (19H2)",
  "os_version": "Bridge OS 4.6 (17P6610)",
  "macos_system_state": "running",
  "incident_id": "853278DF-511E-4C30-A286-D73A50488541",
  "timestamp": "2020-10-15 16:24:43.00 +0000",
  "bug_type": "210",
  "macOSOtherString" : "\n** In Memory Panic Stackshot Succeeded ** Bytes Traced 782880 **\n"

}

I never found a consensus on the cause or solution to these BridgeOS crashes. I don’t even know if they’re even related. Maybe BridgeOS isn’t the cause of these kernel panic but a victim. What if BridgeOS just went down with the rest of the ship?

Nevertheless, here are some interesting observations from various threads on the issue:

  • Many users reported kernel panics after they installed Security Update 2020-002, released in March 2020.
  • Some users reported that upgrading to Big Sur resolved the issue. Big Sur ships with a different build BridgeOS (17.16.16610 at the time of writing.)
  • This issue affects other T2 Enabled Macs, but anecdotally it seems to plague Mac mini owners the most frequently.
  • Possible solutions were all over the place – ranging from not using Safari to logic board replacements.

As an aside, while reading these threads my machine kernel panicked. I guess kernel extensions weren’t to blame, but I hadn’t completely ruled out a software issue.

Erase and Install

One surefire way to resolve a software issue is by erasing the software, which is exactly what I did. I started the Mac Mini into recovery mode, wiped the OS and Data volumes, and proceeded with a clean MacOS install.

To my horror, the machine kernel panicked 15 minutes into the install.

Testing the Hardware

In Part 2 of this post I’ll cover hardware troubleshooting. I’ll also dive more into BridgeOS, firmware restores, and the added complexity of diagnosing a T2 equipped Mac.

Thanks for reading