from Patrick Gundlach |

Fixing macOS notarization issues in the speedata Publisher 5.2.0

Categories: Development

I have rebuilt the macOS packages of the speedata Publisher and released them as version 5.2.1. The Windows and Linux packages remain at 5.2.0, because the changes are macOS-specific.

This post is a short write-up of what broke, how I debugged it, and which tools I used along the way. If you ever need to get a macOS command line tool notarized, this might save you some time (or at least a bit of frustration).

The initial problem

The first hint that something was wrong came from a helpful macOS error message: the sp binary from the downloaded ZIP refused to run and macOS complained about signature / security issues.

At first glance, everything looked fine:

codesign -dv --verbose=4 bin/sp

showed a valid Developer ID Application signature with hardened runtime, the correct team identifier, and a recent timestamp. So far, so good.

However, Gatekeeper (the actual decision-maker on whether something is allowed to run) disagreed:

spctl -a -vvv bin/sp

returned:

bin/sp: rejected
source=Unnotarized Developer ID
origin=Developer ID Application: PATRICK GUENTER GUNDLACH (3Y98DLKYBJ)

In other words: “Nice signature, but this binary (or rather the package it lives in) is not notarized. Rejected.”

The usual suspects: quarantine, signature, notarization

There are a few common reasons why macOS complains:

  • The file still has the quarantine attribute from the browser download:
xattr bin/sp
xattr -d com.apple.quarantine bin/sp
  • The binary is unsigned or only ad-hoc signed.
  • The binary is signed, but the archive is not notarized.
  • The wrong artifact was notarized (e.g. a different build or a different ZIP).

In my case, the binary itself was correctly signed. The problem appeared at the notarization step.

Looking at the notarization log

The key tool here is Apple’s notarization service and its logs.

I submitted my ZIP for notarization using:

xcrun notarytool submit speedata-publisher-macos-arm64-5.2.0.zip \
    --apple-id "..." \
    --team-id 3Y98DLKYBJ \
    --keychain-profile "..." \
    --wait

The result: status: Invalid – the archive contained “critical validation errors”.

The interesting part is in the JSON log, which looks roughly like this (simplified):

{
  "status": "Invalid",
  "statusSummary": "Archive contains critical validation errors",
  "issues": [
    {
      "severity": "error",
      "path": "speedata-publisher-macos-arm64-5.2.0.zip/.../jline-2.14.6.jar/META-INF/native/osx/libjansi.jnilib",
      "message": "The binary is not signed.",
      "architecture": "x86_64"
    }
  ]
}

This is where the real debugging starts.

The important insight: Apple doesn’t just inspect your main executable (bin/sp), it walks through the whole archive and looks at every Mach-O binary it can find:

  • .dylib, .so, .jnilib, helper executables, and so on
  • even native libraries inside JAR files

If any of those are unsigned or signed incorrectly, notarization fails.

The unexpected culprit: jline-2.14.6.jar

One of the problematic paths stood out:

.../jline-2.14.6.jar/META-INF/native/osx/libjansi.jnilib

This is a native library (libjansi.jnilib) embedded inside the JLine 2 JAR, originally used by Saxon for its interactive “Gizmo” tool. It contained old macOS code and was not signed at all.

For the Publisher, I don’t actually need this library:

  • The Publisher uses Saxon for XML/XSLT processing.
  • The interactive JLine-based console is not part of the normal Publisher workflow.
  • Shipping that native library is therefore unnecessary.

Once I realized that, the simplest solution was: remove JLine from the macOS package.

That avoids the need to sign libjansi.jnilib at all, and it removes old native code that is irrelevant on modern macOS systems (especially on arm64).

Making sure the remaining binaries are signed

The notarization log also showed which other binaries Apple considers interesting in the archive, for example:

  • .../bin/sp
  • .../bin/sdluatex (a universal binary in my case)
  • .../share/lib/libsplib.so
  • .../share/lib/luaglue.so

All of these need a proper Developer ID Application signature.

Instead of manually calling codesign on every single file, I use gon to automate this. Gon lets you define a configuration file that lists:

  • which binaries to sign
  • which certificate to use
  • and how to submit the final ZIP for notarization

So the process for a macOS release looks like this now:

  1. Build the binaries and shared libraries.
  2. Run gon with my configuration:
    • it calls codesign with hardened runtime and timestamp
    • it signs all relevant Mach-O binaries
    • it submits the ZIP to Apple for notarization
  3. Wait for the notarization result from notarytool.

After removing JLine and letting gon sign all the relevant binaries, I rebuilt the ZIP and resubmitted it to Apple.

This time the log looked much nicer:

{
  "status": "Accepted",
  "statusSummary": "Ready for distribution",
  "ticketContents": [
    {
      "path": "speedata-publisher-macos-arm64-5.2.1.zip/.../bin/sp",
      "arch": "arm64"
    },
    {
      "path": "speedata-publisher-macos-arm64-5.2.1.zip/.../bin/sdluatex",
      "arch": "x86_64"
    },
    {
      "path": "speedata-publisher-macos-arm64-5.2.1.zip/.../bin/sdluatex",
      "arch": "arm64"
    },
    {
      "path": "speedata-publisher-macos-arm64-5.2.1.zip/.../share/lib/libsplib.so",
      "arch": "arm64"
    },
    {
      "path": "speedata-publisher-macos-arm64-5.2.1.zip/.../share/lib/luaglue.so",
      "arch": "arm64"
    }
  ]
}

“Accepted / Ready for distribution” is exactly what you want to see.

Final check with Gatekeeper

After downloading and unpacking the ZIP on a fresh system, the final sanity check is:

spctl -a -vvv speedata-publisher/bin/sp

Now the result is:

speedata-publisher/bin/sp: accepted
source=Notarized Developer ID
origin=Developer ID Application: PATRICK GUENTER GUNDLACH (3Y98DLKYBJ)

At that point, macOS is happy and the package is ready to be used without scary warnings.

Summary

For macOS users of the speedata Publisher:

  • The macOS package is now 5.2.1 and properly notarized.
  • Windows and Linux stay at 5.2.0 for this release.
  • The changes are purely about:
    • removing an unnecessary native library (JLine / libjansi.jnilib), and
    • making sure all relevant Mach-O binaries are correctly signed and notarized.

For anyone fighting similar issues:

  • codesign -dv --verbose=4 helps you inspect signatures.
  • spctl -a -vvv tells you what Gatekeeper thinks.
  • xcrun notarytool submit ... --wait plus the JSON log tell you exactly which file is causing notarization to fail.
  • Using a tool like gon makes it much easier to consistently sign and notarize all the right files.

And sometimes the best fix is not to sign more, but to ship less.