macOS OpenGL initialization / security entitlements

[michaelgale]

When building a standalone binary of the CQ-Editor for macOS, the app may or may not crash depending on its launch context. That is:

1. direct from terminal
2. inside lldb
3. macOS launchd when double-clicked in the Finder
4. also macOS launchd when launched from the terminal with the open command

Each mechanism may or may not trigger macOS gatekeeper security and execute the app inside a "sandbox" with path randomization. This path randomization is usually transparent from the point of view of the python introspection, but at a lower level, the oce core accesses the file system to initialize the OpenGL driver as follows:

1. In OCC/Core/Graphic3d.py, ShadersFolder() is called which in turn calls the binary object function _Graphic3d.Graphic3d_ShaderProgram_ShadersFolder(*args)
2. This function is in oce/Graphic3d_ShaderProgram.cxx and it is as follows:

// =======================================================================
// function : ShadersFolder
// purpose  :
// =======================================================================
const TCollection_AsciiString& Graphic3d_ShaderProgram::ShadersFolder()
{
  static Standard_Boolean        THE_IS_DEFINED = Standard_False;
  static TCollection_AsciiString THE_SHADERS_FOLDER;
  if (!THE_IS_DEFINED)
  {
    THE_IS_DEFINED = Standard_True;
    OSD_Environment aDirEnv ("CSF_ShadersDirectory");
    THE_SHADERS_FOLDER = aDirEnv.Value();
    if (THE_SHADERS_FOLDER.IsEmpty())
    {
      OSD_Environment aCasRootEnv ("CASROOT");
      THE_SHADERS_FOLDER = aCasRootEnv.Value();
#ifdef OCE_INSTALL_DATA_DIR
      if (THE_SHADERS_FOLDER.IsEmpty())  {
        THE_SHADERS_FOLDER = OCE_INSTALL_DATA_DIR;
      }
#endif
      if (!THE_SHADERS_FOLDER.IsEmpty())
      {
        THE_SHADERS_FOLDER += "/src/Shaders";
      }
    }

    if (THE_SHADERS_FOLDER.IsEmpty())
    {
      std::cerr << "Both environment variables CSF_ShadersDirectory and CASROOT are undefined!\n"
                << "At least one should be defined to use standard GLSL programs.\n";
      Standard_Failure::Raise ("CSF_ShadersDirectory and CASROOT are undefined");
      return THE_SHADERS_FOLDER;
    }

    const OSD_Path aDirPath (THE_SHADERS_FOLDER);
    OSD_Directory aDir (aDirPath);
    const TCollection_AsciiString aProgram = THE_SHADERS_FOLDER + "/Declarations.glsl";
    OSD_File aProgramFile (aProgram);
    if (!aDir.Exists()
     || !aProgramFile.Exists())
    {
      std::cerr << "Standard GLSL programs are not found in: " << THE_SHADERS_FOLDER.ToCString() << std::endl;
      Standard_Failure::Raise ("CSF_ShadersDirectory or CASROOT is set incorrectly");
      return THE_SHADERS_FOLDER;
    }
  }
  return THE_SHADERS_FOLDER;
}

Depending on runtime sandboxing, the path that this function infers may turn out to be incorrect or obscured. Therefore, when it tests Exists() at the end of the function it raises the error. It is difficult to tell if this function falls back on the hardcoded OCE_INSTALL_DATA_DIR since that will defined when oce is built. If it is, then it is yet another mechanism to not find the Shaders folder.

At the end of the day, the program crashes because OpenGL does not get initialized since it cannot find its Shaders/etc. resources in oce/src.

The secondary mechanism of failure related to #108 is that the evaluation of security entitlements delays OpenGL driver initialization and creates a fatal race condition during app initialization due to early requests to draw axis lines in the 3D viewer.

[michaelgale]

@jmwright @adam-urbanczyk Interesting update:

If I hardcode the absolute path of CSF_ShadersDirectory in pyi_rth_occ.py, i.e. changing oce/src/Shaders to /Users/michaelgale/src/cqgui/CQ-editor/dist/CQ-Editor/oce/src/Shaders the resulting pyinstaller built app works without crashing in any launch context.

Therefore, Graphic3d_ShaderProgram::ShadersFolder() genuinely could not resolve a relative path. Its a pity that we cannot trace into this function at run time to see what it acquires. In particular, stepping through these few lines to see how these temporary variables for the path and filename resolve...

    const OSD_Path aDirPath (THE_SHADERS_FOLDER);
    OSD_Directory aDir (aDirPath);
    const TCollection_AsciiString aProgram = THE_SHADERS_FOLDER + "/Declarations.glsl";
    OSD_File aProgramFile (aProgram);

The macOS sandbox context must mangle the current path or oce's low level file access API (OSD_xxxxxxx) resolve an incorrect path when given a relative path spec.

[michaelgale]

@jmwright @adam-urbanczyk Good news! All fixed!

I added some logic to run.py to introspect the full absolute path of the application, lookup the CSF_ShadersDirectory env variable, and perform a runtime substitution of the env variable with the full path. It only does this if it detects the sys.platform == "darwin". I also added some platform specific logic to initialize QApplication--this silences nuisance warnings about initializing pyqtgraph after QApplication. The application bundle launches in all contexts and includes pretty dock icons for macOS. Next, I'll make a DMG builder script with signing and pretty icons so that macOS users should hopefully be able to download and install in one turnkey step just like any other macOS app.

[jmwright]

@michaelgale That's fantastic! Thank you!

[michaelgale]

@jmwright Should I put all my macOS specific build assets into the CQ-Editor repo and issue a PR?

[jmwright]

@michaelgale If you're talking about things like the spec file, I would say yes. @adam-urbanczyk may have thoughts on the specifics of how to organize things though.

[adam-urbanczyk]

@michaelgale it would be great if you could make a PR! Please try to keep the current structure of a spec file + runtime hooks.

[michaelgale]

@adam-urbanczyk Ok sure.

1. Can I add a folder to the root of the repo for the macos specific assets which contain shell scripts to build the dmg file plus the png and icon assets? I've currently called it 'macos'
2. I've made a few tweaks to the .py source files, in particular run.py, main-window.py to perform the platform specific logic for macos to substitute the environ variables at runtime.
3. If you want to make a release tag, do you want the final built dmg file for the repo? I presume you will wait until your next release cycle to include archives for linux, windows, macos. I can give you the dmg via an alternate hosting or AWS link rather than pollute the repo.

[michaelgale]

This issue can be closed since the PR #111 addresses this problem with run time substitution of the environment variable for CSF Shaders. Platform specific guards ensure that this occurs only for macOS targets.