DLL loading fails on UWP platforms due to kernel32.dll not being available

[NoTuxNoBux]

Hello

I’m trying to get Assimp working on a Windows ARM64 device in an UWP context using Unity, but I’m getting stuck on Assimp.NET trying to import functions from kernel32.dll in order to load the actual Assimp DLL, which is likely not available or allowed for direct usage in the protected UWP context I’m using:

DllNotFoundException: Unable to load DLL 'kernel32.dll': The specified module could not be found.
  at Assimp.Unmanaged.UnmanagedLibrary+UnmanagedWindowsLibraryImplementation.NativeLoadLibrary (System.String path) [0x00000] in <00000000000000000000000000000000>:0 
  at Assimp.Unmanaged.UnmanagedLibrary+UnmanagedLibraryImplementation.LoadLibrary (System.String path) [0x00000] in <00000000000000000000000000000000>:0 
  at Assimp.Unmanaged.UnmanagedLibrary.LoadLibrary (System.String libPath) [0x00000] in <00000000000000000000000000000000>:0 
  at Assimp.AssimpUnity.InitializePlugin () [0x00000] in <00000000000000000000000000000000>:0 

I’ve considered simply marking the Assimp DLL as “Load On Startup” in Unity (I always need it anyway) and trying to skip Assimp.NET’s library loading, but it still needs to be able to fetch pointers to the functions from the handle. It also seems impossible to replace anything in the library loading class without making direct changes to the Assimp.NET codebase itself. Replacing AssimpLibrary.Instance with another, custom, instance, is also impossible, as it is read-only.

In case it helps: when DLLs are included in the Plugins folder in Unity, they are flattened in the final UWP build and end up next to the executable. This means you should be able to do LoadLibrarySomehow(“Assimp.dll“) (note that no path is present), and Windows will look for the DLL next to the executable. See also here for more information.

If DLL loading on UWP is not possible directly, perhaps it would be possible to somehow retrieve the functions from the DLL if it already has been loaded into memory?

EDIT: Perhaps LoadPackagedLibrary is an option on UWP?

[Nicholas Woodfield]

I’m not that familiar with UWP. The library doesn’t go well with platforms that are restrictive with dynamic library loading (e.g. iOS is the big one), but if this just needs a slightly different codepath to load the native DLL via LoadPackagedLibrary then we’d just need to add a new unmanaged library implementation and a means to detect UWP vs a regular windows app. Maybe in the process open up the unmanaged library to take in an external implementation so we’re not restricted to the current set of implementations. Feel free to open a PR for this if you’ve experimented with an implementation.

[NoTuxNoBux]

Thanks for the response. I have the misfortune of having to work with UWP everywhere, but I’ve successfully been able to get this working.

The code is probably too dirty for a PR straightaway, but I can show what I’ve done - hopefully you have a better vision of how to integrate this without potentially breaking other platforms .

I changed NativeLoadLibrary to this:

protected override IntPtr NativeLoadLibrary(String path)
{
    IntPtr? libraryHandle;

    try
    {
        libraryHandle = WinNativeLoadLibrary(path);
    }
    catch (DllNotFoundException)
    {
        libraryHandle = null;
    }

    if (libraryHandle is null)
    {
        try
        {
            //If we're running in an UWP context, we need to use LoadPackagedLibrary. On non-UWP contexts, this
            //will fail with APPMODEL_ERROR_NO_PACKAGE, so fall back to LoadLibrary.
            libraryHandle = WinUwpLoadLibrary(path);
        }
        catch (DllNotFoundException)
        {
            libraryHandle = null;
        }
    }

    if((libraryHandle is null || libraryHandle == IntPtr.Zero) && ThrowOnLoadFailure)
    {
        Exception innerException = null;

        //Keep the try-catch in case we're running on Mono. We're providing our own implementation of "Marshal.GetHRForLastWin32Error" which is NOT implemented
        //in mono, but let's just be cautious.
        try
        {
            int hr = GetHRForLastWin32Error();
            innerException = Marshal.GetExceptionForHR(hr);
        }
        catch(Exception) { }

        if(innerException != null)
            throw new AssimpException(String.Format("Error loading unmanaged library from path: {0}\n\n{1}", path, innerException.Message), innerException);
        else
            throw new AssimpException(String.Format("Error loading unmanaged library from path: {0}", path));
    }

    return libraryHandle is null ? IntPtr.Zero : (IntPtr)libraryHandle;
}

What is happening here is I’m trying the standard DLL load first (which is the one that fails now), catch the exception if it does, and then try to load it using the UWP way - which, of course, does not work outside UWP packages according to the MSDN documentation, so both paths need to be present. It might be better to “detect UWP” somehow, here, and choose a path based on that, but that doesn’t seem to be straightforward.

Furthermore, I changed the native methods region in the same file to this:

#region Native Methods

[DllImport("__Internal", SetLastError = true, EntryPoint = "LoadPackagedLibrary")]
public static extern IntPtr WinUwpLoadLibrary([MarshalAs(UnmanagedType.LPWStr)] string libraryName, int reserved = 0);

[DllImport("kernel32.dll", CharSet = CharSet.Ansi, BestFitMapping = false, SetLastError = true, EntryPoint = "LoadLibrary")]
private static extern IntPtr WinNativeLoadLibrary(String fileName);

[DllImport("__Internal", SetLastError = true)]
private static extern bool FreeLibrary(IntPtr hModule);

[DllImport("__Internal")]
private static extern IntPtr GetProcAddress(IntPtr hModule, String procName);

#endregion

The original suggested solution using API-MS-WIN-CORE-LIBRARYLOADER-L2-1-0.DLL and LoadPackagedLibrary supposedly works in an UWP context, but it does not in the ARM64 UWP context I’m using (on the HoloLens 2). What does seem to work is __Internal, which is also a special Unity construction, I believe; it means Unity will look inside the existing process space (also used for statically linked libraries on iOS, it seems), which probably happens to work in my case, because kernel32.dll is in fact already loaded into the process on start-up according to Visual Studio in my case - you just can’t explicitly reference these DLLs in a protected UWP context, it seems.

Application-side, the main thing to take into account is that you likely want to omit the path and just mention the DLL name, such as AssimpNet.dll. This is because you have limited access to most of the filesystem, the DLL hierarchy in Unity is flattened and all DLLs end up next to the executable, which is also one of the locations that Windows searches for DLLs in an UWP context when trying to load one.

(In my case, I also needed to build an ARM64 and UWP-supporting DLL from Assimp, which is not included in AssimpNet, which was fairly straightforward.)

[Mark Preston]

I hope it’s okay to piggyback onto this discussion, but I’d love to see the unmanaged library opened up further.

In my particular case, I’m using a .NET 5 single-file app, with the IncludeNativeLibrariesForSelfExtract option set to true in the project. The unmanaged libraries are embedded inside the single-file app, and then extracted by the runtime to a temporary directory when launching it.

Unfortunately, the directory that it’s extracted to isn’t exposed anywhere, and shouldn’t be relied on as it could change in future. Instead, you should load and free the library using the NativeMethods class, passing only the filename without a full path, and a reference to the single-file assembly. The Load (string libraryName, System.Reflection.Assembly assembly, System.Runtime.InteropServices.DllImportSearchPath? searchPath); method knows to look in the temporary directory when searching for the library.

With the current approach, it’s not possible to load the unmanaged library from a single-file app, so I’ve resorted to adding LibraryLoadFunction and LibraryFreeAction properties to UnmanagedWindowsLibraryImplementation, that I then set from my app to use the NativeMethods class.

[Nicholas Woodfield]

Sounds like a good opportunity for a PR.

At some point I’d love to get these libraries onto .NET 5+ and say goodbye to the .NET Standard targets. Probably can remove a lot of code w.r.t. to interop with the newer APIs.

[NoTuxNoBux]

I upstreamed my fixes for the original issue by adding support for UWP’s LoadPackagedLibrary in PR #5.

For the HoloLens 2 specifically (also UWP), I have to test further, because it worked fine with __Internal as DLL name, but I could not keep that as it breaks Unity’s IL2CPP when targeting other platforms (it wants to reference the symbol regardless of the target platform, and it seems impossible to disable it without splitting the AssimpNet DLL up into multiple assemblies, which I want to avoid, as the problem is specific to Unity’s IL2CPP back end). The current approach uses officially documented UWP DLL’s to reference LoadPackagedLibrary, so it should work at least on desktop UWP (which is what did not work on the HoloLens originally, even though it’s also UWP).

[NoTuxNoBux]

Looks I had an oops when making the changes for Linux in PR #5:

diff --git a/AssimpNet/Unmanaged/UnmanagedLibrary.cs b/AssimpNet/Unmanaged/UnmanagedLibrary.cs
index c335ab0..3b10b2a 100644
--- a/AssimpNet/Unmanaged/UnmanagedLibrary.cs
+++ b/AssimpNet/Unmanaged/UnmanagedLibrary.cs
@@ -341,6 +341,8 @@ namespace Assimp.Unmanaged
                 //If we're running in an UWP context, we need to use LoadPackagedLibrary. On non-UWP contexts, this
                 //will fail with APPMODEL_ERROR_NO_PACKAGE, so fall back to LoadLibrary.
                 WinUwpLoadLibrary("non-existent-dll-that-is-never-used.dll");
+
+                return new UnmanagedUwpLibraryImplementation(defaultLibName, unmanagedFunctionDelegateTypes);
             }
             catch (DllNotFoundException)
             {

The exception is always thrown because I forgot to actually use the back end. (I can’t make a PR right now due to Git LFS issues with the repository; I tried re-forking, but it has problems with the new libassimp.so DLLs, saying I have no permissions to download these artifacts. I tried the Bitbucket workaround for LFS support in forks, but it doesn’t work for me.)

Also, to follow up on my previous post: nope, whilst api-ms-win-core-libraryloader-l2-1-0.dll appears to be the correct DLL to use on UWP (I even verified it exists on the HoloLens 2), on the HoloLens 2 specifically, it doesn’t work; you don’t seem to be allowed to pull this function from this DLL like this; it works fine if you address LoadPackagedLibrary in C++ UWP so that the OS loads in the DLL when the application is loaded, but not if you use DllImport in .NET. The only workaround I’ve found is to use __Internal instead, but this will break Android and other platform builds in Unity IL2CPP (as mentioned above), so we’ve decided to have two AssimpNet assemblies in our Unity project until we find a better solution.

It’s possible there is some additional UWP permission that is necessary, but I’ve not found one so far.