Copy native libs in a local C# project

CSharpNativeLibs.md

I want to copy OS-specific native runtime libraries to the bin directory. This will copy them, but will copy ALL runtimes (win-x64, linux-x64, osx, etc).

<ItemGroup>
  <Content Include="libs/**/*.*" CopyToOutputDirectory="PreserveNewest" Link="%(Filename)%(Extension)" />
</ItemGroup>

---

OK so we can just check the current platform, and only copy those:

<ItemGroup Condition="$([MSBuild]::IsOSPlatform('Windows'))">
  <Content Include="libs/win-x64/**" CopyToOutputDirectory="PreserveNewest" Link="%(Filename)%(Extension)" />
</ItemGroup>

Except this will now copy the host OS runtimes when you specify a runtime identifier (ex. dotnet build -r linux-x64 will still copy Windows libs)...

---

OK, so we can just use the runtime identifer instead:

<ItemGroup Condition="$(RuntimeIdentifier) == 'win-x64'">
  <Content Include="libs/win-x64/**" CopyToOutputDirectory="PreserveNewest" Link="%(Filename)%(Extension)" />
</ItemGroup>

Except now when you do a standard dotnet build, the runtime identifier is never set, so this breaks simple builds...

---

OK, so we can check if the runtime identifier is empty and fallback to the current OS:

<ItemGroup Condition="($(RuntimeIdentifier) == '' and $([MSBuild]::IsOSPlatform('Windows'))) or $(RuntimeIdentifier) == 'win-x64'">
  <Content Include="libs/win-x64/**" CopyToOutputDirectory="PreserveNewest" Link="%(Filename)%(Extension)" />
</ItemGroup>

Except now, when you do ex. dotnet publish -r linux-x64, msbuild runs in multiple steps, and during some of those steps $(RuntimeIdentifer) will not be assigned, so you will end up with both the host OS runtimes and the target runtimes in the bin directory...

---

So now what? I genuinely do not know how to simply copy native libs to the bin directory such that dotnet build and dotnet publish -r [target] both work. I could just always copy all the native libraries, regardless of platform, and use NativeLibrary.SetDllImportResolver to find the correct ones at runtime, but that seems kind of ridiculous.

I spoke with a couple people about this and got some info for you and anyone else having this problem:

If your project doesn't specify a RuntimeIdentifier, it doesn't know which runtime you target and therefore it copies everything into the output directory and let the host pick at runtime. If you specify a RuntimeIdentifier, then nuget (assuming the native asset is coming from a nuget package with the expected relative path) will only copy the nearest applicable asset.

in addition you probably don't want to do this kind of ItemGroup directly inside your project (which causes it to happen for every build of your project, even invisible internal builds) - the way we'd likely guide this is to find the appropriate Target that first during the kinds of builds you care about and add Items for the native libs you want at that point. This should reduce the cases where you copy too much/wrong RID/other failure cases. The trick is finding out what that target to hook off of is

Based on this my personal suggestion would be to try setting a default RID so that in cases where you build without setting an RID, an RID gets set and you can always just conditionally check based on the RID. I think that would look like this, but I haven't tried it:

<PropertyGroup> <!-- near the top of your project -->
  <RuntimeIdentifier Condition="'$(RuntimeIdentifier)' == ''">linux-x64</RuntimeIdentifier>
</PropertyGroup>

If you want to get fancy you can change the default based on the host OS, which you already know how to do. This may still not work right for the publish scenario.

Someone else has also mentioned that properties being missing can mean that your itemgroups are being evaluated 'too early', and the answer for that is to put your items in a Target that runs later in the build process, like mentioned in the second quote. This should be somewhat clear if you capture a binlog of the case where properties are missing.

EDIT: For me, a target running before BeforeBuild always has a RuntimeIdentifier set, even if I don't specify one on the command line.

  <Target Name="DebugSpew" BeforeTargets="BeforeBuild">
    <Message Importance="High" Text="RuntimeIdentifier=$(RuntimeIdentifier)" />
  </Target>

If you continue having issues, please capture a binlog of your build misbehaving (you can do this by passing the -bl switch to the build command) and send it to me and we can look into it!

Hey thanks for the detailed answer! :)

try setting a default RID

This sort of works but has the issue I mentioned in the original post where I believe during a dotnet publish -r [target] it will copy content for both the host runtimes and the target runtimes ...

For me, a target running before BeforeBuild always has a RuntimeIdentifier set

Unfortunately I don't seem to be able to get this to work... RuntimeIdentifier seems to always be empty, even in BeforeBuild and AfterBuild, unless I add <RuntimeIdentifiers> to the PropertyGroup, in which case it will perform both an empty one and then one with the correct value. I guess this makes sense as it does a generic build and then since it has a matching runtime identifier, it performs that build.

assuming the native asset is coming from a nuget package with the expected relative path

I'm not using a nuget package in this case as this is just a local csproj (imagine I wrote a C dll and have it for various runtimes, and want my C# project to use it). I know nuget handles this more gracefully as I have set up public-facing nuget packages that do this before. If that's the intended route I could create a local nuget package I guess, but that also seems weird?

please capture a binlog of your build misbehaving

I'll do this and DM you shortly!

---

I've been investigating a bit more and I do think my project has some weirdness. Alongside this local project with custom libs, I am importing nuget packages with runtimes libs (ImGui), and so even when I don't include -r in my build, it still puts everything into an RID folder - but all my reporting ($(OutputPath), $(PublishDir), $(OutDir), $(TargetDir)) don't include that in their path. I was briefly thinking I could just detect the runtime from the path but

OK, after a lot of debugging with Kate I have come to a solution:

I needed to specify that the csproj was not RID agnostic, and would use the current RID so that dotnet build still worked:

<IsRidAgnostic>false</IsRidAgnostic>
<UseCurrentRuntimeIdentifier>true</UseCurrentRuntimeIdentifier>

This would ensure that an RID always exists, regardless of whether it's passed in through -r or not, for both dotnet build and dotnet publish.

After that, it was fairly trivial to copy the libraries needed to the bin:

<Target Name="CopyNativeLibs" BeforeTargets="PrepareForBuild">
  <Message Importance="High" Text="RuntimeIdentifier=$(RuntimeIdentifier)" />
  <ItemGroup>
    <Content Include="libs/$(RuntimeIdentifier)/*" />
    <Content
      Update="libs/$(RuntimeIdentifier)/*" 
      CopyToOutputDirectory="PreserveNewest"
      TargetPath="%(Filename)%(Extension)"
      Link="%(Filename)%(Extension)"/>
  </ItemGroup>
</Target>

The only other thing that had to happen was to disable default items as otherwise a modern csproj will include all of the libraries anyway. And then it worked!

A few other notes:

• I couldn't use <Copy> because this is a Library project, included several layers deep. <Copy> wouldn't put it in the Exe bin dir, just the Library bin.
• Because it was nested through several references, I had to make sure the whole chain had <IsRidAgnostic> set to false.

Ultimately a lot of this would be avoided if you just copied all of the native libs and accepted they'd all be there, but I wanted to try and figure this out. Appreciate the help from everyone who reached out!