Build process change WRT references

[TahirAhmadov]

Can we change it to check for the referenced DLL's public surface area, and only treat it as a reason to build this project if the referenced DLL's public surface area changed, and not whenever the referenced DLL was touched in any way (such as due to a change in a private member or method body code)?

I envision that each DLL gets a "public surface area image" (PSAI), which is a loss-less compressed signature, basically a byte array. Then, for each project which references DLLs, we add a "cache" of the PSAI for each reference, and store it in something like the obj folder. After we detect a write time change of the referenced DLL, we compare the cached PSAI with the one from the newly modified referenced DLL, and if the PSAI has actually changed, consider this reference to be a trigger for rebuilding our current project.

• This would drastically reduce the build times for large solutions, because if there is a dependency chain of A->B->C->D, and D gets a non-public change, then currently, we rebuild A, B, C, and D; whereas with my proposal, we would only build D, run the public surface area comparison on C (which should be a simple byte array comparison and therefore very quick), and (granted D's public surface area didn't change), skip building C, and do absolutely nothing for A and B (since C wasn't rebuilt, its write time hasn't changed; ditto for B).

• Also, deployments would be faster, because the build/deployment tools can be optimized and only copy the few DLLs that actually need to be copied (or even, get immediate improvements if they already compare write times between build and deployment locations).

• Downside: the build time for each individual project will go up a bit, as the PSAI will need to be generated.

• And finally, this behavior can be made optional, so those users who prefer the old behavior (if they feel it's more reliable) can continue using it.

[rainersigwald]

This is called reference assemblies and has been supported in C# and VB for many years. It's on by default for .NET 5.0+ and can be enabled for any project, as of 17.5 (prior to this it was supported only in projects that used the .NET SDK but caused problems in Visual Studio if enabled for non-SDK projects).

We're planning to turn it on by default for all projects, tracked by #8133.

[TahirAhmadov]

Oh great, thank you very much.
Weird, I use .NET 6 and it's not enabled by default.
I'll give it a shot, using info from https://learn.microsoft.com/en-us/visualstudio/msbuild/common-msbuild-project-properties?view=vs-2022.

[rainersigwald]

Weird, I use .NET 6 and it's not enabled by default.

What makes you say that? It should be basically invisible when it's on.

Oh and I should have mentioned the new-to-17.5 AccelerateBuildsInVisualStudio setting.

[TahirAhmadov]

What makes you say that? It should be basically invisible when it's on.

I still get all DLLs rebuilt regardless of public/private changes.

Oh and I should have mentioned the new-to-17.5 AccelerateBuildsInVisualStudio setting.

I'm still on 17.3.6. I guess, it's this setting which would get rid of all the re-builds?

Once again, thanks for the response.

[TahirAhmadov]

Update: I updated to 17.5, and in regular builds, it does "accelerate" the build significantly, which is great.
However, when I publish, it still causes a full rebuild of the entire solution, if I modify a widely used low level library in a way which doesn't affect it's public surface area. Is it possible to make publish work similarly to build?

[jrdodds]

On the diagnostic verbosity level, MSBuild can output messages about why a project is being built.

There are some things that will cause a project to build every time. Some common causes are:

• Files with CopyToOutputDirectory="Always"
  • The project will always be built so that the copy can always be performed
  • The fix is straight-forward: change to CopyToOutputDirectory="PreserveNewest"
• 'Build Events'
  • Includes
    • Pre-Build Event (C#, C++)
    • Pre-Link Event (C++)
    • Post-Build Event (C#, C++)
    • Custom Build Step with no Outputs specified (C++)
  • All of these shell out and perform commands.
  • With the Pre-Build/Pre-Link/Post-Build which don't support outputs and with the Custom Step without Outputs, MSBuild can't determine that the project is up-to-date.
  • Post-Build Events that perform copies are probably the most common example and can easily be replaced with native MSBuild (which both solves the 'up-to-date' check and eliminates the 'shell out').