Determine what needs to be compiled by given changeset

[lsoft]

Hello.

Imagine, I have ConsoleProjectA.csproj, ConsoleProjectB.csproj, and Shared.csproj which used by both consoles.

And I have a list of files that changed (a commit/set of commits/PR).

If changeset contains changes only for files in ConsoleProjectA, I need to compile ConsoleProjectA only, same for ConsoleProjectB. If changeset contains changes in Shared.csproj I need to compile both.

I have a question:

does msbuild have an option to provide me a list of all files for every csproj without performing a real compilation (faster than perform a compilation)? If so, I will be able to compare list of files against changeset, determine affected csprojes and make a decision.

Of course, I need not only compilation units, I need every included file (resource, embedded resource, included Nones, and every custom types of documents - for example AvaloniaResource).

Any ideas?

Thanks!

[lsoft]

I'm able to parse csproj via Microsoft.Build.Construction.ProjectRootElement.Open, but I don't know how to apply XML items (from <ItemGroup>s) against real files on the disk...

csproj can have a very difficult structure sometimes...

[lsoft]

I'm partially solved this:

    internal class Program
    {
        static void Main(string[] args)
        {
            MSBuildLocator.RegisterDefaults();

            Do();

        }

        private static void Do()
        {
            var solutionFilePath = @"path to sln";
            var solutionFolderPath = new FileInfo(solutionFilePath).Directory.FullName;

            var solution = SolutionFile.Parse(solutionFilePath);
            foreach (var project in solution.ProjectsInOrder)
            {
                var projectFilePath = Path.Combine(solutionFolderPath, project.RelativePath);
                var projectFolderPath = new FileInfo(projectFilePath).Directory.FullName;

                var root = Microsoft.Build.Construction.ProjectRootElement.Open(
                    projectFilePath
                    );

                var evProject = Microsoft.Build.Evaluation.Project.FromProjectRootElement(
                    root,
                    new Microsoft.Build.Definition.ProjectOptions
                    {
                    }
                    );
                foreach (var item in evProject.AllEvaluatedItems)
                {
                    var ei = item.EvaluatedInclude;
                    var rooted = Path.IsPathRooted(ei);
                    if((rooted && File.Exists(ei))
                        || (!rooted  && File.Exists(Path.Combine(projectFolderPath, ei)))
                        )
                    {
                        Console.WriteLine($"{ei}");
                    }

                }
            }
        }
    }

but the results are too noisy:

C:\Temp\AvaloniaApplication1\AvaloniaApplication1\obj\project.assets.json
C:\Temp\AvaloniaApplication1\AvaloniaApplication1\obj\project.assets.json
C:\Temp\AvaloniaApplication1\AvaloniaApplication1\obj\Debug\net7.0\AvaloniaApplication1.assets.cache
C:\Program Files\dotnet\sdk\8.0.200\Sdks\Microsoft.NET.Sdk\targets\..\analyzers\Microsoft.CodeAnalysis.CSharp.NetAnalyzers.dll
C:\Program Files\dotnet\sdk\8.0.200\Sdks\Microsoft.NET.Sdk\targets\..\analyzers\Microsoft.CodeAnalysis.NetAnalyzers.dll
C:\Users\IAm\.nuget\packages\skiasharp.nativeassets.webassembly\2.88.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libSkiaSharp.a\2.0.23\libSkiaSharp.a
C:\Users\IAm\.nuget\packages\skiasharp.nativeassets.webassembly\2.88.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libSkiaSharp.a\2.0.6\libSkiaSharp.a
C:\Users\IAm\.nuget\packages\skiasharp.nativeassets.webassembly\2.88.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libSkiaSharp.a\3.1.12\simd\libSkiaSharp.a
C:\Users\IAm\.nuget\packages\skiasharp.nativeassets.webassembly\2.88.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libSkiaSharp.a\3.1.12\st\libSkiaSharp.a
C:\Users\IAm\.nuget\packages\skiasharp.nativeassets.webassembly\2.88.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libSkiaSharp.a\3.1.7\libSkiaSharp.a
C:\Users\IAm\.nuget\packages\harfbuzzsharp.nativeassets.webassembly\2.8.2.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libHarfBuzzSharp.a\2.0.23\libHarfBuzzSharp.a
C:\Users\IAm\.nuget\packages\harfbuzzsharp.nativeassets.webassembly\2.8.2.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libHarfBuzzSharp.a\2.0.6\libHarfBuzzSharp.a
C:\Users\IAm\.nuget\packages\harfbuzzsharp.nativeassets.webassembly\2.8.2.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libHarfBuzzSharp.a\3.1.12\libHarfBuzzSharp.a
C:\Users\IAm\.nuget\packages\harfbuzzsharp.nativeassets.webassembly\2.8.2.3\buildTransitive\netstandard1.0\..\..\build\netstandard1.0\libHarfBuzzSharp.a\3.1.7\libHarfBuzzSharp.a
C:\Users\IAm\.nuget\packages\avalonia\11.0.2\buildTransitive\AvaloniaItemSchema.xaml
Program.cs
app.manifest
..\AvaloniaApplication1\AvaloniaApplication1.csproj
obj\Debug\net7.0\AvaloniaApplication1.Desktop.dll
C:\Temp\AvaloniaApplication1\AvaloniaApplication1.Desktop\obj\Debug\net7.0\Avalonia.563E3AD1.Up2Date
obj\Debug\net7.0\refint\AvaloniaApplication1.Desktop.dll
obj\Debug\net7.0\AvaloniaApplication1.Desktop.pdb
bin\Debug\net7.0\AvaloniaApplication1.Desktop.pdb
obj\Debug\net7.0\AvaloniaApplication1.Desktop.dll
C:\Temp\AvaloniaApplication1\AvaloniaApplication1.Desktop\obj\Debug\net7.0\AvaloniaApplication1.Desktop.dll
C:\Temp\AvaloniaApplication1\AvaloniaApplication1.Desktop\obj\Debug\net7.0\AvaloniaApplication1.Desktop.pdb
C:\Temp\AvaloniaApplication1\AvaloniaApplication1.Desktop\obj\project.assets.json
C:\Temp\AvaloniaApplication1\AvaloniaApplication1.Desktop\obj\project.assets.json
C:\Temp\AvaloniaApplication1\AvaloniaApplication1.Desktop\obj\Debug\net7.0\AvaloniaApplication1.Desktop.assets.cache
C:\Program Files\dotnet\sdk\8.0.200\Sdks\Microsoft.NET.Sdk\targets\..\analyzers\Microsoft.CodeAnalysis.CSharp.NetAnalyzers.dll
C:\Program Files\dotnet\sdk\8.0.200\Sdks\Microsoft.NET.Sdk\targets\..\analyzers\Microsoft.CodeAnalysis.NetAnalyzers.dll

here is

Program.cs
app.manifest

are the results I need, but something from nuget packages are miss for me.

Could this be filtered somehow or I need a completely different approach?

[rainersigwald]

This is not the way MSBuild thinks of the problem and thus there's no good way to get the answer you want from MSBuild.

You might look at something like Buildalyzer which can do some related things, and the .NET Project System Fast Up-To-Date Check is also related.

Note though that defining what files are in what projects is extremely difficult in general. MSBuild is powerful and flexible and changes that seem totally unrelated can result in other changes.

[lsoft]

@rainersigwald thank you! I'll take a look at these projects.

Could you add more light, why the approach via Microsoft.Build.Evaluation.Project is bad for use-case of tracking changes in real files? For example, I do not care about generated files, because programmer can't change and push them.

I think (probably wrong) that evaluate csproj will give me all files (+noise) which inclulded in that csproj. Am I wrong with this statement?

[rainersigwald]

Yes, unfortunately. State (like "lists of files") in MSBuild is mutable during the build, and it is possible (and done!) to dramatically modify state during the build in relevant ways. For instance one of the things that isn't known at evaluation time (before running any targets) is the list of referenced assemblies (DLLs)--those are built up by targets/tasks that do things like "transform the ProjectReference paths into concrete output assembly paths" and "figure out which assets from NuGet are relevant to this build", both of which are very interesting for incremental compilation.

[lsoft]

@rainersigwald thanks for clarification.

I has chosen if

• any props file has changed -> rebuild everything
• any sln file has changed -> rebuild this sln and all slnf based on it
• any csproj file has changed -> rebuild any sln and slnf contains it
• any other file has been analyzed on evaluation time to determine changed csprojes and make the decision what to rebuild.

I have prototyped. Looks like it works. But I know a little about msbuild and don't see weak points because of that.

You mentioned about referenced assemblies and nugets. If I understand correctly, this is solved by tracking changes in the csproj files...

If you have the time and opportunity (and desire!), please tell me the weak points of my solution based on the logic description given above? If not then no problem, you have helped me a lot already!

Thanks!