Output from publishing file-based app using web SDK is included by default items, causes errors on subsequent publish

[DamianEdwards]

The default output directory for publishing file-based apps, ./artifacts/<app name>, next to the app file, is not excluded from the app's default non-compile items. This is especially problematic for app's using the Microsoft.NET.Sdk.Web SDK which includes *.json files in its default items. This results in issues if you call publish multiple times without clearing the output directory, as the JSON files in the output directory get included in the next publish command.

Repro steps:

1. Create a file-based app, e.g. app.cs using the web SDK:

   #:sdk Microsoft.NET.Sdk.Web
   #:property PublishAot=false
   var builder = WebApplication.CreateBuilder(args);
   var app = builder.Build();
   app.MapGet("/", () => "Hello World!");
   app.Run();

2. Run dotnet publish app.cs
3. There will now be an artifacts/app directory next to the app.cs file, containing the publish output
4. Run dotnet publish app.cs again
   a. This can also be a container publish:
   dotnet publish app.cs --os linux --arch x64 -t:PublishContainer
5. Observe the warnings about files being in use. If the publish succeeds (I've seen it fail, I've seen retries on the file in use succeed), note the publish output directory now contains "recursive" items, e.g. ./artifacts/app/artifacts/app/*.json

We might want to consider updating the default non-compile items globs for file-based apps to exclude the artifacts directory.

[willdean]

Could the default output be in "bin" for consistency with previous approaches and exclusion patterns? (Not just the publish glob, but typical .gitignore, etc)

[DamianEdwards]

@willdean we specifically didn't want a bin but instead wanted to adopt a more modern layout similar to artifacts output layout. artifacts is widely used in the developer ecosystem across different stacks as the output location for build, etc. and is already in common .gitignore files, etc.

[baronfel]

For file-based apps, the decision was made to explicitly allow the publish gesture to leave binaries next to the app in question - functionally speaking this means setting PublishDir to something like ./artifacts/<appName> when publishing file-based apps.

For other projects, PublishDir is generally set by default to a location under the BaseOutputPath or explicitly managed by a user (by the -o/--output flags on dotnet publish). In these cases the published assets generally aren't within the 'cone' of items globbed by default by the SDK.

The SDK already tries to filter out these base output paths from the set of items that are globbed:

sdk/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Sdk.DefaultItems.targets

Lines 34 to 37 in c2d964b

	<!-- bin folder, by default -->
	<DefaultItemExcludes>$(DefaultItemExcludes);$(BaseOutputPath)/**</DefaultItemExcludes>
	<!-- obj folder, by default -->
	<DefaultItemExcludes>$(DefaultItemExcludes);$(BaseIntermediateOutputPath)/**</DefaultItemExcludes>

For these other projects, PublishDir is contained in these locations - but that's not actually an enforced thing. I think we can solve this problem for file-based apps, (as well as the general correctness problem in the SDK overall) by adding another entry here to exclude the PublishDir/** glob in a similar fashion - though we should condition this on the PublishDir actually existing.

[DamianEdwards]

@baronfel nice. What would a workaround be for someone who hits this today? Can they do something like the following in the file-based app .cs file, or do they need to have it in a Directory.Build.targets to ensure ordering?

#:property DefaultItemExcludes=$(DefaultItemExcludes);./artifacts/**

[baronfel]

It looks to me that a property directive like you have would solve the issue. I just tested this, got a binlog, and was able to confirm that no items were created for the artifacts dir. So there ya go!