Skip to content

Conversation

@RikkiGibson
Copy link
Member

@RikkiGibson RikkiGibson commented Oct 20, 2025

This package will be used to support dotnet/roslyn#80575.
Manual review of the first 2 commits may be helpful.

  • 99075fb is a pure move
  • 0c909f4 introduces the source package, refactors to make things work under netstandard2.0, etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why copying this file is needed. Does adding <Compile> to the package project not work? I see you are saying "It seems like a .cs file which is "out of tree" doesn't make it into the package." but I've recently created a source package where all the files come from a different directory and it works fine:

https://github.com/dotnet/roslyn/blob/main/src/NuGet/Microsoft.CodeAnalysis.BuildClient.Package/Microsoft.CodeAnalysis.BuildClient.Package.csproj

Although I wasn't using shproj and projitems, perhaps that's the problem?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my case I tried adding a Compile item to the .projitems. I'll take a look at the build for the source project you linked and try to work out what the difference is.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take another stab at this soon, my current guess is it is related to a LinkBase attribute on the item, or similar.

}

#if FILE_BASED_PROGRAMS_SOURCE_PACKAGE_GRACEFUL_EXCEPTION
internal class GracefulException : Exception
Copy link
Member

@tmat tmat Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove all the exception throwing? We already report diagnostics, right?
The calling code can convert diagnostics to an exception if needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be best to remove usage of this type from the source package. But, it will involve a decent chunk of refactoring.

// Also normalize blackslashes to forward slashes to ensure the directive works on all platforms.
var sourceDirectory = Path.GetDirectoryName(context.SourceFile.Path) ?? ".";
var resolvedProjectPath = Path.Combine(sourceDirectory, directiveText.Replace('\\', '/'));
if (Directory.Exists(resolvedProjectPath))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this may depend on the process current directory. Should we abstract that? Depending on process state makes it hard to test and reuse the code in different environments

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting adding some way to pass in a base directory for resolution here? Perhaps using the compilation’s SourceReferenceResolver if it exists?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps pass additional defaultDirectory parameter or something similar.

Copy link
Member Author

@RikkiGibson RikkiGibson Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit: apologies. Churning on this comment a bit as I think through the scenarios.

I was considering what we should do when context.SourceFile.Path is not absolute.

In the CLI, it looks like a previous layer verifies that the path which is used is absolute. So I think we will not even get to the path that Tomas pointed out when the path is not absolute. So, the non-absolute path scenario probably can't be exercised from the CLI tests.

public VirtualProjectBuildingCommand(
string entryPointFileFullPath,
MSBuildArgs msbuildArgs)
{
Debug.Assert(Path.IsPathFullyQualified(entryPointFileFullPath));
EntryPointFileFullPath = entryPointFileFullPath;

I think the editor scenario where we could get such a path is for an "untitled file". If you just ctrl+N in VS Code, and start writing a file-based app, with directives and all, we have no idea what to resolve those directives relative to. I think in that scenario, we should probably report that we don't know what project the #: refers to, because the current file doesn't have a full path. The message could even hint that the file needs to be saved to disk.

Do also note that the Path which flows in here, would come from the SyntaxTree, and as long as it is saved to disk, we would expect to get a full path for it.

I think we should punt this to a new issue, as we might want additional tests in both the CLI and editor scenarios. I think we definitely don't want a dependency on the process current directory. So the resolution to this issue would involve deleting the ?? ".".

@RikkiGibson RikkiGibson changed the base branch from main to release/10.0.2xx October 23, 2025 19:03
@RikkiGibson
Copy link
Member Author

When attempting to use the package in Roslyn, I get the following exception in the analyzer.

[System.Resources.MissingManifestResourceException thrown]	
[External Code]	
FileBasedProgramsResources.GetResourceString(string resourceKey, string defaultValue) Line 15	C#
FileBasedProgramsResources.CouldNotFindProjectOrDirectory.get() Line 19	C#
CSharpDirective.Project.Parse(CSharpDirective.ParseContext context) Line 492	C#
CSharpDirective.Parse(CSharpDirective.ParseContext context) Line 315	C#
AppDirectiveHelpers.FindLeadingDirectives(SourceFile sourceFile, SyntaxTriviaList triviaList, DiagnosticBag diagnostics, ImmutableArray<CSharpDirective>.Builder builder) Line 165	C#

It looks like the .resources file is not getting packed, I will need to figure out how to do that.

tree output in the source package:

F:\.nuget\packages\microsoft.dotnet.filebasedprograms\10.0.200-dev
│   .nupkg.metadata
│   Icon.png
│   microsoft.dotnet.filebasedprograms.10.0.200-dev.nupkg
│   microsoft.dotnet.filebasedprograms.10.0.200-dev.nupkg.sha512
│   microsoft.dotnet.filebasedprograms.nuspec
│
├───build
│       Microsoft.DotNet.FileBasedPrograms.targets
│
└───contentFiles
    └───cs
        ├───net9.0
        │   │   .editorconfig
        │   │   AppDirectiveHelpers.cs
        │   │   ExternalHelpers.cs
        │   │   MSBuildUtilities.cs
        │   │
        │   └───FileBasedProgramsResources.resx
        │           Microsoft.DotNet.FileBasedPrograms.FileBasedProgramsResources.cs
        │
        └───netstandard2.0
            │   .editorconfig
            │   AppDirectiveHelpers.cs
            │   ExternalHelpers.cs
            │   MSBuildUtilities.cs
            │
            └───FileBasedProgramsResources.resx
                    Microsoft.DotNet.FileBasedPrograms.FileBasedProgramsResources.cs

@RikkiGibson
Copy link
Member Author

I am unsure whether it is better to include the outputs of compiling the .resx/.xlf files, and add some kind of targets to ensure the resulting .resources get embedded into the resource assemblies of the consuming project, or, to actually embed the original .resx/.xlf as items and have the consuming project compile them.

I didn't see any examples of a source-only package which includes resource files. Do we have any examples of that in our org? Tagging @tmat @baronfel in case you happen to know (or know where to look.)

@tmat
Copy link
Member

tmat commented Oct 24, 2025

I am unsure whether it is better to include the outputs of compiling the .resx/.xlf files, and add some kind of targets to ensure the resulting .resources get embedded into the resource assemblies of the consuming project, or, to actually embed the original .resx/.xlf as items and have the consuming project compile them.

I didn't see any examples of a source-only package which includes resource files. Do we have any examples of that in our org? Tagging @tmat @baronfel in case you happen to know (or know where to look.)

Resources in source packages are PITA. Is it possible to avoid?

@RikkiGibson
Copy link
Member Author

RikkiGibson commented Oct 24, 2025

The resources are basically the error messages for bad #:property etc directives. It would be possible to require the consumer of the source package to supply the resource strings, and copy paste them into the .resx of each repo essentially, but it seems unfortunate.

This is the content that would be copied (and possibly translations performed again, unless, perhaps with comments we give the translators a heads up that there is another copy elsewhere to use as a reference.)

<data name="CouldNotFindAnyProjectInDirectory" xml:space="preserve">
<value>Could not find any project in `{0}`.</value>
</data>
<data name="CouldNotFindProjectOrDirectory" xml:space="preserve">
<value>Could not find project or directory `{0}`.</value>
</data>
<data name="MoreThanOneProjectInDirectory" xml:space="preserve">
<value>Found more than one project in `{0}`. Specify which one to use.</value>
</data>
<data name="PropertyDirectiveInvalidName" xml:space="preserve">
<value>Invalid property name: {0}</value>
<comment>{0} is an inner exception message.</comment>
</data>
<data name="PropertyDirectiveMissingParts" xml:space="preserve">
<value>The property directive needs to have two parts separated by '=' like '#:property PropertyName=PropertyValue'.</value>
<comment>{Locked="#:property"}</comment>
</data>
<data name="StaticGraphRestoreNotSupported" xml:space="preserve">
<value>Static graph restore is not supported for file-based apps. Remove the '#:property'.</value>
<comment>{Locked="#:property"}</comment>
</data>
<data name="DirectiveError" xml:space="preserve">
<value>error</value>
<comment>Used when reporting directive errors like "file(location): error: message".</comment>
</data>
<data name="InvalidDirectiveName" xml:space="preserve">
<value>The directive should contain a name without special characters and an optional value separated by '{1}' like '#:{0} Name{1}Value'.</value>
<comment>{0} is the directive type like 'package' or 'sdk'. {1} is the expected separator like '@' or '='.</comment>
</data>
<data name="CannotConvertDirective" xml:space="preserve">
<value>Some directives cannot be converted. Run the file to see all compilation errors. Specify '--force' to convert anyway.</value>
<comment>{Locked="--force"}</comment>
</data>
<data name="DuplicateDirective" xml:space="preserve">
<value>Duplicate directives are not supported: {0}</value>
<comment>{0} is the directive type and name.</comment>
</data>
<data name="QuoteInDirective" xml:space="preserve">
<value>Directives currently cannot contain double quotes (").</value>
</data>
<data name="InvalidProjectDirective" xml:space="preserve">
<value>The '#:project' directive is invalid: {0}</value>
<comment>{0} is the inner error message.</comment>
</data>
<data name="MissingDirectiveName" xml:space="preserve">
<value>Missing name of '{0}'.</value>
<comment>{0} is the directive name like 'package' or 'sdk'.</comment>
</data>
<data name="UnrecognizedDirective" xml:space="preserve">
<value>Unrecognized directive '{0}'.</value>
<comment>{0} is the directive name like 'package' or 'sdk'.</comment>
</data>

@jjonescz
Copy link
Member

to actually embed the original .resx/.xlf as items and have the consuming project compile them.

I would do this and on the consuming side something like this could work?

<PackageReference Include="microsoft.dotnet.filebasedprograms" GeneratePathProperty="true" />

<EmbeddedResource Include="$(PkgMicrosoft_DotNet_FileBasedPrograms)contentfiles\cs\netstandard2.0\FileBasedProgramResources.resx" />

@RikkiGibson
Copy link
Member Author

I was able to get things working on the Roslyn side with @jjonescz's suggestion. Hopefully there is a simpler way to define ExcludeGeneratedResxFromPackage, though.


## Usage in Consuming Projects

To use this package in your project, add the following to your `.csproj` file:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the package contain a build/something.targets file (like for example our microsoft.net.compilers.toolset package has) that would be automatically included and would contain the EmbeddedResource item so users don't need to specify it themselves?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to do this, but, wasn't able to get it working. It was not clear to me what the conventions are with targets files in nuget packages. For example, are all files under build/*.targets imported automatically in the consuming project?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found docs for this: https://learn.microsoft.com/en-us/nuget/concepts/msbuild-props-and-targets

Looks like we'd want a buildMultiTargeting/Microsoft.DotNet.FileBasedPrograms.targets

<!-- Exclude the auto-generated .cs file from the resx from the package -->
<Target Name="ExcludeGeneratedResxFromPackage" BeforeTargets="GenerateNuspec">
<ItemGroup>
<_PackageFiles Remove="@(_PackageFiles)" Condition="$([System.String]::Copy('%(_PackageFiles.Identity)').EndsWith('FileBasedProgramsResources.cs'))" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where exactly is this generated file located originally? Somewhere in obj/bin folder?

Anyway, perhaps a simpler way to exclude it would be something like

<Compile Update="/path/to/the/file.cs" Pack="false" />

(where the path could be perhaps even something like **/FileBasedProgramsResources.cs)

(still might need to live in a Target though if the original item we are updating was defined in a Target as well)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where exactly is this generated file located originally? Somewhere in obj/bin folder?

With net9.0, for example: artifacts\obj\Microsoft.CodeAnalysis.CSharp.Features\Debug\net9.0\Microsoft.CodeAnalysis.CSharp.CSharpAnalyzersResources.cs

Yes, exactly, it needs to be done in a target. I would like to try and simplify this, but, I didn't yet manage to do so successfully.

I believe a target exists related to GenerateNuspec, which looks for which items have Pack="false" and excludes them. So we would try running before that and excluding the file by name or by glob.

I still don't really get how to "debug" msbuild targets. For example, sometimes the target I define appears in grayed-out in the binlog viewer, and I don't know what it means. I imagine the target was skipped, but I don't know how to get info about why it was skipped.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, sometimes the target I define appears in grayed-out in the binlog viewer, and I don't know what it means. I imagine the target was skipped, but I don't know how to get info about why it was skipped.

If the target is skipped, there should be an explanation (see screenshot), otherwise I guess the greyed-out target is just empty or something.

image

@RikkiGibson RikkiGibson marked this pull request as ready for review October 27, 2025 23:28
@RikkiGibson RikkiGibson requested a review from a team as a code owner October 27, 2025 23:28
@RikkiGibson RikkiGibson requested a review from jjonescz October 29, 2025 04:41
Copy link
Member

@jjonescz jjonescz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, just small nits and questions before I think this can be merged

<resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<data name="CouldNotFindAnyProjectInDirectory" xml:space="preserve">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove these strings from the original resx to avoid duplication?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sorry, this should have been included in the original pure move commit.

Copy link
Member Author

@RikkiGibson RikkiGibson Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these are also used in non-file-based programs scenarios, such as CouldNotFindAnyProjectInDirectory, but perhaps changing those sites to use FileBasedProgramsResources isn't really a problem?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did 2 things to try and clean up this area:

  1. Introduced extensions for the CliStrings which were deleted, which redirect to FileBasedProgramsResources, to avoid some churn at use sites.
  2. Migrated the translated strings from the old CliStrings/CliCommandStrings xlf files, to the new FileBasedProgramsResources xlf files.

@RikkiGibson
Copy link
Member Author

Opened #51487 to track follow ups to this PR.

@RikkiGibson
Copy link
Member Author

To merge we will need 2nd compiler sign off and SDK team sign off right? @333fred @MiYanni please take a look when you are available.

/// <summary>
/// Span of the full line including the trailing line break.
/// </summary>
public required TextSpan Span { get; init; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand moving off of Range/Index above, but adding required here. Both need polyfills, why are we saying that we don't expect the consumer to have polyfilled System.Range, but do expect them to have polyfilled the required attributes?

Copy link
Member Author

@RikkiGibson RikkiGibson Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a result of an intermediate state where essentially all this code was pasted directly into the MS.CA.CSharp.Features project and made to compile under netstandard2.0. Then, when that code was brought back into the source package "scaffolding", the contracts package came along with it, adding a bunch of polyfills.

I'm restoring one of the usages of Range as it seems a bit better ergonomically.

Nevermind, the Contracts package provides a declaration of RequiredMemberAttribute, but doesn't declare the string.AsSpan(Range) extensions and similar. I will plan on leaving the current state as-is regarding required and Range/Index unless you have a strong feeling there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do have a strong feeling here. The source package effectively requires consumers to have polyfilled stuff; either it should state what it requires being polyfilled, and should be consistent about the effective TFM that should be polyfilled, or it should not depend on any polyfills being present.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a similar issue exists with the MS.CA.CSharp dependency. As currently written, if consuming project doesn't actually reference MS.CA.CSharp, then the source just won't compile.

It's not clear to me yet how straightforward this is to solve. The stopgap would be to document "you need to also add PackageReference to X, Y, and Z for things to work", but, it feels like we should be able to simply have references to the things we need and have the usual dependency management system work it out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a change to remove PrivateAssets from the dependencies which are needed for this package to compile at the consumption site, and verified that things work internally in the SDK and when consuming from Roslyn. I believe this addresses the concern re: polyfills.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't particularly, as we have still reverted the Range changes, but I'm not going to block the PR on it.

@RikkiGibson RikkiGibson requested a review from 333fred November 4, 2025 05:25
@RikkiGibson RikkiGibson merged commit 30287ee into dotnet:release/10.0.2xx Nov 4, 2025
26 checks passed
@RikkiGibson RikkiGibson deleted the fbp-source-package branch November 4, 2025 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants