Implement support for Privacy Manifests

[rolfbjarne]

We should look into what we need to do with regards to Privacy Manifests: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api

[takeshik]

Are there any updates of this issue, or is there any action in another repository to address this issue?

Apple writes:

From Spring 2024, apps that don’t describe their use of required reason API in their privacy manifest file won’t be accepted by App Store Connect.

If things go as planned by Apple, .NET apps on iOS might be at risk of being rejected for publication or updates by the store this spring. (though Unity seems to be dealing with it in their own way)

We are seeking some workaround which can be done only by ourselves, without framework support, but it might be impossible.
It would be best if the framework could address this issue, before the store's new regulation applies.

[dalexsoto]

Here is an example of a Framework that contains the file PrivacyInfo.xcprivacy which is just a plist dictionary with the described contents of https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api?language=objc

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>NSPrivacyAccessedAPITypes</key>
	<array>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategorySystemBootTime</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>35F9.1</string>
			</array>
		</dict>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategoryFileTimestamp</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>C617.1</string>
				<string>3B52.1</string>
				<string>0A2A.1</string>
			</array>
		</dict>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategoryDiskSpace</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>E174.1</string>
			</array>
		</dict>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategoryUserDefaults</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>CA92.1</string>
			</array>
		</dict>
	</array>
	<key>NSPrivacyTracking</key>
	<false/>
</dict>
</plist>

[rolfbjarne]

If we wanted to implement this well, here's an idea:

We add a new attribute:

[AttributeUsage (AttributeTargets.Assembly | AttributeTargets.Method | AttributeTargets.Field | AttributeTargets.Property | AttributeTargets.Constructor | AttributeTargets.Class, AllowMultiple = true)]
public class PrivacyManifestRequirementAttribute : System.Attribute {
	// The value for the NSPrivacyAccessedAPIType key in the privacy manifest
	// For instance "NSPrivacyAccessedAPICategoryFileTimestamp" for file stamp APIs.
	public string AccessedApiType { get; set; }
	// A description?
	public string? Message { get; set; }
	// A url to a longer description?
	public string? Url { get; set; }

	public PrivacyManifestRequirementAttribute (string accessedApiType)
	{
		AccessedApiType = accessedApiType;
	}
}

Then we decorate every API that triggers the privacy manifest requirement with this attribute.

Note that there are APIs in System.Private.CoreLib.dll that triggers the privacy manifest requirement, so this attribute would probably have to be added to System.Private.CoreLib.dll.

If we use such an API internally (either in the runtime or our own code, and the developer can't not use it in any way), we can add the attribute to the assembly in question (either System.Private.CoreLib.dll or Microsoft.iOS.dll).

And finally we handle this attribute in a way similar to how the linker handles trimmability attributes such as the RequiresUnreferencedCodeAttribute: any method that calls another method with this attribute must also have the equivalent attribute, otherwise a warning is raised. For executable projects, the warning can be silenced by adding a privacy manifest to the project with the correct privacy information.

The downside is that this is a significant amount of work upfront, but it pays off once done: there's no manual validation needed once the initial set of APIs have been annotated, the compiler/code analyzers will yell if the attribute isn't forwarded where it needs to be. If Apple adds more APIs to their list of special APIs, the process is very straight forward on our side: we annotate the corresponding APIs, and then, once again, we'll get warnings everywhere we need to forward the attribute.

If we decided to look into this, we should talk with the linker team for any input from them.

[ivanpovazan]

If we wanted to implement this well, here's an idea:

This sounds like a good and extensible approach and I think we should definitely aim our implementation in that direction.

However, as you rightfully pointed out, I also feel it would take a lot of time/work before we get there. Especially if we consider, acceptance on the dotnet/runtime side, integration with the linker, Mono/NativeAOT toolchain, the build system etc. For these reasons, I think this sounds like a solid long-term plan.

Since Apple announced that the privacy manifests will be checked starting on Spring 2024, I think we should focus on a short-term/backup plan which would be a start-in-the-right-direction, but at the same time make sure that our SDKs/apps are compliant in the timely manner.

Therefore, I think as one of the first things to agree upon (which we can utilize in the future) is to define a format which Xamarin bundler would use when producing the xcprivacy files.

For example, one idea for the short-term solution would be that we (on dotnet/runtime side) manually scan and flag all sensitive API's referenced by our frameworks and runtimes (yes, for now this would be a manual process and it would include all our API's even those not referenced by the project). We create a .targets file with a static ItemGroup in an agreed format, which we can deploy in our runtime packs that will be consumed during the build (like Xamarin SDK imports Microsoft.NET.Runtime.MonoTargets.Sdk targets through the workloads). Xamarin SDK and the bundler can then extend this ItemGroup with Xamarin-internal flagged API's and use it in the bundler to produce the required xcprivacy file.

The static list of privacy accessed APIs and the MSBuild target could be something like this:

  <Target Name="CollectPrivacyRequirements" Condition="'$(_IsPublishing)' == 'true'" AfterTargets="<TBD>">
    <ItemGroup>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>0A2A.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used by C# timestamp APIs</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>C617.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used to check files</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        ...
    </ItemGroup>
  </Target>

NOTE: As an example I used some information Unity includes in their documentation.

This way our users (and other frameworks MAUI?) could also plug-in and extend the ItemGroup for their purposes.

Finally, as a follow-up we would replace this manual process with something more sophisticated and automatic that is compatible with our toolchains similar to what you proposed, but with the same "back-end" that produces the xcprivacy files.

[vitek-karas]

I absolutely agree with Ivan that it seems we need a short-term solution which is reasonably easy to implement.
I also like the attributes as a longer term solution. The trimmer/analyzer/aot might gain a capability to represent the attribute "soon". The design for the more generalized feature attributes seems to allow defining an attribute very similar to this. dotnet/designs#305. I know the Sven is already working on an implementation of this for the analyzer.

[vitek-karas]

Actually, the analyzer change has already been merged: dotnet/runtime#94944

[dalexsoto]

If we wanted to implement this well, here's an idea:

This sounds like a good and extensible approach and I think we should definitely aim our implementation in that direction.

However, as you rightfully pointed out, I also feel it would take a lot of time/work before we get there. Especially if we consider, acceptance on the dotnet/runtime side, integration with the linker, Mono/NativeAOT toolchain, the build system etc. For these reasons, I think this sounds like a solid long-term plan.

Since Apple announced that the privacy manifests will be checked starting on Spring 2024, I think we should focus on a short-term/backup plan which would be a start-in-the-right-direction, but at the same time make sure that our SDKs/apps are compliant in the timely manner.

Therefore, I think as one of the first things to agree upon (which we can utilize in the future) is to define a format which Xamarin bundler would use when producing the xcprivacy files.

For example, one idea for the short-term solution would be that we (on dotnet/runtime side) manually scan and flag all sensitive API's referenced by our frameworks and runtimes (yes, for now this would be a manual process and it would include all our API's even those not referenced by the project). We create a .targets file with a static ItemGroup in an agreed format, which we can deploy in our runtime packs that will be consumed during the build (like Xamarin SDK imports Microsoft.NET.Runtime.MonoTargets.Sdk targets through the workloads). Xamarin SDK and the bundler can then extend this ItemGroup with Xamarin-internal flagged API's and use it in the bundler to produce the required xcprivacy file.

The static list of privacy accessed APIs and the MSBuild target could be something like this:

  <Target Name="CollectPrivacyRequirements" Condition="'$(_IsPublishing)' == 'true'" AfterTargets="<TBD>">
    <ItemGroup>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>0A2A.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used by C# timestamp APIs</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>C617.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used to check files</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        ...
    </ItemGroup>
  </Target>

NOTE: As an example I used some information Unity includes in their documentation.

This way our users (and other frameworks MAUI?) could also plug-in and extend the ItemGroup for their purposes.

Finally, as a follow-up we would replace this manual process with something more sophisticated and automatic that is compatible with our toolchains similar to what you proposed, but with the same "back-end" that produces the xcprivacy files.

What happens if the linker removes all of the usages of said APIs and there is no need for a privacy manifest? Could we make this more opt-in? For example document all the runtime/macios/MAUI usages and then let the final user fill in the above proposed item groups on their own so we can generate the main bundled privacy manifest.

Then once we have the more advanced annotations we can have the analyzer or linker step to produce this automatically in the future.

I absolutely agree with Ivan that it seems we need a short-term solution which is reasonably easy to implement. I also like the attributes as a longer term solution. The trimmer/analyzer/aot might gain a capability to represent the attribute "soon". The design for the more generalized feature attributes seems to allow defining an attribute very similar to this. dotnet/designs#305. I know the Sven is already working on an implementation of this for the analyzer.

I agree, we need a short term solution now if the long term one is not ready to be implemented on the runtime side.

[vitek-karas]

What happens if the linker removes all of the usages of said APIs and there is no need for a privacy manifest? Could we make this more opt-in? For example document all the runtime/macios/MAUI usages and then let the final user fill in the above proposed item groups on their own so we can generate the main bundled privacy manifest.

I think without some more involved changes (attributes, tooling support, ...) it will be hard to detect the need for this reliably. So I would probably go with the "Unity" approach - add the stuff which runtime may need always. If developers don't want it for some reason, it's a single line of MSBuild to remove everything we put there by default anyway.

In fact for the first version of this, we might go with just some docs, where we provide the snippets for the runtime and the frameworks and so on, ant let it up to the developer to construct the final XML.

For the long term solution - just a question:

• Do we need to propagate the attributes?
• The usefulness of propagation comes from the ability to let the library/app developer know, that there's certain compatibility requirements for using certain API.
• So the question is - do we think developers should be aware of all APIs (.NET) which will add stuff to the privacy manifest - and are they informed enough to make the right call?
• Alternative would be to have the attribute but to not propagate it. We would simply annotate the low-level methods with it and use it generate the private manifest after trimming. Basically have the attribute as part of the internal tooling to generate the manifest.

[rolfbjarne]

• So the question is - do we think developers should be aware of all APIs (.NET) which will add stuff to the privacy manifest - and are they informed enough to make the right call?

It's the developers who have to create the privacy manifest. Only they know why they're using a particular API, and they have to explain the reason in the privacy manifest. All we can do is say "hey, if you use this managed API, then you're effectively using this other native API, which means you'll then you'll have to explain why in the privacy manifest".

At most we can create a privacy manifest with a placeholder reason - but the developers still have to review why they're using the APIs in question.

I agree, we need a short term solution now if the long term one is not ready to be implemented on the runtime side.

We might want to wait until Xcode 16 in June to see what else (if anything) Apple comes up with until we start looking at the long term solution, just in case they throw the whole thing out and do something completely different.

[vitek-karas]

Thanks a lot @rolfbjarne for explaining this to me.
Given that, the "propagated attribute" solution seems like a good idea. We might need one more capability from the analyzers/IDE/compilers - the attribute related diagnostics (warnings) should really only kick in for apps targeting iOS. I don't know if the current feature switch design has the capability to do this. But I also don't think it would be a difficult thing to add.

[rolfbjarne]

We might need one more capability from the analyzers/IDE/compilers - the attribute related diagnostics (warnings) should really only kick in for apps targeting iOS.

Couldn't that be accomplished by only adding the attribute in the implementation assembly for iOS? That way other platforms wouldn't even see the attribute.