Open up orchestrator experimentally by Evangelink · Pull Request #7409 · microsoft/testfx

Evangelink · 2026-02-14T15:43:14Z

Copilot

Pull request overview

This PR exposes the test host orchestrator functionality as a public experimental API to enable third-party extensions to implement custom orchestration strategies (e.g., distributing tests across processes/machines or implementing custom retry logic).

Changes:

Made ITestHostOrchestrator and ITestHostOrchestratorManager public interfaces with experimental attribute
Created IInternalTestHostOrchestratorManager to retain internal-only members while exposing core functionality publicly
Added TestHostOrchestrator property to ITestApplicationBuilder as an experimental public API
Simplified RetryExtensions to use the new public API instead of casting to internal types

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
ITestHostOrchestrator.cs	Made interface public and added XML documentation and experimental attribute
ITestHostOrchestratorManager.cs	Split into public interface with experimental attribute and internal interface for advanced functionality
TestHostOrchestratorManager.cs	Updated to implement the internal interface
ITestHostBuilder.cs	Updated to use internal interface type
TestHostBuilder.cs	Updated to use internal interface type
ITestApplicationBuilder.cs	Added public experimental property for TestHostOrchestrator
TestApplicationBuilder.cs	Implemented public experimental property for TestHostOrchestrator
PublicAPI.Unshipped.txt	Added public API entries for new orchestrator interfaces and property
RetryExtensions.cs	Simplified to use public API instead of internal casting

src/Platform/Microsoft.Testing.Extensions.Retry/RetryExtensions.cs

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/ITestHostOrchestratorManager.cs

src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.cs

src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt

src/Platform/Microsoft.Testing.Extensions.Retry/Resources/xlf/ExtensionResources.pl.xlf

src/Platform/Microsoft.Testing.Extensions.Retry/Resources/xlf/ExtensionResources.zh-Hans.xlf

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/ITestHostControllersExtension.cs

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/OrchestratorBackCompat.cs

Co-authored-by: Youssef Victor <youssefvictor00@gmail.com>

Copilot

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

src/Platform/Microsoft.Testing.Platform/TestHostOrchestrator/ITestHostExecutionOrchestrator.cs

Copilot

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.

Copilot · 2026-03-05T14:46:01Z

src/Platform/Microsoft.Testing.Platform/TestHostOrchestrator/OrchestratorBackCompat.cs

+// These types provide binary backward compatibility for extensions compiled
+// against older platform versions where the orchestrator manager type lived in the
+// Microsoft.Testing.Platform.Extensions.TestHostOrchestrator namespace.
+namespace Microsoft.Testing.Platform.Extensions.TestHostOrchestrator;
+
+internal interface ITestHostOrchestratorManager
+{
+    void AddTestHostOrchestrator(Func<IServiceProvider, ITestHostExecutionOrchestrator> factory);
+}


OrchestratorBackCompat claims to provide binary backward compatibility, but the back-compat ITestHostOrchestratorManager no longer matches the previously existing contract (it now accepts ITestHostExecutionOrchestrator and omits members like AddTestHostOrchestratorApplicationLifetime / BuildAsync). This will break load-time compatibility for older extension assemblies compiled against the old internal orchestrator APIs (e.g., older Retry versions) and can cause TypeLoadException/MissingMethodException. Consider reintroducing shims for the old ITestHostOrchestrator/manager shapes (old signatures + members) that forward to the new public APIs, and have the new manager implement/adapt them explicitly.

Fixed. Added the missing ITestHostOrchestrator back-compat shim type (extends ITestHostExecutionOrchestrator) and updated the back-compat ITestHostOrchestratorManager to match the old contract:

AddTestHostOrchestrator now accepts Func<IServiceProvider, ITestHostOrchestrator> (old type name)

Added AddTestHostOrchestratorApplicationLifetime member

The concrete TestHostOrchestratorManager explicitly implements both old signatures, forwarding to the new public APIs

BuildAsync is not included in the back-compat interface since it took ServiceProvider (platform-internal) and was only called by internal platform code (which has been updated).

Copilot · 2026-03-05T14:46:02Z

src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt

 Microsoft.Testing.Platform.Extensions.TestHostControllers.ITestHostProcessLifetimeHandler.OnTestHostProcessStartedAsync(Microsoft.Testing.Platform.Extensions.TestHostControllers.ITestHostProcessInformation! testHostProcessInformation, System.Threading.CancellationToken cancellationToken) -> System.Threading.Tasks.Task!
+[TPEXP]Microsoft.Testing.Platform.Extensions.TestHostOrchestrator.ITestHostExecutionOrchestrator
+[TPEXP]Microsoft.Testing.Platform.Extensions.TestHostOrchestrator.ITestHostExecutionOrchestrator.OrchestrateTestHostExecutionAsync(System.Threading.CancellationToken cancellationToken) -> System.Threading.Tasks.Task<int>!
+Microsoft.Testing.Platform.Builder.ITestApplicationBuilder.TestHostOrchestrator.get -> Microsoft.Testing.Platform.TestHostOrchestrator.ITestHostOrchestratorManager!


ITestApplicationBuilder.TestHostOrchestrator is marked [Experimental("TPEXP", ...)], but its entry in PublicAPI.Unshipped.txt is missing the [TPEXP] prefix. This can cause API compatibility validation to treat it as a non-experimental shipped API; prefix it consistently with the other experimental entries.

Suggested change

Microsoft.Testing.Platform.Builder.ITestApplicationBuilder.TestHostOrchestrator.get -> Microsoft.Testing.Platform.TestHostOrchestrator.ITestHostOrchestratorManager!

[TPEXP]Microsoft.Testing.Platform.Builder.ITestApplicationBuilder.TestHostOrchestrator.get -> Microsoft.Testing.Platform.TestHostOrchestrator.ITestHostOrchestratorManager!

Checked existing conventions in PublicAPI.Shipped.txt — the other [Experimental] properties on ITestApplicationBuilder (Configuration.get, Logging.get) are not prefixed with [TPEXP] in the API files. Only the experimental types/methods themselves carry the prefix. Keeping the entry without [TPEXP] to stay consistent (the build also fails RS0016/RS0017 validation with the prefix added).

Open up orchestrator experimentally

0042781

Evangelink requested a review from Copilot February 14, 2026 15:43

Copilot started reviewing on behalf of Evangelink February 14, 2026 15:43 View session

Copilot AI reviewed Feb 14, 2026

View reviewed changes

Fix

7aee1d7

Youssef1313 reviewed Feb 15, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Extensions.Retry/RetryExtensions.cs Show resolved Hide resolved

Youssef1313 reviewed Feb 15, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/ITestHostOrchestratorManager.cs Outdated Show resolved Hide resolved

Youssef1313 reviewed Feb 15, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/ITestHostOrchestratorManager.cs Outdated Show resolved Hide resolved

Youssef1313 reviewed Feb 15, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/ITestHostOrchestratorManager.cs Outdated Show resolved Hide resolved

Address review comments

33d4cc2

Youssef1313 reviewed Feb 16, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.cs Outdated Show resolved Hide resolved

Youssef1313 reviewed Feb 16, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt Outdated Show resolved Hide resolved

Evangelink added 2 commits February 17, 2026 20:30

Address review comments

d503956

Fix compat issues

04aef20

Youssef1313 reviewed Feb 20, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Extensions.Retry/Resources/xlf/ExtensionResources.pl.xlf Outdated Show resolved Hide resolved

Youssef1313 reviewed Feb 20, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Extensions.Retry/Resources/xlf/ExtensionResources.zh-Hans.xlf Outdated Show resolved Hide resolved

Youssef1313 reviewed Feb 20, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/ITestHostControllersExtension.cs Outdated Show resolved Hide resolved

Youssef1313 reviewed Feb 20, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/TestHostOrcherstrator/OrchestratorBackCompat.cs Outdated Show resolved Hide resolved

Apply suggestions from code review

49d71a9

Co-authored-by: Youssef Victor <youssefvictor00@gmail.com>

Copilot AI review requested due to automatic review settings February 20, 2026 11:39

Copilot started reviewing on behalf of Evangelink February 20, 2026 11:40 View session

Copilot AI reviewed Feb 20, 2026

View reviewed changes

Fixes

aeaa25b

Youssef1313 reviewed Mar 5, 2026

View reviewed changes

src/Platform/Microsoft.Testing.Platform/TestHostOrchestrator/ITestHostExecutionOrchestrator.cs Show resolved Hide resolved

Youssef1313 approved these changes Mar 5, 2026

View reviewed changes

Rename + fix typo on folder

c36e2de

Copilot AI review requested due to automatic review settings March 5, 2026 14:37

Copilot started reviewing on behalf of Evangelink March 5, 2026 14:37 View session

Evangelink enabled auto-merge March 5, 2026 14:43

Copilot AI reviewed Mar 5, 2026

View reviewed changes

Fix backcompat

f4d3236

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Open up orchestrator experimentally#7409

Open up orchestrator experimentally#7409
Evangelink wants to merge 9 commits intomainfrom
dev/amauryleve/orchestrator

Evangelink commented Feb 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

Evangelink Mar 5, 2026

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

Evangelink Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	Microsoft.Testing.Platform.Builder.ITestApplicationBuilder.TestHostOrchestrator.get -> Microsoft.Testing.Platform.TestHostOrchestrator.ITestHostOrchestratorManager!
	[TPEXP]Microsoft.Testing.Platform.Builder.ITestApplicationBuilder.TestHostOrchestrator.get -> Microsoft.Testing.Platform.TestHostOrchestrator.ITestHostOrchestratorManager!

Conversation

Evangelink commented Feb 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Evangelink Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Evangelink Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants