Improve VSTest CLI migration guidance

[Amaury Levé (Evangelink)]

MTP currently reports VSTest-specific --logger and --collect arguments as generic unknown options, leaving users without an actionable migration path. This change keeps those arguments invalid while explaining the canonical MTP replacement instead of introducing a permanent compatibility mapping API.

Summary

• Add value-aware guidance for TRX, console verbosity, Microsoft and XPlat code coverage, and blame arguments.
• Include required extension package hints where applicable and call out cases that are not one-to-one replacements.
• Preserve MTP's single-owner command-line option model and existing invalid-command-line behavior.
• Record RFC 015 as rejected while leaving room to reconsider a narrow temporary shim if migration data shows diagnostics are insufficient.
• Regenerate localized resource files and cover CLI and testconfig.json diagnostics with unit tests.

Testing

• Focused Microsoft.Testing.Platform.UnitTests project build
• CommandLineHandlerTests: 85 passed
• Direct executable check for --logger trx returning the replacement guidance and invalid-command-line exit code

Fixes: #7249

[github-actions]

Review Summary

Clean, well-structured change that adds value-aware migration diagnostics for VSTest CLI options (--logger, --collect) instead of introducing a permanent mapping API. The implementation is sound across all 22 review dimensions.

What looks good

• Design: Correctly preserves MTP's single-owner command-line model. The early-return pattern in TryAppendVSTestOptionGuidance prevents the VSTest guidance from mixing with the "did you mean?" / extension-package suggestions.
• Correctness: The arguments.Count == 1 guard gracefully degrades to generic guidance when no argument or multiple arguments are present. The three accepted --output verbosity values (minimal, normal, detailed) exactly match the registered TerminalTestReporterCommandLineOptionsProvider constants.
• Localization: 新建 resource strings follow the repo conventions — {Locked} markers correctly protect option names and product terms. XLF files are properly regenerated.
• Test coverage: Tests cover all major branches (TRX with/without filename, console verbosity, Code Coverage, XPlat Coverage, blame, generic fallback) plus the testconfig.json path. Tests verify both the VSTest diagnostic banner and the extension-package suggestions.
• RFC / glossary updates: RFC 015 is marked rejected with a clear decision rationale, and the glossary entries are updated to match.

Minor observations (non-blocking)

1. Unrecognized console verbosity values (e.g. quiet, diagnostic) fall through to the generic --logger guidance rather than mentioning that those specific levels have no MTP equivalent. This is an acceptable design choice but could be made slightly more actionable in a follow-up.
2. Multi-argument edge case: --logger trx results.trx (two separate arguments instead of trx;LogFileName=results.trx) would bypass the TRX-specific guidance because arguments.Count != 1. This matches how VSTest parses it (as a single semicolon-delimited value), so users are unlikely to encounter this.

No bugs, no security issues, no public API changes, no breaking behavior. LGTM.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3

新建议题s introduced by this change (3)

Severity	Finding
	src/​Platform/​Microsoft.Testing.Platform/​CommandLine/​CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs — Pass validOptionNames into the VSTest-guidance path and gate each package hint on the replacement…
	src/​Platform/​Microsoft.Testing.Platform/​CommandLine/​CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs — Recognize configured Microsoft collector values rather than only the bare friendly name. A common…
	src/​Platform/​Microsoft.Testing.Platform/​CommandLine/​CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs — Treat XPlat Code Coverage;... as the same recognized collector shape. Coverlet values can include…

What changed in this PR

Improves diagnostics for unsupported VSTest CLI options while preserving MTP’s command-line model.

Changes:

• Adds value-aware migration guidance for --logger and --collect.
• Adds unit coverage for CLI and testconfig.json diagnostics.
• Rejects RFC 015 and updates documentation and localization resources.

File	Description
CommandLineHandlerTests.cs	Tests migration diagnostics.
PlatformResources.cs.xlf	Adds Czech localization entries.
PlatformResources.de.xlf	Adds German localization entries.
PlatformResources.es.xlf	Adds Spanish localization entries.
PlatformResources.fr.xlf	Adds French localization entries.
PlatformResources.it.xlf	Adds Italian localization entries.
PlatformResources.ja.xlf	Adds Japanese localization entries.
PlatformResources.ko.xlf	Adds Korean localization entries.
PlatformResources.pl.xlf	Adds Polish localization entries.
PlatformResources.pt-BR.xlf	Adds Brazilian Portuguese entries.
PlatformResources.ru.xlf	Adds Russian localization entries.
PlatformResources.tr.xlf	Adds Turkish localization entries.
PlatformResources.zh-Hans.xlf	Adds Simplified Chinese entries.
PlatformResources.zh-Hant.xlf	Adds Traditional Chinese entries.
PlatformResources.resx	Defines new diagnostic messages.
CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs	Generates value-aware migration guidance.
015-Command-Line-Option-Mappings.md	Records RFC rejection and rationale.
glossary.md	Updates mapping and coverage terminology.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10864

Only one test file changed: test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineHandlerTests.cs, adding 5 new data-driven [TestMethod]s that cover the new VSTest-option migration guidance in CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs (TryAppendVSTestOptionGuidance, AppendVSTestLoggerGuidance, AppendVSTestCollectGuidance, and the missing-extension suggestion path).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOption_ SuggestsMTPReplacement	4/4 killed	10 DataRow cases assert exact substrings matching each PlatformResources guidance branch (trx, trx+filename, console verbosity, custom logger, Code Coverage, +Format, XPlat, blame, custom collect).	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionWithoutArgument_ ProvidesGeneralGuidance	2/2 killed	Correctly verifies the no-argument fallback path returns the general reporter/collector guidance for both logger and collect.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOption_ SuggestsRequiredExtension	3/3 killed	Asserts the exact "Option '{replacement}' is provided by the '{packageName}' extension" message for every replacement/package pairing, including format variants.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionWithRegisteredReplacement_ DoesNotSuggestPackage	2/2 killed	Registers the replacement option via a mock provider and confirms the missing-extension message is correctly suppressed once it's already valid.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionInTestConfig_ SuggestsMTPReplacement	2/2 killed	Exercises the testconfig.json entry path, verifying both the JSON-prefix wrapper message and the inner VSTest replacement guidance are present.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 89.8 AIC · ⌖ 1.13 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

新建议题s introduced by this change (1)

Severity	Finding
	src/​Platform/​Microsoft.Testing.Platform/​Resources/​PlatformResources.resx — The lock covers only the opening fragment ('--output) instead of the complete command invocation.…

问题 resolved since last review (3)

Severity	Finding
	src/​Platform/​Microsoft.Testing.Platform/​CommandLine/​CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs — Treat XPlat Code Coverage;... as the same recognized collector shape. Coverlet values can include… View resolved comment
	src/​Platform/​Microsoft.Testing.Platform/​CommandLine/​CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs — Recognize configured Microsoft collector values rather than only the bare friendly name. A common… View resolved comment
	src/​Platform/​Microsoft.Testing.Platform/​CommandLine/​CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs — Pass validOptionNames into the VSTest-guidance path and gate each package hint on the replacement… View resolved comment

[github-actions]

🧪 Expert test review — PR #10864

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionWithRegisteredReplacement_ DoesNotSuggestPackage	4/5 killed	For the two "blame" rows, only the registered replacement's suppression is checked; the other collector's (crashdump/hangdump) suggestion still firing is never verified.	For the "blame" rows also assert the un-registered sibling suggestion (crashdump or hangdump) is still present via Assert.Contains.
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOption_ SuggestsMTPReplacement	10/10 killed	DataRows exercise every value-matching branch (trx, trx+LogFileName, console verbosity, Code Coverage(+Format), XPlat variants, blame, custom fallback).	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionWithoutArgument_ ProvidesGeneralGuidance	2/2 killed	Covers the zero-argument (null value) fallback path for both logger and collect.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOption_ SuggestsRequiredExtension	7/7 killed	Verifies the missing-extension suggestion text and package name for each VSTest replacement mapping.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionInTestConfig_ SuggestsMTPReplacement	2/2 killed	Confirms the testconfig.json wiring prefixes the error and still surfaces the VSTest replacement guidance.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 67.5 AIC · ⌖ 1.16 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 67.5 AIC · ⌖ 1.16 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

问题 resolved since last review (1)

Severity	Finding
	src/​Platform/​Microsoft.Testing.Platform/​Resources/​PlatformResources.resx — The lock covers only the opening fragment ('--output) instead of the complete command invocation.… View resolved comment

[github-actions]

🧪 Expert test review — PR #10864

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionWithRegisteredReplacement_ DoesNotSuggestPackage	4/5 killed	Only asserts the suppressed hint is absent; a mutant that drops all guidance text would still pass.	Also assert the positive VSTest replacement text (e.g. Use '--report-trx' instead.) is still present.
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestBlameWithOneRegisteredReplacement_ SuggestsOtherExtension	2/2 killed	Directly follow-up test verifying the sibling collector hint (crashdump/hangdump) survives when only one is registered.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOption_ SuggestsMTPReplacement	10/10 killed	Comprehensive DataRow matrix asserting both the VSTest-syntax header and the value-specific guidance for logger/collect.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOption_ SuggestsRequiredExtension	6/6 killed	Confirms the missing-package hint text and package name for each replacement across coverage/trx/crashdump/hangdump.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionInTestConfig_ SuggestsMTPReplacement	2/2 killed	Verifies the testconfig.json unknown-option path forwards to the same VSTest replacement guidance.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ VSTestOptionWithoutArgument_ ProvidesGeneralGuidance	2/2 killed	Covers the no-argument fallback path for both logger and collect with distinct expected guidance strings.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.5 AIC · ⌖ 1.13 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineHandlerTests.cs:662

• The value-aware console branch accepts minimal, normal, and detailed, but this table only exercises detailed. Removing either of the other two accepted values would leave the suite green while those users silently fall back to generic logger guidance. Add rows for both mappings.

    [DataRow("logger", "console;verbosity=detailed", "For comparable console verbosity, use '--output detailed'.")]

[github-actions]

🧵 Parallel-safety audit — PR #10864

Nothing audited here touches process-global state, shared filesystem paths, or [ResourceLock] / [DoNotParallelize] declarations. Nothing to flag for parallel-safety.

Scope reviewed: the only changed test file is test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineHandlerTests.cs, which adds 6 new [TestMethod]s (ParseAndValidateAsync_VSTestOption_SuggestsMTPReplacement, ..._VSTestOptionWithoutArgument_ProvidesGeneralGuidance, ..._VSTestOption_SuggestsRequiredExtension, ..._VSTestOptionWithRegisteredReplacement_DoesNotSuggestPackage, ..._VSTestBlameWithOneRegisteredReplacement_SuggestsOtherExtension, ..._VSTestOptionInTestConfig_SuggestsMTPReplacement) exercising CommandLineOptionsValidator.ValidateAsync against Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs.

• Each new test constructs its own CommandLineParseResult, local mocks (Mock.Of<ICommandLineOptions>()), and a local ExtensionCommandLineProviderMockValidConfiguration instance — no shared static, env var, CWD, console, or filesystem path is touched.
• The production code under test (CommandLineOptionsValidator) only reads from a static readonly Dictionary<string, string[]> (KnownExtensionOptionsByPackage, never mutated) and its own parameters — no process-global mutation.
• No changed lifecycle member ([TestInitialize]/[ClassInitialize]/ctor/Dispose), no changed [ResourceLock]/[DoNotParallelize]/[Parallelize] declaration, and no changed .runsettings/testconfig.json/.csproj parallelization config in this PR.

Audited Microsoft.Testing.Platform.UnitTests at scope MethodLevel (from [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in Program.cs, unchanged by this PR), workers CPU count.

_{Re-run with /parallel-audit.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 46.9 AIC · ⌖ 0.811 AIC · ⊞ 24.8K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧪 Expert test review — PR #10864

Reviewed the 6 new test methods added in test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineHandlerTests.cs (lines 659-816), covering the new VSTest --logger/--collect migration-guidance feature in CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs. Note: earlier review rounds on this PR already surfaced and the author already fixed several issues (unconditional package hints, Code Coverage;Format=... matching, XPlat Code Coverage;... matching, boolean-comparison style, and — specifically for tests — the missing "other extension still suggested" assertion for the blame collector, now covered by a dedicated test). My pass found no further high-confidence issues.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_VSTestOption_ SuggestsMTPReplacement	12/12 killed	12 data rows exercise every logger/collect guidance branch (trx, trx+filename, console verbosity x3, custom fallback, coverage, coverage+format, XPlat coverage x2, blame, custom fallback) with exact substring assertions.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_VSTestOptionWithoutArgument_ ProvidesGeneralGuidance	2/2 killed	Correctly isolates the "no single-argument value" path that falls back to generic reporter/collector guidance.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_VSTestOption_ SuggestsRequiredExtension	6/6 killed	Confirms each recognized replacement (report-trx, coverage, crashdump, hangdump) triggers a missing-package suggestion when unregistered.	—
A (90–100)	mod CommandLineHandlerTests. ParseAndValidateAsync_VSTestOptionWithRegisteredReplacement_ DoesNotSuggestPackage	5/5 killed	Verifies package hint suppression per replacement once its option is registered, for every recognized VSTest value shape.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_VSTestBlameWithOneRegisteredReplacement_ SuggestsOtherExtension	4/4 killed	Bidirectionally checks the blame collector's two-package case: registered sibling stays suppressed, unregistered sibling is still suggested.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_VSTestOptionInTestConfig_ SuggestsMTPReplacement	2/2 killed	Confirms the testconfig.json unknown-option path also routes through VSTest guidance and preserves the "In testconfig.json under..." prefix.	—

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 123.7 AIC · ⌖ 2 AIC · ⊞ 16.9K · [◷]( · ◷)