xUnit test pass but getting "Catastrophic failure: System.ArgumentException" - .net-core

Question

When running test from command line (dotnet test) in a .net 5.0 project using xunit all tests seems to pass but the process crashes with the following message from the detailed verbosity of dotnet test:
Catastrophic failure: System.ArgumentException : There is at least one object in this array that cannot be serialized (Parameter 'array')
[xUnit.net 00:00:03.74] [FATAL ERROR] System.ArgumentException
[xUnit.net 00:00:03.74] System.ArgumentException : There is at least one object in this array that cannot be serialized (Parameter 'array')
[xUnit.net 00:00:03.74] Stack Trace:
[xUnit.net 00:00:03.74] C:\Dev\xunit\xunit\src\xunit.runner.utility\Extensions\MessageSinkMessageExtensions.cs(44,0): at MessageSinkMessageExtensions.Dispatch[TMessage](IMessageSinkMessage message, HashSet`1 messageTypes, MessageHandler`1 callback)
This just happened when running dotnet test from the command line, running the test from VisualStudio works.
I'm testing a dotnet 5 rest API using TestServer.
Any ideas what could be the cause?
Packages version used:
<PackageReference Include="Microsoft.AspNetCore.TestHost" Version="5.0.8" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.10.0" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
-- UPDATE --
I just realize that the error described here is happening when using an xUnit Theory that take the following example class as argument:
public record TruncatedString
{
public TruncatedString(string value)
{
Value = FormatTruncatedString(value);
}
protected string Value { get; }
protected static string FormatTruncatedString(string value)
{
return value.Substring(0,4);
}
public static implicit operator string(TruncatedString truncated) => truncated.Value;
public static implicit operator TruncatedString(string text) => new (text);
public override string ToString() => Value;
}
And is used in a xUnit Theory like this:
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("ABC")]
public async Task ThestWithTheErrorMessage(TruncatedString value)
{
// ...
// THIS TEST PRODUCE THE SERIALIZATION ERROR
// System.ArgumentException : There is at least one object in this array that cannot be serialized (Parameter 'array')
// Stack Trace:
// C:\Dev\xunit\xunit\src\xunit.runner.utility\Extensions\MessageSinkMessageExtensions.cs(39,0): at MessageSinkMessageExtensions.Dispatch[TMessage](IMessageSinkMessage message, HashSet`1 messageTypes, MessageHandler`1 callback)
}

Answer 1

I've run into the same issue. It seems that xUnit will not run the theory test if there's an implicit conversion happening.
Rewrite your test to be like:
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("ABC")]
public async Task Test(string input)
{
TruncatedString value = input;
// ...
}
This bug in xUnit is documented here: https://github.com/xunit/xunit/issues/1742

Answer 2

As a workaround for unit tests. I have added a support of IXunitSerializable to help XUnit to build the correct test case data.
public sealed record CountryCode(string Value) : IXunitSerializable
{
// It's required by IXunitSerializable
public CountryCode(): this(string.Empty) { }
public override string ToString()
{
return Value;
}
public static implicit operator CountryCode(string self)
{
return new CountryCode(self);
}
public static implicit operator string(CountryCode self)
{
return self.Value;
}
void IXunitSerializable.Serialize(IXunitSerializationInfo info)
{
info.AddValue(nameof(Value), Value, typeof(string));
}
void IXunitSerializable.Deserialize(IXunitSerializationInfo info)
{
throw new NotSupportedException("This should not be used.");
}
}