Xunit CSV streamReader.ReadToEnd returns System.ArgumentOutOfRangeException - css

I would like to evaluate a CSV data series with Xunit.
For this I need to read in a string consisting of int, bool, double and others.
With the following code, the transfer basically works for one row.
But since I want to test for predecessor values, I need a whole CSV file for evaluation.
My [Theory] works with InlineData without errors.
But when I read in a CSV file, the CSVDataHandler gives a System.ArgumentOutOfRangeException!
I can't find a solution for the error and ask for support.
Thanks a lot!
[Theory, CSVDataHandler(false, "C:\\MyTestData.txt", Skip = "")]
public void TestData(int[] newLine, int[] GetInt, bool[] GetBool)
{
for (int i = 0; i < newLine.Length; i++)
{
output.WriteLine("newLine {0}", newLine[i]);
output.WriteLine("GetInt {0}", GetInt[i]);
output.WriteLine("GetBool {0}", GetBool[i]);
}
}
[DataDiscoverer("Xunit.Sdk.DataDiscoverer", "xunit.core")]
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true, Inherited = true)]
public abstract class DataArribute : Attribute
{
public abstract IEnumerable<object> GetData(MethodInfo methodInfo);
public virtual string? Skip { get; set; }
}
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public class CSVDataHandler : DataAttribute
{
public CSVDataHandler(bool hasHeaders, string pathCSV)
{
this.hasHeaders = hasHeaders;
this.pathCSV = pathCSV;
}
public override IEnumerable<object[]> GetData(MethodInfo methodInfo)
{
var methodParameters = methodInfo.GetParameters();
var paramterTypes = methodParameters.Select(p => p.ParameterType).ToArray();
using (var streamReader = new StreamReader(pathCSV))
{
if (hasHeaders) { streamReader.ReadLine(); }
string csvLine = string.Empty;
// ReadLine ++
//while ((csvLine = streamReader.ReadLine()) != null)
//{
// var csvRow = csvLine.Split(',');
// yield return ConvertCsv((object[])csvRow, paramterTypes);
//}
// ReadToEnd ??
while ((csvLine = streamReader.ReadToEnd()) != null)
{
if (Environment.NewLine != null)
{
var csvRow = csvLine.Split(',');
yield return ConvertCsv((object[])csvRow, paramterTypes); // System.ArgumentOutOfRangeException
}
}
}
}
private static object[] ConvertCsv(IReadOnlyList<object> cswRow, IReadOnlyList<Type> parameterTypes)
{
var convertedObject = new object[parameterTypes.Count];
for (int i = 0; i < parameterTypes.Count; i++)
{
convertedObject[i] = (parameterTypes[i] == typeof(int)) ? Convert.ToInt32(cswRow[i]) : cswRow[i]; // System.ArgumentOutOfRangeException
convertedObject[i] = (parameterTypes[i] == typeof(double)) ? Convert.ToDouble(cswRow[i]) : cswRow[i];
convertedObject[i] = (parameterTypes[i] == typeof(bool)) ? Convert.ToBoolean(cswRow[i]) : cswRow[i];
}
return convertedObject;
}
}
MyTestData.txt
1,2,true,
2,3,false,
3,10,true,

The first call to streamReader.ReadToEnd() will return the entire contents of the file in a string, not just one line. When you call csvLine.Split(',') you will get an array of 12 elements.
The second call to streamReader.ReadToEnd() will not return null as your while statement appears to expect, but an empty string. See the docu at
https://learn.microsoft.com/en-us/dotnet/api/system.io.streamreader.readtoend?view=net-7.0
If the current position is at the end of the stream, returns an empty
string ("").
With the empty string, the call to call csvLine.Split(',') will return an array of length 0, which causes your exception when its first element (index 0) is accessed.
All of this could have been easily discovered by simply starting the test in a debugger.
It looks like you have some other issues here as well.
I don't understand what your if (Environment.NewLine != null) is intended to do, the NewLine property will never be null but should have one of the values "\r\n" or "\n" so the if will always be taken.
The parameters of your test method are arrays int[] and bool[], but you are checking against the types int, double and bool in your ConvertCsv method, so the alternative cswRow[i] will always be returned. You'll wind up passing strings to your method expecting int[] and bool[] and will at latest get an error there.

This method reads a data series from several rows and columns and returns it as an array for testing purposes.
The conversion of the columns can be adjusted according to existing pattern.
Thanks to Christopher!
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public class CSVDataHandler : Xunit.Sdk.DataAttribute
{
public CSVDataHandler(string pathCSV)
{
this.pathCSV = pathCSV;
}
public override IEnumerable<object[]> GetData(MethodInfo methodInfo)
{
List<int> newLine = new();
List<int> GetInt = new();
List<bool> GetBool = new();
var reader = new StreamReader(pathCSV);
string readData = string.Empty;
while ((readData = reader.ReadLine()) != null)
{
string[] split = readData.Split(new char[] { ',' });
newLine.Add(int.Parse(split[0]));
GetInt.Add(int.Parse(split[1]));
GetBool.Add(bool.Parse(split[2]));
// Add more objects ...
}
yield return new object[] { newLine.ToArray(), GetInt.ToArray(), GetBool.ToArray() };
}
}

Related

Json.NET: Converting JSON with $ element to XML

Json.NET seems to handle converting many names which would be invalid XML correctly, for example:
JsonConvert.DeserializeXmlNode("{"name!1": "test"}").OuterXml
Results in:
<name_x0021_1>test</name_x0021_1>
But attempting to convert the following, which passes JSONLint:
{"$": "test"}
Results in
Result Message: System.Xml.XmlException : The ':' character, hexadecimal value 0x3A, cannot be included in a name.
This error message in itself seems confusing as it suggests one of the names in the JSON has a : character. There might be a good reason for this but is there a way to get this to convert to XML without throwing an exception as some APIs seem to return the "$": "...." pair.
Update
This is fixed in Json.NET 9.0.1 in commit b71ca75.
Original Answer
This may be a bug in Json.NET's XmlNodeConverter.ReadElement(). Json.NET has several reserved property names, all of which begin with $, shown here:
public const string IdPropertyName = "$id";
public const string RefPropertyName = "$ref";
public const string TypePropertyName = "$type";
public const string ValuePropertyName = "$value";
public const string ArrayValuesPropertyName = "$values";
Then, in ReadElement(), there is special handling for property names that begin with a $:
private void ReadElement(JsonReader reader, IXmlDocument document, IXmlNode currentNode, string propertyName, XmlNamespaceManager manager)
{
if (string.IsNullOrEmpty(propertyName))
{
throw JsonSerializationException.Create(reader, "XmlNodeConverter cannot convert JSON with an empty property name to XML.");
}
Dictionary<string, string> attributeNameValues = ReadAttributeElements(reader, manager);
string elementPrefix = MiscellaneousUtils.GetPrefix(propertyName);
if (propertyName.StartsWith('#'))
{
string attributeName = propertyName.Substring(1);
string attributePrefix = MiscellaneousUtils.GetPrefix(attributeName);
AddAttribute(reader, document, currentNode, attributeName, manager, attributePrefix);
}
else if (propertyName.StartsWith('$'))
{
if (propertyName == JsonTypeReflector.ArrayValuesPropertyName)
{
propertyName = propertyName.Substring(1);
elementPrefix = manager.LookupPrefix(JsonNamespaceUri);
CreateElement(reader, document, currentNode, propertyName, manager, elementPrefix, attributeNameValues);
}
else
{
// Your code throws an exception going down this branch.
string attributeName = propertyName.Substring(1);
string attributePrefix = manager.LookupPrefix(JsonNamespaceUri);
AddAttribute(reader, document, currentNode, attributeName, manager, attributePrefix);
}
}
else
{
CreateElement(reader, document, currentNode, propertyName, manager, elementPrefix, attributeNameValues);
}
}
I suspect the propertyName.StartsWith('$') branch should only be taken for reserved property names, meaning the method should look like:
private void ReadElement(JsonReader reader, IXmlDocument document, IXmlNode currentNode, string propertyName, XmlNamespaceManager manager)
{
if (string.IsNullOrEmpty(propertyName))
{
throw JsonSerializationException.Create(reader, "XmlNodeConverter cannot convert JSON with an empty property name to XML.");
}
Dictionary<string, string> attributeNameValues = ReadAttributeElements(reader, manager);
if (propertyName.StartsWith('#'))
{
string attributeName = propertyName.Substring(1);
string attributePrefix = MiscellaneousUtils.GetPrefix(attributeName);
AddAttribute(reader, document, currentNode, attributeName, manager, attributePrefix);
}
else if (propertyName == JsonTypeReflector.ArrayValuesPropertyName)
{
propertyName = propertyName.Substring(1);
var elementPrefix = manager.LookupPrefix(JsonNamespaceUri);
CreateElement(reader, document, currentNode, propertyName, manager, elementPrefix, attributeNameValues);
}
else if (propertyName == JsonTypeReflector.IdPropertyName
|| propertyName == JsonTypeReflector.RefPropertyName
|| propertyName == JsonTypeReflector.TypePropertyName
|| propertyName == JsonTypeReflector.ValuePropertyName)
{
string attributeName = propertyName.Substring(1);
string attributePrefix = manager.LookupPrefix(JsonNamespaceUri);
AddAttribute(reader, document, currentNode, attributeName, manager, attributePrefix);
}
else
{
var elementPrefix = MiscellaneousUtils.GetPrefix(propertyName);
CreateElement(reader, document, currentNode, propertyName, manager, elementPrefix, attributeNameValues);
}
}
(Honestly, the handling of "$value" looks suspect also, since the value of a polymorphic property could be any JSON object.)
You might want to report an issue.
In the meantime, you could load your JSON into a JToken, manually remap names beginning with $, then convert to an XmlDocument like so:
var token = JToken.Parse(json);
token.RenameReplaceProperties(s => (s.StartsWith("$") && !JsonExtensions.IsReserved(s) ? XmlConvert.EncodeName(s) : s));
var xml = token.ToXmlNode();
Using the extension methods:
public static class JsonExtensions
{
const string IdPropertyName = "$id";
const string RefPropertyName = "$ref";
const string TypePropertyName = "$type";
const string ValuePropertyName = "$value";
const string ArrayValuesPropertyName = "$values";
public static bool IsReserved(string s)
{
return s == IdPropertyName || s == RefPropertyName || s == TypePropertyName || s == ValuePropertyName || s == ArrayValuesPropertyName;
}
public static IEnumerable<JToken> DescendantsAndSelf(this JToken root)
{
var container = root as JContainer;
if (container != null)
return container.DescendantsAndSelf();
else if (root != null)
return new[] { root };
else
return Enumerable.Empty<JToken>();
}
public static JProperty RenameReplace(this JProperty property, string name)
{
if (property == null || name == null)
throw new ArgumentNullException();
var value = property.Value;
property.Value = null;
var newProperty = new JProperty(name, value);
property.Replace(newProperty);
return newProperty;
}
public static JToken RenameReplaceProperties(this JToken root, Func<string, string> map)
{
var query = from property in root.DescendantsAndSelf().OfType<JProperty>()
let name = map(property.Name)
where name != property.Name && name != null
select new KeyValuePair<JProperty, string>(property, name);
foreach (var pair in query.ToList())
{
var newProperty = pair.Key.RenameReplace(pair.Value);
if (pair.Key == root)
root = newProperty;
}
return root;
}
public static XmlDocument ToXmlNode(this JToken root)
{
using (var reader = root.CreateReader())
return DeserializeXmlNode(reader);
}
public static XmlDocument DeserializeXmlNode(JsonReader reader)
{
return DeserializeXmlNode(reader, null, false);
}
public static XmlDocument DeserializeXmlNode(JsonReader reader, string deserializeRootElementName, bool writeArrayAttribute)
{
var converter = new Newtonsoft.Json.Converters.XmlNodeConverter() { DeserializeRootElementName = deserializeRootElementName, WriteArrayAttribute = writeArrayAttribute };
var jsonSerializer = JsonSerializer.CreateDefault(new JsonSerializerSettings { Converters = new JsonConverter[] { converter } });
return (XmlDocument)jsonSerializer.Deserialize(reader, typeof(XmlDocument));
}
}

Trouble with getting codes for Huffman Tree

I need to create the codes to compress a Huffman Tree. (O if you traverse left 1 if you traverse right.)
Right now the codes array will get the correct first code, but afterwards will only return null.
I am not allowed to use any instance/global variables or any predefined java classes besides array and arrayList
public static String compress(final BinaryNodeInterface<Character> root, final String message)
{
String[] codes;
ArrayList<Character> letter = new ArrayList<Character>();
String codeString = "";
boolean addChar=true;
for(int i =0; i<message.length();i++)
{
for(int j=0;j<letter.size();j++)
if(message.charAt(i)==letter.get(j))
addChar=false;
else
addChar=true;
if(addChar)
letter.add(message.charAt(i));
}
codes = new String[letter.size()];
for(int i=0;i<letter.size();i++)
codes[i]=(getPath(root,letter.get(i),""));
return ""; // Do not forget to change this line!!!
}
private static String getPath(final BinaryNodeInterface<Character> root, char toFind, String path)
{
String result;
if (! root.isLeaf()) {
if ((result = getPath(root.getLeftChild(),toFind, path + '0')) == null) {
result = getPath(root.getLeftChild(),toFind, path + '1');
}
}
else {
result = (toFind == root.getData()) ? path : null;
}
return result;
}
You don't seem to be visiting the right sub child. Change
result = getPath(root.getLeftChild(),toFind, path + '1');
to
result = getPath(root.getRightChild(),toFind, path + '1');

How to auto setup MOQ by given parameters and result in an object[]?

I want to automate the Setup code by a given array containing expected result and parameters.
Something like data driven setup.
My existing code:
var mock = new Mock<ILogin>();
var testDataTable = new object[,]
{
{ LoginResult.Success, "Jack", "123!##"}
, { LoginResult.WrongPassword, "Jack", "123321"}
, { LoginResult.NoSuchUser, "Peter", "123!##"}
};
// ForEachRow is my own extension method
testDataTable.ForEachRow((row) =>
{
var result = (LoginResult)row[0];
var username = (string)row[1];
var password = (string)row[2];
mock.Setup(o => o.Login(
It.Is<string>(u => u == username),
It.Is<string>(p => p == password)
)).Returns(result);
});
return mock.Object;
Code that I wish:
var mock = new Mock<ILogin>();
new object[,]
{
{ LoginResult.Success, "Jack", "123!##"}
, { LoginResult.WrongPassword, "Jack", "123321"}
, { LoginResult.NoSuchUser, "Peter", "123!##"}
}.ForEachRow((row) =>
{
var exprTree = (ILogin o)=>o.Login("ANY", "ANY");
AutoSetup(mock, exprTree, row); // <---- How to write this AutoSetup?
});
return mock.Object;
How to write the AutoSetup(mock, exprTree, dataArray) function above?
It takes three parameters:
mock: A mock object, e.g. new Mock()
exprTree: Expression tree that represents a method to be setup
dataArray: An object[], the 0 element is the expected result, and others are parameters that passed to the method
This was an interesting challenge. I think I have a working implementation for your AutoSetup method, using the expressions API. If anyone has a simpler solution, I'd love to see it.
static void AutoSetup<TMock, TResult>(Mock<TMock> mock, Expression<Func<TMock, TResult>> exprTree, object[] items) where TMock : class
{
var methodCallExpr = exprTree.Body as MethodCallExpression;
var arguments = items.Skip(1).Select(o => Expression.Constant(o));
var updatedMethod = methodCallExpr.Update(methodCallExpr.Object, arguments);
var updatedLambda = exprTree.Update(updatedMethod, exprTree.Parameters);
mock.Setup(updatedLambda).Returns((TResult)items[0]);
}
Here is a full working test as a console app, with an implementation of ForEachRow, which you didn't provide.
class Program
{
static void Main(string[] args)
{
var login = SetUp();
Console.WriteLine(login.Login("Jack", "123!##"));
Console.WriteLine(login.Login("Jack", "123321"));
Console.WriteLine(login.Login("Peter", "123!##"));
Console.ReadKey();
}
static ILogin SetUp()
{
var mock = new Mock<ILogin>(MockBehavior.Strict);
var rows = new object[,]
{
{ LoginResult.Success, "Jack", "123!##" },
{ LoginResult.WrongPassword, "Jack", "123321" },
{ LoginResult.NoSuchUser, "Peter", "123!##" }
};
rows.ForEachRow((row) => AutoSetup(mock, (ILogin l) => l.Login("ANY", "ANY"), row));
return mock.Object;
}
private static void AutoSetup<TMock, TResult>(Mock<TMock> mock, Expression<Func<TMock, TResult>> exprTree, object[] items) where TMock : class
{
var methodCallExpr = exprTree.Body as MethodCallExpression;
var arguments = items.Skip(1).Select(o => Expression.Constant(o));
var updatedMethod = methodCallExpr.Update(methodCallExpr.Object, arguments);
var updatedLambda = exprTree.Update(updatedMethod, exprTree.Parameters);
mock.Setup(updatedLambda).Returns((TResult)items[0]);
}
}
public static class ArrayExtensions
{
public static void ForEachRow<T>(this T[,] rows, Action<T[]> action)
{
var x = rows.GetLength(1);
var y = rows.GetLength(0);
for (int i = 0; i < y; i++)
{
var row = new T[x];
for (int j = 0; j < x; j++)
{
row[j] = rows[i, j];
}
action(row);
}
}
}
public interface ILogin
{
LoginResult Login(string p1, string p2);
}
public enum LoginResult
{
Success,
WrongPassword,
NoSuchUser
}
EDIT: You asked in a comment about how to take advantage of the variable parameter matching that Moq provides with the It.IsAny<> method. Because what you pass to your Mock.Setup() is an expression tree, it's able to scan the method parameters and implement special behaviour for any that are calls to It.IsAny<>. However, if you use It.IsAny<> in your test data array, by the time we retrieve it from your items array to set in on the expression, it's not a method call but simply the result of the call it It.IsAny<> which is default(T) (see here).
We need some way of specifying in your test data array that the parameter should be any. Then we can check for this special case and generate the correct MethodCallExpression representing a call to It.IsAny<>.
Here are the changes I made to support this:
Add an Any type to use in your test data
public class Any<T>
{
private Any() { }
public static Any<T> Param { get { return new Any<T>(); } }
}
Update the AutoSetup method to handle this special case:
private static void AutoSetup<TMock, TResult>(Mock<TMock> mock, Expression<Func<TMock, TResult>> exprTree, object[] items) where TMock : class
{
var arguments = items.Skip(1).Select(o => {
var type = o.GetType();
if (type.IsGenericType && type.GetGenericTypeDefinition() == typeof(Any<>))
{
var typeParameter = type.GetGenericArguments();
var genericItIsAny = typeof(It).GetMethod("IsAny");
var itIsAny = genericItIsAny.MakeGenericMethod(typeParameter);
return Expression.Call(itIsAny) as Expression;
}
return Expression.Constant(o);
});
var methodCallExpr = exprTree.Body as MethodCallExpression;
var updatedMethod = methodCallExpr.Update(methodCallExpr.Object, arguments);
var updatedLambda = exprTree.Update(updatedMethod, exprTree.Parameters);
mock.Setup(updatedLambda).Returns((TResult)items[0]);
}
In the test data, use the Any type
{ LoginResult.Success, "NoPasswordUser", Any<string>.Param }
Depending on how you're storing your test data, you might need another way of identifying parameters that you want to be variable (specially formatted string?), but in general this should give you an idea about how to create the correct expressions.

Generic Object to '|'-separated string serialization

I have written this class-methods for .net 2.0 to create objects from '|'-separated strings and vise-versa.
But the problem is, they are not giving right results in case of Inherted types, i.e. inherited properties are coming last and the sequence of the data supplied in the form of a '|'-separated string is not working.
For example:
class A
{
int ID;
}
class B : A
{
string Name;
}
the string is "1|John". the methods are reading as the name==1 and ID=="John".
Please tell me how to do it.
public class ObjectConverter<T>
{
public static T TextToObject(string text)
{
T obj = Activator.CreateInstance<T>();
string[] data = text.Split('|');
PropertyInfo[] props = typeof(T).GetProperties();
int objectPropertiesLength = props.Length;
int i = 0;
if (data.Length == objectPropertiesLength)
{
for (i = 0; i < objectPropertiesLength; i++)
{
props[i].SetValue(obj, data[i], null);
}
}
return obj;
}
public static string ObjectToText(T obj)
{
StringBuilder sb = new StringBuilder();
Type t = typeof(T);
PropertyInfo[] props = t.GetProperties();
int i = 0;
foreach (PropertyInfo pi in props)
{
object obj2 = props[i++].GetValue(obj, null);
sb.Append(obj2.ToString() + "|");
}
sb = sb.Remove(sb.Length - 1, 1);
return sb.ToString();
}
}
I don't think the runtime assures you that when you call getproperties the property info objects will always be in the same order. You will need to do something like get the list of property names sort them and use the same sorting for serialization and deserialization .
There are at least 3 ways to serialize object built into .net is there a reason why you are not using one of those

How can I get parameters values from a lamba expression for my nifty cache extension?

First of all it might be worth looking at this question:
How can I cache objects in ASP.NET MVC?
There some pseudo code that almost does what i want:
public class CacheExtensions
{
public static T GetOrStore<T>(this Cache cache, string key, Func<T> generator)
{
var result = cache[key];
if(result == null)
{
result = generator();
cache[key] = result;
}
return (T)result;
}
}
However, what I'd really like to do, is auto-generate the "key" from the generator. I figure i need to change the method signature to:
public static T GetOrStore<T>(this Cache cache,
System.Linq.Expressions.Expression<Func<T>> generator)
I want to use the method name, but also any parameters and their values to generate the key. I can get the method body from the expression, and the paramter names (sort of), but I have no idea how to get the paramter values...?
Or am I going about this the wrong way? Any ideas much appreciated.
Here's how I did it:
public static class ICacheExtensions
{
public static T GetOrAdd<T>(this ICache cache, Expression<Func<T>> getterExp)
{
var key = BuildCacheKey<T>(getterExp);
return cache.GetOrAdd(key, () => getterExp.Compile().Invoke());
}
private static string BuildCacheKey<T>(Expression<Func<T>> getterExp)
{
var body = getterExp.Body;
var methodCall = body as MethodCallExpression;
if (methodCall == null)
{
throw new NotSupportedException("The getterExp must be a MethodCallExpression");
}
var typeName = methodCall.Method.DeclaringType.FullName;
var methodName = methodCall.Method.Name;
var arguments = methodCall.Arguments
.Select(a => ExpressionHelper.Evaluate(a))
.ToArray();
return String.Format("{0}_{1}_{2}",
typeName,
methodName,
String.Join("|", arguments));
}
}
with this helper to evaluate nodes of an expression tree:
internal static class ExpressionHelper
{
public static object Evaluate(Expression e)
{
Type type = e.Type;
if (e.NodeType == ExpressionType.Convert)
{
var u = (UnaryExpression)e;
if (TypeHelper.GetNonNullableType(u.Operand.Type) == TypeHelper.GetNonNullableType(type))
{
e = ((UnaryExpression)e).Operand;
}
}
if (e.NodeType == ExpressionType.Constant)
{
if (e.Type == type)
{
return ((ConstantExpression)e).Value;
}
else if (TypeHelper.GetNonNullableType(e.Type) == TypeHelper.GetNonNullableType(type))
{
return ((ConstantExpression)e).Value;
}
}
var me = e as MemberExpression;
if (me != null)
{
var ce = me.Expression as ConstantExpression;
if (ce != null)
{
return me.Member.GetValue(ce.Value);
}
}
if (type.IsValueType)
{
e = Expression.Convert(e, typeof(object));
}
Expression<Func<object>> lambda = Expression.Lambda<Func<object>>(e);
Func<object> fn = lambda.Compile();
return fn();
}
}
When calling a function that produces a collection i want to cache i pass all my function's parameters and function name to the cache function which creates a key from it.
All my classes implement an interface that has and ID field so i can use it in my cache keys.
I'm sure there's a nicer way but somehow i gotta sleep at times too.
I also pass 1 or more keywords that i can use to invalidate related collections.

Resources