How can I get parameters values from a lamba expression for my nifty cache extension? - asp.net

First of all it might be worth looking at this question:
How can I cache objects in ASP.NET MVC?
There some pseudo code that almost does what i want:
public class CacheExtensions
{
public static T GetOrStore<T>(this Cache cache, string key, Func<T> generator)
{
var result = cache[key];
if(result == null)
{
result = generator();
cache[key] = result;
}
return (T)result;
}
}
However, what I'd really like to do, is auto-generate the "key" from the generator. I figure i need to change the method signature to:
public static T GetOrStore<T>(this Cache cache,
System.Linq.Expressions.Expression<Func<T>> generator)
I want to use the method name, but also any parameters and their values to generate the key. I can get the method body from the expression, and the paramter names (sort of), but I have no idea how to get the paramter values...?
Or am I going about this the wrong way? Any ideas much appreciated.

Here's how I did it:
public static class ICacheExtensions
{
public static T GetOrAdd<T>(this ICache cache, Expression<Func<T>> getterExp)
{
var key = BuildCacheKey<T>(getterExp);
return cache.GetOrAdd(key, () => getterExp.Compile().Invoke());
}
private static string BuildCacheKey<T>(Expression<Func<T>> getterExp)
{
var body = getterExp.Body;
var methodCall = body as MethodCallExpression;
if (methodCall == null)
{
throw new NotSupportedException("The getterExp must be a MethodCallExpression");
}
var typeName = methodCall.Method.DeclaringType.FullName;
var methodName = methodCall.Method.Name;
var arguments = methodCall.Arguments
.Select(a => ExpressionHelper.Evaluate(a))
.ToArray();
return String.Format("{0}_{1}_{2}",
typeName,
methodName,
String.Join("|", arguments));
}
}
with this helper to evaluate nodes of an expression tree:
internal static class ExpressionHelper
{
public static object Evaluate(Expression e)
{
Type type = e.Type;
if (e.NodeType == ExpressionType.Convert)
{
var u = (UnaryExpression)e;
if (TypeHelper.GetNonNullableType(u.Operand.Type) == TypeHelper.GetNonNullableType(type))
{
e = ((UnaryExpression)e).Operand;
}
}
if (e.NodeType == ExpressionType.Constant)
{
if (e.Type == type)
{
return ((ConstantExpression)e).Value;
}
else if (TypeHelper.GetNonNullableType(e.Type) == TypeHelper.GetNonNullableType(type))
{
return ((ConstantExpression)e).Value;
}
}
var me = e as MemberExpression;
if (me != null)
{
var ce = me.Expression as ConstantExpression;
if (ce != null)
{
return me.Member.GetValue(ce.Value);
}
}
if (type.IsValueType)
{
e = Expression.Convert(e, typeof(object));
}
Expression<Func<object>> lambda = Expression.Lambda<Func<object>>(e);
Func<object> fn = lambda.Compile();
return fn();
}
}

When calling a function that produces a collection i want to cache i pass all my function's parameters and function name to the cache function which creates a key from it.
All my classes implement an interface that has and ID field so i can use it in my cache keys.
I'm sure there's a nicer way but somehow i gotta sleep at times too.
I also pass 1 or more keywords that i can use to invalidate related collections.

Related

Xunit CSV streamReader.ReadToEnd returns System.ArgumentOutOfRangeException

I would like to evaluate a CSV data series with Xunit.
For this I need to read in a string consisting of int, bool, double and others.
With the following code, the transfer basically works for one row.
But since I want to test for predecessor values, I need a whole CSV file for evaluation.
My [Theory] works with InlineData without errors.
But when I read in a CSV file, the CSVDataHandler gives a System.ArgumentOutOfRangeException!
I can't find a solution for the error and ask for support.
Thanks a lot!
[Theory, CSVDataHandler(false, "C:\\MyTestData.txt", Skip = "")]
public void TestData(int[] newLine, int[] GetInt, bool[] GetBool)
{
for (int i = 0; i < newLine.Length; i++)
{
output.WriteLine("newLine {0}", newLine[i]);
output.WriteLine("GetInt {0}", GetInt[i]);
output.WriteLine("GetBool {0}", GetBool[i]);
}
}
[DataDiscoverer("Xunit.Sdk.DataDiscoverer", "xunit.core")]
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true, Inherited = true)]
public abstract class DataArribute : Attribute
{
public abstract IEnumerable<object> GetData(MethodInfo methodInfo);
public virtual string? Skip { get; set; }
}
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public class CSVDataHandler : DataAttribute
{
public CSVDataHandler(bool hasHeaders, string pathCSV)
{
this.hasHeaders = hasHeaders;
this.pathCSV = pathCSV;
}
public override IEnumerable<object[]> GetData(MethodInfo methodInfo)
{
var methodParameters = methodInfo.GetParameters();
var paramterTypes = methodParameters.Select(p => p.ParameterType).ToArray();
using (var streamReader = new StreamReader(pathCSV))
{
if (hasHeaders) { streamReader.ReadLine(); }
string csvLine = string.Empty;
// ReadLine ++
//while ((csvLine = streamReader.ReadLine()) != null)
//{
// var csvRow = csvLine.Split(',');
// yield return ConvertCsv((object[])csvRow, paramterTypes);
//}
// ReadToEnd ??
while ((csvLine = streamReader.ReadToEnd()) != null)
{
if (Environment.NewLine != null)
{
var csvRow = csvLine.Split(',');
yield return ConvertCsv((object[])csvRow, paramterTypes); // System.ArgumentOutOfRangeException
}
}
}
}
private static object[] ConvertCsv(IReadOnlyList<object> cswRow, IReadOnlyList<Type> parameterTypes)
{
var convertedObject = new object[parameterTypes.Count];
for (int i = 0; i < parameterTypes.Count; i++)
{
convertedObject[i] = (parameterTypes[i] == typeof(int)) ? Convert.ToInt32(cswRow[i]) : cswRow[i]; // System.ArgumentOutOfRangeException
convertedObject[i] = (parameterTypes[i] == typeof(double)) ? Convert.ToDouble(cswRow[i]) : cswRow[i];
convertedObject[i] = (parameterTypes[i] == typeof(bool)) ? Convert.ToBoolean(cswRow[i]) : cswRow[i];
}
return convertedObject;
}
}
MyTestData.txt
1,2,true,
2,3,false,
3,10,true,
The first call to streamReader.ReadToEnd() will return the entire contents of the file in a string, not just one line. When you call csvLine.Split(',') you will get an array of 12 elements.
The second call to streamReader.ReadToEnd() will not return null as your while statement appears to expect, but an empty string. See the docu at
https://learn.microsoft.com/en-us/dotnet/api/system.io.streamreader.readtoend?view=net-7.0
If the current position is at the end of the stream, returns an empty
string ("").
With the empty string, the call to call csvLine.Split(',') will return an array of length 0, which causes your exception when its first element (index 0) is accessed.
All of this could have been easily discovered by simply starting the test in a debugger.
It looks like you have some other issues here as well.
I don't understand what your if (Environment.NewLine != null) is intended to do, the NewLine property will never be null but should have one of the values "\r\n" or "\n" so the if will always be taken.
The parameters of your test method are arrays int[] and bool[], but you are checking against the types int, double and bool in your ConvertCsv method, so the alternative cswRow[i] will always be returned. You'll wind up passing strings to your method expecting int[] and bool[] and will at latest get an error there.
This method reads a data series from several rows and columns and returns it as an array for testing purposes.
The conversion of the columns can be adjusted according to existing pattern.
Thanks to Christopher!
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public class CSVDataHandler : Xunit.Sdk.DataAttribute
{
public CSVDataHandler(string pathCSV)
{
this.pathCSV = pathCSV;
}
public override IEnumerable<object[]> GetData(MethodInfo methodInfo)
{
List<int> newLine = new();
List<int> GetInt = new();
List<bool> GetBool = new();
var reader = new StreamReader(pathCSV);
string readData = string.Empty;
while ((readData = reader.ReadLine()) != null)
{
string[] split = readData.Split(new char[] { ',' });
newLine.Add(int.Parse(split[0]));
GetInt.Add(int.Parse(split[1]));
GetBool.Add(bool.Parse(split[2]));
// Add more objects ...
}
yield return new object[] { newLine.ToArray(), GetInt.ToArray(), GetBool.ToArray() };
}
}

is it a good idea to do transformation in a database model with ASP.Net

we are a small development team.
We develop in ASP.NET and we are starting to use generic controllers and services.
The goal is to have solid methods for things that are repetitive.
What we ask ourselves is if it is a good idea to do some transformation in the data models to allow us to reuse our functions that we know are working?
Exemple: we have a combobox and we want to manage the display and search. It's always the same and redundant.
This is my class
[Table("stage.Test")]
public partial class Test : IBaseEntity, ICombobox
{
public virtual Product Product { get; set; }
public string nom { get; set; }
public string prenom { get; set; }
public string title { get; set; }
[NotMapped]
public virtual string AffichageCombobox => nom + prenom;
[NotMapped]
public virtual string TexteRecherche => Product.Gabarit.Description;
}
as you can see i have two columns with the tag [NotMapped]. These are the columns in the interface ICombobox
public interface ICombobox
{
string AffichageCombobox { get;}
string TexteRecherche { get; }
}
this is the first service where I use one of my two columns which redirects to other columns. [We use the column "AffichageCombobox" from the model]
public List<ComboboxViewModel> GetComboboxViewModel(int? id, bool actifseulement, string text)
{
var query = _requestDatabaseService.GetComboboxQuery<T>(id, actifseulement, text);
var list = query.Select(table => new ComboboxViewModel
{
Id = table.Id,
AffichageCombobox = table.DateHFin == null ? table.AffichageCombobox : table.AffichageCombobox + " (inactif)"
}).ToList();
return list;
}
This is the RequestDatabaseService [We use the column "TexteRecherche" from the model]
public List<T> GetComboboxQuery<T>(int? id, bool actifseulement, string text) where T : class, IBaseEntity, ICombobox
{
text = text.ToLower();
var list = _dbContext.Set<T>()
.If(id.HasValue,
q => q.Where(x => x.Id == id))
.If(actifseulement,
q => q.Where(x => x.DateHFin == null))
.If(text != "",
q => q.Where(x => x.TexteRecherche.ToLower() == text))
.ToList();
return list;
}
As you can see, I am using an interface to add columns to redirect to the correct columns to my data model to avoid overriding my methods for two column.
Is it a good idea, a good practice ?
What do you think is the best practice if we want to do generic functions, but the columns are not called the same way?
Thank you!
Your solution has a lot of weaknesses
You have extended Model to handle specific UI cases. In my opinion it is bad practice.
Your virtual properties will not work in LINQ query. EF translates only Expression because it canot look into compiled property body.
What we can do here is simplifying of building such comboboxes. I have defind set fo extensions which can be reused for such scenarios. Sorry if there some mistakes, written from memory.
How it can be used:
Assuming that GetComboboxViewModel is not in generic class
public List<ComboboxViewModel> GetComboboxViewModel(int? id, bool actifseulement, string text)
{
// uncover DbContext. All that we need is IQueryable<Test>
var ctx = _requestDatabaseService.GetContext();
var query = ctx.Test.AsQueryable();
var comboItems = query
.FilterItems(id, actifseulement)
.GetComboboxQuery(text, e => e.Product.Gabarit.Description, e => e.nom + e.prenom)
.ToList();
return comboItems;
}
Think about this solution and yes, we can register somewhere pair of Lmbdas Dictionary<Type, (LambdaExpression: searchProp, LambdaExpression: displayProp)> and dynamically build call above.
Realisation:
public static class QueryableExtensions
{
// more simlified version for filtering
public static IQueryable<T> WhereIf(this IQueryable<T> query, bool condition, Expression<Func<T, bool>> predicate)
{
return condition ? query.Where(predicate) : query;
}
// handy function for filtering
public static IQueryable<T> FilterItems<T>(this IQueryable<T> query, int? id, bool onlyActive)
where T : IBaseEntity
{
query = query
.WhereIf(id.HasValue, x => x.Id == id)
.WhereIf(onlyActive, x => x.DateHFin == null)
return query;
}
// dynamic generation of filtering and projection
public static IQueryable<ComboboxViewModel> GetComboboxQuery<T>(this IQueryable<T> query, string text, Expression<Func<T, string>> searchProp, Expression<Func<T, string>> dsiplayProp)
where T : IBaseEntity
{
if (!string.IsNullOrEmpty(text))
{
text = text.ToLower();
// defining search pattern
// this also extension point, you may use here `Contains` or FullText search functions
Expression<Func<string, string, bool>> filterFunc = (s, t) => s.ToLower() == t;
// reusing parameter from searchProp lambda
var param = searchProp.Parameters[0];
// applying pattern to searchprop
var filterBody = ExpressionReplacer.GetBody(filterFunc, searchProp.Body, Expression.Constant(text));
// applying generated filter
var filterPredicate = Expression.Lambda<Func<T, bool>>(filterBody, param);
query = query.Where(filterPredicate);
}
// defining template for Select
Expression<Func<T, string, ComboboxViewModel>> createTemplate = (entity, dp) => new ComboboxViewModel
{
Id = entity.Id,
AffichageCombobox = entity.DateHFin == null ? dp : dp + " (inactif)"
};
// reusing parameter from dsiplayProp lambda
var entityParam = dsiplayProp.Parameters[0];
// injecting dsiplayProp into createTemplate
var selectBody = ExpressionReplacer.GetBody(createTemplate, entityParam, dsiplayProp.Body);
var selectLambda = Expression.Lambda<Func<T, ComboboxViewModel>>(selectBody, entityParam);
// applying projection
var comboQuery = query.Select(selectLambda);
return comboQuery;
}
// helper class for correcting expressions
class ExpressionReplacer : ExpressionVisitor
{
readonly IDictionary<Expression, Expression> _replaceMap;
public ExpressionReplacer(IDictionary<Expression, Expression> replaceMap)
{
_replaceMap = replaceMap ?? throw new ArgumentNullException(nameof(replaceMap));
}
public override Expression Visit(Expression exp)
{
if (exp != null && _replaceMap.TryGetValue(exp, out var replacement))
return replacement;
return base.Visit(exp);
}
public static Expression Replace(Expression expr, Expression toReplace, Expression toExpr)
{
return new ExpressionReplacer(new Dictionary<Expression, Expression> { { toReplace, toExpr } }).Visit(expr);
}
public static Expression Replace(Expression expr, IDictionary<Expression, Expression> replaceMap)
{
return new ExpressionReplacer(replaceMap).Visit(expr);
}
public static Expression GetBody(LambdaExpression lambda, params Expression[] toReplace)
{
if (lambda.Parameters.Count != toReplace.Length)
throw new InvalidOperationException();
return new ExpressionReplacer(Enumerable.Range(0, lambda.Parameters.Count)
.ToDictionary(i => (Expression) lambda.Parameters[i], i => toReplace[i])).Visit(lambda.Body);
}
}
}
Well, after writing this sample, I think, it can be cardinally simplified by using LINQKit. Will post another answer with LINQKit usage if you are interested,

ASP.NET MVC - Session Variables are null in Callback from server ActionResult method

I am implementing CoinPayments IPN in my application and I have trouble with passing data to the method that take their Callback. I have tried like everything I could find: TempData, SessionVariables, tried to implement it somewhere in forms and maybe Request it but that didn`t work for me. So I also tried to implement it with Global static variables. And it worked! But then came another issue: if more than one user were to buy something from website at the same time or even in between any callbacks their data will get mixed. So here I am, trying again to make Session Variables work and have no clue why they are not working as I used them before. Probably what I can think of is that because its a callback from CoinPayments and I handle something wrongly.
Here is the code I have right now: Tho I tried different variations like implementing Session in Get Payment method. Now I ended up with it and it still comes out as null in POST METHOD.
Class for handling Session Variables:
public static class MyGlobalVariables
{
public static int TokenChoice
{
get
{
if (System.Web.HttpContext.Current.Session["TokenChoice"] == null)
{
return -1;
}
else
{
return (int)System.Web.HttpContext.Current.Session["TokenChoice"];
}
}
set
{
System.Web.HttpContext.Current.Session["TokenChoice"] = value;
}
}
public static int PaymentChoice
{
get
{
if (System.Web.HttpContext.Current.Session["PaymentChoice"] == null)
{
return -1;
}
else
{
return (int)System.Web.HttpContext.Current.Session["PaymentChoice"];
}
}
set
{
System.Web.HttpContext.Current.Session["PaymentChoice"] = value;
}
}
public static string CurrentUser
{
get
{
System.Web.HttpContext.Current.Session["CurrentUser"] = System.Web.HttpContext.Current.User.Identity.Name;
return (string)System.Web.HttpContext.Current.Session["CurrentUser"];
}
}
}
Class that returns view where you click on CoinPayments button:
public ActionResult Payment(int tokenChoice, int paymentChoice)
{
ViewBag.Payment = paymentChoice;
MyGlobalVariables.PaymentChoice = paymentChoice;
MyGlobalVariables.TokenChoice = tokenChoice;
return View();
}
Callback class that handles Callback from CoinPayments:
[HttpPost]
public ActionResult Payment()
{
NameValueCollection nvc = Request.Form;
var merchant_id = id;
var ipn_secret = secret;
var order_total = MyGlobalVariables.PaymentChoice;
if (String.IsNullOrEmpty(nvc["ipn_mode"]) || nvc["ipn_mode"] != "hmac")
{
Trace.WriteLine("IPN Mode is not HMAC");
return View();
}
if (String.IsNullOrEmpty(HTTP_HMAC))
{
Trace.WriteLine("No HMAC signature sent");
return View();
}
if (String.IsNullOrEmpty(nvc["merchant"]) || nvc["merchant"] != merchant_id.Trim())
{
Trace.WriteLine("No or incorrect Merchant ID passed");
return View();
}
//var hmac = hash_hmac("sha512", request, ipn_secret.Trim());
var txn_id = nvc["txn_id"];
var item_name = nvc["item_name"];
var item_number = nvc["item_number"];
var amount1 = nvc["amount1"];
var amount2 = float.Parse(nvc["amount2"], CultureInfo.InvariantCulture.NumberFormat);
var currency1 = nvc["currency1"];
var currency2 = nvc["currency2"];
var status = Convert.ToInt32(nvc["status"]);
var status_text = nvc["status_text"];
Trace.WriteLine(status);
if (currency1 != "USD") {
Trace.WriteLine("Original currency mismatch!");
return View();
}
if (Convert.ToInt32(amount1) < Convert.ToInt32(order_total))
{
Trace.WriteLine("Amount is less than order total!");
return View();
}
if (status >= 100 || status == 2) {
using (MyDatabaseEntities1 dc = new MyDatabaseEntities1())
{
var account = dc.Users.Where(a => a.Username == MyGlobalVariables.CurrentUser).FirstOrDefault();
if (account != null && account.Paid == 0)
{
Trace.WriteLine("Payment Completed");
Trace.WriteLine("Tokens to add: " + MyGlobalVariables.TokenChoice);
account.Tokens += MyGlobalVariables.TokenChoice;
account.Paid = 1;
dc.Configuration.ValidateOnSaveEnabled = false;
dc.SaveChanges();
}
}
} else if (status < 0)
{
Trace.WriteLine(
"payment error, this is usually final but payments will sometimes be reopened if there was no exchange rate conversion or with seller consent");
} else {
using (MyDatabaseEntities1 dc = new MyDatabaseEntities1())
{
var account = dc.Users.Where(a => a.Username == MyGlobalVariables.CurrentUser).FirstOrDefault();
if (account != null)
{
account.Paid = 0;
dc.Configuration.ValidateOnSaveEnabled = false;
dc.SaveChanges();
}
}
Trace.WriteLine("Payment is pending");
}
return View();
}
As you can see there are only 3 variables I need to handle.
Also someone might ask why I use Session Variable for Current.User?
Well for some reason Callback method can not read Current.User as it return null. And well... nothing really changed as for now.
If you have ever experienced something like that or can find an issue I would be so thankful since I wasted already over 2 days on that issue.
EDIT:
After some testing I found out variables works fine if I run Post method on my own. So the problem is with handling callback from CoinPayments. Is there a specific way to deal with this?

How to auto setup MOQ by given parameters and result in an object[]?

I want to automate the Setup code by a given array containing expected result and parameters.
Something like data driven setup.
My existing code:
var mock = new Mock<ILogin>();
var testDataTable = new object[,]
{
{ LoginResult.Success, "Jack", "123!##"}
, { LoginResult.WrongPassword, "Jack", "123321"}
, { LoginResult.NoSuchUser, "Peter", "123!##"}
};
// ForEachRow is my own extension method
testDataTable.ForEachRow((row) =>
{
var result = (LoginResult)row[0];
var username = (string)row[1];
var password = (string)row[2];
mock.Setup(o => o.Login(
It.Is<string>(u => u == username),
It.Is<string>(p => p == password)
)).Returns(result);
});
return mock.Object;
Code that I wish:
var mock = new Mock<ILogin>();
new object[,]
{
{ LoginResult.Success, "Jack", "123!##"}
, { LoginResult.WrongPassword, "Jack", "123321"}
, { LoginResult.NoSuchUser, "Peter", "123!##"}
}.ForEachRow((row) =>
{
var exprTree = (ILogin o)=>o.Login("ANY", "ANY");
AutoSetup(mock, exprTree, row); // <---- How to write this AutoSetup?
});
return mock.Object;
How to write the AutoSetup(mock, exprTree, dataArray) function above?
It takes three parameters:
mock: A mock object, e.g. new Mock()
exprTree: Expression tree that represents a method to be setup
dataArray: An object[], the 0 element is the expected result, and others are parameters that passed to the method
This was an interesting challenge. I think I have a working implementation for your AutoSetup method, using the expressions API. If anyone has a simpler solution, I'd love to see it.
static void AutoSetup<TMock, TResult>(Mock<TMock> mock, Expression<Func<TMock, TResult>> exprTree, object[] items) where TMock : class
{
var methodCallExpr = exprTree.Body as MethodCallExpression;
var arguments = items.Skip(1).Select(o => Expression.Constant(o));
var updatedMethod = methodCallExpr.Update(methodCallExpr.Object, arguments);
var updatedLambda = exprTree.Update(updatedMethod, exprTree.Parameters);
mock.Setup(updatedLambda).Returns((TResult)items[0]);
}
Here is a full working test as a console app, with an implementation of ForEachRow, which you didn't provide.
class Program
{
static void Main(string[] args)
{
var login = SetUp();
Console.WriteLine(login.Login("Jack", "123!##"));
Console.WriteLine(login.Login("Jack", "123321"));
Console.WriteLine(login.Login("Peter", "123!##"));
Console.ReadKey();
}
static ILogin SetUp()
{
var mock = new Mock<ILogin>(MockBehavior.Strict);
var rows = new object[,]
{
{ LoginResult.Success, "Jack", "123!##" },
{ LoginResult.WrongPassword, "Jack", "123321" },
{ LoginResult.NoSuchUser, "Peter", "123!##" }
};
rows.ForEachRow((row) => AutoSetup(mock, (ILogin l) => l.Login("ANY", "ANY"), row));
return mock.Object;
}
private static void AutoSetup<TMock, TResult>(Mock<TMock> mock, Expression<Func<TMock, TResult>> exprTree, object[] items) where TMock : class
{
var methodCallExpr = exprTree.Body as MethodCallExpression;
var arguments = items.Skip(1).Select(o => Expression.Constant(o));
var updatedMethod = methodCallExpr.Update(methodCallExpr.Object, arguments);
var updatedLambda = exprTree.Update(updatedMethod, exprTree.Parameters);
mock.Setup(updatedLambda).Returns((TResult)items[0]);
}
}
public static class ArrayExtensions
{
public static void ForEachRow<T>(this T[,] rows, Action<T[]> action)
{
var x = rows.GetLength(1);
var y = rows.GetLength(0);
for (int i = 0; i < y; i++)
{
var row = new T[x];
for (int j = 0; j < x; j++)
{
row[j] = rows[i, j];
}
action(row);
}
}
}
public interface ILogin
{
LoginResult Login(string p1, string p2);
}
public enum LoginResult
{
Success,
WrongPassword,
NoSuchUser
}
EDIT: You asked in a comment about how to take advantage of the variable parameter matching that Moq provides with the It.IsAny<> method. Because what you pass to your Mock.Setup() is an expression tree, it's able to scan the method parameters and implement special behaviour for any that are calls to It.IsAny<>. However, if you use It.IsAny<> in your test data array, by the time we retrieve it from your items array to set in on the expression, it's not a method call but simply the result of the call it It.IsAny<> which is default(T) (see here).
We need some way of specifying in your test data array that the parameter should be any. Then we can check for this special case and generate the correct MethodCallExpression representing a call to It.IsAny<>.
Here are the changes I made to support this:
Add an Any type to use in your test data
public class Any<T>
{
private Any() { }
public static Any<T> Param { get { return new Any<T>(); } }
}
Update the AutoSetup method to handle this special case:
private static void AutoSetup<TMock, TResult>(Mock<TMock> mock, Expression<Func<TMock, TResult>> exprTree, object[] items) where TMock : class
{
var arguments = items.Skip(1).Select(o => {
var type = o.GetType();
if (type.IsGenericType && type.GetGenericTypeDefinition() == typeof(Any<>))
{
var typeParameter = type.GetGenericArguments();
var genericItIsAny = typeof(It).GetMethod("IsAny");
var itIsAny = genericItIsAny.MakeGenericMethod(typeParameter);
return Expression.Call(itIsAny) as Expression;
}
return Expression.Constant(o);
});
var methodCallExpr = exprTree.Body as MethodCallExpression;
var updatedMethod = methodCallExpr.Update(methodCallExpr.Object, arguments);
var updatedLambda = exprTree.Update(updatedMethod, exprTree.Parameters);
mock.Setup(updatedLambda).Returns((TResult)items[0]);
}
In the test data, use the Any type
{ LoginResult.Success, "NoPasswordUser", Any<string>.Param }
Depending on how you're storing your test data, you might need another way of identifying parameters that you want to be variable (specially formatted string?), but in general this should give you an idea about how to create the correct expressions.

How to Optimize this method

private static void ConvertToUpper(object entity, Hashtable visited)
{
if (entity != null && !visited.ContainsKey(entity))
{
visited.Add(entity, entity);
foreach (PropertyInfo propertyInfo in entity.GetType().GetProperties())
{
if (!propertyInfo.CanRead || !propertyInfo.CanWrite)
continue;
object propertyValue = propertyInfo.GetValue(entity, null);
Type propertyType;
if ((propertyType = propertyInfo.PropertyType) == typeof(string))
{
if (propertyValue != null && !propertyInfo.Name.Contains("password"))
{
propertyInfo.SetValue(entity, ((string)propertyValue).ToUpper(), null);
}
continue;
}
if (!propertyType.IsValueType)
{
IEnumerable enumerable;
if ((enumerable = propertyValue as IEnumerable) != null)
{
foreach (object value in enumerable)
{
ConvertToUpper(value, visited);
}
}
else
{
ConvertToUpper(propertyValue, visited);
}
}
}
}
}
Right now it works fine for objects with lists that are relatively small, but once the list of objects get larger it takes forever. How would i optimize this and also set a limit for a max depth.
Thanks for any help
I didn't profile the following code, but it must be very performant on complex structures.
1) Uses dynamic code generation.
2) Uses type-based cache for generated dynamic delegates.
public class VisitorManager : HashSet<object>
{
delegate void Visitor(VisitorManager manager, object entity);
Dictionary<Type, Visitor> _visitors = new Dictionary<Type, Visitor>();
void ConvertToUpperEnum(IEnumerable entity)
{
// TODO: this can be parallelized, but then we should thread-safe lock the cache
foreach (var obj in entity)
ConvertToUpper(obj);
}
public void ConvertToUpper(object entity)
{
if (entity != null && !Contains(entity))
{
Add(entity);
var visitor = GetCachedVisitor(entity.GetType());
if (visitor != null)
visitor(this, entity);
}
}
Type _lastType;
Visitor _lastVisitor;
Visitor GetCachedVisitor(Type type)
{
if (type == _lastType)
return _lastVisitor;
_lastType = type;
return _lastVisitor = GetVisitor(type);
}
Visitor GetVisitor(Type type)
{
Visitor result;
if (!_visitors.TryGetValue(type, out result))
_visitors[type] = result = BuildVisitor(type);
return result;
}
static MethodInfo _toUpper = typeof(string).GetMethod("ToUpper", new Type[0]);
static MethodInfo _convertToUpper = typeof(VisitorManager).GetMethod("ConvertToUpper", BindingFlags.Instance | BindingFlags.Public);
static MethodInfo _convertToUpperEnum = typeof(VisitorManager).GetMethod("ConvertToUpperEnum", BindingFlags.Instance | BindingFlags.NonPublic);
Visitor BuildVisitor(Type type)
{
var visitorManager = Expression.Parameter(typeof(VisitorManager), "manager");
var entityParam = Expression.Parameter(typeof(object), "entity");
var entityVar = Expression.Variable(type, "e");
var cast = Expression.Assign(entityVar, Expression.Convert(entityParam, type)); // T e = (T)entity;
var statements = new List<Expression>() { cast };
foreach (var prop in type.GetProperties())
{
// if cannot read or cannot write - ignore property
if (!prop.CanRead || !prop.CanWrite) continue;
var propType = prop.PropertyType;
// if property is value type - ignore property
if (propType.IsValueType) continue;
var isString = propType == typeof(string);
// if string type but no password in property name - ignore property
if (isString && !prop.Name.Contains("password"))
continue;
#region e.Prop
var propAccess = Expression.Property(entityVar, prop); // e.Prop
#endregion
#region T value = e.Prop
var value = Expression.Variable(propType, "value");
var assignValue = Expression.Assign(value, propAccess);
#endregion
if (isString)
{
#region if (value != null) e.Prop = value.ToUpper();
var ifThen = Expression.IfThen(Expression.NotEqual(value, Expression.Constant(null, typeof(string))),
Expression.Assign(propAccess, Expression.Call(value, _toUpper)));
#endregion
statements.Add(Expression.Block(new[] { value }, assignValue, ifThen));
}
else
{
#region var i = value as IEnumerable;
var enumerable = Expression.Variable(typeof(IEnumerable), "i");
var assignEnum = Expression.Assign(enumerable, Expression.TypeAs(value, enumerable.Type));
#endregion
#region if (i != null) manager.ConvertToUpperEnum(i); else manager.ConvertToUpper(value);
var ifThenElse = Expression.IfThenElse(Expression.NotEqual(enumerable, Expression.Constant(null)),
Expression.Call(visitorManager, _convertToUpperEnum, enumerable),
Expression.Call(visitorManager, _convertToUpper, value));
#endregion
statements.Add(Expression.Block(new[] { value, enumerable }, assignValue, assignEnum, ifThenElse));
}
}
// no blocks
if (statements.Count <= 1)
return null;
return Expression.Lambda<Visitor>(Expression.Block(new[] { entityVar }, statements), visitorManager, entityParam).Compile();
}
}
It looks pretty lean to me. The only thing I can think of would be to parallelize this. If I get a chance I will try to work something out and edit my answer.
Here is how to limit the depth.
private static void ConvertToUpper(object entity, Hashtable visited, int depth)
{
if (depth > MAX_DEPTH) return;
// Omitted code for brevity.
// Example usage here.
ConvertToUppder(..., ..., depth + 1);
}
What you could do is have a Dictionary with a type as the key and relevant properties as the values. You would then only need to search through the properties once for the ones you are interested in (by the looks of things IEnumerable and string) - after all, the properties the types have aren't going to change (unless you're doing some funky Emit stuff but I'm not too familiar with that)
Once you have this you could simply iterate all the properties in the Dictionary using the objects type as the key.
Somehting like this (I haven't actually tested it but it does complile :) )
private static Dictionary<Type, List<PropertyInfo>> _properties = new Dictionary<Type, List<PropertyInfo>>();
private static void ExtractProperties(List<PropertyInfo> list, Type type)
{
if (type == null || type == typeof(object))
{
return; // We've reached the top
}
// Modify which properties you want here
// This is for Public, Protected, Private
const BindingFlags PropertyFlags = BindingFlags.DeclaredOnly |
BindingFlags.Instance |
BindingFlags.NonPublic |
BindingFlags.Public;
foreach (var property in type.GetProperties(PropertyFlags))
{
if (!property.CanRead || !property.CanWrite)
continue;
if ((property.PropertyType == typeof(string)) ||
(property.PropertyType.GetInterface("IEnumerable") != null))
{
if (!property.Name.Contains("password"))
{
list.Add(property);
}
}
}
// OPTIONAL: Navigate the base type
ExtractProperties(list, type.BaseType);
}
private static void ConvertToUpper(object entity, Hashtable visited)
{
if (entity != null && !visited.ContainsKey(entity))
{
visited.Add(entity, entity);
List<PropertyInfo> properties;
if (!_properties.TryGetValue(entity.GetType(), out properties))
{
properties = new List<PropertyInfo>();
ExtractProperties(properties, entity.GetType());
_properties.Add(entity.GetType(), properties);
}
foreach (PropertyInfo propertyInfo in properties)
{
object propertyValue = propertyInfo.GetValue(entity, null);
Type propertyType = propertyInfo.PropertyType;
if (propertyType == typeof(string))
{
propertyInfo.SetValue(entity, ((string)propertyValue).ToUpper(), null);
}
else // It's IEnumerable
{
foreach (object value in (IEnumerable)propertyValue)
{
ConvertToUpper(value, visited);
}
}
}
}
}
Here is a blog of code that should work to apply the Max Depth limit that Brian Gideon mentioned as well as parallel things a bit. It's not perfect and could be refined a bit since I broke the value types and non-value type properties into 2 linq queries.
private static void ConvertToUpper(object entity, Hashtable visited, int depth)
{
if (entity == null || visited.ContainsKey(entity) || depth > MAX_DEPTH)
{
return;
}
visited.Add(entity, entity);
var properties = from p in entity.GetType().GetProperties()
where p.CanRead &&
p.CanWrite &&
p.PropertyType == typeof(string) &&
!p.Name.Contains("password") &&
p.GetValue(entity, null) != null
select p;
Parallel.ForEach(properties, (p) =>
{
p.SetValue(entity, ((string)p.GetValue(entity, null)).ToUpper(), null);
});
var valProperties = from p in entity.GetType().GetProperties()
where p.CanRead &&
p.CanWrite &&
!p.PropertyType.IsValueType &&
!p.Name.Contains("password") &&
p.GetValue(entity, null) != null
select p;
Parallel.ForEach(valProperties, (p) =>
{
if (p.GetValue(entity, null) as IEnumerable != null)
{
foreach(var value in p.GetValue(entity, null) as IEnumerable)
ConvertToUpper(value, visted, depth +1);
}
else
{
ConvertToUpper(p, visited, depth +1);
}
});
}
There are a couple of immediate issues:
There is repeated evaluation of property information for what I am assuming are the same types.
Reflection is comparatively slow.
Issue 1. can be solved by memoizing property information about types and caching it so it does not have to be re-calculated for each recurring type we see.
Performance of issue 2. can be helped out by using IL code generation and dynamic methods. I grabbed code from here to implement dynamically (and also memoized from point 1.) generated and highly efficient calls for getting and setting property values. Basically IL code is dynamically generated to call set and get for a property and encapsulated in a method wrapper - this bypasses all the reflection steps (and some security checks...). Where the following code refers to "DynamicProperty" I have used the code from the previous link.
This method can also be parallelized as suggested by others, just ensure the "visited" cache and calculated properties cache are synchronized.
private static readonly Dictionary<Type, List<ProperyInfoWrapper>> _typePropertyCache = new Dictionary<Type, List<ProperyInfoWrapper>>();
private class ProperyInfoWrapper
{
public GenericSetter PropertySetter { get; set; }
public GenericGetter PropertyGetter { get; set; }
public bool IsString { get; set; }
public bool IsEnumerable { get; set; }
}
private static void ConvertToUpper(object entity, Hashtable visited)
{
if (entity != null && !visited.Contains(entity))
{
visited.Add(entity, entity);
foreach (ProperyInfoWrapper wrapper in GetMatchingProperties(entity))
{
object propertyValue = wrapper.PropertyGetter(entity);
if(propertyValue == null) continue;
if (wrapper.IsString)
{
wrapper.PropertySetter(entity, (((string)propertyValue).ToUpper()));
continue;
}
if (wrapper.IsEnumerable)
{
IEnumerable enumerable = (IEnumerable)propertyValue;
foreach (object value in enumerable)
{
ConvertToUpper(value, visited);
}
}
else
{
ConvertToUpper(propertyValue, visited);
}
}
}
}
private static IEnumerable<ProperyInfoWrapper> GetMatchingProperties(object entity)
{
List<ProperyInfoWrapper> matchingProperties;
if (!_typePropertyCache.TryGetValue(entity.GetType(), out matchingProperties))
{
matchingProperties = new List<ProperyInfoWrapper>();
foreach (PropertyInfo propertyInfo in entity.GetType().GetProperties())
{
if (!propertyInfo.CanRead || !propertyInfo.CanWrite)
continue;
if (propertyInfo.PropertyType == typeof(string))
{
if (!propertyInfo.Name.Contains("password"))
{
ProperyInfoWrapper wrapper = new ProperyInfoWrapper
{
PropertySetter = DynamicProperty.CreateSetMethod(propertyInfo),
PropertyGetter = DynamicProperty.CreateGetMethod(propertyInfo),
IsString = true,
IsEnumerable = false
};
matchingProperties.Add(wrapper);
continue;
}
}
if (!propertyInfo.PropertyType.IsValueType)
{
object propertyValue = propertyInfo.GetValue(entity, null);
bool isEnumerable = (propertyValue as IEnumerable) != null;
ProperyInfoWrapper wrapper = new ProperyInfoWrapper
{
PropertySetter = DynamicProperty.CreateSetMethod(propertyInfo),
PropertyGetter = DynamicProperty.CreateGetMethod(propertyInfo),
IsString = false,
IsEnumerable = isEnumerable
};
matchingProperties.Add(wrapper);
}
}
_typePropertyCache.Add(entity.GetType(), matchingProperties);
}
return matchingProperties;
}
While your question is about the performance of the code, there is another problem that others seem to miss: Maintainability.
While you might think this is not as important as the performance problem you are having, having code that is more readable and maintainable will make it easier to solve problems with it.
Here is an example of how your code might look like, after a few refactorings:
class HierarchyUpperCaseConverter
{
private HashSet<object> visited = new HashSet<object>();
public static void ConvertToUpper(object entity)
{
new HierarchyUpperCaseConverter_v1().ProcessEntity(entity);
}
private void ProcessEntity(object entity)
{
// Don't process null references.
if (entity == null)
{
return;
}
// Prevent processing types that already have been processed.
if (this.visited.Contains(entity))
{
return;
}
this.visited.Add(entity);
this.ProcessEntity(entity);
}
private void ProcessEntity(object entity)
{
var properties =
this.GetProcessableProperties(entity.GetType());
foreach (var property in properties)
{
this.ProcessEntityProperty(entity, property);
}
}
private IEnumerable<PropertyInfo> GetProcessableProperties(Type type)
{
var properties =
from property in type.GetProperties()
where property.CanRead && property.CanWrite
where !property.PropertyType.IsValueType
where !(property.Name.Contains("password") &&
property.PropertyType == typeof(string))
select property;
return properties;
}
private void ProcessEntityProperty(object entity, PropertyInfo property)
{
object value = property.GetValue(entity, null);
if (value != null)
{
if (value is IEnumerable)
{
this.ProcessCollectionProperty(value as IEnumerable);
}
else if (value is string)
{
this.ProcessStringProperty(entity, property, (string)value);
}
else
{
this.AlterHierarchyToUpper(value);
}
}
}
private void ProcessCollectionProperty(IEnumerable value)
{
foreach (object item in (IEnumerable)value)
{
// Make a recursive call.
this.AlterHierarchyToUpper(item);
}
}
private void ProcessStringProperty(object entity, PropertyInfo property, string value)
{
string upperCaseValue = ConvertToUpperCase(value);
property.SetValue(entity, upperCaseValue, null);
}
private string ConvertToUpperCase(string value)
{
// TODO: ToUpper is culture sensitive.
// Shouldn't we use ToUpperInvariant?
return value.ToUpper();
}
}
While this code is more than twice as long as your code snippet, it is more maintainable. In the process of refactoring your code I even found a possible bug in your code. This bug is a lot harder to spot in your code. In your code you try to convert all string values to upper case but you don't convert string values that are stored in object properties. Look for instance at the following code.
class A
{
public object Value { get; set; }
}
var a = new A() { Value = "Hello" };
Perhaps this is exactly what you wanted, but the string "Hello" is not converted to "HELLO" in your code.
Another thing I like to note is that while the only thing I tried to do is make your code more readable, my refactoring seems about 20% faster.
After I refactored the code I tried to improve performance of it, but I found out that it is particularly hard to improve it. While others try to parallelize the code I have to warn about this. Parallelizing the code isn't as easy as others might let you think. There is some synchronization going on between threads (in the form of the 'visited' collection). Don't forget that writing to a collection is not thread-safe. Using a thread-safe version or locking on it might degrade performance again. You will have to test this.
I also found out that the real performance bottleneck is all the reflection (especially the reading of all the property values). The only way to really speed this up is by hard coding the code operations for each and every type, or as others suggested lightweight code generation. However, this is pretty hard and it is questionable whether it is worth the trouble.
I hope you find my refactorings useful and wish you good luck with improving performance.

Resources