As these questions point out, Guid.NewGuid will return the same value for all rows due to the enforced deterministic nature of U-SQL i.e if it's scaled out if an element (vertex) needs retrying then it should return the same value....
Guid.NewGuid() always return same Guid for all rows
auto_increment in U-SQL
However.... the code example in the officials documentation for a User Defined Extractor purposefully uses Guid.NewGuid().
I'm not querying the validity of the answers for the questions above, as they are from an authoritative source (the programme manager for u-sql, so very authoritative!). However, what I'm wondering if the action of using an Extractor means NewGuid can be used as normal? Is it simply within c# expressions in u-sql and User Defined Functions in which NewGuid is unsafe?
[SqlUserDefinedExtractor(AtomicFileProcessing = true)]
public class FullDescriptionExtractor : IExtractor
{
private Encoding _encoding;
private byte[] _row_delim;
private char _col_delim;
public FullDescriptionExtractor(Encoding encoding, string row_delim = "\r\n", char col_delim = '\t')
{
this._encoding = ((encoding == null) ? Encoding.UTF8 : encoding);
this._row_delim = this._encoding.GetBytes(row_delim);
this._col_delim = col_delim;
}
public override IEnumerable<IRow> Extract(IUnstructuredReader input, IUpdatableRow output)
{
string line;
//Read the input line by line
foreach (Stream current in input.Split(_encoding.GetBytes("\r\n")))
{
using (System.IO.StreamReader streamReader = new StreamReader(current, this._encoding))
{
line = streamReader.ReadToEnd().Trim();
//Split the input by the column delimiter
string[] parts = line.Split(this._col_delim);
int count = 0; // start with first column
foreach (string part in parts)
{
if (count == 0)
{ // for column “guid”, re-generated guid
Guid new_guid = Guid.NewGuid();
output.Set<Guid>(count, new_guid);
}
else if (count == 2)
{
// for column “user”, convert to UPPER case
output.Set<string>(count, part.ToUpper());
}
else
{
// keep the rest of the columns as-is
output.Set<string>(count, part);
}
count += 1;
}
}
yield return output.AsReadOnly();
}
yield break;
}
}
https://learn.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-programmability-guide#use-user-defined-extractors
I am trying to setup a product key system in my application, but I want to ensure the attribute has the right size (16 characters).
I tried the following
public class ProductKey
{
public const int ProductKeyLength = 16;
[StringLength(ProductKeyLength, MinimumLength = ProductKeyLength)]
private string _value;
[Required]
[Index(IsUnique = true)]
public string Value {
get
{
var temp = Regex.Replace(this._value, ".{4}", "$0-");
return temp.Trim('-');
}
set { this._value = value.Replace("-", "");}
}
}
I want to enable the user to insert the key with our without hyphen. I get the following error with above code:
Column 'Value' in table 'dbo.ProductKeys' is of a type that is invalid for use as a key column in an index.
As I understood, I need to set a limit to Value so it can be used as a unique key. But, _value has a limit and _value is the actual representation of Value in the database.
Is there a way to set the limit correctly in this case?
Thanks in advance.
You are getting the error because without a StringLength attribute on the Value field, the database column gets created as VARCHAR(MAX) which cannot be used as a key. You need a [StringLength] on the field being used as a key. However, as your getter is returning the key formatted with dashes, you need the key length to be 19:
public class ProductKey
{
public const int ProductKeyLength = 19;
private string _value { get; set; }
[Key]
[Required]
[StringLength(ProductKeyLength, MinimumLength = ProductKeyLength)]
[Index(IsUnique = true)]
public string Value
{
get
{
var temp = Regex.Replace(this._value, ".{4}", "$0-");
return temp.Trim('-');
}
set { this._value = value.Replace("-", ""); }
}
}
You might be better off doing your format conversion in ViewModels and client-side code, as one problem you'll have here is searching - for example...
db.Keys.Add(new ProductKey { Value = "1234-5678-9012-3456" });
db.Keys.Add(new ProductKey { Value = "1234567890123455" });
db.SaveChanges();
Console.WriteLine(db.Keys.Count(k => k.Value.Contains("89"))); // 0
Console.WriteLine(db.Keys.Count(k => k.Value.Contains("8-9"))); // 2
// Hi everyone
i do this call in Action :
[HttpGet]
public virtual ActionResult JsonGetProvinces(int countryId)
{
//WebSiteContext WbContext = new WebSiteContext();
//UnitOfWork UnitofWork = new UnitOfWork(WbContext);
var provinces =
(
from province in unitofWork.ProvinceRepository.All
where province.CountryId == countryId
select new
{
Id = province.Id,
Name = province.GetType().GetProperty("Name_" + CultureManager.GetCurrentCultureShortName()).GetValue(province)
}
).ToList();
return Json(provinces, JsonRequestBehavior.AllowGet);
}
something is wrong with my query :
var provinces =
(
from province in unitofWork.ProvinceRepository.All
where province.CountryId == countryId
select new
{
Id = province.Id,
Name = province.GetType().GetProperty("Name_" + CultureManager.GetCurrentCultureShortName()).GetValue(province)
}
).ToList();
Particulary,
Name = province.GetType().GetProperty("Name_" + CultureManager.GetCurrentCultureShortName()).GetValue(province)
In BDD, there is Name_fr, Name_en columns
and i'm trying to take one dynamically... Is it possible ?
Of course, i can take both and choose dynamically the column in View but i would to know how do...
Thank you for your help
The short answer is you need to change your code a bit and using expression tree inside. Look at this question
EF can not translate function calls to SQL. Using expression trees can be comlicated see this question
Here is a sample with expression trees. The GetQuery2 is the same as GetQuery but with expression tree and a propertyname parameter.
public static IQueryable<Foo> GetQuery(BlogContext context)
{
var query = from x in context.BlogEntries
select new Foo
{
NameX = x.Name
};
return query;
}
public static IQueryable<Foo> GetQuery2(BlogContext context, string propertyName)
{
ConstructorInfo ci = typeof(Foo).GetConstructor(new Type[0]);
MethodInfo miFooGetName = typeof(Foo).GetMethod("set_NameX");
MethodInfo miBlogEntry = typeof(BlogEntry).GetMethod("get_" + propertyName);
ParameterExpression param = Expression.Parameter(typeof(BlogEntry), "x");
IQueryable<Foo> result = Queryable.Select<BlogEntry, Foo>(
context.BlogEntries,
Expression.Lambda<Func<BlogEntry, Foo>>(
Expression.MemberInit(
Expression.New(ci, new Expression[0]),
new MemberBinding[]{
Expression.Bind(miFooGetName,
Expression.Property(param,
miBlogEntry))}
),
param
)
);
return result;
}
It is easier the fetch all all language strings and write an additional Property Name that does the magic.
I know i can do this
var nv = HttpUtility.ParseQueryString(req.RawUrl);
But is there a way to convert this back to a url?
var newUrl = HttpUtility.Something("/page", nv);
Simply calling ToString() on the NameValueCollection will return the name value pairs in a name1=value1&name2=value2 querystring ready format. Note that NameValueCollection types don't actually support this and it's misleading to suggest this, but the behavior works here due to the internal type that's actually returned, as explained below.
Thanks to #mjwills for pointing out that the HttpUtility.ParseQueryString method actually returns an internal HttpValueCollection object rather than a regular NameValueCollection (despite the documentation specifying NameValueCollection). The HttpValueCollection automatically encodes the querystring when using ToString(), so there's no need to write a routine that loops through the collection and uses the UrlEncode method. The desired result is already returned.
With the result in hand, you can then append it to the URL and redirect:
var nameValues = HttpUtility.ParseQueryString(Request.QueryString.ToString());
string url = Request.Url.AbsolutePath + "?" + nameValues.ToString();
Response.Redirect(url);
Currently the only way to use a HttpValueCollection is by using the ParseQueryString method shown above (other than reflection, of course). It looks like this won't change since the Connect issue requesting this class be made public has been closed with a status of "won't fix."
As an aside, you can call the Add, Set, and Remove methods on nameValues to modify any of the querystring items before appending it. If you're interested in that see my response to another question.
string q = String.Join("&",
nvc.AllKeys.Select(a => a + "=" + HttpUtility.UrlEncode(nvc[a])));
Make an extension method that uses a couple of loops. I prefer this solution because it's readable (no linq), doesn't require System.Web.HttpUtility, and it supports duplicate keys.
public static string ToQueryString(this NameValueCollection nvc)
{
if (nvc == null) return string.Empty;
StringBuilder sb = new StringBuilder();
foreach (string key in nvc.Keys)
{
if (string.IsNullOrWhiteSpace(key)) continue;
string[] values = nvc.GetValues(key);
if (values == null) continue;
foreach (string value in values)
{
sb.Append(sb.Length == 0 ? "?" : "&");
sb.AppendFormat("{0}={1}", Uri.EscapeDataString(key), Uri.EscapeDataString(value));
}
}
return sb.ToString();
}
Example
var queryParams = new NameValueCollection()
{
{ "order_id", "0000" },
{ "item_id", "1111" },
{ "item_id", "2222" },
{ null, "skip entry with null key" },
{ "needs escaping", "special chars ? = &" },
{ "skip entry with null value", null }
};
Console.WriteLine(queryParams.ToQueryString());
Output
?order_id=0000&item_id=1111&item_id=2222&needs%20escaping=special%20chars%20%3F%20%3D%20%26
This should work without too much code:
NameValueCollection nameValues = HttpUtility.ParseQueryString(String.Empty);
nameValues.Add(Request.QueryString);
// modify nameValues if desired
var newUrl = "/page?" + nameValues;
The idea is to use HttpUtility.ParseQueryString to generate an empty collection of type HttpValueCollection. This class is a subclass of NameValueCollection that is marked as internal so that your code cannot easily create an instance of it.
The nice thing about HttpValueCollection is that the ToString method takes care of the encoding for you. By leveraging the NameValueCollection.Add(NameValueCollection) method, you can add the existing query string parameters to your newly created object without having to first convert the Request.QueryString collection into a url-encoded string, then parsing it back into a collection.
This technique can be exposed as an extension method as well:
public static string ToQueryString(this NameValueCollection nameValueCollection)
{
NameValueCollection httpValueCollection = HttpUtility.ParseQueryString(String.Empty);
httpValueCollection.Add(nameValueCollection);
return httpValueCollection.ToString();
}
Actually, you should encode the key too, not just value.
string q = String.Join("&",
nvc.AllKeys.Select(a => $"{HttpUtility.UrlEncode(a)}={HttpUtility.UrlEncode(nvc[a])}"));
Because a NameValueCollection can have multiple values for the same key, if you are concerned with the format of the querystring (since it will be returned as comma-separated values rather than "array notation") you may consider the following.
Example
var nvc = new NameValueCollection();
nvc.Add("key1", "val1");
nvc.Add("key2", "val2");
nvc.Add("empty", null);
nvc.Add("key2", "val2b");
Turn into: key1=val1&key2[]=val2&empty&key2[]=val2b rather than key1=val1&key2=val2,val2b&empty.
Code
string qs = string.Join("&",
// "loop" the keys
nvc.AllKeys.SelectMany(k => {
// "loop" the values
var values = nvc.GetValues(k);
if(values == null) return new[]{ k };
return nvc.GetValues(k).Select( (v,i) =>
// 'gracefully' handle formatting
// when there's 1 or more values
string.Format(
values.Length > 1
// pick your array format: k[i]=v or k[]=v, etc
? "{0}[]={1}"
: "{0}={1}"
, k, HttpUtility.UrlEncode(v), i)
);
})
);
or if you don't like Linq so much...
string qs = nvc.ToQueryString(); // using...
public static class UrlExtensions {
public static string ToQueryString(this NameValueCollection nvc) {
return string.Join("&", nvc.GetUrlList());
}
public static IEnumerable<string> GetUrlList(this NameValueCollection nvc) {
foreach(var k in nvc.AllKeys) {
var values = nvc.GetValues(k);
if(values == null) { yield return k; continue; }
for(int i = 0; i < values.Length; i++) {
yield return
// 'gracefully' handle formatting
// when there's 1 or more values
string.Format(
values.Length > 1
// pick your array format: k[i]=v or k[]=v, etc
? "{0}[]={1}"
: "{0}={1}"
, k, HttpUtility.UrlEncode(values[i]), i);
}
}
}
}
As has been pointed out in comments already, with the exception of this answer most of the other answers address the scenario (Request.QueryString is an HttpValueCollection, "not" a NameValueCollection) rather than the literal question.
Update: addressed null value issue from comment.
The short answer is to use .ToString() on the NameValueCollection and combine it with the original url.
However, I'd like to point out a few things:
You cant use HttpUtility.ParseQueryString on Request.RawUrl. The ParseQueryString() method is looking for a value like this: ?var=value&var2=value2.
If you want to get a NameValueCollection of the QueryString parameters just use Request.QueryString().
var nv = Request.QueryString;
To rebuild the URL just use nv.ToString().
string url = String.Format("{0}?{1}", Request.Path, nv.ToString());
If you are trying to parse a url string instead of using the Request object use Uri and the HttpUtility.ParseQueryString method.
Uri uri = new Uri("<THE URL>");
var nv = HttpUtility.ParseQueryString(uri.Query);
string url = String.Format("{0}?{1}", uri.AbsolutePath, nv.ToString());
I always use UriBuilder to convert an url with a querystring back to a valid and properly encoded url.
var url = "http://my-link.com?foo=bar";
var uriBuilder = new UriBuilder(url);
var query = HttpUtility.ParseQueryString(uriBuilder.Query);
query.Add("yep", "foo&bar");
uriBuilder.Query = query.ToString();
var result = uriBuilder.ToString();
// http://my-link.com:80/?foo=bar&yep=foo%26bar
In AspNet Core 2.0 you can use QueryHelpers AddQueryString method.
As #Atchitutchuk suggested, you can use QueryHelpers.AddQueryString in ASP.NET Core:
public string FormatParameters(NameValueCollection parameters)
{
var queryString = "";
foreach (var key in parameters.AllKeys)
{
foreach (var value in parameters.GetValues(key))
{
queryString = QueryHelpers.AddQueryString(queryString, key, value);
}
};
return queryString.TrimStart('?');
}
This did the trick for me:
public ActionResult SetLanguage(string language = "fr_FR")
{
Request.UrlReferrer.TryReadQueryAs(out RouteValueDictionary parameters);
parameters["language"] = language;
return RedirectToAction("Index", parameters);
}
You can use.
var ur = new Uri("/page",UriKind.Relative);
if this nv is of type string you can append to the uri first parameter.
Like
var ur2 = new Uri("/page?"+nv.ToString(),UriKind.Relative);
I'm stumped by this easy data problem.
I'm using the Entity framework and have a database of products. My results page returns a paginated list of these products. Right now my results are ordered by the number of sales of each product, so my code looks like this:
return Products.OrderByDescending(u => u.Sales.Count());
This returns an IQueryable dataset of my entities, sorted by the number of sales.
I want my results page to show the rank of each product (in the dataset). My results should look like this:
Page #1
1. Bananas
2. Apples
3. Coffee
Page #2
4. Cookies
5. Ice Cream
6. Lettuce
I'm expecting that I just want to add a column in my results using the SQL ROW_NUMBER variable...but I don't know how to add this column to my results datatable.
My resulting page does contain a foreach loop, but since I'm using a paginated set I'm guessing using that number to fake a ranking number would NOT be the best approach.
So my question is, how do I add a ROW_NUMBER column to my query results in this case?
Use the indexed overload of Select:
var start = page * rowsPerPage;
Products.OrderByDescending(u => u.Sales.Count())
.Skip(start)
.Take(rowsPerPage)
.AsEnumerable()
.Select((u, index) => new { Product = u, Index = index + start });
Actually using OrderBy and then Skip + Take generates ROW_NUMBER in EF 4.5 (you can check with SQL Profiler).
I was searching for a way to do the same thing you are asking for and I was able to get what I need through a simplification of Craig's answer:
var start = page * rowsPerPage;
Products.OrderByDescending(u => u.Sales.Count())
.Skip(start)
.Take(rowsPerPage)
.ToList();
By the way, the generated SQL uses ROW_NUMBER > start and TOP rowsPerPage.
Try this
var x = Products.OrderByDecending(u => u.Sales.Count());
var y = x.ToList();
for(int i = 0; i < y.Count; i++) {
int myNumber = i; // this is your order number
}
As long as the list stays in the same order, which should happen unless the sales number changes. You could be able to get an accurate count;
There is also this way of doing it.
var page = 2;
var count = 10;
var startIndex = page * count;
var x = Products.OrderByDecending(u => u.Sales.Count());
var y = x.Skip(startIndex).Take(count);
This gives the start index for the page, plus it gives you a small set of sales to display on the page. You just start the counting on your website at startIndex.
Here is a long winded answer. First create a class to house the number/item pair like so:
public class NumberedItem<T>
{
public readonly int Number;
public readonly T Item;
public NumberedItem(int number, T item)
{
Item = item;
Number = number;
}
}
Next comes an abstraction around a page of items (numbered or not):
class PageOf<T> : IEnumerable<T>
{
private readonly int startsAt;
private IEnumerable<T> items;
public PageOf(int startsAt, IEnumerable<T> items)
{
this.startsAt = startsAt;
this.items = items;
}
public IEnumerable<NumberedItem<T>> NumberedItems
{
get
{
int index = 0;
foreach (var item in items)
yield return new NumberedItem<T>(startsAt + index++, item);
yield break;
}
}
public IEnumerator<T> GetEnumerator()
{
foreach (var item in items)
yield return item;
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
}
Once you have that you can "Paginate" a particular queryable collection using this:
class PaginatedQueryable<T>
{
private readonly int PageSize;
private readonly IQueryable<T> Source;
public PaginatedQueryable(int PageSize, IQueryable<T> Source)
{
this.PageSize = PageSize;
this.Source = Source;
}
public PageOf<T> Page(int pageNum)
{
var start = (pageNum - 1) * PageSize;
return new PageOf<T>(start + 1, Source.Skip(start).Take(PageSize));
}
}
And finally a nice extension method to cover the ugly:
static class PaginationExtension
{
public static PaginatedQueryable<T> InPagesOf<T>(this IQueryable<T> target, int PageSize)
{
return new PaginatedQueryable<T>(PageSize, target);
}
}
Which means you can now do this:
var products = Products.OrderByDescending(u => u.Sales.Count()).InPagesOf(20).Page(1);
foreach (var product in products.NumberedItems)
{
Console.WriteLine("{0} {1}", product.Number, product.Item);
}