I have been using Linq to SQL for a while now on one of my sites and over time the code I am using to query the database has gotten a little messy so I decided to re-write, originally my queries were all handled exclusively by Linq but recently there has been a demand for more advanced search features which has led me more towards using ExecuteQuery and handwriting my SQL statements the problem is that I cannot for the life of me get the Join statement to work properly.
I have two tables in my databases, t_events and t_clients. The only thing similar between the two tables is that they both have a clientid field (the id of the client the event is for). What I need to be able to do is pull all of the events into the page (which works fine) but I dont want to show the clientid on the page I need to show the client name. Originally I had a join clause that handled this nicely:
var eve = from p in db.t_events
join c in db.Clients on p.clientid equals c.clientid
where p.datedue <= thisDay && p.status != "complete"
select new { p.eventname, p.datedue, p.details, p.eventid, p.status, c.clientname };
With the redesign of the page however I am having issues recreating what linq has done here with the join. My current code:
StringBuilder sqlQuery = new StringBuilder("SELECT * FROM dbo.t_events JOIN dbo.t_clients ON dbo.t_events.clientid=dbo.t_clients.clientid");
var query = db.ExecuteQuery<t_events>(sqlQuery.ToString());
foreach (var c in query)
{
counter = counter + 1;
MyStringBuilder.Append("<tr class='"+c.status+"'><td><a href='searchdetails.aspx?id="+c.eventid+"'>"+c.eventname+"</a></td><td>" +c.clientname+ "</td></tr>");
}
in the foreach loop I have you can see I am trying to pull in c.clientname (which doesnt work) as it is not on the t_events database, changing this to c.clientid makes the code work, I am not sure what the issue is as taking that same SQL and running the query directly off the sql management tool works like a charm. Any ideas on this issue would be greatly appreciated!
FIXED!
DaveMarkle suggested using a view, which was by far a much easier way of doing this. I created a view that joins the two tables together with the fields I need and run my queries against it, simple and effective, I thank you!
Erm - so maybe we should have an answer here then so the question drops off the 'unanswered' list.
As Dave Markle stated.
Use a view.
Another option!
Execute the query twice: once with db.ExecuteQuery<t_events> and once db.ExecuteQuery<t_clients>. Now that you have both events and clients you can re-join them client-side by matching client_id.
Related
I first created a SQL query and now I'm converting it into an AOT query in AX 2012 R3. I have everything else working except for one thing. I have a join in my SQL that is: JOIN dbo.INVENTJOURNALTABLE AS INV3 ON INV.INVENTBATCHID LIKE left(INV3.DESCRIPTION,17). In SQL this works and returns the data that I need.
I don't know how to create this join in the AOT query though.
That INV3.DESCRIPTION is a string that contains the InventBatchID.
Is there a way to do this kind of join?
In order to accomplish what you want with a Query object, it's difficult/unusual, and AX isn't designed to do this in a straight forward way and requires a very creative solution.
I would generally push towards writing a pure x++ query as I don't believe LEFT and LIKE can be natively combined, especially in a query. You can use * in the value for "like" as an option.
You may be able to accomplish using expressions in query ranges somehow.
If you must have a query, a way I can think is by combining a View, Computed Column, and a Query...and I can't guarantee it will work, but I can give you enough to have something to test with. Some of the information in this answer is pretty concentrated so look closely at everything to understand it.
Create a View, add the below computed column, then add it to a Query as pictured below. For the computed column, you need to add a new String field to the view and set the ViewMethod property to the name of the method. See here for more info about adding Computed Columns.
The Computed Column accomplishes the left(..., 17) part and you can browse the view to confirm.
The last part is trying to join either by a relation (pictured below, but it does not accomplish the like) or setting the Value property using an expression by following the link above. You may need to create a custom expression in \Classes\SysQueryRangeUtil. You have some experimenting to do to see if it works.
private static server str compColDescLeft()
{
#define.ViewName(InventJournalTableView)
#define.DataSourceName("InventJournalTable_1")
#define.FieldDescription("Description")
#define.LeftCount("17")
str sReturn;
str sLeftDesc;
DictView dictView;
dictView = new DictView(tableNum(#ViewName));
sLeftDesc = dictView.computedColumnString(#DataSourceName, #FieldDescription, FieldNameGenerationMode::FieldList, true);
sReturn = "left(" + sLeftDesc + ", " + #LeftCount + ")";
return sReturn;
}
So far, I've been using classic ADO.NET model for database access. I have to tell you that I'm quite happy with it. But I have also been hearing much about Entity Framework recently so I thought I could give it a try. Actually the main reason which pushed me was the need to find a way to build the WHERE clause of my Stored Procedures. With the classic way I have to do either of the following:
Build the WHERE clause in the client side based on the user inputs and send it as a VARCHAR2 argument to the Stored Procedure, concatenate the WHERE clausewith the main part of the SQL and pass the whole string to EXECUTE_IMMEDIATE function. I personally hate to have to do so.
Inside the Stored Procedure construct lots and lots of SQL statements, which means I have to take all the possible combinations that WHERE clause might be composed of into account. This seems worse than the first case.
I know that EF has made it possible to use Stored Procedures as well. But will it be possible to build the WHERE part dynamically? Can EF rescue me somehow?
yes, you can use Dynamic queries in Linq.
Dynamic Query LIbrary
from scott gu example
var query = Northwind.Products.Where("Lastname LIKE "someValue%");
or some complex query
var query =
db.Customers.
Where("City = #0 and Orders.Count >= #1", "London", 10).
OrderBy("CompanyName").
Select("new(CompanyName as Name, Phone)");
or from this answer Where clause dynamically.
var pr = PredicateBuilder.False<User>();
foreach (var name in names)
{
pr = pr.Or(x => x.Name == name && x.Username == name);
}
return query.AsExpandable().Where(pr);
I am thinking of using Simple.Data Micro-ORM for my ASP.NET 4.5 website. However, there is something that I need to know before deciding whether to use it or not.
Let's take the following Join query for example:
var albums = db.Albums.FindAllByGenreId(1)
.Select(
db.Albums.Title,
db.Albums.Genre.Name);
This query will be translated to:
select
[dbo].[Albums].[Title],
[dbo].[Genres].[Name]
from [dbo].[Albums]
LEFT JOIN [dbo].[Genres] ON ([dbo].[Genres].[GenreId] = [dbo].[Albums].[GenreId])
WHERE [dbo].[Albums].[GenreId] = #p1
#p1 (Int32) = 1
Let's assume that the 'Genres' table is a a table with thousands or even millions of rows. I think that it might be very inefficient to filter the data after the JOIN has taken place, which is what this query translated for in Simple.Date.
Would it be better to filter the data firs in the Generes table, which means create make a SELECT statement first and make the JOIN with that filtered table?
Wouldn't it be better to filter the data ahead of time?
Furthermore, is there an option to make that type of complex (JOIN on a filtered table) query using Simple.Data.
Need your answer to know if to proceed with Simple.Data, or damp it in favor of another micro-ORM.
You are confused about how SQL is interpreted and executed by the database engine. Modern databases are incredibly smart about the best way to execute queries, and the order in which instructions appear in SQL statements has nothing to do with the order in which they are executed.
Try running some queries through SQL Management Studio and looking at the Execution Plan to see how they are actually optimised and executed. Or just try the SQL you think would work better and see how it actually performs compared to what is generated by Simple.Data.
The sql that Simple.Data is generating is idomatic T-SQL, too be honest its what I would be writing if I was drafting the sql myself.
This sql allows Sql Server to optimise the execution plan which should mean the most efficient retrieval of data.
The beauty of Simple.Data is that if you have any doubts or issues with the sql it generates you can just call a stored proc:
db.ProcedureWithParameters(1, 2);
I am using Linq to Sql as my ORM and I have a list of Ids (up to a few thousand) passed into my retriever method, and with that list I want to grab all User records that correspond to those unique Ids. To clarify, imagine I have something like this:
List<IUser> GetUsersForListOfIds(List<int> ids)
{
using (var db = new UserDataContext(_connectionString))
{
var results = (from user in db.UserDtos
where ids.Contains(user.Id)
select user);
return results.Cast<IUser>().ToList();
}
}
Essentially that gets translated into sql as
select * from dbo.Users where userId in ([comma delimmited list of Ids])
I'm looking for a more efficient way of doing this. The problem is the in clause in sql seems to take too long (over 30 seconds).
Will need more information on your database setup like index's and type of server (Mitch Wheat's post). Type of database would help as well, some databases handle in clauses poorly.
From a trouble shooting standpoint...have you isolated the time delay to the sql server? Can you run the query directly on your server and confirm it's the query taking the extra time?
Select * can also have a bit of a performance impact...could you narrow down the result set that's being returned to just the columns you require?
edit: just saw the 'view comment' that you added...I've had problems with view performance in the past. Is it a materialized view...or could you make it into one? Recreating the view logic as a stored procedure may aslo help.
Have you tried converting this to a list, so the application is doing this in-memory? i.e.:
List<IUser> GetUsersForListOfIds(List<int> ids)
{
using (var db = new UserDataContext(_connectionString))
{
var results = (from user in db.UserDtos.ToList()
where ids.Contains(user.Id)
select user);
return results.Cast<IUser>().ToList();
}
}
This will obviously be memory-intensive if this is being run on a public-facing page on a hard-hit site. If this still takes 30+ seconds though in staging/development, then my guess is that the View itself takes that long to process -OR- you're transferring 10's of MB of data each time you retrieve the view. Either way, my only suggestions are to access the table directly and only retrieve the data you need, rewrite the view, or create a new view for this particular scenario.
I just started learning LINQ to SQL, and so far I'm impressed with the easy of use and good performance.
I used to think that when doing LINQ queries like
from Customer in DB.Customers where Customer.Age > 30 select Customer
LINQ gets all customers from the database ("SELECT * FROM Customers"), moves them to the Customers array and then makes a search in that Array using .NET methods. This is very inefficient, what if there are hundreds of thousands of customers in the database? Making such big SELECT queries would kill the web application.
Now after experiencing how actually fast LINQ to SQL is, I start to suspect that when doing that query I just wrote, LINQ somehow converts it to a SQL Query string
SELECT * FROM Customers WHERE Age > 30
And only when necessary it will run the query.
So my question is: am I right? And when is the query actually run?
The reason why I'm asking is not only because I want to understand how it works in order to build good optimized applications, but because I came across the following problem.
I have 2 tables, one of them is Books, the other has information on how many books were sold on certain days. My goal is to select books that had at least 50 sales/day in past 10 days. It's done with this simple query:
from Book in DB.Books where (from Sale in DB.Sales where Sale.SalesAmount >= 50 && Sale.DateOfSale >= DateTime.Now.AddDays(-10) select Sale.BookID).Contains(Book.ID) select Book
The point is, I have to use the checking part in several queries and I decided to create an array with IDs of all popular books:
var popularBooksIDs = from Sale in DB.Sales where Sale.SalesAmount >= 50 && Sale.DateOfSale >= DateTime.Now.AddDays(-10) select Sale.BookID;
BUT when I try to do the query now:
from Book in DB.Books where popularBooksIDs.Contains(Book.ID) select Book
It doesn't work! That's why I think that we can't use thins kinds of shortcuts in LINQ to SQL queries, like we can't use them in real SQL. We have to create straightforward queries, am I right?
You are correct. LINQ to SQL does create the actual SQL to retrieve your results.
As for your shortcuts, there are ways to work around the limitations:
var popularBooksIds = DB.Sales
.Where(s => s.SalesAmount >= 50
&& s.DateOfSale >= DateTime.Now.AddDays(-10))
.Select(s => s.Id)
.ToList();
// Actually should work.
// Forces the table into memory and then uses LINQ to Objects for the query
var popularBooksSelect = DB.Books
.ToList()
.Where(b => popularBooksIds.Contains(b.Id));
Yes, query gets translated to a SQL string, and the underlying SQL can be different depending on what you are trying to do... so you have to be careful in that regard. Checkout a tool called linqpad, you can try your query in it and see the executing SQL.
Also, it runs when iterating through the collection or calling a method on it like ToList().
Entity framework or linq queries can be tricky sometimes. Sometimes you are surprised at the efficiency of the sql query generated and sometimes the query is so complicated and inefficient that you would smack your forehead.
Best idea is that if you have any suspicions about a query, run an sql profiler at the backend that would monitor all the queries coming in. That way you know exactly what is being passed on to the sql server and correct any inefficiencies if need be.
http://damieng.com/blog/2008/07/30/linq-to-sql-log-to-debug-window-file-memory-or-multiple-writers
This will help you to see what and when queries are being run. Also, Damiens blog is full of other linq to sql goodness.
You can generate an EXISTS clause by using the .Any method. I have had more success that way than trying to generate IN clauses, because it likes to retrieve all the data and pass it all back in as parameters to a query
In linq to sql, IQueryable expression fragments can be combined to create a single query, it will try to keep everything as an IQueryable for as long as it can, before you do something that cannot be expressed in SQL. When you call ToList you are directly asking it to resolve that query into an IEnumerable stored in memory.
In most cases you are better off not selecting the book ids in advance. Keep the fragment for popular books in a single place in the code and use it when necessary, to build on another query. An IQueryable is just an expression tree, which is resolved into SQL at some other point.
If you think your application will perform better by storing the popular books elsewhere (memcache or whatever), then you may consider pulling them out before hand, and checking against that later. This will mean each book id will be passed in as a sproc parameter and used in an IN clause.