I've got two un-related (no FK's defined) tables. The first table contains some tasks for which a user may not have access. I need to find all those tasks - in this case, the joined table would contain nulls. How do I get them?
Here's the setup:
TimeData table
- userID
- taskID
- hours
ApprovedTasks table (the one that should contain nulls)
- taskID
- userID
The SQL query would look like this:
select * from TimeData td
left join ApprovedTasks at
on at.taskID = td.taskID and at.userID = td.userID
where at.taskID is null
Any way to pull that off using a LINQ to Entity query?
TIA
Check out... Disjoint Union in LINQ
This should work...
var approvedTaks = from at in ApprovedTasks.Except(
from at2 in ApprovedTasks
where at2.userID == userId and at2.taskID==taskId
select at2)
where at.userID == userId and at.taskID==taskId
select at;
but sorry don't have the database handy to test it.
Related
Edit
I would rather explicitly state my problem I'm facing rather than assuming the approach. There could be an easiest solution to my problem.
I need to select records by joining two different table based on the result from another table. And I have to use different joins based on the result from the first table.
If a particular record is present in the first table, I have two use inner join the first table whereas if it's not present, then I have to left join.
bool recordPresent = select exists (select * from firstTable where access_id = 13) as access
if (recordPresent)
results = select * from secondTable s left join firstTable f on f.access_id = s.access_id where f.access_id is null order by access_id
else
results = select * from secondTable s inner join firstTable f on f.access_id = s.access_id
I am comming to ASP .NET Core from PHP w/ MySQL.
The problem:
For the illustration, suppose the following two tables:
T: {ID, Description, FK} and States: {ID, ID_T, Time, State}. There is 1:n relationship between them (ID_T references T.ID).
I need all the records from T with some specific value of FK (lets say 1) along with the related newest record in States (if any).
In terms of SQL it can be written as:
SELECT T.ID, T.Description, COALESCE(s.State, 0) AS 'State' FROM T
LEFT JOIN (
SELECT ID_T, MAX(Time) AS 'Time'
FROM States
GROUP BY ID_T
) AS sub ON T.ID = sub.ID_T
LEFT JOIN States AS s ON T.ID = s.ID_T AND sub.Time = s.Time
WHERE FK = 1
I am struggling to write an efficient equivalent query in LINQ (or the fluent API). The best working solution I've got so far is:
from t in _context.T
where t.FK == 1
join s in _context.States on t.ID equals o.ID_T into _s
from s in _s.DefaultIfEmpty()
let x = new
{
id = t.ID,
time = s == null ? null : (DateTime?)s.Time,
state = s == null ? false : s.State
}
group x by x.id into x
select x.OrderByDescending(g => g.time).First();
When I check the resulting SQL query in the output window when executed it is just like:
SELECT [t].[ID], [t].[Description], [t].[FK], [s].[ID], [s].[ID_T], [s].[Time], [s].[State]
FROM [T] AS [t]
LEFT JOIN [States] AS [s] ON [T].[ID] = [s].[ID_T]
WHERE [t].[FK] = 1
ORDER BY [t].[ID]
Not only it selects more columns than I need (in the real scheme there are more of them). There is no grouping in the query so I suppose it selects everything from the DB (and States is going to be huge) and the grouping/filtering is happening outside the DB.
The questions:
What would you do?
Is there an efficient query in LINQ / Fluent API?
If not, what workarounds can be used?
Raw SQL ruins the concept of abstracting from a specific DB technology and its use is very clunky in current Entity Framework Core (but maybe its the best solution).
To me, this looks like a good example for using a database view - again, not really supported by Entity Framework Core (but maybe its the best solution).
What happens if you try to do a more straight forward translation to LINQ?
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID_T = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID_T into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States on new { t.ID, sub.Time } equals new { s.ID, s.Time } into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? 0 : s.State) };
Apparently the ?? operator will translate to COALESCE and may handle an empty table properly, so you could replace the select with:
select new { t.ID, t.Description, State = s.State ?? 0 };
OK. Reading this article (almost a year old now), Smit's comment to the original question and other sources, it seems that EF Core is not really production ready yet. It is not able to translate grouping to SQL and therefore it is performed on the client side, which may be (and in my case would be) a serious problem. It corresponds to the observed behavior (the generated SQL query does no grouping and selects everything in all groups). Trying the LINQ queries out in Linqpad it always translates to a single SQL query.
I have downgraded to EF6 following this article. It required some changes in my model's code and some queries. After changing .First() to .FirstOrDefault() in my original LINQ query it works fine and translates to a single SQL query selecting only the needed columns. The generated query is much more complex than it is needed, though.
Using a query from NetMage's answer (after small fixes), it results in a SQL query almost identical to my own original SQL query (there's only a more complex construct than COALESCE).
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States
on new { ID_T = t.ID, sub.Time } equals new { s.ID_T, s.Time }
into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? false : s.State) };
In LINQ it's not as elegant as my original SQL query but semantically it's the same and it does more or less the same thing on the DB side.
In EF6 it is also much more convenient to use arbitrary raw SQL queries and AFAIK also the database views.
The biggest downside of this approach is that full .NET framework has to be targeted, EF6 is not compatible with .NET Core.
I have the following columns in a SQLite DB.
id,ts,origin,product,bid,ask,nextts
1,2016-10-18 20:20:54.733,SourceA,Dow,1.09812,1.0982,
2,2016-10-18 20:20:55.093,SourceB,Oil,7010.5,7011.5,
3,2016-10-18 20:20:55.149,SourceA,Dow,18159.0,18161.0,
How can I populate the 'next timestamp' column (nextts) with the next timestamp for the same product (ts), from the same source? I've been trying the following, but I can't seem to put a subquery in an UPDATE statement.
UPDATE TEST a SET nextts = (select ts
from TEST b
where b.id> a.id and a.origin = b.origin and a.product = b.product
order by id asc limit 1);
If I call this, I can display it, but I haven't found a way of updating the value yet.
select a.*,
(select ts
from TEST b
where b.id> a.id and a.origin = b.origin and a.product = b.product
order by id asc limit 1) as nextts
from TEST a
order by origin, a.id;
The problem is that you're using table alias for table in UPDATE statement, which is not allowed. You can skip alias from there and use unaliased (but table-name prefixed) reference to its columns (while keeping aliased references for the SELECT), like this:
UPDATE TEST
SET nextts = (
SELECT b.ts
FROM TEST b
WHERE b.id > TEST.id AND
TEST.origin = b.origin AND
TEST.product = b.product
ORDER BY b.id ASC
LIMIT 1
);
Prefixing unaliased column references with the table name is necessary for SQLite to identify that you're referencing to unaliased table. Otherwise the id column whould be understood as the id from the closest[*] possible data source, in which case it's the aliased table (as b alias), while we're interested in the unaliased table, therefore we need to explicitly tell SQLite that.
[*] Closest data source is the one listed in the same query, or parent query, or parent's parent query, etc. SQLite is looking for the first data source (going from inner part to the outside) in the query hierarchy that defines this column.
As far as I know, doctrine dql does not allow subqueries inside a join.
I have a table and a trasnlation table. The relation is one to many. One record has many translations.
In order to get the right translation row from the translatiosn table I did subselects in the select clause:
$query = $this->getDoctrine()->getEntityManager()
->createQuery('
SELECT w.id, w.pastid, w.name, w.jsonParameters as params, m.id as milestone_id, m.name as milestone_name,
m.slug as milestone_slug, m.startdate as milestone_start, m.enddate as milestone_end,
w.expand as expand, w.backgroundcolor as background, w.colorschema as colorschema, w.headline as headline, w.subheadline as subheadline, w.text as text, w.expandheight as expandheight,
w.url as url, w.created as created, w.updated as updated, wt.name as wtname, ws.weight as width, ws.height as height
, (SELECT t.headline FROM AdminBundle:widgetTranslation t WHERE t.widget = w.id and t.locale = :published) AS headline_trans
, (SELECT t2.subheadline FROM AdminBundle:widgetTranslation t2 WHERE t2.widget = w.id and t2.locale = :published) AS subheadline_trans
, (SELECT t3.text FROM AdminBundle:widgetTranslation t3 WHERE t3.widget = w.id and t3.locale = :published) AS text_trans
FROM AdminBundle:Widget w
JOIN w.milestone m
JOIN w.widgetType wt
JOIN w.widgetShape ws
WHERE w.published = 1
ORDER BY m.order, w.order
')->setParameter('published', $currentLocale);
$result = $query->getArrayResult();
This query does the job but Iam worried about performance, is there a better query to do this?
First, I need to clarify a little thing.
There is something really wrong in your question.
You are using an ORM, right? But you're talking about tables.
An ORM doesn't care about how your tables are organized or linked together. It only matters on the relationship between the objects themselves.
However, if you are worried about performance, the best thing to do, as you would do it with any SQL query, is to EXPLAIN it.
Since you're using Symfony 2.0, you may use the profiler (accessible in dev env) to see the EXPLAIN results of your query.
Make sure that is using index, and the number of rows to lookup is not too high.
Depending on your result you may need to use another strategy to fetch your translation or re-write your query in native SQL to allow more flexibility.
I need to get column names and their tables in a SQLite database. What I need is a resultset with 2 columns: table_name | column_name.
In MySQL, I'm able to get this information with a SQL query on database INFORMATION_SCHEMA. However the SQLite offers table sqlite_master:
sqlite> create table students (id INTEGER, name TEXT);
sqlite> select * from sqlite_master;
table|students|students|2|CREATE TABLE students (id INTEGER, name TEXT)
which results a DDL construction query (CREATE TABLE) which is not helpful for me and I need to parse this to get relevant information.
I need to get list of tables and join them with columns or just get columns along with table name column. So PRAGMA table_info(TABLENAME) is not working for me since I don't have table name. I want to get all column metadata in the database.
Is there a better way to get that information as a result set by querying database?
You've basically named the solution in your question.
To get a list of tables (and views), query sqlite_master as in
SELECT name, sql FROM sqlite_master
WHERE type='table'
ORDER BY name;
(see the SQLite FAQ)
To get information about the columns in a specific table, use PRAGMA table_info(table-name); as explained in the SQLite PRAGMA documentation.
I don't know of any way to get tablename|columnname returned as the result of a single query. I don't believe SQLite supports this. Your best bet is probably to use the two methods together to return the information you're looking for - first get the list of tables using sqlite_master, then loop through them to get their columns using PRAGMA table_info().
Recent versions of SQLite allow you to select against PRAGMA results now, which makes this easy:
SELECT
m.name as table_name,
p.name as column_name
FROM
sqlite_master AS m
JOIN
pragma_table_info(m.name) AS p
ORDER BY
m.name,
p.cid
where p.cid holds the column order of the CREATE TABLE statement, zero-indexed.
David Garoutte answered this here, but this SQL should execute faster, and columns are ordered by the schema, not alphabetically.
Note that table_info also contains
type (the datatype, like integer or text),
notnull (1 if the column has a NOT NULL constraint)
dflt_value (NULL if no default value)
pk (1 if the column is the table's primary key, else 0)
RTFM: https://www.sqlite.org/pragma.html#pragma_table_info
There are ".tables" and ".schema [table_name]" commands which give kind of a separated version to the result you get from "select * from sqlite_master;"
There is also "pragma table_info([table_name]);" command to get a better result for parsing instead of a construction query:
sqlite> .tables
students
sqlite> .schema students
create table students(id INTEGER, name TEXT);
sqlite> pragma table_info(students);
0|id|INTEGER|0||0
1|name|TEXT|0||0
Hope, it helps to some extent...
Another useful trick is to first get all the table names from sqlite_master.
Then for each one, fire off a query "select * from t where 1 = 0". If you analyze the structure of the resulting query - depends on what language/api you're calling it from - you get a rich structure describing the columns.
In python
c = ...db.cursor()
c.execute("select * from t where 1=0");
c.fetchall();
print c.description;
Juraj
PS. I'm in the habit of using 'where 1=0' because the record limiting syntax seems to vary from db to db. Furthermore, a good database will optimize out this always-false clause.
The same effect, in SQLite, is achieved with 'limit 0'.
FYI, if you're using .Net you can use the DbConnection.GetSchema method to retrieve information that usually is in INFORMATION_SCHEMA. If you have an abstraction layer you can have the same code for all types of databases (NOTE that MySQL seems to swich the 1st 2 arguments of the restrictions array).
Try this sqlite table schema parser, I implemented the sqlite table parser for parsing the table definitions in PHP.
It returns the full definitions (unique, primary key, type, precision, not null, references, table constraints... etc)
https://github.com/maghead/sqlite-parser
The syntax follows sqlite create table statement syntax: http://www.sqlite.org/lang_createtable.html
This is an old question but because of the number of times it has been viewed we are adding to the question for the simple reason most of the answers tell you how to find the TABLE names in the SQLite Database
WHAT DO YOU DO WHEN THE TABLE NAME IS NOT IN THE DATABASE ?
This is happening to our app because we are creating TABLES programmatically
So the code below will deal with the issue when the TABLE is NOT in or created by the Database Enjoy
public void toPageTwo(View view){
if(etQuizTable.getText().toString().equals("")){
Toast.makeText(getApplicationContext(), "Enter Table Name\n\n"
+" OR"+"\n\nMake Table First", Toast.LENGTH_LONG
).show();
etQuizTable.requestFocus();
return;
}
NEW_TABLE = etQuizTable.getText().toString().trim();
db = dbHelper.getWritableDatabase();
ArrayList<String> arrTblNames = new ArrayList<>();
Cursor c = db.rawQuery("SELECT name FROM sqlite_master WHERE
type='table'", null);
if (c.moveToFirst()) {
while ( !c.isAfterLast() ) {
arrTblNames.add( c.getString( c.getColumnIndex("name")) );
c.moveToNext();
}
}
c.close();
db.close();
boolean matchFound = false;
for(int i=0;i<arrTblNames.size();i++) {
if(arrTblNames.get(i).equals(NEW_TABLE)) {
Intent intent = new Intent(ManageTables.this, TableCreate.class
);
startActivity( intent );
matchFound = true;
}
}
if (!matchFound) {
Toast.makeText(getApplicationContext(), "No Such Table\n\n"
+" OR"+"\n\nMake Table First", Toast.LENGTH_LONG
).show();
etQuizTable.requestFocus();
}
}