I never should've expected that knowing mySQL I'd be safe using Access.
I have two tables: users and scores
users table contains: id(auto increment primary key), username, password, etc..
scoers table contains: id(number - foreign key to users.id), highScore
I've previously asked help for INSERT command, which now works as it should. Now I've got issues with a similar UPDATE command.
The non-working command looks like this:
string updateCommand = #"UPDATE scores
SET
id = (SELECT id FROM users WHERE username = #username),
highScore = #score
WHERE highScore = (SELECT MIN(highScore) FROM scores);";
which throws a: Operation must use an updateable query.
To rationalize what I'm trying to accomplish here: I'm INSERT-ing high scores until I reach 10 scores in the table, afterwards instead of adding any new scores and filling up the database needlessly I decided It'd be more sensible to just "overwrite" the currently lowest score using UPDATE.
I am supplied a username and the high score and since the scores table contains only id I need to reach the id of the current user so that's what the first subquery is doing, the second subquery in the WHERE clause is to specify which score to replace (though there is possibly a bug here if there are multiple people with the lowest score, any ideas how to fix that?)
I've also tried using OUTER RIGHT JOIN like this:
string updateCommand = #"UPDATE scores
OUTER RIGHT JOIN users ON scores.id = users.id
SET
scores.id = users.id,
scores.highScore = #score
WHERE (highScore = (SELECT MIN(highScore) FROM scores)) AND (username = #username);";
With no luck(I get a generic Syntax error in UPDATE statement.).
Browsing the net I've found that I possibly "can't" use subqueries in UPDATE statements but I seem to find conflicting opinions on the matter.
I've also tried using the DLookup function in place of subqueries like:
#"...
id = DLookup(""id"", ""users"", ""username = #username""),
...
WHERE highScore = DLookup(""MIN(highScore)"", ""scores"");";
elipses represent extraneous code which is identical to the code above.
Also as a last resort I've tried dividing into multiple queries however userId query which looks like this:
string userIdQuery = "SELECT id FROM users WHERE username = #username"
seems to return a null judging by the NullReferenceException i recieve (Object reference not set to an instance of an object.) when trying to use the variable userId after I've done this:
int userId = 0;
userId = (Int32)command.ExecuteScalar();
I'm supposed to get an integer however I get a null I think. The almost identical query for getting the minimum highscore works flawlessly and the int variable is filled with the correct value so I'm assuming that hte problem is in the query itself somehow. I've tried adding single quotes around the #username parameter assuming that it might not be recognizing the string but it seems that's not it.
Phew.. took me a while to write this. Anyone got any ideas on how to make this all work? If you need more info let me know.
So after some messing around I've found out the causes of my troubles. The bad side is that I increased the amount of code so that I'd avoid subqueries as much as possible since, at least from my experience, Access doesn't really like the use of subqueries in UPDATE or INSERT commands.
What I did first is split the command into 3 separate ones:
"SELECT id FROM users WHERE username = ?;" - To get the id of the user whose score
I'm putting in the database.
#"SELECT scores.id, scores.highScore, scores.dateTime FROM scores WHERE (((scores.highScore)=DMin(""highScore"",""scores"")));" - which gets the id, high score
and time when the entry was... well entered, of the lowest score currently in the high scores list. Thanks to a suggestion from HansUp I used DMin function instead of a subquery with MIN to avoid the Must use an updateable query error. The extraneous parentheses are due to Access since this command was generated by the Access query designer and I'm too afraid to change anything lest I break it.
#"UPDATE scores SET scores.id = ?, scores.highScore = ?, scores.[dateTime] = Now() WHERE (((scores.id)=?) AND ((scores.highScore)=?) AND ((scores.dateTime)=?));" - The update command itself, not much to say here except that it takes the previously extracted data and uses it as values for the command.
One thing I noticed is that even if I got the command working the .ExecuteNonQuery() would always return 0 rows affected. After poking around I found out that named parameters for commands in ASP.NET / C# don't always work and that instead ? should be used instead. It's kind of inconvenient but I can't complain too much.
Related
I've been attempting to increase my knowledge and trying out some challenges. I've been going at this for a solid two weeks now finished most of the challenge but this one part remains. The error is shown below, what am i not understanding?
Error in sqlite query: update users set last_browser= 'mozilla' + select sql from sqlite_master'', last_time= '13-04-2019' where id = '14'
edited for clarity:
I'm trying a CTF challenge and I'm completely new to this kind of thing so I'm learning as I go. There is a login page with test credentials we can use for obtaining many of the flags. I have obtained most of the flags and this is the last one that remains.
After I login on the webapp with the provided test credentials, the following messages appear: this link
The question for the flag is "What value is hidden in the database table secret?"
So from the previous image, I have attempted to use sql injection to obtain value. This is done by using burp suite and attempting to inject through the user-agent.
I have gone through trying to use many variants of the injection attempt shown above. Im struggling to find out where I am going wrong, especially since the second single-quote is added automatically in the query. I've gone through the sqlite documentation and examples of sql injection, but I cannot sem to understand what I am doing wrong or how to get that to work.
A subquery such as select sql from sqlite_master should be enclosed in brackets.
So you'd want
update user set last_browser= 'mozilla' + (select sql from sqlite_master''), last_time= '13-04-2019' where id = '14';
Although I don't think that will achieve what you want, which isn't clear. A simple test results in :-
You may want a concatenation of the strings, so instead of + use ||. e.g.
update user set last_browser= 'mozilla' || (select sql from sqlite_master''), last_time= '13-04-2019' where id = '14';
In which case you'd get something like :-
Thanks for everyone's input, I've worked this out.
The sql query was set up like this:
update users set last_browser= '$user-agent', last_time= '$current_date' where id = '$id_of_user'
edited user-agent with burp suite to be:
Mozilla', last_browser=(select sql from sqlite_master where type='table' limit 0,1), last_time='13-04-2019
Iterated with that found all tables and columns and flags. Rather time consuming but could not find a way to optimise.
Similar to this question and this solution for PostgreSQL (in particular "INSERT missing FK rows at the same time"):
Suppose I am making an address book with a "Groups" table and a "Contact" table. When I create a new Contact, I may want to place them into a Group at the same time. So I could do:
INSERT INTO Contact VALUES (
"Bob",
(SELECT group_id FROM Groups WHERE name = "Friends")
)
But what if the "Friends" Group doesn't exist yet? Can we insert this new Group efficiently?
The obvious thing is to do a SELECT to test if the Group exists already; if not do an INSERT. Then do an INSERT into Contacts with the sub-SELECT above.
Or I can constrain Group.name to be UNIQUE, do an INSERT OR IGNORE, then INSERT into Contacts with the sub-SELECT.
I can also keep my own cache of which Groups exist, but that seems like I'm duplicating functionality of the database in the first place.
My guess is that there is no way to do this in one query, since INSERT does not return anything and cannot be used in a subquery. Is that intuition correct? What is the best practice here?
My guess is that there is no way to do this in one query, since INSERT
does not return anything and cannot be used in a subquery. Is that
intuition correct?
You could use a Trigger and a little modification of the tables and then you could do it with a single query.
For example consider the folowing
Purely for convenience of producing the demo:-
DROP TRIGGER IF EXISTS add_group_if_not_exists;
DROP TABLE IF EXISTS contact;
DROP TABLE IF EXISTS groups;
One-time setup SQL :-
CREATE TABLE IF NOT EXISTS groups (id INTEGER PRIMARY KEY, group_name TEXT UNIQUE);
INSERT INTO groups VALUES(-1,'NOTASSIGNED');
CREATE TABLE IF NOT EXISTS contact (id INTEGER PRIMARY KEY, contact TEXT, group_to_use TEXT, group_reference TEXT DEFAULT -1 REFERENCES groups(id));
CREATE TRIGGER IF NOT EXISTS add_group_if_not_exists
AFTER INSERT ON contact
BEGIN
INSERT OR IGNORE INTO groups (group_name) VALUES(new.group_to_use);
UPDATE contact SET group_reference = (SELECT id FROM groups WHERE group_name = new.group_to_use), group_to_use = NULL WHERE id = new.id;
END;
SQL that would be used on an ongoing basis :-
INSERT INTO contact (contact,group_to_use) VALUES
('Fred','Friends'),
('Mary','Family'),
('Ivan','Enemies'),
('Sue','Work colleagues'),
('Arthur','Fellow Rulers'),
('Amy','Work colleagues'),
('Henry','Fellow Rulers'),
('Canute','Fellow Ruler')
;
The number of values and the actual values would vary.
SQL Just for demonstration of the result
SELECT * FROM groups;
SELECT contact,group_name FROM contact JOIN groups ON group_reference = groups.id;
Results
This results in :-
1) The groups (noting that the group "NOTASSIGNED", is intrinsic to the working of the above and hence added initially) :-
have to be careful regard mistakes like (Fellow Ruler instead of Fellow Rulers)
-1 used because it would not be a normal value automatically generated.
2) The contacts with the respective group :-
Efficient insertion
That could likely be debated from here to eternity so I leave it for the fence sitters/destroyers to decide :). However, some considerations:-
It works and appears to do what is wanted.
It's a little wasteful due to the additional wasted column.
It tries to minimise the waste by changing the column to an empty string (NULL may be even more efficient, but for some can be confusing)
There will obviously be an overhead BUT in comparison to the alternatives probably negligible (perhaps important if you were extracting every Facebook user) but if it's user input driven likely irrelevant.
What is the best practice here?
Fences again. :)
Note Hopefully obvious, but the DROP statements are purely for convenience and that all other SQL up until the INSERT is run once
to setup the tables and triggers in preparation for the single INSERT
that adds a group if necessary.
I have taken over some code and I see that database updates are performed like this:
dbcon = DependencyService.Get<ISQLite>().GetConnection();
public void UpdateAnswered(string id)
{
lock(locker)
{
dbcon.Query<Phrase>("UPDATE Phrase SET Answered = Answered + 1 " +
"WHERE Id = ?", id);
}
}
I am new to using SQLite with Xamarin but it looks strange to me that this update is handled with a dbcon.Query and that the table name is passed as . Can someone confirm is this the optimal way to handle a table update? Also why is it coded as a query with the table name being passed?
Update<T>
This method allows you to pass in an instance of an object that this stored in the database which has a primary key. SQLite then recognizes the primary key and updates the rest of the object's values.
You would just call connection.Update( phrase ); where the phrase is an instance of the Phrase class with properties you want to set. Be aware that all columns except ID will be updated.
Query<T>
Performs a query and returns the results. The type parameter specifies the type of the items returned. This is most appropriate for SELECT queries.
Execute
This returns the number of affected rows by the query as an int. This is probably the best choice for your UPDATE query after the Update<T> method.
ExecuteScalar<T>
Use for queries that return scalar types - like COUNT, etc., where T is the type of the value.
In summary, Update is the most natural way to update a row in the database (with an instance you have), but Query<T> and Execute<T> are very useful if you just want to UPDATE one column like in your example.
I'm experiencing a really odd result when I do a count in X++, something I've not experienced before. I am performing what I thought was a really simply count query, but I can't seem to get the result I am after.
WMSOrderTrans orderTrans;
WMSOrderTrans orderTransChk;
;
select count(RecId) from orderTrans group by shipmentid where orderTrans.inventTransRefId == 'XXXXXX';
info(strFmt('Count is %1', orderTrans.RecId));
while select orderTransChk group by shipmentid where orderTransChk.inventTransRefId == 'XXXXXX' {
info(strFmt('Shipment is %1', orderTransChk.shipmentId));
}
The data set that I am selecting all have only 1 shipmentid, so the first select I am expecting a count of 1, instead I get 4 (which is how many lines for that transrefid exist). If I change the count from 'RecId' to 'ShipmentId', then instead of the count, I get actual shipmentId. I simply want it to return the count of the records, which is what I believe I've asked it to do.
I really can't see what I am missing.
In the while select, I get what I expect (the shipmentid), only 1 infolog message for the loop. This tells me that the group by with the where clause is working, but it doesn't explain why the first count select statement isn't behaving as I would expect.
For reference, this is AX2012 R1 system.
For anyone who might be interested in knowing my answer, it's tied up with Jeff's response. At the end of the day, I didn't look at data well enough, and the query returned the correct results. I initially thought there were a number of unique shipments, but I was wrong. My expected result was erroneous. There were 4 lines in the file, but the lines were unique for the item, not the shipment. They were all on the same shipment. So really, my own fault, it goes to show that one really needs to look at the data closely.
Thanks to all that responded, greatly appreciated.
I would try a database sync, then restarting the AOS. I don't see anything obviously wrong, so it points to bouncing everything.
Try getting the select statement via this method (from memory so check syntax) and then review the query against SQL directly. It uses generateOnly.
select generateOnly count(RecId) from orderTrans group by shipmentid where orderTrans.inventTransRefId == 'XXXXXX';
info(orderTrans.getSQLStatement());
If I understand what you try to achieve, you'd like to get something like this SQL query:
select count(distinct shipmentid) from orderTrans
where inventTransRefId = 'XXXXXX'
The 'distinct' keyword is not available in AX select command. The group by clause will allow you to iterate all distinct values but not to use an aggregate on top of it.
You may use a sql connection to push the exact sql command you want.
In AX, aggregate values are stores in the field used: Count(RecId), the count will go in the recid field otherwise the system may need to add new field on the buffer on the fly. I don't think you can aggregate on the group by clause as it's important to have its value.
You can try (I don't have an AX to test it) to use a query:
Query query = new Query();
QueryRun queryRun;
QueryBuildDataSource qbd;
qbd = query.addDataSource(tablenum(OrderTrans));
qbd.addRange(fieldNum(OrderTrans, InventTransId)).value("xxxx");
qbd.addSortField(fieldNum(OrderTrans, ShipmentId));
qbd.SortOrder(SortOrder::GroupBy);
queryRun = new QueryRun(query);
info(strfmt("Total Records in Query %1",SysQuery::countTotal(queryRun)));
I'm creating a caching system to take data from an SQLite database table using a sorted/filtered query and display it. The tables I'm pulling from can be potentially very large and, of course, I need to minimize impact on memory by only retaining a maximum number of rows in memory at any given time. This is easily done by using LIMIT and OFFSET to load only the records I need and update the cache as needed. Implementing this is trivial. The problem I'm having is determining where the insertion index is for a new record inserted into a particular query so I can update my UI appropriately. Is there an easy way to do this? So far the ideas I've had are:
Dump the entire cache, re-count the Query results (there's no guarantee the new row will be included), refresh the cache and refresh the entire UI. I hope it's obvious why that's not really desirable.
Use my own algorithm to determine whether the new row is included in the current query, if it is included in the current cached results and at what index it should be inserted into if it's within the current cached scope. The biggest downfall of this approach is it's complexity and the risk that my own sorting/filtering algorithm won't match SQLite's.
Of course, what I want is to be able to ask SQLite: Given 'Query A' what is the index of 'Row B', without loading the entire query results. However, so far I haven't been able to find a way to do this.
I don't think it matters but this is all occurring on an iOS device, using the objective-c programming language.
More Info
The Query and subsequent cache is based off of user input. Essentially the user can re-sort and filter (or search) to alter the results they're seeing. My reticence in simply recreating the cache on insertions (and edits, actually) is to provide a 'smoother' UI experience.
I should point out that I'm leaning toward option "2" at the moment. I played around with creating my own caching/indexing system by loading all the records in a table and performing the sort/filter in memory using my own algorithms. So much of the code needed to determine whether and/or where a particular record is in the cache is already there, so I'm slightly predisposed to use it. The danger lies in having a cache that doesn't match the underlying query. If I include a record in the cache that the query wouldn't return, I'll be in trouble and probably crash.
You don't need record numbers.
Save the values of the ordered field in the first and last records of the LIMITed query result.
Then you can use these to check whether the new record falls into this range.
In other words, assuming that you order by the Name field, and that the original query was this:
SELECT Name, ...
FROM mytab
WHERE some_conditions
ORDER BY Name
LIMIT x OFFSET y
then try to get at the new record with a similar query:
SELECT 1
FROM mytab
WHERE some_conditions
AND PrimaryKey = LastInsertedValue
AND Name BETWEEN CachedMin AND CachedMax
Similarly, to find out before (or after) which record the new record was inserted, start directly after the inserted record and use a limit of one, like this:
SELECT Name
FROM mytab
WHERE some_conditions
AND Name > MyInsertedName
AND Name BETWEEN CachedMin AND CachedMax
ORDER BY Name
LIMIT 1
This doesn't give you a number; you still have to check where the returned Name is in your cache.
Typically you'd expect a cache to be invalidated if there were underlying data changes. I think dropping it and starting over will be your simplest, maintainable solution. I would recommend it unless you have a very good reason.
You could write another query that just returned the row count (example below) to see if your cache should be invalidated. That would save recreating the cache when it did not change.
SELECT name,address FROM people WHERE area_code=970;
SELECT COUNT(rowid) FROM people WHERE area_code=970;
The information you'd need from sqlite to know when your cache was invalidated would require some rather intimate knowledge of how the query and/or index was working. I would say that is fairly high coupling.
Otherwise, you'd want to know where it was inserted with regards to the sorting. You would probably key each page on the sorted field. Delete anything greater than the insert/delete field. Any time you change the sorting you'd drop everything.
Something like the below would be a start if you were using C++. I realize you aren't doing C++, but hopefully it is evident as to what I'm trying to do.
struct Person {
std::string name;
std::string addr;
};
struct Page {
std::string key;
std::vector<Person> persons;
struct Less {
bool operator()(const Page &lhs, const Page &rhs) const {
return lhs.key.compare(rhs.key) < 0;
}
};
};
typedef std::set<Page, Page::Less> pages_t;
pages_t pages;
void insert(const Person &person) {
if (sql_insert(person)) {
pages_t::iterator drop_cache_start = pages.lower_bound(person);
//... drop this page and everything after it
}
}
You'd have to do some wrangling to get different datatypes of key to work nicely, but its possible.
Theoretically you could just leave the pages out of it and only use the objects themselves. The database would no longer "own" the data though. If you only fill pages from the database, then you'll have less data consistency worries.
This may be a bit off topic, you aren't re-implementing views are you? It doesn't cache per se, but it isn't clear if that is a requirement of your project.
The solution I came up with is not exactly simple, but it's currently working well. I realized that the index of a record in a Query Statement is also the Count of all it's previous records. What I needed to do was 'convert' all the ORDER statements in the query to a series of WHERE statements that would return only the preceding records and take a count of those records. It's trickier than it sounds (or maybe not...it sounds tricky). The biggest issue I had was making sure the query was, in fact, sorted in a way I could predict. This meant I needed to have an order column in the Order Parameters that was based off of a column with unique values. So, whenever a user sorts on a column, I append to the statement another order parameter on a unique column (I used a "Modified Date Stamp") to break ties.
Creating the WHERE portion of the statement requires more than just tacking on a bunch of ANDs. It's easier to demonstrate. Say you have 3 Order columns: "LastName" ASC, "FirstName" DESC, and "Modified Stamp" ASC (the tie breaker). The WHERE statement would have to look something like this ('?' = record value):
WHERE
"LastName" < ? OR
("LastName" = ? AND "FirstName" > ?) OR
("LastName" = ? AND "FirstName" = ? AND "Modified Stamp" < ?)
Each set of WHERE parameters grouped together by parenthesis are tie breakers. If, in fact, the record values of "LastName" are equal, we must then look at "FirstName", and finally "Modified Stamp". Obviously, this statement can get really long if you're sorting by a bunch of order parameters.
There's still one problem with the above solution. Mathematical operations on NULL values always return false, and yet when you sort SQLite sorts NULL values first. Therefore, in order to deal with NULL values appropriately you've gotta add another layer of complication. First, all mathematical equality operations, =, must be replace by IS. Second, all < operations must be nested with an OR IS NULL to include NULL values appropriately on the < operator. This turns the above operation into:
WHERE
("LastName" < ? OR "LastName" IS NULL) OR
("LastName" IS ? AND "FirstName" > ?) OR
("LastName" IS ? AND "FirstName" IS ? AND ("Modified Stamp" < ? OR "Modified Stamp" IS NULL))
I then take a count of the RowID using the above WHERE parameter.
It turned out easy enough for me to do mostly because I had already constructed a set of objects to represent various aspects of my SQL Statement which could be assembled to generate the statement. I can't even imagine trying to manipulate a SQL statement like this any other way.
So far, I've tested using this on several iOS devices with up to 10,000 records in a table and I've had no noticeable performance issues. Of course, it's designed for single record edits/insertions so I don't really need it to be super fast/efficient.