I've overcome many hurdles while trying to work with SQLite on Blackberry. Now I've reached the point where everything seems to work fine and I'm looking for ways to speed up my queries.
Sometimes my app retrieves a lot of data from a web service call, this data is parsed and stored into my database. A lot of DELETE, INSERT and UPDATE going on. The database calls seem to be taking a lot of time.
I would like to know some best practices when dealing with SQLite. Preferably from people who have experience with it on the Blackberry platform specifically. Any tricks to speed up DELETEs or INSERTs etc....
Links to good tutorials would be great. Or some snippets of useful code even better.
Thanks in advance.
EDIT:
Here is some sample code from Blackberry using transactions.
import net.rim.device.api.ui.*;
import net.rim.device.api.ui.component.*;
import net.rim.device.api.ui.container.*;
import net.rim.device.api.database.*;
import net.rim.device.api.io.*;
public class UsingTransactions extends UiApplication
{
public static void main(String[] args)
{
UsingTransactions theApp = new UsingTransactions();
theApp.enterEventDispatcher();
}
public UsingTransactions()
{
}
}
class UsingTransactionsScreen extends MainScreen
{
Database d;
public UsingTransactionsScreen()
{
LabelField title = new LabelField("SQLite Using Transactions Sample",
LabelField.ELLIPSIS |
LabelField.USE_ALL_WIDTH);
setTitle(title);
add(new RichTextField(
"Updating data in one transaction in MyTestDatabase.db."));
try
{
URI myURI = URI.create("file:///SDCard/Databases/SQLite_Guide/" +
"MyTestDatabase.db");
d = DatabaseFactory.open(myURI);
d.beginTransaction();
Statement st = d.createStatement("UPDATE People SET Age=7 " +
"WHERE Name='Sophie'");
st.prepare();
st.execute();
st.close();
st = d.createStatement("UPDATE People SET Age=4 " +
"WHERE Name='Karen'");
st.prepare();
st.execute();
st.close();
d.commitTransaction();
d.close();
}
catch ( Exception e )
{
System.out.println( e.getMessage() );
e.printStackTrace();
}
}
}
Is there a reason why they're closing the statement every time?? Isn't it better just to close it once at the end (in a Finally block perhaps??).
The standard technique is to use a prepared insert statement, inside a loop inside a transaction. That will give you almost optimal efficiency in most instances. It will speed things up by at least an order of magnitude.
I personally don't know many blackberry-specific practices, but these seem to be helpful (sorry I couldn't find code snippets):
"Top 8 SQL Best Practices"
"Best Practice: Optimizing SQLite Database Performance"
Hope these help!
You should do one thing is that, create different methods for database handling code.
and simply call these methods from your main code and open and close database also in each and every block. It will reduce redundancy and will be better for your future. This will also helps you in your App performance.
Related
I want some kind of mechanism to have more information about a caught exception. (Specifically exceptions I throw myself to abort transactions) I've looked around and pretty much the only thing I could find was "Use the info log". This to me does not seem like a good idea. For one it is cumbersome to access and find the last message. And it is limited in size so at some point the new messages won't even show up.
So my idea is the following: Create a class NuException and pass an instance of that through all methods store an instance in the class where the work methods are located. When I need to throw an exception I call a method on it similar to Global::error() but this one takes an identifier and a message.
Once I reach my catch block I can access those from my object the class that contains the work methods similarly to how CLRExceptions work.
class NuException
{
"public" str identifier;
"public" str message;
public Exception error(str _id, str _msg)
{
//set fields
return Exception::Error;
}
}
class Worker
{
"public" NuException exception;
void foo()
{
throw this.exception.error("Foo", "Record Foo already exists");
}
void bar()
{
this.foo();
}
}
void Job()
{
Worker w = new Worker();
try
{
w.bar(ex);
}
catch (Exception::Error)
{
info(w.exception().message());
}
}
It works but isn't there a better way? Surely someone must have come up with a solution to work around this shortcoming in AX?
Short answer: yes.
While your "brilliant" scheme "works", it gets boring pretty fast, as you now must transport your NuException object deep down 20 level from the listener (job) to the thrower (foo). Your bar method and other middle men has no interest or knowledge about your exception scheme but must pass it on anyway.
This is no longer the case after the update.
There are several ways to go.
Use an observer pattern like the Event broker or in AX 2012 and newer use delegates.
Stick to the infolog system and you use an InfoAction class to peggy bag your information to be used later. It can be used to display a stack trace or other interesting information.
Use a dedicated table for logging.
The third way may seem impractical, as any errors will undo the insert in the log. This is the default behavior but can be circumvented.
MyLogTable log;
Connection con = new UserConnection();
con.ttsBegin();
log.setConnection(con);
... // Set your fields
log.insert();
con.ttsCommit();
Your way to go depends on circumstances you do not mention.
I have a query which uses code like:
criteria.AddOrder(
Order.Asc(
Projections.SqlFunction(
new StandardSQLFunction("NEWID"),
new NHType.GuidType(),
new IProjection[0])));
The purpose is to get a random ordering. I run this against an SQL server, but I would also like to run it against an SQLite, since we use this for testing. SQLite does not support NEWID() but has Random instead. Is it possible to write the code (or configure) such that the same query will work against both databases?
I think the way to do this is to create two custom dialects. Have each one implement a random function differently:
public class MyMsSqlDialect : MsSql2012Dialect
{
protected override void RegisterFunctions()
{
base.RegisterFunctions();
RegisterFunction("random", new StandardSQLFunction("NEWID", NHibernateUtil.Guid));
}
}
public class MySqliteDialect : SQLiteDialect
{
protected override void RegisterFunctions()
{
base.RegisterFunctions();
RegisterFunction("random", new StandardSQLFunction("random", NHibernateUtil.Guid));
}
}
Now, the following query should work fine in either database:
criteria.AddOrder(
Order.Asc(
Projections.SqlFunction("random", NHibernateUtil.Guid)));
Note that this is cheating a bit. random doesn't return a Guid in the Sqlite flavor, but since NHibernate doesn't need that information to do the ORDER BY, nothing should go wrong. I would consider calling it random_order or something to make it clear that this is only for ordering.
I was previously using a repository with this code:
public virtual void Delete(T entity)
{
DbEntityEntry dbEntityEntry = DbContext.Entry(entity);
if (dbEntityEntry.State != EntityState.Deleted)
{
dbEntityEntry.State = EntityState.Deleted;
}
else
{
DbSet.Attach(entity);
DbSet.Remove(entity);
}
}
My application is simple and I noticed many people saying "EF is already a repository so you don't need to use another one". The problem is that without that external repository wrapping I now find that when I want to code a simple delete I find it's necessary to do something like this:
DbEntityEntry dbEntityEntry1 = db.Entry(objectiveDetail);
if (dbEntityEntry1.State != EntityState.Deleted)
{
dbEntityEntry1.State = EntityState.Deleted;
}
else
{
db.ObjectiveDetails.Attach(objectiveDetail);
db.ObjectiveDetails.Remove(objectiveDetail);
}
My one line repo.delete has now changed to ten lines. Is there a way I can get back to simplifying the removal of an entry with EF without having to hard code all the lines for checking if the entry is already attached etc. ?
Actually, although EF is a repository in and of itself, it still makes sense in many situations from a separation of concerns perspective to abstract EF out and use a Repository pattern to achieve this. I use a repository pattern quite often with EF. If you are using automated unit tests, the repository pattern makes isolating your code for testing much easier.
need to pass two parameters from List . need read each element ABCDTO from List and then pass to select statment where a=abcDTO.a and b= abcDTO.b like that.
please let me know which method of jdbc template to be used.
I am not quite sure what you are asking for. It sounds to me like you want to know how to write the SQL statement for your jdbcTemplate to use.... or maybe you are asking for how to iterate through a resultSet? Or, maybe which method will get you a resultSet as a List?
I'd love to help but please clarify your question. There are numerous posts out there that explain (very well) each of these topics. I would first start learning by using springs tuts and the API.
Also, if you want to make things a little easier for your co-workers or self in the future consider using JdbcDaoTemplate class or NamedParameterJdbcTemplate. These help to simplify your code and makes maintenance easier in the long run (imo).
Here is an example of using JdbcTemplate, and the SQL to go with it:
private static final String SQL_RETRIEVE = "SELECT FATHER, MOTHER, SON " +
"FROM DATABASENAME.TABLENAME WHERE FATHER = ? AND MOTHER = ?";
#Autowired
private JdbcTemplate jdbcTemplate;
public Family retrieve(String pFather, String pMother) {
return this.jdbcTemplate.queryForObject(SQL_RETRIEVE, Object[] {pFather, pMother},
new RowMapper<Family>() {
#Override
public Family mapRow(ResultSet pResultSet, int pRowNumber) throws SQLException {
final Family family = new Family();
family.setFather(pFather);
family.setMother(pMother);
family.setSon(pResultSet.getString("SON"));
return family;
}
});
}
I'm just learning nHibernate and have come across what probably is a simple issue to resolve.
Right so I've figured out so far that you can't/shouldn;t nest nHibernate Transactions within each other; in my case I figured this out when scope went to another routine and I started a new Transaction.
So should I be doing the following?
using (ITransaction transaction = session.BeginTransaction())
{
NHibernateMembership mQuery =
session.QueryOver<NHibernateMembership>()
.Where(x => x.Username == username)
.And(x => x.ApplicationName == ApplicationName)
.SingleOrDefault();
if (mQuery != null)
{
mQuery.PasswordQuestion = newPwdQuestion;
mQuery.PasswordAnswer = EncodePassword(newPwdAnswer);
session.Update(mQuery);
transaction.Commit();
passwordQuestionUpdated = true;
}
}
// Assume this is in another routine elsewhere but being
// called right after the first in the same request
using (ITransaction transaction = session.BeginTransaction())
{
NHibernateMembership mQuery =
session.QueryOver<NHibernateMembership>()
.Where(x => x.Username == username)
.And(x => x.ApplicationName == ApplicationName)
.SingleOrDefault();
if (mQuery != null)
{
mQuery.PasswordQuestion = newPwdQuestion;
mQuery.PasswordAnswer = EncodePassword(newPwdAnswer);
session.Update(mQuery);
transaction.Commit();
passwordQuestionUpdated = true;
}
}
Note: I know they are simply a copy, i'm just demonstrating my question
First Question
Is this the way it is MEANT to be done? Transaction per operation?
Second Question
Do I need call transaction.Commit(); each time or only in the last set?
Third Question
Is there a better way, automated or manual, to do this?
Third Question
Can I use session.Transaction.IsActive to determine if the "Current Session" already is part of a transaction - so in this case I can make the "Transaction wrap" in the highest level, let's say the Web Form code, and let routines be called within it and then commit all work at the end. Is this a flawed method?
I really want to hammer this down so I start as I mean to go on; I don;t want to find 1000s of lines of code in that I need to change it all.
Thanks in advance.
EDIT:
Right so I wrote some code to explain my issue exactly.
private void CallingRoutine()
{
using(ISession session = Helper.GetCurrentSession)
{
using (ITransaction transaction = session.BeginTransaction())
{
// RUN nHIbernate QUERY to get an OBJECT-1
// DO WORK on OBJECT
// Need to CALL an EXTERNAL ROUTINE to finish work
ExternalRoutine();
// DO WORK on OBJECT-1 again
// *** At this point ADO exception triggers
}
}
}
private bool ExternalRoutine()
{
using(ISession session = Helper.GetCurrentSession)
{
using (ITransaction transaction = session.BeginTransaction())
{
// RUN nHIbernate QUERY to get an OBJECT-2
// DO WORK on OBJECT
// Determine result
if(Data)
{
return true;
}
return false;
}
}
}
Hopefully this demonstrates the issue. This is how I understood to write the Transactions but notice how the ADO exception occurs. I'm obviously doing something wrong. How am I meant to write these routines?
Take for example if I was to write a helper object for some provider and within each routine exposed there is a nHibernate query run - how wold I write those routines, in regards to Transactions, assuming I knew nothing about the calling function and data - my job is to work with nHibernate effectively and efficiently and return a result.
This is what I assumed by writing the transaction how I did in ExternalRoutine() - to assume that this is the only use of nHibernate and to explicitly make the Transaction.
If possible, I would suggest using System.Transactions.TransactionScope:
using(var trans = new TransactionScope())
using(var session = .. create your session...) {
... do your stuff ...
trans.Complete();
}
The trans.Complete will cause the session to flush and commit the transaction, in addition you can have nested transactionscopes. The only "downside" to this is that it will escalate to DTC if you have multiple connections (or enlisted resources such as MSMQ), but this is not necessarily a downside unless you're using something like MySQL which doesn't play nicely with DTC.
For simple cases I would probably use a transaction scope in the BeginRequest and commit it in EndRequest if there were no errors (or use a filter if u're using ASP.NET MVC), but that really depends a lot on what you're doing - as long as your operations are short (which they should be in a web app), this should be fine. The overhead of creating a session and transaction scope is not that big that it should cause problems for accessing static resources (or resources that don't need the session), unless you're looking at a really high volume / high performance website.