PouchDB / SQLite indexing crashes - needs to much memory on IOS - sqlite

Intro
I have build an IOS app with PouchDB and Ionic and Cordova stuff, but now i'm failing with the database and can't see a solution...
SQLite
Sqlite plugin and the synchronization works great, but as soon i want to access the data PouchDB starts indexing which uses to much memory and my app crashes.
Here is a sampleApp with everything prepared for interested helpers to reproduce the crash
SampleApp Github
WebSQL
In WebSQL I can't even sync the database, because I get the "QuotaExceededError" Error. Is there a way to ask for more space in an IOS uiwebview?
IndexedDB
Indexedb isn't usable, cause of too many bugs. And will it also not be better in IOS 9
So any ideas ideas or hints?

The solution to the problem of storage limits on iOS is indeed to use the SQLite Plugin. You can't exceed 50MB on WebSQL.
Sqlite plugin and the synchronization works great, but as soon i want to access the data PouchDB starts indexing which uses to much memory and my app crashes.
Are you sure this is a bug with PouchDB and not with the SQLite Plugin? If it's a crash in Objective-C land, it belongs to the SQLite Plugin. If it's a crash in JavaScript land, it probably belongs to PouchDB.
Either way, if you could provide a test to reproduce, we would welcome a bug report on either the PouchDB Github issues page or the SQLite Plugin issues page. :)

About WebSQL, please consider this:
PouchDB limits
Some versions of Safari have a bug where they only let you request additional storage once, so you'll need to request the desired space up-front. PouchDB allows you to do this using the size option.
Try allocating size upfront, and allocate your biggest expectations about the app. Hope that helps.
Good luck.

Related

Which database should I use for a local application?

I am programming a standalone application which needs a local database storage.
I have read about SQLite and it seems like it could perfectly fit my needs.
Nonetheless someone told me that SQLite is going to be discontinued. Is that true ? If so, what technology should I use instead ?
The SQLite documentation says:
The intent of the developers is to support SQLite through the year 2050.
The developers are working on a new major version, actually. Maybe whoever told you that heard about it and got confused thinking this means work on SQLite 3 is stopped (it isn't). But even if it were discontinued, the current version isn't going to disappear or stop working.

Is there the ability to store encrypted data on a desktop app created with tideSDK?

I have searched but couldn't find anything. We need to be able to store data offline in our desktop app and we are looking for cross-platform solutions. TideSDK looks promising but I am not sure if it has this functionality.
We can certainly help there. SQLite certainly provide for encrypted data support and is completely cross-platform and supported by TideSDK. We can use an encrypted database just as easily in TideSDK and could offer support in the API for this. If this is of interest, please get in touch by filing a feature request at https://github.com/TideSDK/TideSDK/issues.
If we are talking about encryption of the sources - short term there is obfuscation of the JS sources available in 1.3.0 that is due.
We created a Titanium Desktop module to support encrypted SQLite databases last year. We could help with this and dig out the old code...

Secure cross-platform storage options using Xamarin

I'm currently reviewing Xamarin as an option for developing a tablet app (right now aim for iOS and Android but in the future think of porting it to Win8 tablets)
Data to be stored is really sensitive so security is my main concern.
So far I'm thinking of two options:
a) SQLite + SQLCipher: Assuming that Win8 tablets will have SQLite support and that SQLCipher will also work on Win8.
b) Isolated Storage + DPAPI: Assuming that this DPAPI approach will work on Mono platforms and that it will still be available on Win8 tablets.
My concern here is that to securely store the key I guess I'll still need to have separated implementations for each platform (Data Protection on iOS and I'm still reviewing options for Android and Win8 tablets).
Is there any cross platform solution that will work to securely store the encryption key or another completely cross platform way to securely store data? If there is no completely cross platform solution then any recommended options to use on each platform?
I have written a plugin for to address secure storage. It works cross platform and secures the data natively on each platform. It is open source.
Please check it out.
https://www.nuget.org/packages/sameerIOTApps.Plugin.SecureStorage/
Blog:
https://sameer.blog/2018/01/19/whats-new-in-secure-storage-2-0/
Let me know, should you have any questions.
We use SQLite + SQLCipher in our solution, it works really well on a cross-platform environment.
If I am not mistaken there is no problem with using SQLCipher because it uses AES. I would suggest you to check it anyway but I don't think it's a concern. SQLCipher itself doesn't make the data impossible to read, maybe you have a problem if you allow the option to use Passwords with SQLCipher, but still. The process to get a permission is really simple for AES and can be completed in a few days.

Alternatives to SQLite Manager, with WAL support?

What application clients are there that can connect SQLite3 databases that have Write-Ahead Logging(WAL) are out there? I am not finding one with my google foo today, I have the SQLite Manager Firefox extension. Again I am not finding where it says that it has wal support for it however? Can someone point me in the right direction please?
If I where to guess, I would say that as long as the client can open the SQLite3 file with wal, that the client supports wal. However that is just a guess on my part.
EDIT 8/31/2012
As much as I don't like answering my own questions :p So I am turning this into a, "what alternatives are there" question
Yes, there is a Client application that connects SQLite3 databases, it is a FireFox extension called SQLite Manager. They added support for it apparently in Apr 24, 2011 (or sometime there abouts) according to their issue tracker. For those of you who where wondering, there is the answer. I needed this for work, wound up using the manager without knowing and figured I would take the time after to do a little more research. I still have not found any other SQLite db clients that are independent of an IDE.
SQLite Manager - FireFox Extension
SQLite Manager - Project Page
SQLite Manager - Issue 602
If you got any SQLite client with seperated sqlite3.dll you can easily swap that file with the latest one supporting WAL.
With that in mind you can find some more SQLite managers out there.

Recommendation for Sqlite DB manager application? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
There are quite a few Sqlite GUI applications listed here:
http://www.sqlite.org/cvstrac/wiki?p=ManagementTools
some appear to be incomplete, buggy, not maintained, etc. Do you have any recommendations?
I use the SQLite manager plugin for Firefox.
It seems pretty stable to me.
Here's another free option. It has been pretty stable for me. SQLiteStudio
For Windows: I've been looking for functionality and a comfortable GUI - it's been particularly hard to satisfy the latter requirement, but these two picks are both fine:
a) Good enough: SQLite Expert
Less expensive, fully featured manager. The author is very responsive to comments and bug reports, and publishes updates frequently. The flip side, if you look at version history, is that new releases seem to introduce new bugs, which are then fixed in sunsequent builds. The GUI is fine, very good for quickly designing new databases; a little less so for designing queries and working with large amounts of data. Main gripe: you can't see the schema while editing a query (without flipping tabs in the program).
b) Nearly perfect: SQLite Maestro
Pricier. Rich UI, easy access to all features, nice visual query builder and automatic SQL formatter for readability, lots of eye candy. There is a separate, more specialized query builder (SQLite Code Factory), but you can make do with just the main Maestro application. Main gripe: can't seem to be able to change font size for table display and the default is a little too small; at the beginning it's easy to get lost in the thicket of tabs, though overall I find the GUI very productive.
Both solutions are very stable in my experience, and both seem to offer occasional discounts, if you can afford to wait.
For just browsing data, try SQLite Spy - free and lets you execture queries, but no or little GUI support for editing. Very convenient for quick lookups though.
Spent the morning looking for a good Sqlite Database Manager/Browser and have settled on SqliteStudio, currently v2.20.28, which I'm running on Ubuntu Linux 10.04.
The download is one 4.3MB uncompressed executable file. Period!
It's fast, well-behaved, and uses modest resources.
The interface is clean and attractive with logical functionality.
I don't say this often, but it's just a "A Joy to Use".
It was developed and is actively maintained by Pawel Salawa who is is to be commended for producing a very nice program.
I just used it to merge two Firefox FloatNotes databases (Sqlite v3), which just use a single table...
The target database is on the local host where SqliteStudio is running (i.e. /0/LX02)
The source database file is on a remote host mounted via SSHFS to /0/LX04
Opened the two database files using Add Database.
Used the SQL Editor to execute an INSERT to the LX02 database from a SELECT on the LX04.
Click the Commit icon when all goes well or Rollback if there are errors to fix.
Refresh the table data to see the inserted rows.
There's no Import GUI functionality but merging is pretty simple if you know SQL. I supplied NULL for the first column which is the unique-id primary-key so that Sqlite would autoincrement, thus renumbering the rows being merged in...
INSERT INTO [floatnotes.sqlite].floatnotes
SELECT NULL,
url,
protocol,
content,
x,
y,
w,
h,
color,
status,
guid,
creation_date,
modification_date
FROM [floatnotes-LX04.sqlite].floatnotes;
I like SQLite Administrator
I've been using SQLite Professional and it's been reliable. The only downside was that changes I made to the db via the iOS simulator didn't resolve in the app (I had to close it and reopen for changes to show) but that feature was added in a recent version. So now it does everything I need. I'm happy with the support. It's good for testing and I use it to build out the db structure. Never been buggy.
I've used Sqliteman in the past. Quite nice.
SQLiteSpy is a good choice
Sqlite.org has a list of management tools available here:
http://www.sqlite.org/cvstrac/wiki?p=ManagementTools
If your using OSX you may like SQLPro for SQLite (App Store).
The app has a few neat features such as:
Versions Integration (rollback to previous versions).
Inline data filtering.
Exporting options to CSV, JSON, XML and MySQL.
Column reordering.
Full screen support.
I've been using SQL Explorer. The firefox plugin is awesome, but it couldn't handle BIGINTs properly (it truncated them). I have noticed that the .jar driver doesn't seem to support FTS4 (but it does support FTS3) and doesn't show indexes in the data structure.
I like SQL explorer as you can use other providers like mySQL too with the one client.
You didn't mention a platform, so here's a great comparison of Mac OS X SQLite tools. I personally found MesaSQLite to be most like my preference for database tools, which was CocoaMySQL incidentally. (For Windows, I just used the Firefox add-on mentioned above.)
If you're within Visual Studio most of the time then System.Data.SQLite is good, and as a plus handles encrypted databases.
Navicat SQLite is very good and they support Windows, OS X and Linux too.
You might want to check MYZSQLExplorer, here.
Unlike the other tools, it is running on iOS devices (or in the simulator). It is a Viewer, not a management tool, and is not as feature-complete as some of the other tools, but is convenient as you can launch it from within your code and browse your databse from within your app.
Or you can "Open In..." it sqlite stores, by sending them as email attachments.
I developed it in order to help myself during the development and testing phases, and it did help.
If you have any feedback on it, I'd love to hear it.
If you want just CRUD operations on the sqlite database file, then SQuirreL is a very option as it has an auto complete feature which drastically improve the speed and efficiency of typing the sql queries.
To use the SQLite database in SQuirreL first download the JDBC driver of the SQLite from here then drop the jar in the lib folder of the SQuirreL folder. Now open SQuirreL and choose Create a New Driver.
In the Example URL field put
jdbc:sqlite:$file_url
and in Class Name put
org.sqlite.JDBC
After that choose Create a New Alias and choose the driver that you just added and replace $file_url with the actual location of the sqlite file then click ok and you are done.

Resources