Due to the differences in file structure etc. between platforms, I was wondering if the database creation (with connection strings) need to be platform specific? Or if there's maybe a way to create a database from OnAppLoad() platform agnostic?
The SQLite file format is fully portable.
A database in SQLite is a single disk file. Furthermore, the file
format is cross-platform. A database that is created on one machine
can be copied and used on a different machine with a different
architecture. SQLite databases are portable across 32-bit and 64-bit
machines and between big-endian and little-endian architectures.
You do not need to worry about it at all. Things to worry about that are not platform related are few and can include the WAL journal-mode due to lack of backward compatibility.
You can also read:
http://www.sqlite.org/atomiccommit.html#sect_9_0
and:
http://www.sqlite.org/lockingv3.html#how_to_corrupt
Related
I need to use an embedded database in my java application that will be run in a Linux device. The application uses Hibernate and derby database. This is not a Android application.
Due to slow performance of the database, we are looking for a better embedded database framework.
Looking at all the options, H2 seems to be better than SQLite as there is no cross-compilation involved and no JNI interface to build.
So, why isn't there a more usage of H2. Are there any drawbacks or issues that I am not aware of.
The SQLite library is implemented in C, so it indeed needs (cross-)compilation and a JNI interface.
However, SQLite is so widely used that it is likely that the SQLite interface already exists (as part of your language's runtime, or as a JDBC driver), and that using it is simpler than explicitly adding H2 to your project. (This might not actually be true in your specific environment.)
If you're looking to speed up your application, you have to measure yourself.
I recently switched from H2 to SQLite because of database corruptions in the H2 mv store.
If the application is not shutdown properly, or in case of unexpected reboots, the H2 database stored on a file using the MV store (the default) can get corrupt, and you can't restore data.
SQLite is much more robust to corruption.
Speed wise H2 was much faster in my case. With SQLite transactions are particularly costly, so you should prefer doing bulk operations within transactions or via batches.
As for cross compilation, you can use the jdbc driver from xerial which ships with all the native binaries precompile : https://github.com/xerial/sqlite-jdbc
I can't seem to find an application to monitor SQLite DB performance. Currently I have a test server that uses SQLite. I'm primarily concerned with obtaining a benchmark of storage requirements and performance for scaling this server to production.
I know for MySQL there is the standard Nagios for monitoring (changing to mySQL is not an option at this point). Is there anything analogous for SQLite?
SQLite has functions like sqlite3_status() and sqlite3_db_status(), but those do not really give you the information you want, and might not even be available in all languages.
Anyway, SQLite is an embedded library, so you'd have to monitor your actual application. Tools like Nagios allow to monitor a server's CPU load and disk usage, but you can also use any other tool of your OS.
We have an application that can use both Postgres and SQLite as its backend, depending on what the customer requires: High load and concurrency, or easy setup.
But I fear that some customers will be too clever for their own good, and to get around the (slightly) complex Postgres installation, they use SQLite, but put the file on a network disk.
Then the inevitable happens and the database gets corrupted, because SQLite is not meant for that, and we have Postgres support for that very reason.
Is there an ideal way to prevent SQLite from working on a network drive? There are some questionable ideas like looking for a \\ at the beginning, or the colon in "C:\" (it's purely a windows app), or parsing for IPs, but none of these are fool-proof and can be circumvented by making a link to a network disk.
I have seen some applications using proprietary databases crash or experience data corruption if the application was running when the OS (Windows in this case) performs a disk defragmentation. My question is this: Does SQLite (sqlite3) suffer from this issue? In other words, would it ever be dealing with the disk on a block level, or just on a file level?
SQLite uses only the OS's file access functions, so as long as the defrag tool works with concurrent file accesses, you should be fine.
I have a large (several Gb) berkeley db that I am thinking of migrating from windows (2K) to Linux (either Redhat or Ubuntu). I am not sure how to go about this. Can I merely move the db files accross, or do I need a special conversion utility?
Database and log files are portable across different endian systems. Berkeley DB will recognize the kind of system it is on and swap bytes accordingly for data structures it manages that make up the database itself. Berkeley DB's region files, which are memory mapped, are not portable. That's not such a big deal because they region files hold the cache and locks which, because your application will not be running during the transition, will be re-created on the new system.
But, be careful, Berkeley DB doesn't know anything about the byte-order or types in your data (in your keys and values, stored at "DBTs"). Your application code is responsible for knowing what kind of system it is running on, how it stored the data (big or little endian) and how to transition it (or simple re-order on access). Also, pay close attention to your btree comparison function. That too may be different depending on your system's architecture.
Database and log files are also portable across operating systems with the same caveat as with byte-ordering -- the application's data is the application's responsibility.
You might consider reviewing the following:
Selecting a Byte Order
DB->set_lorder()
Berkeley DB's Getting Started Guide for Transactional Applications
Berkeley DB's Reference Guide
Voice-over presentation about Berkeley DB/DS (Data Store)
Voice-over presentation about Berkeley DB/CDS (Concurrent Data Store)
Berkeley DB's Documentation
Disclosure: I work for Oracle as a product manager for Berkeley DB products. :)
There's a cross-platform file transfer utility described here.
You may also need to be concerned about the byte order on your machine, but that's discussed a little here.
If you're using Java Berkeley though it shouldn't matter?