I have been developing locally for some time and am now pushing everything to production. Of course I was also adding data to the development server without thinking that I hadn't reconfigured it to be Postgres.
Now I have a SQLite DB who's information I need to be on a remote VPS on a Postgres DB there.
I have tried dumping to a .sql file but am getting a lot of syntax complaints from Postgres. What's the best way to do this?
For pretty much any conversion between two databases the options are:
Do a schema-only dump from the source database. Hand-convert it and load it into the target database. Then do a data only dump from the source DB in the most compatible form of SQL dump it offers. Try loading that into the target DB. When you hit problems, script transformations to the dump using sed/awk/perl/whatever and try again. Repeat until it loads and the results match.
Like (1), hand-convert the schema. Then write a script in your preferred language that connects to both databases, SELECTs from one, and INSERTs into the other, possibly with some transformations of data types and representations.
Use an ETL tool like Talend or Pentaho to connect to both databases and convert between them. ETL tools are like a "somebody else already wrote it" version of (2), but they can take some learning.
Hope that you can find a pre-written conversion too. Heroku one called sequel that will work for SQLite -> PostgreSQL; is it available without Heroku and able to function without all the other Heroku infrastructure and code?
After any of those, some post-transfer steps like using setval() to initialize sequences is typically required.
Heroku's database conversion tool is called sequel. Here are the ruby gems you need:
gem install sequel
gem install sqlite3
gem install pg
Then this worked for me for a sqlite database file named 'tweets.db' in the current working directory:
sequel -C sqlite://tweets.db postgres://pgusername:pgpassword#localhost/pgdatabasename
PostgreSQL supports "foreign data wrappers", which allow you to directly access any data source through the DB, including sqlite. Even up to automatically importing the schema. You can then use create table localtbl as (select * from remotetbl) to get your data into the actual PG storage.
https://wiki.postgresql.org/wiki/Foreign_data_wrappers
https://github.com/pgspider/sqlite_fdw
Related
I have a website solution that uses on SQLite database for each tenant account. Without going into much depth about why we chose this solution, we chose it due to SQLite support on distributed/offline systems.
All databases are manipulated using the same PHP file structure. I wish to update the database version iteratively for all accounts so that they are all at the same version number.
I have a script that loops over each, and can use either PHP(Yii) or the shell to execute queries.
I would like to wrap the manipulation to each database in a transaction. It appears as though DDL commands may already be auto-commit in nature.
Question: How to accomplish a SQLite DB upgrade which, if it fails, will report a failure? Using that report, I could prompt the system to re-attempt or report an error to an admin. Ideally, I would like to wrap the whole upgrade in a transaction to prevent inconsistencies, but I'm fairly certain that this is not possible.
One thought I had was to backup the database temporarily during upgrade, and delete it on success.
As CL stated in the comments, DDL is fully transactional in SQLite. So, for each database in our system, we wrap it in a transaction, and it executes atomically. We leverage the application to ensure that all databases upgrade successfully if any fail during upgrade.
I have installed Oracle XE 11g R2 on my machine. I ran few scripts which does the setup by creating schemas, procedures for our application. Now I want to clone this database so that other people by using the cloned dbf file can see the base schema on their respective machine and work on their individual requirement on top of that.
Now it has 6 dbf files
CONTROL.DBF
SYSAUX.DBF
SYSTEM.DBF
TEMP.DBF
UNDO.DBF
USER.DBF
Can i just give them the files or I need to create server parameter file (SPFILE) or Control file. What about the REDO logs.
I have very little knowledge in Database administration. Please suggest. I understand that it is not Enterprise Edition so all things might not supported but assuming cloning process is similar for XE.
While it is possible to restore a database using the data files, I strongly suspect that is not what you're really after. If you're not an experienced DBA, the number of possible issues you'll encounter trying to restore a backup on a different machine and then creating an appropriate database instance are rather large.
More likely, what you really want to do is generate a full export of your database. The other people that need your application would then install Oracle and import the export that you generated.
The simplest possible approach would be at a command line to
exp / as sysdba full=y file=myDump.dmp
You would then send myDump.dmp to the other users who would import that into their own database
imp / as sysdba full=y file=myDump.dmp
This will only be a logical backup of your database. It will not include things like the parameters that the database has been set to use so other users may be configured to use more (or less) memory or to have a different file layout or even a slightly different version of Oracle. But it does not sound like you need that degree of cloning. If you have a large amount of data, using the DataPump version of the export and import utilities would be more efficient. My guess from the fact that you haven't even created a new tablespace is that you don't have enough data for this to be a concern.
For more information, consult the Oracle documentation on the export and import utilities.
Removing content as it is not valid here
I have been using a central MS SQL database located in the cloud to develop a web site project. I have recently found myself in situations, when I need to develop without the internet connection. I want to begin to use a locally available copy of the existing database, put it in App_Data folder.
What is the correct set of steps I need to undertake to get the project to work with local DB?
For example:
Detach a db from an existing SQL instance.
Copy to a development machine.
etc.
Moving a SQL-Server DB is not that hard. Look here for some methods to do it.
http://support.microsoft.com/kb/314546
I usually find the sp_detach + sp_attach method really easy.
I would create an empty shell database locally, then use one of the many schema comparison solutions available to make the local database look exactly like the cloud database.
Correct way is to create and regularly update your standalone copy of database using import/export. In particular MS SQL Server provides Import/Export Wizard tool for such purpose.
Make a backup
Copy backup file
Restore on your server
Restore/organise users
I am working on a firm application in which I need to create a local database on my device.
I create my local database through create statement[ It works well]
Then I use that file and perform insert operation through fire-fox sqlite plugin, I need to insert aprox 2000 rows at a time so I can not use code. I just run insert manually through sqlite plugin in fir-fox.
After that I just use that file in my place of my local database.
When I run select query through my code, It show Exception:java.lang.Exception: Exception: In create or prepare statement in DBnet.rim.device.api.database.DatabaseException: SELECT distinct productline FROM T_Electrical ORDER BY productline: file is encrypted or is not a database
I got the solution of this problem, I was doing a silly mistake by creating a file manually by right click in my RES folder, that is not correct. We need to create the database completely from SQlite plugin, then it will work fine. "Create data base from SQLITE(FIle too) and perform insertion operation from SQLITE, then it will work fine"
This is very rare problem, but i think it might be helpful for someone like me....!:)
You should check to see if there is a version problem between the SQLite used by your Firefox installation and that on the BlackBerry. I think I had the same error when I tried to build a database file with SQLite version 2.
You also shouldn't need to create the database file on the device. To create large tables I use a Ubuntu machine and the sqlite3 command line. Create the file, create the tables, insert the data and build indexes. Then I just copy the file onto the device in the proper directory.
For me it was a simple thing. One password was set to that db. I just used it and prolem got solved.
I need to manipulate some tables in a JET database housed on a web-server:
check existing indexes
change table cluster/primary key
see what tables exist
rename tables
add tables
drop tables
browse data
etc
I don't have the option of installing PlaneDisaster or Access (even if i had it) on the local machine.
I've already written a generic web-based query tool. I'd rather not have to get into writing a whole web-based database maintenance GUI. Someone must have done this already, and probably many times over.
A partial answer might be Compare'Em
http://home.gci.net/~mike-noel/CompareEM-LITE/CompareEMscreens/CompareEM-About.htm The Pro version allows you to create SQL statements to update the Access database file. This will allow you to generate the differences between one version and a newer version.
His website isn't very clear but as I recall the price for the Pro version was $10.
As you say you have already done a generic web based query tool. The problem with JET is that you cannot connect with it as database server like you can do with one SQL server in order to process changes to tables and other maintenance procedures. Jet is is not a client/server RDBMS. You need to have an application in the server to do that for you as you already have done with your generic web based tool, or download the database to your machine. That's why you have done some procedures and locate them in the server as asp pages.
Anyway you can use JetSQLConsole, if you don't want to use Planedisaster or Access, but remember that you need always an application on the server to to the job for you
You can also use access "in your machine" and connect to a database located in a URL (http://myserver/mydatabase.mdb) but remember when you are doing this you are downloading all the database and when you save it you are uploading it again.