I need to export the data of a SQLite database and import this data to another SQLite database. But the two databases have different schemas.
Does exist an open source tool that can help me doing that job?
The only opensource tool that i know is opendbcopy that i'm using for migrate from a database server to another and also for a similar kind of job that you want to do with SQLite but i've done it with PostgreSQL.
However opendbcopy is JDBC compliant and can connect to every database that have a JDBC driver, so you can try, also if the schema is not the same you can use the column mapping feature :
In addition i know also a good commercial alternative (that is easier to use) that is ESF Database Migration Toolkit .
Related
In which way can I access another database (not OpenEdge) via ODBC from OpenEdge without using DataDirect?
The use case is data migration from one system to another, so performance cannot be neglected completely but it's a one time thing that is allowed to take a little longer.
Why without DataDirect? Extra cost. Our client doesn't have the license.
Why not dump and load (via CSV f.e.)? The client doesn't want to do the mapping between the systems this way but with database views.
As far as I know there is no way to directly access other database if you're not using DataDirect or something like DataServer for Oracle etc.
However, you could call a third party ODBC library as external functions, and write your handle your queries to the foreign database by accessing. This wouldn't allow you to use OpenEdge constructs like FOR EACH, buffers etc, but it would allow you to retrieve the data and process it using custom functions, and then insert into the OpenEdge tables etc.
See the following KB for accessing external library functions:
https://knowledgebase.progress.com/articles/Article/P183546
Another approach you could use, assuming your tables are in OpenEdge already, is to use the OpenEdge SQL92 ODBC driver from another language (C/VB/Java/whatever works for you), and read the data from the source database and insert into OpenEdge via SQL92 ODBC.
Looking at the website there are downloadable ODBC drivers for most platforms:
https://www.progress.com/odbc/openedge
So just exploring the possibility of using flyway to maintain my DDL statements against Amazon Athena using Athena provided JDBC driver, Athena supports only CREATE statements (hive DDL) and no INSERTS.
So if database metadata table is the only one that flyway creates and updates, Is there anyway I can externalize the creating, storing into a totally different database ?
Currently this is not possible in flyway, as the schema history is read from/written to the database defined by the current jdbc connection. You can see this for yourself in the JDBCTableSchemaHistory file.
If you wish to add this support, you could create a pull request on the repo, or just add an issue detailing exactly what you want the behaviour to be.
I'm trying to create a database using Scala and SQLite. I'm using Slick as the library for the SQLite.
I've been googling around for hours and still can't figure out how to get this working. I have eclipse project with Slick installed. I'm trying to instanciate the database with
val db = Database.forUrl("url",driver = "org.SQLite.Driver")
I have no idea what to put in the url. I'm not very sure about the driver part either. Should I use that one or does "scala.slick.driver.SQLiteDriver" work too? Or does it even matter?
I'm really confused about all this.Any help is appreciated
Thanks!
JDBC relies on drivers that implement the JDBC API, and provide access to the low-level functionality of working with particular databases.
URLs are how you tell a JDBC driver which database you want to connect to. The first part of the URL is always jdbc:<driverId>:, where driverId is the specific name that the driver expects to see (e.g. postgresql, mysql or, in your case sqlite.) The format of the URL after the driver ID is specific to the particular driver implementation. With mysql and postgres, where you typically connect over TCP to the database server, you'll see a format like this:
jdbc:mysql://dbserver:dbport/databaseName
jdbc:postgresql://dbserver:dbport/databaseName
But, since SQLite is an in-process, local database, the part of the URL after the driver ID is just a filesystem path, like so:
jdbc:sqlite:/home/me/my-db-file.sqlite
I have installed Oracle XE 11g R2 on my machine. I ran few scripts which does the setup by creating schemas, procedures for our application. Now I want to clone this database so that other people by using the cloned dbf file can see the base schema on their respective machine and work on their individual requirement on top of that.
Now it has 6 dbf files
CONTROL.DBF
SYSAUX.DBF
SYSTEM.DBF
TEMP.DBF
UNDO.DBF
USER.DBF
Can i just give them the files or I need to create server parameter file (SPFILE) or Control file. What about the REDO logs.
I have very little knowledge in Database administration. Please suggest. I understand that it is not Enterprise Edition so all things might not supported but assuming cloning process is similar for XE.
While it is possible to restore a database using the data files, I strongly suspect that is not what you're really after. If you're not an experienced DBA, the number of possible issues you'll encounter trying to restore a backup on a different machine and then creating an appropriate database instance are rather large.
More likely, what you really want to do is generate a full export of your database. The other people that need your application would then install Oracle and import the export that you generated.
The simplest possible approach would be at a command line to
exp / as sysdba full=y file=myDump.dmp
You would then send myDump.dmp to the other users who would import that into their own database
imp / as sysdba full=y file=myDump.dmp
This will only be a logical backup of your database. It will not include things like the parameters that the database has been set to use so other users may be configured to use more (or less) memory or to have a different file layout or even a slightly different version of Oracle. But it does not sound like you need that degree of cloning. If you have a large amount of data, using the DataPump version of the export and import utilities would be more efficient. My guess from the fact that you haven't even created a new tablespace is that you don't have enough data for this to be a concern.
For more information, consult the Oracle documentation on the export and import utilities.
Removing content as it is not valid here
I have been developing locally for some time and am now pushing everything to production. Of course I was also adding data to the development server without thinking that I hadn't reconfigured it to be Postgres.
Now I have a SQLite DB who's information I need to be on a remote VPS on a Postgres DB there.
I have tried dumping to a .sql file but am getting a lot of syntax complaints from Postgres. What's the best way to do this?
For pretty much any conversion between two databases the options are:
Do a schema-only dump from the source database. Hand-convert it and load it into the target database. Then do a data only dump from the source DB in the most compatible form of SQL dump it offers. Try loading that into the target DB. When you hit problems, script transformations to the dump using sed/awk/perl/whatever and try again. Repeat until it loads and the results match.
Like (1), hand-convert the schema. Then write a script in your preferred language that connects to both databases, SELECTs from one, and INSERTs into the other, possibly with some transformations of data types and representations.
Use an ETL tool like Talend or Pentaho to connect to both databases and convert between them. ETL tools are like a "somebody else already wrote it" version of (2), but they can take some learning.
Hope that you can find a pre-written conversion too. Heroku one called sequel that will work for SQLite -> PostgreSQL; is it available without Heroku and able to function without all the other Heroku infrastructure and code?
After any of those, some post-transfer steps like using setval() to initialize sequences is typically required.
Heroku's database conversion tool is called sequel. Here are the ruby gems you need:
gem install sequel
gem install sqlite3
gem install pg
Then this worked for me for a sqlite database file named 'tweets.db' in the current working directory:
sequel -C sqlite://tweets.db postgres://pgusername:pgpassword#localhost/pgdatabasename
PostgreSQL supports "foreign data wrappers", which allow you to directly access any data source through the DB, including sqlite. Even up to automatically importing the schema. You can then use create table localtbl as (select * from remotetbl) to get your data into the actual PG storage.
https://wiki.postgresql.org/wiki/Foreign_data_wrappers
https://github.com/pgspider/sqlite_fdw