I started to use Flyway in my current project for database migrations and I like it very much. I currently use Oracle in PROD- and Derby in TEST-Environment.
Pretty soon, I did run in the problem of database specific sql commands, e.g.
ALTER TABLE T1 MODIFY F1 VARCHAR(256); on Oracle vs
ALTER TABLE T1 ALTER F1 SET DATA TYPE VARCHAR(256); on Derby.
I can't see a way to write a "vendor neutral alter table modify column datatype" sql.
What's the best way to deal with this problem using Flyway?
You can use the flyway.locations property.
In test in would look like this:
flyway.locations=sql/common,sql/derby
and in prod:
flyway.locations=sql/common,sql/oracle
You could then have the common statements (V1__Create_table.sql) in common and different copies of the DB-specific statements (V2__Alter_table.sql) in the db-specific locations.
An even better solution, in my opinion, is to have the same DB in prod and test. Yes, you do lose a bit of performance, but on the other hand you also eliminate another difference (and potential source of errors) between the environments.
The differences in SQL between Oracle and some of these desktop databases is minor. Is it possible for a developer to insert custom code to do some light-weight dynamic stripping of the SQL at runtime based on the environment (e.g. remove tablespace designation)?
I prefer this approach to relying on each developer to manually keep two sets of SQL in sync.
Related
I currently have a Kotlin-Exposed project that I would like to add Flyway to. The problem I am having is that most documentation and answers online indicate that the best way to add Flwyay to an existing schema is to just have the first script be a data definition script. This usually would work, but since I'm dynamically generating my SQL with an ORM, this doesn't really make sense. Are there ways around this?
I really just want to use Flyway to add/delete persistent data that I will always need in certain tables. I don't want to insert it at the ORM level because if the application is run multiple times, then it can insert the data each time it's run (as opposed to Flyway where it will just migrate the database to the newest constructed state).
I think another way to word this question is: "Can I use Flyway for static data only, and not schema?"
Yes, you can.
Some info:
You are not required to have a first script containing the data definition / "baseline" of the schema. You can simply skip that.
When running Flyway against a non-empty database for the first time, you will still need to run the baseline command. In your case this will simply indicate to Flyway that it can assume the baseline schema is present and that it's safe to run migrations. (Your baseline schema was deployed by the ORM instead of a baseline script -- that's totally fine, Flyway won't check/doesn't care.)
You could choose to write your scripts that insert static data in a way that they are idempotent / use a guard clause so that they don't insert the data twice. That way it would be safe to use at the ORM level if you choose.
I'm working on some R-scripts which require the use of database queries. My IT department mentioned to me that these scripts require too much of the database (shared pool) and I should use bind-variables.
I am not very familiar with database designing, but I found out that bind variables are like variables i.e. if you use for example and date=:var in a statement.
Is it possible to use such statements when working with R (RODBC)?
Thank u very much
It seems that using bind variables is a part of the SQL query you write. So I presume that when you connect to the Oracle database using RODBC, you can pass an valid SQL query that contains bind variable syntax, and that this would simply work.
I am having a question i.e when i tried to alter a data type of a table by query or by modifying the data type directly i used to get some errors stating this can not be done. But where as if i open the same table in server explorer from visual studio and modify the data type it was easily done with out any errors.
I tried to modify the primary key which is a foreign key for some tables from data type int to varchar using Management Studio it throw me some errors.
But the same thing when i done through Server explorer from Visual Studio it was done in a fraction.
Can any one tell what was the difference
I think I had this issue (at least with sql server 2008):
Take a look at Tools->Options->Designers.
uncheck "Prevent saving changes that require table re-creation"
and try again.
You can see more info here about when it occurs if the option is checked.
Without seeing your specific error, all I can say is that the UI does far more than a simple command. It sometimes creates temporary storage and the copies data as it needs to, to make the experience pleasant. You should use the scripting function in the UI to capture the change and see what it's doing?
It can be done using a query, because that's exactly what SQL Server does. However, the script is considerably more complex than a simple ALTER TABLE statement; generally involving creating a temporary table with the changed datatype, copying the data from the original table, dropping the original table, then renaming the temporary table.
When modifying the structure of a table in the Design view in SSMS there is an option to generate the script for the change(s), so you can run it on multiple servers, etc. This will show you the exact process that is actually happening behind the scenes.
I am currently developing an application that will store data in an SQLite database. The database will have much more read- than write-access (in fact, it will be filled with data once, and then almost only reading will happen). The read-performance is therefore much mre important. The schema I am currently developing is very likely to change in the future, with additional columns and tables being added. I do not have very much experience with databases in general. My question is, specifically in SQLite, are there any pitfalls to be considered when changing a schema? Are there any patterns or best practices to plan ahead for such cases?
Here are some suggestions:
Don't use select * from ... because the meaning of * changes with schema changes; explicitly name the columns your query uses
Keep the schema version number in the database and keep code in the application to convert from schema version N to version N+1; then all the code in the application works with the latest schema version; this may mean having default values to fill added columns
You can avoid copying tables for schema updates with SQLite version 3.1.3 or better which supports ALTER TABLE ADD COLUMN...
Look into data-marts and star schema design. This might be overkill for your situation, but at least it will prevent you from designing at random.
I'm currently looking at a terrible legacy ColdFusion app written with very few stored procedures and lots of nasty inline SQL statements (it has a similarly bad database too).
Does anyone know of any app which could be used to search the files of the app picking out any SQL statements and listing the tables/stored procedures which are referenced?
Dreamweaver will allow you to search the code of the entire site. If the site is setup properly including the RDS password and provide a data source it can tell you a lot of information. I've only set it up once so I can't remember exactly what information it gives you, I think maybe just the DB structure. Application window > databases. Even if it isn't set up properly just searching for "cfquery" will quickly find all your queries.
You could also write a CF script using CFDirectory/CFFile to loop the .cfm files and parse everything between cfquery and /cfquery tags.
CFBuilder may have some features like that but I'm not to familiar with it yet.
edit I've heard that CFBuilder can't natively find all your cfqueries that don't have cfqueryparam but you can use CF to extend CFB to do so. I imagine you could find/write something for CFB to help you with your problem.
another edit
I know it isn't indexing the contents of the query, but you can use regex to search using the editor as well. searching for <cfquery.+(select|insert|update|delete) checking the regex box should find the queries that aren't using cfstoredProc (be sure to uncheck the match case option if there is one). I know Dreamweaver and Eclipse can both search for Regex.
HTH
As mentioned above I would try a grep with a regex looking for
"<cfquery*" "</cfquery>" and "<cfstoredproc*" "</cfstoredproc>"
In addition if you have tests that have good code coverage or even just feel like the app is fully exercised in production you could try turning on "Log Database Calls" in Admin - > Datasources or maybe even at the JDBC driver level, just monitor performance to make sure it does not slow the site down unacceptably.
In short: no. You'd have to do alot of tricky parsing to make sure you get all the SQL. And because you can glob SQL together from lots of strings, you'll almost always miss some of it.
The best you're likely to do will be a case insensitive grep for "SELECT|INSERT|UPDATE|DELETE" and then manually pulling out the table names.
Depending on how the code is structured, you might be able to get the table names by regexing the SQL from clause. But that's not foolproof. Alot of people use string concatenation to build SQL statements. This is bad because it can introduce SQL injection attacks, and it also make this particular problem harder.