I was wondering if there is a way for Flyway to accept an actual SQL migration as a string or a stream instead of searching for it on a classpath?
I'm constructing the SQL migration in Java on the fly and would like to call Flyway API and pass the migration as a paramter.
Please, let me know if this is possible.
Thank you
Not entirely what you are asking for, but looks like Java-based migrations might be a solution.
Basically instead of V1_0__script.sql you write V1_0__script.java class implementing JdbcMigration. Inside that class you have access to JDBC Connection:
class V1_0__script implements JdbcMigration {
public void migrate(Connection connection) throws Exception {
//...
}
}
In migrate() you are free to run your custom SQL queries.
There is no API available for this.
However, if you construct your SQL on the fly, it surely must be possible to construct it one statement at a time. Each statement can then be executed using the Connection parameter you get in a JdbcMigration
Related
Does RocksJavaAPI have the support for transactions? I see that there is a Transaction DB class present in the JAR. I am not able to do a begin transaction on transaction Db class.
RocksDB db = TransactionDB.open(options, "/Users/jagannathan/Desktop/My Files/db/rocksdb")
I am not able to do db.beginTransaction as such methods are not available. Any pointers on how to accomplish in Java are appreciated.
You need to use a different open method. You currently use the open method of the base class (RocksDB).
Use either:
public static TransactionDB open(Options options,
TransactionDBOptions transactionDbOptions,
java.lang.String path)
or
public static TransactionDB open(DBOptions dbOptions,
TransactionDBOptions transactionDbOptions,
java.lang.String path,
java.util.List<ColumnFamilyDescriptor> columnFamilyDescriptors,
java.util.List<ColumnFamilyHandle> columnFamilyHandles)
To get a TransactionDB object. This object you can then use to call #beginTransaction, which will return an Transaction object. This transaction can then be used similar to a RocksDB, where you can put, delete etc. and commit if you're done.
I'm currently working on a C# UWP application that runs on Windows 10 IoT Core OS on an ARM processor. For this application, I am using a SQLite DB for my persistence, with Entity Framework Core as my ORM.
I have created my own DBContext and call the Migrate function on startup which creates my DB. I can also successfully create a DBContext instance in my main logic which can successfully read/write data using the model. All good so far.
However, I've noticed that the performance of creating a DbContext for each interaction with the DB is painfully slow. Although I can guarantee that only my application is accessing the database (I'm running on custom hardware with a controlled software environment), I do have multiple threads in my application that need to access the database via the DbContext.
I need to find a way to optimize the connection to my SQLite DB in a way that is thread safe in my application. As I mentioned before, I don't have to worry about any external applications.
At first, I tried to create a SqliteConnection object externally and then pass it in to each DbContext that I create:
_connection = new SqliteConnection(#"Data Source=main.db");
... and then make that available to my DbContext and use in in the OnConfiguring override:
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlite(_connection);
}
... and then use the DbContext in my application like this:
using (var db = new MyDbContext())
{
var data = new MyData { Timestamp = DateTime.UtcNow, Data = "123" };
db.MyData.Add(data);
db.SaveChanges();
}
// Example data read
MyDataListView.ItemsSource = db.MyData.ToList();
Taking the above approach, I noticed that the connection is closed down automatically when the DbContext is disposed, regardless of the fact that the connection was created externally. So this ends up throwing an exception the second time I create a DbContext with the connection.
Secondly, I tried to create a single DbContext once statically and share it across my entire application. So instead of creating the DbContext in a using statement as above, I tried the following:
// Where Context property returns a singleton instance of MyDbContext
var db = MyDbContextFactory.Context;
var data = new MyData { Timestamp = DateTime.UtcNow, Data = "123" };
db.MyData.Add(data);
db.SaveChanges();
This offers me the performance improvements I hoped for but I quickly realized that this is not thread safe and wider reading has confirmed that I shouldn't do this.
So does anyone have any advice on how to improve the performance when accessing SQLite DB in my case with EF Core and a multi-threaded UWP application? Many thanks in advance.
Secondly, I tried to create a single DbContext once statically and share it across my entire application. So instead of creating the DbContext in a using statement as above, I tried the following...This offers me the performance improvements I hoped for but I quickly realized that this is not thread safe and wider reading has confirmed that I shouldn't do this.
I don't know why we shouldn't do this. Maybe you can share something about what you read. But I think, you can make the DBContext object global and static and when you want to do CRUD, you can do it in main thread like this:
await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
//App.BloggingDB is the static global DBContext defined in App class
var blog = new Blog { Url = NewBlogUrl.Text };
App.BloggingDB.Add(blog);
App.BloggingDB.SaveChanges();
});
But do dispose the DBContext at a proper time as it won't automatically get disposed.
I'am working on a Symfony app that provides a rest web service (simple HTTP Request with JSON).
That service check some rules and inserts few lines in two MySQL table (write only).
For optimize reason, even if Doctrine bundle is available, i use native MySQL Query (with bind params) to insert this lines.
My need is : If for any reason, the database is not available, write "runnables" queries into a log file.
The final purpose is that when database is back, i want to be able to execute directly the file's content on the database.
Note that there is no unique constraint (pk is a generated uuid) and no lock or transaction to handle (simple insert statements).
I write a custom SQLLogger, but when $connection->insert(...) is called, the connect fail before logger is called.
So, my question is : There is a way to get the final query (with binded parameters) without database connection ?
Or should i rewrite the mecanism that bind params into query and log it myself when database is not available ?
Best regards,
Julien
As the final query with parameters is build by the database, there is just no way to build the query with PHP and to be garanteed that the query will be the same as the database.
The only way si to build query without binded parameters, but this is clearly not a good practice.
So, i finally decided to store all the JSON (API request body) in a file if the database is not available.
So when the database is back, instead of replay SQL queries, i can replay the original HTTP query.
Hope this late self-anwser will help someone.
Best regards.
I am using PersistenceIOParticipant in WF4.0 to save something into database together with the persistence of the workflow instance. I have no idea that how to use the same connection object with the workflow persistence and I am forced to use the distributed transaction. Are there any ways to avoid using DTC?
I found the WF4 Sample project "WorkflowApplication ReadLine Host" useful
to see an example of persistenceIOParticipant in action.
I toggled the booleans in the constructor to verify that a transaction was being used and that
MSDTC was required.
See http://msdn.microsoft.com/en-us/library/dd764467.aspx
If using SQL Server 2008+, then it shouldn't matter if multiple connections are required. After using reflector on the SqlWorkflowInstanceStore, I discovered it was setting some additional properties on the connection string. Here is the code it uses to create a connection string:
SqlConnectionStringBuilder builder2 = new SqlConnectionStringBuilder(connectionString);
builder2.AsynchronousProcessing = true;
builder2.ConnectTimeout = (int)TimeSpan.FromSeconds(15.0).TotalSeconds;
builder2.ApplicationName = "DefaultPool";
SqlConnectionStringBuilder builder = builder2;
return builder.ToString();
I verified with profiler that MSDTC is not involved when using a custom IO participant and this connection string code. Don't forget to pass true to the base PersistenceIOParticipant constructor and flow Transaction.Current appropriately. Obviously, Microsoft could change that at anytime so use at your own discretion.
I'm trying to create a bunch of MS visual studio unit tests for my n-tiered web app but for some reason I can't run those tests and I get the following error -
"Object reference not set to an
instance of an object"
What I'm trying to do is testing of my data access layer where I use LINQ data context class to execute a certain function and return a result,however during the debugging process I found out that all the tests fail as soon as they get to the LINQ data context class and it has something to do with the connection string but I cant figure out what is the problem.
The debugging of tests fails here(the second line):
public EICDataClassesDataContext() :
base(global::System.Configuration.ConfigurationManager.ConnectionStrings["EICDatabaseConnectionString"].ConnectionString, mappingSource)
{
OnCreated();
}
And my test is as follows:
TestMethod()]
public void OnGetCustomerIDTest()
{
FrontLineStaffDataAccess target = new FrontLineStaffDataAccess(); // TODO: Initialize to an appropriate value
string regNo = "jonh"; // TODO: Initialize to an appropriate value
int expected = 10; // TODO: Initialize to an appropriate value
int actual;
actual = target.OnGetCustomerID(regNo);
Assert.AreEqual(expected, actual);
}
The method which I call from DAL is:
public int OnGetCustomerID(string regNo)
{
using (LINQDataAccess.EICDataClassesDataContext dataContext = new LINQDataAccess.EICDataClassesDataContext())
{
IEnumerable<LINQDataAccess.GetCustomerIDResult> sProcCustomerIDResult = dataContext.GetCustomerID(regNo);
int customerID = sProcCustomerIDResult.First().CustomerID;
return customerID;
}
}
So basically everything fails after it reaches the 1st line of DA layer method and when it tries to instantiate the LINQ data access class...
I've spent around 10 hours trying to troubleshoot the problem but no result...I would really appreciate any help!
UPDATE:
Finally I've fixed this!!!!:) I dont know why but for some reasons in the app.config file the connection to my database was as follows:
AttachDbFilename=|DataDirectory|\EICDatabase.MDF
So what I did is I just changed the path and instead of |DataDirectory| I put the actual path where my MDF file sits,i.e
C:\Users\1\Documents\Visual Studio 2008\Projects\EICWebSystem\EICWebSystem\App_Data\EICDatabase.mdf
After I had done that it worked out!But still it's a bit not clear what was the problem...probably incorrect path to the database?My web.config of ASP.NET project contains the |DataDirectory|\EICDatabase.MDF path though..
Is LINQDataAccess.EICDataClassesDataContext looking to the web.config or some other outside source of data for its setup?
I can tell you for a fact that you must jump thru hoops to get web.config accessible to your test code.
Update
Ah, yes. I see that you're using ConfigurationManager on the line where your test fails... ConfigurationManager looks to web.config for configuration. This has been a sticking point for me when I write my tests.
You need to either change the code so that the class can be instantiated without web.config, or you need to make it so that your tests can access web.config.
Does your test project have it's own configuration file? This type of behavior usually means the app can't find the connection string. Test projects require their own file since they are not running in the context of the client app.
UPDATE The error you describe after adding an app.config is common when testing web applications built on SQLExpress and attaching an .mdf. SQLExpress cannot be run in more than one process at a time. So if you have previously run your web application it may still be active and will conflict with the attempt to attach the database.
You can use SQL Management Studio to attach the database permanently and then use a more traditional connection string like:
Server=myServer;Database=EICDatabase;Trusted_Connection=True;
For me it seems like your problem is the connection string, which is not set.
I assume your unit test is in a different project than the DAL.
You call the 'new' command on the datacontext constructor without a connection string. So it should usually use its default, when set. But since this setting normally is stored in the web.config of the other project there is no connection string set and you get the error.
If its right, what i assume, you have to get the settings from the DAL project into the unit-testing project. Simplest solution should be to copy web.config connection string to app.config of unit test project.
There are other possibilities for sure, but i think its not easy to get the web.config configuration into your unit-testing project.