TransactionScope: nested transactions with different database connections (SQL Server & Postgresql) - .net-core

I am writing an SDK method with transaction using NpgsqlConnection for others to use.
When they were calling my method, they used SqlConnection with another transaction to wrap their DB stuff and my SDK's DB stuff.
If I set my SDK method without a transaction, the outer code was fine and my SDK method could be rolled back. (Which was odd too. Still figuring out why.)
If I set my SDK method with a transaction though, the outer code crashed with a TransactionAbortedException:
System.Transactions.TransactionAbortedException : The transaction has aborted.
---- Npgsql.PostgresException : 55000: prepared transactions are disabled
Currently we're using enlist=false at the SDK's connection string to prevent the inner transaction from joining the outer one but I'd like to know the reason behind this behavior.
Here's the code I'm reproducing the problem with:
using (var scope = new TransactionScope(
TransactionScopeOption.Required,
new TransactionOptions
{
IsolationLevel = IsolationLevel.ReadCommitted,
},
TransactionScopeAsyncFlowOption.Enabled))
{
await using (var conn = new SqlConnection(#"Server=(localdb)\mssqllocaldb;Database=Test;ConnectRetryCount=0"))
using (var cmd = new SqlCommand("insert into [Test].[dbo].[Test] (Id, \"Name\") values (1, 'A')", conn))
{
await conn.OpenAsync();
var result = await cmd.ExecuteNonQueryAsync();
await SdkMethodToDoStuffWithNpgsql(1);
scope.Complete();
}
}
I had SdkMethodToDoStuffWithNpgsql() to mock a method in a repository with Postgres context injected.
public async Task SdkMethodToDoStuffWithNpgsql(long id)
{
var sqlScript = #"UPDATE test SET is_removal = TRUE WHERE is_removal = FALSE AND id = #id;
INSERT INTO log(id, data) SELECT id, data FROM log WHERE id = #id";
using (var scope = new TransactionScope(
TransactionScopeOption.RequiresNew,
new TransactionOptions
{
IsolationLevel = IsolationLevel.ReadCommitted,
},
TransactionScopeAsyncFlowOption.Enabled))
{
await using (var conn = new NpgsqlConnection(this._context.ConnectionString))
{
await conn.OpenAsync();
using (var cmd = new NpgsqlCommand(sqlScript, conn))
{
cmd.Parameters.Add(new NpgsqlParameter("id", NpgsqlDbType.Bigint) { Value = id });
await cmd.PrepareAsync();
var result = await cmd.ExecuteNonQueryAsync();
if (result != 2)
{
throw new InvalidOperationException("failed");
}
scope.Complete();
}
}
}
}

The above is the expected behavior - enlisting two connections in the same TransactionScope triggers a "distributed transaction"; this is known in PostgreSQL terminology as a "prepared transaction", and you must enable it in the configuration (this is the cause of the error you're seeing above). If the intention is to have two separate transactions (one for SQL Server, one for PostgreSQL) which commit separately, then opting out of enlisting is the right thing to do. You should also be able to use TransactopScopeOption.Suppress.
Note that distributed transactions aren't currently supported in .NET Core, only in .NET Framework (see this issue). So unless you're on .NET Framework, this won't work even if you enable prepared transactions in PostgreSQL.

Related

How can I perform search through files like queries to Windows Search platform using .net-core?

I need to search files by file name and content. Current implementation used Ole DB connection to windows search. But as I understand, Ole db wouldn't be implemented in .net core. I guess that I should use solution like Lucene .So I need advice, how to access windows search from .net-core at least or any ideas how to make that in cross-platform manner without windows search.
You basically have a couple of options. You can get hold of the FTQuery Tool (FTQuery.exe) in Windows 7 SDK and use Process Class in System.Diagnostics to get results by parsing standard output.
Alternatively you can create a separate ASP.NET Web API 2 project to act as a bridge between Windows Search and .NET Core using the following code as an action in your controller.
public async Task<IHttpActionResult> Post()
{
var data = new ArrayList();
var conn = new OleDbConnection("Provider=Search.CollatorDSO;Extended Properties='Application=Windows'");
await conn.OpenAsync();
string sql = await Request.Content.ReadAsStringAsync();
var cmd = new OleDbCommand(sql, conn);
using (var rdr = await cmd.ExecuteReaderAsync(CommandBehavior.CloseConnection))
{
while (await rdr.ReadAsync())
{
var row = new Dictionary<string, object>();
for (int i = 0; i < rdr.FieldCount; i++)
{
if (!rdr.IsDBNull(i))
row.Add(rdr.GetName(i), rdr.GetValue(i));
}
data.Add(row);
}
}
return Json(data);
}

Cannot execute Raw Sql in .Net COre

I am using Core 2.0 using entity framework.
I have successfully generated context using scaffold DBContext.
I have DBSet for table EMployee.
I need to execute SToredProcedure which will give list of employee.
I cannot see .FromSql nor.ExecuteCommand option.
I have added EntityFrameworkCore.SqlServer(2.0.1),EntityFrameworkCore.SqlServer.Design(1.1.5),Microsoft.VisualStudio.Web.CodeGeneration.Design(2.0.2) and EntityFrameworkCore.Tools(2.0.1) but to no awail.
Please guide for mentioned concerns.
If you want to execute row SQL using EF Core, try the following.
var employees = context.Employees
.FromSql("SELECT * FROM dbo.Employees")
// If you want to execute a stored procedure, then below
// .FromSql("EXECUTE {SP_NAME}")
.ToList();
But note, there are certain limitations present as described here:
https://learn.microsoft.com/en-us/ef/core/querying/raw-sql#limitations
This is the only way to execute Raw SQL in .NET at the moment:
var conn = _context.Database.GetDbConnection();
try
{
await conn.OpenAsync();
using (var command = conn.CreateCommand())
{
command.CommandText = "SELECT * From Table1 WHERE sender = #sender";
DbParameter sender = command.CreateParameter();
sender.ParameterName = "sender";
sender.Value = "Value";
command.Parameters.Add(sender);
DbDataReader reader = await command.ExecuteReaderAsync();
if (reader.HasRows)
{
while (await reader.ReadAsync())
{
int SubscriptionID = reader.GetInt32(0);
}
}
reader.Dispose();
}
}
finally { conn.Close(); }
You can use it for stored procedures as well.

Bulk updating data in DocumentDB

I have a desire to add a property with a default value to a set of documents that I retrieve via a SELECT query if they contain no value.
I was thinking of this in two parts:
SELECT * FROM c article WHERE article.details.locale = 'en-us'
I'd like to find all articles where article.details.x does not exist.
Add the property, article.details.x = true
I was hoping this EXEC command could be supported via the Azure Portal so I don't have to create a migration tool to run this command once but I couldn't find this option in the portal. Is this possible?
You can use Azure Document DB Studio as a front end to creating and executing a stored procedure. It can be found here. It's pretty easy to setup and use.
I've mocked up a stored procedure based on your example:
function updateArticlesDetailsX() {
var collection = getContext().getCollection();
var collectionLink = collection.getSelfLink();
var response = getContext().getResponse();
var docCount = 0;
var counter = 0;
tryQueryAndUpdate();
function tryQueryAndUpdate(continuation) {
var query = {
query: "select * from root r where IS_DEFINED(r.details.x) != true"
};
var requestOptions = {
continuation: continuation
};
var isAccepted =
collection
.queryDocuments(collectionLink,
query,
requestOptions,
function queryCallback(err, documents, responseOptions) {
if (err) throw err;
if (documents.length > 0) {
// If at least one document is found, update it.
docCount = documents.length;
for (var i=0; i<docCount; i++){
tryUpdate(documents[i]);
}
response.setBody("Updated " + docCount + " documents");
}
else if (responseOptions.continuation) {
// Else if the query came back empty, but with a continuation token;
// repeat the query w/ the token.
tryQueryAndUpdate(responseOptions.continuation);
} else {
throw new Error("Document not found.");
}
});
if (!isAccepted) {
throw new Error("The stored procedure timed out");
}
}
function tryUpdate(document) {
//Optimistic concurrency control via HTTP ETag.
var requestOptions = { etag: document._etag };
//Update statement goes here:
document.details.x = "some new value";
var isAccepted = collection
.replaceDocument(document._self,
document,
requestOptions,
function replaceCallback(err, updatedDocument, responseOptions) {
if (err) throw err;
counter++;
});
// If we hit execution bounds - throw an exception.
if (!isAccepted) {
throw new Error("The stored procedure timed out");
}
}
}
I got the rough outline for this code from Andrew Liu on GitHub.
This outline should be close to what you need to do.
DocumentDB has no way in a single query to update a bunch of documents. However, the portal does have a Script Explorer that allows you to write and execute a stored procedure against a single collection. Here is an example sproc that combines a query with a replaceDocument command to update some documents that you could use as a starting point for writing your own. The one gotcha to keep in mind is that DocumentDB will not allow sprocs to run longer than 5 seconds (with some buffer). So you may have to run your sproc multiple times and keep track of what you've already done if it can't complete in one 5 second run. The use of IS_DEFINED(collection.field.subfield) != true (thanks #cnaegle) in your query followed up by a document replacement that defines that field (or removes that document) should allow you to run the sproc as many times as necessary.
If you didn't want to write a sproc, the easiest thing to do would be to export the database using the DocumentDB Data Migration tool. Import that into Excel to manipulate or write a script to do the manipulation. Then upload it again using the Data Migration tool.

Cannot Perform Linq-To-Sql Stored Procedures with MVC-Mini-Profiler

I am using the MVC-Mini-Profiler in my ASP.net 4.0 C# environment along with MSSQL and Linq-to-SQL. I am having an issue with using L2S with the profiler. Whenever I return new DataClassesDataContext(), it allows me to get data from L2S stored procedures. But if try to return the Mvc-Mini-Profiler ProfilesDbConnection, I can get the data from the stored procedure on the first time after I build, but then ever time after that, zero data is returned. When it returns the DataClassesDataContext using ProfiledDbConnection, I can still iterate through the Db tables, but for some reason, the stored procedures do not allow me to send/receive data. Any ideas on why this might be?
try
{
var miniProfiler = MiniProfiler.Current;
var connstring = new DataClassesDataContext().Connection.ConnectionString;
SqlConnection connection = new SqlConnection(connstring);
var profiledConnection = ProfiledDbConnection.Get(connection);
var context = new DataClassesDataContext(profiledConnection);
return context;
}
catch
{
return new DataClassesDataContext();
}

Storing data from database in static property

I have got an ASP.Net website, where the data is brought in from ISeries.
The data connection to ISeries is quite slow and the speed is quite important for this website. Because of the slow speed of data retrieval from ISeries, I want to make as less database connections as possible.
So, I was thinking about storing tables from the database which rarely changes as static properties in my website. Whenevera user logs in I submit a thread which refreshes the data in the static property.
Is this approach correct? If not, what are the problems with this approach and what are the possible alternatives?
Example:-
For list of ports, I submit the below thread when user logs on:-
// Get Ports list
Thread threadPorts = new Thread(delegate()
{
Ports.getPortList();
});
threadPorts.Start();
Session["threadPorts"] = threadPorts;
In class Ports, there are 2 methods -
one for populating the static property PortList,
and the other checks if the thread is alive and waits for the thread to complete and retrieve the list of ports, once it is complete. The second method is the one which I use in my application whenever I need the list of ports (populating a dropdown, etc).
public static void getPortList()
{
DataTable dt = new DataTable();
DB2Connection conn = new DB2Connection(ConfigurationManager.ConnectionStrings["db2IBM"].ConnectionString);
conn.Open();
string query = query to get ports from ISeries;
DB2Command cmd = new DB2Command(query, conn);
cmd.CommandType = CommandType.Text;
DB2DataAdapter adp = new DB2DataAdapter(cmd);
adp.Fill(dt);
cmd.Dispose();
conn.Close();
List<Port> list = new List<Port>();
foreach (DataRow row in dt.Rows)
{
list.Add(new Port(row[0].ToString(), row[1].ToString(), row[2].ToString(), row[3].ToString()));
}
StaticProp.PortList = list;
}
public static List<Port> getPortListfromSession()
{
List<Port> portList = new List<Port>();
if (System.Web.HttpContext.Current.Session["threadPorts"] != null)
{
Thread t = (Thread)System.Web.HttpContext.Current.Session["threadPorts"];
if (t != null)
{
if (t.IsAlive)
{
t.Join();
}
}
}
if (System.Web.HttpContext.Current.Session["threadPorts"] != null)
System.Web.HttpContext.Current.Session.Remove("threadPorts");
portList = StaticProp.PortList;
return portList;
}
I take it that ISeries, is an external database!
Why not take data from that database and stick it in your own, and update it separately?
You can then query your own database quickly, and update your database, as often as you see fit, alternatively you can use a file, I personally my preferred file data format is Json, over XML - but database is much better.

Resources