I develop in the last days a web page (http://www.srtbox.com/) for testing my architecture, more info here. With NoRM, MongoLab or MongoHQ for DB hosting. And I having a a lot of errors with NoRM. All with the Norm.BSON.BsonDeserializer class. I could fix one, but now im getting some errors in the connection. Error:
System.Net.Sockets.SocketException
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 count)
at System.IO.BinaryReader.ReadBytes(Int32 count)
at Norm.BSON.BsonDeserializer.Deserialize[T](BinaryReader stream, Int32 length)
The truth is that this NoRM driver seems really unstable... The official driver does not offer Linq support and I did not find one example with a POCO object. Some one have a good experience with and NoSQL database and asp.net?? Scalable like MongoDB? RavenDB seems a nice option, but MongoDB have more success stories, of course without .Net .
So the big question is: Which NoSQL Scalable DB will you recommend for ASP.Net? Do you have any success story?
PD: I will be thankful if you visit my site: (http://www.srtbox.com/) for testing.
That looks like a network error, not a problem with the driver. If the database isn't local you will see that if you have a bad connection or didn't set up the firewall right - not much you can do differently in code to change it. Trying to keep a single connection open for too long can also cause connection errors.
Most of the NoSQL databases available work well with .NET so you can choose based on functional requirements rather than .NET compatibility. However, you shouldn't expect it to work just like SQL or have that many examples - most of them are used primarily on other platforms and since they all have different ways of running queries, LINQ isn't always a good fit anyway.
Also, what do you mean by POCO? The serialization attributes can make the classes look complicated, but they are just regular objects, not the lazy loaded self updating objects you get from a typical ORM.
Related
I have a Windows 2008 R2 server that hosts many back end NServiceBus endpoints. All of the services that rely on the NServiceBus.Host.exe host (installed as Windows Services) are able to interact with MSDTC perfectly, averaging a small handful of concurrent distributed transactions throughout the day. There are 2 small Web.API applications, however, that self host NServiceBus endpoints (as publishers) that constantly receive the following error when trying to process subscription requests:
NServiceBus.Transports.Msmq.MsmqDequeueStrategy Error in receiving
messages. System.Transactions.TransactionAbortedException: The
transaction has aborted. --->
System.Transactions.TransactionManagerCommunicationException:
Communication with the underlying transaction manager has failed. --->
System.Runtime.InteropServices.COMException: The Transaction Manager
is not available. (Exception from HRESULT: 0x8004D01B) at
System.Transactions.Oletx.IDtcProxyShimFactory.ConnectToProxy(String
nodeName, Guid resourceManagerIdentifier, IntPtr managedIdentifier,
Boolean& nodeNameMatches, UInt32& whereaboutsSize, CoTaskMemHandle&
whereaboutsBuffer, IResourceManagerShim& resourceManagerShim) at
System.Transactions.Oletx.DtcTransactionManager.Initialize() ---
End of inner exception stack trace --- at
System.Transactions.Oletx.OletxTransactionManager.ProxyException(COMException
comException) at
System.Transactions.Oletx.DtcTransactionManager.Initialize() at
System.Transactions.Oletx.DtcTransactionManager.get_ProxyShimFactory()
at
System.Transactions.Oletx.OletxTransactionManager.CreateTransaction(TransactionOptions
properties) at
System.Transactions.TransactionStatePromoted.EnterState(InternalTransaction
tx) --- End of inner exception stack trace --- at
System.Transactions.TransactionStateAborted.CheckForFinishedTransaction(InternalTransaction
tx) at System.Transactions.Transaction.Promote() at
System.Transactions.TransactionInterop.ConvertToOletxTransaction(Transaction
transaction) at
System.Transactions.TransactionInterop.GetDtcTransaction(Transaction
transaction) at
System.Messaging.MessageQueue.StaleSafeReceiveMessage(UInt32 timeout,
Int32 action, MQPROPS properties, NativeOverlapped* overlapped,
ReceiveCallback receiveCallback, CursorHandle cursorHandle, IntPtr
transaction) at
System.Messaging.MessageQueue.ReceiveCurrent(TimeSpan timeout, Int32
action, CursorHandle cursor, MessagePropertyFilter filter,
MessageQueueTransaction internalTransaction,
MessageQueueTransactionType transactionType) at
System.Messaging.MessageQueue.Receive(TimeSpan timeout,
MessageQueueTransactionType transactionType) at
NServiceBus.Transports.Msmq.MsmqDequeueStrategy.ReceiveMessage(Func`1
receive) in
c:\BuildAgent\work\31f8c64a6e8a2d7c\src\NServiceBus.Core\Transports\Msmq\MsmqDequeueStrategy.cs:line
313
Some other notes:
Both the erroring ApplicationPools' identities and the Windows
Services' Log On users are the same.
This actually worked well before
a recent reboot, as the Web.API services were able to successfully
process subscription requests, and are able to publish messages just
fine (though publishing does not automatically use MSDTC, and we are
not using a TransactionScope explicitly). Since the local reboot, we
simply get the above error if a subscription request message sits
in either of the Web.API publisher's input queue.
I've used both procmon.exe and MSDTC tracing and have found nothing of interest. The typical event viewer logs also do not provide any information.
All endpoints are running .NET 4.5 and NServiceBus 4.6
We cannot
recreate this in any other environment.
Additional notes from below conversations
The thread which throws the exception is pure NServiceBus subscription management where none of "my" code is involved. When the application pool starts the w3wp.exe worker process on demand, NSB is spawning a worker thread unbeknownst to the application to process subscription requests. It should only ever work across the publisher's input queue and the subscription storage, which I'm using MSMQ for that as well, in a queue right beside the other (i.e. no other server is involved to my knowledge).
The "code" of the website didn't change across reboots, and the application pool stopped and restarted several times before the reboot without issue.
Not really an answer, but too long for a comment.
What part of your operation requires DTC? A Distributed Transaction gets enlisted automatically when needed, usually when you are talking to two different DTC-supporting bits of infrastructure (e.g. MSMQ and a database).
You said you tested via DTC tracing--do you mean DTC Ping? Did you test by having it run on both machines (or all machines if there are more than two involved in the transaction)? The DTC tool is pretty esoteric, and its output can be confusing.
Also, if it did work before the reboot, is it possible the reboot reset firewall settings? Firewalls are a common cause of DTC problems.
Also, I assume you checked and rechecked your DTC settings on the local machine? Did you ensure that your MSMQ queues are set up to be transactional?
From your comments:
Note that this particular failure occurs when attempting to dequeue a
message from a local private MSMQ queue [...]
The stack trace makes it appear that that's all it's doing, but I suspect that as it is attempting dequeue it is also trying to enlist the transaction between multiple servers. See below.
Why MSDTC? It's the original way to support exactly-once messaging in
NServiceBus (see here).
Right, but what I'm asking is why the particular operation requires a distributed transaction. If all a handler is doing is reading from a queue and (for example) writing output to the console, MSDTC will never be enlisted, even though the handler is wrapped in a transaction scope. It will simply use a local transaction to read from the queue. The escalation to a distributed transaction is automatic, and only happens when it is needed to support multiple bits of infrastructure.
So if you recently deployed code in a handler that writes data to a new database server, you may be getting a failure because you are now enlisting a transaction that includes the new server, which may be where the failure is happening.
So determining all the servers involved in the distributed transaction is the first step. The next step would be to check the DTC settings on all involved servers. If DTC settings aren't the problem, I'd recommend testing communication between the servers using DTCPing. The NServiceBus documentation has some good instructions for using DTCPing.
What "fixed" this for us in the production environment was adding the application pool identity user to the local Administrators group on the server. Unfortunately we don't have time to determine what setting required that security setup, as this isn't a required configuration in other similar servers. Also, this isn't the most desirable solution from a security perspective, but in our particular situation, we're willing to live with it.
When using web services (we're specifically using asmx and WCF) with ASP.NET, what is the best way to establish a SQL connection? Right now, I'm establishing a new connection for each web service call, but I'm not convinced this will be too efficient when there will be thousands of users connecting. Any insight on this topic would be much appreciated.
What you are doing is fairly standard.
Assuming you are using the same connection string, the connections will be coming from the connection pool, which is the most efficient way to get connections already.
Only doing the work required and closing the connection on each call is good practice.
One thing you can do is cache results and return the cached results for calls that are not likely to result in changed data over the life of the cache item. This will reduce database calls.
It is strongly recommended that you always close the connection when you are finished using it so that the connection will be returned to the pool. You can do this using either the Close or Dispose methods of the Connection object, or by opening all connections inside a using statement in C#. Connections that are not explicitly closed might not be added or returned to the pool.
You should add "Pooling = true" (and add a non-zero "Min Pool Size") to the connection string.
Let the provider handle connection pooling for you; don't try to do better than it - you will fail.
With the default connection settings the provider will maintain a connection pool. When you close/dispose, the connection is actually just released to the pool. it is not necessarily really closed.
By default, SqlConnections make use of connection pooling, which will allow the system to manage the re-use of previous connection objects rather than truly creating "new" connections for each request - up to a pool maximum value. And its built-in, so you don't really have to do anything to leverage it.
Writing your own pooling/connection manager is fraught with peril, and leads to all manner of evil, so it seems to me allowing the system to manage your connections from the pool is probably your best bet.
I have heard the term connection pooling and looked for some references by googling it... But can't get the idea when to use it....
When should i consider using
connection pooling?
What are the advantages and
disadvantagesof connection pooling?
Any suggestion....
The idea is that you do not open and close a single connection to your database, instead you create a "pool" of open connections and then reuse them. Once a single thread or procedure is done, it puts the connection back into the pool and, so that it is available to other threads. The idea behind it is that typically you don't have more than some 50 parallel connections and that opening a connection is time- and resource- consuming.
When should i consider using
connection pooling?
Always for production system.
What are the advantages and
disadvantages of connection pooling?
Advantages:
Performance. Use a fixed pool of connection and avoid the costly creation and release of connections.
Shared infrastructure. If your database is shared between several apps, you don't want one app to exhaust all connections. Pooling help to limit the number of connection per app.
Licensing. Depending on your database license, the number of concurrent client is limited. You can set a pool with the number of authorized connections. If no connection is available, client waits until one is available, or times out.
Connectivity issue. The connection pool that is between the client and the database, can provide handy features such as "ping" test, connection retry, etc. transparently for the client. In worse case, there is a time-out.
Monitoring. You can monitor the pool, see the number of active connections, etc.
Disadvantage:
You need to set it up and configure it, which is really peanuts usually.
You should use connection pooling whenever the time to establish a connection is greater than zero (pretty much always) and when there is a sufficient average usage such that the connection is likely to be used again before it times out.
Advantages are it's much faster to open/close new connections as they're not really opened and closed, they're just checked out/in to a pool.
Disadvantage would be in some connection pools you'll get an error if all pooled connections are in use. This usually is a good thing as it indicates a problem with the calling code not closing connections, but if you legitimately need more connections than are in the pool and haven't configured it properly, you could get errors where you wouldn't otherwise.
And of course there will be other pros and cons depending on the specific environment you're working in and database.
In .NET, if you are using the same connection string for data access then you already have connection pooling. The concept is to reuse an idle connection without having to tear it down & recreate it, thereby saving server resources.
This is of-course taking into consideration that you are closing open connections upon completion of your work.
connection pooling enables re-use of an existing, but not used database connection. by using it you eliminate the overhead of the connection/disconnection to the database server. it provides a significant performance boost in most cases. the only reason i can think of not to use it is if your software won't be connecting frequently enough to keep the connections alive or if there's some bug in the pooling layer.
If we required to communicate with the database multiple times then it is not recommended to create a separate Connection Object every time, because creating and destroying connection object impacts performance.
To overcome this problem we should use a connection pool.
If we want to communicate with the database then we request a connection pool to provide a connection. Once we got the connection, by using it we can communicate with the database.
After completing our work, we can return the connection object back to the pool instead of destroying it.
The main advantage of connection pooling is to reuse the same connection object multiple times.
I have a BizTalk solution deployed at a customer site.
(BizTalk 2006 R2 with the non WCF adapters)
One of the orchestrations issues an RFC call to a SAP system and returns a big message.
The message is mapped to a SQL Adapter Send Port schema which is calling a stored procedure.
Recently I'm running into a OutOfMemory exception when the message is sent to the SQL Adapter.
I did a trace with SQL profiler which resulted in a 90MB+ tracefile for just a few seconds.
As I imagined, the Stored Procedure is executed rapidly one after another.
But even this does not explain why I get the error message. The Server has 32GB RAM and I can't even see it peaking out...
Any suggestions what else I could check?
Have you looked at how much memory the host of the Send Port is configured to use? 32GB of memory is irrelevant if the host is not set to take advantage of it. That being said, I think the default is 25% of the available memory. So barring any change this may not be the problem.
You could also try playing with the throttling settings for the host. Although I have never had much luck with it myself.
One way to get more information is to hook up Perfmon and look at the memory around that specific host process.
I've got a number of ASP.Net websites (.Net v3.5) running on a server with a SQL 2000 database backend. For several months, I've been receiving seemingly random InvalidOperationExceptions with the message "Internal connection fatal error". Sometimes there's a few days in between, while other times there are multiple errors per day.
The exception is not limited to one site in particular, though they share business and data access assemblies. The error seems to always be thrown from SqlClient.TdsParser.Run(). It sometimes is thrown from old-school direct SqlCommand.Execute() calls, while other times it is thrown from Linq2Sql code.
I've been assured by the network guys that there are no errors or packets lost on their end. Has anyone else experienced this? Could it be a driver problem? We have been unable as of yet to pinpoint a specific trigger for this exception.
We're running II6 on Windows Server 2003.
After a few months of ignoring this issue, it started to reach a critical mass as traffic gradually increased. Under heavy load, including some crawlers, things got crazy and these errors poured in nonstop.
Through trial and error, we eventually tracked down a handful of SqlCommand or LINQ queries whose SqlConnection wasn't closed immediately after use. Instead, through some sloppy programming originating from a misunderstanding of LINQ connections, the DataContext objects were disposed (and connections closed) only at the end of a request rather than immediately.
Once we refactored these methods to immediately close the connection with a C# "using" block (freeing up that pool for the next request), we received no more errors. While we still don't know the underlying reason that a connection pool would get so mixed up, we were able to cease all errors of this type. This problem was resolved in conjunction with another similar error I posted, found here: Why is my SqlCommand returning a string when it should be an int?
Sounds like the database connection is getting dropped or timing out.
We recently had similar issues moving to IIS 6 from IIS 5 connecting to SQL 2000. Our issue was solved by increasing number of ephemeral ports available.
Look at the usage of the ephemeral ports by the IIS server. The default max no. of ports available is normally 4000. You might want to consider increasing this if the sites on your server are particularly busy or your application is making a lot of database calls.
You can monitor these first to see if going over max limit.
Search Microsoft Knowledge base for "MaxUserPort" and "TcpTimedWaitDelay" and make necessary registry changes. Make sure you back up registry or snapshot server before making the changes. Will need to reboot for changes to take effect.
You should double check your database and recordset connection are being closed after use. Not closing will use up this port range unnecessarily.
Check the efficiency of your stored procedures anyway as they might be taking longer than they need too.
"If you rapidly open and close 4000 sockets in less than four minutes, you will reach the default maximum setting for client anonymous ports, and new socket connection attempts fail until the existing set of TIME_WAIT sockets times out." - from http://support.microsoft.com/kb/328476
Check your server's LOG folder (\program files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG or similar) for files named SqlDump*.mdmp and SqlDump*.txt. If you do find any you'll have to take it to Product Support.
I was creating a new EF Core project and was trying to create the database to an external Linux server instead of a Windows Server or local one. After hours of searching I found out that I am using MySQL instead of the Microsoft SQL server.
I found it weird that everyone was using 1433 instead of the usual 3306. So to fix my 'Internal connection fatal error' I had to set up a docker instance of SQL Server bound to its default port of 1433.
It literally was that simple. In the docker repo look for "microsoft-mssql-server" and run the image as described neatly in the description below. Everything works now and I am able to push my database from my EF Core project to an external server.