Hi and thanks for your attention.
First some background on the question:
I have an Inline Table Function which is used ubiquitously throughout an ASP.net application I have developed, running on IIS7. It's the basis for 30 or more stored procedures, all of while will normally process in a maximum of 200ms (more than fast enough). The application always uses the same connection string for all database connections. The function itself runs in a maximum of ~10ms.
Occasionally a call to a page which calls a stored procedure in which the function is used will result in a timeout after 15 seconds. This timeout applies to all users of the page, but other pages which also have stored procedures which use this function still perform normally while this is occurring, suggesting that it is a specific stored procedure having problems. However this has occurred on multiple pages, suggesting it is either multiple stored procedures or the function itself.
Running any (or all) of the stored procedures on the page from a management studio session with different user credentials is <200ms, even when a timeout is occurring for the web application.
Running an sp_recompile on the function will always "clear" the timeout, from any login credentials.
Because this is a critical portion of the application, sp_recompile is run as soon as possible, and little time is available for debugging. Also, I have never been able to recreate the timeout at will.
I've tried to do a lot of research on Inline Table Functions and haven't come across anything which suggests that this is a common problem with them and they should therefore be avoided.
The Question:
Is it possible that these timeouts are being caused by using the function, or are they guaranteed to be an issue with the stored procedures which rely upon it? To put it another way, is there likely to be any benefit in refactoring the stored procedures to use either a view or by encapsulating the appropriate logic inline?
My guess is that it is the stored procedures only and I'll likely solve this by adding optimize for unknown, option recompile where appropriate, or localising the parameters, but in all honesty I'd prefer to find a solution which applies to the underlying function such that I can apply a fix in a single location.
The function:
CREATE FUNCTION [dbo].[fn_ObjectIDs] (
#DateFrom AS DATETIME = NULL
,#DateTo AS DATETIME = NULL
,#Region AS INT = NULL
,#FamilyID AS INT = NULL
,#ParentID AS INT = NULL
,#ChildID AS INT = NULL
) RETURNS TABLE AS
RETURN
SELECT DISTINCT
obj.ID AS IDs
FROM tblObjects obj WITH (NOLOCK)
INNER JOIN tblFamily fam WITH (NOLOCK)
ON obj.famID = fam.famID
LEFT JOIN tblCountry cntry WITH (NOLOCK)
ON (#Region IS NOT NULL) AND (fam.countryId = cntry.countryId)
LEFT JOIN tblParent parent WITH (NOLOCK)
ON (#ParentID IS NOT NULL) AND (obj.ID = parent.objectID)
LEFT JOIN tblChild child WITH (NOLOCK)
ON (#ChildID IS NOT NULL) AND (obj.ID = child.objectID)
WHERE
obj.Visible = 1
AND obj.statusID IN (3,4,6,8)
AND ((#DateFrom IS NULL) OR (obj.CreatedDate >= #DateFrom))
AND ((#DateTo IS NULL) OR (obj.CreatedDate <= #DateTo))
AND ((#Region IS NULL) OR (cntry.regionID = #Region))
AND ((#FamilyID IS NULL) OR (obj.famID = #FamilyID))
AND ((#ParentID IS NULL) OR (parent.parentID = #ParentID))
AND ((#ChildID IS NULL) OR (child.childID = #ChildID))
Temporarily add some logging to verify what is actually causing the timeout problem. You can do this in your application and it the stored procedure itself. Have the procedure write the current timestamp and the parameters used to invoke the procedure to a logging tables when it's executed. Also add logging to your application. Then you'll be able to identify when specific timeouts occur if there are certain parameters causing the problem or if the problem is in the procedure at all.
In general user defined functions aren't a good idea, though my understanding is that a table inline function is better than some of the others. It adds a lot of overhead and the optimizer can't work properly with UDFs.
Related
We appear to have a problem with MDriven generating the same ECO_ID for multiple objects. For the most part it seems to happen in conjunction with unexpected process shutdowns and/or server shutdowns, but it does also happen during normal activity.
Our system consists of one ASP.NET application and one WinForms application. The ASP.NET app is setup in IIS to use a single worker process. We have a mixture of WebForms and MVC, including ApiControllers. We're using a rather old version of the ECO packages: 7.0.0.10021. We're on VS 2017, target framework is 4.7.1.
We have it configured to use 64 bit integers for object id:s. Database is Firebird. SQL configuration is set to use ReadCommitted transaction isolation.
As far as I can tell we have configured EcoSpaceStrategyHandler with EcoSpaceStrategyHandler.SessionStateMode.Never, which should mean that EcoSpaces are not reused at all, right? (Why would I even use EcoSpaceStrategyHandler in this case, instead of just creating EcoSpace normally with the new keyword?)
We have created MasterController : Controller and MasterApiController : ApiController classes that we use for all our controllers. These have a EcoSpace property that simply does this:
if (ecoSpace == null)
{
if (ecoSpaceStrategyHandler == null)
ecoSpaceStrategyHandler = new EcoSpaceStrategyHandler(
EcoSpaceStrategyHandler.SessionStateMode.Never,
typeof(DiamondsEcoSpace),
null,
false
);
ecoSpace = (DiamondsEcoSpace)ecoSpaceStrategyHandler.GetEcoSpace();
}
return ecoSpace;
I.e. if no strategy handler has been created, create one specifying no pooling and no session state persisting of eco spaces. Then, if no ecospace has been fetched, fetch one from the strategy handler. Return the ecospace. Is this an acceptable approach? Why would it be better than simply doing this:
if (ecoSpace = null)
ecoSpace = new DiamondsEcoSpace();
return ecoSpace;
In aspx we have a master page that has an EcoSpaceManager. It has been configured to use a pool but SessionStateMode is Never. It has EnableViewState set to true. Is this acceptable? Does it mean that EcoSpaces will be pooled but inactivated between round trips?
It is possible that we receive multiple incoming API calls in tight succession, so that one API call hasn't been completed before the next one comes in. I assume that this means that multiple instances of MasterApiController can execute simultaneously but in separate threads. There may of course also be MasterController instances executing MVC requests and also the WinForms app may be running some batch job or other.
But as far as I understand id reservation is made at the beginning of any UpdateDatabase call, in this way:
update "ECO_ID" set "BOLD_ID" = "BOLD_ID" + :N;
select "BOLD_ID" from "ECO_ID";
If the returned value is K, this will reserve N new id:s ranging from K - N to K - 1. Using ReadCommitted transactions everywhere should ensure that the update locks the id data row, forcing any concurrent save operations to wait, then fetches the update result without interference from other transactions, then commits. At that point any other pending save operation can proceed with its own id reservation. I fail to see how this could result in the same ID being used for multiple objects.
I should note that it does seem like it sometimes produces id duplicates within one single UpdateDatabase, i.e. when saving a set of new related objects, some of them end up with the same id. I haven't really confirmed this though.
Any ideas what might be going on here? What should I look for?
The issue is most likely that you use ReadCommitted isolation.
This allows for 2 systems to simultaneously start a transaction, read the current value, increase the batch, and then save after each other.
You must use Serializable isolation for key generation; ie only read things not currently in a write operation.
MDriven use 2 settings for isolation level UpdateIsolationLevel and FetchIsolationLevel.
Set your UpdateIsolationLevel to Serializable
With Doctrine and Symfony in my PHPUnit test method :
// Change username for user #1 (Sheriff Woody to Chuck Norris)
$form = $crawler->selectButton('Update')->form([
'user[username]' => 'Chuck Norris',
]);
$client->submit($form);
// Find user #1
$user = $em->getRepository(User::class)->find(1);
dump($user); // Username = "Sheriff Woody"
$user = $em->createQueryBuilder()
->from(User::class, 'user')
->andWhere('user.id = :userId')
->setParameter('userId', 1)
->select('
user
')
->getQuery()
->getOneOrNullResult()
;
dump($user); // Username = "Chuck Norris"
Why my two methods to fetch the user #1 return different results ?
diagnosis / explanation
I assume* you already created the User object you're editing via crawler before in that function and checked that it is there. This leads to it being a managed entity.
It is in the nature of data, to not sync itself magically with the database, but some automatism must be in place or some method executed to sync it.
The find() method will always try to use the cache (unless explicitly turned off, also see side note). The query builder won't, if you explicitly call getResult() (or one of its varieties), since you explicitly want a query to be executed. Executing a different query might lead to the cache not being hit, producing the current result. (it should update the first user object though ...) [updated, due to comment from Arno Hilke]
((( side note: Keeping objects in sync is hard. It's mainly about having consistency in the database, but all of ACID is wanted. Any process talking to the database should assume, that it only is working with the state at the moment of its first query, and is the only user of the database. Unless additional constraints must be met and inconsistent reads can occur, in which case isolation levels should be raised (See also: transactions or more precisely: isolation). So, automatically syncing is usually not wanted. Doctrine uses certain assumptions for performance gains (mainly: isolation / locking is optimistic). However, in your particular case, all of those things are of no actual concern... since you actually want a non-repeatable read. )))
(* otherwise, the behavior you're seeing would be really unexpected)
solution
One easy solution would be, to actively and explicitly sync the data from the database by either calling $em->refresh($user), or - before fetching the user again - to call $em->clear(), which will detach all entities (clearing the cache, which might have a noticable performance impact) and allowing you to call find again with the proper results being returned.
Please note, that detaching entities means, that any object previously returned from the entity manager should be discarded and fetched again (not via refresh).
alternate solution 1 - everything is requests
instead of checking the database, you could instead do a different request to a page that displays the user's name and checks that it has changed.
alternate solution 2 - using only one entity manager
using only one entity manager (that is: sharing the entity manager / database in the unit test with the server on the request) may be a reasonable solution, but it comes with its own set of problems. mainly omitted commits and flushes may avoid detection.
alternate solution 3 - using multiple entity managers
using one entity manager to set up the test, since the server is using a new entity manager to perform its work, you should theoretically - to do this actually properly - create yet another entity manager to check the server's behavior.
comment: the alternate solutions 1,2 and 3 would work with the highest isolation level, the initial solution probably wouldn't.
In a clustered intershop environment, we see a lot of error messages. I'm suspecting the communication between the application servers is not reliable.
Caused by: com.intershop.beehive.orm.capi.common.ORMException:
Could not UPDATE object: com.intershop.beehive.bts.internal.orderprocess.basket.BasketPO
Is there safe way to for the local application server, to load the latest instance.
BasketPO basket = null;
try{
BasketPOFactory factory = (BasketPOFactory) NamingMgr.getInstance().lookupFactory(BasketPOFactory.FACTORY_NAME);
try(ORMObjectCollection<BasketPO>baskets = factory.getObjectsBySQLWhere("uuid=?", new Object[]{basketID},CacheMode.NO_CACHING);){
if(null != baskets && !baskets.isEmpty()){
basket = baskets.stream().findFirst().get();
}
}
}
catch(Throwable t){
Logger.error(this, t.getMessage(),t);
}
Does the ORMObject#refresh method help ?
try{
if(null != basket)
basket.refresh();
}
catch(Throwable t){
Logger.error(this, t.getMessage(),t);
}
You experience that error because an optimistic lock "fails". To understand the problem better I'll try to explain how the optimistic locking works in particular in the Intershop ORM layer.
There is a column named OCA in the PO tables (OCA == optimistic control attribute?). Imagine that two servers (or two different threads/transactions) try to update the same row in a table. For performance reasons there is no DB locking involved by default (e.g. by issuing select for update). Instead the first thread/server increments the OCA by one when it updates the row successfully within its transaction.
The second thread/server knows the value of the OCA from the time that it created its own state. It then tries to update the row by issuing a similar query:
UPDATE ... OCA = OCA + 1 ... WHERE UUID = <uuid> AND OCA = <old_oca>
Since the OCA is already incremented by the first thread/server this update fails (in reality - updates 0 rows) and the exception that you posted above is thrown when the ORM layer detects that no rows were updated.
Your problem is not the inter-server communication but rather the fact that either:
multiple servers/threads try to update the same object;
there are direct updates in the database that bypass the ORM layer (less likely);
To solve this you may:
Avoid that situation altogether (highly recommended by me :-) );
Use the ISH locking framework (very cumbersome imHo);
Use pesimistic locking supported by the ISH ORM layer and Oracle (beware of potential performance issues, deadlocks, bugs);
Use Java locking - but since the servers run in different JVM-s this is rarely an option;
OFFTOPIC remarks: I'm not sure why you use getObjectsBySQLWhere when you know the primary key (uuid). As far as I remember ORMObjectCollection-s should be closed if not iterated completely.
UPDATE: If the cluster is not configured correctly and the multicasts can't be received from the nodes you won't be able to resolve the problems programatically.
The "ORMObject.refresh()" marks the cached shared state as invalid. Next access to the object reloads the state from the database. This impacts the performance and increase the database server load.
BUT:
The "refresh()" method does not reload the PO instance state if it already assigned to the current transaction.
Would be best to investigate and fix the server communication issues.
Other possibility is that it isn't a communication problem (multicast between node in the cluster i assume), but that there are simply two request trying to update the basket at the same time. Example two ajax request to update something on the basket.
I would avoid trying to "fix" the orm, it would only cause more harm than good. Rather investigate further and post back more information.
In web.config you can enable session compression.
<sessionState mode="InProc" customProvider="DefaultSessionProvider" compressionEnabled="true" >
What are positive and negative sides of this action?
Well, on the positive side, you need less space.
On the negative side, it needs time to compress, so it's slower.
Let me add, that in my opinion, if you use sessions at all, you've made an architectural mistake (exceptions my apply to this rule, but very very rarely).
It's not a good idea, because if a page writes something in a session, this gets overwritten if I simultanously open the same page in another browser window (it's the same session).
And because InProc sessions expire when you change something in the web.config file, you can create unlimited number of bugs for EVERY currently active user...
Plus you loose inProc sessions, if the VM gets moved to another server (cloud environments, failover, dynamic scaleOut).
Also, the InProc provider doesn't require objects to be marked as serializable.
If you change to, for example, an SQL session provider, you'll get exceptions in all places where you put an object that hasn't been marked as serializable into the session.
For example, when you need to query all the locations a user may access (according to portofolio rights in T_SYS_LocationRights):
You get the UserID from the formsAuth-cookie, then use it as the parameter:
DECLARE #userID integer
SET #userID = 12435
SELECT * FROM T_Locations
WHERE (1=1)
AND
(
(
SELECT ISNULL(MAX(CAST(T_SYS_LocationRights.LR_IsRead AS integer)), 0)
FROM T_SYS_LocationRights
INNER JOIN T_User_Groups
ON T_User_Groups.USRGRP_GRP = T_SYS_LocationRights.LR_GRANTEE_ID
WHERE T_SYS_LocationRights.LR_LC_UID = T_Locations.LC_UID
AND T_User_Groups.USRGRP_USR = #userID
) = 1
)
Don't just query something after the maxim:
if you'll ever need it, it's already there.
Design a web-application (which is multi-threaded by design) after that maxim, is a very bad idea.
If you don't need it, don't query it.
If you need it, query it.
If you needed it, don't store it in the session, it's better to query it again, if necessary
You can win much more time by executing all database operations at once, get all the data you need into a System.Data.DataSet (in one query-operation, one connection open-and-close), and then use that. When the page reloads, you can always reload the data (as a matter of fact, you even should).
Don't use the session as cache. It's not the cache
I am using ASP.net Identity 2.0 with a user id of an integer. Performing a password update is an incredibly expensive database operation with 2 (unneeded) queries both averaging 128,407 db time units, or about a 7 in the query plan, based on the amount of data I have.
Code I am calling (either async or sync are the same)
var result = await UserManager.ChangePasswordAsync(userId, oldPassword, newPassword);
// or
var result = UserManager.ChangePassword(userId, oldPassword, newPassword);
In the database this causes two large sql calls which contain in their guts
AspNetUserRoles ... WHERE ((UPPER([Extent1].[Email])) = (UPPER(#p__linq__0))) ...
query 2:
AspNetUserRoles ... WHERE ((UPPER([Extent1].[UserName])) = (UPPER(#p__linq__0))) ...
From my perspective
There is no reason to call this sql at all - a lookup by the int ID is fast, and the sql it is calling is looking up role data.
using "Upper" is probably what makes it slow, and if there is no other better solution I can add a computed index (System.Web.Providers.DefaultMembershipProvider having performance issues/deadlocks)
At a high level my question is - is there a work around for this, or can someone from the Identity team fix the code (if it is indeed broken).
Update
The same behavior can be observed for the following calls (and probably many others)
UserManager.ResetPasswordAsync
UserManager.CreateAsync
Well you can certainly do it yourself instead of calling that method.
Imagine you have, in scope, a UserManager<ApplicationUser> called userManager and a DbContext called context.
var user = await context.Users.Single(u => u.Id == knownId);
// You can skip this if you don't care about checking the old password...
if (userManager.PasswordHasher.VerifyHashedPassword(user.HashedPassword, "myOldPassword") == PasswordVerificationResult.Failed) { return; };
user.HashedPassword = userManager.PasswordHasher.HashPassword("myNewPassword");
await context.SaveChangesAsync();
If that's still not optimized enough for you, DbContext exposes a Sql method, so you could just use the password hasher to get a hash, then issue a single Update() query.
I wouldn't do either of these things unless this is really a bottleneck or you have some other good reason, though. I'm doing something similar in my application because, at least in ASP.NET Identity v1, there's no method to just change the password without checking the old one.
I believe the answer provided by #emodendroket correctly and efficiently answered my original question. Due to the implications of needing to modify numerous parts of the API in the short term I am going with a simpler (but not as good solution)
Add a computed and persisted index on email and usernames
ALTER TABLE dbo.aspnetusers ADD UpperFieldEmail AS UPPER(email) PERSISTED
CREATE NONCLUSTERED INDEX IX_aspnetusers_UpperFieldEmail ON dbo.aspnetusers(UpperFieldEmail)
ALTER TABLE dbo.aspnetusers ADD UpperFieldUsername AS UPPER(username) PERSISTED
CREATE NONCLUSTERED INDEX IX_aspnetusers_UpperFieldUsername ON dbo.aspnetusers(UpperFieldUsername)