I am using ASP.net Identity 2.0 with a user id of an integer. Performing a password update is an incredibly expensive database operation with 2 (unneeded) queries both averaging 128,407 db time units, or about a 7 in the query plan, based on the amount of data I have.
Code I am calling (either async or sync are the same)
var result = await UserManager.ChangePasswordAsync(userId, oldPassword, newPassword);
// or
var result = UserManager.ChangePassword(userId, oldPassword, newPassword);
In the database this causes two large sql calls which contain in their guts
AspNetUserRoles ... WHERE ((UPPER([Extent1].[Email])) = (UPPER(#p__linq__0))) ...
query 2:
AspNetUserRoles ... WHERE ((UPPER([Extent1].[UserName])) = (UPPER(#p__linq__0))) ...
From my perspective
There is no reason to call this sql at all - a lookup by the int ID is fast, and the sql it is calling is looking up role data.
using "Upper" is probably what makes it slow, and if there is no other better solution I can add a computed index (System.Web.Providers.DefaultMembershipProvider having performance issues/deadlocks)
At a high level my question is - is there a work around for this, or can someone from the Identity team fix the code (if it is indeed broken).
Update
The same behavior can be observed for the following calls (and probably many others)
UserManager.ResetPasswordAsync
UserManager.CreateAsync
Well you can certainly do it yourself instead of calling that method.
Imagine you have, in scope, a UserManager<ApplicationUser> called userManager and a DbContext called context.
var user = await context.Users.Single(u => u.Id == knownId);
// You can skip this if you don't care about checking the old password...
if (userManager.PasswordHasher.VerifyHashedPassword(user.HashedPassword, "myOldPassword") == PasswordVerificationResult.Failed) { return; };
user.HashedPassword = userManager.PasswordHasher.HashPassword("myNewPassword");
await context.SaveChangesAsync();
If that's still not optimized enough for you, DbContext exposes a Sql method, so you could just use the password hasher to get a hash, then issue a single Update() query.
I wouldn't do either of these things unless this is really a bottleneck or you have some other good reason, though. I'm doing something similar in my application because, at least in ASP.NET Identity v1, there's no method to just change the password without checking the old one.
I believe the answer provided by #emodendroket correctly and efficiently answered my original question. Due to the implications of needing to modify numerous parts of the API in the short term I am going with a simpler (but not as good solution)
Add a computed and persisted index on email and usernames
ALTER TABLE dbo.aspnetusers ADD UpperFieldEmail AS UPPER(email) PERSISTED
CREATE NONCLUSTERED INDEX IX_aspnetusers_UpperFieldEmail ON dbo.aspnetusers(UpperFieldEmail)
ALTER TABLE dbo.aspnetusers ADD UpperFieldUsername AS UPPER(username) PERSISTED
CREATE NONCLUSTERED INDEX IX_aspnetusers_UpperFieldUsername ON dbo.aspnetusers(UpperFieldUsername)
Related
I have implemented MediatR in my asp.net core web api application.
Our controllers simply send a command or query to MediatR and return the result.
[HttpPost]
public async Task<IActionResult> CreateQuestion(CreateQuestionCommand command)
{
await Mediator.Send(command);
return Ok();
}
Due to the CQRS pattern that says commands should not return any value we don't return any value in our MediatR commands.
everything was normal since we decided to write some BDD tests.
In our BDD tests there is a simple scenario like this:
Scenario: [Try to add a new Question]
Given [I'm an authorized administrator]
When [I create a new Question with Title '<Title>' and '<Description>' and '<IsActive>' and '<IndicatorId'>]
Then [A question with Title '<Title>' and '<Description>' and '<IsActive>' and '<IndicatorId'> should be persisted in database]
Examples:
| Title | Description | IsActive | IndicatorId |
| This is a test title | this is a test description | true | 3cb23a10-107a-4834-8c1a-3fd893217861 |
We set Id property of Question object in its constructor. which means we don't know what Id we have set to this newly created Question and therefore we can't read it after adding it to database in a test environment.
my question was how to test it in BDD?
this is my test step implementation:
[When(#"\[I create a new Question with Title '([^']*)' and '([^']*)' and '([^']*)' and '([^']*)'>]")]
public async void WhenICreateANewQuestionWithTitleAndAndAnd(string title, string description, bool isActive, Guid indicatorId)
{
var command = new CreateQuestionCommand
{
Title = title,
Description = description,
IndicatorId = indicatorId
};
await questionController.CreateQuestion(command);
}
[Then(#"\[A question with Title '([^']*)' and '([^']*)' and '([^']*)' and '([^']*)'> should be persisted in database]")]
public void ThenAQuestionWithTitleAndAndAndShouldBePersistedInDatabase(string title, string description, string isActive, string indicatorId)
{
//? how to retrieve the created data. We don't have the Id here
}
How can I retrieve the added Question?
Should I change my command handlers to return my object after inserting it to database?
And if I do so, wouldn't it be a CQRS violation?
Thank you for your time.
There's a couple of ways to go about this, but it depends on how you actually expect the system to behave. If you're doing BDD you should focus on the kind of behaviour that would be observable by real users.
If you're implementing a system that allows users to create (and save, I infer) questions in a (Q&A?) database, then should they not have the ability to view or perhaps edit the question afterwards?
As the OP is currently phrased, the trouble writing the test seems to imply a lacking capability of the system in general. If the test can't identify the entity that was created, then how can a user?
The CreateQuestion method only returns Ok(), which means that there's no data in the response. Thus, as currently implemented, there's no way for the sender of the POST request to subsequently navigate to or retrieve the new resource.
How are users supposed to interact with the service? This should give you a hint about how to write the test.
A common REST pattern is to return the address of the new resource in the response's Location header. If the CreateQuestion method were to do that, the test would also be able to investigate that value and perhaps act on it.
Another option is to return the entity ID in the response body. Again, if that's the way things are supposed to work, the test could verify that.
Other systems are more asynchronous. Perhaps you only put the CreateQuestionCommand on a queue, to be handled later. In such a case I'd write a test that verifies that the command was added to the queue. You could also write a more long-running test that then waits for an asynchronous message handler to process the command, but if you want to test something like, you also need to deal with timeouts.
In a comment, you write:
I think he might add the question and then read all questions from database and see if his question is amongst them. He'd probably test without knowing Id.
Do you expect real users of the system to spelunk around in your database? Unless you do, the database is not part of a system's observable behaviour - it's an implementation detail. Thus, behaviour-driven design shouldn't concern itself with the contents of databases, but how a system behaves.
In short, find out how to observe the behaviour you want the system to have, and then test that.
With Doctrine and Symfony in my PHPUnit test method :
// Change username for user #1 (Sheriff Woody to Chuck Norris)
$form = $crawler->selectButton('Update')->form([
'user[username]' => 'Chuck Norris',
]);
$client->submit($form);
// Find user #1
$user = $em->getRepository(User::class)->find(1);
dump($user); // Username = "Sheriff Woody"
$user = $em->createQueryBuilder()
->from(User::class, 'user')
->andWhere('user.id = :userId')
->setParameter('userId', 1)
->select('
user
')
->getQuery()
->getOneOrNullResult()
;
dump($user); // Username = "Chuck Norris"
Why my two methods to fetch the user #1 return different results ?
diagnosis / explanation
I assume* you already created the User object you're editing via crawler before in that function and checked that it is there. This leads to it being a managed entity.
It is in the nature of data, to not sync itself magically with the database, but some automatism must be in place or some method executed to sync it.
The find() method will always try to use the cache (unless explicitly turned off, also see side note). The query builder won't, if you explicitly call getResult() (or one of its varieties), since you explicitly want a query to be executed. Executing a different query might lead to the cache not being hit, producing the current result. (it should update the first user object though ...) [updated, due to comment from Arno Hilke]
((( side note: Keeping objects in sync is hard. It's mainly about having consistency in the database, but all of ACID is wanted. Any process talking to the database should assume, that it only is working with the state at the moment of its first query, and is the only user of the database. Unless additional constraints must be met and inconsistent reads can occur, in which case isolation levels should be raised (See also: transactions or more precisely: isolation). So, automatically syncing is usually not wanted. Doctrine uses certain assumptions for performance gains (mainly: isolation / locking is optimistic). However, in your particular case, all of those things are of no actual concern... since you actually want a non-repeatable read. )))
(* otherwise, the behavior you're seeing would be really unexpected)
solution
One easy solution would be, to actively and explicitly sync the data from the database by either calling $em->refresh($user), or - before fetching the user again - to call $em->clear(), which will detach all entities (clearing the cache, which might have a noticable performance impact) and allowing you to call find again with the proper results being returned.
Please note, that detaching entities means, that any object previously returned from the entity manager should be discarded and fetched again (not via refresh).
alternate solution 1 - everything is requests
instead of checking the database, you could instead do a different request to a page that displays the user's name and checks that it has changed.
alternate solution 2 - using only one entity manager
using only one entity manager (that is: sharing the entity manager / database in the unit test with the server on the request) may be a reasonable solution, but it comes with its own set of problems. mainly omitted commits and flushes may avoid detection.
alternate solution 3 - using multiple entity managers
using one entity manager to set up the test, since the server is using a new entity manager to perform its work, you should theoretically - to do this actually properly - create yet another entity manager to check the server's behavior.
comment: the alternate solutions 1,2 and 3 would work with the highest isolation level, the initial solution probably wouldn't.
I'm trying to implement simple DDD/CQRS architecture without event-sourcing for now.
Currently I need to write some code for adding a notification to a document entity (document can have multiple notifications).
I've already created a command NotificationAddCommand, ICommandService and IRepository.
Before inserting new notification through IRepository I have to query current user_id from db using NotificationAddCommand.User_name property.
I'm not sure how to do it right, because I can
Use IQuery from read-flow.
Pass user_name to domain entity and resolve user_id in the repository.
Code:
public class DocumentsCommandService : ICommandService<NotificationAddCommand>
{
private readonly IRepository<Notification, long> _notificationsRepository;
public DocumentsCommandService(
IRepository<Notification, long> notifsRepo)
{
_notificationsRepository = notifsRepo;
}
public void Handle(NotificationAddCommand command)
{
// command.user_id = Resolve(command.user_name) ??
// command.source_secret_id = Resolve(command.source_id, command.source_type) ??
foreach (var receiverId in command.Receivers)
{
var notificationEntity = _notificationsRepository.Get(0);
notificationEntity.TargetId = receiverId;
notificationEntity.Body = command.Text;
_notificationsRepository.Add(notificationEntity);
}
}
}
What if I need more complex logic before inserting? Is it ok to use IQuery or should I create additional services?
The idea of reusing your IQuery somewhat defeats the purpose of CQRS in the sense that your read-side is supposed to be optimized for pulling data for display/query purposes - meaning that it can be denormalized, distributed etc. in any way you deem necessary without being restricted by - or having implications for - the command side (a key example being that it might not be immediately consistent, while your command side obviously needs to be for integrity/validity purposes).
With that in mind, you should look to implement a contract for your write side that will resolve the necessary information for you. Driving from the consumer, that might look like this:
public DocumentsCommandService(IRepository<Notification, long> notifsRepo,
IUserIdResolver userIdResolver)
public interface IUserIdResolver
{
string ByName(string username);
}
With IUserIdResolver implemented as appropriate.
Of course, if both this and the query-side use the same low-level data access implementation (e.g. an immediately-consistent repository) that's fine - what's important is that your architecture is such that if you need to swap out where your read side gets its data for the purposes of, e.g. facilitating a slow offline process, your read and write sides are sufficiently separated that you can swap out where you're reading from without having to untangle reads from the writes.
Ultimately the most important thing is to know why you are making the architectural decisions you're making in your scenario - then you will find it much easier to make these sorts of decisions one way or another.
In a project i'm working i have similar issues. I see 3 options to solve this problem
1) What i did do is make a UserCommandRepository that has a query option. Then you would inject that repository into your service.
Since the few queries i did need were so simplistic (just returning single values) it seemed like a fine tradeoff in my case.
2) Another way of handling it is by forcing the user to just raise a command with the user_id. Then you can let him do the querying.
3) A third option is ask yourself why you need a user_id. If it's to make some relations when querying the data you could also have this handles when querying the data (or when propagating your writeDB to your readDB)
Hi and thanks for your attention.
First some background on the question:
I have an Inline Table Function which is used ubiquitously throughout an ASP.net application I have developed, running on IIS7. It's the basis for 30 or more stored procedures, all of while will normally process in a maximum of 200ms (more than fast enough). The application always uses the same connection string for all database connections. The function itself runs in a maximum of ~10ms.
Occasionally a call to a page which calls a stored procedure in which the function is used will result in a timeout after 15 seconds. This timeout applies to all users of the page, but other pages which also have stored procedures which use this function still perform normally while this is occurring, suggesting that it is a specific stored procedure having problems. However this has occurred on multiple pages, suggesting it is either multiple stored procedures or the function itself.
Running any (or all) of the stored procedures on the page from a management studio session with different user credentials is <200ms, even when a timeout is occurring for the web application.
Running an sp_recompile on the function will always "clear" the timeout, from any login credentials.
Because this is a critical portion of the application, sp_recompile is run as soon as possible, and little time is available for debugging. Also, I have never been able to recreate the timeout at will.
I've tried to do a lot of research on Inline Table Functions and haven't come across anything which suggests that this is a common problem with them and they should therefore be avoided.
The Question:
Is it possible that these timeouts are being caused by using the function, or are they guaranteed to be an issue with the stored procedures which rely upon it? To put it another way, is there likely to be any benefit in refactoring the stored procedures to use either a view or by encapsulating the appropriate logic inline?
My guess is that it is the stored procedures only and I'll likely solve this by adding optimize for unknown, option recompile where appropriate, or localising the parameters, but in all honesty I'd prefer to find a solution which applies to the underlying function such that I can apply a fix in a single location.
The function:
CREATE FUNCTION [dbo].[fn_ObjectIDs] (
#DateFrom AS DATETIME = NULL
,#DateTo AS DATETIME = NULL
,#Region AS INT = NULL
,#FamilyID AS INT = NULL
,#ParentID AS INT = NULL
,#ChildID AS INT = NULL
) RETURNS TABLE AS
RETURN
SELECT DISTINCT
obj.ID AS IDs
FROM tblObjects obj WITH (NOLOCK)
INNER JOIN tblFamily fam WITH (NOLOCK)
ON obj.famID = fam.famID
LEFT JOIN tblCountry cntry WITH (NOLOCK)
ON (#Region IS NOT NULL) AND (fam.countryId = cntry.countryId)
LEFT JOIN tblParent parent WITH (NOLOCK)
ON (#ParentID IS NOT NULL) AND (obj.ID = parent.objectID)
LEFT JOIN tblChild child WITH (NOLOCK)
ON (#ChildID IS NOT NULL) AND (obj.ID = child.objectID)
WHERE
obj.Visible = 1
AND obj.statusID IN (3,4,6,8)
AND ((#DateFrom IS NULL) OR (obj.CreatedDate >= #DateFrom))
AND ((#DateTo IS NULL) OR (obj.CreatedDate <= #DateTo))
AND ((#Region IS NULL) OR (cntry.regionID = #Region))
AND ((#FamilyID IS NULL) OR (obj.famID = #FamilyID))
AND ((#ParentID IS NULL) OR (parent.parentID = #ParentID))
AND ((#ChildID IS NULL) OR (child.childID = #ChildID))
Temporarily add some logging to verify what is actually causing the timeout problem. You can do this in your application and it the stored procedure itself. Have the procedure write the current timestamp and the parameters used to invoke the procedure to a logging tables when it's executed. Also add logging to your application. Then you'll be able to identify when specific timeouts occur if there are certain parameters causing the problem or if the problem is in the procedure at all.
In general user defined functions aren't a good idea, though my understanding is that a table inline function is better than some of the others. It adds a lot of overhead and the optimizer can't work properly with UDFs.
I am using an sqlite database for a producer-consumer queue.
One or more producers INSERT one row at a time with a new autoincremented primary key.
There is one consumer (implemented in java, uses the sqlite-jdbc library) and I want it to read a batch of rows and delete them. It seems like I need transactions to do this but trying to use SQLite with transactions seems to not work right. Am I overthinking this?
If I do end up needing transactions, what's the right way to do this in Java?
Connection conn;
// assign here
boolean success = false;
try {
// do stuff
success = true;
}
finally
{
if (success)
conn.commit();
else
conn.rollback();
}
See this trail for an introduction on transaction handling with Java JDBC.
As for your use case, I think you should use transactions, especially if the consumer is complex. The tricky part is always to decide when a row has been consumed and when it should be considered again. For example, if you have an error before the consumer can actually do its job, you'll want a rollback. But if the row contains illegal data (like a text in a number field), then the rollback will turn into an infinite loop.
Normally, with SQLite there are explicit (not implicit!) transactions. So you need something like "START TRANSACTION" of course, it could be that your Java binding has this incorporated -- but good bindings don't.
So you might want to add the necessary transaction start (there might be a specialiced method in your binding).