I am creating a server for an app with a Notifications table which will have A LOT of records (about 100 million records) but only two small values in each record. From what I understand, the name of the Datastore Entity is part of the Entity key and it looks as if the key gets longer with longer Entity names, so I am guessing it is not hashed to the same length for all names? Is this true? Will storage increase with long names?
If so, can I tell Objectify (3.x) what the Entity name in Datastore will be or do I need to rename my classes?
Initially I wrote that space used by keys is not being counted toward your quota because I heard something like that in one of the Google I/O videos. After extensive googling I can't find any info that would confirm this. So I think we should assume the longer the names of your entities the more you'll pay.
You can define name of the entity in annotation's attribute
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.TYPE})
public #interface Entity
{
/**
* Controls the actual kind name used in the datastore.
*/
String name() default "";
}
Related
Realm:
We have the following scenario: There are several stores with employees and customers, several employees that could work at more than one store, and several customers that may shop at several stores. This could be represented with these classes
class Store {
dynamic var id = ""
dynamic var address = ""
let workers = List<Employee>()
let customers = List<Customer>()
}
class Customer {
dynamic var id = ""
dynamic var name = ""
let stores = LinkingObjects(fromType: Store.self, property: "customers")
// ... many more fields about this customer
}
class Employee {
var id
var name
let work = LinkingObjects(fromType: Store.self, property: "workers")
}
The catch here is that we must protect customer information, so none of the customer info can be present in a shared realm and needs to be secure. Neither the store nor Employee data is a security matter. Our current approach is to give each customer their own realm, however, the major drawbacks to this is that requires massive duplication since each customer realm must copy the data of the store. The other drawback is that we would be copying customer data into a shared realm which is a security risk. What would be the best way to architect this scenario that allows for relationships across different Realms?
Realm doesn't currently support "direct" object links across Realms analogous to object properties within the same Realm.
Instead, what I suggest you do is to give your objects primary keys (you can probably just declare your existing id fields as such, or create a new internalId field if your existing id field can't be used for this purpose).
Primary keys are mandatory, must be unique, and can't be changed after they are set, which makes them great for uniquely identifying objects. Our documentation discusses them in greater detail.
Then, instead of directly storing customer info/a customer object in a shared Realm, you can just store the primary keys for the relevant customers, for example in a list. (Right now you'll have to make a wrapper CustomerKey object for example to store the customer's primary key, but we plan to support collections directly containing strings or other primitive types very soon.)
You can enhance this further by adding helper methods on your objects that can be passed in a customer Realm and return the user object (or whatever object's primary key is being stored), looking it up in the Realm automatically. You can use Realm's object(ofType:forPrimaryKey) method to look up an object based on its primary key.
The main limitation is that you won't get the automatic updating of links you would get with object, list, and LinkingObjects properties. You'll have to manually perform the bookkeeping yourself.
If you have ideas for functionality you want to see in Realm that would go beyond what I've posted here, feel free to share your thoughts at our GitHub issue tracker. We welcome feature requests.
I have the following:-
Visual Studio 2013.
i created a new asp.net MVC-5 web project.
the project is using asp.net identity 2.2.
for the authentication method i chose "Individual user accounts"
this process created a new database named aspnet-OurProjectNanme-number
inside the automatically generated database, i have a table named AspNetUSers which store the user info.
now i am working on building an ERP system. and inside the ERP system i want to add the following:-
a table named "Asset" to store the asset info.
the "Asset" table will have 2 columns named "CreatedBy" + "ModifiedBy" which should store the userId who created and modified the asset item.
now i am not sure how i need to achieve this? as i need to add a foreign key between my custom table "Asset" and the "AspNetUsers" table which have been created automatically.. so can i add my custom table "Asset" inside the automatically generated database, and build the foreign key between the Asset.CreatedBy and AspNetUsers.Id ??
if the answer is Yes then can this relation break in the future if we want to upgrade our aspnet identity version ? as upgrading the identity might result in creating new tables or renaming existing ones etc.. which might break the relation between the Asset table and the AspNetUsers table?
If the answer is No (i should not add custom tables inside the automatically generated database ) then how i can build the foreign key ?? and where i need to add the Asset table in this case??
The most common approach to what you want to do is simply to add your additional model as a DbSet in your ApplicationDbContext.
public class Asset
{
public string CreatedBy { get; set; }
public string UserId { get; set; }
public ApplicationUser User { get; set; }
}
public class ApplicationUser : IdentityUser
{
public IList<Asset> Assets { get; set; }
}
public class ApplicationDbCotext : IdentityDbContext<ApplicationUser>
{
public DbSet<Asset> Assets { get; set; }
}
As I mentioned this is the most common approach as updating the Identity packages should have no impacting affects on your schema. That said you should always test updates before pushing to production.
UPDATE:
Note that when you're working with One to Many relationship's you will see in our Asset model a property for the User Id foreign key as well as the User object. Because of the relationship we are then able to create a List<Asset> in our User to complete the One to Many relationship. This will then allow us to directly query Assets belonging to a User.
As for Code First vs Database First the difference really comes down to how you define the mapping between Entity Framework and the Database.
As I mentioned below there is no one size fits all answer to should you separate the Identity context from your business context, or should you separate them into separate databases. The reality is that only you can answer that question for your needs. It is far more common to have all of the data in a single database. That said, there is something to be said for the security of having identifying information about a user such as their name, email and password hash separated from information like their address or payment information. The trade off is that you can find yourself trying to maintain objects that are supposed to be tied together but are only loosely related because they reside in different databases. Also you would then need to make sure you're using different users/passwords to connect to the different databases, and it's better to have the databases on different servers because if the server gets compromised you went through the entire exercise for nothing. The trade off to get the theoretical security ends up being so impractical with there consistently being another thing you have to do, that you end up seeing everything in one database where you can focus all of your hardening efforts.
Both the ApplicationDbContext and ApplicationUser objects should typically be created for you when you File -> New a project with Individual Authentication. You can add as many properties and relationships to your User as you require.
-- Update --
The answer is growing and growing and so is the discussion. I think I've shown all kinds of variations, which may not have helped to make it understandable. So here is a summary. For explanation, read the full answer and discussion.
Out of the box you have two contexts, identity and business. These are decoupled, so you can change your security without interfering with your business. This way, upgrading security won't break your application or other models. Since the contexts are seperate, changes to either one of them won't affect the other.
As a sidenote: you are not intended to directly access the AspNet identity tables. Implement the UserManager and use the avaiable methods of the manager to perform actions.
Now it comes to logic, where should information be stored? As a simple rule just ask yourself the question: is it part of security or business?
In both contexts you have users. For your requirement this is a logical 1:1 relation. But they are actually seperate. You can create people without supplying a login or delete a login, without deleting a user (people), e.g. for historical reasons.
All you want is to find all information for the current user. So all you need is the People.Id.
Without having to change the IdentityUser you can create the 1:1 relation by just overriding the AspNetUser.Id.
var appUser = new IdentityUser
{
UserName = model.Email,
Email = model.Email,
Id = Convert.ToString(People.Id)
};
var identityResult = await userManager.CreateAsync(appUser, model.Password);
You do not need the identity context for your business. All you need is People.Id. The identity context is only used when tokens are issued and users are created / modified.
To obtain the id use something like this:
var peopleId = int.Parse(Request.User.Identity.GetUserId());
Now you can query your business model using the Id.
When registering, extend the View and ViewModel with the People information you want to store. This will allow you to add both People and AspNetUser at the same time. Though this is not one transaction. But I think it is highly unlikely that creating either one would fail if you perform checks first.
You can validate the username and password (use the methods in the UserManager) and check the ModelState of the viewmodel before creating the user. Use attributes to force Required fields to be filled.
-- Original answer --
In order not to repeat myself, read my answer here.
In short, keep identity and business seperated.
Just in case the identity logic is removed from the same database, like when implementing IdentityServer.
It seems you have business information in AspNetUser. If so, create a Person table and move the information to that table. Relate to that table in your model. In table Person you can add a reference to AspNetUser.
-- update --
I think you understand correctly, but I will just add the details to this answer.
In most cases all tables are defined in one database. But that doesn't mean they are all part of the same model. There can be multiple contexts. In this case one for Identity and one (or more) for Business.
Now why seperate those two? The most important difference between the Business model and Identity model is that Identity tables are not to be called directly. We use the Owin context to call the UserManager / RoleManager.
That is why we cannot add these tables to the business model. Things can be altered in a way that is not secure. Also we do not want the business to have any knowledge about authorization. It shouldn't matter how this is done, as long as a user is identified and authorized.
Also you may want to implement OpenId and claim based authorization. In that case information doesn't have to be available in the database.
The idea is to create a 1:1 relation of the identity table AspNetUsers and business table People. There can be some redundancy, like email or (user)name. But that isn't a problem. The People table should contain all information you want to use in your business model. And the business tables should only relate to People, not AspNetUsers.
Now about the link between AspNetUsers and People. There are four options:
Set People.Id = AspNetUser.Id. Please note that AspNetUser.Id doesn't have to be a GUID. You can add your own value as key.
Set AspNetUser.Id = People.Id.
Add column AspNetUserId to People. No modifications to Identity are needed. You can add People to the Identity Model as well, but I don't think you can create both records in one transaction. You can use User.Identity.GetId() to get AspNetUser.Id. You may however ask yourself if the business should have knowledge about this information.
Add column PeopleId to AspNetUsers. You'll need to extend the IdentityUser to add PeopleId. An advantage is that you don't need the AspNetUser Id, but you can use the actual Id of People. When using OpenId or claims you can get People.Id from claims and you won't have to add AspNetUser.Id to the business. Optionally you can add People to the Model and as navigation property of the extended IdentityUser. When creating the user, you can do this in one transaction.
In case you are creating the user in seperate contexts, you'll need to handle the rollback yourself. But before adding a record to People, you can already test if an AspNetUser can be added: has a valid name/email and password.
Since your business model relates to the People table, you can query all assets and join with the People table for additional information. Or you can get all assets for the current user.
o yes, there are two contexts. The identity model, which contains the AspNet... tables + optionally People. And the business model, which contains all ERP tables + Asset + People.
You may consider to use code first for identity framework 2 and database first for the business model.
I hope that this helps. If not, let's continue in chat.
-- update --
The answer focused on seperation of domains: identity and business. That is why I didn't discuss one possible alternative concerning the AspNetUsers table.
The two models are representations of the database, which means that the database doesn't have to be an exact match. You are free to map tables and fields as you like, as long as they don't break database logic.
Since AspNetusers and People has a 1:1 relation and when both tables are present in the same database, you may as well merge the two into the AspNetUsers table. You can also add relations to the AspNetUsers table, though you may want to add an extra Id (int) column instead of using the current Id (string).
This does not mean the People class can be discarded, except that we have to change the table mapping: AspNetUsers.
Example:
[Table("AspNetUsers")]
public class People
{
[Required]
[StringLength(128)]
public string Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
As you can see, the sensitive fields are not mapped. We need however the Id field. You can now read and update the mapped fields.
You don't have to extend IdentityUser. You can add an AspNetUser and then update the fields using People in the other context. But if you want to add a user in one single transaction it may be easier to extend the IdentityUser (make sure you'll define the new fields in both People and ApplicationUser):
public class ApplicationUser : IdentityUser
{
public string FirstName { get; set; }
public string LastName { get; set; }
}
There are multiple advantages:
There is only one transaction to add the user.
You cannot expose the sensitive fields since they are not mapped in People.
You cannot add People to the database, since some required fields are not mapped in People.
Please note that this may not work for all types of models (code first/database first + migrations).
Background: I have a fairly big table (2000 rows, 10 columns) full of small int (0,1,2 only).
Given an entity and a field of that entity, depending on the values of the 10 columns I get the information if I want to display and or make required that field of that entity when I render the form.
Instead of querying the DB for each field, every time I have my ->add in the form builder, I'd like to cache the whole table "somewhere" in server's memory.
How can I do that? Is it necessary that each user "loads" the table, or can it be done once for all of the users and each user simply reads server's RAM?
That table almost never changes. Still I need to be able to upload an updated version of it every once in a while (veeery seldom).
Thank you!
SN
All you need to do is use useResultCache method.
Doctrine uses cache driver set in Symfony configuration, so when you write queryBuilder which is fetching data from your table just tell Doctrine you want result to be cached (for how long and under what key is optional):
class YourEntityRepository extends EntityRepository
{
public function getAllRowsCached()
{
$cachedResult = $this->createQueryBuilder('e')
->getQuery()
->useResultCache(true, 3600, 'cache_key')
->getArrayResult();
return $cachedResult;
}
}
Cache is common for all users so first user will query DB, second will use data from cache. Also note, that by default in dev environment cache is disabled.
Check this blog entry for more detailed info
I have a problem concerning the usage of a DataTransformer.
Basically, I am developing a translation tool for my application whose goal is to be as generic as possible.
For that, I chose to follow that model : Database modeling for international and multilingual purposes
So, in different entities in my application, I have translatable attributes that simply are references to i18n elements. Then, this i18n ID is referenced in Translation table entries, that handle translation strings.
I succeed handling my translation interface, but I now have a problem with my forms : Indeed, I want some of my entities to be created/updated via forms. The problem is that I don't want the user to set a i18n ID for the translatable fields, of course, but a text, so that it can be handled by my application to either update or create the related translation in database.
I thought then that creating a DataTransformer could be a good idea, so that I can get the related translation string from the i18nID that is in my Entity entry (for that way, no problem). But my problem here is for the opposite way :
How can I deal with creating/updating i18n entries in my reverseTransform() method without knowing the entity values context?
Is there any way to get the previous entity values so that I could get the i18 ID that is stored originally in my entity? I understand that a Data Transformer is theorically totally independent from my forms and my entities, but I'm totally blocked about how to handle this case.
Indeed, when I save my entity with my translated string, I have no way to know the entity context in my reverseTransform() method, that would have permitted me to get the i18nID of the entity and to update it.
I just have the string that typed the user, but I can't do anything with that, because I can't know if it is an update or not since I don't have access to my entities.
Do you have any clue to do that? Is trying to use a DataTransformer to perform this a bad idea?
Thank you !
Using the Dynamics CRM I'm trying to create an instance of an entity. I would like to manually set the GUID, but if I had the attribute that is the primary key to the DynamicEntity, I get following error.
Service could not process request
I am building a DynamicEntity, and setting the [entityname]id attribute causes the request to fail. It's moving data between two CRM instances, so if anyone knows of a better way to copy records between CRMs, that'd work too. Otherwise, I'd like the GUID to match across instances... as that's the point of a GUID.
Happily, it IS possible to do this across two CRM instances! A co-worker knew the solution, so credit really belongs to him.
My mistake was creating a Property with type UniqueIdentifierProperty. The primary key attribute on an entity needs to be filled in with a KeyProperty. These two properties are nearly identical -- the Property types are, except that one holds a Key, the other a UniqueIdentifier. The Key/UniqueIdentifier both hold GUIDs. (Another day in the mind of Microsoft!)
Precisely, what I'm doing is creating a DynamicEntity, filling in the entity name, and filling in the majority of the attributes. The PK attribute (which you can determine from the metadata) can be filled in with a KeyProperty. I was filling it in with a UniqueIdentifierProperty, which CRM rejects and responds with a nondescript and unhelpful error message.
I apologize if I am over-simplifying the solution, but why not add a custom field in both instances that would be a mirror of the other instances guid?