I have WCF Data service on top of EF (old-fashioned edmx; not code-first) that has millions of rows of data. I am restricting result set to 500 rows. So far, so good!
I want to give the consumer the ability to "drill" into the data. Ideally, consumer code would run something like select distinct Province from my service, user would choose the state, and then code would run select distinct Zip where Province == p (my business domain is very different, but hopefully you get the picture!).
Obviously, I can't get everything and run distinct on the client side. How do I provide this ability in the service? I don't have EntitySet "Provinces" or "Zips" exposed in the service. Should I extend MyEntities class, and try to simulate those sets there? Or is there a simpler way to expose other collections, besides the ones that are automatically exposed from Entity Framework?
I hope the question makes sense...
UPDATE: I think I caused the confusion when I mentioned that I was using WCF service. I actually am using WCF Data Service trying to provide OData wrapper over Entity Framework 4.1. Hopefully, this clarifies the question...
Have you considered dropping your supposedly propietary interafce and going standards?
It is quit easy to expose ODATA from .NET and that allows the user to drill down. THen all you need are some transformations to expose elements like Province or Zip as in a sane data model and there you go ;)
The good thing about OData is that it allows you to easily pass LINQ to the backend and do projections (get me all proovinces from customers where customer name starts with "a") and doesn ot necessarily deal with compelte entities.
Related
I am trying to use Microsoft's Face API, for facial recognition for my company employees. I see that you need to create a database in Microsoft's serverS.
Is there a way to use their API's on our company database (without creating another DB on their server? Also any changes you make to this DB will be taken care of.
If no, then how will you take care of the changes you want (I know that there are delete API calls as well, but will not it be cumbersome?)
I think you may have misunderstood how this API works. Whenever there is a change to your list of employees, the PersonGroup (a friendly name for the image classifier model), must be retrained in order for it to start recognizing the added faces and stop recognizing removed ones. So even if there was a way to store the model locally (which there isn't), you will still need to track add/removes and take the additional step of training.
I'm starting a new project and am interested in architecting it as microservices. I'm trying to wrap my head around it:
Say that I have an order service and a product service. Now I want to make a report service that gives me all orders that contain a product from a certain product category.
Since order's dont know about products that means that I would need to fetch all orders, loop them and fetch products for each order and then return those how match.
Is this assumption correct or is there any more efficient way of doing this with microservices?
In a microservices architecture, the procedure is to distill the use cases and the service boundaries of the application. In the question above, there are at least two service boundaries, namely one for transactions and another for reporting.
When you have two different service boundaries, the typical approach is to duplicate some data elements between them eg. whenever you make a sale, the data, should be sent to both the reporting and transactional services. One possible approach of broadcasting the data to the different boundaries is to use a message queue. Duplicating the data allows them to be evolve and operate independently and become self sufficient which is one of the goals of microservices.
A personal word of advice though, you might want to start with a monolith before going the microservices route. Microservices are generally more operationally heavy; it will be difficult to reason about its advantages during the initial application stages. It tends to work better after having developed the monolithic application since it would be easier to see what didn't work and what could be improved by a microservices-like system.
I have a question abut what is the best practice/pattern to ensure proper authorization in my SOA application. I have a bunch of services that allow users access certain data (stored in DB). An example of a typical scenario is...
We have a EMPLOYEE table that has say a one to many relationship with PROJECT. So AN EMPLOYEE can have many PROJECTS. Also each employee belongs to a region. The users of the system are allowed to edit information on employee and projects. However Each user only manages data for a few regions and therefore can only modify employees and their projects that belong to a region that the user manages. So user may have access to regions A, B, C and can edit employee/project data for Region A employee but not region Z employee.
I have one service that lets you edit an employee and another to edit an project. Similarly a project has relationships to other entities, for e.g. SCHEDULE, and I have services to edit those as well.
However my problem is this -- whether an user can edit or not a project or schedule, by calling the corresponding service, is determined by which region the employee (to which these projects and schedules are related) belongs to. So for every service to modify a project or schedule or any entity in that hierarchy of data starting at employee, I am having to query the corresponding employee and enforce the region constraints. Which can become very expensive operation considering the database calls and number of joins (my real example has lot of such entities and corresponding services) I have to make on every service call. Is there a more elegant light weight solution for my scenario?
First of all, I'd call these requirements regular business rules rather than authorization. Authorization is usually much more generic in nature, meaning whether a user is allowed access to a specific system or is allowed to invoke a certain function (regardless of parameters).
Second, and based on a business rules view, you should take into account the consistency needs around these business rules before dividing things up into services.
Third, relating to your statement "my real example has lot of such entities and corresponding services", it usually not a good idea to have services that correspond to entities.
So, in summary, you need to redesign your service boundaries such that the rules that need to be enforced (in a highly consistent manner) are contained within a single service. One way to do that is to take different parts of a given entity and put them in different services - provided that there aren't any business rules that demand consistency across those diffe
I am working on an Adobe Flex front-end with a Java back-end using JPA for persistence. The protocol I am using is remote objects (AMF) implemented with BlazeDS.
I started out with a service-facade and entity DAOs, but without any specific DTOs. The same POJOs, the domain objects, were passed in the service-facade as those used as DTOs passed to the Hibernate DAOs.
However, the latest few days I have been thinking whether this is a good approach or not. The latest article on the subject I read was this one:
Interesting JPA Pattern blog
The situation:
Say I have POJO Book with a unidirection ManyToOne relation with the POJO Category (i.e. each book may only be associated with one category, but the same category may be associated with many books). I see some alternatives:
Alternative 1:
I expose a method/operation addUpdateBook(Book book). In the implementation of this operation I add/update both the book and the referenced category. I mean, if the client submits a book having a category that doesn't exist from before, this would mean that the client implicitly may edit categories using the addUpdateBook operation.
the client is working directly with the domain model!
the entire category information will be sent when a new book is added
even though a reference to the category would be sufficient
Alternative 2:
I expose a method/operation addUpdateBook(Book book,Long categoryId). In the implementation I retrieve the category for the given categoryId and replace the category given in the book POJO and then I persist the book. In other words, I ignore any category in the book object, I just look at the categoryId. This means that the client would need to use another operation in order to modify the category.
pro: the client can still work more or less on the domain model, but ...
con: ... it is confusing for the client that the category of the book
object will be ignored
con: the entire category information of the book will be sent, even
if the server never will read it
pro: it may be more clear when a separate operation should be used
for category modifications
con: I need to retrieve the category before persisting the book. I
guess this means some overhead.
Alternative 3:
I expose a method/operation addUpdateBook(BookDTO bookDto). The POJO BookDTO looks as the POJO Book, but instead of a field Category category it has a field Long categoryId. In the implementation I retrieve the Category for the given categoryId before I persist the Book.
pro: not confusing for the client
con(?): what should the method getBook(Long bookId) return? should it
return only the BookDTO? Then it would be required to invoke also the
operation getCategory(Long categoryId) in order to have "the entire
book information". Then the client would need to set together the
different parts to a local domain representation of the book.
Compared to alternative 1 this would be more complex on the client
side?
con: I need to retrieve the category before persisting the book. I
guess this means some overhead.
con: being forced to use the DTOs in the client makes it deal with physical details thereby and makes it somewhat distant from the actual domain model. It seems like I am missing the point with having an domain model and using JPA in the business layer.
I guess (!) alternative 3 is the way you would design the operations in a SOA context. However, for me, it is not that important to be loosely-coupled between the client and server. My focus is not to provide multiple client-platform support.
Which alternative would you propose? Are there other better alternatives? Do you know any nice resources, such as code examples, which could help me?
I'm using something related to "Alternative 3". In the beginning I've started to use domain objects (probably also because of my experience with dataservices), after a while I found too many problems and I've switched to DTO's. All the publing services are exposing only DTO's (both for input/output parameters).
Some of the problems that I've met during working directly with the domain objects and BlazeDS:
a)you need to break the domain objects encapsulation (like exposing properties, or exposing private constructors) in order to use them for data transfer. Otherwise you will have to write your own serialization/deserialization.
b)you need to use tricks in order to allow data conversion between client/server. For example, using strings instead of dates in order to prevent timezone difference. Or using strings instead of int/double. You can solve some of these issues by writing custom proxies, but I still think that it's easier to use strings instead of other data types.
c)most of the time you don't need all the data from the domain objects, and in order to deal with that you need to use various frameworks with support for data pagination/lazy instantiation on the client. This frameworks are introducing complexity, and I try to stay away from that.
The main disadvantage of using DTO is the amount of boiler code in order to do the conversion between the domain objects-DTOs...but I still prefer using them.
I have worked on a timesheet application application in MVC 2 for internal use in our company. Now other small companies have showed interest in the application. I hadn't considered this use of the application, but it got me interested in what it might imply.
I believe I could make it work for several clients by modifying the database (Sql Server accessed by Entity Framework model). But I have read some people advocating multiple databases (one for each client).
Intuitively, this feels like a good idea, since I wouldn't risk having the data of various clients mixed up in the same database (which shouldn't happen of course, but what if it did...). But how would a multiple database solution be implemented specifically?
I.e. with a single database I could just have a client register and all the data needed would be added by the application the same way it is now when there's just one client (my own company).
But with a multiple database solution, how would I create a new database programmatically when a user registers? Please note that I have done all database stuff using Linq to Sql, and I am not very familiar with regular SQL programming...
I would really appreciate a clear detailed explanation of how this could be done (as well as input on whether it is a good idea or if a single database would be better for some reason).
EDIT:
I have also seen discussions about the single database alternative, suggesting that you would then add ClientId to each table... But wouldn't that be hard to maintain in the code? I would have to add "where" conditions to a lot of linq queries I assume... And I assume having a ClientId on each table would mean that each table would have need to have a many to one relationship to the Client table? Wouldn't that be a very complex database structure?
As it is right now (without the Client table) I have the following tables (1 -> * designates one to many relationship):
Customer 1 -> * Project 1 -> * Task 1 -> * TimeSegment 1 -> * Employee
Also, Customer has a one to many relationship directly with TimeSegment, for convenience to simplify some queries.
This has worked very well so far. Wouldn't it be possible to simply have a Client table (or UserCompany or whatever one might call it) with a one to many relationship with Customer table? Wouldn't the data integrity be sufficient for the other tables since the rest is handled by the relationships?
as far as whether or not to use a single database or multiple databases, it really all depends on the use cases. more databases means more management needs, potentially more diskspace needs, etc. there are alot more things to consider here than just how to create the database, such as how will you automate the backup process creation, etc. i personally would use one database with a good authentication system that would filter the data to the appropriate client.
as to creating a database, check out this blog post. it describes how to use SMO (sql management objects) in c#.net to create a database. they are a really neat tool, and you'll definitely want to familiarize yourself with them.
to deal with the follow up question, yes, a single, top level relationship between clients and customers should be enough to limit the new customers to their appropriate data.
without any real knowledge about your application i can't say how complex adding that table will be, but assuming your data layer is up to snuff, i would assume you'd really only need to limit the customers class by the current client, and then get all the rest of your data based on the customers that are available.
did that make any sense?
See my answer here, it applies to your case as well: c# database architecture