We have a requirement to allow customising our core product and adding additional fields on a per client basis e.g. People entity some client wants to record their favourite colour etc. As far as I know we can't add properties to EF at runtime as it needs classes defined at startup. Each customer has their own database but we are deploying the same solution to all customers with all additional code. We are then detecting which customer they are and running customer specific services etc.
Now the last thing I want is to be forking my project or alternatively adding all fields for all clients. This would seem likely to become a nightmare. Also more often than not the extra fields would only be required in a very limited amount of place. Maybe some reports, couple of screens etc.
I found this article from Jermey Miller http://codebetter.com/jeremymiller/2010/02/16/our-extension-properties-story/ describing how they are adding extension properties and having them go from domain to the web front end.
Has anyone else implemented anything similar using EF? How did it work out? Are there any blogs/samples that anyone has seen? I am not sure if I am searching for the right thing even if someone could tell me the generic name for what we want to do that would help. I'm guessing it is a problem that comes up for other people.
Linked question still requires some forking or implementing all possible extensions in single solution because you are still creating strongly typed extensions upfront (= you know upfront what extensions customer wants). It is not generally extensible solution. If you want generic extensible solution you must leave strongly typed world and describe extensions as data.
You will need to use some metamodel. Your entity classes will contain only properties used by all customers and navigation property to special extension entity (additional table per every extensible entity) where you will be able to put additional properties as name / value pair (you can add other columns like type, validation, etc. if needed).
This will in general moves part of your model from hardcoded scenario to configuration based scenario and your customers will even be allowed to define extensions at runtime (if you implement such feature).
Related
Just started using Drupal and tried to understand the core concepts. I have a developer background but I would like to use Drupal as a site builder and not digging into the code.
I'm trying to build a website which lists various vendors. One could be a Restaurant, another can be Photographer and other possible services (I have like 15 different ones).
They all have some things in common like Title, a Location (used Taxonomy/Vocabulary for that), description, image gallery, address, website, office hours and so on.
But they also have some custom fields. Restaurants can have fields like Facility options:Parking, Smoking area, etc or Capacity; Photographers can have others.
So there are lots of fields which are common for each vendor and some are are unique per each vendor.
What's the best way to implement this kind of structure as a Site builder?
I tried using Entities via ECK (Entity Construction Kit) and defining Entity types (as Vendor) and Bundles (as specific Vendors) but then I'm really limited in defining the common fields on Entity type level since Properties does not seem to be flexible in this regard, meaning that I cannot define them as normal fields and can't associate to them various widgets but only as a text input. Not sure if this a limitation of ECK or of Drupal 7 itself?
On the other hand I see the option of creating normal Content types for each kind of vendor which seems like alot of repetitive work, not sure if this is the right way (that's my only option at the moment)?
Maybe I should start learning more of Drupal and do some coding to create specific entity types? - but this means being more than a site builder. Since it will be a big project will this save me of some trouble later on or you see that I can accomplish the task easily without this extra effort?
Also by coding I'm not sure if there are easy ways of defining fields/widgets for Entity type Properties.
I would later on want to use faceting as well for filtering which will be based both on the fields which are common and unique for each vendor type, not sure if this is an important factor when creating the structure.
Any feedback is appreciated!
So, I need some input refactoring an asp.net (c#) application that is basically a framework for creating dynamic forms (any forms). From a high level point of view, there is a table that has the forms, and then there is a table that has all the form fields, where it is one to many between the two. There is a validation table, where each field can have multiple types of validation, and it is a one to many from the form fields table to the validation table.
So the issue is that this application has been sold as the be-all-end-all customizable solution to all the clients. So, the idea is whatever form they want, we can build it jsut using DB configurations. The thing is, that is not always possible, because there is complex relationship between the fields, and complex relationship between the forms themselves. Also, there is only once codebase, and this is for multiple clients - all of whom host it on their own. There is very specific logic for each of the clients, and they are ALL in the same codebase, with no real separation. Sometimes it was too difficult to make it generic, so there are instances where it has hard coded logic (as in if formID = XXX then do _). You can also have nested forms, as in, one set of fields on its own within each form.
So usually, when one client requests a change, we make the change and deploy it to that client - but then another client requests a different change, and we make the change and deploy it for THAT client, but the change from the earlier client breaks it, and its a headache trying to debug, because EVERYTHING is dynamic. There is no way we can rollback the earlier change, because then the other client would be screwed.
Its not done in a real 3-tier architecture - its a web site with references to a DB class, and a class library. There is business logic in the web site itself, in the class library, and the database stored procs (Validation is done in the stored procs).
I've been put in charge of re-organizing the whole thing, and these are my thoughts/questions:
I think this is a bad model in general, because one of the things I heard one of the developers say is that anytime any client makes a change, we should deploy to everybody - but that is not realistic, if we have say 20 clients - there will need to be regression testing on EVERYTHING, since we don't know the impact...
There are about 100 forms in total, and their is some similarity in them (not much). But I think the idea that a dynamic engine can solve ALL form requests was not realistic as well. Clients come up with the most weird requests. For example, they have this engine doing a regular data entry form AND a search form.
There is a lot of preserving state between pages, and it is all done using session variables, which is ok, except that it is not really tracked, and so sessions from the same user keep getting overwritten, and I think sessions should be got rid of.
Should I really just rewrite the whole thing? This app is about 3 years old, and there has been lots of testing and things done, and serious business logic implemented, so I hate to get rid of all that (joel's advice). But its really a mess of a sphagetti code, and everything takes forever to do, and things break all the time because of minor changes.
I've been reading Martin Fowlers "Refactoring" and Michael Feathers "working effectively with legacy code" - and they are good, but I feel they were written for an application that was 'slightly' better architected, where it is still a 3-tiered architecture, and there is 'some' resemblance of logic..
Thoughts/input anyone?
Oh, and "Help!"
My current project sounds like almost exactly the same product you're describing. Fortunately, I learned most of my hardest lessons on a former product, and so I was able to start my current project with a clean slate. You should probably read through my answer to this question, which describes my experiences, and the lessons I learned.
The main thing to focus on is the idea that you are building a product. If you can't find a way to implement a particular feature using your current product feature set, you need to spend some additional time thinking about how you could turn this custom one-off feature into a configurable feature that can benefit all (or at least many) of your clients.
So:
If you're referring to the model of being able to create a fully customizable form that makes client-specific code almost unnecessary, that model is perfectly valid and I have a maintainable working product with real, paying clients that can prove it. Regression testing is performed on specific features and configuration combinations, rather than a specific client implementation. The key pieces that make this possible are:
An administrative interface that is effective at disallowing problematic combinations of configuration options.
A rules engine that allows certain actions in the system to invoke customizable triggers and cause other actions to happen.
An Integration framework that allows data to be pulled from a variety of sources and pushed to a variety of sources in a configurable manner.
The option to inject custom code as a plugin when absolutely necessary.
Yes, clients come up with weird requests. It's usually worthwhile to suggest alternative solutions that will still solve the client's problem while still allowing your product to be robust and configurable for other clients. Sometimes you just have to push back. Other times you'll have to do what they say, but use wise architectural practices to minimize the impact this could have on other client code.
Minimize use of the session to track state. Each page should have enough information on it to track the current page's state. Information that needs to persist even if the user clicks "Back" and starts doing something else should be stored in a database. I have found it useful, however, to keep a sort of breadcrumb tree on the session, to track how users got to a specific place and where to take them back to when they finish. But the ID of the node they're actually on currently needs to be persisted on a page-by-page basis, and sent back with each request, so weird things don't happen when the user is browsing to different pages in different tabs.
Use incremental refactoring. You may end up re-writing the whole thing twice by the time you're done, or you may never really "finish" the refactoring. But in the meantime, everything will still work, and you'll have new features every so often. As a rule, rewriting the whole thing will take you several times as long as you think it will, so don't try to take the whole thing in a single bite.
I have a number of similar apps for building dynamic forms that I support.
There's a whole lot of things you could/could not do & you're right to think hard before throwing away 3 years of testing/development.
My input for you to consider is to implement a plug-in architecture on top of what you're got. Any custom code for a form goes in the plug-in & the name of this plug-in is stored with the form. When you generate a form, the correct plug-in is called to enhance the base functionality. that way you get to move all the custom code out of the existing library. It should also mean less breaking changes, each plug-in only affects the form it's attached to.
From that point it'll be easy to refactor the core engine as it's common functionality across all clients & forms.
Since your application seems to have become a big ball of mud, a complete (or an almost complete rewrite) might make sense.
You should also take into account new technologies like document-oriented databases (couchDB, MongoDB)
Most of the form definitions could probably fit pretty well in document-oriented databases. For exemple:
To define a customer form, you could use a document that looks like:
{Type:"FormDefinition",
EntityType: "Customer",
Fields: [
{FieldName:"CustomerName",
FieldType:"String",
Validations:[
{ValidationType:"Required"},
{ValidationType:"StringLength", Minimum:15, Maximum:50},
]},
...
{FieldName:"CustomerType",
FieldType:"Dropdown",
PossibleValues: ["Standard", "Valued", "Gold"],
DefaultValue: ["Standard"]
Validations:[
{ValidationType:"Required"},
{
ValidationType:"Custom",
ValidationClass:"MySystem.CustomerName.CustomValidations.CustomerStatus"
}
]},
...
]
};
With this kind of document to define your forms, you could easily add forms and validations which are customer specific.
You could easily add subforms using a fieldtype of SubForm or whatever.
You could define FieldTypes for all common types of fields like e-mail, phone numbers, address, etc.
namespace System.CustomerName.CustomValidations {
class CustomerStatus: IValidator {
private FormContext form;
private List<ValidationErrors> validationErrors;
CustomerStatus(FormContext fc) {
this.validationErrors = new List<ValidationErrors>();
this.form = fc;
}
public List<ValidationErrors> Validate() {
if (this.formContext.Fields["CustomerType"] == "Gold" && Int.Parse(this.form.Fields["OrderCount"]) < 10) {
this.validationErrors.Add(new ValidationError("A gold customer must have at least 10 orders"))
}
if (this.formContext.Fields["CustomerType"] == "Valued" && Int.Parse(this.form.Fields["OrderCount"]) < 5) {
this.validationErrors.Add(new ValidationError("A valued customer must have at least 5 orders"))
}
return this.validationErrors;
}
}
}
A record of a document with that definition could look like this:
{Type:"Record",
EntityType: "Customer",
Fields: [
{FieldName:"CustomerName", Value:"ABC Corp.",
{FieldName:"CustomerType", Value:"Gold",
...
]
};
Sure, this solution is a lot of work, but if/when realized it could be really easy to create/update/customize forms.
This is a common but (IMO) somewhat naive design approach. "Instead of solving the customer's problem, let's build a tool to let them solve their own problems!". But the reality is, that generally customers want YOU to solve their ACTUAL problems. So build things that solve their problems.
If you can architect it in a way that allows you to reuse some parts for different customers, fine. But that is generally what the frameworks have done for you already - work out the common features that applications need and make them available in neat packages.
We're currently migrating our ASP Intranet to .NET and we started to develop this Intranet in one ASP.NET website. This, however, raised some problems regarding Visual Studio (performance, compile-time, ...).
Because our Intranet basically exists of modules, we want to seperate our project in subprojects in Visual Studio (each module is a subproject).
This raises also some problems because the modules have references to each other.
Module X uses Module Y and vice versa... (circular dependencies).
What's the best way to develop such an Intranet?
I'll will give an example because it's difficult to explain.
We have a module to maintain our employees. Each employee has different documents (a contract, documents created by the employee, ...).
All documents inside our Intranet our maintained by a document module.
The employee-module needs to reference the document-module.
What if in the future I need to reference the employee-module in the document-module?
What's the best way to solve this?
It sounds to me like you have two problems.
First you need to break the business orientated functionality of the system down into cohesive parts; in terms of Object Orientated design there's a few principles which you should be using to guide your thinking:
Common Reuse Principle
Common Closure Principle
The idea is that things which are closely related, to the extent that 'if one needs to be changed, they all are likely to need to be changed'.
Single Responsibility Principle
Don't try to have a component do to much.
I think you also need to look at you dependency structure more closely - as soon as you start getting circular references it's probably a sign that you haven't broken the various "things" apart correctly. Maybe you need to understand the problem domain more? It's a common problem - well, not so much a problem as simply a part of designing complex systems.
Once you get this sorted out it will make the second part much easier: system architecture and design.
Luckily there's already a lot of existing material on plugins, try searching by tag, e.g:
https://stackoverflow.com/questions/tagged/plugins+.net
https://stackoverflow.com/questions/tagged/plugins+architecture
Edit:
Assets is defined in a different module than employees. But the Assets-class defines a property 'AssignedTo' which is of the type 'Employee'. I've been breaking my head how to disconnect these two
There two parts to this, and you might want to look at using both:
Using a Common Layer containing simple data structures that all parts of the system can share.
Using Interfaces.
Common Layer / POCO's
POCO stands for "Plain Old CLR Objects", the idea is that POCO's are a simple data structures that you can use for exchanging information between layers - or in your case between modules that need to remain loosely Coupled. POCO's don't contain any business logic. Treat them like you'd treat the String or DateTime types.
So rather than referencing each other, the Asset and Employee classes reference the POCO's.
The idea is to define these in a common assembly that the rest of your application / modules can reference. The assembly which defines these needs to be devoid of unwanted dependencies - which should be easy enough.
Interfaces
This is pretty much the same, but instead of referring to a concrete object (like a POCO) you refer to an interface. These interfaces would be defined in a similar fashion to the POCO's described above (common assembly, no dependencies).
You'd then use a Factory to go and load up the concrete object at runtime. This is basically Dependency Inversion.
So rather than referencing each other, the Asset and Employee classes reference the interfaces, and concrete implementations are instantiated at runtime.
This article might be of assistance for both of the options above: An Introduction to Dependency Inversion
Edit:
I've got the following method GetAsset( int assetID ); In this method, the property asset.AssignedTo (type IAssignable) is filled in. How can I assign this properly?
This depends on where the logic sits, and how you want to architect things.
If you have a Business Logic (BL) Layer - which is mainly a comprehensive Domain Model (DM) (of which both Asset and Employee were members), then it's likely Assets and Members would know about each other, and when you did a call to populate the Asset you'd probably get the appropriate Employee data as well. In this case the BL / DM is asking for the data - not isolated Asset and Member classes.
In this case your "modules" would be another layer that was built on top of the BL / DM described above.
I variation on this is that inside GetAsset() you only get asset data, and atsome point after that you get the employee data separately. No matter how loosely you couple things there is going to have to be some point at which you define the connection between Asset and Employee, even if it's just in data.
This suggests some sort of Register Pattern, a place where "connections" are defined, and anytime you deal with a type which is 'IAssignable' you know you need to check the register for any possible assignments.
I would look into creating interfaces for your plug-ins that way you will be able to add new modules, and as long as they follow the interface specifications your projects will be able to call them without explicitly knowing anything about them.
We use this to create plug-ins for our application. Each plugin in encapsulated in user control that implements a specific interface, then we add new modules whenever we want, and because they are user controls we can store the path to the control in the database, and use load control to load them, and we use the interface to manipulate them, the page that loads them doesn't need to know anything about what they do.
I am struggling with the following use-case:
User amends an existing order. The order is complex - lots of related 'entities' (addresses, post options, suppliers, makes, models, various items etc). Across multiple http posts.
User wants to discard the changes.
--
I have an order entity and as the user is editing this I am making various changes to the entity associations e.g changing order.address, order.items.add(item)...
In a single post this is fine, but across posts I don't know how best store state. If I store the entities then I cannot save the changes as they are across different data contexts. I have read that it is bad practice to store the data context in the session state i.e. long-lived context. I can't save changes after each edit/post because I cannot roll-back (?). I really would like to work with the entities during the editing process rather than one big save at the end (taking UI settings and applying these in one chunk).
This must be a pretty common problem - it's driving me mad. Any help really appreciated.
Cheers!
We have a similar problem where we are building a complex business object through a multi-page wizard.
Instead of creating a partially complete business object at each step of the wizard, we create a dedicated wizard object that looks pretty similar to the business object, populate that through the wizard. At each step in the wizard, the wizard object is saved into the database. At the end the user can accept it and it is converted to a real business object and then becomes visible to everyone else, or they can bin it and no-one else ever knows it existed.
If this kind of approach was not suitable, I suspect you're looking at some kind of difference tracking, either at the entity or database levels. Neither are simple to implement, work with or manage in a system. The former would be some kind of calculation and storage of n changes to the entities and developing an algorithm to undo them, the latter depends on your RDBMS, but might include versioned rows or similar.
Yes its pretty much common for us. In most scenarios we use the MVC approach. Even without the actual ASP .NET MVC Projects, we use similar ViewModel with our Views/Pages/Scenarios etc. where there is no direct/single entity mapping to the Business Layer (in other words, Business.Entities). This is pretty much similar to DTOs.
It is always easy to use Disconnected EF. We retrieve data and discard the context, then transform the Entities into ViewModels/DTOs if necessary. When you need to persist changes, all you have to do is to create a new context, find the latest entity instance do the changes.
The Views/Pages/Controllers will be managing these ViewModels/DTOs. Tracking Changed and Deleted content can be done by introducing a HistoryList<T> (you can extend a List<T> to implement this).
Once done, using a Controller/Workflow/Component you can observe the ViewModel/DTO and do the necessary changes to your Entities using a new Context to retrieve and persist.
It involves a bit of a coding and I would say its not a perfect solution since it has its own pros and cons.
/KP
I am developing a site that is saving non visible user data upon login (e.g. external ID on another site). We are going to create/save this data as soon as the account is created.
I could see us saving data using
the content profile module (already in use on our side)
the profile module
the data column in the user table
creating our own table
I feel like #1 is probably the most logical place, however creating a node within a module does not seem to be a trivial thing.
#3 feels like a typical way to solve this, but just having a bunch of serialized data in a catchall field does not feel like the best design.
Any suggestions?
IMO, each option has its pro's and con's, and you should be the one to make the final call, given that you are the only one to know what your project is about, what are the critical points of the project, what is the expected typical user pattern, what are the resources available, etc...
If I was totally free to chose, my personal favourites would be option #4, #1 and #5 (wait! #5? Yes: see below!). My guiding principles in making the choice would be:
Keep it clean
Keep it simple
Make it extensible
#1 - The content profile module
This would be a clean solution in that you would make easier for developer to maintain your code, as all the alteration to the user would pass through the same channel, and it would be easier to track down problems or add new functionality.
I do not find it particularly simple as it requires you to interact with the custom API of that module.
As for extensibility that depends from how well the content profile module API is designed. The temptation could be to simply use the tables done by said module for your purpose, bypassing the API's but that exposes you to the possibility that on a critical security update one day in which you are in a hurry the entire system will break down because the schema has changed...
#4 - Creating your own table
This would be a clean solution because you could design your table (and your module to do exactly what you need to), and you could create your own API to be used by other modules. On the other hand you would introduce yet another piece of code altering the registration process, and this might make it more difficult for devs to track problems and expand the system in a consistent way.
This would be very simple to implement code-wise. Also the DB design would benefit though: another thing to consider is that the tables would be very easy to inspect and query. Creating a new handler for views is dead easy in most of the cases: 4 out of 5 you simply use one of the prototype objects shipping with views.
This would be extremely easy to extend, of course. Once you created the module for one field, you could manage as many as you want by mostly copy/pasting the code for one field to another (or to inherit from the same ancestor if you go OOP).
I understand you are already knowledgeable about drupal, but if you need a pointer on how to do that, I gave some direction in this other answer.
#5 - Creating your own table and porting already existing fields there
That would essentailly be #4 minus the drawback of scattering functionality across various modules... Of course if you are already managing 200 fields the other way this is not a viable option, but if you are early into your design, you could consider this.
In my experience nearly every project that requires system integration (meaning: synchronising data for the same user in multiple systems) has custom needs for user registration, and I found this solution the one that suits my need best for two reasons:
I found I reuse a lot of the custom code I wrote from project to project.
It's the most flexible way to integrate data with the other system (in some case I even split data for the user in two custom tables managed by the same module: one that contains custom fields used by drupal only and one that contains the "non visible fields", as you call them. I find this very handy in a lot of scenarios, as it makes very easy to inspect and manipulate the data of the two systems separately.
HTH!
If you're already using the Content Profile module, then I'd really suggest continuing to use it and attach the field to it. You're saving the data when the account is created, and creating a node for the user at the same time isn't that hard. Really.
$node = new stdClass();
$node->title = $user->name; // what I'd use, or you can have node auto title handle the title for you. Up to you!
$node->field_hidden_field[0]['value'] = '$*#$f82hff';
$node->uid = $user->uid;
node_save($node);
Bob's your uncle!
I would go for option 3. Eventually even other modules will store the data in the database against a user. So you could directly save it yourself probably in a more efficient way than those modules.
Of course depending on the size of the data, you can take a call whether to add a new column in the users table or to create a new table for your data with the user's id as the foreign key.