Related
So, I need some input refactoring an asp.net (c#) application that is basically a framework for creating dynamic forms (any forms). From a high level point of view, there is a table that has the forms, and then there is a table that has all the form fields, where it is one to many between the two. There is a validation table, where each field can have multiple types of validation, and it is a one to many from the form fields table to the validation table.
So the issue is that this application has been sold as the be-all-end-all customizable solution to all the clients. So, the idea is whatever form they want, we can build it jsut using DB configurations. The thing is, that is not always possible, because there is complex relationship between the fields, and complex relationship between the forms themselves. Also, there is only once codebase, and this is for multiple clients - all of whom host it on their own. There is very specific logic for each of the clients, and they are ALL in the same codebase, with no real separation. Sometimes it was too difficult to make it generic, so there are instances where it has hard coded logic (as in if formID = XXX then do _). You can also have nested forms, as in, one set of fields on its own within each form.
So usually, when one client requests a change, we make the change and deploy it to that client - but then another client requests a different change, and we make the change and deploy it for THAT client, but the change from the earlier client breaks it, and its a headache trying to debug, because EVERYTHING is dynamic. There is no way we can rollback the earlier change, because then the other client would be screwed.
Its not done in a real 3-tier architecture - its a web site with references to a DB class, and a class library. There is business logic in the web site itself, in the class library, and the database stored procs (Validation is done in the stored procs).
I've been put in charge of re-organizing the whole thing, and these are my thoughts/questions:
I think this is a bad model in general, because one of the things I heard one of the developers say is that anytime any client makes a change, we should deploy to everybody - but that is not realistic, if we have say 20 clients - there will need to be regression testing on EVERYTHING, since we don't know the impact...
There are about 100 forms in total, and their is some similarity in them (not much). But I think the idea that a dynamic engine can solve ALL form requests was not realistic as well. Clients come up with the most weird requests. For example, they have this engine doing a regular data entry form AND a search form.
There is a lot of preserving state between pages, and it is all done using session variables, which is ok, except that it is not really tracked, and so sessions from the same user keep getting overwritten, and I think sessions should be got rid of.
Should I really just rewrite the whole thing? This app is about 3 years old, and there has been lots of testing and things done, and serious business logic implemented, so I hate to get rid of all that (joel's advice). But its really a mess of a sphagetti code, and everything takes forever to do, and things break all the time because of minor changes.
I've been reading Martin Fowlers "Refactoring" and Michael Feathers "working effectively with legacy code" - and they are good, but I feel they were written for an application that was 'slightly' better architected, where it is still a 3-tiered architecture, and there is 'some' resemblance of logic..
Thoughts/input anyone?
Oh, and "Help!"
My current project sounds like almost exactly the same product you're describing. Fortunately, I learned most of my hardest lessons on a former product, and so I was able to start my current project with a clean slate. You should probably read through my answer to this question, which describes my experiences, and the lessons I learned.
The main thing to focus on is the idea that you are building a product. If you can't find a way to implement a particular feature using your current product feature set, you need to spend some additional time thinking about how you could turn this custom one-off feature into a configurable feature that can benefit all (or at least many) of your clients.
So:
If you're referring to the model of being able to create a fully customizable form that makes client-specific code almost unnecessary, that model is perfectly valid and I have a maintainable working product with real, paying clients that can prove it. Regression testing is performed on specific features and configuration combinations, rather than a specific client implementation. The key pieces that make this possible are:
An administrative interface that is effective at disallowing problematic combinations of configuration options.
A rules engine that allows certain actions in the system to invoke customizable triggers and cause other actions to happen.
An Integration framework that allows data to be pulled from a variety of sources and pushed to a variety of sources in a configurable manner.
The option to inject custom code as a plugin when absolutely necessary.
Yes, clients come up with weird requests. It's usually worthwhile to suggest alternative solutions that will still solve the client's problem while still allowing your product to be robust and configurable for other clients. Sometimes you just have to push back. Other times you'll have to do what they say, but use wise architectural practices to minimize the impact this could have on other client code.
Minimize use of the session to track state. Each page should have enough information on it to track the current page's state. Information that needs to persist even if the user clicks "Back" and starts doing something else should be stored in a database. I have found it useful, however, to keep a sort of breadcrumb tree on the session, to track how users got to a specific place and where to take them back to when they finish. But the ID of the node they're actually on currently needs to be persisted on a page-by-page basis, and sent back with each request, so weird things don't happen when the user is browsing to different pages in different tabs.
Use incremental refactoring. You may end up re-writing the whole thing twice by the time you're done, or you may never really "finish" the refactoring. But in the meantime, everything will still work, and you'll have new features every so often. As a rule, rewriting the whole thing will take you several times as long as you think it will, so don't try to take the whole thing in a single bite.
I have a number of similar apps for building dynamic forms that I support.
There's a whole lot of things you could/could not do & you're right to think hard before throwing away 3 years of testing/development.
My input for you to consider is to implement a plug-in architecture on top of what you're got. Any custom code for a form goes in the plug-in & the name of this plug-in is stored with the form. When you generate a form, the correct plug-in is called to enhance the base functionality. that way you get to move all the custom code out of the existing library. It should also mean less breaking changes, each plug-in only affects the form it's attached to.
From that point it'll be easy to refactor the core engine as it's common functionality across all clients & forms.
Since your application seems to have become a big ball of mud, a complete (or an almost complete rewrite) might make sense.
You should also take into account new technologies like document-oriented databases (couchDB, MongoDB)
Most of the form definitions could probably fit pretty well in document-oriented databases. For exemple:
To define a customer form, you could use a document that looks like:
{Type:"FormDefinition",
EntityType: "Customer",
Fields: [
{FieldName:"CustomerName",
FieldType:"String",
Validations:[
{ValidationType:"Required"},
{ValidationType:"StringLength", Minimum:15, Maximum:50},
]},
...
{FieldName:"CustomerType",
FieldType:"Dropdown",
PossibleValues: ["Standard", "Valued", "Gold"],
DefaultValue: ["Standard"]
Validations:[
{ValidationType:"Required"},
{
ValidationType:"Custom",
ValidationClass:"MySystem.CustomerName.CustomValidations.CustomerStatus"
}
]},
...
]
};
With this kind of document to define your forms, you could easily add forms and validations which are customer specific.
You could easily add subforms using a fieldtype of SubForm or whatever.
You could define FieldTypes for all common types of fields like e-mail, phone numbers, address, etc.
namespace System.CustomerName.CustomValidations {
class CustomerStatus: IValidator {
private FormContext form;
private List<ValidationErrors> validationErrors;
CustomerStatus(FormContext fc) {
this.validationErrors = new List<ValidationErrors>();
this.form = fc;
}
public List<ValidationErrors> Validate() {
if (this.formContext.Fields["CustomerType"] == "Gold" && Int.Parse(this.form.Fields["OrderCount"]) < 10) {
this.validationErrors.Add(new ValidationError("A gold customer must have at least 10 orders"))
}
if (this.formContext.Fields["CustomerType"] == "Valued" && Int.Parse(this.form.Fields["OrderCount"]) < 5) {
this.validationErrors.Add(new ValidationError("A valued customer must have at least 5 orders"))
}
return this.validationErrors;
}
}
}
A record of a document with that definition could look like this:
{Type:"Record",
EntityType: "Customer",
Fields: [
{FieldName:"CustomerName", Value:"ABC Corp.",
{FieldName:"CustomerType", Value:"Gold",
...
]
};
Sure, this solution is a lot of work, but if/when realized it could be really easy to create/update/customize forms.
This is a common but (IMO) somewhat naive design approach. "Instead of solving the customer's problem, let's build a tool to let them solve their own problems!". But the reality is, that generally customers want YOU to solve their ACTUAL problems. So build things that solve their problems.
If you can architect it in a way that allows you to reuse some parts for different customers, fine. But that is generally what the frameworks have done for you already - work out the common features that applications need and make them available in neat packages.
We want a LMS(coded in ASP.NET/vb.net) which is able to import SCORM packages & display it to learner for viewing content. I am totally new to SCORM and have been shifted to this project. I want to know how can I access SCORM Assessment object's (Test) result, like Learner ID, passed/fail, time.
Can you please guide me what will I need to implement in ASP.NET code to accomplish my goal ?
Task that I have done so far is,
Reading a manifest zip file, unzipping the file and get all information from the file(content name,description,items and launching page) and when user clicks on a particular course a pop up window is launching the page.
I eagerly want to know what I can do next to communicate with the LMS with the APIs. Shall I need to develop my own LMS to get the result,If there is a quiz which is running, all I need to know is the no of questions attempted by the user, whether the user is pass or fail and I need to store all information in the database for individual user so that I can review the result afterwards.
So the task remaining.
Tracking mechanism to deliver the content.
SCORM/LMS sequencing engine that controls the navigation between parts of SCORM conformant course.
Please help.
SLK at codeplex provides a good starting point. However, if you are truly wanting to provide an in-house written SCORM play that is fully compliant, you have a major task ahead of you. In essence there are three party you need to fully develop:
CAM - the unzipping process, which it sounds like you have already achieved.
RTE - the javascript host for SCORM, providing the 8 specified methods. Behind this you also need to implement the SCORM object model, which SLC does help with. If you have implemented all of this, then there should be data entries on the data model that indicate completion etc.
SN - the sequencing and navigation processing. This is significantly the most complex part. I am still in the process of trying to implement this, using SLC, and it is hard. It is the completion of this that will potentially give you more information that will enable you to know what has been done.
it is also worth looking at scorm.com, who are a consultancy, but provide a lot of useful information about the scorm standard.
That is true. SCORM is one of these stadarts where you can implement as little as possible. But you will need some of Javascript with a Backend-Script (JSON to the rescue) so you can track the scorm data, and save it your database.
But let me tell you this: This is the easiest task! Making your own course-creator is a whole other beast.
We are in the process of designing a web site were users can fill an application form and submit it. Once an application is created, it goes through different departments for review.After each review, the department persons log on to the website and update the status of the application. Once review is completed, the application is said to be 'approved'. Also, communication(email) is sent to the applicants updating them about the status of their application.
I am thinking of using Windows Workflow for this application, but am new to it.
Can anyone tell me if it would make sense to use WF for such an app..or would it be an overkill..I was thinking of using a State machine workflow, as each application created goes through different states. One of my major concerns is, this process also involves manual work such as reviewing the application(this could take days) and then updating the status(say by clicking a button on the website). Can workflow handle this kind of thing?
I have gone through many websites, but they talk about very basic examples where states change automatically.But in my case there is a lot of human interaction.
Any help is appreciated
Thanks!!!
Here is a sample on MSDN that describes a very similar workflow that you've described - although they use a Windows Forms client rather than a web client - but those details shouldn't matter given the use of the Workflow Runtime.
I would say that your situation would be well suited to Workflow - long-running workflows are supported by having the runtime automatically persist workflow details at certain points while it runs, so steps that take days or weeks to fulfill are perfectly fine.
"my case there is a lot of human interaction".
That's the easiest kind of thing to build. It's just transactions.
Each stage in the process is a simple web application.
Fetch all items that are waiting at this state. Display them in a list. User picks one.
Display that piece of work. User makes changes. Saves it. Some change will move it to the next state. It's still just an update.
That's it. Nothing fancy.
You just need to very, very clearly define each state.
You must have a simple query that finds work in this state.
You must have a simple update that moves work on to the next state.
The states must be very, very clear and simple definitions. One column with a state name, for example.
The state transition rules, however, may be complicated. If the work is highly manual, it may be as simple as a drop-down list of available states.
If the state transitions are complex, then, perhaps you need something more sophisticated to embody the transition rules. But since it's manual, you don't need much.
Do I need a separate workflow instance for each application created?
I don't know what this can possibly mean. Each "application" that's moving through the pipeline of manual processing steps has a "state" -- the step of the pipeline where it's waiting, right now.
Each application also has a complete history of each state change. I don't know which of this is a "workflow" that could have an instance. They're just states of being of an object.
It is a web application. So it will have some other logic such as view an appln., navigation etc. do I need to accommodate this in the workflow as well?
Yes and No. Yes, each processing stage which show the application so the user at that stage can do whatever value-add thing they do. They add information or they approve information.
Each stage in the workflow is a place where some human being makes a decision and takes an action. So each stage displays all the information the person needs to make the decision. There's no additional view required.
Navigation isn't terribly interesting. People go to pages to see their queues of unprocessed work. I guess you'll have to build a page with some URL's, but there isn't much to that.
Would it be ok to have just a part of the application based on workflow.
Sure. Why not? I'm not sure I understand the question.
Your web apps just queries a queue of stuff out of the database and present that queue of stuff to a person. The person can request details and make changes to a specific item in the queue. One change the person can make will be to move the stuff out of their queue and into someone else's queue.
This isn't very complex. It shouldn't be. What's hard is defining the allowed state changes.
We use MS Dynamics 4.0 at work for our CRM. This handles all contact management, marketing, resource sharing w/ sharepoint integration, workflow management / collaboration and essentially is used by every department in the firm in some way or another.
We have requirements from business for a new application that we have a tight timeline on. We have only just started rolling out CRM, and most of the custom development was done by a consulting firm.
We need a relatively simple application that we need to track some data for sharing for a specific group. Some of this information already lives in our 'company' and 'client' CRM entity.
This new project would require us to add around 26 fields - we don't want to bloat our already large company entity - especially since only around 5% of our companies would use these extra fields.
We are basically debating a design right now - hybrid solution (create our own ASP.NET app that looks like CRM and communicates to it via web services and store all the 'supplemental' fields on our own database, possibly living on the same DB server as our CRM DB so we can easily write queries). The other alternative is to do it 100% in CRM.
I'm just looking for advice for people who have done something similar to this. Would you recommend doing a hybrid solution such as this or should we do 100% CRM? Our deadline is tight and the developers working on the project have limited CRM knowledge; this is why it's a bit of a debate. For those working with MS Dynamics - how would you typically handle a project like this, where we need to add many fields (and even sub fields with parent->child relationships of their own) that would only apply to a very small percentage of our main 'company' entity.. something to note: we are already having performance issues when people load up this company entity as is (it could take 5 seconds for the page to render) and the same goes for advanced finds.
Last thing to note - this portion of the application is only for storing the data. In the end, the user will be opening a VBA Excel workbook, pushing a 'pull down data' button, that will pull this data from wherever we wind up storing it. We just aren't sure where we should store/manage this data/UI.
Thanks very much for any advice.
EDIT: How can I create 2 list boxes next to each other with 2 buttons in the middle where one listbox is a lost of 'my foos' and the other is 'all foos' and you add/take away from 'my foos' list box??? the classic 'i have these foos as part of me' UI control with 2 list boxes and 2 arrow buttons... Should/can I use jquery for this? and does anyone happen to know of any jquery control that already does all of this out of the box? This is such a common control I'm sure it must be out there somewhere. I've browsed some toolkits and controls and some threads on here and seen some really awesome, even more complicated controls but not this particular one..
EDIT2: After doing more research, it seems like keeping the UI all in CRM would be more complicated then just making an ASP.NET app for that portion and putting it in an iframe or modal popup in CRM.
We can still setup all the data fields and relationships in CRM - and have the ASP.NET do the CRUD using Webservice calls.
It seems we would wind up having to do the same amount of work to get the functionality needed in CRM - except it would be more hackish and done in javascript. At what benefit? Keeping the UI in one place??? Not that much of a trade off IMO...
so far we are leaning towards keeping all data in CRM but putting the UI in ASP.NET
Any advice is greatly appreicated. Is what I'm saying sane? Thanks
I agree, you're better off going with 100% CRM.
If (and I stress if) you find the performance impact is significant, consider using a related entity to hold the additional fields.
CRM doesn't provide a 1-to-1 relationship type so you'll have to manage that yourself. Make your company entity the N side of the relationship so the related entity appears as a lookup.
Alternately, if the related entity lookup is too abstract for your users, add a tab with an iFrame to the company entity form. Use javascript to show/hide the tab and also to set the src of the iFrame to the url of the related entity.
I'd use CRM to store the data. You can stick the new fields in a separate pane in the UI so that it won't clutter. You can even add some Javascript to the UI to hide pane/fields from users who are not part of the group that requires them. I know this sounds a little hacke-ty, but it's a lot less work than coming up with an entirely different app and users will get a consistent experience. Having the data in one place is also a boon for reporting and such.
I can't say for sure, but I don't think adding a few columns to an entity (which already has a bazzilion columns) will deteriorate performance much further. I'd go over the installation and check for the usual performance pitfalls.
Creating ASP.NET applications to create a complex UI in an iframe is a simple solution that I use frequently for MS Dynamics CRM 4.0 applications.
Keeping all of the data in CRM makes a lot of sense, but make the UI however you want it.
The iframe calls your ASP.NET application with a Querystring containing the entity's GUID so that you can use web services to pull any related information.
You can both modify the fields showing on the form with JavaScript, directly update the database or both for consistency. Frequently it is easier just to hide the fields being updated in the ASP.NET application so there is no confusion.
An example from a long time ago was a loan morgage calculator that I built for an iframe of an opportunity that a sales representative would have up. It would find all of the customer's related loan balances and calculate different options that the sale representative could then turn into a quote. Click a few check boxes, and press a button and they were done without having to rekey a lot of information. Data was written to a number of CRM entities, emails were generated and the autodialer list would be modified not to call that customer again.
Learning to use MS CRM as a big development toolbox is the first step to being able to do some serious business process automation.
If you have any questions let me know.
I know that this doesn't fit your situation as you are deep in MS CRM, but there is a good article by Neal Ford that was recently posted to IBM Developer Works (http://www.ibm.com/developerworks/java/library/j-eaed10/) that discusses COTS vs home grown software. Here's a snippet.
One of the common questions that arise in big companies is the decision whether to build or buy: for the current requirements, should we buy COTS (Commercial Off-the-Shelf Software) or build it ourselves? The motivation for this decision is understandable — if the company can find some already written software that does exactly what's needed, it saves time and money. Unfortunately, lots of software vendors understand this desire and write packaged software that can be customized if it doesn't do exactly what the client needs. They are motivated to build the most generic software they can because it will potentially fit into more ecosystems. But the more general it is, the more customization is required. That's when an army of consultants shows up, sometimes taking years to get all the custom coding done.
I am working on a design spec for a new application that will be heavily workflow driven.
Before I re-invent the wheel, is there a decent lightweight workflow engine that plugs into ASP.NET already around?
Basically, I'm looking for something that handles moving through a defined set of workflow pages while handling state management automatically.
If this isn't around already, I'll definitely try to abstract the engine from my app and put it on codeplex, as it would be really handy.
Any suggestions?
Note: .NET 2.0, so no WWF, though I think WWF is overkill for my needs.
EDIT: Seems like there is a legitimate need for this, and there isn't a product out there...So I might build this.
Here is what I'm picturing:
Custom Page class called WebFlowPage
All WebFlowPage's are registered in a Workflow mapper.
Each WebFlowPage has some form of state object.
A HttpHandler handles picking the appropriate WebFlowPage based upon the workflow, and populating it from the state object.
Is the workflow dynamic, or static?
If the workflows are simple, you could roll your own workflow engine.
In certain situations, it can be fairly simple, and just a couple of data tables to handle the rules, processing and state.
Alot of workflow engines are built for large scale processing (credit card applications, for example). For small scale, you should at least consider your own, which would eliminate the overhead and dependency of/on an engine.
Not sure exactly what you wish to do here, but Ra-Ajax can easily keep state at least if you want your solution ajaxified...
For reference purposes you might want to check out the Ajax Calendar sample or even the (banalistically implemented) Ajax Wizard sample. It surely beats the hell out of doing it with JavaScript...
And every time you "do something" you're in "server-land" which means you can store temporaries all the time as you wish...
The project is LGPL
(PS!
Yes I do work with it)
Building a custom workflow engine is not trivial, although it may seem simple at first. We've tried that. It depends a lot on the complexity of the logic you need it to cover.
Given the current state of the Windows Workflow Foundation and the lack of another framework that abstracts the workflow concepts, I would choose WF if you need complex logic, asynchronous handling or branches in your workflows.
Tracking your state through the workflow can be accomplished by carrying some kind of xml payload or storing the state in a database,
If your workflow is actually a sequential set of forms that need to be filled in by the user, tracking the steps and guiding the user to the next step can be accomplished with some simple custom solution.
You could take a look at the InRule engine too.
Also, there is nxBRE.
These too are mostly used for business rules.
InRule is proprietary, whereas nxBRE supports RuleML (the defacto standard).
You might need to make your own implementation for the pages, and use the rule engine as the "structure".
At this moment, I know that Sharepoint 2007 supports page workflows (using WF), but this would imply using .NET Framework 3 and deployng sharepoint.
My suggestion would be to use whatever you find more light and easier to use.
I think the term "workflow" is very open to interpretation. I have been working lately with a type of workflow that is very different from what you seem to be describing. Mine is a state machine based workflow where the state of a particular record determines what actions a user can take to move the record to the next step in the business process. So "workflow" in this instance means how the record flows from one state to another until it is finally completed.
Your usage of workflow seems to have more to do with moving a user from one page to another in a linear multi-step process, which is a completely different use case (correct me if I'm wrong). So before coming up with a general purpose "workflow" engine that anyone could use, I would recommend defining a little bit better exactly what types of situations this system would handle.
I've been using this for a few months http://objectflow.codeplex.com. Not asp specific but it may fit your needs
While browsing the web for some workflow & BPM resources, I found the following project: NetBPM. Unfortunately, the project seems to be stopped.
I don't think there is a workflow engine that will automatically handle state for you, but if you are moving through a set of pages like a process such as checkout on an ecommerce site, perhaps the ASP.NET wizard control could help you?
There are few workflow options. "Aspose" and "Skelta" are the offers I´m evaluating.
Fábio
you can use WorkFlow Engine, just read the document and run the Demo.
all of the features you need for a dynamic workflow engine they added in there.