We have a client who is currently using Lotus Notes/Domino as their content management system and web server. For many reasons, we are recommending they sunset their Notes/Domino implementation and transition onto a more modern platform--such as Drupal.
The client has several web applications which would be a natural fit for Drupal. However, I am unsure of the best way to implement one of the web applications in Drupal. I am running into a knowledge barrier and wondered if any of you could fill in the gaps.
Situation
The client has a Lotus Domino application which serves as a front-end for querying a large DB2 data store and returning a result set (generally in table form) to a user via the web. The web application provides access to approximately 100 pre-defined queries--50 of which are public and 50 of which are secured. Most of the queries accept some set of user selected parameters as input. The output of the queries is typically returned to users in a list (table) format. A limited number of result sets allow drill-down through the HTML table into detail records.
The query parameters often involve database queries themselves. For example, a single query may pull a list of company divisions into a drop-down. Once a division is selected, second drop-down with the departments from that division is populated--but perhaps only departments which meet some special criteria--such as those having taken a loss within a specific time frame. Most queries have 2-4 parameters with the average probably being 3.
The application involves no data entry. None of the back-end data is ever modified by the web application. All access is purely based around querying data and viewing results.
The queries change relatively infrequently, and the current system has been in place for approximately 10 years. There may be 10-20 query additions, modifications, or other changes in a given year. The client simply desires to change the presentation platform but absolutely does not want to re-do the 100 database queries.
Once the project is implemented, the client wants their staff to take over and manage future changes. The client's staff have no background in Drupal or PHP but are somewhat willing to learn as necessary.
How would you transition this into Drupal? My major knowledge void relates to how we would manage the query parameters and access the queries themselves. Here are a few specific questions but feel free to chime in on any issue related to this implementation.
Would we have to build 100 forms by hand--with each form containing the parameters for a given query? If so, how would we do this?
Approximately how long would it take to build/configure each of these forms?
Is there a better way than manually building 100 forms? (I understand using CCK to enter data into custom content types but since we aren't adding any nodes, I am a little stuck as to how this might work.)
Would it be possible for the internal staff to learn to create these query parameter forms--even if they are unfamiliar with Drupal today? Would they be required to do any PHP programming?
How would we take the query parameters from a form and execute a query against DB2? Would this require a custom module? If so, would it require one module total or one module per query? (Note: There is apparently a DB2 driver available for Drupal. See http://groups.drupal.org/node/5511.)
Note: I am not looking for CMS recommendations other than Drupal as Drupal nicely fits all of the client's other requirements, and I hope to help them standardize on a single platform.
Any assistance you can provide would be helpful. Thank you in advance for your help!
Have a look at the Data module - it might be able to get you a long way towards a solution.
The biggest problem you are likely to have is connecting to the DB2 server through Drupal since it's DBA layer doesn't support it without patches (as you've discovered).
A couple things come to mind.
You can use Table Wizard to expose any custom tables to views. Views basically gives you a UI for writing custom sql queries. Once your tables are exposed to views you can use filters to "parameterize" the query. Also, views supports many display options including tables and pagers.
If you do need to write your own forms for advanced queries, take a look at the Drupal Form API which makes creating custom php forms a peice of cake.
Views is fairly easy to learn and, in my experience, most clients pick it up quickly. It really depends on the complexity of the query.
Form API should be easy for a developer to pick up, but does require a basic knowledge of php and writing Drupal code.
Does Lotus provide a web service to query it? If so, you can easily do this using a custom form (in your own module) and use Services module to query the data.
Related
I have a Drupal 7 website with content types like "events" and "news".
I would like for nodes of these content types to be automatically imported into other websites.
I played with Feeds, XPath on the 'client' websites and Views RSS fields' onn the 'server' side, but I realized that there would be problems with content type fields like files... Any suggestions? I would like to be able to create new views for this content in the other websites.
P.S. The content types will be identical between the websites (but they don't have to, if your solution includes something else).
You probably have more success with services and content Distribution. RSS feeds are not well suited for transfer of semantic data. They are highly focused on lists-of-articles and typically lack information such as "event-start-date".
Services allows you to expose services on the server-drupal-site, exposing the nodes as e.g. RESTfull json. The client side-drupalsite can then use services and content-distribution to import nodes from said server.
That said, the services suit plugs into views, and is really heavy, large and complex. If you are allergic to large and complex projects (like I am), you may be better of writing simple modules:
events-service: a 20+ lines module that grabs the events from the database, and presents them as json.
news-service: a 10+ lines module that fetches a list of news-nodes and presents them as json.
events-client: a small module (~400-800 lines?) that eats said json at the given url and turns them into nodes. It will keep a register of some UUID next to the nodes table, to avoid re-creating nodes on changes upstream (but instead finding the associated one and updating that).
news-client: a small module. Same as above.
Writing such modules is very rewarding, because instead of fighting with poorly documented views-plugins, complex layers around services and such, you have full control and full understanding. It also allows for a lot better tuning and performance.
The one large downside is that Drupal, more specific: CCK or Fields, dictate the database and its structure. There will be a point when some tiny config change on your site breaks your modules SQL query: all of a sudden you are blasting SQL errors because Drupal decided to rename or move out some table, column or reference.
Maybe you can just share data by creating xmls/json (server side) that will be used by the client side.
services is a good way to go. But I find it complex for simple stuffs.
What you can do is create views that will output as xml/json... You can do this by doing preprocess functions in your module/template file.
After which the client side (maybe run cron) will take the xml/json and create nodes programmatically.
My professor recently gave my class an assignment and I need help finding the tools to do so.
We have a database of tables (Customer, Payment, Order, etc.) and I need to use Drupal 7 to allow users to see customer's info, view customer data (i.e. payment history), or edit customer information. This would involve select statements, aggregate clauses, joins, etc. It would also involve forms to input customer name/information.
Here is the catch: I am not allowed to write any code. That means no API, no SQL queries, and no PHP. I am only allowed to use ready-built modules.
So my questions is: What module(s) should I use, and how do I use them?
I know the view is a good place to start, but the default options of view are for content. I have tables on a database, not content. Also from what I understand, custom queries of view require coding.
I believe this is what you are looking for:
http://drupal.org/project/data
If it integrates with the Views module properly, then you won't be required to do any hand-coding to make a View which queries your tables.
(Disclaimer: I haven't actually used it)
I've recently created a very simple CRUD table where the user stores some data. For the data, I created a custom node. The functionality works great for creating, editing, and deleting data in the CRUD table using the basic node functionality (I'm actually amazed how fast and easy it was to program the basic functionality with proper access controls using only a tiny bit of code)....
Since the data isn't meant to be treated the same way as 'content' such as a blog post (no title, no body, no commments, no revisions, shouldn't show up on ?q=node page, no previews, no teasers, etc)... I find that I'm spending most of my time 'turning off' and modifying the stuff that drupal does automatically for nodes.
I know its a matter of taste, but where should one draw the line on what should be treated as a node and what shouldn't? In other words, would it be better to program this stuff from scratch without using nodes?
Using nodes for custom data has quite some additional benefits besides easy edit/update/delete functionality:
possible categorization via taxonomy
implicit 'ownership' via author tracking
implicit tracking of creation/modification time
basic access control by default, expandable by a huge selection of modules
flexible query generation/listing/filtering via views
possible ad hoc extensions/annotations via CCK fields
possible definition of workflows, actions and the like
a huge number of hooks to programmatically intercept/adjust almost every usage aspect/scenario
commenting, voting, rating and tons of other functionality provided by all contributed modules that work on/with nodes ...
Given all this, I'd say you need a very good reason to not use nodes to store data in Drupal. Nodes are simply the fundamental building blocks for just about everything in the Drupal ecosystem, and the overhead of removing some unwanted default 'features' seems pretty small in comparison to the gains.
That said, one possible reason/argument to handle data separate from the node system might be if that data is directly aimed at annotating other nodes (think taxonomy). But since you can easily reference nodes from other nodes (with lots of different options on how to do this), the argument is not to strong.
Another (much stronger) argument would be data integrity - Drupal is not very strong (to put it politely) concerning normalized, relational data storage, referential integrity, transaction handling and other related topics. If you have requirements in that direction, you might have no choice but to skip the node concept and create and maintain a separate data island within the system on your own.
It helps to think also that a node doesn't need to be public either. Some nodes are private/internal and can be controlled further with access controls. The way you are doing it, whatever you're doing, makes all the scalability and extending it on your shoulders.
I would probably approach it with CCK/Taxonomy depending on what I was doing. That way, I get the added benefit of Views/Panels/etc module integration without writing any additional code.
We use MS Dynamics 4.0 at work for our CRM. This handles all contact management, marketing, resource sharing w/ sharepoint integration, workflow management / collaboration and essentially is used by every department in the firm in some way or another.
We have requirements from business for a new application that we have a tight timeline on. We have only just started rolling out CRM, and most of the custom development was done by a consulting firm.
We need a relatively simple application that we need to track some data for sharing for a specific group. Some of this information already lives in our 'company' and 'client' CRM entity.
This new project would require us to add around 26 fields - we don't want to bloat our already large company entity - especially since only around 5% of our companies would use these extra fields.
We are basically debating a design right now - hybrid solution (create our own ASP.NET app that looks like CRM and communicates to it via web services and store all the 'supplemental' fields on our own database, possibly living on the same DB server as our CRM DB so we can easily write queries). The other alternative is to do it 100% in CRM.
I'm just looking for advice for people who have done something similar to this. Would you recommend doing a hybrid solution such as this or should we do 100% CRM? Our deadline is tight and the developers working on the project have limited CRM knowledge; this is why it's a bit of a debate. For those working with MS Dynamics - how would you typically handle a project like this, where we need to add many fields (and even sub fields with parent->child relationships of their own) that would only apply to a very small percentage of our main 'company' entity.. something to note: we are already having performance issues when people load up this company entity as is (it could take 5 seconds for the page to render) and the same goes for advanced finds.
Last thing to note - this portion of the application is only for storing the data. In the end, the user will be opening a VBA Excel workbook, pushing a 'pull down data' button, that will pull this data from wherever we wind up storing it. We just aren't sure where we should store/manage this data/UI.
Thanks very much for any advice.
EDIT: How can I create 2 list boxes next to each other with 2 buttons in the middle where one listbox is a lost of 'my foos' and the other is 'all foos' and you add/take away from 'my foos' list box??? the classic 'i have these foos as part of me' UI control with 2 list boxes and 2 arrow buttons... Should/can I use jquery for this? and does anyone happen to know of any jquery control that already does all of this out of the box? This is such a common control I'm sure it must be out there somewhere. I've browsed some toolkits and controls and some threads on here and seen some really awesome, even more complicated controls but not this particular one..
EDIT2: After doing more research, it seems like keeping the UI all in CRM would be more complicated then just making an ASP.NET app for that portion and putting it in an iframe or modal popup in CRM.
We can still setup all the data fields and relationships in CRM - and have the ASP.NET do the CRUD using Webservice calls.
It seems we would wind up having to do the same amount of work to get the functionality needed in CRM - except it would be more hackish and done in javascript. At what benefit? Keeping the UI in one place??? Not that much of a trade off IMO...
so far we are leaning towards keeping all data in CRM but putting the UI in ASP.NET
Any advice is greatly appreicated. Is what I'm saying sane? Thanks
I agree, you're better off going with 100% CRM.
If (and I stress if) you find the performance impact is significant, consider using a related entity to hold the additional fields.
CRM doesn't provide a 1-to-1 relationship type so you'll have to manage that yourself. Make your company entity the N side of the relationship so the related entity appears as a lookup.
Alternately, if the related entity lookup is too abstract for your users, add a tab with an iFrame to the company entity form. Use javascript to show/hide the tab and also to set the src of the iFrame to the url of the related entity.
I'd use CRM to store the data. You can stick the new fields in a separate pane in the UI so that it won't clutter. You can even add some Javascript to the UI to hide pane/fields from users who are not part of the group that requires them. I know this sounds a little hacke-ty, but it's a lot less work than coming up with an entirely different app and users will get a consistent experience. Having the data in one place is also a boon for reporting and such.
I can't say for sure, but I don't think adding a few columns to an entity (which already has a bazzilion columns) will deteriorate performance much further. I'd go over the installation and check for the usual performance pitfalls.
Creating ASP.NET applications to create a complex UI in an iframe is a simple solution that I use frequently for MS Dynamics CRM 4.0 applications.
Keeping all of the data in CRM makes a lot of sense, but make the UI however you want it.
The iframe calls your ASP.NET application with a Querystring containing the entity's GUID so that you can use web services to pull any related information.
You can both modify the fields showing on the form with JavaScript, directly update the database or both for consistency. Frequently it is easier just to hide the fields being updated in the ASP.NET application so there is no confusion.
An example from a long time ago was a loan morgage calculator that I built for an iframe of an opportunity that a sales representative would have up. It would find all of the customer's related loan balances and calculate different options that the sale representative could then turn into a quote. Click a few check boxes, and press a button and they were done without having to rekey a lot of information. Data was written to a number of CRM entities, emails were generated and the autodialer list would be modified not to call that customer again.
Learning to use MS CRM as a big development toolbox is the first step to being able to do some serious business process automation.
If you have any questions let me know.
I know that this doesn't fit your situation as you are deep in MS CRM, but there is a good article by Neal Ford that was recently posted to IBM Developer Works (http://www.ibm.com/developerworks/java/library/j-eaed10/) that discusses COTS vs home grown software. Here's a snippet.
One of the common questions that arise in big companies is the decision whether to build or buy: for the current requirements, should we buy COTS (Commercial Off-the-Shelf Software) or build it ourselves? The motivation for this decision is understandable — if the company can find some already written software that does exactly what's needed, it saves time and money. Unfortunately, lots of software vendors understand this desire and write packaged software that can be customized if it doesn't do exactly what the client needs. They are motivated to build the most generic software they can because it will potentially fit into more ecosystems. But the more general it is, the more customization is required. That's when an army of consultants shows up, sometimes taking years to get all the custom coding done.
I am looking for guidance regarding the best practice around the use of the Profile feature in ASP.NET.
How do you decide what should be kept in the built-in user Profile, or if you should create your own database table and add a column for the desired fields? For example, a user has a zip code, should I save the zip code in my own table, or should I add it to the web.config xml profile and access it via the user profile ASP.NET mechanism?
The pros/cons I can think of right now are that since I don't know the profile very well (it is a bit of a Matrix right now), I probably can do whatever I want if I go the table route (e.g., SQL to get all the users in the same zip code as the current user). I don't know if I can do the same if I use the ASP.NET profile.
Ive only built 2 applications that used the profile provider. Since then I have stayed away from using it. For both of the apps I used it to store information about the user such as their company name, address and phone number.
This worked fine until our client wanted to be able to find a user by one of these fields.
Searching involved looping through every users profile and comparing the information to the search criteria. As the user base grew the search time became unacceptable to our client. The only solution was to create a table to store the users information. Search speed was increased immensely.
I would recommend storing this type of information in its own table.
user profile is a nice clean framework for individual customization(AKA. Profile Properties). (e.g. iGoogle)
the problem of it is its not designed for query and not ideal for data sharing to public user.(you still would be able to do it, with low performance)
so, if you want to enhance the customized user experience, user profile would be a good way to go. otherwise, use your own class and table would be a much better solution.
In my experience its best to keep an the info in the profile to a bare minimum, only put the essentials in there that are directly needed for authentication. Other information such as addresses should be saved in your own database by your own application logic, this approach is more extensible and maintainable.
I think that depends on how many fields you need. To my knowledge, Profiles are essentially a long string that gets split at the given field sizes, which means that they do not scale very well if you have many fields and users.
On the other hand, they are built in, so it's an easy and standardized way, which means there is not a big learning curve and you can use it in future apps as well without needing to tweak it to a new table structure.
Rolling your own thing allows you to put it in a properly normalized database, which drastically improves performance, but you have to write pretty much all the profile managing code yourself.
Edit: Also, Profiles are not cached, so every access to a profile goes to the database first (it's then cached for that request, but the next request will get it from the database again)
If you're thinking about writing your own thing, maybe a custom Profile Provider gives you the best of both worlds - seamless integration, yet the custom stuff you want to do.
I think it is better off using it for supplementary data that is not critical to the user that is only normally important when that user is logging in anyway. Think data that would not break anything important if it was all wiped.
of course thats personal preference but others have raised some other important issues.
Also very useful considering it can be used for an unauthenticated user whose profile is maintained with an anonymous cookie.