I have a question about queries optimization and data "grouping"
Here's my setup :
I have a single application for multiple sites
Items I display to my sites are stored in a single table (one item can be shown in different sites)
So I have joins between my "sites" table and "items" table
When a user authenticate in site01 i'd like to present him only the data for this site.
I was wondering if there was a genius doctrine (or other) trick to make it transparent in my code:
For example:
User authenticates for site01
I set in my siteManager that it's 01
And all my entities of items (for this session) now refer to a subset of my data according to the JOIN with site01.
Like a view or something.
My concern is as much performance optimization than transparency in code.
Performance coming first.
nb: the choice of using a single item table is not definite yet, it might be a bit to meta and/or conceptual to provide good performance.
Thank you in advance for any input on my question.
Related
I’m absolutely stumped by what I suspect is probably simple for those experienced with Access. I’m brand new to Access (2010) and studying a lot, but unfortunately still confused by many basic concepts. I’m embarrassed to admit that I have spent about 40 hours trying (unsuccessfully) to solve the specific issue below. Please forgive me if I haven’t included enough detail here - I’m not sure how much someone needs to know to address this. I’m happy to edit and/or provide more information.
My question:
How do I create a Form for creating new records / editing existing records in Table1 where two fields in Table1 have different values from a single field in Table2? (I have better detail below)
I want the Form to have all the records from Table1. I have tried many different ways with queries, sub-forms, etc., but can’t pull it off. I’m fairly certain the issue is related to how I address Table2. Ideally, the user would be able to select from dropdowns in the form for the two fields to be updated in Table1.
I am including screenshots of a mockup of my intended Form concept, the object Relationships as I currently have them, the design and datasheet views of Table1 and the design and datasheet views of Table2.
“Table1” above is “t_PEOPLE” in the images while “Table2” is “t_COLORS.”
The object relationship types are currently one-to-many with enforced referential integrity (cascade update related fields) and the join properties are "include ALL records from 't_PEOPLE' and only those records from 't_COLORS' where the joined fields are equal."
I'm happy to send the actual database file if that helps.
I will be very grateful for any guidance - thank you!!
The general approach to this is as follows:
A) If I understand what you're trying to do here, your t_colors table is usually referred to as a Reference table or Lookup table. You need to make one form to add, edit, and delete records in this t_colors table. How the user accesses that form varies. I'll get to that in a minute.
B) The form for your People table needs to have drop down menus for your two color selections. In the dropdown menus' RowSource, you will use a query that looks up values in your t_colors table.
C) Depending what you are using your colors for in t_people, you should consider making a third table with PeopleID and ColorID in it. It would then link to both t_people and t_colors. This would allow you to have multiple colors specified for a single person, and you wouldn't be limited to two. In your People form, you would use a subform for these colors. The subform would probably need to be a datasheet form or a continuous form. If you are using a datasheet form for your people form, then you would need to use a datasheet form for the colors subform.
If the user wants to use a color that isn't already in your t_colors table, you need to give them a way of inserting that color. There are various approaches to this. You could use a union query in the dropdowns RowSource that shows a "" option. If selected you would bring up your Colors form and when they close the colors form you have to requery your dropdown menu. Or you could insert the color for them using VBA when they enter a value that is "Not In List" (an event that Combobox's have).
Please note that the relationships you've defined are not overly helpful or important in this case. Yes, they can be helpful when it comes to using the Update Cascade or Delete Cascade features. But quite truthfully, relationships are basically for programmers, to make sure you get an error if and when referential integrity is violated. Users should never see these errors and properly designed forms will prevent them from occurring. The main reason to use them is that it will force you to design your forms properly by giving you an error when something is wrong, hopefully during your own testing phase of the project.
I have about 20 different tables that each have a different parent / child relationship built into them. I've recently been asked to create a breadcrumb and Site Map for our website based off of all of these tables.
One idea I had, was to remove the parent / child relationship from each of these tables and create basically one table that holds the id and parentId and whenever I need to pull the parent child relationship I would just join the parent_child_relationships table to whatever table I was pulling from specifically.
Does this make sense?
Anyway, the problem with this idea is that i don't like it. haha.
Does anyone else have any other ideas of how this could be done? Or what the correct way of building a breadcrumb and sitemap based off of a site comprised of 20 tables or so?
If it helps, my site is comprised of asp.net, ColdFusion and uses a MSSQL database.
Thanks!
Do not let the implementation of the UI effect the design of your model and especially not your DB. Prototype the front end, involve your customer(s), give them a voice. Build your breadcrumbs and site map without it initially tied into your actual DB. Once your customer says "thats what we want, just like that", then freeze the prototype, then work on the actual implementation - how will your app request the data, what type of dataobject will you use AND THEN build your db,
"One idea I had, was to remove the parent / child relationship from each of these tables and create basically one table that holds the id and parentId"
This is not a very scalable solution, do not *reverse normalize your db. Follow standard relation database modeling/normalization techniques. Lots of small cohensive tables with lots of association tables.
I have an ASP.NET data entry application that is used by multiple clients. The application consists of multiple data entry modules that are common to all clients.
I now have multiple clients that want their own custom module added which will typically consist of a dozen or so data points. Some values will be text, others numeric, some will be dropdown selections, etc.
I'm in need of suggestions for handling the data model for this. I have two thoughts on how to handle. First would be to create a new table for each new module for each client. This is pretty clean but I don't particular like it. My other thought is to have one table with columns for each custom data point for each client. This table would end up with a lot of columns and a lot of NULL values. I don't really like either solution and suspect there's a better way to do this, so any feedback you have will be appreciated.
I'm using SQL Server 2008.
As always with these questions, "it depends".
The dreaded key-value table.
This approach relies on a table which lists the fields and their values as individual records.
CustomFields(clientId int, fieldName sysname, fieldValue varbinary)
Benefits:
Infinitely flexible
Easy to implement
Easy to index
non existing values take no space
Disadvantage:
Showing a list of all records with complete field list is a very dirty query
The Microsoft way
The Microsoft way of this kind of problem is "sparse columns" (introduced in SQL 2008)
Benefits:
Blessed by the people who design SQL Server
records can be queried without having to apply fancy pivots
Fields without data don't take space on disk
Disadvantage:
Many technical restrictions
a new field requires DML
The xml tax
You can add an xml field to the table which will be used to store all the "extra" fields.
Benefits:
unlimited flexibility
can be indexed
storage efficient (when it fits in a page)
With some xpath gymnastics the fields can be included in a flat recordset.
schema can be enforced with schema collections
Disadvantages:
not clearly visible what's in the field
xquery support in SQL Server has gaps which makes getting your data a real nightmare sometimes
There are maybe more solutions, but to me these are the main contenders. Which one to choose:
key-value seems appropriate when the number of extra fields is limited. (say no more than 10-20 or so)
Sparse columns is more suitable for data with many properties which are filled out infrequent. Sounds more appropriate when you can have many extra fields
xml column is very flexible, but a pain to query. Appropriate for solutions that write rarely and query rarely. ie: don't run aggregates etc on the data stored in this field.
I'd suggest you go with the first option you described. I wouldn't over think it. The second option you outlined would be a bad idea in my opinion.
If there are fields common to all the modules you're adding to the system you should consider keeping those in a single table then have other tables with the fields specific to a particular module related back to the primary key in the common table. This is basically table inheritance (http://www.sqlteam.com/article/implementing-table-inheritance-in-sql-server) and will centralize the common module data and make it easier to query across modules.
I want to use ASP.NET Dynamic Data for my next project, but there is a problem a can't manage to solve. In the database we manage authorization on a per-row basis. For example no user is permitted to see all rows of the Contracts table. So there is a Many to Many Relationship between Contracts and Users. So everytime Dynamic Data performs a Select to show all Contracts it has to look into the ContractUsers junction table to see what contracts the current user is permitted to see (filtered by UserID which will be stored in a session variable). Of course these junction tables should be invisible to the users.
By default Dynamic Data returns all rows of a table, so is it possible to customize this behaviour for every query the user performs?
I want to use Dynamic Data together with LINQ to SQL but if this task would much easier to accomplish using Entity Framework I would look into that too.
Thanks for your help and time.
Implementing such a solution in Dynamic Data it will probably require the creation of a custom Entity Template; not really easy but once done it will not require the creation of custom pages just the editing of the page templates.
I think it will be really usefull to check the excellent work on DD done by S.J.Naughton and presented on his blog.
Greetings, F.
You should not use dynamic data because you need full control over querying and manually write all linq queries to add your data level security. If you still insist on dynamic data be aware that you will still write most of pages yourselves and you will only use dynamic templates. You will have to manually define ever data source and correctly pass where condition to filter results based on logged user.
In addition linq-to-sql is not able to hide junction table and entity framework is able to do that only if junction table contains just two FKs for many-to-many relation. If this table contains any other column you want to use in the application you will have to map it as any other entity and dynamic data will show it as an entity.
Dynamic data are technology for quick creation of simple application where you need to provide access to database through web interface but what you describe is not a simple scenario. You need per record authorization which can differ among entity types.
I am facing this question in a new little project:
The system to be built will allow user to add new columns to a table in the system, and then the user will be able to maintain the data, I think there is two ways to implement this:
1) create a few tables including "columns" table with "columnName" "columnValue" "datatype" etc to store the column definition, aonther table "XXCoumn" to store the value of the column (entered by user), and user a store procedure to query/update column data.
2) create the column in the table schema when user enter a new column, then the maintain of the table data is just as normal
which way do you guys reckon? or any new suggestion?
Some additional info: the data volumn is small, and I need to create reports.
Any good recommendations would require a much better understanding of your requirements, but here are some comments on the options you mentioned, as well as some additional thoughts.
1) Entity-Attribute-Value (EAV) Design: This is the option you describe where you have a table that has columns for ColumnName, Type and Value. This option has the advantage of being able to accomodate unlimited new columns easily, but I have found it to be painful when the time comes to retrieve meaningful data back. For example, say you have rows in this EAV table for {Color, varchar}{Red, Green, Blue}, and {Size, varchar}{Small, Medium, Large}. If you want to find all the small green items, you need something like this (untested SQL of course):
SELECT *
FROM ITEMS
WHERE ITEMID IN (SELECT ITEMID
FROM ITEM_ATTRIBUTES ATT INNER JOIN ITEM_VALUES VLS
ON ATT.AttributeID = VLS.AttributeID
WHERE ATT.ColumnName = 'Color' AND VLS.Value = 'Green')
AND ITEMID IN (SELECT ITEMID
FROM ITEM_ATTRIBUTES ATT INNER JOIN ITEM_VALUES VLS
ON ATT.AttributeID = VLS.AttributeID
WHERE ATT.ColumnName = 'Size' AND VLS.Value = 'Small')
Contrast this with having actual columns on the items table for color and size:
SELECT *
FROM ITEMS
WHERE COLOR = 'Green' AND SIZE = 'Small'
In addition, you will have a difficult time maintaining data integrity, if that is important for this app (and it is almost always important, even when you are told otherwise). In the example above, you will need to implement extra logic if "Color" should be limited to Blue, Green, and Red. Also, you will need to implement even more logic if certain colors only come in certain sizes (example - blue items are only available in small and medium)
2) User-Defined Columns: Just giving the user the ability to add additional columns to the table has the advantage of making data retrieval simpler, but all the data integrity issues remain. Also, your app usually requires extra logic to deal with the unknown columns.
3) Pre-Existing Custom Columns: I have worked with a few apps, such as CRMs, that provide a dozen or more columns already in place for user definition. Basically, the designers put in columns like "Text1","Text2","Text3","Number1","Number2", etc. The users then provide header and description information for these columns, and that is what the app uses for display purposes. This model has the advantage of easy data retrieval, as well as a pre-defined DB schema which should simplify app logic. Data integrity issues remain, however. The other obvious downside is that you will run out of pre-defined columns, which is what you are usually trying to avoid with this type of solution.
As with most design issues, there are tradeoffs to each solution. My experience has been that while many users/clients say they want solutions like these, in reality they are simply trying to ensure they don't get trapped with an app that can't grow with their needs. I have found that there are actually very few places where a design like this is needed. I can almost always create a design that addresses the expansion desires of the client without putting them into the role of database designer.
"The system to be built will allow user to add new columns to a table in the system..."
Really - that's the user story? Sounds like you've already made up your mind on the solution, to me.
Whether it's a good idea or not to allow a user to extend schemas is pretty context dependent. I'd have little problem in an admin-like, limited use way. But it'd be a horribly bad idea in a MySpace type way. I suspect your situation lies somewhere between those 2 extremes.
Extending the schema would lead to greatly more efficient queries - as you could add indexes and such - but it does expose some relational rules on your users. Also, the extension would (probably) lock the entire table and concurrent edits would need to be dealt with.
If this is centrally hosted by you, I would suggest NOT allowing user-input data to change the schema of the database (i.e. drive the creation of new tables).
Rather you may want to look into using XML fields in SQL to store variable field names of data, or a more generic table structure... this technique works pretty well if we're not talking crazy amounts of data...
Is it possible you're looking at your solution sideways? It sounds like you need a mapping table (sort of like your #1). You have a table, say "objects" for example, a table called "properties" which holds what you're calling columns and then a table that holds the values, so it just has object_id, property_id, value.
To put in a smarter way than I said it, take a look at the Entity-attribute-value model.