As I explained in my post a few days before I'm programming an ASP MVC3 multi-language website which should contain facetted search, full text search and a distance search. To realize that I've installed solr 3.3 on a Tomcat 7. I'm also successfully integrated a dataimporthandler.
Now I want to index the data from my relational ms sql database. I read the index structure looks like one table containing all the data of one object. That means if I've got a object like a car my schema catains fields like Branding, Color and so on.
But what about n-m realtions? Does the index "table" have one column for each relation?
And what about multi language items? Should I create one object/row int the index for each language?
And should I save just the id of objects in the index or the whole names?
And last how to index (query) a Object like on the database image? (I read something about dynamic fields and multiplevalue fields but I'm not sure if it is the solution for my problem)
I've a example of a database design I'm talking about attached.
Thanks for all the answers!!!
Update:
The people should be able to have different way to search.
They should have the possibility to search the tbl_text_local.text by full text searching and the miscellaneous are are facettes.
The Result should be a list of objects that match to the search and a list of facetts.
But how should I index the Miscellaneous? Is there a posibility to index them in a form like that:
<cattegory name = "cat1">
<Miscellaneous>
name...
</Miscellaneous>
<Miscellaneous>
...
</Miscellaneous>
<Miscellaneous>
...
</Miscellaneous>
</cattegory>
<cattegory name = "cat2">
<Miscellaneous>
</Miscellaneous>
<Miscellaneous>
</Miscellaneous>
<Miscellaneous>
</Miscellaneous>
</cattegory>
People should have a searchfield like:
Text input (to search the text)
Facettes:
Miscellaneous-Cattegory1
Miscellaneous1 (9)
Miscellaneous2 (39)
Miscellaneous3 (49)
Miscellaneous-Cattegory2
Miscellaneous5 (59)
Miscellaneous6 (69)
Miscellaneous-Cattegory3
Miscellaneous7 (7)
Miscellaneous8 (8)
Miscellaneous-Cattegory4
Miscellaneous9 (19)
There is no single, "best" way to model relationships in Solr. Unlike relational databases, where you design tables by following normalization, in Solr the schema design is very much ad-hoc, a function of the searches you will perform on the index. Ask yourself these questions as guidance:
What are users searching for? What is the "result type"? The schema should be designed around this.
What information do I need to facet?
What information do I need to include in full-text search?
What information do I need to use to sort results?
What information will I search by? I.e. what information will I use to filter search results, and how will I use that information?
What will I process at index-time and what will I process at query-time?
Finally, don't be afraid of duplicating data in the index for specific search purposes.
Related
I wanted to fetch the document which have the particular element attribute value.
So, I tried the cts:element-attribute-value-query but I didn't get any result. But the same element attribute value, I am able to get using cts:element-attribute-range-query.
Here the sample snippet used.
let $s-query := cts:element-attribute-range-query(xs:QName("tit:title"),xs:QName("name"),"=",
"SampleTitle",
("collation=http://marklogic.com/collation/codepoint"))
let $s-query := cts:element-attribute-value-query(xs:QName("tit:title"),xs:QName("name"),
"SampleTitle",
())
return cts:search(fn:doc(),($s-query))
The problem with range-query is it needs the range index. I have hundreds of DB's in multiple hosts. I need to create range indexes on each DB.
What could be the problem with attribute-value-query?
I found the issue with a couple of research.
Actually the result document is a french language document. It has the structure as follows. This is a sample.
<doc xml:lang="fr:CA" xmlns:tit="title">
<tit:title name="SampleTitle"/>
</doc>
The cts:element-attribute-value-query is a language dependent query. To get the french language results, then language needs to be mentioned in the option as follows.
cts:element-attribute-value-query(xs:QName("tit:title"),xs:QName("name"), "SampleTitle",("lang=fr"))
But cts:element-attribute-range-query don't require the language option.
Thanks for the effort.
We have a db driven asp.net /sql server website and would like to investigate how we can allow users to create a new database category and fields - is this crazy?. Is there any examples of such organic websites out there - the fact that I havent seen any maybe suggest i am?
Interested in the best approach which would allow some level of control by Admin.
I've implemented things along these lines with a dictionary table, rather than a more traditional table.
The dictionary table might look something like this:
create table tblDictionary
(id uniqueidentifier, --Surrogate Key (PK)
itemid uniqueidentifier, --Think PK in a traditional database
colmn uniqueidentifier, --Think "column name" in a traditional database
value nvarchar, --Can hold either string or number
sortby integer) --Sorting columns may or may not be needed.
So, then, what would have been one row in a traditional table would become multiple rows:
Traditional Way (of course I'm not making up GUIDs):
ID Type Make Model Year Color
1 Car Ford Festiva 2010 Lime
...would become multiple rows in the dictionary:
ID ITEMID COLUMN VALUE
0 1 Type Car
1 1 CarMake Ford
2 1 CarModel Festiva
3 1 CarYear 2010
4 1 CarColor Lime
Your GUI can search for all records where itemid=1 and get all of the columns it needs.
Or it can search for all records where itemid in (select itemid from tblDictionary where column='Type' and value='Car' to get all columns for all cars.
In theory, you can put the user-defined types into the same table (Type='Type') as well as the user-defined columns that that Type has (Type='Column', Column='ColumnName'). This is where the sortby column comes into it - to help build the the GUI in the correct order, if you don't want to rely on something else.
A number of times, though, I have felt that storing the user-defined dictionary elements in the dictionary was a bit too much drinking-the-kool-aid. Those can be separate tables because you already know what structure they need at design time. :)
This method will never have the speed or quality of reporting that a traditional table would have. Those generally require the developer to have pre-knowledge of the structures. But if the requirement is flexibility, this can do the job.
Often enough, what starts out as a user-defined area of my sites has had a later project to normalize the data for reporting, etc. But this allows users to get started in a limited way and work out their requirements before engaging the developers.
After all that, I just want to mention a few more options which may or may not work for you:
If you have SharePoint, users already have the ability to create
their own lists in this way.
Excel documents in a shared folder that are saved in such a way
to allow multiple simultaneous edits would also serve the purpose.
Excel documents, stored on the webserver and accessed via ODBC
would also serve as single-table databases like this.
i'm using Drupal 6
I have this table relation and I've translated into CCK complete with it's relation.
Basically when I view a Period node, I have tabs to display ALL Faculty nodes combined with Presence Number.
here's the table diagram: http://i.stack.imgur.com/7Y5cU.png
Translated into CCK like these:
CCK Faculty (name),
CCK Period (desc,from,to) and
CCK Presence(node-reference-faculty, node-reference-period, presence_number)
Here's my simple manual SQL query that achieve this result: http://i.stack.imgur.com/oysd3.png
SELECT faculty.name, presence.presence_number FROM Faculty AS faculty
LEFT JOIN (SELECT * FROM Presence WHERE Period_id=1) AS presence ON faculty.id=presence.Faculty_id
The value of 1 for Period_id will be given by the Period Node ID from the url argument.
Now the hardest part, is simulating simple SQL query above into Views. How can I make such query into Views in Drupal-6 or Drupal-7 ?
thanks for any helps.
The main issue, which I think you've noticed, is that if you treat Faculty as the base for your join, then there is no way to join on the Presence nodes. Oppositely, if you treat Presence as the base, then you will not see faculties that have no presence number.
There is no easy way, using your currently defined structure, to do these joins in views.
I would say your easiest option is to remove the 'node-reference-faculty' field from the presence node and add a node-reference-presence field to the faculty. Since CCK fields can have multiple values, you can still have your one-to-many relationship properly.
The one downside of this is that then you need to manage the presence-faculty relationship from the faculty nodes instead of the presence nodes. If that's a show stopper, which it could be depending on your workflow, you could have BOTH node-reference fields, and use a module like http://drupal.org/project/backreference to keep them in sync.
Once you have your reference from faculty -> presence, you will need to add a relationship in Views. Just like adding a field or a filter, open the list of relationships and find the one for your node-reference field.
Next, you will need to add an argument for period id and set it up to use a node id from the url. The key thing is that when you add the argument, it will ask which relationship to use in its options. You will want to tell it to use your newly added presence relationship.
You don't really need to do a subquery in your SQL like that. This should be the same thing and won't make mysql try to create a temporary table. I mention it because you can't really do subqueries in Views unless you are writing a very custom Views handler, but in this case you don't really need the subquery anyway.
Ex.
SELECT f.name, p.presence_number
FROM Faculty AS f
LEFT JOIN Presence AS p ON f.id=p.Faculty_id
WHERE p.Period_id=1;
I wrote an article about how to achieve a similar outcome here. http://scottanderson.com.au/#joining-a-views-query-to-a-derived-table-or-subquery
Basically how to alter a Views query to left join on a sub-query.
I have three content types: Artist, Artwork, Exhibition. An Exhibition has a field 'artworks' (unlimited values). An Artwork has a field 'artist' (1 value, required).
And there is the relation I can't seem to find with Views: I want all Exhibitions an Artist ever participated in. Which means: on the Artist page, show all Exhibitions all Artworks of this Artist were ever in.
The problem (I think) is that one field (Exhibition.artworks) has many values. Yet Artwork.artist has just 1 value.
I don't know what the problem is =) but it's not working and I've tried a million things. At this point, I'll accept writing SQL queries, but the drupal content database is so incredibly untransparent that I have no idea what to query and how.
Obviously I'd be happiest with an unhacked Views solution, but I'm not getting my hopes up. Anyone experience with relations like this?
You can build dependent relationships that should help you to accomplish this. Use a relationship (Artwork) on exhibition.artworks and a relationship (Artist) on (Artwork).Artist
It would be easier to understand what you're doing with exports of the views & content types.
The database structure for content types in Drupal works as follows;
The node is the base table, with nid as index. Your content types have their own tables, content_type_XXXXXX with all single entry fields (that aren't shared among content types) members of that table. Multiple entry and shared fields get their own table content_field_XXXXXX. All of the tables relate on the nid field, and multiple entry fields use a "delta" to indicate the entry order.
I'm looking to create a table for user preferences and can't figure the best way to do it. The way that the ASP.NET does it by default seems extremely awkward, and would like to avoid that. Currently, I'm using one row per user, where I have a different column for each user preference (not normalized, I know).
So, the other idea that I had come up with was to split the Preferences themselves up into their own table, and then have a row PER preference PER user in a user preferences table; however, this would mean each preference would need to be the exact same datatype, which also doesn't sound too appealing to me.
So, my question is: What is the best/most logical way to design a database to hold user preference values?
Some of the ideas that I try to avoid in database work, is data duplication and unnecessary complication. You also want to avoid "insert, update, and deletion anomalies". Having said that, storing user preferences in one table with each row = one user and the columns, the different preferences that are available, makes sense.
Now if you can see these preferences being used in any other form or fashion in your database, like multiple objects (not just users) using the same preferences, then you'll want to go down your second route and reference the preferences with FK/PK pairs.
As for what you've described I see no reason why the first route won't work.
I usually do this:
Users table (user_id, .... etc.)
.
Options table (option_id, data_type, ... etc.)
(list of things that can be set by user)
.
Preferences table (user_id, option_id, setting)
I use the new SQLVARIANT data type for the setting field so it can be different data types and record the data type of the option as part of the option definition in the Options table for casting it back to the right type when queried.
If you store all your user preferences in a single row of a User table you will have a maintenance nightmare!
Use one row per preference, per user and store the preference value as a varchar (length 255 say, or some value large enough to meet your requirements). You will have to convert values in/out of this column obviously.
The only situation where this won't work easily is if you want to store some large binary data as a User preference, but I have not found that to be a common requirement.
Real quick, one method:
User(UserID, UserName, ...)
PreferenceDataType(PreferenceDataTypeID, PreferenceDataTypeName)
PreferenceDataValue(PreferenceDataValueID, PreferenceDataTypeID, IntValue, VarcharValue, BitValue, ...)
Preference(PreferenceID, PreferenceDataTypeID, PreferenceName, ...)
UserHasPreference(UserID, PreferenceID, PreferenceDataValueID)