I want to add a search engine to my website. I want it to handler boolean searches and give me a list of results in order or best match. I need it to be able to work with LINQ, because I want to add additional where clauses to the final query that gets run. I am looking for the best open source .NET search engine that works with LINQ. I like lucene.net but the problem is the LINQ interface (LINQ to Lucene) hasn't been updated since 2008. Are there any good options out there?
You could try and use the free Search Server Express from Microsoft. It's available in beta for the 2010 version but will be released soon. The (SharePoint) Search API is very similar similar to SQL, so you could append additional where clauses.
It's not linq or open source, but it's free and might work in your case. I've looked a bit at the lucene linq api myself, and came to the same conclusion you have. It's not updated, while Lucene is still being worked on.
The other option is to create your own Lucene Linq provider, but it will require some work.
Documentation for the FullTextSqlQuery class. (old version docs with sample here)
Here's a code snippet to show what it looks like:
FullTextSqlQuery fullTextSqlQuery = new FullTextSqlQuery(site)
fullTextSqlQuery.QueryText = String.Format("SELECT Title, SiteName, Path FROM Scope() WHERE \"scope\"='All Sites' AND CONTAINS('\"{0}\"')", searchPhrase),
and you could append more to the WHERE part of the query.
I decided to use full-text indexing feature of sql server. It's not as full featured as lucene.net but for my requirements it gets the job done pretty well.
Related
I have a task to optimize search engine in asp.net ecommerce store based on nopcommerce tempate.
I would like to hear on what should I pay most attention to improve the search engine and to deliver faster results, since current search engine is taking forever to display results.
Full Text Search is one of the options to be implemented too.
Thanks in advance, Laziale
Make sure that all search queries are thru database
Make sure that all the search fields have the proper indecies
Return as minimum info as needed (probably create stored procedures)
Look at your search queries, perhaps they can be rewritten an optimized
Profile your .net code and find the place where it slow and optimize it
Cache your results or even sql queries
For FULL TEXT SEARCH look at Lucene.NET
Skip the EF and have your own data layer, at least for the purpose of Search Optimization.
I think the best way will be reading this document provided by Google which tells you what are the most important tweaks you should pay attention to, i used it myself and was very rewarded indeed:
http://static.googleusercontent.com/external_content/untrusted_dlcp/www.google.com/en//webmasters/docs/search-engine-optimization-starter-guide.pdf
I am currently testing Telerik Reporting to replace our current report solution which is some kind of scripting tool that pulls the localized version of a term from the database.
Example:
A label called "customer" would have a english definition (Customer) and a french one (Client).
We get the correct definition from the database by calling a function like GetLabel("customer",user_language).
Is there a way to do the same thing except for including each "label" separately in the SQL query of the datasource?
We plan to use Telerik Reporting in an ASP.NET environment.
I found a way of doing this quite easily.
I only had to use the User Functions that Telerik Reporting already support and then return the correct value.
I am tinkering with Freebase, and trying things with the query editor and everything looks great. I'm still reading the fine manual, but can't seem to get the point if this may be used as a web search replacement for showing refined data to the user. The main question is:
If {q1, q2, q3 ....} be the query the user submits - how do I programatically map each query term to the freebase query key:value pair?
I am not sure if it can entirely replace the current search engines as of now.I have just written a blog post for handling a basic query(more will follow) in C# for freebase.
http://2guysfrommumbai.wordpress.com/
If you like the java stuff you can go to,
https://github.com/narphorium/freebase-java-api
this is a more complete api,and i have used it with good success.
You can append multiple queries using q1,q2,q3 as parameters,more details are available on the freebase developer site.
Hope this helps.
I'm currently looking at a terrible legacy ColdFusion app written with very few stored procedures and lots of nasty inline SQL statements (it has a similarly bad database too).
Does anyone know of any app which could be used to search the files of the app picking out any SQL statements and listing the tables/stored procedures which are referenced?
Dreamweaver will allow you to search the code of the entire site. If the site is setup properly including the RDS password and provide a data source it can tell you a lot of information. I've only set it up once so I can't remember exactly what information it gives you, I think maybe just the DB structure. Application window > databases. Even if it isn't set up properly just searching for "cfquery" will quickly find all your queries.
You could also write a CF script using CFDirectory/CFFile to loop the .cfm files and parse everything between cfquery and /cfquery tags.
CFBuilder may have some features like that but I'm not to familiar with it yet.
edit I've heard that CFBuilder can't natively find all your cfqueries that don't have cfqueryparam but you can use CF to extend CFB to do so. I imagine you could find/write something for CFB to help you with your problem.
another edit
I know it isn't indexing the contents of the query, but you can use regex to search using the editor as well. searching for <cfquery.+(select|insert|update|delete) checking the regex box should find the queries that aren't using cfstoredProc (be sure to uncheck the match case option if there is one). I know Dreamweaver and Eclipse can both search for Regex.
HTH
As mentioned above I would try a grep with a regex looking for
"<cfquery*" "</cfquery>" and "<cfstoredproc*" "</cfstoredproc>"
In addition if you have tests that have good code coverage or even just feel like the app is fully exercised in production you could try turning on "Log Database Calls" in Admin - > Datasources or maybe even at the JDBC driver level, just monitor performance to make sure it does not slow the site down unacceptably.
In short: no. You'd have to do alot of tricky parsing to make sure you get all the SQL. And because you can glob SQL together from lots of strings, you'll almost always miss some of it.
The best you're likely to do will be a case insensitive grep for "SELECT|INSERT|UPDATE|DELETE" and then manually pulling out the table names.
Depending on how the code is structured, you might be able to get the table names by regexing the SQL from clause. But that's not foolproof. Alot of people use string concatenation to build SQL statements. This is bad because it can introduce SQL injection attacks, and it also make this particular problem harder.
I have an SQL database with multiple tables, and I am working on creating a searching feature. Other than having multiple queries for the different tables, is there a different way to go about said searching function?
I should probably add that a lot of my content is database driven to make upkeep easier. Lucene will not work for this, correct?
Different approaches to consider:
1) Multiple queries pre-baked, like you described.
2) Dynamic sql that you put together on the fly based on user-entered criteria.
3) If text is involved, based on SQL Server full text search or Lucene.
In my open source app BugTracker.NET, I do both 2 and 3 (using Lucene.NET).
I documented how I use Lucene.NET here:
http://www.ifdefined.com/blog/post/2009/02/Full-Text-Search-in-ASPNET-using-LuceneNET.aspx
Since you have tagged the question with Asp.net I suppose you want to search your webpages. In that case you can use Indexing Server to perform freetext searches easily that search the generated html and any keywords you have set up.
As Corey Trager suggested, using Lucene.NET is also an option. It has a good reputation of being fast and quite easy to use.
Although the other answers provide good suggestions such as using Lucene, I have much preferred using a custom caching method.
So for a website that I help create, we cached the searchable data every couple of hours, from many tables, into one simple table with columns such as:
URL
Item/Page Name
Main Keywords
Text Only Contents
Date Updated
I would then write my SQL statement to search this field using different functions to determin the rank.
You might want to check out this post i wrote on writing full text queries, its in C#, but its easilly portable, or just stick it in a library and use it as it.
How to build an SQL full text index search term in c#