I'm about to embark on the ASP.net project which involves building a pretty powerful search function. The application is very database heavy. Essentially, organisations will be adding a lot of metadata about themselves in the form of multi-selects, free text boxes etc. which are all stored in SQL 2008.
When it comes to search I'm loath to re-invent the wheel. Normally with a content driven site I'd use a component such as Zoom Search or ASP.net Search engine (http://www.aspnetsearchengine.com/UltimateSearch/Features.aspx)
But I don't think these type of content driven search controls are apppropriate for what I need given the data driven nature of the search.
I'm thinking full text search is the way to go but then I'm thinking I'll probably lose a lot of the bells and whistles I'd typically get with a packaged search module like spelling suggestions, document search, synonyms, ignore words etc.
Are there any good hybrid solutions (paid or free) for .net sites that provides these nice features within a search framework of sorts?
Thanks,
Ed
Lucene is pretty highly regarded across a number of languages. It's in use on some pretty large sites too, i know monster.com use it and their search is pretty extensive.
https://lucene.apache.org/
Edit Found some more resources:
Lucene.Net and SQL Server
SQL Server 2008 Full Text Search (FTS) versus Lucene.NET
http://ifdefined.com/blog/post/Full-Text-Search-in-ASPNET-using-LuceneNET.aspx
open source faceted search / guided navigation for ecommerce sites with .net apis
Related
I am taking part in Imagine Cup and got only three months for building a site in ASP.NET. Is it possible to learn enough to build a working site containing an cms, login system in a month. Its not that I am beginning web development, I already know a little asp.net and know AJAX, JAVASCRIPT, CSS, HTML, SQL but have never worked with databases. I was never into web development but started because of Imagine cup. In 2 months I already know HTML, CSS, JAVASCRIPT, HTML and AJAX but I am worried about the complexity of server-side scripting and ask for the best way to get me working in ASP.NET in a month. I know c#.
Although you can't be an expert in a month, you can get up and running fairly quickly with webforms. Since it already includes a login system, you can get that up and running quickly.
As far as CMS, it depends what you need. An advanced CMS system is a major undertaking.
Getting going with ASP.NET
Actually, if you want to re-apply everything you know about creating JS enabled static html sites, you'd have a smoother path using MVC because you don't have to guess what the tag id's are, you don't have to guess what kind of HTML a server side control is going to create, etc.
That said, Jonathan Wood is correct, to just create a page with labels and buttons, Web Forms has fewer concepts you need to learn, but you will find it harder to apply the same techniques that work well with JS + static html.
Data bases
If you use SQL Compact, then it is easier to get started because you needn't learn to mess with the numerous things going on with SQL Server (like a security model appropriate for big companies, but not for learners). Another easy alternative is MS-Access because the "Query by Example (QBE)" query designer is still better than anything that the Visual Studio or SSMS teams ever put together.
Keep in mind that if you use MS-Access, while it will be really easy to learn (more so than SQL Compact), MS-Access uses Jet SQL and has many other differences from SQL Server, where as everything you learn about SQL Compact will apply to SQL Server.
Not sure what the conditions etc of Imagine cup are, but I strongly recommend to use an existing CMS (in particular Umbraco) instead of developing all from scratch. Some advantages are:
No need to dive into SQL and databases right away
Building blocks are available (login system) and leverage existing ASP.NET technologies
Razor can be used instead of xslt which will be a skill you'll be able to apply to pure ASP.NET MVC development later on
Very friendly and helpful community if you need some help
Starter kits available - basic web sites can be created easily
By diving into any detail of the CMS, you'll learn a lot (source code available)
You could probably pick up the basics with a few good books. Sam's, Wrox, APress and O'Reilly are great. You can also check http://www.asp.net/get-started for Microsoft's official tutorials.
You can also download the express versions of Visual Studio 2010 and SQL Server. If you want to continue on after the competition I'd strongly recommend you learn database programming on SQL Server 2010 Express. (It'll help you down the road)
Good luck with it.
We host websites in a shared hosting environment where Microsoft SQL Server full text searching is not allowed. We would love an ASP.NET API that allowed similar functionality to get around this restriction.
We can't easily install software on the shared servers, so the API would have to be written in ASP.NET.
SQL "like" queries are our alternative and they are fast enough (our websites never exceed more than 50Mb of text) but they don't rank results well, have a dictionary, do stemming etc
For this type of circumstance I'd rely on Google and create a proper sitemap. You can integrate google search right into your website too with Google SiteSearch.
If you need more control over full-text search, you can use features of the RDBMS to support this. You don't say which brand of RDBMS you're using. I assume it's likely Microsoft SQL Server if you're using ASP.NET.
See the docs for Full-Text Search at MSDN.
For other brands of RDBMS, see my answer: How best to develop the sql to support Search functionality in a web application?
Lucene is what we were looking for http://incubator.apache.org/lucene.net/
I have no experience building a search solution, but I'd like to have a search box within my solution and I don't know where to even begin. Are there cool SQL Server tricks that I can use to make my search solution performant (I'm using a hosted SQL 2008 server) I'd love pointers to a multi-step tutorial that starts me off with a simple query search solution...and then layers on more advanced code and features.
You don't actually say whether you need/want a 'spider' to index your site "as is" (like Google; which is useful if your searchable content on each page comes from many different tables/objects/entities) or whether you just want to query EF using full-text-search-like syntax to return a collection of Entities?
If you are interesting in the 'spider' approach - here's a CodeProject article for a small ASP.NET Search Engine "Searcharoo". It is a web-crawling search engine for small-ish sites (it doesn't use a database at all), so it may not be applicable for your situation.
The code is also at searcharoo.codeplex.com and there are 7 articles on how it works/was built at Searcharoo.net (disclaimer: I wrote them; I hope they are interesting/useful).
If you need to search your database directly, you should probably look into SQL Server 2008's Full Text Search feature (assuming LIKE isn't sophisticated enough for your needs). We used info from this article (free registration) to set-up SQL Full Text Search on a work project... no EF in our solution though.
Also, as you might know StackOverflow is built with ASP.NET MVC - they blogged about some problems with SQL 2008 FTS. There's also some info on SQL FTS versus Lucene.NET (which is another search engine you could research) that might be useful.
You might be interested in reading this.
Read this article:
Create a Site Search Engine in ASP.NET
If you don't have to program an engine yourself you could consider using Google Custom Search Engine. There are couple of articles about this:
Using Google Co-op's Custom Search
Engine
Implementing Search in ASP.NET with Google Custom Search
Also could be useful:
Helping Visitors Search Your Site By Creating an OpenSearch Provider
Which is the best way to create a site search engine for a dynamic asp.net site with hundreds of dynamic pages. I have seen many products and articles
http://www.karamasoft.com/UltimateSearch/overview.aspx
http://www.sitesearchasp.net
http://www.easysearchasp.net/
http://msdn.microsoft.com/en-us/magazine/cc163355.aspx
http://www.codeproject.com/KB/asp/indexserver.aspx
Priyan,
Another high-quality open-source option would be the .NET port of Lucene
CodeProject - Introducing Lucene
dotlucene
lucene.net
You haven't mentioned Google's SiteSearch "product". Is one of your requirements that you'd like to host the search engine/catalog yourself?
Microsoft also has a product Search Server 2008 Express although I'm not sure if you can install it on any hosting provider.
And (disclaimer: I am the author) there is also a very basic open source project on CodeProject called Searcharoo (also at searcharoo.net). It is really meant as a 'demonstration/learning experience' - hence the six how to articles - but it might suffice for a small dynamic site.
I have used SQL Server Full Text Search for some projects - works well but it's really just searching database content, not a combination of static and dynamic Html/Pdf/Word/Jpg etc documents which a "real" web crawler will do.
What's your preferred method of providing a search facility on a website? Currently I prefer to use Lucene.net over Indexing Service / SQL Server full-text search (as there's nothing to set up server-side), but what other ways are being used out there?
We used both Lucene.net, Indexing Service and SQL Server full-text. For a project with large and heavy DB search functionality SQL search has an upper hand in terms of performance/resource hit. Otherwise Lucene is much better in all aspects.
Take a look at Solr. It uses Lucene for text indexing, but it is a full blown http server so you can post documents over http and do search using urls. The best part is that it gives you faceted searching out of the box which will require a lot of work if you do it yourself.
you could use google, it's not going to be the fastest indexer but it does provide great results when you have no budget.
dtSearch is one we've often used, but I'm not really that big a fan of it.
A lot of people are using Google's custom search these days; even a couple of banks that I know of use it for their intranet.
If you need to index all the pages of your site (not just the ones Google indexes) or if you want to create a search for your intranet web sites, the Google Mini is pretty sweet. It will cost you some money, but it is really easy to have it up and running within just a couple of hours. Depending on how many pages you need to index it can be expensive though.
I'm using dtSearch and I (kind of) like it. The API isn't the greatest in the world for .NET but it can get the job done and it's pretty fast. And it's cheap, so your boss will like it (~$1,000 US).
The results leave something to be desired as it doesn't do any kind of semantic relevance rankings or anything fancy. It does a better job than anything you can get out of MS SQL server though.
It has a web spider that makes it easy to do quick search apps on a website. If you need to you can use the API to create hooks into your database and to provide item level security - but you have to do the work yourself. Their forum leaves something to be desired as well but maybe people will start posting dtSearch stuff here. :)
Has anyone tried Microsoft search server express?
http://www.microsoft.com/enterprisesearch/serverproducts/searchserverexpress/default.aspx
I haven't tried it yet, but it could potentially be powerful.
From the site it looks primarily geared towards sharepoint users but given its sdk I don't see why you couldn't use it for a regular old site search
I also recommend SOLR. It's easy to set up, maintain, and configure. I've found it to be stable and easy to scale. There's a c# package for interfacing with solr.