What's your preferred method of providing a search facility on a website? Currently I prefer to use Lucene.net over Indexing Service / SQL Server full-text search (as there's nothing to set up server-side), but what other ways are being used out there?
We used both Lucene.net, Indexing Service and SQL Server full-text. For a project with large and heavy DB search functionality SQL search has an upper hand in terms of performance/resource hit. Otherwise Lucene is much better in all aspects.
Take a look at Solr. It uses Lucene for text indexing, but it is a full blown http server so you can post documents over http and do search using urls. The best part is that it gives you faceted searching out of the box which will require a lot of work if you do it yourself.
you could use google, it's not going to be the fastest indexer but it does provide great results when you have no budget.
dtSearch is one we've often used, but I'm not really that big a fan of it.
A lot of people are using Google's custom search these days; even a couple of banks that I know of use it for their intranet.
If you need to index all the pages of your site (not just the ones Google indexes) or if you want to create a search for your intranet web sites, the Google Mini is pretty sweet. It will cost you some money, but it is really easy to have it up and running within just a couple of hours. Depending on how many pages you need to index it can be expensive though.
I'm using dtSearch and I (kind of) like it. The API isn't the greatest in the world for .NET but it can get the job done and it's pretty fast. And it's cheap, so your boss will like it (~$1,000 US).
The results leave something to be desired as it doesn't do any kind of semantic relevance rankings or anything fancy. It does a better job than anything you can get out of MS SQL server though.
It has a web spider that makes it easy to do quick search apps on a website. If you need to you can use the API to create hooks into your database and to provide item level security - but you have to do the work yourself. Their forum leaves something to be desired as well but maybe people will start posting dtSearch stuff here. :)
Has anyone tried Microsoft search server express?
http://www.microsoft.com/enterprisesearch/serverproducts/searchserverexpress/default.aspx
I haven't tried it yet, but it could potentially be powerful.
From the site it looks primarily geared towards sharepoint users but given its sdk I don't see why you couldn't use it for a regular old site search
I also recommend SOLR. It's easy to set up, maintain, and configure. I've found it to be stable and easy to scale. There's a c# package for interfacing with solr.
Related
Asana looks like it is easy to use and extremely fast.
My company currently uses the slow and terrible Microsoft CRM, would it be possible to use Asana as the front end and send the inputted data to the MS CRM database?
I understand that a MS API is most likely needed as well, however does Asana have the capabilities?
The short answer is yes, it's possible to do this, both products have the capability to do what you describe. Asana's API is pretty new though, so you might have some initial limitations; but based on what I suspect you'd want to do, it should be adequate.
But unless you have an incredibly nimble and forward thinking IT group, I don't imagine they would want to build or support this. And if they were nimble and forward thinking, you probably wouldn't be using Microsoft CRM (or at least it wouldn't be slow).
Is there any good search tools for asp.net I can buy to carry out search indexes easily on data I have in my database?
What I require is something that would carry out a general site search of articles but also faceted search as well. Faceted search is quite important feature.
Thanks.
You can implement facets with Lucene.NET, Solr, Sphinx, FAST, Endeca, probably also Xapian.
None of these will be trivial to integrate though, since they use very different data structures from relational databases. Usually you have to denormalize your data to feed it to a full-text search engine.
SolrNet includes a sample ASP.NET MVC application that shows how to implement facets with Solr.
Disclaimer: I'm the author of SolrNet.
I agree with Mauricio Scheffer, using Solr.NET will help achieve what you want. I implemented a basic ASP.NET web form example around a year back. I found this link to be very useful to get me started: http://crazorsharp.blogspot.co.uk/2010/01/full-text-search-using-solr-lucene-and.html
Using the example (above), I managed to create this (DISCLAIMER: personal site): http://surinder.computing-studio.com/post/2011/01/14/At-Last!-Created-My-Own-eBay-Style-Search-Using-Solrnet.aspx
Lucene .net great http://incubator.apache.org/lucene.net/
Its a .net port of the very popular Java search library Lucene.
Now, I prefer having a stand along search engine to remove the strain from my site. I use Solr which is Java, but you can use SolrSharp http://solrsharp.codeplex.com/ or SolrNet http://code.google.com/p/solrnet/ to intergrate with solr.
I am looking for a very insecure ASP.NET application. Ideally i'm looking for an application that was written by a noob who has made a lot of security mistakes. If the app has a MS-SQL back-end that would be a bonus. I know of two cool projects for PHP and J2EE which fill my needs. Do you know anything like this for ASP.NET?
Hacme Bank is a pretty good example from Foundstone of what not to do. It's an older example, written in .NET 1.1 against SQL 2000, but covers most of the common vulnerabilities found in web applications.
I know one... there's a business near where I live, and I refuse to join their loyalty program because I accidentally stumbled across a SQL injection vulnerability on their site. I can't tell you who it is though, because they're a real business and I'm sure they don't want your students to hack them.
Why not write your own? Maybe start a codeplex project called LearnSecurity, which has a combination of pages - some which use bad security practices and some which don't.
Yes, as Rob suggested ... you could make a couple of pages ... on the first get the input from a field and execute it against the database :) On the second, just get some text from the db that contains javascript alert or something and render it directly on the page :) Just an idea ...
You can search the vulnerabilities databases, like NISTs one.
I search for a web crawler solution which can is mature enough and can be simply extended. I am interested in the following features... or possibility to extend the crawler to meet them:
partly just to read the feeds of several sites
to scrape the content of these sites
if the site has an archive I would like to crawl and index it as well
the crawler should be capable to explore part of the Web for me and it should be able to decide which sites matches the given criteria
should be able to notify me, if things possibly matching my interest were found
the crawler should not kill the servers by attacking it by too many requests, it should be smart doing crawling
the crawler should be robust against freak sites and servers
Those things above can be done one by one without any big effort, but I am interested in any solution which provide a customisable, extendible crawler. I heard of Apache Nutch, but very unsure about the project so far. Do you have experiences with it? Can you recommend alternatives?
I've used Nutch extensively, when I was building the open source project index for my Krugle startup. It's hard to customize, being a fairly monolithic design. There is a plug-in architecture, but the interaction between plug-ins and the system is tricky and fragile.
As a result of that experience, and needing something with more flexibility, I started the Bixo project - a web mining toolkit. http://openbixo.org.
Whether it's right for you depends on the weighting of factors such as:
How much flexibility you need (+)
How mature it should be (-)
Whether you need the ability to scale (+)
If you're comfortable with Java/Hadoop (+)
A quick search on GitHub threw up Anemone, a web spider framework which seems to fit your requirements - particularly extensiblility. Written in Ruby.
Hope it goes well!
I heartily recommend heritrix. It is VERY flexible and I'd argue is the most battle tested freely available open source crawler, as it's the one the Internet Archive uses.
You should be able to find something that fits your needs here.
I have no experience building a search solution, but I'd like to have a search box within my solution and I don't know where to even begin. Are there cool SQL Server tricks that I can use to make my search solution performant (I'm using a hosted SQL 2008 server) I'd love pointers to a multi-step tutorial that starts me off with a simple query search solution...and then layers on more advanced code and features.
You don't actually say whether you need/want a 'spider' to index your site "as is" (like Google; which is useful if your searchable content on each page comes from many different tables/objects/entities) or whether you just want to query EF using full-text-search-like syntax to return a collection of Entities?
If you are interesting in the 'spider' approach - here's a CodeProject article for a small ASP.NET Search Engine "Searcharoo". It is a web-crawling search engine for small-ish sites (it doesn't use a database at all), so it may not be applicable for your situation.
The code is also at searcharoo.codeplex.com and there are 7 articles on how it works/was built at Searcharoo.net (disclaimer: I wrote them; I hope they are interesting/useful).
If you need to search your database directly, you should probably look into SQL Server 2008's Full Text Search feature (assuming LIKE isn't sophisticated enough for your needs). We used info from this article (free registration) to set-up SQL Full Text Search on a work project... no EF in our solution though.
Also, as you might know StackOverflow is built with ASP.NET MVC - they blogged about some problems with SQL 2008 FTS. There's also some info on SQL FTS versus Lucene.NET (which is another search engine you could research) that might be useful.
You might be interested in reading this.
Read this article:
Create a Site Search Engine in ASP.NET
If you don't have to program an engine yourself you could consider using Google Custom Search Engine. There are couple of articles about this:
Using Google Co-op's Custom Search
Engine
Implementing Search in ASP.NET with Google Custom Search
Also could be useful:
Helping Visitors Search Your Site By Creating an OpenSearch Provider