We have built an ASP.NET application (with C#.net language) and hosted on Windows Server 2003 Operating System.
The database is SQL Server 2008.
Now we need to do a search involving 4 tables and one of the column of table is varchar (2400) and the other columns are of normal lengths (E.g. Varchar(50) etc.). This search gets fired whenever an user enters a keyword in the search box and presses enter key. The number of searchable records in future could be in lakhs.
Would you please let me know the best method of searching these tables?
If would be great if you could provide a reference document or web reference (If needed).
Many Thanks,
Regards,
Venkat
To search text columns you should use Full-text search
More resources:
Understanding SQL Server Full-Text Indexing, SQL Server Full Text Search: Language Features and Full Text Search Step by Step Tutorial.
However it depends on the type of queries you make. Regular search using LIKE can be also acceptable.
Related
I am building a web application in ASP.NET 4.5 C#. My app will be heavily relied on searching capabilities. I am searching for a high-performing searching solution for my project. I am looking at one that is efficient / fast and easy to implement.
After going over a few solution, I found ElasticSearch to offer excellent searching capabilities, as well it's supported by Amazon Web Services if I decide to deploy my project there.
However, I've read that MySQL 5.6 now features Full Text Search foor InnoDB tables. I wanted to know the difference, cons and pros of each approach in order to decide which one to use in my next project.
My main goal: being able to make very fast searches against my database and enjoy the benefits of natural language search, scoring, etc. I will actually have very small text fields of 100 characters for my searchable text fields.
Summary of my questions:
What is the foundemental differences between MySQL 5.6 Full Text Search and ElasticSearch? (advantages and disadvantages)
With MySQL Full Text Search, do I need to separate the searching index / functionality from the database (for example, with ElasticSearch I am using the ELasticSearch server which I can deploy on a different server. I wonder if that works the same with MySQL Full Text Search too?
Which one is easier to implement?
Is there any advantages of using either in ASP.NET project (like supported client libraries, etc.)
Thanks.
I'm not very familiar with MySQL 5.6 Full Text Search capabilities, but I'm using ElasticSearch in my ASP.NET Web App.
MySQL is queried in SQL, ES is queried in JSON, as it is RESTful
While MySQL is database management system itself, ElasticSearch is only a search engine. The data on which search is performed is stored in it's index (ElasticSearch data store is called index). This index sometimes likes to fail, and then you need to have your data backuped in some external database (I'm using NoSQL solution - MongoDB, as I struggle with BigData problem), from which you restore it.
ElasticSearch is distributed system. It uses Shards - when you perform search, the index is splitted into number of shards, which are searched independently by separate threads. Then search results are consolidated into one set. As scoring is calculated in some way based on number of records on which search is performed, it may be different for the same record depending on which shard it was and how many records were assigned to this shard (although it is configurable in search query options - see DFSThenFetch)
ES can be interacted with from C# code by using by NEST library, which allows you to map ES entities into C# classes and query them by LINQ, by lambda syntax (utilizing expressions more than predicates)
Querying ElasticSearch can be pretty challenging if your entities have many fields. In my case I had to implement a pretty complex Query builder, as there is very many ways of performing searching query on ElasticSearch - see http://www.elasticsearch.org/guide/reference/query-dsl/ (all those query types are implemented in NEST library). You query pretty differently for matching string, than filtering data, eg. GreaterThan or LessOEqual.
ES allows you to utilize fuzzy matching based on Levenshtein Distance, which is p retty nice when struggling with typos.
ElasticSearch is very, very fast. Where MS SQL Server searched my data (by simple query) for more than 1 minute, ES is doing it in less than 20 ms (even by using very complex query). It is distributed implementation of Lucene search engine.
I know my post is not straight answer to your questions, but I hope it provides you some info concerning ElasticSearch, and helps you in any way. Cheers.
I'm about to embark on the ASP.net project which involves building a pretty powerful search function. The application is very database heavy. Essentially, organisations will be adding a lot of metadata about themselves in the form of multi-selects, free text boxes etc. which are all stored in SQL 2008.
When it comes to search I'm loath to re-invent the wheel. Normally with a content driven site I'd use a component such as Zoom Search or ASP.net Search engine (http://www.aspnetsearchengine.com/UltimateSearch/Features.aspx)
But I don't think these type of content driven search controls are apppropriate for what I need given the data driven nature of the search.
I'm thinking full text search is the way to go but then I'm thinking I'll probably lose a lot of the bells and whistles I'd typically get with a packaged search module like spelling suggestions, document search, synonyms, ignore words etc.
Are there any good hybrid solutions (paid or free) for .net sites that provides these nice features within a search framework of sorts?
Thanks,
Ed
Lucene is pretty highly regarded across a number of languages. It's in use on some pretty large sites too, i know monster.com use it and their search is pretty extensive.
https://lucene.apache.org/
Edit Found some more resources:
Lucene.Net and SQL Server
SQL Server 2008 Full Text Search (FTS) versus Lucene.NET
http://ifdefined.com/blog/post/Full-Text-Search-in-ASPNET-using-LuceneNET.aspx
open source faceted search / guided navigation for ecommerce sites with .net apis
I'm writing a ASP.net web app where users will be able to add wordy descriptions to a database table and was just wondering if there's some sort of (free) .net search plugin I could use to search through the database. I could write a simple SQL query to do it but I'd rather it be more robust and like a google search.
If your database is SQL Server, then you use SQL Server full text search. It's right there, and has google-like semantics.
you are probably looking for lucene.net which a port of lucene. Provide search capabilities for arbitrary document (so you can also index database text with it).
But I think you would rather index the html renders of your database entries (which lucene does beautifully).
I'm deploying a web site and I need to run large TSQL scripts contained in a single file in a production server where I don't have full access to SQL Server console and I can't connect remotely. The scripts is a mixed of table, stored procedures and views creations. All I can do is to run 1 group of TSQL sentences, like the ones for a stored procedure.
I have two options: to parse the file manually looking for GO's sentences and run each block of sentences before that GO, or to do the same task but with a tool. Using a tool I will be very fast doing the task, but I don't know any tool such that.
Do you know any tool that I can use?
I think it must be something like a control, with an editor where I will paste or load the scripts to run, and it will be able to parse and run them in sequence, like the Microsoft SQL Server Management usually does.
You could check out some articles on CodeProject on the topic and maybe use one of those tools / component for your needs?
Universal Database Admin for ASP.NET and SQL Server (Reloaded)
ASP.NET Database Admin Control
Web SQL Utility
Most of those come with full source and could also serve you as a starting point for a custom version of your own.
Marc
you got many options
usually i create stored procedure in sql management studio then save it as string, then use linq2sql to execute the stored procedure, works very well.
use sql server smo object where you can really do mostly many things like creating DBs, tables, it is really cool, i create 1 page on my site and use it to update DB with it.
here is a good link for that
http://www.sqldbatips.com/showarticle.asp?ID=34
you can use Redgate SQL data compare, and Compare. they rock really to synchronize DBs, it have saved me a lot of time and it is super easy, highly recommended really.
hope this helps.
If those don't work, it's pretty easy to write such a tool yourself. Just use Regex.Split to split the text on the GO lines, then loop, calling ExecuteNonQuery for each section.
when I am querying the local database instead of live database I get the following error:
Cannot use a CONTAINS or FREETEXT
predicate on table or indexed view
'Shop' because it is not full-text
indexed.
Why is this happening?
It means pretty much what it says; Full-text indexing isn't turns on for your Shop table. Full-text indexing allows all the text within a field to be searched while querying, which is what I assume you want to do.
You probably have full-text indexing turned off on your development system, but on on your live server.
Please read this page which explains how to enable it.