Advice to implement a search for a big table - asp.net

I've created a table in asp.net (with Gridview) and it's expected to have lots of rows. There's also pagination and because I didn't want the page to refresh every time the user use it, everything go through Javascript.
Now I want to implement a search option with regex support but I'm not sure what the best way is to do it. I need the search on all the rows (not only the rows of the current page).
I'm not sure if it's possible to use regex in an sql query (I searched a bit on Google but didn't really find anything recent that helped me). The other option would be to query all the rows and filter them on the server but I'm afraid it will slow things down when we have like 2000 rows.
What would you do? Is there another option I didn't think of?

Related

Refine search results on entering location using ajax?

I have this textbox in asp.net webform page used to enter a city. On entering some text it provides suggestions just like facebook does of matching results.
I tried these two methods to implement this.
I first used onTextChanged event and AJAX and found out it only works when the textbox loses focus. I wanted a solution to work as you type. Advantage of using this was that I could use a database and it would be fast, because no xml files will be transferred in the process.
2.I used ajax, clientside using js. But the problem is the xml containing cities, there states, country is a massive 30MB file. So, it was impossible to use it, so thought of making 26 small xml files of each alphabet out of that big one but still they would be big enough to actually use. So, now I am planning to use 26*26 files containing the cities with same first two alphabets but I think its ineffective way to do what I want.
Is there any other efficient way of accomplishing it?
The best way would be to use a database, if I could.
You need to use onkeypress and/or onkeyup events instead.
Did you know that there are plug-and-play auto-complete components out there that are free? For example http://jqueryui.com/demos/autocomplete/
Use JSON! It's much more compact. You'll probably save 30-40% on the size of that data.
Did you know that you don't need to pass the whole data set for that to work? You can have it live on the server (e.g. in the database, or cached on the webserver for faster access and less db traffic), and have clients only pull small set of data at a time, based on characters that they type. That JQuery UI AutoComplete supports that feature.
If you cannot use JQuery and JQuery UI (not wanting would be an unacceptable answer), then I'm pretty sure there are other free alternatives, including this one: http://www.asp.net/ajaxLibrary/AjaxControlToolkitSampleSite/AutoComplete/AutoComplete.aspx

What is a strategy for a simple site site search in a SQL Server 2008 and ASP.NET MVC environment?

I am trying to hash out a strategy for implementing a very simple site search in ASP.NET MVC and SQL Server 2008.
Really, all I want to to do is to be able to rank search results based on the number of times a search word or phrase is found in the webpage. I attempted to do this using LINQtoSQL but I ran into a lot of issues where some LINQ commands don't have a SQL equivalent. This was a few months ago so I don't remember specific errors.
So, I'm just trying to figure out an approach. What I'm thinking is this:
Approach 1:
I should probably write a program to spider the site and somehow index the site's text - I'm thinking I should save information in a table like:
ID
Word
URL
I could then query that and rank based on how many time that word is associated with a certain URL. But then I realized that this technique would completely breakdown if a user was searching for a phrase.
Approach 2:
Then I was toying with the idea of using SPROCs to create a temporary table with a record for each URL that would somehow parse the text and determine how many times the phrase or word appeared in each individual URL. and then we would return the results from the temp table. I am thinking the temporary table would look something like this:
ID
SearchText
URL
Frequency
And then select * from temptable order by Frequency asc or something like that.
However, I'm not sure if SPROCs are capable of parsing text like that, or if simultanious searching would be possible.
I am looking for something very lightweight. I'm not really interested in using Lucene or Solr or anything like that because the learning curve seems very steep and those applications' features are far away more than what I need.
Any thoughts on how I should approach this problem? Is there a different approach that I should consider?
For your phrase versus word issue, why not use wildcards and LIKE operators?
Select Count(*) from temptable where SearchPhrase LIKE '%Apple%'
Maybe not exactly what you want, but Windows SharePoint Search Server isn't all that bad.
Yes, it has the word 'SharePoint' in it, which would usually make me grab the scissors on my desk and start stabbing my eyes out, but having to use it once in a pinch, I was actually somewhat impressed with it.
It's free, so maybe worth a couple of hours playing with it for comparison to writing something custom.
After a little poking around, it looks like SQL Server 2008's Full Text Search is what I would want to use. I'm not 100% sure yet, but it looks promising.
http://msdn.microsoft.com/en-us/library/ms142547.aspx
If you're considering Full Text Search, then also check out lucene.net.
I used FTS for one project, and later used lucene.net for another, and although the requirements were different from yours, I'd never go back to FTS now.

How to improve page loading delays?

I am using gridview control of .net framework 4.0. My list contains 1000 of rows which i am binding to a gridview on each postback. Hence, My page is taking time to load I want to speed up the system. Is there any other control available which can enhance the performance or is there any other way to achieve this?
What all i want is faster performance
You might be able to use faster controls, such as the Repeater, but it depends on what feature you really need. Are you only displaying the data or is it editable?
With such a large amount of data you can look at optimizing what HTML you use for rendering, as you may be able to split the page size by half...
First step should be to do a preformance check to find out what exacly is slow.
Check where in the code things take time, it could be one of many things.
1) If the control uses javascript, perhaps the users are on a old version of their browser with a slow javascript engine.
2) Perhaps the issue is bandwidth?
3) Perhaps its missing SQL index
and on it goes.
Dont guess at what is wrong, find out for sure what is taking the time, and solve then one at the time.
Like Forgotten Semicolon suggested loading via Ajax might be solution, it would give the user a better idea what is going on.
Other then that i would would heavly suggest caching if posible, you can use the built in Cache options to cache the datatable.
In addition to the point about not binding on every postback, I am assuming the datasource is an SQL database. You should probably check to see how fast the query runs. Make sure you have properly indexed your tables.

Good way to allow people to select a lot of things?

I'm using jQuery, ASP.NET, SQL Server, and the other usual suspects to design a company CRM. After they put in contact info, notes, dates, places and so forth they have to be able to select many different people to be "CC'ed." A group of people will be required to be one either "CC'ed" or "ToDo." The rest of the people can be nothing or "CC" or "ToDo." Currently we have it set up as a huge databind to templates with radio buttons for each option. Looks like shit. Anyone have any suggestions? I'd like to use a template with a datasource and have a good way to retrieve their answers and use them.
I'm leaning jQuery direction but like I said I'll need there to be up to 3 possible options for the people. This is going to be all opinion so I'm just looking for options.
Just to re-clarify, this concept is similar to email but I don't want them to have to type anything in as it is a set group of names that they're allowed to select from.
Looking for quick simple and pretty. somewhere in the range of 120 names.
If you intend to look down the jQuery route, I suggest that this widget could possibly help you out (even if only for inspiration sakes). http://quasipartikel.at/multiselect/
I'm struggling to "visualize" your form for terms of "real-estate expendature" etc.
Not directly what you are looking for, but this plugin may help
http://devgrow.com/slidernav-jquery-plugin/
Typing with intellisense. Sorry - any graphical thing will look overloaded.
Or: A table with filter options on top (again, typing). THere simply is no other way.
What I would most likely do to achieve this is implement the auto filter pattern that you type in a text box a few letters of the name and then it would filter down all of the overall results to those containing that pattern. Then have a select all button that will let you check all of them, and then the user can manually uncheck a few instead of having to check all.
The other thing to do would be to offer some type of categorization of the data so that they could filter by category that would put people in probable groups that would want all them all together. Like IT, HR, Executive or something similar.

Best way to create a search function ASP.NET and SQL server

I have an SQL database with multiple tables, and I am working on creating a searching feature. Other than having multiple queries for the different tables, is there a different way to go about said searching function?
I should probably add that a lot of my content is database driven to make upkeep easier. Lucene will not work for this, correct?
Different approaches to consider:
1) Multiple queries pre-baked, like you described.
2) Dynamic sql that you put together on the fly based on user-entered criteria.
3) If text is involved, based on SQL Server full text search or Lucene.
In my open source app BugTracker.NET, I do both 2 and 3 (using Lucene.NET).
I documented how I use Lucene.NET here:
http://www.ifdefined.com/blog/post/2009/02/Full-Text-Search-in-ASPNET-using-LuceneNET.aspx
Since you have tagged the question with Asp.net I suppose you want to search your webpages. In that case you can use Indexing Server to perform freetext searches easily that search the generated html and any keywords you have set up.
As Corey Trager suggested, using Lucene.NET is also an option. It has a good reputation of being fast and quite easy to use.
Although the other answers provide good suggestions such as using Lucene, I have much preferred using a custom caching method.
So for a website that I help create, we cached the searchable data every couple of hours, from many tables, into one simple table with columns such as:
URL
Item/Page Name
Main Keywords
Text Only Contents
Date Updated
I would then write my SQL statement to search this field using different functions to determin the rank.
You might want to check out this post i wrote on writing full text queries, its in C#, but its easilly portable, or just stick it in a library and use it as it.
How to build an SQL full text index search term in c#

Resources