Use a SQL Server stored procedure with unlimited number of multiple criteria instead of a view - asp.net

Question

I have a page where users select multiple search criteria to retrieve data from a SQL Server 2014 view. The view is grabbing data from a table on a linked server (I am not able to put the view directly on that server, and the table I am reading from has over 800 million rows so copying that data onto the local server isn't going to happen).
Of course, I can't index the view either (on linked server) so I'm trying to find a way to stop the timeouts from happening when the query is run. Is it possible to do something like this in a stored procedure?
SELECT
cast(trees as varchar(3)) as Trees
, MIN(fruitnumber) AS FN_Start
, MAX(fruitnumber) AS FN_End
, COUNT(CASE WHEN fruitType = 'apple' THEN 1 ELSE NULL END) AS apple
, COUNT(CASE WHEN fruitType = 'banana' THEN 1 ELSE NULL END) AS banana
FROM
view_fruitReport
WHERE
(orchard = #orchard) and
and here's where it gets wonky. Users select the orchard from a dropdown (not a combobox because we use IE11 and ajaxtoolkit combo box still doesn't work there) so only one selection possible but.
They are able to add criteria to listboxes. Unlimited criteria. And they don't need to select any of the criteria, they can just search by orchard.
So the rest of the WHERE clause is built based on what they have added to the listboxes.
Like this:
' check if items selected in both listboxes'
If trees_Listbox.Items.Count > 0 Then
If fruitminListBox.Items.Count > 0 Then
'cycle through items in fruitnum listbox to create an "in" clause for sql query'
For Each item As ListItem In trees_Listbox.Items
whereString += String.Join(",", item) + ", "
Next
whereString = Left(whereString, Len(whereString) - 2) + ")"
selectQry += "(" + wherecls + whereString + ")"
whereFNcls = "(fruitNumber between "
For Each itemFNmin As ListItem In fruitminListBox.Items
'create a "between" clause for the min and max FN values entered by user.'
whereOEcls += itemFNmin.Value + " and " + fruitmaxListBox.Items(i).ToString + ") or (fruitNumber between " '(fruitnumber between number and number) or '
i += 1
Next
'trim off the last text portion of the whereOEcls'
whereOEcls = Left(whereOEcls, Len(whereFNcls) - 25)
selectQry += " and (" + whereFNcls + ") GROUP BY trees ORDER BY trees"
fruityData.SelectCommand = selectQry
WeeklyGridView.Visible = True
Else
'see if FN is empty but trees is selected'
For Each item As ListItem In trees_Listbox.Items
whereString += String.Join(",", item) + ", "
Next
whereString = Left(whereString, Len(whereString) - 2)
selectQry += wherecls + whereString + ") GROUP BY trees ORDER BY trees"
fruityData.SelectCommand = selectQry
WeeklyGridView.Visible = True
End If
Else
Essentially ending up with a where clause that could look like this:
WHERE (orchard = #orchard)
and trees in (100,200,300,400)
and fruitnumber between (itemFNmin.Value and itemFNmax.Value)
or fruitnumber between (itemFNmin.Value and itemFNmax.Value)
etc etc etc
Which works except it makes things very ugly and I am certain is a poor way of doing this.
I have no clue if/how I can make these lists of variables pass to a stored procedure as multiple arrays or tables etc.
Probably anything is better than having them tied to a view, whose linked server table isn't even an indexed table (not my fault haha)

Answer 1

For your first question: You can return the Count of each fruit type, but it will have performance implications as it requires a subquery for each one. This also requires that you hard code each possible fruit type in the query. I assume that the fruit types can change or have other types added to them, so this isn't the most desirable in terms of maintenance either. You can't dynamically add columns to a query unless you build SQL in your proc and make use of sp_executesql, which is more convoluted than doing in line SQL in your .Net code.
SELECT
cast(trees as varchar(3)) as Trees
, MIN(fruitnumber) AS FN_Start
, MAX(fruitnumber) AS FN_End
, CASE
WHEN fruitType = 'apple' THEN (SELECT COUNT(fruitType) FROM view_fruitReport WHERE fruitType = 'apple') ELSE NULL
END AS [apple]
, CASE
WHEN fruitType = 'banana' THEN (SELECT COUNT(fruitType) FROM view_fruitReport WHERE fruitType = 'banana') ELSE NULL
END AS [banana]
FROM
view_fruitReport
WHERE
(orchard = #orchard)
For your second question, you can pass in lists/tables into a stored procedure. One method is to pass some sort of delimited string and parse it using T-SQL. I recommend a different approach, however, which is
Table Value Parameters. This is a parameter that acts as a table that you can join with in your stored procedure.
Here is an example for implementing a Table Value Parameter for the Trees column.
You will first need to declare a SQL Type:
CREATE TYPE [dbo].[Trees] AS TABLE (Trees INT)
Then you can reference it in your stored procedure as a parameter that acts as a table. Note that you can't use WITH(NOLOCK) with these and must specify READONLY in the paramter:
CREATE PROCEDURE [dbo].[up_getOrchardInfo]
(
#Trees As [dbo].[Trees] READONLY
, #Orchard INT
)
AS
BEGIN
SELECT
cast(trees as varchar(3)) as Trees
, MIN(fruitnumber) AS FN_Start
, MAX(fruitnumber) AS FN_End
, COUNT(CASE WHEN fruitType = 'apple' THEN 1 ELSE NULL END) AS apple
, COUNT(CASE WHEN fruitType = 'banana' THEN 1 ELSE NULL END) AS banana
FROM
view_fruitReport AS F
INNER JOIN #Trees AS T
ON F.Trees = T.Trees
WHERE
(orchard = #orchard)
END
GO
The above example will filter by the Trees passed in. Note that if want to return everything for the Orchard if #Trees is Null or the count is 0 you will need to include that conditional logic in your stored procedure.
IF (#Trees IS NULL OR (SELECT COUNT(1) FROM #Trees = 0))
BEGIN
--No Join to #Trees
END
ELSE
BEGIN
--Query from above.
END
Finally, on the .Net side you will need to pass in a DataTable object as a Parameter on the SqlCommand with the Type of Structured:
Dim sqlCommand As New SqlCommand("up_getOrchardInfo", sqlConnection.SqlConnection)
sqlCommand.CommandType = CommandType.StoredProcedure
Dim sqlTreesParameter As New SqlParameter("#Trees", SqlDbType.Structured)
sqlOrchardParameter.Direction = ParameterDirection.Input
Dim tblExample As New DataTable
tblExample.Columns.Add("Trees", New Integer().GetType())
Dim drExample As DataRow = tblExample.NewRow()
drExample.Item("Trees") = 100
tblExample.Rows.Add(drExample)
'Adjust if Orchard is a VarChar/String'
Dim sqlOrchardParameter As New SqlParameter("#Orchard", SqlDbType.Int)
sqlOrchardParameter.Direction = ParameterDirection.Input
sqlOrchardParameter.Value = intYourOrchardValue
sqlCommand.Parameters.Add(sqlTreesParameter)
sqlCommand.Parameters.Add(sqlOrchardParameter)
'Execute Dataset
All of this said, you may want to consider making multiple stored procedures. One stored procedure could be optimized for returning everything when only an Orchard is passed and another for when Trees are also passed. This depends on how many parameters you're dealing with.

Answer 2

Maybe there's a UX answer. When you know it's not going to come back for a long time give the user a heads up and confirm if they want to wait or not. (With check box on the confirmation that says, "don't show me this again.")
But don't re-event the godawful estimated file transfer time from Windows Explorer.