I have a program written in asp.net with lucene.net. At first I create an index from 28000 documents.
Secondly I'm executing a search, but sometimes there is an error. (I think this error is thrown when there are many results)
The important part of code:
Dim hits As Hits = searcher.Search(query)
Dim results As Integer = hits.Length() 'ergebnisse (größe der hits)
'#####################
'####### RESULTS #####
'#####################
trefferanzahl = results
If (results > 0) Then
Dim i As Integer
Dim h As Integer = results - 1
ReDim array_results(h, 6) 'array zum speichern von den "feldern"
Dim cellX As New TableCell()
For i = 0 To results - 1 Step 1
Dim tmpdoc As Document = hits.Doc(i) ' HERE THE ERROR!
Dim score As Double = hits.Score(i)
MsgBox("2. Docname: " & hits.Doc(i).Get("title"))
array_results(i, 0) = tmpdoc.Get("title")
array_results(i, 0) += tmpdoc.Get("doc_typ")
array_results(i, 1) = tmpdoc.Get("pfad")
array_results(i, 2) = tmpdoc.Get("date_of_create")
array_results(i, 3) = tmpdoc.Get("last_change")
array_results(i, 4) = tmpdoc.Get("id")
array_results(i, 5) = tmpdoc.Get("doc_typ")
array_results(i, 6) = CStr(score)
Next
' Load this data only once.
ItemsGrid.DataSource = CreateDataSource()
ItemsGrid.DataBind()
Else
bool_Suchergebnis = False
End If
searcher.Close()
Thanks in advance
A good principle when performing searches accross very large collections is to limit the results that you are processing as soon as possible. I will assume that you are implementing paging in your grid. And lets assume that PageSize is 20.
What you need to do is make sure that you have access to the PageSize and the current PageNo within this method. Then use Linq accross the result set to Take(PageSize) and Skip (PageNo * PageSize). Then you will only have to process 20 records.
Then, you have two options. If you are binding directly to the array, you might be able to get away with empty items, but I am not sure, so you might have to place dummy items into the datasource array in all positions that won't be displayed. Not ideal, but certainly quicker than processing 1000s of Hits.
The second option is to bind only the 20 items to the grid, which will be quick, switch off paging on the grid as it will only show one page and then implement your own paging behaviour as you know the PageSize, and the current PageNo. This will take more work but it will perform a lot faster than the out-of-the-box gridview binding to a large datasource.
And it will help you solve your memory problem.
Related
Scenario:
I have a calculated SQL that returns 100 results.
Added a table (from this calculated SQL) and limited the size of the page by 25 results.
This will generate 4 pages.
Pager form AppMaker works well (navigates between pages) but i need a button that navigates directly from page 1 to the page 4.
is this possible?
Anyone got a solution for this?
Regards
If you need to know how many entries your table has (in your case it's seems fixed to 100, but maybe it could grow), you can still do what you want:
E.g. say your table on YOURPAGE depends on a datasource called Customers.
Create a new Data item called CustomerCount, with just one field, called Count (integer).
Its data source would be a sql query script:
Select count(CustomerName) as Count from Customers
on the page you are having the table on, add a custom property (say called
Count of type integer)
In the page attach event, set the property asynchronously with this custom action:
app.datasources.CustomerCount.load(function() {
app.pages.YOURPAGE.properties.Count = app.datasources.CustomerCount.count;
app.datasources.Customers.query.pageIndex = #properties.Count / 25;
app.datasources.Customers.datasource.load();
});
I tried similar things successfully in the past.
Found a solution for this:
ServerScript:
function CandidateCountRows() {
var query = app.models.candidate.newQuery();
var records = query.run();
console.log("Number of records: " + records.length);
return records.length;
}
in the button code:
var psize = widget.datasource.query.pageSize;
var pidx = widget.datasource.query.pageIndex;
var posicao = psize * pidx;
var nreg = posicao;
google.script.run.withSuccessHandler(function(Xresult) {
nreg = Xresult;
console.log('position: ' + posicao);
console.log('nreg: ' + nreg);
console.log('psize: ' + psize);
console.log('pidx: ' + pidx);
var i;
for (i = pidx; i < (nreg/psize); i++) {
widget.datasource.nextPage();
}
widget.datasource.selectIndex(1);
}).CandidateCountRows();
This will allow to navigate to last page.
If you know for a fact that your query always returns 100 records and that your page size will always be 25 records then the simplest approach is to make sure your button is tied to the same datasource and attach the following onClick event:
widget.datasource.query.pageIndex = 4;
widget.datasource.load();
I have the Following XML which is a result from a certain WEB-Service.
<?xml version="1.0" encoding="UTF-8"?>
-<ArrayOfString xmlns="http://tempuri.org/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<string>16/May/2016 - 20/May/2016</string>
<string>20/May/2016 - 23/May/2016</string>
<string>23/May/2016 - 27/May/2016</string>
<string>27/May/2016 - 30/May/2016</string>
</ArrayOfString>
I have the Following VBA Code to read the Above XML
strRet = PostWebservice(strUrlEBenefit, strSoapAction, strXml)
intPos1 = InStr(strRet, "<string>") + 8
intPos2 = InStr(strRet, "</string>")
If intPos1 > 11 And intPos2 > 0 Then
xmlresult = Mid(strRet, intPos1, intPos2 - intPos1)
End If
as a result I'm Getting "16/May/2016 - 20/May/2016" in xmlresult.
What I want to do is getting all the date values Between all the [string] tags.
Can you please guide me how can I achieve the result? I understand I need to read it into array but I don't know how and didn't saw any useful tutorials for me(beginner in VBA and XML) to ref.
Read the xml-result which was returned from the web service into xml-document and use e.g. SelectSingleNode to select the node ArrayOfString. This node has namaspaces so you need to use namespaces in the xpath as well. Then just read all the child node texts e.g. into a collection here declared as result. HTH
Note: Add reference to Microsoft XML, v6.0 dll
Sub GetDateValues()
Dim xmlDocument As MSXML2.DOMDocument60
Dim xmlNamespaces As String
Dim arrayOfStringNode As IXMLDOMNode
Dim result As Collection
Dim xmlNode As IXMLDOMNode
Dim strRet As String
strRet = PostWebservice(strUrlEBenefit, strSoapAction, strXml)
Set xmlDocument = New DOMDocument60
If Not xmlDocument.LoadXML(strRet) Then
Err.Raise xmlDocument.parseError.ErrorCode, , xmlDocument.parseError.reason
End If
xmlNamespaces = "xmlns:myns='http://tempuri.org/'"
xmlDocument.setProperty "SelectionNamespaces", xmlNamespaces
Set arrayOfStringNode = xmlDocument.SelectSingleNode("/myns:ArrayOfString")
Set result = New Collection
For Each xmlNode In arrayOfStringNode.ChildNodes
result.Add xmlNode.Text
Next xmlNode
End Sub
I'm extrapolating using the answer from: 'dee', but reading in to an array with x number of columns. The reason for dNode below is because webservices often return auxiliary metadata at the same level as the actual value you want in your array. So you have to use SelectSingleNode at that point.
xmlDoc.SetProperty "SelectionNamespaces", xmlNamespaces
Set xmlNodes = xmlDoc.SelectNodes("/myns:.../...")
rws = xmlNodes.length
cols = var '# of columns from the request sent or extrapolate this code to count children of a single node at the level necessary
ReDim xay(1 To rws, 1 To cols)
For Each xNode In xmlNodes
r = r + 1
For c = 1 To UBound(xay, 2)
Set dNode = xNode.ChildNodes(c - 1)
Set dNode = dNode.SelectSingleNode("myns:THE_VALUE")
xay(r, c) = dNode.Text
Next c
Next xNode
Im writing SSAS MDX queries involving more than 2 axis' to retrieve a value. Using ADOMD.NET, I can get the returned cellset and determine the value by using
lblTotalGrossSales.Text = CellSet.Cells(0).Value
Is there a way I can get the CellSet's Cell(0) Value in my MDX query, instead of relying on the data returning to ADOMD.NET?
thanks!
Edit 1: - Based on Daryl's comment, here's some elaboration on what Im doing. My current query is using several axis', which is:
SELECT {[Term Date].[Date Calcs].[MTD]} ON 0,
{[Sale Date].[YQMD].[DAY].&[20121115]} ON 1,
{[Customer].[ID].[All].[A612Q4-35]} ON 2,
{[Measures].[Loss]} ON 3
FROM OUR_CUBE
If I run that query in Management Studio, I am told Results cannot be displayed for cellsets with more than two axes - which makes sense since.. you know.. there's more than 2 axes. However, if I use ADOMD.NET to run this query in-line, and read the returning value into an ADOMD.NET cellset, I can check the value at cell "0", giving me my value... which as I understand it (im a total noob at cubes) is the value sitting where all these values intersect.
So to answer your question Daryl, what I'd love to have is the ability to have the value here returned to me, not have to read in a cell set into the calling application. Why you may ask? Well.. ultimately I'd love to have one query that performs several multi-axis queries to return the values. Again.. Im VERY new to cubes and MDX, so it's possible Im going at this all wrong (Im a .NET developer by trade).
Simplify your query to return two axis;
SELECT {[Measures].[Loss]} ON 0, {[Term Date].[Date Calcs].[MTD] * [Sale Date].[YQMD].[DAY].&[20121115] * [Customer].[ID].[All].[A612Q4-35]} ON 1 FROM OUR_CUBE
and then try the following to access the cellset;
string connectionString = "Data Source=localhost;Catalog=AdventureWorksDW2012";
//Create a new string builder to store the results
System.Text.StringBuilder result = new System.Text.StringBuilder();
AdomdConnection conn = new AdomdConnection(connectionString);
//Connect to the local serverusing (AdomdConnection conn = new AdomdConnection("Data Source=localhost;"))
{
conn.Open();
//Create a command, using this connection
AdomdCommand cmd = conn.CreateCommand();
cmd.CommandText = #"SELECT { [Measures].[Unit Price] } ON COLUMNS , {[Product].[Color].[Color].MEMBERS-[Product].[Color].[]} * [Product].[Model Name].[Model Name]ON ROWS FROM [Adventure Works] ;";
//Execute the query, returning a cellset
CellSet cs = cmd.ExecuteCellSet();
//Output the column captions from the first axis//Note that this procedure assumes a single member exists per column.
result.Append("\t\t\t");
TupleCollection tuplesOnColumns = cs.Axes[0].Set.Tuples;
foreach (Microsoft.AnalysisServices.AdomdClient.Tuple column in tuplesOnColumns)
{
result.Append(column.Members[0].Caption + "\t");
}
result.AppendLine();
//Output the row captions from the second axis and cell data//Note that this procedure assumes a two-dimensional cellset
TupleCollection tuplesOnRows = cs.Axes[1].Set.Tuples;
for (int row = 0; row < tuplesOnRows.Count; row++)
{
for (int members = 0; members < tuplesOnRows[row].Members.Count; members++ )
{
result.Append(tuplesOnRows[row].Members[members].Caption + "\t");
}
for (int col = 0; col < tuplesOnColumns.Count; col++)
{
result.Append(cs.Cells[col, row].FormattedValue + "\t");
}
result.AppendLine();
}
conn.Close();
TextBox1.Text = result.ToString();
} // using connection
Source : Retrieving Data Using the CellSet
This is fine upto select on columns and on Rows. It will be helpful analyze how to traverse sub select queries from main query.
I'm using the code below to create dynamic panes in an accordion control. Info is read from a data set and the controls are generated based on that info. I'm now stuck when it comes to finding these controls. When a user clicks a button I need to loop through all the controls and get the information inside the textboxes... but all I really need to know is how to call the darn things!
Do Until b = 0
holder = ds.Tables(0).Rows(i).Item("Issue" & z).ToString
If holder <> "" Then
lblTitle = New Label()
txtContent = New TextBox()
lblTitle.Text = "Issue" & z & " " & ds.Tables(0).Rows(i).Item("Issue" & z)
txtContent.Text = ds.Tables(0).Rows(i).Item("Issue" & z)
pn = New AjaxControlToolkit.AccordionPane()
pn.ID = "Pane" & z
pn.HeaderContainer.Controls.Add(lblTitle)
pn.ContentContainer.Controls.Add(txtContent)
arcPane.Panes.Add(pn)
End If
pncount = pncount + 1
z = z + 1
b = b - 1
Loop
Every control has a property called Controls, which is a collection of immediate child controls. Looping through them is possible, where you can examine them one by one until you find the one that you want. Each control instance also has a method called FindControl, which you can use to look up controls by their IDs. You should be able to find them this way. Start from the first common parent control (e.g. arcPane).
foreach (Control pane in arcPane.Panes)
{
foreach (Control c in pane.ContentContainer.Controls)
{
//examine c.ClientID or c.GetType() or some other
//property that you can recognize the control by
}
}
i want to use an List to store the title, path,... from Documents.
I declared the list like this:
Dim MyDocuments As New List(Of Document)
But i don't really know how to handle the list.
i want to use the list instead of an ReDim Array.
For i = 0 To results - 1 Step 1 ' forschleife zum durchlaufen der Ergebnisse
Try
MyDocuments.Add(New Document())
array_results(i, 0) = hits.Doc(i).Get("title")
array_results(i, 0) += hits.Doc(i).Get("doc_typ")
array_results(i, 1) = hits.Doc(i).Get("pfad")
'array_results(i, 2) = hits.Doc(i).Get("date_of_create") '
array_results(i, 2) = hits.Doc(i).Get("last_change")
array_results(i, 3) = CStr(hits.Score(i))
array_results(i, 4) = hits.Doc(i).Get("doc_typ")
Can I store the object Document, or do i have to create an own class??
Is there a good tutorial for using the list? (i searched, but didn't found something good)
Is the List of (T) the right data structure?
but how can i do like mylist(i) ->gettitle() or something like this?
thanks in advance!
Yes, you can store your documents in a generic List. Deciding if a List<T> is the right data structure or not depends on what you want to do with it. Maybe if you provide more information someone could come up with a better example. I don't know VB.NET so i'll do it in C#.
// i assume you're using the Document class of Lucene.NET
List<Document> documents = new List<Document>();
// add the documents to your collection
for (i = 0; i < hits.Length(); i++)
{
// each result in the list contains a Document
// which you can add to your list
documents.Add(hits.Doc(i));
}
// you can search the list for a Document following a specific rule, using lambda expressions
Document myDoc = documents.Find(d => d.Get("title") == "a value");
// you can get a document by a specific index
Document myOtherDoc = documents[0];
// you can search the list for multiple Documents following a specific rule, using lambda expressions
List<Document> myDocs = documents.FindAll(d => d.Get("doc_typ") == "a type");
More information about the List<T> can be found here: http://msdn.microsoft.com/en-us/library/6sh2ey19.aspx
More information on lambda expressions can be found here: http://msdn.microsoft.com/en-us/library/bb397687.aspx
This article on SO shows how to use lambdas to search a List<T>