Circumventing query page size limitations - google-app-maker

In the Project Tracker template there is a feature where statistics for a certain project's items are displayed. You can filter the project items, but the statistics will only show those statistics for all the project items, i.e. the filters do not affect the statistics.
I would like to add the feature of filters affecting those statistics in a similar implementation of mine. My current solution passes the keys of those project items (affected by filters, too) to a calculated data source, which then calculates the statistics using those item keys, essentially applying the filters used in the page.
My issue is that my calculations are restricted by the query page size. For example, if I apply filters that limit the number of items to 15 records, but the page size is 10 records, I will only have statistics of those first 10 items, which is not useful. I'd need to have the statistics on all the records that are left after filtering.
One way to solve this would be to get rid of the query page size and leave it at 0. However, similarly to the Project Tracker template, I'm displaying the project items on the page in a table, and if I do that, the page becomes too heavy.
How can I circumvent the query page size? I'm thinking I could
limit the items displayed in the page by some other way than query page size (i.e. hiding items from the UI)
use a different datasource for the statistics, but in some way copying the filters used in the datasource that is displaying the project items
Both of these ways I could think of, I can't seem to implement. I don't know how I could hide items from the UI to make it less heavy, as query page size pretty much does it. I have also tried copying the filters from a datasource to a similar one, but that does not seem to work.
EDIT: I might have come up with a way to solve this myself, but I still need to implement it. Now I'm using the page size restricted Items Ds to apply the filters on, and the statistics are build from this data source. If instead, I use a non-restricted Ds called AllItems, and apply the filters on it, and then pass the item keys to a page size restricted Ds (to show the items in UI) AND to a calculated Ds (for the stats). Will make a response once I've verified it works.

I solved the issue myself.
To produce (refreshable) statistics subject to filtering but not restricted by page size, I used the following datasource structure:
In this structure, the filters practically flow to the data sources below, as they are passed the ItemKeys that fit the filter. What this completes is that my Statistics (used in piegraphs etc.) can be filtered dynamically and account for all the records that fit the filter, while the UI does not get crowded over too many records, as the data source used in UI has a query page size limitation.

Related

Drupal views - Show no exposed filter options when no results come from view

I have a problem. I have a view that is getting back no results like it should, for the context it is in. I have a list of resources (file nodes) that are associated to an organic group. For this view I use exposed filters that should only show filter options that apply to the result set.
For example, if I have only one "folder" associated with this group then I should only see one folder in the exposed filter for folders. If I have none then none should show up. The problem is that when we have none it is showing ALL the folders for the site. Same with the associated topics and other filter fields.
Currently we are using the views_hacks > views_filters_selective module to accomplish the initial filtering. But it doesn't seem to handle the case when the view has no results. And I am having trouble figuring out how (and where) to identify when it has no results so that I can just eliminate all the filter options.
Now to be a little extra honest about this, we actually copied some of the functions from the above mentioned module and improved it (for performance purposes) within a custom module we wrote. But like I said, I am having a tough time trying to identify when the view has no initial results at all.
Does anyone have any idea as to how to do this? We are running this in Drupal/Pressflow 6 with Views 3.
It turns out I just needed to set it to not empty for that field and it works fine.

How can I specify the number of a instances of a given field in a View

I have standard basic article content type which, amongst other things, contains an image field which is configured to contain an unlimited amount of values.
I'm now trying to create a view that lists some of the fields in this content type but I want to limit the number of images that are returned for this specific field to 3 random images in the list (or less). Is this something that can be achieved right in my view configuration or do I need to do something ninja? I'm not aware of everything that's possible php wise to "rewrite" / "filter" the results of my view but I'm confortable with anything programming related.
I'm using drupal-7 with Views 7.x-3.0-alpha1
Thanks!
AFAIK there is no way through the Views UI to randomize the selections of a grouped field.
I can see two possible ways, in theory to achieve what you want.
Use hook_views_query_alter to alter the SQL query for the view. I'm not sure Views support what you need to randomize the results, but you have the ability to alter the query that views generates, and might be able to randomize it.
Select all results, but using theming to only show the randomly selected options. This could be done in a preprocess function. Views provides several, depending on your display option, hook_views_view_unformatted might be what you need.

How to manage complex data entry validation

I feel like I am fighting against the current when I develop ASP.NET Webform apps. I frequently run into the same problems, and while I eventually find some kind of workaround i'm never fully satisfied with the results.
Here is an example of a typical problem:
The design requires a grid or grid-like result set. This result set is pulled from a database, however, there are additional controls on each row that are not data bound, but their contents are used to insert data into other records.
A good example of this would be displaying a list of products, then adding selected products to a shopping cart based on values entered into quantity fields, and options selected per product. Throw into the mix that you have to allow multiple lines to be added to the cart at the same time, and it starts to get more complex.
Let's add to that mix that you can't select certain products together (mutually exclusive), that you can only select a certain number of one product, but not another, that prices may change while the user is selecting their items, that you you get an overall discount per item based on quantity purchased (both per item and overall order), that you are using a line of credit and cannot exceed the line of credit, nor can you buy more of a given item than an arbitrary amount set in your account or in the product by your account representitive (think certain over the counter medications that the government limits how many you can buy), etc.. etc.. etc..
What starts out as a simple grid with an add to cart becomes a hopeless mess of business logic, which then of course requires validation and notification to the user of various errors in their choices.
How does one deal with very complex data entry schemes in asp.net? How do you even begin to design a piece of software to do all this?
EDIT:
Please don't suggest changing the interface, as the interface is not the problem. Users are fine with it, and they demand that it function the way it does. I'm looking for help on how to design and solve the problem of implementing it.
Don't put anything other than the basic validation in your code behind. The code behind should just take what the user entered, build a business object (or collection of business objects) and let those business objects validate themselves.
Each business rule should be a single function call on the business object that deals with just the one rule and nothing else. You then simply call them one after another and keep track of which ones pass and which ones fail.
When a validation fails, the business objects can provide the code behind enough information that it can display the correct errors and highlight the fields that have errors.
Some options to consider:
Separate out your validation logic so you can call it from both client-side and server-side code (enables next point).
Use AJAX to perform server-side validation and provide immediate feedback as the user performs various tasks, ie increases quantity.
Provide abundant and clear instructions/feedback to the user both before and after any actions are performed. Before: warn that product X can't be bought with product Y (especially if Y is already in the cart). After: explain the problem exactly and suggest ways to correct, eg "why don't you remove product Y?")
Fail gracefully, ie if only one product fails validation then ensure all other products are added.
Simplify the entry process, eg only allow one product to be added to the cart at a time.
The last point is important. A complicated data entry process can confuse users before they start, and makes trying to understand the numerous validation errors difficult. And this is even before you start coding the validation logic.

Dojo datagrid and treegrid help - datagrid has a reformating flash?

I'm having a bit of a time trying to get Dojo grids (1.5) to play nice. Specifically I've spent about two weeks of work trying to implement a grid that allows for our result set data to collapse into rows, where rows can be expanded. Data comes in as a full set in JSON format, using ItemFileReadStore as the store. Any subsequent sorts or pagings are handled by GETing a new json from the application, and passing in new query parameters in the url.
The nested data was only two layers deep - a top layer to always be displayed and an array of child data with identical structure as the top layer. Each node has a unique ID and a cluster ID - on a parent node the unique ID and cluster ID will match.
I was initially very excited with TreeGrid - but I couldn't see how I could format it to do what I needed - namely eliminate the 'summary row' and one extra row full of null cells (???) that I just couldnt figure out how to remove unless I focused the query to only one cluster. I studied the test examples, built many test pages myself, tried to understand the forestModel, which for what I could tell was unnecessary... I found so little documentation, and sources I found online hinted that TreeGrid might not be reliable...
So I decided I would try to implement the expandable/collapsible rows in dataGrid.
I flattened the JSON data and added another attribute to indicate being a top level node ('alwaysShow' = true). I built my grid programaticaly and applied grid.filter() to pull only those top level nodes. I modified that filter by extending the ItemFileReadStore _FetchItems "filter" method to allow for OR querying instead of AND, and also modified it to allow for keys to point to arrays - when a top level node (small +/- icon in the cell) is clicked, the cluster ID of the parent node is added to the grid.filter.allowed[] and the filter is updated, allowing nodes with that cluster_id value to be displayed.
This worked fine on my small test set of five records (although id say a little slugish...) - but now I am pulling ~900 rows back from the application, and on expanding large clusters (~80 rows) I am seeing a very long flash of blue and white on the filter updates. I've spent most of my day trying to step through in firebug to find where its happening, but the dojo logic is so spread out. Seems to be happening before the call to _Grid.js defaultUpdate.
Its so bad that I am considering trying again with TreeGrid. Im also considering just doing this by hand... Im kicking myself for spending so much time trying to get Dojo to work to begin with. I would also consider a commercial "JSON->table with collapsible row" library if anyone has any recommendations...
Any suggestions or insights? Familiarity with the flashing problem or how I could adapt TreeGrid to my needs? I'm aware this is a bit of a rant... Many thanks for any help.
-robbie
EDIT: I eventually gave up trying to get Dojo to do what I needed and coded it myself in less than a day. Not the best use of three weeks...
EDIT:
I just found a solution that works for me, I have added the following CSS:
.dojoxGridSummaryRow {
visibility: collapse
}
Basically the summaries are probably still created but they are not visible nor taken into account in the table layout. That's good for me. Hope this will solve your issue.
This won't help but just to let you know that:
"- but I couldn't see how I could format it to do what I needed - namely eliminate the 'summary row' "
is the very EXACT same thing I'm trying to achieve and did not find the solution even though this looks like a very simple feature... Will let you know if I found a solution...

Performane improvement in getting paged data for asp.net grid

I am encountering a performance problem in the development of current UI. The problem, I suppose, is however general.
I have a page with a simple asp.net grid. Grid will display data from a table based on certain search criteria. Moreover, grid has fixed page size (say 10). There is pager at the bottom which can be used to navigate b/w pages. On the back end, whenever search button is pressed a stored procedure is called which returns the desired data.
Stored procedure has parameters like currentpageIndex, pagesize, other search criteria, etc. Here is a pseudo code for sp:
-- SP begins
-- calculate the page index range to return required using current page index and page size
-- query the table in a CTE and do all filtering. Also calculate row numbers so that
-- correct record range can be returned.
-- use the cte to return the correct record based on the row number calculated in CTE
-- SP ends
I have following problems/queries in this approach
When Db table size is large (say 10 million records), performance degrades and this approach becomes impractical.
Is using table variable or a temporary table more useful?
Is there any other efficient way to get paged data from database?
Hi Dan, the article provided a new insight for calculation of total rows. Really helpful. Thanks.
But still is there better way than using CTE when data is large?
Update: I have found few other performant approaches for efficiently getting paged records.
There's a good article on SqlServerCentral called SQL Server 2005 Paging – The Holy Grail that shows a few techniques for server-side paging. You will need to register to view it, though.
I know for really large result sets then software like Google will simply estimate how many rows will be returned, bypassing the need to get a count of all the rows returned.
Sorry, if I can't give more help.

Resources