I have a table (key=username, value=male or female) and an index on the values.
After I add an item to the table, I want to update the counts of males and females. However, after a successful write, as the index is a Global Secondary Index, the count query is not consistent.
Is there a way (dynamo db Streams, Lambda, ...) to monitor when the index is up to date?
Note that Im not looking for a solution that involves something else (keep count of increments in redis or ...), what I describe here is a simplified problem to especially ask a question about how can I monitor an index in dynamo.
Thanks!
I am not sure if there is any mechanism currently provided to check this but, you can easily solve this problem by adding a single line of code to your query.
ConsistentRead = True
DynamoDB has a parameter when set as true will make sure that you get latest updated value.
Now, when you add/update the item and then query the data add ConsistentRead option in it, this will ensure that you will have latest count value.
Here is the reference link.
If you are able to accomplish using other technique then please do share it.
Hope that helps.
Related
I have a piece of code which does a Buffer-Copy method, but is there any way to check before doing the buffer copy of the record already exists? I do not want to check 'unique keys' in my data dictionary.
This is the code I have at this moment:
CREATE QUERY hQuery.
hQuery:SET-BUFFERS(hBuffer).
hQuery:QUERY-PREPARE("FOR EACH " + hBuffer:NAME + " NO-LOCK ").
hQuery:QUERY-OPEN().
hQuery:GET-FIRST().
DO WHILE NOT hQuery:QUERY-OFF-END:
DO TRANSACTION ON ERROR UNDO:
hDBBuffer:BUFFER-CREATE().
hDBBuffer:BUFFER-COPY(hBuffer) NO-ERROR.
It is unclear what you are trying to accomplish and why you don't want to check unique keys "in my data dictionary" or even what you mean by that.
Your example code is very sketchy and incomplete, maybe someone else can figure out what you are trying to do and why, but I am at a loss to divine the purpose behind it. The use of handles and dynamic queries is especially puzzling. There doesn't seem to be a reason for that or any need to do that.
None the less, if I were coding a routine to copy a buffer, couldn't look up unique indexes in the dictionary, and wanted to pro-actively avoid potential collisions I might write something like this:
define temp-table oLine like orderLine.
for each orderline no-lock:
find oLine of orderLine no-error.
if not available( oLine ) then create oLine.
buffer-copy orderLine to oline.
end.
(Using static coding to keep the example simple.)
(I wouldn't really use OF - it is on my personal forbidden list, I think it is terrible from a documentation and maintenance perspective.)
I believe, as Tom has mentioned in his reply, it'd be most appropriate to have another dynamic query directed at the hDBBuffer using the BUFFER-FIELDs and BUFFER-VALUEs from hBuffer and check the NUM-RESULTS after you use QUERY-OPEN. Then delete the query for memory purposes.
But yes, you would be looking for the metadata unique keys to achieve that. I understand you don't want to do it, but it's REALLY the best way, can't stress it enough.
Now if you would really like to check for the existence of ALL the record data, look into the BUFFER-COMPARE method. You could create a second dynamic query, then cycle all records there by using buffer-compare to match the entire record you're looking at to the one you're assessing whether to create, or list the ones you wish to include or exclude. This approach is way less performatic, though, please keep that in mind.
I want to know the query for kibana, i tried terms and agg, but didnt get right output, so need to filter the data based on distinct query in kibana.
I want to apply query in following input data in elasticsearch
Rows and columns
CELLID|MCC|MNC|
1222|405|861|
1222|405|861|
1222|405|122|
1233|406|861|
1233|406|861|
1224|407|777|
1224|407|777|
need to apply query such a way, it will remove the same CELL ID with different MNC, so expecting output like this
CELLID|MCC|MNC|
1233|406|861|
1233|406|861|
1224|407|777|
1224|407|777|
As you know, it's impossible to have such row vs column infrastructure within Kibana graphs as of now. Cos this feature is yet be made to the new versions as an enhancement.
But then if you're simply trying to print out the count|sum or let it be whatever the aggregation you need, you can have a Data Table visualization with a metric of count and then within your buckets you could define multiple terms-aggregation. In your case, you should have CELLID|MCC|MNC being split by terms-aggregation which should do the job for you. Hope this helps!
I'm using the new gcloud-java API (https://github.com/GoogleCloudPlatform/gcloud-java/tree/master/gcloud-java-datastore/src/main/java/com/google/cloud/datastore) for working with the Cloud Datastore. My specific question is on using GQL for pagination with cursors. I was able to page through the results one page at a time in the forward direction using cursors, but not having any luck with paging backwards.
Example Scenario:
Let's say I've 20 entities in a Kind with IDs 1 through 20. I have a page size of 5. Once I'm on the 3rd page (IDs 11 through 15), if I need to go one page back; i.e. retrieve IDs 6 through 10, what would be the correct GQL/sample code? Again, I prefer not to use offset with a number, but would like to use Cursors.
From what I can tell (actually tested), it looks like one needs to keep track of Start/End cursors for each page as they navigate in the forward direction, then use the saved cursors when there is a need to go back. I just want to make sure if this is the correct/only way or there is a simpler way to accomplish this.
Thanks in advance for your help.
If you add to your original query a sort by key (appended to the end of your "order by" clause), you should be able to reverse each property's sort order and use the latest cursor from your original query to get results in reverse.
Suppose you've iterated through some of the values from your forward query's QueryResults. You can call QueryResults's cursorAfter() method, which will return a cursor pointing right after the last result you saw from your original query. Now you can issue a new query (with the opposite sort order on each property, including the key property) using that cursor as the start cursor. You'll probably want to skip the first result, since it will be the last result you saw from the original query.
I am having a field named rpc in my elasticsearch database and I am displaying it using Kibana. When I search in search bar of kibana like:
rpc:*
It display all the values of rpc field but I want to have only those value to be displayed which are unique.
I have been playing around with Kibana4 since a couple of weeks now. I find it intuitive and simple and the experience has been great till now. Following your question, I tried getting unique results via a Data Table visualization. Why? Because I personally find it easier to understand. Following are the steps:
1. Get unique count
Create the visualization (Visualize -> Data Table). First lets get
the count of how many unique entries we have for a particular field
(We will use this in the later part for verification). I'm using
clientip.raw but as I see, it will work just fine with any friendly
field name too.
2. Set the aggregation right
Set you aggregation back to count and have a Split Rows as follows. Not doing this will give you count 1 for each field value (since it is looking for unique counts) when you populate the table. Noteworthy part is setting the Top field to 0. Because Kibana won't let you enter anything else than a digit (Obviously!). This was the tricky part. Hit Apply and you'll get the results. Unique field values and the count of each of them.
3. Verification:
Going to the last page of the table, we see there are exactly 543 results. This is how I know it works.
What Next?
You save this visualization and add it to a Dashboard. There you can always check the request, query, response and other stats.
Just an addition to the above mathakoot answer.
For the user of newer version (which do not allow bucket size of 0 anymore) just set a value greater than the maximum number of result
And report the value in the Options>Per Page field
I am using Kibana 6 so the UI looks a bit different than the older answers here.
Here is what worked for me
Create a visualization from your query, I used a line graph type (don't think it matters)
Under Data, set metrics aggregation = "Unique Count" and set field to your field.
Set x-axis aggregation = "Terms" and set field to your field.
Set Size > your number of records
Under Metrics and Axes, disable drawing of the graph, circles, and labels (this really helps the UI not lag)
Run query and then click "Inspect" and download CSV
Data
Metrics & Axes
I wanted to achieve something similar but I'm stuck with Kibana 3.1.
I simply added a panel of type "TERMS" and configured its Field = User-agent and left everything else on default values. This gave me a nice bar chart with one bar for each User-agent.
I have a problem that I've been going round and round with in Access 2010. Imagine a table with these columns:
Name Date Time
Now, I have a query that asks the user to input a begin date and an end date and returns all records that are between those two dates. This works fine. However, as soon as I add a sort to the Date column things go awry. Once you put a sort on a column with a parameter the user gets asked to enter the parameter twice. From what I've been able to find out this is normal (although annoying) behavior in Access.
If I add the Date column in a second time and show the column with the sort and don't show the column with the parameter it works fine. The query would look something like:
Name Date (shown & sorted) Date (not shown & parameters) Time
Now when I run the query it all works well and comes out the way I want it to. This would obviously be a great solution then. However, there's another problem. When I save the query, leave, and reopen the query the two columns are merged back into each other. Thus, the change is lost and the user again sees two inputs.
My question is this: what can I do differently to achieve the desired results?
Some possible things I've thought about but don't know the answer to are:
Is there a way to make it so the columns don't merge? Do I have to use a form with the input boxes and take the data from that (I'd prefer not to do that as it will require a lot of additional work to handle the various things I am doing in the database). Is there some obvious thing I'm missing?
Thanks for any suggestions.
FYI: Here is the SQL from the query
SELECT Intentions.Intention, Intentions.MassDate, Intentions.[Time Requested], Intentions.[Place Requested], Intentions.[Offered By], Intentions.Completed
FROM Intentions
WHERE (((Intentions.MassDate) Between [Enter start date] And [Enter end date]))
ORDER BY Intentions.MassDate, Intentions.[Time Requested];
It is true that sometimes the Query Designer in Access will "reorganize" a query when you save it. However, I don't recall an instance where such a reorganization actually broke anything.
For what it's worth, the following query seems to do what you desire. After saving and re-opening it looks and behaves just the same:
For reference, the SQL behind it is
PARAMETERS startDate DateTime, endDate DateTime;
SELECT NameDateTime.Name, NameDateTime.Date, NameDateTime.Time
FROM NameDateTime
WHERE (((NameDateTime.Date) Between [startDate] And [endDate]))
ORDER BY NameDateTime.Date DESC , NameDateTime.Time DESC;
I have had the same problem and I have discovered the reason:
If, after you have run your query, sort a collumn in the result grid and the say yes to save changes to the query the sort action will be stored with the query. This will actually cause the query to run twice. First to create the result and then one more time to sort. You'll therefore be asked twice for the parameters.
SOLUTION: Run the query (entering your parameters twice ;-) ). Then remove the Sorting by clicking on the AZ-eraser symbol in the task bar above (in the sorting compartment).
Then open your query in design-mode and add the sorting order to the appropriate collumn.
Your are then good to go.
Regards
Jan