I have a dataset where the attributes are ID , procedure and date in RDF format. I need a SPARQL query to give me the list of patients with no date.
I need a filter option which filters out the data by ID who has no date. I have the entire data visible in GraphDB but I can't figure out a way to filter it by the missingness of an attribute
Related
I am having the following table in my DynamoDB.
I want to get/extract all the data using the following condition or filters
This Month data : This will be the set of records that belongs to 1st of this month to today. ( I think this I can achieve using the BEGINS_WITH filter , again not sure whether this is the correct approach )
This Quarter data : This will be the set of records that belongs to this quarter, basically from 1st of April 2021 to 30th June 2021
This Year data : This will be set of records that belongs to this entire year
Question : How I can filter/query the data using the date column from the above table to get these 3 types (Month , Quarter ,Year ) of data.
Other Details
Table Size : 25 GB
Item Count : 4,081,678
It looks like you have time-based access patterns (e.g. fetch by month, quarter, year, etc).
Because your sort key starts with a date, you can implement your access patterns using the between condition on your sort key. For example (in pseudo code):
Fetch User 1 data for this month
query where user_id = 1 and date between 2021-06-01 and 2021-06-30
Fetch User 1 data for this quarter
query where user_id = 1 and date between 2021-01-01 and 2021-03-31
Fetch User 1 data for this month
query where user_id = 1 and date between 2021-06-01 and 2021-06-30
If you need to fetch across all users, you could use the same approach using the scan operation. While scan is commonly considered wasteful/inefficient, it's a fine approach if you run this type of query infrequently.
However, if this is a common access pattern, you might want to consider re-organizing your data to make this operation more efficient.
As mentioned in the above answer by #Seth Geoghegan , the above table design is not correct, ideally you should think before placing your Partition Key and Sort Key, still for the people like me who already have such kind of scenarios, here is the steps which I followed to mitigate my issue.
Enabled DynamoDB Steams
Re-trigger the data so that they can pass through the DDB Streams ( I added one additional column updated_dttm to all of my records using one of the script )
Process the Streams record , in my case I broken down the date column above to three more columns , event_date , category , sub_category respectively and updated back to the original record using the Lambda
Then I was able to query my data using event_date column , I can also create index over event_date column and make my query/search more effective
Points to Consider
Cost for updating records so that they can go to DDB Streams
Cost of reprocessing the records
Cost for updating records back to DDB
I have a table which is partitioned on a date column.
TRANS_DT between '2010-01-0' and '2030-12-31' EACH INTERNAL '1' MONTH.
A view is created on the table where the TRANS_DT is CAST as CAST(CAST(TRANS_DT AS DATE FORMAT 'MMM-YYYY') AS VARCHAR(8)) to denote date like 'MAR-2021'.
When SQL's are running on the VIEW by applying condition as TRANS_DT='MAR-2021', the explain plan states that a full table scan is performed instead of Partition Level Scan.
I have a dynamodb table which stores creation_date epoch in string format. This date is neither hash key nor sort key. Ultimate goal is querying the creation_date for a range i.e. I need all the ids in the give time range.
The table schema is:
id, version, creation_date, info.
id is hash key and version is sort key.
I was thinking of creating a cloudsearch domain and link that to dynamodb table. Is it possible to use a range query in cloudsearch using java if the date is in string format? If yes how?
Here’s how you can accomplish this in DynamoDB using a GSI with a hash key of creation_y_m and a GSI range key of creation_date.
When you’re querying for a range of creation dates, you need to do a bit of date manipulation to find out all of the months in between your two dates, but then you can query your GSI with a key condition expression like this one.
creation_y_m = 2019-02 AND creation_date BETWEEN 2019-02-05T12:00.00Z AND 2019-02-18T06:00:00Z
Given that most of your queries are a two week range, you will usually only have to make only one or two queries to get all of the items.
You may need to backfill the creation_y_m field, but it’s fairly straightforward to do that by scanning your table and updating each item to have the new attribute.
There are, of course, many variations on this. You could tweak how granular your hash key is (maybe you want just year, maybe you want year-month-day). You could use epoch time instead of ISO 8601 strings.
I need your support for a query in SQLite Studio.
I am dealing with a database made by 1,000 different tables.
Half of them (all named "news" + an identification number, like 04AD86) contain the column "category" which I am interested in. This column can have from 100 to 200 records for each table.
Could you suggest me a query that extracts "category" from every table and returns a list of all possible categories (without duplicates records)?
Thanks a lot
You will probably need dynamic SQL to handle this in a single query. If you don't mind doing this over several queries, then here is one option. First do a query to obtain all the tables which contain the category column:
SELECT name
FROM sqlite_master
WHERE type = 'table' AND name LIKE 'news%'
Next, for the actual queries to obtain the unique categories, you can perform a series of unions to get your list. Here is what it would look like:
SELECT DISTINCT category
FROM news04AD86
UNION
SELECT DISTINCT category
FROM news 05BG34
UNION
...
The DISTINCT keyword will remove duplicates within any given name table, and UNION will remove duplicates which might occur between one table and another.
As part of migrating from SQL to DynamoDB I am trying to create a DynamoDB table. The UI allows users to search based on 4 attributes start date, end date, name of event and source of event.
The table has 6 attributes and the above four are subset of it with other attributes being priority and location. The query as described above makes it mandatory to search based on the above four values. whats the best way to store the information in DynamoDB that will help me in querying based on start date and end date fairly easy.
I thought of creating a GSI with hashkey as startdate, rangekey as end date and GSI on the rest two attributes ?
Inshort:
My table in DynamoDB will have 6 attributes
EventName, Location, StartDate, EndDate, Priority and source.
Query will have 4 mandatory attributes
StartDate, EndDate, Source and Event Name.
Thanks for the help.
You can use greater than/less than comparison operators as part of your query http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html
So you could try to build a table with schema:
(EventName (hashKey), "StartDate-EndDate" (sortKey), other attributes)
In this case the sort-key is basically a combination of start and end date allowing you to use >= (on the first part) and <= (on the second part)... dynamodb uses ASCII based alphabetical ordering... so lets assume your sortKey looks like the following: "73644-75223" you could use >= "73000-" AND <= "73000-76000" to get the given event.
Additionally, you could create a GSI on your table for each of your remaining attributes that need to be read via query. You then could project data into your index that you want to fetch with the query. In contrast to LSI, queries from GSI do not fetch attributes that are not projected. Be aware of the additional costs (read/write) involved by using GSI (and LSI)... and the additional memory required by data projections...
Hope it helps.