I have a collection "foo". Each document in the collection has a property/field "bar" that is an array of objects like so:
foo = [{
bar: [{
id: "random_string",
"status": "string"
}, {
id: "random_string",
"status": "string2"
}]
}, {
bar: [{
id: "random_string",
"status": "string"
}, {
id: "random_string",
"status": "string2"
}]
}]
What I want to achieve is I want to be able to query the db so that I can get two different collections, one with all the documents in the "foo" collection that have "string" as a value of at least one of the objects in the "bar" array, and another collection for all the documents that have "string2" as a value of at least one of the objects in the "bar" array.
Is that even possible? I'm struggling quite a lot to this one, so any help would be greatly appreciated. I'm also happy to change the db schema if needed, totally open to suggestions!
You can't do this with a single query because Firestore currently doesn't support logical OR conditions. In other words, you can't have a query that gives you all the documents where any one of a set of conditions is true.
Also, you need to be able to call out a particular field in a document in order to perform a query against it. Without a specific field to use, the query can't use an index to speed things up, and the query would never scale at the magnitude offered by Firestore.
Related
I'm using AWS DynamoDB to store data in JSON format. The Partition key is "device" and sort key is "timestamp". I can query the table for a specific device in a range of dates. I can then filter the content by the specific endpoint (in the nested "reports" object) the application is interested in.
{
"device": "AAA111",
"attr1": "bbb",
"reports": [
{
"endpoint": 1,
"value": "23"
},
{
"endpoint": 3,
"value": "26"
},
{
"endpoint": 4,
"value": "20"
}
],
.........
............
...........
"timestamp": "2017-11-30T03:50:30z"
}
The problem I have is if for example, I want to retrieve the latest value of an specific "endpoint". So, I can retrieve the latest record for a "device" based on the latest "timestamp", but it doesn't guarantee this record will contain value for this particular endpoint (not all records contains all endpoints). To solve this I have to basically scan the latest records (in descending order) and return the first object where the endpoint is found. Also, I don't know how many records I have to retrieve to find one...
I'm wondering if there is a better way of doing this... I tried with secondary indexes but this would require to duplicate the data, creating an object for each endpoint value (duplicating the common data). I would like to avoid this...
I would appreciate any hints on how to solve this issue.
Thanks
Gus
I'm wondering if it is possible to get all info about a specific date from Freebase.
I can easily retrieve info about a date giving a specific topic, for example, to grab all persons of interest who were born on a specific date:
[{
"type":"/people/person",
"limit":1000,
"sort":"name",
"name":null,
"guid":null,
"timestamp":null,
"/people/person/date_of_birth":"1955-02-24"
}]
Is it possible to grab all types? I'm after things like people born on that date (which I have), major events (start of a war, assassination of a person of interest, etc), and so on.
Essentially I want to match all fields that are dates and return the full information about that entry, regardless of type.
Reflection is what you need here:
[{
"/type/reflect/any_value": [{
"type": "/type/datetime",
"value": "1955-02-24",
"link": {
"source": {
"id": null
},
"master_property": null
}
}]
}]
A couple of notes on that: the MQL manual I've linked to is somewhat bitrotted in its details but is still the best documentation that exists on MQL. Secondly, there's what I'm pretty sure is in MQL bug if you use "*": null or more specifically "target_value": null in the link clause above which makes it ignore the outer value you specified... so don't do that :-)
Let's say I want to get all movies in which at least two (different) actors called "John" played:
Example query:
[{
"type":"/film/film",
"name":null,
"limit":10,
"/film/film/initial_release_date":"2005"
"starring":[{
"a:actor": [{
"type": "/film/actor",
"name": null,
"name~=": "John",
}],
"b:actor": [{
"type": "/film/actor",
"name": null,
"name~=": "John",
}]
}]
}]
If you run the example query, you will see that it will list movies with only one "John" in them. How can I fix my query to exclude these results with duplicated children?
In general, you'll have to do the filtering client-side; queries in MQL are "tree-like" in that one part of the query can't refer to another part rather than being a generic graph.
In this case, you could look for films which have more than one "John" acting in them; however, MQL doesn't allow you to filter on a derived property like "count", so the best you can do is to reverse sort based on the count and then just stop processing as soon as you hit the first entry with "count": 1. However, that query times out if you remove the fixed 1935 release date (sorting in MQL kills performance), so you're probably stuck with just simple client-side filtering.
I tried Googling and searching everywhere, but couldn't find a definitive authority on this topic. While being true to REST principles, how should I design the HTTP interface for:
An ordered list (get, add, insert into position, reorder, remove)
A set (get, add, remove)
A hash-table (get, add, remove)
NOTE: These data structures are to contain references to existing resources with known ids
That's how I would do it for an ordered list and hash table. I guess the methods would be the same for a set and a list:
Ordered list
Get item 123:
GET /list/123
Append an item to the list:
POST /list/
Insert new item into position 5:
POST /list/?position=5
Move item 123 to position 3:
PUT /list/123?position=3
Delete item 123:
DELETE /list/123
Delete item at position 3:
DELETE /list/?position=3
Of course, your API should update the indexes of all the elements when doing insertion and deletion.
Hash table
Get item "somekey":
GET /hashtable/somekey
Add item "somekey":
POST /hashtable/somekey
Remove item "somekey":
DELETE /hashtable/somekey
#dadads
You can not define such interface directly.
An ordered list (get, add, insert into position, reorder, remove)
By excluding "insert into position" and "reorder" you can perfectly implement "get", "add" and "remove" for example:
You define your resource /service/users
You can use POST /service/users to add new user to the "users" collection
You can GET /service/users to retrieve users
You can GET /service/users/user-id to retrieve particular user
You can DELETE /service/users/user-id from users collection
This is a very rough example, though it outlines some ideas. In order to achieve "reorder" and "insert into position" you need to implement your own action semantics which you can include in your resource representation and let client know HOW to perform these operations. As a reference you can see this JSON PATCH specification proposal: https://www.rfc-editor.org/rfc/rfc6902 which tries to describe such operations.
It is not necessary to use already existing media format, you can define your own under your own namespace for example: application/vnd.your-company.format-name+json which describes these capabilities and also advertises this information to clients.
You should decouple the transport mechanism from the underlying application. I would consider designing the application correctly, then figure out how to access it via HTTP. This way you could easily add or change the transport mechanisms (SOAP, SCA, etc) without affecting the underlying application.
Once you have the application correctly designed, consider accessing it from the HTTP requests via something like an Adapter or Visitor pattern.
This is my idea for reordering.
There is a HTTP method called PATCH that is used to update fragments of a resource. Give your resource a new property called index, then make a call with PATCH method
PATCH /collection
[
{
"id: "original index 0"
"index": 1
}
{
"id: "original index 1"
"index": 0
}
]
Then your server back-end needs to figure out how to do this atomically. But interface-wise, I think this is the best way to stay true to RESTful.
Alternatively, there is a better solution, but it may not apply to everyone's case. Since ordering always depends some sort of criteria, it can even be as simple as insertion order. Let your collection url support an orderBy query string, and let this orderBy dictate on how the result gets ordered. Then during your reordering call from client, just update the resource's property used for the ordering criteria.
I came to this question mostly looking for a RESTful way to reorder. I don't really like any of the answers, so here is what I think is most RESTful.
For reorder you could make the order a resource:
/list/order
Then you can do normal operations on it (for these examples assume a list with 5 items currently in it):
"items":" [
{
"id": "A",
"name": "Monkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "D",
"name": "Turkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]
Note that "order" is not included in the resource response. It's not needed - the order is implicitly specified by the response order of the items.
GET /list/order
returns a list of item ids in their correct order
['A','B','C','D','E']
POST /list/order
with payload ['D','B','C','A','E']
GET /list/order
returns a list of item ids in their correct order
['D','B','C','A','E']
Also obviously you would return the items in the list in the correct order when you do a GET on /list.
GET /list
returns a list of items in their correct order
"items":" [
{
"id": "D",
"name": "Turkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "A",
"name": "Monkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]
Trying to get some movies and their genres but leave out any records that contain the genre "Thriller" in the array of genres.
How do I not only ignore the genre key itself for "Thriller", but squelch that entire movie result? With my current query, Thriller is removed from the array of genres, but the parent object (film) is still displayed.
Here's my current workup in the query editor:
http://tinyurl.com/d2g54lj
[{
'type':'/film/film',
'limit':5,
'name':null,
'/film/film/genre': [],
'/film/film/genre!=': "Thriller",
}]
The answer provided is correct, but changes some other stuff in the query too. Here's the direct equivalent to the original query:
[{
"type": "/film/film",
"limit": 5,
"name": null,
"genre": [],
"x:genre": {"name":"Thriller",
"optional":"forbidden"},
}]
The important part is the "optional":"forbidden". The default property used is "name", but we need to specify it explicitly when we use a subclause (to allow us to specify the "optional" keyword). Using ids instead of names, as #kook did, is actually more reliable, so that's an improvement, but I wanted people to be able to see the minimum necessary to fix the broken query.
We can abbreviate the property name to "genre" from "/film/film/genre" since "type":"/film/film" is included (we also never need to use /type/object for properties like /type/object/name).
Answering my own question.
So the trick is to not use the != (but not) operator, but to actually flip it on its head and use the "|=" (one of) operator with 'forbid', like so:
[{
'type':'/film/film',
'limit':5,
'name':null,
'/film/film/genre': [{
"id": null,
"optional": true
}],
"forbid:/film/film/genre": {
"id|=": [
"/en/thriller",
"/en/slapstick"
],
"optional": "forbidden"
}
}]
Thanks to the following post:
Freebase query - exclusion of certain values