I totally made up the name "type specifiers." What I mean is the stringValue key in front of a value. Usually I would expect a more-standard response: "name" : "name_here".
{
"fields": {
"name": {
"stringValue": "name_here"
}
}
}
Is it possible to remove those when making a GET call?
More importantly, it be nice to understand why it's structured like it is. Even for POST-ing data? The easy answer is probably because Cloud Firestore, unlike Realtime Database, needs to know the specific types, but what are all the deeper reasons? Is there an "official" name for formatting like this where I could do more research?
For example, is the reasoning any related to Protocol Buffers? Is there a way to request a protobuf instead of JSON?
Schema:
Is it possible to remove those when making a GET call?
In short No. The Firestore REST API GET returns an instance of Document.
See https://firebase.google.com/docs/firestore/reference/rest/v1beta1/projects.databases.documents#Document
{
"name": string,
"fields": {
string: {
object(Value)
},
...
},
"createTime": string,
"updateTime": string,
}
Regarding the "Protocol Buffer": When the data is deserialized you could just have a function to convert into the structure you wish to use, e.g. probably using the protocol buffers if you wish but as there appear to be libraries for SWIFT, OBJECTIVE-C, ANDROID, JAVA, PYTHON, NODE.JS, GO maybe you won’t need to use the REST API and craft a Protocol Buffer.
Hopefully address your “More Importantly” comment:
As you eluded to in your question Firestore has a different data model to the Realtime Database.
Realtime database data model allows JSON objects with the schema and keywords as you want to define it.
As you point out, the Firestore data model uses predefined schemas, in that respect some of the keywords and structure cannot be changed.
The Cloud Firestore Data Model is described here: https://firebase.google.com/docs/firestore/data-model
Effectively the data model is / where a document can contain a subcollection and the keywords “name”, “fields”, “createdTime”, “upTime” are in a Firestore document (a pre-defined JSON document schema).
A successful the Firestore REST API GET request results in a Document instance which could contain collection of documents or a single document. See https://firebase.google.com/docs/firestore/reference/rest/. Also the API discovery document helps give some detail about the api:
https://firestore.googleapis.com/$discovery/rest?version=v1beta1
An example REST API URL structure is of the form:
https://firestore.googleapis.com/v1beta1/projects/<yourprojectid>/databases/(default)/documents/<collectionName>/<documentID>
It is possible to mask certain fields in a document but still the Firestore Document schema will persist. See the three examples GET:
collection https://pastebin.com/98qByY7n
document https://pastebin.com/QLwZFGgF
document with mask https://pastebin.com/KA1cGX3k
Looking at another example, the REST API to run Queries
https://firebase.google.com/docs/firestore/reference/rest/v1beta1/projects.databases.documents/runQuery
the response body is of the form:
{
"transaction": string,
"document": {
object(Document)
},
"readTime": string,
"skippedResults": number,
}
In summary:
The Realtime database REST API will return the JSON for the object according to the path/nodes as per your “more-standard response”.
The Firestore REST API returns a specific Firestore predefined response structure.
There API libraries available for several language so maybe it’s not necessary to use the REST API and craft your own Protocol Buffer but if you needed to you it’s probably feasible.
I don't understand why somebody just say that you can't and don't try think some solution for help! Seriously that this is a really problem solver?
Anyway, I created a script that will help you (maybe it's late now hahaha).
The script encode json and after replace it as string to modify and remove Google type fields (low process).
It's a simple code, I know that you can improve it if necessary!
WARNING!!
Maybe you will have problems with values that contain '{}' or '[]'. This can be solved with a foreach that convert all strings that contains this elements in other char (like '◘' or '♦', some char that you know that doesn't will be in value.
Ex.: Hi {Lorena}! ------> Hi ◘Lorena♦!
After the process, convert again to '{}' or '[]'
YOU CAN'T HAVE FIELDS WITH THE SAME NAME THAT GOOGLE FIELDS
Ex.: stringValue, arrayValue, etc
You can see and download the script in this link:
https://github.com/campostech/scripts-helpers/blob/master/CLOUD%20STORE%20JSON%20FIELDS%20REMOVER/csjfr.php
Related
Users of our app can add other users as friends. Each friends list is implemented as a map where the key is the id of the users and where the value is some data related to the user. The following map is an example of how it looks like :
{'id_1' : {displayName: 'John Doe', color: 3412445}, 'id_2' : {displayName: 'Bob Alison', color: 84655467}}
We want to add rules to make data validation on the fields in the values of the map. The displayName needs to be a string and the color needs to be a number.
In the firestore rules we can check the new added friends by doing like so :
let addedFriendsKeys = newFriendsList.diff(oldFriendsList).addedKeys()
But is there a way to retrieve the values related to that list of keys?
something like :
let newFriendsList.getAll(addedFriendsKeys).forEach((p0)=>isDataValid(p0))
There are no looping operations in Firestore security rules. You will have to enumerate the keys that you want to validate.
The following answer is not mine (Credits to
Frank van Puffelen). It's coming from this post but answers perfectly to the question :
If you're asking about doing this in server-side security rules, then you've precisely hit the nail on the head: there is no ability to loop in Firebase's server-side security rules. See the reference documentation for the operations that can be performed on a List in a document. This limits what can be accomplished in security rules, and as far as I can see none of the use-cases you mention can be implemented with just security rules.
The simplest approach I can think of is by using Cloud Functions to implement the logic. You could either have the Cloud Function inspect the documents in place in the current collection, or you can have the client write to a different collection (of "pending" documents), have the Cloud Function validate the document, and move it to the actual collection.
I am actually looking for a way to validate elements in json. I thought there is a way to list them out to strictly avoid accepting wrong elements. For instance instead of "gender": "male" as illustrated below, someone could send "sex": "male" and I am trying to avoid it.
I have a data field (column) called Profile
profile = {'name': 'Payne', 'gender': 'male', 'favourites': [{'drinks': 'soda'}, {'colour': 'blue'}, {'game': 'scrabble'}], 'dob': '1962'}
I am using a third party API to populate the database using HttpClient.
My response is returning JSON and I want to make some decisions with it and store it in the database but I need to validate it in conformity with what is expected strictly.
About validation:
If you know how to do it with arrays, you can still decode it and validate it as an array, then encode it again. Symfony has a validator service but I do not know exactly how to correctly use it in all cases.
The official Symfony documentation for the Validator Service and how to use it can be found in this link and it's anchor:
https://symfony.com/doc/current/validation.html
https://symfony.com/doc/current/validation.html#using-the-validator-service
Some info about JSON in PHP:
The most typical use for json, even for updating and downloading content to and from a DataBase, is the use of json_encode() and json_decode(), which will basically help you make your arrays more "portable" in different use cases.
For example, in MySQL, which accepts the JSON type, you can insert arrays into a column by encoding them with JSON.
If you want to declare a JSON type variable in Symfony, you can do it as in this example:
/**
* #ORM\Column(type="json", ...)
*/
private $yourColumn;
Both json_encode() and json_decode() are available since PHP v5.2.0
As an example of a DB, the JSON type was added to MySQL in version 5.7.8 (https://dev.mysql.com/doc/refman/5.7/en/json.html)
You should take a look at these links:
https://www.php.net/manual/es/function.json-encode.php
https://www.php.net/manual/es/function.json-decode.php
https://www.w3schools.com/js/js_json_php.asp
https://dev.mysql.com/doc/refman/8.0/en/
I cannot find a solution for validating the contents of Lists in firestore. Is there any solution for doing this?
I don't want to use a new collection as it doesn't make sense for such a small amount of data (and will end up costing many more reads than necessary, and is much more convenient to keep the data under the original document).
Storing a few image details per document:
[
{
"filename": "example.png",
"author": "example",
"caption": "...",
},
...
]
or, storing a few tags per document:
[
"tag text 1",
"tag text 2",
...
]
How can I access each item (or loop through?) to validate them, so that in the first example with the images, it can be validated so (for example):
each item is a map,
filename is a string,
author is a string,
caption is a string of a particular length
Or, for the tags example,
each item is a string of a particular length?
Or is there a better solution to storing this sort of data in a document without creating a new collection or subcollection?
How can I access each item (or loop through?) to validate them.
If you're asking about doing this in server-side security rules, then you've precisely hit the nail on the head: there is no ability to loop in Firebase's server-side security rules. See the reference documentation for the operations that can be performed on a List in a document. This limits what can be accomplished in security rules, and as far as I can see none of the use-cases you mention can be implemented with just security rules.
The simplest approach I can think of is by using Cloud Functions to implement the logic. You could either have the Cloud Function inspect the documents in place in the current collection, or you can have the client write to a different collection (of "pending" documents), have the Cloud Function validate the document, and move it to the actual collection.
I haven't been able to find a clear answer, hoping someone can help.
If we have, say, a blog with posts, and each post can have comments, and each comment a related user. Is it against convention, if I request the comment, to include both the user and the post in the relationships?
data: {
type: 'comments',
id: '1',
relationships: {
post: {...}, //should this be here?
user: {...},
}
attributes: {...},
},
included: {...}
As paulsm4 correctly stated: "it is up to you".
But I can give you some advice about that.
In such situations, you can give the caller of the API the choice of having such links or not via a querystring flag, like
?relationships=post,user
In this case, if you do not specify the relationship flag, you'll get the simple comment data or you can decide to give them all; in the second case, you can use relationships as a sort of filter.
In some APIs, I've seen also a more invasive approach: embed the related object directly in the returned JSON.
With the same technique as before:
?embed=post,user
This should produce an embedded JSON object in the current JSON reply including the original objects just as you were asking something like "GET /post/123" or "GET /user/456" separately. This can be handy in some situations.
Often this flag is named "expand" denoting same or similar behaviour.
For an example open this API documentation from Atlassian and search for "expand".
It does exist an old "standard" for your problem called HAL that speaks about linking and embedding in REST APIs.
Even the Wordpress API offers such features, give it a look in the official documentation.
An alternative to this is to rewrite the entire API in GraphQL leveraging the REST approach.
Q: Should JSON API entities include a relationship for its parent?
A: I assume that's entirely up to you!
If your JSON is defined by some third party, then you have to live with what they gave you. Please post details on how the JSON is specified.
Otherwise, if you're "inventing" the format yourself:
One possibility is to have a relationships: field with a link to the "parent".
Perhaps a better solution might to invent a "container" (perhaps a simple array!) to hold your "records".
If this were a database, I'd have a "posts" table, and a "comments" table. The "comments" table would have a "Post ID" column as a foreign key into the "posts" table.
'Hope that helps ... at least a bit...
JSON API specification does not make any requirements on the attributes and relationships being included in a resource object. The specification is just saying how they must be formatted if they are included. If I did not missed anything, specification does not even require that all resource objects of the same type must have same attributes and relationships.
But I would argue that there isn't any value in not including the relationships. JSON API specification does not require a relationship object to include resource linkage data. On the contrary it's only talking about resource linkage data in context of a compound document, in which it's used "to link together all of the included resource objects without having to GET any URLs via links."
It's totally valid and could be considered best practice to only provide related resource link if the related resources are not included in the payload. Constructing such a link would not put any workload on your server since it does not require to query the database. It also does not make any relevant difference in payload size.
An example of a payload using both techniques would look like this. It assumes that the request explicitly asked to include related user using include query param.
// GET https://examples.com/api/v1/comments/1?include=user
{
data: {
type: 'comments',
id: '1',
relationships: {
post: {
links: {
related: 'https://examples.com/api/v1/comments/1/post'
}
},
user: {
data: {
type: 'users',
id: '2'
},
links: {
related: 'https://examples.com/api/v1/comments/1/user'
}
},
}
},
included: [
{
type: 'users',
id: '2',
attributes: {
name: 'John Doe'
}
}
]
}
You may also want to include a relationship link, which "allows the client to directly manipulate the relationship." Update relationships chapter of spec gives a deep dive into what you could accomplish using relationship links.
I'm developing an application using Sails.js in the backend, and I'm having trouble validating requests as follows:
I want to block access to certain resources based on an attribute of the logged in user. I have the REST API blueprints enabled, and the shortcut routes disabled. I have the following piece of code:
User.findOne()
.where(query)
.exec(function(err, user) {
if (user.team !== req.user.team) {
return res.view('403');
}
return next();
});
where query is the criteria by which I'd like to do the database search. The idea is that the user can only access the requested user if they're in the same "team".
The problem is that the user can make at least the following kinds of requests to the backend, maybe more (I'm not too experienced with sails, please enlighten me if there's even more types):
localhost:1337/user?id=1
In this case, the req object will have a req.query attribute, which I can pass on as query as it is. Now, the following kind of request is also possible:
localhost:1337/user/1
Here req.query will be an empty object, while req.params is [ id: '1'].
Now this is troublesome; if I understand correctly, the the type of req.params isn't a JSON object, so I can't pass it as query as it is. In addition, I'd have to convert the id parameter into Int since it's originally a string for some reason (?).
What I'm asking is if there's a way I may have missed that handles both kinds of requests in the same way, or whether I'll have to take both cases into account in a way like
var query = isEmpty(req.query) ? req.params : req.query
in which case I'd have to convert req.params into something I could pass to the database search and would generally be troublesome and inconvenient. I'd like to know what the best way to handle this situation is.
Well, it's funny how right after posting a question you happen to find an answer. Apparently there's a function called req.allParams() which "Includes parameters parsed from the url path, the query string, and the request body." according to the official docs. I've no idea how I never bumped into this before, but now I did and it seems to work, so hooray!