Related
I'm making an app with database structure like this:
{
"Locations": {
"location1": {
"name": "Nice location"
}
},
"User_posts": {
"user1": {
"post1": {
"location_name": "Nice location",
"location_id": "location1",
"description": "Wow!"
},
"post2": {
"location_name": "Nice location",
"location_id": "location1",
"description": "Nice"
}
}
}
If I have to change location1 name, how to change all location_name's that all users posts have? I have to download all the data before and update it or there is other method?
I think that using location id only to get location name for every location when user enters his posts is not a good idea.
By duplicating data you improve your read performance/scalability at the cost of decreased write performance. This is a normal trade-off in NoSQL databases and in highly scaleable systems in general.
If you want to update the location_name of all posts, you will indeed have to query the posts and update each. If you need to do this regularly, consider keeping a separate lookup list for each location to find the posts where it used. Such an inverted index is another common occurrence in NoSQL databases.
I covered strategies for updating the duplicated data in my answer here: How to write denormalized data in Firebase
Coming from a relational/SQL background, this may initially feel uncomfortable, since it goes against the normalization rules we've been taught. To counter that feeling, I recommend reading NoSQL data modeling, watching Firebase for SQL developers and in general just read some more NoSQL data modeling questions.
You can add one more attribute to location1 , say isLocationOf , which will store all the user id or perhaps post id/post names. Like
"Locations": {
"location1": {
"name": "Nice location",
"isLocationOf": {
'post1': true,
'post2': true
}
}
}
Here isLocationOf is an attribute of Locations whose value is an object.Now if locations1's name gets changed then you can retrieve its isLocationOf object , iterate through it , get all posts id/name containing that location.Then use the post ids to update all entries having this address .
Also whenever you add new post , you have to add its post id/name to isLocation object.
I'm creating my first app in Firebase. I have no experience with NoSQL, so working out my data structure is proving to be a challenge. Let's say my app is similar Reddit where users visit the site and read/write posts. I want the app to have a list view where it sorts the post data in several ways, however it is all centered around the date posts where submitted:
Views
Show the latest posts in descending order.
Show the latest posts for a specific tag.
Show the most liked posts in descending order for the last day (24 hours).
I assume the data structure to look this:
{
"posts": {
"post_0": {
"content": "...",
"created_at": 1497112445748,
"likes": 100,
"tags": {
"tag_0": true,
"tag_2": true
}
},
"post_1": {
"content": "...",
"created_at": 1497112549374,
"likes": 30,
"tags": {
"tag_1": true
}
},
"post_2": {
"content": "...",
"created_at": 1497112640376,
"likes": 70,
"tags": {
"tag_1": true,
"tag_2": true
}
},
...
}
}
View 1
This is probably the easiest to resolve. I imagine the script to retrieve the data would be something like this:
const ref = firebase.database().ref("posts");
const query = ref.orderByChild("created_at").limitToLast(50);
query.on("child_added", (snapshot) => {
// Do stuff like add to array for sorting
});
View 2
This is where things get tricky. Since you can only have one orderBy* per query, the only way I can see to pull this off is to have a tags node that duplicates the date and post ID. For example:
{
"tags": {
"tag_2": {
"post_0": {
"created_at": 1497112445748
},
"post_2": {
"created_at": 1497112640376,
}
},
...
}
}
I've read this is the whole concept of denormalization and structuring your data around your views, but isn't there a better way?
View 3
I don't know how to solve this one at all. As the last 1 day is changing every time the view is requested and the likes are fluctuating often, how can I possibly structure my data around this view?
I've read that push keys, which would take place of the post_n key I have in my example, are sequential and can somewhat be relied on as a timestamp. I'm not sure if there's some way to take advantage of that.
I've found a few useful videos by the Firebase team and articles on Medium, but I'm afraid they don't go far enough for me to understand how to accomplish the needs of my app.
Common SQL Queries converted for the Firebase Database
Firebase Data Structures: Pagination
I'm just find this aspect of Firebase really confusing to get my head around to have it return the data I need for my views.
If anybody can provide me with an example of how to accomplish these things, it would be much appreciated! Thanks!
I have a GA account, with defined properties and views. Now, I gave viewing rights for a few users to a specific view. How can I construct/get programmatically a direct URL that will bring those users right to that view/report?
Thanks a lot!
First lets take a look at an typical report url for a specific view:
https://analytics.google.com/analytics/web/#report/visitors-actives/a40777649w70913173p73156703/
Notice the pattern:
BASE_URL = 'https://analytics.google.com/analytics/web/#report/'
REPORT_TYPE = 'visitors-actives/'
ACOUNT_ID = '40777649'
WEBPROPERTY_ID = '70913173'
PROFILE_ID = '73156703' # Also called the view Id.
You can retrieve this information programmatically by calling the Account Summaries: list API method which returns a list of Account Summaries:
{
"id": string,
"kind": "analytics#accountSummary",
"name": string,
"starred": boolean,
"webProperties": [
{
"kind": "analytics#webPropertySummary",
"id": string,
"name": string,
"internalWebPropertyId": string,
"level": string,
"websiteUrl": string,
"starred": boolean,
"profiles": [
{
"kind": "analytics#profileSummary",
"id": string,
"name": string,
"type": string,
"starred": boolean
}
]
}
]
}
The ACCOUNT_ID is the top level acountSumaries.id.
The WEBPROPERTY_ID is the accountsumaries.webproperties[X].internalWebPropertyId.
The PROFILE_ID is the accountsumaries.webproperties[X].profiles[X].id
Now with this information you can recustruct the URL link to the report of interest for a particular view.
FULL_URL = BASE_URL + REPORT_TYPE + 'a' + ACCOUNT_ID + 'w' + WEBPROPERTY_ID + 'p' + PROFILE_ID + '/'
Further to Matt's brilliant answer, you can use the "Try this API" section in their documentation here to get this information without writing a line of code:
https://developers.google.com/analytics/devguides/config/mgmt/v3/mgmtReference/management/accountSummaries/list
There is also some code snippets to do this programmatically.
Don't forget you will need to be logged in with an account with GA access for this to work.
I have a few points to add to Matt and Adam's answers:
I chose to build a generic URL for the main view instead of a report. That way a user can navigate to a report of their choosing. The URL structure is https://analytics.google.com/analytics/web/#/report-home/a[account id]w[internal web property id]p[view id] Important: a user has to have at least READ_AND_ANALYZE permissions for the web property in order to access its default view.
internalWebPropertyId is a resource in the web properties collection and can be obtained through various GA Management API calls. For example, I extracted this value from the response object after creating a new web property using insert call.
I want to create a dashboard which shows information about a limited set of request values :
request:("/path1" OR "/path2" OR "/path3")
What I've tried so far:
I can add filters to the dashboard by clicking on a part of a pie chart, but all these filters are applied as AND filters and not OR. This way of working also requires actual data for all possible request values. Which is not always the case in a test environment.
in Discover I created a saved search but I don't know how I can apply this to my Dashboard so it gets part of the dashboard definition.
Is their a way to do this using the Dashboard editor or does it require some json scripting via Settings->Objects->Dashboards ? If so can you point me a good reference to this (escaped) syntax ?
In Kibana 3 you could define filters of type "either". Does this functionality exist in Kibana 4 ?
I'm using Kibana 4.0.2
I am not sure if this is an answer to your actual question, I'll write it anyway as someone may benefit and I found examples on the Kibana filter syntax to be elusive when googling.
I am trying to define a boolean filter instead of a boolean query in my Discover tab, to unclutter the search field and fascilitate further filtering on a limited set of values.
I found this link to the documentation where AND, OR, NOT filter syntax is described. After a bit of experimenting this was what worked for me, example:
I have a field named host containing the name of the server shipping the log entry. There are quite a few servers, each belonging to one of several redundancy groups. To filter only for log entries produced by the servers "SERVER06 OR SERVER07 OR SERVER08" which happen to belong to a distinct redundancy group B-Servers I can make an OR filter like so:
{
"bool": {
"should": [
{
"query": {
"match": {
"host": {
"query": "SERVER06",
"type": "phrase"
}
}
}
},
{
"query": {
"match": {
"host": {
"query": "SERVER07",
"type": "phrase"
}
}
}
},
{
"query": {
"match": {
"host": {
"query": "SERVER08",
"type": "phrase"
}
}
}
}
]
}
}
and save it as a search called B-Servers. Now I get a filtered list, where I can cherry pick a server with a further and more restrictive filter. Before I had all servers and the quick count only listed the five top entries, so I had to pick one and then edit the filter manually if my target wasn't in the list.
This should be useful for other string type fields too. The documentation should have included a couple of more examples I think, to set the context for the placement of the bool statement and not just a demonstration of the principle.
This link was also useful for demonstrating how to do booleans from the search field rather than as a filter.
[EDIT] An update for Kibana 5.2 as I could not get the previous syntax to work. The following did the trick with 5.2, I used this link to figure it out:
{
"query": {
"bool": {
"should": [
{
"match": {
"host": "SERVER06"
}
},
{
"match": {
"host": "SERVER07"
}
},
{
"match": {
"host": "SERVER08"
}
}
],
"minimum_should_match": 1
}
}
}
Kibana 4 is a total rewrite and apparently not all Kibana 3 features are yet implemented. I've found an "enhancement" ticket in the Kibana github:
https://github.com/elastic/kibana/issues/3693
This closes my question for the moment.
Definitely you can add OR filters in your dashboard. As dashboard is created from saved visualizations, In your visualization you can add filter containing OR which will reflect such data.
As per my understanding of your question I am posting my answer (Feel free to correct me):-
Clicking on pie chart under visualize tab (Create a new visualization).
Selecting a search source - From a new search
Go to Split Slices, select aggregation as filters.
Under Query1 enter the filter you want to apply such as request:("/path1" OR "/path2")
Click on add Filter and add Query2 as: request:("/path2" OR "/path3")
Click on Apply to view the changed pie chart as per filters.
Save Visualization by selecting it from toolbar (2nd option beside search bar).
Go to Dashboard & Click on Add Dashboard & select your saved visualization which will reflect your pie chart.
Hope it answers your question :)
The lucene syntax is "OR" (uppercase), so "foo OR bar".
Once your query is corrected, you can save the search in the Discover tab and refer to that saved search in your Visualization.
You can also manually set the query in the visualization if you don't want the overhead of saving it separately.
I tried Googling and searching everywhere, but couldn't find a definitive authority on this topic. While being true to REST principles, how should I design the HTTP interface for:
An ordered list (get, add, insert into position, reorder, remove)
A set (get, add, remove)
A hash-table (get, add, remove)
NOTE: These data structures are to contain references to existing resources with known ids
That's how I would do it for an ordered list and hash table. I guess the methods would be the same for a set and a list:
Ordered list
Get item 123:
GET /list/123
Append an item to the list:
POST /list/
Insert new item into position 5:
POST /list/?position=5
Move item 123 to position 3:
PUT /list/123?position=3
Delete item 123:
DELETE /list/123
Delete item at position 3:
DELETE /list/?position=3
Of course, your API should update the indexes of all the elements when doing insertion and deletion.
Hash table
Get item "somekey":
GET /hashtable/somekey
Add item "somekey":
POST /hashtable/somekey
Remove item "somekey":
DELETE /hashtable/somekey
#dadads
You can not define such interface directly.
An ordered list (get, add, insert into position, reorder, remove)
By excluding "insert into position" and "reorder" you can perfectly implement "get", "add" and "remove" for example:
You define your resource /service/users
You can use POST /service/users to add new user to the "users" collection
You can GET /service/users to retrieve users
You can GET /service/users/user-id to retrieve particular user
You can DELETE /service/users/user-id from users collection
This is a very rough example, though it outlines some ideas. In order to achieve "reorder" and "insert into position" you need to implement your own action semantics which you can include in your resource representation and let client know HOW to perform these operations. As a reference you can see this JSON PATCH specification proposal: https://www.rfc-editor.org/rfc/rfc6902 which tries to describe such operations.
It is not necessary to use already existing media format, you can define your own under your own namespace for example: application/vnd.your-company.format-name+json which describes these capabilities and also advertises this information to clients.
You should decouple the transport mechanism from the underlying application. I would consider designing the application correctly, then figure out how to access it via HTTP. This way you could easily add or change the transport mechanisms (SOAP, SCA, etc) without affecting the underlying application.
Once you have the application correctly designed, consider accessing it from the HTTP requests via something like an Adapter or Visitor pattern.
This is my idea for reordering.
There is a HTTP method called PATCH that is used to update fragments of a resource. Give your resource a new property called index, then make a call with PATCH method
PATCH /collection
[
{
"id: "original index 0"
"index": 1
}
{
"id: "original index 1"
"index": 0
}
]
Then your server back-end needs to figure out how to do this atomically. But interface-wise, I think this is the best way to stay true to RESTful.
Alternatively, there is a better solution, but it may not apply to everyone's case. Since ordering always depends some sort of criteria, it can even be as simple as insertion order. Let your collection url support an orderBy query string, and let this orderBy dictate on how the result gets ordered. Then during your reordering call from client, just update the resource's property used for the ordering criteria.
I came to this question mostly looking for a RESTful way to reorder. I don't really like any of the answers, so here is what I think is most RESTful.
For reorder you could make the order a resource:
/list/order
Then you can do normal operations on it (for these examples assume a list with 5 items currently in it):
"items":" [
{
"id": "A",
"name": "Monkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "D",
"name": "Turkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]
Note that "order" is not included in the resource response. It's not needed - the order is implicitly specified by the response order of the items.
GET /list/order
returns a list of item ids in their correct order
['A','B','C','D','E']
POST /list/order
with payload ['D','B','C','A','E']
GET /list/order
returns a list of item ids in their correct order
['D','B','C','A','E']
Also obviously you would return the items in the list in the correct order when you do a GET on /list.
GET /list
returns a list of items in their correct order
"items":" [
{
"id": "D",
"name": "Turkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "A",
"name": "Monkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]