Related
If my data structure looks like this:
documents.json:
{
"-NG8qzvgs46A5gojZbJO": {
"-NG8r-2q1-47MWK35aT2": {
"description": "My Description",
"title": "My Title"
},
"author": "jim",
"date": "05/11/2022"
},
"-NG8ta4xpHGZxA4JRUQZ": {
"-NG8ta9e90ChdMQclirn": {
"description": "My Description",
"title": "My Title tom"
},
"author": "tom",
"date": "04/11/2022"
},
"-NG8tjfP_TYJHZcjouY8": {
"-NG8tjiryoxnWbb4wwQQ": {
"description": "My Description ccc",
"title": "My Title jim"
},
"author": "jim",
"date": "05/11/2022"
}
}
How to get all the entries where author="jim"?
This does not work:
https://testing-11f41-default-rtdb.firebaseio.com/documents.json?orderBy="author"&equalTo="jim"
Is it possible using Firebase REST API?
Thanks in advance.
When I access the URL you give, I get this response:
{
"error" : "Index not defined, add ".indexOn": "author", for path "/documents", to the rules"
}
As this message says, you need to add an index to allow this query.
In your rules:
{
"rules": {
...
"documents": {
".indexOn": "author"
}
}
}
The following is my Company object that I store in Cosmos DB. I have all the essential information about employees in Employees property. I also have a Departments property that both defines departments as well as its members.
{
"id": "company-a",
"name": "Company A",
"employees": [
{
"id": "john-smith",
"name": "John Smith",
"email": "john.smith#example.com"
},
{
"id": "jane-doe",
"name": "Jane Doe",
"email": "jane.doe#example.com"
},
{
"id": "brian-green",
"name": "Brian Green",
"email": "brian.green#example.com"
}
],
"departments": [
{
"id": "it",
"name": "IT Department",
"members": [
{
"id": "john-smith",
"name": "John Smith",
"isDepartmentHead": true
},
{
"id": "brian-green",
"name": "Brian Green",
"isDepartmentHead": false
}
]
},
{
"id": "hr",
"name": "HR Department",
"members": [
{
"id": "jane-doe",
"name": "Jane Doe",
"isDepartmentHead": true
}
]
}
]
}
I'm trying to return a list of a particular department, including the employee's email which will come from employees property.
Here's what I did but this is including all employees in the output:
SELECT dm.id, dm.name, e.email, em.isDepartmentHead
FROM Companies c
JOIN d IN c.departments
JOIN dm IN d.members
JOIN e IN c.Employees
WHERE c.id = "company-a" AND d.id = "hr"
The correct output would be:
[
{
"id": "jane-doe",
"name": "Jane Doe",
"email": "jane.doe#example.com",
"isDepartmentHead": true
}
]
How do I form my SQL statement to get all members of a department AND include employees' email addresses?
I'm pretty sure you cannot write a query like this. You are trying to correlate data twice in the same query across two arrays which I don't think is possible. (at least I've never been successful doing this).
Even if this was possible though, there are other issues with your data model. This data model will not scale. You also need to avoid unbounded or very large arrays within documents (e.g. employees and departments). You also do not want to store unrelated data in the same document. Objective here is to model data for high concurrency operations in the way you use it. This reduces both latency and cost.
There are many ways in which you can remodel this data. But if this is a very small data set, you could probably do something like this below with a partition key of companyId (assuming that you always query within a single company). This will store all employees for one company in the same logical partition which can store up to 20GB of data. I would also model this such that one document stores data specific to the company itself (address, phone number, number of employess, etc) with the id and companyId having the same value. This lets you do things like store materialized aggregates like # of employees and update it in a transaction. Also, since this approach mixes different types of entities (a bonus for NoSQL database, you need a discriminator property that allows you to filter for specific entities within the container so you can deserialize them directly into your model classes.
Here is a data model you could try (please note, you need to determine if this works for you by scaling it up to the amount of data you believe you will need to store. You also need to test and measure the RU/s cost for the CRUD operations you will execute with high concurrency).
Example company document:
{
"id": "aaaaa",
"companyId": "aaaaa",
"companyName": "Company A",
"type": "company",
"numberOfEmployees: 3,
"addresses": [
{
"address1": "123 Main Street",
"address2": "",
"city": "Los Angeles",
"state": "California",
"zip": "92568"
}
]
}
Then an employee document like this:
{
"id": "jane-doe",
"companyId": "aaaaa",
"type": "employee",
"employeeId": "jane-doe",
"employeeName": "Jane Doe",
"employeeEmail": "jane.doe#example.com",
"departmentId": "hr",
"departmentName": "HR Department",
"isDepartmentHead": true
}
Then last, here's the query to get the data you need.
SELECT
c.employeeId,
c.employeeName,
c.employeeEmail,
c.IsDepartmentHead
FROM c
WHERE
c.companyId = "company-a" AND
c.type = "employee" AND
c.departmentId = "hr"
I'm working on a project which requires the category which google uses to define that business/store. I'm using beautiful soup library but I am unable to extract that specific part of the result. I'll show you with a help of an example using the image below:
click to open the image
when I search "bell canada" for example, I get search results and also you can see a box which google shows on the right side of the page which summaries the search "bell canada".
So my task required, extracting the subtitle under BELL CANADA named as Telecommunications company .
How can I achieve this for any search on google using web scraping.
Make sure you're using the latest version of user-agent e.g Chrome version.
The result will be parsed only if the search result would have a Knowledge graph displayed.
If you need to extract just one element, you can use select_one() method from beautifulsoup which uses CSS selectors to find an element. Sometimes it's better than find() (check out SelectorGadget to grab CSS selectors by clicking on the desired element in your browser)
Code and full example in the online IDE:
from bs4 import BeautifulSoup
import requests, lxml
# https://docs.python-requests.org/en/master/user/quickstart/#passing-parameters-in-urls
params = {
"q": "bell canada", # search query
"gl": "us", # country to search from
"hl": "en" # language
}
# https://docs.python-requests.org/en/master/user/quickstart/#custom-headers
headers = {
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36"
}
response = requests.get("https://www.google.com/search", params=params, headers=headers, timeout=30)
soup = BeautifulSoup(response.text, "lxml")
subtitle = soup.select_one(".wwUB2c span").text
print(subtitle)
Output:
Telecommunications company
Alternatively, you can use Google Knowledge Graph API from SerpApi, it's a paid API with a free plan.
The difference is that you don't need to create a parser from scratch, maintain it, or figure out how to bypass blocks from Google or other search engines.
Example code to integrate:
from serpapi import GoogleSearch
import os
params = {
# https://docs.python.org/3/library/os.html#os.getenv
"api_key": os.getenv("API_KEY"), # your serpapi API key
"engine": "google", # search engine
"q": "bell canada", # search query
"gl": "us", # country to search from
"hl": "en" # language
}
search = GoogleSearch(params)
results = search.get_dict()
result = results["knowledge_graph"].get("type")
print(result)
# Telecommunications company
Full JSON Knowledge Graph output:
{
"title": "Bell Canada",
"type": "Telecommunications company",
"place_id": "ChIJQaXUmDm7j4ARY4k9pa41rcs",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc20b07f241e63c1a8af.png",
"website": "http://www.bell.ca/",
"description": "Bell Canada is a Canadian telecommunications company headquartered at 1 Carrefour Alexander-Graham-Bell in the borough of Verdun in Montreal, Quebec, Canada. It is an ILEC in the provinces of Ontario and Quebec; as such, it was a founding member of the Stentor Alliance.",
"source": {
"name": "Wikipedia",
"link": "https://en.wikipedia.org/wiki/Bell_Canada"
},
"headquarters": "Montreal, Canada",
"headquarters_links": [
{
"text": "Headquarters",
"link": "https://www.google.com/search?q=bell+canada+headquarters&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vS0spOttLPL0pPzMusSizJzM9D4VhlpCamFJYmFpWkFhUvYpVISs3JUUhOzEtMSVRAlgIAJLjc3VQAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ6BMoAHoECFUQAg"
},
{
"text": "Montreal, Canada",
"link": "https://www.google.com/search?q=Montreal&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tS4gAxTY0KzLW0spOt9POL0hPzMqsSSzLz81A4VhmpiSmFpYlFJalFxYtYOXzz80qKUhNzdrAy7mJn4mAAAKldbGxWAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoAXoECFUQAw"
}
],
"ceo": "Mirko Bibic (Jan 2020\u2013)",
"ceo_links": [
{
"text": "CEO",
"link": "https://www.google.com/search?q=bell+canada+ceo&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vSUsxOttLPL0pPzMusSizJzM9D4Vglp-YvYuVPSs3JUUhOzEtMSVQAigAAyDZ50kIAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ6BMoAHoECFoQAg"
},
{
"text": "Mirko Bibic",
"link": "https://www.google.com/search?q=Mirko+Bibic&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tS4tFP1zc0MjWpMKyoSNFSzE620s8vSk_My6xKLMnMz0PhWCWn5i9i5fbNLMrOV3DKTMpM3sHKuIudiYMBAJXG1PxUAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoAXoECFoQAw"
}
],
"revenue": "23.96 billion CAD (2019)",
"revenue_links": [
{
"text": "Revenue",
"link": "https://www.google.com/search?q=bell+canada+revenue&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vSUskot9JPzs_JSU0uyczP088vSk_My6xKBHGKrYpSy1LzSlMXsQonpebkKCQn5iWmJCpARQEd2EpcSQAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ6BMoAHoECFgQAg"
}
],
"founded": "April 29, 1880",
"founded_links": [
{
"text": "Founded",
"link": "https://www.google.com/search?q=bell+canada+founded&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vSUs1OttLPL0pPzMusSizJzM9D4Vil5ZfmpaSmLGIVTkrNyVFITsxLTElUgIoCAKNSAMBKAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ6BMoAHoECFcQAg"
}
],
"subsidiaries": "Bell Mobility, Bell Fibe TV, Bell Media, Bell Aliant, MORE",
"subsidiaries_links": [
{
"text": "Subsidiaries",
"link": "https://www.google.com/search?q=bell+canada+subsidiaries&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vS0swot9JPzs_JSU0uyczP088vSk_My6xKBHGKrYpLk4ozUzITizJTixexSiSl5uQoJCfmJaYkKiBLAQC0i8LLUwAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ6BMoAHoECFYQAg"
},
{
"text": "Bell Mobility",
"link": "https://www.google.com/search?q=Bell+Mobility&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tSAjONLCotk7Q0M8qt9JPzc3JSk0sy8_P084vSE_MyqxJBnGKr4tKk4syUzMSizNTiRay8Tqk5OQq--UmZOZkllTtYGXexM3EwAgDhYoVcWwAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoAXoECFYQAw"
},
{
"text": "Bell Fibe TV",
"link": "https://www.google.com/search?q=Bell+Fibe+TV&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tS4gIxk_MqzIvStDQzyq30k_NzclKTSzLz8_Tzi9IT8zKrEkGcYqvi0qTizJTMxKLM1OJFrDxOqTk5Cm6ZSakKIWE7WBl3sTNxMAIAwCwzNFsAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoAnoECFYQBA"
},
{
"text": "Bell Media",
"link": "https://www.google.com/search?q=Bell+Media&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tSAjONLCotC7Q0M8qt9JPzc3JSk0sy8_P084vSE_MyqxJBnGKr4tKk4syUzMSizNTiRaxcTqk5OQq-qUCBHayMu9iZOBgBIdpKolgAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoA3oECFYQBQ"
},
{
"text": "Bell Aliant",
"link": "https://www.google.com/search?q=Bell+Aliant&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tSAjONs6qKMrQ0M8qt9JPzc3JSk0sy8_P084vSE_MyqxJBnGKr4tKk4syUzMSizNTiRazcTqk5OQqOOZmJeSU7WBl3sTNxMAIAPghHPVkAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoBHoECFYQBg"
},
{
"text": "MORE",
"link": "https://www.google.com/search?q=bell+canada+subsidiaries&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vS0swot9JPzs_JSU0uyczP088vSk_My6xKBHGKrYpLk4ozUzITizJTixexSiSl5uQoJCfmJaYkKiBLAQC0i8LLUwAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ44YBKAV6BAhWEAc"
}
],
"parent_organization": "BCE Inc.",
"parent_organization_links": [
{
"text": "Parent organization",
"link": "https://www.google.com/search?q=bell+canada+parent+organization&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vSMsgot9JPzs_JSU0uyczP088vSk_My6xKBHGKrQoSi1LzShSQBRexyiel5uQoJCfmJaYkKmBRAQDkOSRXYQAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ6BMoAHoECFIQAg"
},
{
"text": "BCE Inc.",
"link": "https://www.google.com/search?q=BCE+Inc.&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tS4tVP1zc0TM6pSs8ryCnSMsgot9JPzs_JSU0uyczP088vSk_My6xKBHGKrQoSi1LzShSQBRexcjg5uyp45iXr7WBl3MXOxMEAAJpETrZhAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoAXoECFIQAw"
}
],
"founders": "Alexander Graham Bell, Charles Fleetford Sise",
"founders_links": [
{
"text": "Founders",
"link": "https://www.google.com/search?q=bell+canada+founders&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0vSUs9OttJPKi3OzEstLoYz4vMLUosSSzLz86zS8kvzUlKLFrGKJKXm5CgkJ-YlpiQqQEWLAVSO90hNAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQ6BMoAHoECFMQAg"
},
{
"text": "Alexander Graham Bell",
"link": "https://www.google.com/search?q=Alexander+Graham+Bell&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tSYgcxs80NtNSzk630k0qLM_NSi4vhjPj8gtSixJLM_DyrtPzSvJTUokWsoo45qRWJILaCe1FiRmKuglNqTs4OVsZd7EwcjADv87IIXwAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoAXoECFMQAw"
},
{
"text": "Charles Fleetford Sise",
"link": "https://www.google.com/search?q=Charles+Fleetford+Sise&stick=H4sIAAAAAAAAAOPgE-LUz9U3MMxJz0tS4gIxjcqrzKoqtdSzk630k0qLM_NSi4vhjPj8gtSixJLM_DyrtPzSvJTUokWsYs4ZiUU5qcUKbjmpqSVp-UUpCsGZxak7WBl3sTNxMAIAu9x5NWMAAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQmxMoAnoECFMQBA"
}
],
"profiles": [
{
"name": "Facebook",
"link": "https://www.facebook.com/BellCanada",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=bell+canada",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc2074c7067793bbe8078324a0faa409aa72ae97cc00dde05d3d.png"
},
{
"name": "LinkedIn",
"link": "https://www.linkedin.com/company/bell",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=bell+canada",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc2074c7067793bbe8077149819227ec3fe420490a616cb7a98b.png"
},
{
"name": "Instagram",
"link": "https://www.instagram.com/bell",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=bell+canada",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc2074c7067793bbe80701ded0bf9c7339eb0750d18f70d41a8e.png"
},
{
"name": "Twitter",
"link": "https://twitter.com/Bell_News",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=bell+canada",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc2074c7067793bbe807ac0352770d2c1364de5875551883bbf0.png"
},
{
"name": "YouTube",
"link": "https://www.youtube.com/user/bellcanada",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=bell+canada",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc2074c7067793bbe807aca6f6c897bc56669105a3fa347dd780.png"
}
],
"people_also_search_for": [
{
"name": "Bell Mobility",
"link": "https://www.google.com/search?q=Bell+Mobility&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1R9UAvwmMtikZ7R6go3IFYLEBqyQrZaG4BLS5p3WFt0rrwQLldwNDkd8S1-6xh6XzROLNmWPjEn63m_JtfsUpeGnJ1sZ5ZrfygGyNjqzZYRQUeiBl6zZ3AHaXiz88P22q3s_YpZGtyao5_7p1pRjhBRMQyhXg-PRV6Xf5Cqukh3H14ooklpf80wV_TODtWV1EKdhiC82QvehgQ6nBqVdkEN0OY9i136n3i8xQnKQVXOmGQKlgnE7ZoUYz6VvineQnxQhTjk%3D&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQxA16BAhUEAU",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=Bell+Mobility&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1R9UAvwmMtikZ7R6go3IFYLEBqyQrZaG4BLS5p3WFt0rrwQLldwNDkd8S1-6xh6XzROLNmWPjEn63m_JtfsUpeGnJ1sZ5ZrfygGyNjqzZYRQUeiBl6zZ3AHaXiz88P22q3s_YpZGtyao5_7p1pRjhBRMQyhXg-PRV6Xf5Cqukh3H14ooklpf80wV_TODtWV1EKdhiC82QvehgQ6nBqVdkEN0OY9i136n3i8xQnKQVXOmGQKlgnE7ZoUYz6VvineQnxQhTjk%3D",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc20c56ba723d853f90fc6315fa2689fcfb45a4b2a45d9fae90403c0d82e4a40d42b.png"
},
{
"name": "Rogers Communications",
"link": "https://www.google.com/search?q=Rogers+Communications&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1dzM_m9DQbE44YUabMcVuyLEzIAhZ3H2Ffy87BzHf7w7Eyu5_QEgCc-vJB0RWaeMzFlsDk1chAWxn8TCFqAnREbojgzTt19P5wVcvFN-qGPr88al2xsnGebQ167KSGOKLrBJhLPuvhjMdtl5xGEenGV8M6hmipQ-UhzJhuTftp1PKWGWSR9ZleVlYAaFN-oPOINFXJ9mt4tOhnfVLP-auA9-ZdIA6-h22xSlZWQqm7W9w12ezMpjfZuQqJO0hOXwoIk__ZO-iER0syJJSIceMmG9NPg2&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQxA16BAhUEAc",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=Rogers+Communications&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1dzM_m9DQbE44YUabMcVuyLEzIAhZ3H2Ffy87BzHf7w7Eyu5_QEgCc-vJB0RWaeMzFlsDk1chAWxn8TCFqAnREbojgzTt19P5wVcvFN-qGPr88al2xsnGebQ167KSGOKLrBJhLPuvhjMdtl5xGEenGV8M6hmipQ-UhzJhuTftp1PKWGWSR9ZleVlYAaFN-oPOINFXJ9mt4tOhnfVLP-auA9-ZdIA6-h22xSlZWQqm7W9w12ezMpjfZuQqJO0hOXwoIk__ZO-iER0syJJSIceMmG9NPg2",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc20c56ba723d853f90fc6315fa2689fcfb4c7166733dc5a6aba38cf161f2a757494.png"
},
{
"name": "Telus",
"link": "https://www.google.com/search?q=Telus&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1avwx721CKW5xFPqY8J24d9HoDRlbSmY_hJ3bLjDSyrYAhUDJ7NPnSUsZP-EodZiQ7lcNnn6zw6GmEG97o7wfrx9GGsw36qfjTUEeKsmbjB1Eu6-3VdGNvabNha80f9AJyZugh2oTfK3Je7UW6NJKddrU1s_rpzpuMpaIaq_XEgweuW_Oh-qyor8ealTV2tasEWtmrNH9ELh8Qr0WjipAWgyWPAOy33BAPYdul4_tNs2tgPq8rQr0SS6yFXbTgS2YySzrH0%3D&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQxA16BAhUEAk",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=Telus&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1avwx721CKW5xFPqY8J24d9HoDRlbSmY_hJ3bLjDSyrYAhUDJ7NPnSUsZP-EodZiQ7lcNnn6zw6GmEG97o7wfrx9GGsw36qfjTUEeKsmbjB1Eu6-3VdGNvabNha80f9AJyZugh2oTfK3Je7UW6NJKddrU1s_rpzpuMpaIaq_XEgweuW_Oh-qyor8ealTV2tasEWtmrNH9ELh8Qr0WjipAWgyWPAOy33BAPYdul4_tNs2tgPq8rQr0SS6yFXbTgS2YySzrH0%3D",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc20c56ba723d853f90fc6315fa2689fcfb4ef927ce4febcc787bb372f154e1ab257.png"
},
{
"name": "Bell Satellite TV",
"link": "https://www.google.com/search?q=Bell+Satellite+TV&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1R_XB3ccGKnGM0ZWrwNs4AfVgIEjRog7cnGYpULPf3NHggGWhZu06mYq6FwuKWiqDluDkFpFfHCqo6EZT-fSbk237oI3ER1G7BPe5c23kxRoP2DjMeuiCj36HGT0G1mHYk4oqhO2mfL5vBe9SNAG9vkMwDPBevv39jCosyf4unz8msh_oPcxbmt9guiYsqPDsuWMW5qbjJk9VnI178i8sIRuww27hhsmsfX7tjqEyIb1yHfG0Jc-1fxB50qXmo8kZ1rhmjQ%3D&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQxA16BAhUEAs",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=Bell+Satellite+TV&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1R_XB3ccGKnGM0ZWrwNs4AfVgIEjRog7cnGYpULPf3NHggGWhZu06mYq6FwuKWiqDluDkFpFfHCqo6EZT-fSbk237oI3ER1G7BPe5c23kxRoP2DjMeuiCj36HGT0G1mHYk4oqhO2mfL5vBe9SNAG9vkMwDPBevv39jCosyf4unz8msh_oPcxbmt9guiYsqPDsuWMW5qbjJk9VnI178i8sIRuww27hhsmsfX7tjqEyIb1yHfG0Jc-1fxB50qXmo8kZ1rhmjQ%3D",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc20c56ba723d853f90fc6315fa2689fcfb438c5d18c0d72fc2c338cfdf89f7d79ed.jpeg"
},
{
"name": "Bell Internet",
"link": "https://www.google.com/search?q=Bell+Internet&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1fuHSkPMK_DYNjyh5MCv1jI4fqIb0Y65kvtJEsFZiVMzyVFwIrZzQDRGHfsOb2zza3VseJPDDPsHk8DFp-GpKcL81MCRae8eeO5kgTN-q6oCXW-8ae-Vk7qtQGOkJ2RrivNU8rh_LlzRAmI57laCuyA5Q7twiXJPUIWn9od4hQKfSJSzQBHc6IeuuNIjv9VHiRJVuoHIVBRTTpdRWXy-_3LUfIfr1DtDobSoycKlY6qR9txE6iT53DlIcIKRLzDAzE5Ra7M%3D&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQxA16BAhUEA0",
"serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&q=Bell+Internet&si=ANhW_NqMq05hm_NHatki2oB83qRn9HU6YTauVAJv82wgMbWR1fuHSkPMK_DYNjyh5MCv1jI4fqIb0Y65kvtJEsFZiVMzyVFwIrZzQDRGHfsOb2zza3VseJPDDPsHk8DFp-GpKcL81MCRae8eeO5kgTN-q6oCXW-8ae-Vk7qtQGOkJ2RrivNU8rh_LlzRAmI57laCuyA5Q7twiXJPUIWn9od4hQKfSJSzQBHc6IeuuNIjv9VHiRJVuoHIVBRTTpdRWXy-_3LUfIfr1DtDobSoycKlY6qR9txE6iT53DlIcIKRLzDAzE5Ra7M%3D",
"image": "https://serpapi.com/searches/624d3ef03d69195733337a68/images/f8d5105f4a126e851d64734cda40fc20c56ba723d853f90fc6315fa2689fcfb4291421c753a92f6fe72135100c226cb4.jpeg"
},
{
"name": "Pacific Bell",
"extensions": [
"Telephone company"
],
"place_id": "7872218611778819527",
"link": "https://www.google.com/search?tbm=lcl&q=Bell+Telephone+Canada&rflfq=1&num=20&stick=H4sIAAAAAAAAACVQu20cMRDFBRYU30oKGG0J8_-kdgMO3MBCWkHB4k7QdeQKXJeq0KM8AEnwceZ9eH83lqwU4QrmzCpulxwPrOLeydlFpD1LgFKqm6szmZZTVRpQcYtMwJpirC4SY2lyYhJGq3t6mPJYhCyMoVhGraVAxxKchvL5ZqnqMaW4wWBW3WyCpTbn2ZOsKFRNfI5jPtt7moLsNylMjDPcoZc54Ngjy2EpBCEhA8JABCWbSYlZBGmSq029kGIs2sTW2AuGJPN_q7kBLzXCSa7FBdQp8TtUDlpRK_gZi4PNlXleiVKiaCwQsuxJw94s7SzojO7yCpKWZFV8-DhLSTbG05E8rIP-nU6fp6ff-_X92NftuF3X2759PL-tr9ePvz8ef-7Hsf7Zj_397XrZ11_bZXvZvgBP7Lvh3wEAAA",
"image": "/maps/vt/data=35Ij67gvdt8gd0NJKUYCZj3ZRJBpEjaJcHDm9Xl3KmNVaH1OJ9CYXbiidynalny2EBchT2rufd2QVU5FUbQfp7nchYu5KUA0_J3a1yYy46SsXK4KkeeXh3rkQndrBx8aAtQLmzqFonP-PQ7C"
},
{
"name": "PLALCA12 Central Office",
"extensions": [
"Telephone company"
],
"place_id": "13255971798003999992",
"link": "https://www.google.com/search?tbm=lcl&q=Bell+Telephone+Canada&rflfq=1&num=20&stick=H4sIAAAAAAAAACVQu20cMRDFBRYU30oKGG0J8_-kdgMO3MBCWkHB4k7QdeQKXJeq0KM8AEnwceZ9eH83lqwU4QrmzCpulxwPrOLeydlFpD1LgFKqm6szmZZTVRpQcYtMwJpirC4SY2lyYhJGq3t6mPJYhCyMoVhGraVAxxKchvL5ZqnqMaW4wWBW3WyCpTbn2ZOsKFRNfI5jPtt7moLsNylMjDPcoZc54Ngjy2EpBCEhA8JABCWbSYlZBGmSq029kGIs2sTW2AuGJPN_q7kBLzXCSa7FBdQp8TtUDlpRK_gZi4PNlXleiVKiaCwQsuxJw94s7SzojO7yCpKWZFV8-DhLSTbG05E8rIP-nU6fp6ff-_X92NftuF3X2759PL-tr9ePvz8ef-7Hsf7Zj_397XrZ11_bZXvZvgBP7Lvh3wEAAA",
"image": "/maps/vt/data=4rthHhRCfZXURzK-kzMzSCtsBPnG4xHtl1JuySthFDTg7w9NM3hPnGZjVRIpR49EMbd6kP1Y1ZDxIjaDdX7v_JGvZLrbdqj5EO5C_wNEL8r8RPtIg_jkAdnhgPvskot35L6unuxB9PJ5phjk"
},
{
"name": "AT&T Internet Service",
"extensions": [
"Internet service provider"
],
"place_id": "10735453510438508874",
"link": "https://www.google.com/search?tbm=lcl&q=Bell+Telephone+Canada&rflfq=1&num=20&stick=H4sIAAAAAAAAACVQu20cMRDFBRYU30oKGG0J8_-kdgMO3MBCWkHB4k7QdeQKXJeq0KM8AEnwceZ9eH83lqwU4QrmzCpulxwPrOLeydlFpD1LgFKqm6szmZZTVRpQcYtMwJpirC4SY2lyYhJGq3t6mPJYhCyMoVhGraVAxxKchvL5ZqnqMaW4wWBW3WyCpTbn2ZOsKFRNfI5jPtt7moLsNylMjDPcoZc54Ngjy2EpBCEhA8JABCWbSYlZBGmSq029kGIs2sTW2AuGJPN_q7kBLzXCSa7FBdQp8TtUDlpRK_gZi4PNlXleiVKiaCwQsuxJw94s7SzojO7yCpKWZFV8-DhLSTbG05E8rIP-nU6fp6ff-_X92NftuF3X2759PL-tr9ePvz8ef-7Hsf7Zj_397XrZ11_bZXvZvgBP7Lvh3wEAAA",
"image": "/maps/vt/data=TMQY8k8OAt51uboJ-OAxE8rQDF7N3xtmkvaaWOdr_uxxnKj5-fs1bPNRvrzDR9NcNNrGvRrcXGbPORZLxV7yrah4W714KMFrSTnPP6E7oU2wO4vFbPBGB2BsvdRTdE0ryTBpjJ35nrCTCL2l"
},
{
"name": "Professional Communications Services/ProComm",
"extensions": [
"Telephone answering service"
],
"place_id": "12546775353724135226",
"link": "https://www.google.com/search?tbm=lcl&q=Bell+Telephone+Canada&rflfq=1&num=20&stick=H4sIAAAAAAAAACVQu20cMRDFBRYU30oKGG0J8_-kdgMO3MBCWkHB4k7QdeQKXJeq0KM8AEnwceZ9eH83lqwU4QrmzCpulxwPrOLeydlFpD1LgFKqm6szmZZTVRpQcYtMwJpirC4SY2lyYhJGq3t6mPJYhCyMoVhGraVAxxKchvL5ZqnqMaW4wWBW3WyCpTbn2ZOsKFRNfI5jPtt7moLsNylMjDPcoZc54Ngjy2EpBCEhA8JABCWbSYlZBGmSq029kGIs2sTW2AuGJPN_q7kBLzXCSa7FBdQp8TtUDlpRK_gZi4PNlXleiVKiaCwQsuxJw94s7SzojO7yCpKWZFV8-DhLSTbG05E8rIP-nU6fp6ff-_X92NftuF3X2759PL-tr9ePvz8ef-7Hsf7Zj_397XrZ11_bZXvZvgBP7Lvh3wEAAA",
"image": "/maps/vt/data=9dPkYHZyTO-CbttEkSVhUJCe91_uc1MydmP8hF5owQhQ7HFwUsUnVO9EIfSzg7FfYoOHaiMETvxEKpHpxsIXNZFTn687rsldtI7GPQEpwwr2SjmKYq25SJpl7otLS6kg031psXCO6xYKhLbW"
},
{
"name": "Fiber Internet Center",
"extensions": [
"Internet service provider"
],
"place_id": "9050102110455756431",
"link": "https://www.google.com/search?tbm=lcl&q=Bell+Telephone+Canada&rflfq=1&num=20&stick=H4sIAAAAAAAAACVQu20cMRDFBRYU30oKGG0J8_-kdgMO3MBCWkHB4k7QdeQKXJeq0KM8AEnwceZ9eH83lqwU4QrmzCpulxwPrOLeydlFpD1LgFKqm6szmZZTVRpQcYtMwJpirC4SY2lyYhJGq3t6mPJYhCyMoVhGraVAxxKchvL5ZqnqMaW4wWBW3WyCpTbn2ZOsKFRNfI5jPtt7moLsNylMjDPcoZc54Ngjy2EpBCEhA8JABCWbSYlZBGmSq029kGIs2sTW2AuGJPN_q7kBLzXCSa7FBdQp8TtUDlpRK_gZi4PNlXleiVKiaCwQsuxJw94s7SzojO7yCpKWZFV8-DhLSTbG05E8rIP-nU6fp6ff-_X92NftuF3X2759PL-tr9ePvz8ef-7Hsf7Zj_397XrZ11_bZXvZvgBP7Lvh3wEAAA",
"image": "/maps/vt/data=BKw9RQ3VtjggI2Pv_zM9xpB8ENL2dEdyZZwgge81TdfAzzgwfw2vF9u1NvnGmqxYdw-oMs0qhdvirPaIFNOLT1tcXe5bvi9pf-f484dREAef1M3xiH5N27GhnBDLov7joDapvJsk_yNeaMWk"
}
],
"people_also_search_for_link": "https://www.google.com/search?q=Bell+Canada&stick=H4sIAAAAAAAAAONgFuLUz9U3MMxJz0tSQjC1BIMzU1LLEyuL_VIrSoJLUguKF7FyO6Xm5Cg4J-YlpiTuYGUEAMq8ASI8AAAA&sa=X&ved=2ahUKEwjRtYnu8v72AhVAg3IEHeNNAtwQMSgAegQIVBAB",
"people_also_search_for_stick": "H4sIAAAAAAAAAONgFuLUz9U3MMxJz0tSQjC1BIMzU1LLEyuL_VIrSoJLUguKF7FyO6Xm5Cg4J-YlpiTuYGUEAMq8ASI8AAAA"
}
Disclaimer, I work for SerpApi.
I use the self href from my HAL resources on the client side to find the right path for the CRUD operations. In single(ton) resources this is working fine (see the address resource below, the _links containing the self href is included in the embedded resource) but when it comes down to collections this is a different story. The _links of a collection are not rendered when the collection is in _embedded.
Earlier I worked around this problem by reading the url form the first child. But this is not sufficient. In case the collection is empty I only have
an empty array with no possibility to extract the url like that. If I want to create a new item in the collection I would like that my client knows where to send data using POST by reading the self href from _links. Is it a good idea to include the _links to my collection like this:
{
"_links": {
"self": {
"href": "http://example.com/api/v1/users/1"
}
},
"_embedded": {
"contacts": {
Now I have access to the self href here:
"_links": {
"self": {
"href": "http://example.com/api/v1/users/1/contacts"
}
},
"_embedded": {
"contacts": [
{
"_links": {
"self": {
"href": "http://example.com/api/v1/users/1/contacts/2"
}
},
"id": "2",
"name": "John Smith"
},
{
"_links": {
"self": {
"href": "http://example.org/api/v1/users/1/contacts/3"
}
},
"id": "3",
"name": "Jane Doe"
}
],
}
},
"address": {
"_links": {
"self": {
"href": "http://example.com/api/v1/addresses/1"
}
},
"street": "Bakerstreet 11",
"postal code": "123456",
"city": "Some city",
"country": "Some country",
}
},
"id": "1",
"name": "John Doe"
}
EDIT (one year later)
In the end I solved this by always adding the links of embedded resources to the parent resource. So in the example above my response object would look like this:
{
"_links": {
"self": {
"href": "http://example.com/api/v1/users/1"
},
"contacts": {
"href": "http://example.com/api/v1/users/1/contacts"
},
"address": {
"href": "http://example.com/api/v1/addresses/1"
}
},
"_embedded": {
"contacts": [
{
"_links": {
"self": {
"href": "http://example.com/api/v1/users/1/contacts/2"
}
},
"id": "2",
"name": "John Smith"
},
{
"_links": {
"self": {
"href": "http://example.org/api/v1/users/1/contacts/3"
}
},
"id": "3",
"name": "Jane Doe"
},
],
"address": {
"_links": {
"self": {
"href": "http://example.org/api/v1/addresses/1"
}
},
"street": "Bakerstreet 11",
"postal code": "123456",
"city": "Some city",
"country": "Some country",
}
},
"id": "1",
"name": "John Doe"
}
So no matter whether I embedded the resources or not I always know where they are located. And for the contacts collection I will have the link to my collection endpoint in the _links array and the contacts themselves in _embedded.
Yes. It's not only good practice to do so; it's recommended by the HAL spec:
Each Resource Object SHOULD contain a 'self' link that corresponds with the IANA registered 'self' relation (as defined by RFC5988) whose target is the resource's URI.
A collection of resources is itself a resource, don't forget.
I want to find the ceo of IBM. What would be the MQL query for this?
The MQL for this search looks like the following.
This particular instance may be a tat more complicated than necessary because I got it initially produced from a Freebase interactive search and then simply added/improved the filters manually.
I verified it with various company names with relative success, i.e. it works provided that the underlying data is properly codified in Freebase (some companies are missing, for some companies the leadership data is incomplete etc.)
There are a few tricks to this query:
the company name in u0 fitler needs to match precisely the company name as recorded in Freebase. You could use a contains predicate rather than an equal one, but that could introduce many irrelevant hits. For example you need to use "IBM", "Apple Inc.", "General Motors" rather than common alternatives to these names ("International Business Machines", "Apple", "GM"...")
the u1 filter, on the leadership role is expressed in a extensive One of predicate because unfortunately the nomenclature for these roles is relatively loose, with duplicates (e.g. could be CEO or Chief Executive Officer) and with the fact that the role of CEO is often coupled with other corporate roles such as Chairman [of the board] and/or President etc. I hand picked this list by first looking up (in Freebase) the instances of Leadership Roles which contained "CEO" or "Chief Executive".
the u2 filter expresses that the to date should be empty, to select only the person currently in office, as opposed to former CEOs (for which hopefully Freebase recorded the end date of their tenure).
Depending on your application, you may need to test that the query returns one and exactly one record, and adapt accordingly if it doesn't.
Freebase MQL editor is a convenient tool test and edit with this kind of queries.
[
{
"from": null,
"id": null,
"limit": 20,
"organization": {
"id": null,
"name": null,
"optional": true
},
"person": {
"id": null,
"name": null,
"optional": true
},
"role": {
"id": null,
"name": null,
"optional": true
},
"s0:type": [
{
"id": "/organization/leadership",
"link": [
{
"timestamp": [
{
"optional": true,
"type": "/type/datetime",
"value": null
}
],
"type": "/type/link"
}
],
"type": "/type/type"
}
],
"sort": "s0:type.link.timestamp.value",
"title": null,
"to": null,
"type": "/organization/leadership",
"u0:organization": [
{
"id": null,
"name": "IBM",
"type": "/organization/organization"
}
],
"u1:role": [
{
"id": null,
"name|=": ["Chief Executive Officer", "President and CEO", "Chairman and CEO", "Interim CEO", "Interim Chief Executive Officer", "Founder and CEO", "Chairman, President and CEO", "Managing Director and CEO", "Executive Vice President and Chief Operating Officer", "Co-Founder, Chairman and Chief Executive Officer"],
"type": "/organization/role"
}
],
"u2:to": [
{
"value": null,
"optional": "forbidden"
}
]
}
]
Sample return (for "IBM", specifically)
{
"code": "/api/status/ok",
"result": [{
"from": "2012-01-01",
"id": "/m/09t7b08",
"organization": {
"id": "/en/ibm",
"name": "IBM"
},
"person": {
"id": "/en/virginia_m_rometty",
"name": "Virginia M. Rometty"
},
"role": {
"id": "/en/chairman_president_and_ceo",
"name": "Chairman, President and CEO"
},
"s0:type": [{
"id": "/organization/leadership",
"link": [{
"timestamp": [{
"type": "/type/datetime",
"value": "2010-01-23T08:02:57.0006Z"
}],
"type": "/type/link"
}],
"type": "/type/type"
}],
"title": "Chairman, President and CEO",
"to": null,
"type": "/organization/leadership",
"u0:organization": [{
"id": "/en/ibm",
"name": "IBM",
"type": "/organization/organization"
}],
"u1:role": [{
"id": "/en/chairman_president_and_ceo",
"type": "/organization/role"
}],
"u2:to": []
}