I'm facing a problem when I'm trying to get the data shallowed from Firebase Database.
Here is the node that I'm trying to shallow:
and here is the response after I've called the Rest API:
{
"-KzPO6veQ986jxq6jlHK": true,
"-KzQeHwhWQozjb_bqRXl": true,
"-KzPLigBLTL5w8wcSsnA": true,
"-KzPOMEB7C7GawO_7o8b": true,
"-KzVcYwM3IMnaF8DbhOe": true,
"-Kz5XqskF0zLLogT_sl-": true
}
You can try yourself by open the following URL: https://librostic-fa290.firebaseio.com/books.json?&shallow=true
This is not the response that I'm specting because as you can see it is unordered in a weird way.
Am I getting the correct response?
The REST API Guide explains that query results are unordered:
The REST API Returns Unsorted Results: JSON interpreters do not
enforce any ordering on the result set. While orderBy can be used in
combination with startAt, endAt, limitToFirst, or limitToLast to
return a subset of the data, the returned results will not be sorted.
Therefore, it may be necessary to manually sort the results if
ordering is important.
Related
One of the data in datastore is 7766277975020011920 and similarities.
The error shown in nodejs is -
Error: We attempted to return all of the numeric values, but chain value 7766277975129421920 is out of bounds of 'Number.MAX_SAFE_INTEGER'.
It suggested "options.wrapNumbers=true" to the file
"node_modules/#google-cloud/datastore/build/src/entity.js:412:19".
But I am using google cloud RUN version and not able to edit the files. How can I pass it ?
Moving my comment into an answer, according to the Node.js client reference for Datastore, when you run queries or calls for entities, you can pass options as an additional argument, which supports passing the wrapNumbers = True option:
const [values] = await datastore.runQuery(query,{"wrapNumbers":true});
You can use that in your calls to avoid receiving out of bounds errors for large integers. The rest of the supported options are documented in this code snippet from the official repository:
const options = {
consistency: 'string',
gaxOptions: {},
wrapNumbers: true,
};
I am trying to access the Time Entries object via the Kronos API v2.
The documentation says that there are two required Query Parameters: start_date and end_date.
I am able to query the endpoint including one of the parameters at a time but am not able to enter both. And, I find the documentation quite lacking.
The root of the endpoint is:
https://secure.saashr.com/ta/rest/v2/companies/{cid}/time-entries
Here are things I have tried to suffix to the above endpoint:
?start_date=2019-11-01&end_date=2019-12-01
?start_date=2019-11-01|end_date=2019-12-01
?start_date=2019-11-01 end_date=2019-12-01
?start_date=2019-11-01?end_date=2019-12-01
?start_date=2019-11-01:end_date=2019-12-01
?filter=start_date:=:2019-11-01|end_date:=:2019-12-01
I also tried including quotes around the dates.
Everything results in some 400 level error when querying the API. With most of the above suffixes, it recognizes start_date but not the end_date. In this case, the error is:
{'code': 400, 'message': 'Missing required: end_date'}]
Note, above {cid} is replaced with the company's id.
In summary, how should I include two query parameters in the Kronos API?
The first option is correct.
https://secure.saashr.com/ta/rest/v2/companies/{cid}/time-entries?start_date=2019-11-01&end_date=2019-12-01
should work just fine.
Could you provide full URL you set in request?
I am using firebase for data storage. The data structure is like this:
products:{
product1:{
name:"chocolate",
}
product2:{
name:"chochocho",
}
}
I want to perform an auto complete operation for this data, and normally i write the query like this:
"select name from PRODUCTS where productname LIKE '%" + keyword + "%'";
So, for my situation, for example, if user types "cho", i need to bring both "chocolate" and "chochocho" as result. I thought about bringing all data under "products" block, and then do the query at the client, but this may need a lot of memory for a big database. So, how can i perform sql LIKE operation?
Thanks
Update: With the release of Cloud Functions for Firebase, there's another elegant way to do this as well by linking Firebase to Algolia via Functions. The tradeoff here is that the Functions/Algolia is pretty much zero maintenance, but probably at increased cost over roll-your-own in Node.
There are no content searches in Firebase at present. Many of the more common search scenarios, such as searching by attribute will be baked into Firebase as the API continues to expand.
In the meantime, it's certainly possible to grow your own. However, searching is a vast topic (think creating a real-time data store vast), greatly underestimated, and a critical feature of your application--not one you want to ad hoc or even depend on someone like Firebase to provide on your behalf. So it's typically simpler to employ a scalable third party tool to handle indexing, searching, tag/pattern matching, fuzzy logic, weighted rankings, et al.
The Firebase blog features a blog post on indexing with ElasticSearch which outlines a straightforward approach to integrating a quick, but extremely powerful, search engine into your Firebase backend.
Essentially, it's done in two steps. Monitor the data and index it:
var Firebase = require('firebase');
var ElasticClient = require('elasticsearchclient')
// initialize our ElasticSearch API
var client = new ElasticClient({ host: 'localhost', port: 9200 });
// listen for changes to Firebase data
var fb = new Firebase('<INSTANCE>.firebaseio.com/widgets');
fb.on('child_added', createOrUpdateIndex);
fb.on('child_changed', createOrUpdateIndex);
fb.on('child_removed', removeIndex);
function createOrUpdateIndex(snap) {
client.index(this.index, this.type, snap.val(), snap.name())
.on('data', function(data) { console.log('indexed ', snap.name()); })
.on('error', function(err) { /* handle errors */ });
}
function removeIndex(snap) {
client.deleteDocument(this.index, this.type, snap.name(), function(error, data) {
if( error ) console.error('failed to delete', snap.name(), error);
else console.log('deleted', snap.name());
});
}
Query the index when you want to do a search:
<script src="elastic.min.js"></script>
<script src="elastic-jquery-client.min.js"></script>
<script>
ejs.client = ejs.jQueryClient('http://localhost:9200');
client.search({
index: 'firebase',
type: 'widget',
body: ejs.Request().query(ejs.MatchQuery('title', 'foo'))
}, function (error, response) {
// handle response
});
</script>
There's an example, and a third party lib to simplify integration, here.
I believe you can do :
admin
.database()
.ref('/vals')
.orderByChild('name')
.startAt('cho')
.endAt("cho\uf8ff")
.once('value')
.then(c => res.send(c.val()));
this will find vals whose name are starting with cho.
source
The elastic search solution basically binds to add set del and offers a get by wich you can accomplish text searches.
It then saves the contents in mongodb.
While I love and reccomand elastic search for the maturity of the project, the same can be done without another server, using only the firebase database.
That's what I mean:
(https://github.com/metaschema/oxyzen)
for the indexing part basically the function:
JSON stringifies a document.
removes all the property names and JSON to leave only the data
(regex).
removes all xml tags (therefore also html) and attributes (remember
old guidance, "data should not be in xml attributes") to leave only
the pure text if xml or html was present.
removes all special chars and substitute with space (regex)
substitutes all instances of multiple spaces with one space (regex)
splits to spaces and cycles:
for each word adds refs to the document in some index structure in
your db tha basically contains childs named with words with childs
named with an escaped version of "ref/inthedatabase/dockey"
then inserts the document as a normal firebase application would do
in the oxyzen implementation, subsequent updates of the document ACTUALLY reads the index and updates it, removing the words that don't match anymore, and adding the new ones.
subsequent searches of words can directly find documents in the words child. multiple words searches are implemented using hits
SQL"LIKE" operation on firebase is possible
let node = await db.ref('yourPath').orderByChild('yourKey').startAt('!').endAt('SUBSTRING\uf8ff').once('value');
This query work for me, it look like the below statement in MySQL
select * from StoreAds where University Like %ps%;
query = database.getReference().child("StoreAds").orderByChild("University").startAt("ps").endAt("\uf8ff");
I have a Dynamodb table with a few fields - my_id is the PrimaryKey. In the API gateway I set up a response with a method that takes in a parameter {my_id}.
Then I have an Integration Request mapping template that takes the passed in parameter and queries the table to return all the fields that match.
Then I have an Integration response mapping template that cleans up the returned items the way I want.
This all works perfect.
The thing I can't figure out how to do is if the parameter that is passed in doesn't match anything in the table, how do I get it to change from a 200 status into a 404?
From what I can tell when the passed in parameter doesn't match anything it doesn't cause an error, it just doesn't return anything.
It seems like I need to change the mapping template on the Integration response to first check if the params are empty and then somehow tell it to change the response status.
I can find info about this type of thing with people using Lambda, but I am not using Lambda - just the Dynamodb table and the API Gateway.
You can use Mapping Template to convert the response that you get from DDB and overrride the response code. You can get more details in the link https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-override-request-response-parameters.html
If you are using cloud formation, you can do this by using below snippet
IntegrationResponses:
- StatusCode: "200"
ResponseTemplates:
application/json: |
{
"payload" : {
}
},
}
IntegrationResponses:
- StatusCode: "200"
ResponseTemplates:
application/json: |
#set($inputRoot = $input.path('$'))
#if($inputRoot.toString().contains("Item"))
$input.json("$")
#set($context.responseOverride.status = 200)
#else
#set($context.responseOverride.status = 404)
#end
Api gateway currently supports mapping the status code using the status code of the integration response (Here dynamodb response code). The only workaround is to use a lambda function which outputs different error messages that can be mapped using a error regex http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-method-settings-execution-console.html.
Is it possible to filter data returned by the Firebase REST API using query parameters? I don't see it mentioned one way or an other in the docs, but the client libraries support it, so I'm hoping it's possible. Thanks.
It might be a bit late to answer, but Firebase does allow querying data via REST.
You can use the orderby option together with limitToLast, startAt etc just like you would when using the SDK.
Checkout the Firebase guide for more details
I fought a little bit to have it working.
I actually needed 2 things:
combine limitToLast with orderBy as mentioned by idan
URL getUrl = new URL( url + "news.json?orderBy=\"timestamp\"&limitToLast=5" );
add a rule in the database to declare an index on this "column"
"news" : { ".indexOn": "timestamp" }
Firebase provides querying parameters. However, I don't think they are the querying parameters you are expecting them to be, which are ones that filter data. Firebase REST API provides querying options like auth, print, callback, format, and download. Check docs here
Without ordering by certain field, By combining params orderBy="$key" and limitToLast=5 you can get the last 5 of inserted data ordered by it's key
The documentation can be looked at here