I'm new to Google Earth Engine and I can't modify or choose the computing power like the Compute Engine service on Google Cloud Platform.
It's this possible! Thanks!
Generally, no. Earth Engine is a shared service — your computation runs on the same machines as everyone else's.
Your main choice is between whether you use the interactive API with things like print, evaluate, interactive maps, etc. or whether you use exports which can run for much longer times and with more resources. But your first step should always be to ensure that your code is written to run efficiently and makes use of Earth Engine's strengths.
Related
I am working on a small project trying to control some steps of a workflow in a web application using MS teams. My idea is to use R as an intermediate step between the application (which has a number of API endpoints I can call from R) and Microsoft Teams chats (or channels). Users would then use a set of keywords in the chat to lead to an action in the application. For example they might use "publish ABC-123" in a specific chat and this would lead to the application publishing document ABC-123 somewhere via R which would orchestrate everything.
I have a couple of ideas but there are drawbacks:
I thought originally about using microsoft365r. We have an app registered in Microsoft 365 which would allow us to monitor a specific chat for messages that trigger actions in R. The problem with this approach is that we would need to have the R code running and checking MS Teams every couple of minutes. It is certainly doable, but not very elegant.
Another option could be setting up a plumber API and an outgoing webhook in MS Teams. This seems like the ideal way to do it, but webhooks in MS Teams require https and as far as I understand this is not straightforward to implement in plumber.
I would appreciate any ideas on how to do this. I know I am not very specific, but mostly looking for high level pointers of what I could look at. Many thanks!
You actually have a bunch of options for this:
Create a bot directly in code, e.g. per https://learn.microsoft.com/en-us/microsoftteams/platform/bots/what-are-bots . There's a bit of a learning curve of course, and it depends on whether you have development skills outside of r, e.g. python, .net, whatever. The bot would then call your code as needed.
Create a no-code bot using Power Virtual Agents. This is the equivalent, for bots, of Power Apps or Power Automate, if you're familiar with those.
Create a workflow, either in Power Automate or Azure Logic Apps, that can listen for and respond to messages. This is kind of similar to a bot, but with finer scope (and therefore less capability). If you want it to call out to your app, e.g. to an endpoint, you'd need a Premium Connecter for Power Automate, or you can use an Azure Logic App directly (uses the same engine, but the pricing model is different for these and Power Automate is a little easier to work with.
Outgoing webhook - you can implement these as standalone, but actually from your use case it sounds like a bot would be better anyway, and it's kind of what you need to build to make this kind of webhook work properly anyway.
I am using Firebase as my authentication and database platform in my React Native-Expo app. I have not yet decided if I will be using the realtime-database or Firestore database.
I need to perform statistical analysis on daily data gathered from my users, which is stored in the database. I.e. the users type in their daily intake of protein, from it I would like to calculate their weekly average, expected monthly average, provide suggestions of types of food if protein intake is too low and etc.
What would be the best approach in order to achieve the result wanted in my specific situation?
I am really unfamiliar and stepping into uncharted territory regarding on how I can accomplish this. I have read that Firebase Analytics generates different basic analytics regarding usage of the app, number crash-free users etc. But can it perform analytics on custom events? Can I create a custom event for Firebase analytics to keep track of a certain node in my database, and output analytics from that? And then of course, if yes, does it work with React Native-Expo or do I need to detach from Expo? In addition, I have read that Firebase Analytics can be combined with Google BigQuery. Would this be an alternative for my case?
Are there any other ways of performing such data analysis on my data stored in Firebase database? For example, export the data and use Python and SciKit Learn?
Whatever opinion or advice you may have, I would be grateful if you could share it!
You're not alone - many people building web apps on GCP have this question, and there is no single answer.
I'm not too familiar with Firebase Analytics, but can answer the question for Firestore and for your custom analytics (e.g. weekly avg protein consumption)
The first thing to point out is that Firestore, unlike other NoSQL databases, is storage only. You can't perform aggregations in real time like you can with MongoDB, so the calculations have to be done somewhere else.
The best practice recommended by GCP in this case is indeed to do a regular export of your Firestore data into BQ (BigQuery), and you can run analytical calculations there in the meantime. You could also, when a user inputs some data, send that to Pub/Sub and use one of GCP Dataflow's streaming templates to stream the data into BQ, and have everything in near real time.
Here's the issue with that however: while this solution gives you real time, and is very scalable, it gets expensive fast, and if you're more used to Python than SQL to run analytics it can be a steep learning curve. Here's an alternative I've used for smaller webapps, which scales well for <100k users and costs <$20 a month on GCP's current pricing:
Write a Python script that grabs the data from Firestore (using the Firestore Python SDK), generates the analytics you need on it, and writes the results back to a Firestore collection
Create an endpoint for that function using Flask or Django
Deploy that server application on Cloud Run, preventing unauthenticated invocations (you'll only be calling it from within GCP) - see this article, steps 1 and 2 only. You can also deploy the Python script(s) to GCP's Vertex AI or hosted Jupyter notebooks if you're more comfortable with that
Use Cloud Scheduler to call that function every x minutes - see these docs for authentication
Have your React app query the "analytics results" collection to get the results
My solution is a FlutterWeb based Dashboard that displays relevant data in (near) realtime like the Regular Flutter IOS/Android app and likewise some aggregated data.
The aggregated data is compiled using a few nodejs based triggers in the database that does any analytic lifting and hence is also near realtime. If you study pricing you will learn, that function invocations are pretty cheap unless of-course you happen to make a 'desphew' :)
I came up with a great solution.
I used the inbuilt firebase BigQuery plugin. Then I used Cube.js (deployed on GCP - cloud run on docker) on top of bigquery.
Cube.js just makes everything just so easy. You do need to make a manual query It tries to do optimize queries. On top of that, it uses caching so you won't get big bills on GCP. I think this is the best solution I was able to find. And this is infinitely scalable and totally real-time.
Also if you are a small startup then it is mostly free with GCP - free limits on cloud run and BigQuery.
Note:- This is not affiliated in any way with cubejs.
If I use HERE's Android SDK and preload map data to the device storage,
do I need to reload these data every n days?
We are operating a demand responsive transport system in an area with poor cellular coverage. Having to relocate the cars periodically to some designated wifi access point would be inconvenient and integrating our vehicle dispatcher algorithm with off-the-shelve SatNavs is just not possible.
No, offline data is not expiring, nor needs to be refreshed.
If you are using also online services (and don't operate fully offline) it might happen that some online services (e.g. online routing, traffic, online search) respond with no or bad results with older mapdata (mostly something older than a year). So, if you also have online usecases, please consider a mapupdate every half a year.
Also keep in mind, that outdated mapdata might cause a bad user experience with newly build roads or changed road networks.
I'd like to know if it is possibile using waze's API about the community like danger points for an external app and integrated them with some our features.
I'm afraid Waze has strict rules concerning to who they expose the reports sent in by users. They provide a feed of reports for specific areas through the Waze Connected Citizens Program (CCP), but this program is generally only available for governmental institutions or road authorities. Generally the program also expects a dataflow towards Waze (road closures, for example).
That said, I don't work for Waze and I don't know all the details of the app you are working on. Depending on the type of app you are talking about, they may like to collaborate. You could always try to apply as a CCP partner and perhaps they may make an exception. Just be aware that while Waze has a lot of users, it is actually not a very big company so support queries may take a while to get a reply.
Note that it is technically possible to scrape the information from the Waze Live Map, but I'd strongly advice against doing that without permission as it could lead to legal actions.
I'm looking for a good way to track CPU and memory usage for a variety of web applications and to be able to cross-reference this information with information on Google Analytics. For example, I'd like to be able to generate a report that shows the CPU and memory usage along with number of hits averaged over minute periods. One way I thought this could be solved is by adding custom page-level variables to Google Analytics for tracking CPU and memory usage. My questions:
For those familiar with GA reporting as it pertains to custom variables, is this possible?
Is there a better way to generate the kind of report I'm seeking? Perhaps even without using GA?
Thanks.
You can use the Google analytics API to push this data directly from the web page via javascript, or from the server using whatever language is relevant.
I've seen at least one large implementation use the API for UX A/B testing by way of event tracking, but there's no reason you couldn't store whatever related data you'd like.