Is there a way to get a HealthKit query run on watchos to return the full set of results in a way that is not limited by time? The same code which I have running on ios and which returns all results, when run on watchos, drops any results that are older than a week. Results that were showing from 6 days ago from a particular day are not included in query results the next day when run on the watch.
This question:
1] Is HealthKit query on WatchOS limited?
was answered with a suggestion that a query performed with a release build would return all results, but this doesn't seem to be correct and also is not consistent with the week-long limit that I've observed.
If this is simply a limit that is inherent with WatchOS, where is it documented?
If there is a way to overcome the limit, please specify how.
Apple Watch does not have all health data on device. It only contains a weeks worth of data so the results you are getting are expected. The full data is available on the paired iPhone.
Related
Today I found an error about Firestore Batchwrite in my stock control application.
My database structure was designed to store the order detail information in one collection (orders) and in other collection (report_monthly for count purposes) for monthly reports, I stored the quantity information such as total no of stocks, price, and debts.
The problem was this morning I make offline orders of stocks of about 20-25 orders. After I switch my phone from offline to online instantly, I can see the numbers are increasing (clearly firebase is updating the data) but I suddenly clear the app from working like clear recent apps in the android system. Then when I re-run(open) my app, these data are incorrect. Although the data in order collection(orders) is correct, the report number of quantity value in the monthly collection(report_monthly) was somehow repeated increment.
I tested this process multiple times with my friend and still facing the problems. I also tested without using BatchWrite. In that case, I can still see the issues but some of Fieldvalue increments are right and some are wrong not like in BatchWrite which is the whole list of Fieldvalue increments is wrong.
Can someone suggest to me how can I handle this kind of problem?
We have a couple of environments still running Time Series Insights Preview version. This is really fast and we are really satisfied by it. However, new environments really seem a lot slower with the official release. Warm path extraction is a lot slower, but still doable, while cold path extraction becomes unbearable.
EDIT: We need to add &storeType=WarmStore if we would like to query warm data. Cool! This works really fast again! Question about cold store still persists:
It is hard to compare the different environments, because the datasets are not exactly the same, but for our new environment we have about 4.5 TB sensor data imported in TSI.
The following screenshot shows a query that tries to retrieve one minute of data for one device (each device only sends data each 10 seconds) in the far past of 2018. However, the server returns the call after 30 seconds with a continuationtoken, saying it couldn't retrieve all the 6 values in time. Sometimes it manages to return all 6 of the values, but it still takes 30 seconds.
My internet download speed, while performing the query, was over 80 Mb per second, so that shouldn't be an issue either.
Is this something we should be worried about in the new release?
please submit a support ticket through the Azure portal with all of these details and the product team will investigate.
Background:
I'm having the Firebase analytics data exported to BigQuery. And I'm using cron jobs to crunch data in BigQuery for getting insight.
Problem:
To be able to only crunch delta data i.e. the data that has arrived since last time I ran my cron job I need a way to figure out the time when the data arrived at server, since the event_timestamp is generated at client and can be cached at client before sent.
Insights:
I have laborated with event_server_timestamp_offset (offset) which I thought I could use together with event_timestamp. But I was expecting the offset to only be positive but it can also be negative. And when I look at the MAX and MIN for the offset in the entire exported Firebase analytics dataset and re-calculate it to years instead of microseconds I can get more than 18 years offset.
Query:
SELECT
MAX(event_server_timestamp_offset)/(1000000*60*60*24) max_days,
MIN(event_server_timestamp_offset)/(1000000*60*60*24) min_days
FROM
`analytics_<project_id>.events_*`
Result: max_days=6784.485790436655,
min_days=-106.95833052104166
Question:
How can I figure out the server arrival time for my Firebase exported BigQuery data so I can run cron jobs crunching only delta data?
Can I use event_server_timestamp_offset together with event_timestamp? If so, how?
Best regards,
Daniel
Surprisingly enough, this question not having a clear answer for almost 2 years, I am leaving here the answers I got from the Firebase support team. The format is - question asked followed by the answer of the support staff.
Q1. event_date - The date on which the event was logged (YYYYMMDD format in the registered timezone of your app). Does it mean that the event occurred on that date, or that it was actually collected on that date?
A1. Per documentation, event_date refers to the date on which the event is logged/occurred. Note that event_date is based on the Analytics timezone setting of your Firebase Project.
Q2. event_timestamp - The time (in microseconds, UTC) at which the event was logged on the client. Is it safe to assume that this is the exact timestamp the event occurred on client side (in the app timezone of course)?
A2. Yes, this is based on the device timezone setting. However, event_timestamp may be skewed if the device time is incorrect.
Q3. event_server_timestamp_offset - Timestamp offset between collection time and upload time in micros. This is the main field that causes all the misunderstandings - in our BigQuery table for the year 2020 this field takes values in a range between 5 days and -2 days. I mean how can the colleciton time be 2 days ahead?
A3. The event_server_timestamp_offset field in the export schema is the time difference between when the event took place and the app uploaded it to our server. In other words, this is the estimated difference between the client's local time and the actual time, according to our servers. The values of this field are usually positive, but can be negative as well if the device time setting is incorrect.
Q4. One last question is very important - can we ignore the
event_server_timestamp_offset field and just rely on event_timestamp -
as the exact date and time the event occurred on the clientside (not
collected, not uplaoded, etc). If not- please explain how we can get
the exact datetime of the event occuring on the clientside. But if yes
please let me know why do we need the event_server_timestamp_offset field?
A4. Yes, you may actually ignore it and use event_timestamp alone. However, as mentioned earlier, event_timestamp could be off if the device time setting incorrect, but it shouldn't really affect the bigger picture of your analytics data as cases like this are usually one-off.
We use the event_date as the indicator and load the data once a day.
According to Firebase Analytics docs (https://support.google.com/firebase/answer/6317517#active-users), the active number of users is the number of unique users who initiated sessions on a given day. Also according to the docs, every time a session is started an event with session_start name is sent. I am trying to get that metric using BigQuery's export, but my query is giving me different results (15636 on BigQuery, 14908 on FB analytics)
I have also tried converting to different timezones to see if that might be the issue, but no matter which timezone I try I never get the same (or similar) results
Which query should I run to get the same results I get on Firebase Analytics dashboard for active users?
My query is
SELECT EXACT_COUNT_DISTINCT(user_dim.app_info.app_instance_id)
FROM table_date_range([XXXXX.app_events_], timestamp('2016-11-26'), timestamp('2016-11-29'))
WHERE DATE(event_dim.timestamp_micros) = '2016-11-27'
AND event_dim.name ='session_start'
Thanks
Update
After #djabi's answer I changed my query to use user_engagement rather than session_start and it works much better now. Still some minor differences though (they range from under ten to under 50 out of 16K, depending on the date).
I have tried once again using different timezones by playing around with DATE(date_add(event_dim.timestamp_micros,1,'hour')) but I never got the exact number I get on Firebase Analytics dashboard.
The new numbers are good enough to be considered statistically acceptable, but wondering if anyone has a suggestion to improve the query and get exact results?
The current query is:
SELECT
COUNT(*) AS active_users
FROM (
SELECT
COALESCE(user_dim.user_id, user_dim.app_info.app_instance_id) AS user_id
FROM
TABLE_DATE_RANGE([XXXXX.app_events_], TIMESTAMP('2016-11-24'), TIMESTAMP('2016-11-29'))
WHERE
DATE(event_dim.timestamp_micros) = '2016-11-25'
AND event_dim.name ='user_engagement'
GROUP BY
user_id )
Note: At the moment we are not sending user_id, so the COALESCE will always return the app_instance_id, in case anyone was going to suggest that could be the problem
You need to wait for full 3 days for data from offline devices to be uploaded. Your query correctly filter the events based on the event timestamp and you pull data from 3 days but that is only day and half from today and that is enough for all data to be uploaded. Try including 3 days from yesterday.
Also try using user_engagement event instead of session_start. I believe active user count is based on user_engagement and not on session_start events.
Also FB reports take a bit to process so you wight want and check the FB reports the next day.
FB reports are done on the time zone on the account and events are timestamped in UTC so the day in FB reports is different from UTC calendar day. You want to control for that discrepancy as well to get matching numbers.
Sessions are by-default measured after user activity of 10 seconds in the respective app which you can change. Try changing the sessions start time count to the least number possible and then you may arrive at a number closer to what you are expecting.
For Android stats I used:
user_dim.device_info.resettable_device_id
instead of
user_dim.app_info.app_instance_id
and it produced better results.
I have an public transport app with realtime departure data for trains.
I would like to add a complication that shows the departure time of the next train.
Is it possible to show (or refresh) realtime data on a complication? For example, showing "3 min. to station X." The data could change every minute, based on info that comes from the public transport API.
How should I accomplish this on watchOS 2 or watchOS 3?
I know the ETA app shows travel times in a complication, but I'm not sure how they achieve that.
Are realtime updates possible?
Complications aren't designed to show realtime data. Frequent updates can affect energy efficiency and impact the battery (on both watch and phone).
To minimize power usage, ClockKit asks you to provide as much data as you have available and then caches the data and renders it when needed.
While there is no fixed number of times a complication timeline can be reloaded, the complication data source is subject to a daily execution time budget.
If your app’s data changes frequently, it might be difficult to provide enough data to display in a complication. Worse, if you refresh your complication data too frequently, you may exceed your execution time budget and your complication might not be updated until the following day.
Once the daily budget is exhausted, calls to reloadTimeline (and extendTimeline) do nothing.
If your complication has already exceeded its allotted daily budget for execution time, calls to this method do nothing. Call this method sparingly.
How can a complication display relative times?
You can use a CLKRelativeDateTextProvider to create a formatted relative time that can change on a minute-by-minute basis.
A CLKRelativeDateTextProvider object creates a formatted string that conveys the difference in time between the current date and a date that you specify. You use a relative date text provider to implement timers or other relative time values in an efficient way. Instead of using multiple timeline entries to replicate a countdown timer, create a single timeline entry with a relative date text provider. When the user views the clock face, ClockKit automatically updates the relative time value in your complication, providing up-to-date time information.
How could a complication be frequently updated?
You could use a complication push update (either from a remote server, or locally from the phone in iOS 10).
There is a limit of 50 complication push updates per day.
You could fetch data on the phone and use transferCurrentComplicationUserInfo.
In watchOS 2, this was only subject to the daily budget. In watchOS 3, this is now limited to 50 transfers per day.
See Is transferCurrentComplicationUserInfo more suitable for complication update? for more details.
In watchOS 2, you could use getNextRequestedUpdateDate to schedule the next time to update your complication.
This can't occur more often than every ten minutes.
Note that watchOS 3 apps should be upgraded to use background refresh app tasks. The main benefit is that background tasks would be able to do more than merely update your complication. They can also handle fetching data, updating your model once the data arrives, as well as updating your dock snapshot.
Finally, you can schedule a manual update. In watchOS 3, the recommended way to do this would via a background refresh app task.
The task budget permits 4 tasks per hour. See scheduleBackgroundRefresh for more details.
Note that background refresh app tasks must not use more than 10% CPU.
Recommended WWDC 2016 sessions
208 What's New in watchOS 3 introduces some of these topics.
804 Designing Great Apple Watch Experiences discusses when and why to update your watch apps.
218 Keeping Your Watch App Up to Date provides details about using background tasks to update your complication, app, and dock snapshot.
As mentioned in the talks, you should schedule your updates around the times when they would be needed.
For your use case, examples would be only when public transit is running, and only when the regularly scheduled departure times would be affected by a delay.
Apple sample code
Apple provides WatchBackgroundRefresh sample code demonstrating how to use WKRefreshBackgroundTask to update WatchKit apps in the background.
To update any active complication within a background task, you would simply add code to reload (or extend) the timeline:
let complicationServer = CLKComplicationServer.sharedInstance()
for complication in activeComplications {
complicationServer.reloadTimelineForComplication(complication)
}