I configured Firebase Remote Config A/B testing for Android, and we did rollout on at least 10K devices.
For some reason, I see "0 users" in my A/B test after more than 24 hours.
Firebase GMS version is: 11.8.0
Should it show A/B participants in real-time or it's ok to see 0 users after 24 hours?
P.S: We are able to get AB test variants on test devices through Firebase Instance Id, it works well.
The simplest experiment which is running has only app package as a target, with no additional filters. And it shows 0 users as well.
Finally, we found an answer!
Maybe somebody will find it helpful:
For now, it happens (no data in Firebase remote config A/B test experiment) if you have an activation event configured for A/B test experiment.
If you have 2 different experiments, both will fail to get results even if you have "activation event" configured only in 1 of them.
Additionally, remote config will not work as well, you'll be able to get only default values.
We already reported to Google about, so they'll fix it at some point I hope.
Another useful info which is really hard to get:
How long is it ok to see "0 Total Users" in experiment I've just
started?
It takes many hours before you can see any data in your experiment. We were able to see results only after 21 hours after experiment start, so if you configured everything well, don't worry and wait for at least 24 hours. It will show 0 "Total Users" for many hours after the start.
Should I use app versionName or versionCode in "Version" field of
experiment setup?
You should use versionName.
Some useful info from support:
Firebase SDK
Make sure your users have the version of your app with the latest SDK.
Since your experiment is with Remote Config
When activateFetched() is called, all events from that point on will be tagged with the experiment. If you have a goal or activation event that happens before activateFetched(), such as automatic events like first_open, session_start, etc., the experiment setup might be wrong.
Are you using an Activation Event?
Make sure to call fetch() and activateFetched() before the activation event occurs.
Experiment ID of the experiments (if support asks you about)
It's the number at the end of the URL while viewing experiment results.
This debugging log could be useful to get what is going on
Also:
The good way to check if your experiment is working now is to set it to a specific version you didn't publish yet and check logs from remote config with the fresh app install(or erase all app data & restart).
It should show different variant every time you reinstall the app, since your Firebase Instance ID changes after app reinstall/app data erase.
If you see variants change - then A/B test is running well.
In your "build.graddle": don't forget to set the same versionName which you set in experiment setup.
In my case, I was receiving results of A/B testing but suddenly, it stopped to appear. It had continued for 7 days and then results appeared. Firebase Support manager said:
what I suspected here is just a delay in showing the result in the
experiments
Additionally, she said that
With that, I would suggest always using the latest SDK version and
enabling Google Analytics data sharing.
In my case, I used I wasn't using the latest SDK version, but Google Analytics was enabled for "Benchmarking", "Technical Support", "Account Specialists" except for "Google products & services". I believe these settings were enabled by default (the screenshot from Google Analytics):
Related
I recently built 120 dags using cloud composer. They all functioned for a while.
They were all approximately the same. Each used python operator. Each made API calls to google search console. Each collected 7-9k rows of GSC data into a pandas dataframe, then uploaded this to GCS buckets and BigQuery (partitioned and clustered).
Occasionally I'd have all fail one day because the GSC auth token had been revoked, but no problem, create new credentials, upload and continue. That situation lasted a couple of months. Now nothing runs.
From the start, the cloud composer health had occasional red spots, but now the health is static red every day.
I have found documentation about how to check the health, but not how to find why the health is so poor and fix it.
Can anyone point me in the right direction?
The environment health metric depends on a Composer-managed DAG named airflow_monitoring which is triggered periodically by the airflow-monitoring pod. If this DAG isn't deleted, you can check the airflow-monitoring logs to see if there are any problems related to reading the DAG's run statuses. Consequently, you can also try troubleshooting the error in Cloud Logging using the filter:
resource.type="cloud_composer_environment"
severity=ERROR
The liveness check failure could be due to the following reasons:
Any resource constraint(Memory and CPU)
Known issue with the composer version. Please check composer
release
notes for any
known issues.
Airflow configuration as core:default_timezone(If you’ve
configured core: default_timezone airflow configuration composer
environment health will be shown as unhealthy. It is a known
issue and the composer product team is working on the resolution.)
Refer to this documentation for information on Cloud Composer’s environment health metric.
I was lucky enough to talk to someone from Google yesterday who said what I need to do is recreate my cloud composer environment because I have insufficient CPU. He suggested the flexible choice when recreating.
I need to actively receive crash notifications for firebase functions.
Is there any way to set up Slack webhooks to receive a message when Firebase Functions throw an Error, functions crash, or something like that?
I would love to receive issue messages by velocity ie: Firebase Functions crash 50 times a day.
Thank you so much.
First you have to create a log based (counter) metric that will be counting specific error occurencies and second - you create alerting policy with Slack notification channel.
Let's start from finding corresponding logs that appear when the function throws an error. Since I didn't have one that would crash I used logs that indicated that it was started.
Next you have to create a log based metric. Ignore the next screen and go to Monitoring > Alerting. Click on "Create new policy", find your metric and select "Rolling Window" to whatever time period you need. For testing I used 1 minute. Then set "Rollind windows function" to "mean".
Now configure when the alert has to be triggered - I chose over 3 (within 1 minute window).
On the next screen you select notification channel. In case of Slack it has to be configured first in "Notification Channels".
You can save policy the policy now.
After a few minutes I gathered enough data to generate two incidents:
And here's some alerting related documentation that may help you understand how to use them.
I am attempting to publish events via FlutterAnalytics but I am experiencing very sporadic behaviour.
Using latest firebase_core and firebase_analytics packages
Using Firebase project on PAYG Blaze plan
Add pushing of events to BigQuery
Using vanilla flutter create project for testing
Downloaded and added google-services.json to android/app and android/app/debug folder
Added firebaseAnalytics.logEvent(name: 'testevent'); in onPressed where counter is incremented
Click button until counter reaches 100
Expect to see 100 events in Firebase Analytics but I see none.
Look in StreamView, after 5 minutes a part of them show up, alongside the automatically collected screen_view etc.
Look in DebugView (after activating adb) they show up instantly.
Look in Events tab, nothing
Look in BigQuery, nothing, not even tables created
They say events don't show up instantly, wait up to 24h, okay:
Wait 24h, no event in Events tab beyond the automatically collected ones
No BigQuery table generated
Wait 48h, no event showing up.
I then proceeded to create several other test firebase projects, with varying degrees of events showing up:
One project has 12 events out of 100 in BigQuery and 100 in Events tab
Another project has no events
Another project has 27 events in Events tab and 12 in BigQuery
Is anybody getting better mileage out of Firebase Analytics ? It cannot be a misconfiguration on my part on a vanilla project as then no events would show up, not this sporadic behaviour across all the projects.
since you are seeing varying levels of events logged, we need to determine if each of the event is sent to the server for processing. this can be checked by enabling the verbose mode.
adb shell setprop log.tag.FA VERBOSE
adb shell setprop log.tag.FA-SVC VERBOSE
adb logcat -v time -s FA FA-SVC
This will help you to verify if the event is logged or not immediately instead of waiting for 24-48 hours for the UI to show that.
If the events are not logged in the verbose mode, then you might want to rephrase your code to send 100 sequential events, if that is required. Another thought is like lots of same events from same user, are so quick and so they are packaged together for processing resulting in various count. ALways do verbose mode to ensure your events are created and sent as you wanted to analyze further.
I've configured firebase ab-testing. Everything works fine except there is no impact user on console.
Actually, I can see UI and log show ab-testing is applied.
Moreover, by checking the other StackoverFlow topic, activateFetched also invoked after fetch successfully.
Moreover, I've referenced
Firebase Remote Config A/B testing shows no results after 24 hours
Firebase Remote Config results on initial request
Remote Config A/B Test does not provide results on iOS
But those are no work on my case.
Is there anything miss or any other need to check so that client can response AB testing result to firebase console.
Thanks for your help first.
Code snippet:
[FIRApp configure];
FIRRemoteConfigSettings* configSettings = [[remoteConfig configSettings] initWithDeveloperModeEnabled:YES];
[[FIRRemoteConfig remoteConfig] setConfigSettings:configSettings];
[[FIRRemoteConfig remoteConfig] fetchWithExpirationDuration:duration completionHandler:^(FIRRemoteConfigFetchStatus status, NSError *error) {
if (status == FIRRemoteConfigFetchStatusSuccess) {
BOOL configFound = [[FIRRemoteConfig remoteConfig] activateFetched];
A couple things to check or take note of:
Make sure you're using and have deployed the latest Remote Config SDK. Earlier versions don't work with A/B test experiments.
Be sure to verify your experiment on a test device by following the documentation here
It can take a couple days for data to come in for your experiment.
Please call the functions in the following order:
fetch()
Call activatefetched() in the completion handler of fetch().
Fire activation event. If you need to call activation event immediately after activatefetched(), add a time delay of a few seconds. This is because activatefetched() process asynchronously and hence the function may not execute completely, before the activation event is fired.
Once done, test a running experiment on test device. In the debug logs search with string "exp_X" where 'X' is the experiment Id. You will find the experiment Id in the URL of the experiment. If you find the experiment ID in the debug logs while executing the code on test device, it means the device was covered in experiment.
Also if the experiment setup is correct, the running experiment will show 1 active experiment user in the console.
I am implementing Paypal express checkout (using paypal rest sdk for php )in one of my projects for recurring billing (for subscription), every thing is working fine for the initial requests (about 10 checkouts) after that I start to get Error 400 for few days and then everything starts working again.
I just wanted to confirm if there is any sort of limit on creating billing agreement in sandbox environment?
Thanks in advance
Finally found the solution, the reason I was getting error 400 after some times was because I was setting a static time while creating billing agreement.
$agreement = new Agreement();
$agreement->setName('My Billing Agreement')
->setDescription('Subscription to My Billing Agreement')
->setStartDate(date('Y-m-d').'T9:45:04Z');
The reason this snippet was resulting in error is because the start date/time of the billing agreement can only be a future time.
All I needed to do was replace
setStartDate(date('Y-m-d').'T9:45:04Z')
with
setStartDate(date("c", time() + 1800))
and everything started working as expected. Hope this helps some one.