How to reset firestore emulator in integration test - firebase

I'm running my integration-tests in flutter and dart using the firestore emulator. First I start the firestore emulator with some data like so: firebase emulators:start --import=./dir.
Then I start an android emulator and start the app I want to test on the android emulator. The app is configured to use the firestore emulator. Then I run a series of tests, which all write to the firestore emulator.
But on the beginning of each test, I want the data to be reset to the state, when I first started the emulator. So e.g. if the tests are executed in this order:
Test A
Test B
Test C
I don't want to have the data, Test A created to be present in the database, when Tests B and C are executed. I could terminate the firestore emulator and start it again at the beginning of each test. But this would make my tests a lot slower.
Do you know of a way to reset the data, which is present in the firebase emulator?

I am assuming you're referring to firestore when you say you want to 'reset the data'.
Per the documentation at https://firebase.google.com/docs/emulator-suite/install_and_configure#use_the_emulator_hub_rest_api
import fetch from 'node-fetch';
import firebaseConfig from '../../../firebase.json';
const hubEmulatorPort = firebaseConfig.emulators.hub.port;
const firestoreEmulatorPort = firebaseConfig.emulators.firestore.port;
async function clearDb() {
const response = await fetch(
`http://localhost:${firestoreEmulatorPort}/emulator/v1/projects/${process.env.PROJECT_ID}/databases/(default)/documents`,
{
method: 'DELETE',
}
);
if (response.status !== 200) {
throw new Error('Trouble clearing Emulator: ' + (await response.text()));
}
}
async function populateDb(data) {
// Logic for adding in any data you want in the db
// before each test run
}
async function enableBackgroundTriggers() {
const response = await fetch(`http://localhost:${hubEmulatorPort}/functions/enableBackgroundTriggers`, {
method: 'PUT',
});
if (response.status !== 200) {
throw new Error('Trouble enabling database triggers in emulator: ' + (await response.text()));
}
}
async function disableBackgroundTriggers() {
const response = await fetch(`http://localhost:${hubEmulatorPort}/functions/disableBackgroundTriggers`, {
method: 'PUT',
});
if (response.status !== 200) {
throw new Error('Trouble disabling database triggers in emulator: ' + (await response.text()));
}
}
async function resetDb(data) {
await disableBackgroundTriggers();
await clearDb();
await populateDb(data);
await enableBackgroundTriggers();
}
export { resetDb };
I can't find a source for the clearing of the db, but the RESTful call in clearDb does what you want.
It's important to disable the triggers before clearing or populating the database, in case you have firestore triggers that modify data in ways your tests don't expect. I write tests by passing full DB state to the populateDb method, then reenable triggers before running tests so I can test said triggers. If you aren't running any firestore triggers, the clearDb call alone should be enough for your purposes.
My tests all have calls to resetDb() in my beforeEach hook in jest to ensure clean runs of each test. I recommend adding this to whatever 'beforeEach'-like hook your favorite testing API exposes.
If your tests do things like create users in Firebase Authentication you'll have to find another way to clear them between test runs.
If anyone can find documentation on how to clear other emulators in the Firebase Emulator Suite, please drop it in the comments. I am currently trying to find a way to clear Authentication emulators, which is actually how I found this question.
Best of luck!

If you want to clear out all the collections programatically, like in a setUp() or tearDown() there's a reference for that here: Delete data from Cloud Firestore - Delete Collections
Note that it's not recommended for all implementations, but there are examples in Java, Python, Node.js, go, PHP, C#, and Ruby.
Here's an example of how to iterate through all your collections and delete them all in Java, using the deleteCollection() method from that link.
public static void main(String[] args) throws IOException {
final int BATCH_SIZE = 5;
Firestore firestore = initializeCloudFirestore();
for (CollectionReference listCollection : firestore.listCollections()) {
deleteCollection(listCollection, BATCH_SIZE);
}
}
/**
* One way of initializing Firestore,
* see other options at https://firebase.google.com/docs/firestore/quickstart#initialize
*/
private static Firestore initializeCloudFirestore() throws IOException {
// Use the application default credentials
GoogleCredentials credentials = GoogleCredentials.getApplicationDefault();
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials)
.setProjectId("projectId")
.build();
FirebaseApp.initializeApp(options);
Firestore firestore = FirestoreClient.getFirestore();
return firestore;
}
/**
* Delete a collection in batches to avoid out-of-memory errors. Batch size may be tuned based on
* document size (atmost 1MB) and application requirements.
* See https://firebase.google.com/docs/firestore/manage-data/delete-data#java_5
*/
static void deleteCollection(CollectionReference collection, int batchSize) {
try {
// retrieve a small batch of documents to avoid out-of-memory errors
ApiFuture<QuerySnapshot> future = collection.limit(batchSize).get();
int deleted = 0;
// future.get() blocks on document retrieval
List<QueryDocumentSnapshot> documents = future.get().getDocuments();
for (QueryDocumentSnapshot document : documents) {
document.getReference().delete();
++deleted;
}
if (deleted >= batchSize) {
// retrieve and delete another batch
deleteCollection(collection, batchSize);
}
} catch (Exception e) {
System.err.println("Error deleting collection : " + e.getMessage());
}
}
For the entire file, including imports, see this Github Gist.

Related

How to Read and Write Data from Firebase Database?

I have many problems to read data from a simple Firebase DB and to store it.
It's the first time that I use it so any help will be appreciated.
The solution is Xamarin.Forms and this is the Android part, my main problem.
This is my DB Structure:
I need to read and write the "number" value stored.
I add all Firebase package like Firebase.Database and Firebase.Connection
In my Android Prj I have a class with these:
MainActivity.cs
private void InitFirebaseAuth()
{
var options = new FirebaseOptions.Builder()
.SetApplicationId("******")
.SetApiKey("*******")
.SetDatabaseUrl("******")
.Build();
if (app == null)
app = FirebaseApp.InitializeApp(this, options, "SaloneB");
}
FirebaseHelper.cs
DatabaseReference databaseReference;
FirebaseDatabase database
public void Connect()
{
database = FirebaseDatabase.GetInstance(MainActivity.app);
}
First I call Connect then I try something like this without result:
FirebaseClient firebase = new FirebaseClient("*****");
//This to retrieve value
var items = await firebase
.Child("number_of_user") //name of the table
.OnceAsync(); //retrieve the value
Items result is 0 elements
Thanks for help

Flutter moor insert hangs on isolate

When I create a database I want to initialize it with a ton of data.
I have the following initialization service.
// This needs to be a top-level method because it's run on a background isolate
DatabaseConnection _backgroundConnection() {
// construct the database. You can also wrap the VmDatabase in a "LazyDatabase" if you need to run
// work before the database opens.
final database = VmDatabase.memory();
return DatabaseConnection.fromExecutor(database);
}
Future<void> _initDatabase(Map<String, dynamic> args) async {
var moorIsolate = await MoorIsolate.spawn(_backgroundConnection);
var connection = await moorIsolate.connect();
var db = BillingDatabase.connect(connection);
_initBillingSpecialties(db, args["specialties"]);
}
Future<void> _initBillingSpecialties(BillingDatabase db, String specialtiesJson) async {
var json = jsonDecode(specialtiesJson);
var jsonSpecialties = json["specialties"] as List<dynamic>;
var specialities = jsonSpecialties.map((s) =>
DbSpecialtiesCompanion(name: Value(s["specialty_name"]),
mohNumber: Value(s["moh_specialty"]))).toList();
return db.specialtyDao.saveAllSpecialties(specialities);
}
#injectable
class InitDbService {
Future<void> initDatabase() async {
WidgetsFlutterBinding.ensureInitialized();
var specialties = await rootBundle.loadString("lib/assets/billing_specialties.json");
compute(_initDatabase, {"specialties": specialties});
//initDbSync(specialties);
}
Future<void> initDbSync(String specialtiesJson) async {
var json = jsonDecode(specialtiesJson);
var jsonSpecialties = json["specialties"] as List<dynamic>;
var specialities = jsonSpecialties.map((s) =>
DbSpecialtiesCompanion(name: Value(s["specialty_name"]),
mohNumber: Value(s["moh_specialty"]))).toList();
var dao = GetIt.instance.get<SpecialtyDao>();
return dao.saveAllSpecialties(specialities);
}
}
initDbSync runs and inserts just fine. While db.specialtyDao.saveAllSpecialties(specialities); never actually exectues any SQL. I have it printing log statements for the moment so I can see what it's doing.
Update: I found out that VmDatabase.memory(logStatements: true); was needed to see the SQL. I can see it printing the statements.
I'm running on a simulator so I can look at the raw db file. And there's nothing there. When I query in the app there's also nothing there.
So what's not really clear in the documentation is that VmDatabase.memory(); opens up a new database in memory. Not takes the database from memory.
You want to take your reference to the file that you pass in the constructor, and use
VmDatabase(File(dbFile));
then it will actually run on your sql.

Flutter Firestore take long retrieving data while offline

I am using Firestore in flutter application. Each time user launch the application it retrieves some data from Firestore Cloud.
QuerySnapshot dataSnapshot = await Firestore.instance
.collection('/data')
.getDocuments();
When user opens the application on first time, it required from him to connect online, to get the data, and as Firebase documents say
For Android and iOS, offline persistence is enabled by default. To disable persistence, set the PersistenceEnabled option to false.
So, it should save the data that application have been read before to retrieve it while the device is offline; so user can access application at anytime with the same data that have been read.
The problem is: it takes too long time to retrieve the data while the device is offline, with the same codes and nothing changed!.
I tried to configure how much time it takes? On offline, it takes about 8 minutes and 40 seconds. But while on online, it takes just 10 seconds, maybe less.
So how can I solve this problem?
============
UPDATE
I manged to get more logs about this problem, which after take a lot of time, and will start application with the offline saved data, it prints this log
This typically indicates that your device does not have a healthy Internet connection at the moment. The client will operate in offline mode until it is able to successfully connect to the backend.
And then take 3 second for example (not much time) and continue with the next works.
I did open a new issue in GitHub too.
Is there a way to limit the time it takes?
And finally, with the help of diegoveloper comment in GitHub issue, I have reached the solution.
This comment
await Firestore.instance
.collection("Collection")
.getDocuments(source: source)
was a good solution if I decided to check source each time and then use it or I can use it in starting of a new Flutter project, but now I already have a lot of codes that need a better solution. So I decided to fork the cloud_firestore package and edit it.
You can find it here: https://github.com/ShadyBoshra2012/flutterfire/tree/master/packages/cloud_firestore
What I have edited:
firestore.dart
// The source of which the data will come from.
static Source _source = Source.serverAndCache;
static Source get source => _source;
Future<void> settings(
{bool persistenceEnabled,
String host,
bool sslEnabled,
bool timestampsInSnapshotsEnabled,
int cacheSizeBytes,
Source source}) async {
await channel.invokeMethod<void>('Firestore#settings', <String, dynamic>{
'app': app.name,
'persistenceEnabled': persistenceEnabled,
'host': host,
'sslEnabled': sslEnabled,
'timestampsInSnapshotsEnabled': timestampsInSnapshotsEnabled,
'cacheSizeBytes': cacheSizeBytes,
});
if (source != null) _source = source;
}
query.dart
source = Firestore.source; Line 92
document_reference.dart
source = Firestore.source; Line 83
How you can use it?
So you can use my forked repository in this way with using connectivity package from Google : https://pub.dev/packages/connectivity .
Add my forked repository in pubspec.yaml file
cloud_firestore:
git:
url: https://github.com/ShadyBoshra2012/flutterfire.git
path: packages/cloud_firestore
Then in your first screen or main
var connectivityResult = await (Connectivity().checkConnectivity());
if (connectivityResult == ConnectivityResult.none) {
await Firestore.instance.settings(source: Source.cache);
} else {
await Firestore.instance.settings(source: Source.serverAndCache);
}
and if you want to refresh the source when change the connection state:
StreamSubscription subscription;
void initState() {
super.initState();
// Check the internet connection after each change
// of the connection.
subscription = Connectivity()
.onConnectivityChanged
.listen((ConnectivityResult result) async {
// Check the internet connection and then choose the appropriate
// source for it.
var connectivityResult = await (Connectivity().checkConnectivity());
if (connectivityResult == ConnectivityResult.none) {
await Firestore.instance.settings(source: Source.cache);
} else {
await Firestore.instance.settings(source: Source.serverAndCache);
}
});
}
#override
void dispose() {
super.dispose();
subscription.cancel();
}
So I hope it works with everyone see it, and waiting for Flutter Team to code a better and better solution. Thanks for everyone has participated.
In addition to Shady Boshra's answer you can use FirebaseFirestore.instance.disableNetwork() functionality so your code will look like this:
StreamSubscription subscription;
void initState() {
super.initState();
// Check the internet connection after each change
// of the connection.
subscription = Connectivity()
.onConnectivityChanged
.listen((ConnectivityResult result) async {
// Check the internet connection and then choose the appropriate
// source for it.
var connectivityResult = await (Connectivity().checkConnectivity());
if (connectivityResult == ConnectivityResult.none) {
await FirebaseFirestore.instance.disableNetwork();
} else {
await FirebaseFirestore.instance.enableNetwork();
}
});
}

flutter firebase database transaction crash

I'm trying to create child by using transaction.
I did like so.
void _runTransaction(DatabaseReference ref, Map<String, dynamic> data) async {
await ref.runTransaction((MutableData tx) async {
tx.value = data;
return tx;
});
}
But app will crash and child won't be created.
Am I doing it wrong?
Does anyone have an idea what went wrong?
As document says
performs an optimistic-concurrency transactional update to the data at this
Firebase Database location.
I think you can't create with transaction.
After created the child, then update it with transaction worked.

Detect if Firebase connection is lost/regained

Is there a strategy that would work within the current Firebase offering to detect if the server connection is lost and/or regained?
I'm considering some offline contingencies for mobile devices and I would like a reliable means to determine when the Firebase data layer is available.
This is a commonly requested feature, and we just released an API update to let you do this!
var firebaseRef = new Firebase('http://INSTANCE.firebaseio.com');
firebaseRef.child('.info/connected').on('value', function(connectedSnap) {
if (connectedSnap.val() === true) {
/* we're connected! */
} else {
/* we're disconnected! */
}
});
Full docs are available at https://firebase.google.com/docs/database/web/offline-capabilities.
Updated:
For many presence-related features, it is useful for a client to know when it is online or offline. Firebase Realtime Database clients provide a special location at /.info/connected which is updated every time the client's connection state changes. Here is an example:
DatabaseReference connectedRef = FirebaseDatabase.getInstance().getReference(".info/connected");
connectedRef.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot snapshot) {
boolean connected = snapshot.getValue(Boolean.class);
if (connected) {
System.out.println("connected");
} else {
System.out.println("not connected");
}
}
#Override
public void onCancelled(DatabaseError error) {
System.err.println("Listener was cancelled");
}
});
I guess this changed in the last couple of months. Currently the instructions are here:
https://firebase.google.com/docs/database/web/offline-capabilities
In summation:
var presenceRef = firebase.database().ref("disconnectmessage");
// Write a string when this client loses connection
presenceRef.onDisconnect().set("I disconnected!");
and:
var connectedRef = firebase.database().ref(".info/connected");
connectedRef.on("value", function(snap) {
if (snap.val() === true) {
alert("connected");
} else {
alert("not connected");
}
});
I'll admit I don't know a lot about how references are set, or what that means (are you making them out of thin air or do you have to have already created them beforehand?) or which one of those would trigger something on the server as opposed to something on the front end, but if the link is still current when you read this, a little more reading might help.
For android you can make user offline by just a single function called onDisconnect()
I did this in one of my chat app where user needs to get offline automatically if network connection lost or user disconnected from Internet
DatabaseReference presenceRef = FirebaseDatabase.getInstance().getReference("USERS/24/online_status");
presenceRef.onDisconnect().setValue(0);
On disconnecting from network Here I am making online_status 0 of user whose Id is 24 in firebase.
getReference("USERS/24/online_status") is the path to the value you need to update on offline/online.
You can read about it in offline capabilities
Note that firebase takes time around 2-10 minutes to execute onDisconnect() function.
firebase for web
firebase.database().ref(".info/connected").on("value",(snap)=> {});
The suggested solution didn't work for me, so I decided to check the connection by writing and reading 'health/check' value. This is the code:
const config = {databaseURL: `https://${projectName.trim()}.firebaseio.com/`};
//if app was already initialised delete it
if (firebase.apps.length) {
await firebase.app().delete();
}
// initialise app
let cloud = firebase.initializeApp(config).database();
// checking connection with the app/database
let connectionRef = cloud.ref('health');
connectionRef.set('check')
.then(() => {
return connectionRef.once("value");
})
.then(async (snap) => {
if (snap.val() === 'check') {
// clear the check input
await connectionRef.remove();
// do smth here becasue it works
}
});

Resources