SQLite database is broken when executing query soon after app loading
This is Expo(react-native) app and the database is loaded from local storage.
If I wait a few seconds to use app(execute queries), the issue does not happen.
Maybe, this issue happens when a query is executed before all data loaded from local storage.
in APP.js
async function loadResourcesAsync() {
await Promise.all([
FileSystem.makeDirectoryAsync(`${FileSystem.documentDirectory}SQLite`, {
intermediates: true
}).then(
FileSystem.downloadAsync(
Asset.fromModule(require("./assets/db/mydb.db")).uri,
`${FileSystem.documentDirectory}SQLite/mydb.db`
)
)
]);
}
and in a Screen.js like this
const db = SQLite.openDatabase("mydb.db");
db.transaction(
tx => {
tx.executeSql(sql, [queryChar], (_, { rows: { _array } }) => {
// console.log(JSON.stringify(_array));
setSearchResult(_array);
});
},
e => {
console.log(e);
});
},
null
);
I expect a callback from openDatabase function, but it does not have such a callback.
How can I know a database loading done?
Related
i have another problem while using RTK Query.
We initially fetch data with a get request. Once the result added to the cache, we want add the websocket connection and listen for any changes.
But the only thing i can find in the docs is, just adding further entries via .push.
But we have to update already existing data.
The structure of the data is:
[
{
resources: event.resources, // Array
events: event.events, // Array
resourceTimeRanges: event.resourceTimeRanges, // Array
calendars: event.calendars, // Array
project: event.project, // Array
},
];
Via websocket we get changes only for the events prop of the object.
So we have to update an entry in the events array.
How would it looks like?
Our code:
async onCacheEntryAdded(arg, { updateCachedData, cacheDataLoaded, cacheEntryRemoved }) {
const state = store.getState();
const currentUser = state.user.uuid;
console.log(currentUser);
try {
// wait for the initial query to resolve before proceeding
await cacheDataLoaded;
// when data is received from the socket connection to the server,
// if it is a message and for the appropriate channel,
// update our query result with the received message
const listener = (event) => {
schedulerData.util.updateData('getSchedulerEvents', undefined, (draft) => {
// Test data
const newEvent = { id: 2, name: 'newName' };
// How do i update data here???
});
updateCachedData((draft) => {
console.log(draft);
});
};
// client-side
socketClient.emit('schedulerRoom', 'join');
socketClient.on('scheduler', (payload) => {
console.log('Joined room!');
console.log(payload);
listener(payload);
});
} catch (e) {
console.log(e);
// no-op in case `cacheEntryRemoved` resolves before `cacheDataLoaded`,
// in which case `cacheDataLoaded` will throw
}
// cacheEntryRemoved will resolve when the cache subscription is no longer active
await cacheEntryRemoved;
// perform cleanup steps once the `cacheEntryRemoved` promise resolves
},
Thank you very much for your suggestions.
I want to set state in catch even if user offline but firebase V9 setDoc does not give anything in catch when user offline
For Example: in Authentication, if the user offline firebase gives (network error) in catch but in firestore "add document" no message from catch...
This is by design thanks to Firestore's Offline Behaviour (queued up to the right spot, but I do recommend watching in full).
The promise will resolve once the server has acknowledged the request. If the server is currently unavailable, that request is cached within the SDK and attempted as soon as a connection is restored. During this window, the Promise will be kept in its pending state because that's the state its actually in - pending. While the promise may not resolve, all your local realtime listeners and such will still fire off and your app will function as normal - just offline.
Dealing with this behaviour is an exercise for the developer. One way to approach this would be to use Promise.race() to implement your own offline-handling logic.
As a quick & dirty example, here's a setDocWithTimeout implementation:
const setDocWithTimeout = (ref, data, options) => {
const timeoutMS = options && options.timeout || 10000;
const setDocPromise = setDoc(ref, data);
return Promise.race([
setDocPromise.then(() => ({ timeout: false })),
new Promise((resolve, reject) => setTimeout(resolve, timeoutMS, { timeout: true, promise: setDocPromise }));
]);
}
which you can apply using:
try {
const result = await setDocWithTimeout(doc(db, "cities", "new-city-2"), data);
if (result.timeout) {
// offline? (could be poor connection too)
console.log("Document added to Firestore queue");
// dismiss any blocking UIs/loading bubbles
// tell user will be written to server later
await result.promise; // wait for write to complete as before
}
// online! (or back online)
console.log("Document written successfully!");
} catch (err) {
console.error(`error found! ${err}`);
}
Alternatively where an error is thrown:
const setDocWithTimeoutError = (ref, data, options) => {
const timeoutMS = options && options.timeout || 10000;
const setDocPromise = setDoc(ref, data);
return Promise.race([
setDocPromise,
new Promise((_, reject) => setTimeout(reject, timeoutMS, new Error("timeout"));
]);
}
which you can apply using:
try {
await setDocWithTimeoutError(doc(db, "cities", "new-city-2"), data);
console.log("Document written successfully!");
} catch (err) {
console.error(`error found! ${err}`);
}
works on web v9, see
docs from v8.
import { onLog } from 'firebase/app';
onLog((e) => {
const { level, message } = e;
if (level === 'warn') {
console.log('connection interruption after intial load was success:', message);
}
if (level === 'error') {
console.log('no connection on inital load:', message);
}
});
I implemented a chatscreen inside my app and the following code represents the important sample of the code and I noticed that something about the data usage is very odd. The code is a little bit longer code sample but I will explain it after that.
const CountryChat = props =>{
var chosenLanguage = useSelector(state => state.myLanguage.myLanguage);
const countryId = props.navigation.getParam("countryId");//already upper case so no worries about correct firestore adress
const countryName = props.navigation.getParam("countryName");
const userId = useSelector(state => state.auth.userId);
const [TItext, setTItext] = useState("");
const [chatmessages, setChatMessages] = useState(() => []);//dummydata so FlatList wont crash because messages are empty during first renderprocess
const [refreshFlatlist, setRefreshFlatList] = useState(false);
const [myProfilePic, setMyProfilePic] = useState(null);
useEffect(() => {
downloadProfilePic();
var loadnewmessages = firebase.firestore().collection("group_rooms").doc("group_rooms").collection(`${countryId}`).orderBy("timestamp").limit(30).onSnapshot((snapshot) => {
var newmessages = [];
var deletedmesssages = [];
snapshot.docChanges().forEach((change) => {
if(change.type === "added"){
newmessages.push({
counter: change.doc.data().counter,
sender: change.doc.data().sender,
timestamp: change.doc.data().timestamp.toString(),
value: change.doc.data().value,
displayedTime: new Date(change.doc.data().displayedTime),
senderProfilePic: change.doc.data().senderProfilePic
})
};
if(change.type === "removed"){
deletedmesssages.push({
counter: change.doc.data().counter,
sender: change.doc.data().sender,
timestamp: change.doc.data().timestamp.toString(),
value: change.doc.data().value,
displayedTime: new Date(change.doc.data().displayedTime),
senderProfilePic: change.doc.data().senderProfilePic
})
};
})
if(newmessages.length > 0){
setChatMessages(chatmessages => {
return chatmessages.concat(newmessages)
});
};
if(deletedmesssages.length > 0){
setChatMessages(chatmessages => {
var modifythisarray = chatmessages;
let index = chatmessages.map(e => e.timestamp).indexOf(`${deletedmesssages[0].timestamp}`);
let pasttime = Date.now() - parseInt(modifythisarray[index].timestamp);
modifythisarray.splice(index, 1);
if(pasttime > 300000){
return chatmessages
}else{
return modifythisarray
}
});
setRefreshFlatList(refreshFlatlist => {
//console.log("Aktueller Status von refresher: ", refreshFlatlist);
return !refreshFlatlist
});
}
newmessages = [];
deletedmesssages = [];
});
return () => { //for removing listeners
try{
loadnewmessages();
}catch(error){console.log(error)};
}
}, []);
const pushMessagetoDB = async (filter, imageName) => {
//sending message to the chatroom in Firestore
if(filter == 1){
await firebase.firestore().collection("group_rooms").doc("group_rooms").collection(`${countryId}`).add({
"counter": 1,
"sender": userId,
"timestamp": Date.now(),
"value": TItext,
"displayedTime": (new Date()).toISOString(),
"senderProfilePic": myProfilePic
})
.then(() => {
console.log("Chat written in DB!");
})
.catch((error) => {
console.error("Error writing Chat into DB: ", error);
});
}else{
await firebase.firestore().collection("group_rooms").doc("group_rooms").collection(`${countryId}`).add({
"counter": 2,
"sender": userId,
"timestamp": Date.now(),
"senderProfilePic": myProfilePic,
"value": await firebase.storage().ref(`countrychatimages/${countryId}/${imageName}`).getDownloadURL().then((url) => {
return url
}).catch((error) => { //incase something bad happened
console.log(error);
})
})
.then(() => {
console.log("Image passed to DB!");
})
.catch((error) => {
console.error("Error passing Image to DB: ", error);
});
}
};
What you can see here is my listener loadnewmessages which is beeing called inside my useEffect. This listener downloads the recent 30 messages in the chat and stores them in a state. The chat works perfect and I can even send a message (store a document on the firestore inside a collection which represents the chat). After I leave the screen the return in the useEffect is fired and my listener is getting canceled.
My problem is: I went back and forth around 4 times and I had 6 messages in my collection. After I did that I closed my app and checked my usage in "Usage and billing" in firebase and saw that suddenly I had around 25 reads. I was expecting that my listener will only download the collection with the documents once and will maintain in on the phone even if I leave the screen, not that I redownload it always when I check the screen, that is what I assume is happening after I saw my usage in my firebase console. If I launch my app and I receive 100 or more users, my billings will explode this way.
I know that I detach my listener and relaunch it but I expected firebase to maintain the already loaded data on the phone so I (if no new files will be written) I only get 1 read because the query run without loading any new data.
Can somebody pls explain to me what I did wrong or how I could improve my code to shrink down the reads? How can I change my code so it stays efficient and does not download already loaded data? Its really important for me to maintain the reads on a low level, I have big problems getting this under control and my money is very limited.
That is the intended behavior. When you switch your pages/activities the listener is closed. A listener will fetch all the matching documents specified in query when it's reconnected (just like being connected for first time) as mentioned in the docs:
An initial call using the callback you provide creates a document snapshot immediately with the current contents of the single document. Then, each time the contents change, another call updates the document snapshot.
You can try:
Enabling offline persistence which caches a copy of the Cloud Firestore data that your app is actively using, so your app can access the data when the device is offline. If the documents are fetched from the cache then you won't be charged reads. However I am not sure if this will be the best option for your use case.
Storing messages fetched so far in local storage of that platform and then query messages sent after message using the listener. You would have to remove messages from local storage if any message is deleted.
const messagesRef = db..collection("group_rooms").doc("group_rooms").collection(`${countryId}`);
return messagesRef.doc("last_local_msg_id").get().then((doc) => {
// Get all messages sent after last local msg
const newMessagesQuery = messagesRef
.orderBy("timestamp")
.startAt(doc)
.limit(30);
});
Using for example async storage suits good, even increasing the size of the memory of async storage is not a problem so that its possible to store more data and therefore more chats as showed here.
I was doing some POC on firebase cloud functions and made a CF with below snippet (This is working code snippet).
app.post('/create-pdf', (req, res) => {
pdfPromise.toFile( os.tmpdir() + '/template.pdf', (err, data) => {
if(err) {
console.log('Error Saving File', err);
res.send(Promise.reject());
}
res.send(Promise.resolve());
});
})
app.get('/get-pdf', (req, res) => {
res.sendFile(`${os.tmpdir()}/template.pdf`);
})
And the call to above api will be like this.
axios.post(url+'/create-pdf', { data : poBody }).then((res) => {
}).then(() => {
axios.get(url+'/get-pdf', { responseType: 'blob' }).then(res => {
const pdfBlob = new Blob([res.data], { type: 'application/pdf' });
saveAs(pdfBlob, 'payout.pdf')
})
})
This code is working fine..
I just want to know if server side code is running on GCP as CF, then do we need to clear the memory consumed by os.tmpdir(), or will it be cleared automatically?
Yes, you do need to delete the temporary files created in the temporary directory (which is an in-memory filesystem), because "files that you write consume memory available to your function, and sometimes persist between invocations".
There is a specific doc section and a video about that: https://firebase.google.com/docs/functions/tips#always_delete_temporary_files and https://www.youtube.com/watch?v=2mjfI0FYP7Y&feature=youtu.be
I'm using Ionic 2 (Angular 2) for a Hybrid app. I inject a shared provider into the page that will display data from my SQLite3 database and then proceed to load the data. However, on creation of my database provider opening the database takes some time (very little). My code (as of this moment) however does not wait for the database to be opened before querying, which obviously results in an error.
How can I structure my code that it will wait for the database to be opened in order to evade a crash?
The constructor of my database provider:
constructor(private platform: Platform) {
this.platform.ready().then(() => {
if(this.isOpen !== true) {
this.storage = new SQLite();
this.storage.openDatabase({name: "data.db", location: "default"}).then(() => {
this.isOpen = true;
this.storage.executeSql("CREATE TABLE IF NOT EXISTS people (id INTEGER PRIMARY KEY AUTOINCREMENT, firstname TEXT, lastname TEXT)", []);
});
}
});
console.log('Hello Database Provider');
This provider gets injected into the constructor of my page.
When the page (home page) is loaded it triggers an event that calls a load() function.
ionViewDidLoad() {
this.load();
console.log('Hello Home Page');
The load function:
public load() {
this.database.getPeople().then((result) => {
this.itemList = <Array<Object>> result;
}, (error) => {
console.log("LOAD ERROR: ", error);
});
I'm very much hoping someone can point me in the right direction :)
I've finally found a solution to my problem.
For starters I've added a function to my provider that checks if the database is loaded, if it isn't it proceeds to load it:
public openSQLiteDatabase() {
return new Promise((resolve, reject) => {
if(this.isOpen) {
console.log("DB IS OPEN");
resolve(this.isOpen);
}
else {
console.log("DB IS NOT OPEN");
this.platform.ready().then(() => {
this.storage.openDatabase({name: "data.db", location: "default"}).then(() => {
this.appsettings.openSQLiteDatabase().then(() => {
this.appsettings.getSettings().then((result) => {
let settings: Settings = <Settings> result;
this.selectedDataset = settings.selectedDataset;
this.isOpen = true;
resolve(this.isOpen);
});
});
}, (error) => {
reject(error);
});
});
}
});}
As this function returns a promise (JS Promises) it allows me to wait for the database to be opened before doing anything else (such as querying).
My function in the page-specific TypeScript file:
ionViewDidLoad() {
this.database.openSQLiteDatabase().then(() => {
this.loadDictionary();
});}
With code like this I never have problems with queries being performed before my database has been opened!