I am implementing a process to store hundreds of thousands of records and retrieve them by realm query.
The simplified code is as follows
Realm Object
final class Friend: Object {
#objc dynamic var number: Int64 = 0
#objc dynamic var name: String = ""
convenience init(number: Int64, label: String) {
self.init()
self.number = number
self.label = label
}
override class func primaryKey() -> String? {
return "number"
}
}
extension Realm {
static var `default`: Realm {
get {
do {
let realm = try Realm(configuration: RealmConstants.configuration)
return realm
} catch {
return self.default
}
}
}
}
RealmClient class
struct RealmClient {
static func save(_ list: [Friend],
completion: #escaping (Result<Void, Error>) -> Void) {
DispatchQueue.global().async {
autoreleasepool {
do {
let realm = Realm.default
try realm.write {
realm.add(list)
completion(.success(()))
}
} catch {
completion(.failure(error))
}
}
}
}
}
DataStore class (shared file with Call directory extension target)
class DataStore {
let realm = realm
init(realm: Realm.default) {
self.realm = realm
}
var recordcounts: Int {
return realm.objects(Friend.self).count
}
}
However, sometimes the realm query returns 0 records.
Query results
RealmClient.save(friendList) -> friendList is hundreds of thousands of data fetched from server
let dataStore = DataStore()
dataStore.recordcounts -> sometimes return 0
So I have implemented refreshing the realm instance, but it doesn't help.
Like this.
realm.refresh()
Question
I want to be able to always get the data in a query.
How can I implement stable data using a query?
Realm writes are transactional - the transaction is a list of read and write operations that Realm treats as a single indivisible operation.
The reads and writes within the closure {} following the write are part of the transaction; the data is not committed until all of those complete. Transactions are also all or nothing - either all operations complete or none.
Most importantly for this case, transactions are synchronous so adding a callback within the transaction is going to present intermittent results.
Move the callback outside the write transaction
try realm.write {
realm.add(list)
}
completion(.success(()))
It can be illustrated by how we used to write realm transactions:
realm.beginWrite()
write some data
// Commit the write transaction to make this data available to other threads
try! realm.commitWrite() <- data is not valid until after it's been committed
//data has been committed
Related
This question already has answers here:
How to return a DocumentSnapShot as a result of a method?
(2 answers)
Closed 10 months ago.
I am very sorry if I break some rules, or if this has already been asked before. I have used so much time to google examples, and questions on stack overflow and other recourses. But I can simply not understand how I can get a document field from a firestore collection, and show the string value in a jetpack compose text function.
I am a very beginner in programming and Android. So I properly has some fundamental misunderstanding how I should do it but here is my attempt which doesn't work, and I can not understand why.
In Firestore I have a collection, called users with a document called holidaySavings that has a field of type string called name.
I want to show the value of name in a composable text function.
I have a class called storedData that handles Firestore. It has methods for creating and update a collection /document /fields. That works.
But I cant seem to be able to read a field value from a document to a jetpack composable text.
I can read the value from a document field, to the Log in Android studio.
Here is my function in my class where I handle the Firestore database
fun readDataTestFinal(): String{
val docRef = db.collection("users").document("holidaySavings")
var returnTest = ""
docRef.get()
.addOnSuccessListener { document ->
if (document != null) {
Log.d("Rtest", "DocumentSnapshot data: ${document.data}")
// I want to return this so I can use it in a composable text view
returnTest = document.get("name").toString()
} else {
Log.d("Rtest", "No such document")
}
}
.addOnFailureListener { exception ->
Log.d("Rfail", "get failed with ", exception)
}
return returnTest
}
And here I try to read the value into a jetpack compose Text function.
var newStringFromStoredData by remember {
mutableStateOf(storedData().readDataTestFinal())
}
Text(
modifier = Modifier.background(color = Color.Blue),
text = newStringFromStoredData
)
When I run the app. everything compiles fine, and I get the value from the document field fine, and can see it in the Log in Android Studio.
But the Compose function where call Text with the value newStringFromStoredData it doesn't show on the screen?
Can anyone tell me what it is I don't understand, and how it could be done so I can use the firestore document field and show the value in a jetpack compose Text function?
The Firebase call is async, which means it will not return the data immediately. This is why the API uses a callback. Therefore, your function readDataTestFinal is always returning an empty string.
One solution you can use is transform your function in a suspend function and call it using a coroutine scope. For example:
suspend fun readDataTestFinal(): String {
val docRef = firestore.collection("users")
.document("holidaySavings")
return suspendCoroutine { continuation ->
docRef.get()
.addOnSuccessListener { document ->
if (document != null) {
continuation.resume(document.get("name").toString())
} else {
continuation.resume("No such document")
}
}
.addOnFailureListener { exception ->
continuation.resumeWithException(exception)
}
}
}
In the code above, we are converting a callback call to suspend function.
In your composable, you can do the following:
var newStringFromStoredData by remember {
mutableStateOf("")
}
Text(newStringFromStoredData)
LaunchedEffect(newStringFromStoredData) {
newStringFromStoredData =
try { readDataTestFinal() } catch(e: Exception) { "Error!" }
}
The LaunchedEffect will launch your suspend function and update the result as soon it loads.
A better option would be define this call in a View Model and call this function from it. But I think this answers your question and you can improve your architecture later. You can start from here.
The most convenient, quickest, and apparently the best-in-your-case patch would be to use what are called valueEventListeners.
Firebase provides these helpful methods for you, so that you can keep your app's data up-to-date with the firebase servers.
val docRef = db.collection("cities").document("SF")
docRef.addSnapshotListener { snapshot, e -> // e is for error
// If error occurs
if (e != null) {
Log.w(TAG, "Listen failed.", e)
return#addSnapshotListener
}
// If backend value not received, use this to get current local stored value
val source = if (snapshot != null && snapshot.metadata.hasPendingWrites())
"Local"
else
"Server"
// If request was successful,
if (snapshot != null && snapshot.exists()) {
Log.d(TAG, "$source data: ${snapshot.data}")
//Update your text variable here
newStringFromStoredData = snapshot.data // Might need type-conversion
} else {
Log.d(TAG, "$source data: null")
}
}
This will not only solve your problem as described in the question, but will also ensure that whenever the value on the server is changed/updated, your text will update alongside it. It is usually a good best practice to use these listeners, and these are often converted into LiveData objects for respecting the 'separation-of-concerns' principle, but you can use this simple implementation for the simple use-case described.
Another thing, this would usually go in a viewModel, and hence, you should declare you text variable inside the viewmodel too.
Try it in the init block.
calss MVVM: ViewModel() {
init {
/* Paste Code From Above Here */
}
var newStringFromStoredData by mutableStateOf("")
}
Then read it in the Composable
Text(viewModel.newStringFromStoredData)
I have a problem with Firebase.
I also use Firestore on a website and in a Kotlin app.
On the website, I can save the result of the query to a variation in the following way:
const addStudentManu = async($this) => {
const userId = await db.collection('users').where('neptun','==',ASD123).get();
const getUserId = userId.docs.map(doc=>doc.id);
}
How can i do this in kotlin?
This is how it goes:
db.collection("users")
.whereEqualTo("neptun", "ASD123")
.get()
.addOnSuccessListener { documents ->
val list = mutableListOf<String>()
for (document in documents) {
Log.d(TAG, "${document.id}")
list.add(document.id)
}
println(list)
}
.addOnFailureListener { exception ->
Log.w(TAG, "Error getting documents: ", exception)
}
You can checkout the sample code snippets in the documentation.
While #Dharmaraj answer will work perfectly fine, when it comes to Kotlin, the most convenient way for saving the result of a query would be to use Kotlin Coroutines, We can create a suspend function and map all documents to their corresponding IDs, similar with your example. So please try the following lines of code:
private suspend fun getIdsFromFirestore(): List<String> {
val ids = db.collection("users").whereEqualTo("neptun", "ASD123").get().await()
return ids.documents.mapNotNull { doc ->
doc.id
}
}
As you can see, we have now an extension function called await() that will interrupt the Coroutine until the data from the database is available and then return it. That's almost the same thing when using async on the web.
Now we can simply call this from another suspend method like in the following lines of code:
private suspend fun getIds() {
try {
val ids = getIdsFromFirestore()
// Do what you need to do with the list of IDs
} catch (e: Exception) {
Log.d(TAG, e.getMessage()) //Don't ignore potential errors!
}
}
I'm having problems to execute a https requests, if the request don't have any error i never get the message, this is a command line tool application and i have a plist to allow http requests, i always see the completion block.
typealias escHandler = ( URLResponse?, Data? ) -> Void
func getRequest(url : URL, _ handler : #escaping escHandler){
let session = URLSession.shared
var request = URLRequest(url:url)
request.cachePolicy = .reloadIgnoringLocalCacheData
request.httpMethod = "GET"
let task = session.dataTask(with: url ){ (data,response,error) in
handler(response,data)
}
task.resume()
}
func startOp(action : #escaping () -> Void) -> BlockOperation{
let exOp = BlockOperation(block: action)
exOp.completionBlock = {
print("Finished")
}
return exOp
}
for sUrl in textFile.components(separatedBy: "\n"){
let url = URL(string: sUrl)!
let queu = startOp {
getRequest(url: url){ response, data in
print("REACHED")
}
}
operationQueue.addOperation(queu)
operationQueue.waitUntilAllOperationsAreFinished()
One problem is that your operation is merely starting the request, but because the request is performed asynchronously, the operation is immediately completing, not actually waiting for the request to finish. You don't want to complete the operation until the asynchronous request is done.
If you want to do this with operation queues, the trick is that you must subclass Operation and perform the necessary KVO for isExecuting and isFinished. You then change isExecuting when you start the request and isFinished when you finish the request, with the associated KVO for both. This is all outlined in the Concurrency Programming Guide: Defining a Custom Operation Object, notably in the Configuring Operations for Concurrent Execution section. (Note, this guide is a little outdated (it refers to the isConcurrent property, which has been replaced is isAsynchronous; it's focusing on Objective-C; etc.), but it introduces you to the issues.
Anyway, This is an abstract class that I use to encapsulate all of this asynchronous operation silliness:
/// Asynchronous Operation base class
///
/// This class performs all of the necessary KVN of `isFinished` and
/// `isExecuting` for a concurrent `NSOperation` subclass. So, to developer
/// a concurrent NSOperation subclass, you instead subclass this class which:
///
/// - must override `main()` with the tasks that initiate the asynchronous task;
///
/// - must call `completeOperation()` function when the asynchronous task is done;
///
/// - optionally, periodically check `self.cancelled` status, performing any clean-up
/// necessary and then ensuring that `completeOperation()` is called; or
/// override `cancel` method, calling `super.cancel()` and then cleaning-up
/// and ensuring `completeOperation()` is called.
public class AsynchronousOperation : Operation {
override public var isAsynchronous: Bool { return true }
private let lock = NSLock()
private var _executing: Bool = false
override private(set) public var isExecuting: Bool {
get {
return lock.synchronize { _executing }
}
set {
willChangeValue(forKey: "isExecuting")
lock.synchronize { _executing = newValue }
didChangeValue(forKey: "isExecuting")
}
}
private var _finished: Bool = false
override private(set) public var isFinished: Bool {
get {
return lock.synchronize { _finished }
}
set {
willChangeValue(forKey: "isFinished")
lock.synchronize { _finished = newValue }
didChangeValue(forKey: "isFinished")
}
}
/// Complete the operation
///
/// This will result in the appropriate KVN of isFinished and isExecuting
public func completeOperation() {
if isExecuting {
isExecuting = false
isFinished = true
}
}
override public func start() {
if isCancelled {
isFinished = true
return
}
isExecuting = true
main()
}
}
And I use this Apple extension to NSLocking to make sure I synchronize the state changes in the above (theirs was an extension called withCriticalSection on NSLock, but this is a slightly more generalized rendition, working on anything that conforms to NSLocking and handles closures that throw errors):
extension NSLocking {
/// Perform closure within lock.
///
/// An extension to `NSLocking` to simplify executing critical code.
///
/// - parameter block: The closure to be performed.
func synchronize<T>(block: () throws -> T) rethrows -> T {
lock()
defer { unlock() }
return try block()
}
}
Then, I can create a NetworkOperation which uses that:
class NetworkOperation: AsynchronousOperation {
var task: URLSessionTask!
init(session: URLSession, url: URL, requestCompletionHandler: #escaping (Data?, URLResponse?, Error?) -> ()) {
super.init()
task = session.dataTask(with: url) { data, response, error in
requestCompletionHandler(data, response, error)
self.completeOperation()
}
}
override func main() {
task.resume()
}
override func cancel() {
task.cancel()
super.cancel()
}
}
Anyway, having done that, I can now create operations for network requests, e.g.:
let queue = OperationQueue()
queue.name = "com.domain.app.network"
let url = URL(string: "http://...")!
let operation = NetworkOperation(session: .shared, url: url) { data, response, error in
guard let data = data, error == nil else {
print("\(error)")
return
}
let string = String(data: data, encoding: .utf8)
print("\(string)")
// do something with `data` here
}
let operation2 = BlockOperation {
print("done")
}
operation2.addDependency(operation)
queue.addOperations([operation, operation2], waitUntilFinished: false) // if you're using command line app, you'd might use `true` for `waitUntilFinished`, but with standard Cocoa apps, you generally would not
Note, in the above example, I added a second operation that just printed something, making it dependent on the first operation, to illustrate that the first operation isn't completed until the network request is done.
Obviously, you would generally never use the waitUntilAllOperationsAreFinished of your original example, nor the waitUntilFinished option of addOperations in my example. But because you're dealing with a command line app that you don't want to exit until these requests are done, this pattern is acceptable. (I only mention this for the sake of future readers who are surprised by the free-wheeling use of waitUntilFinished, which is generally inadvisable.)
I'm new to RxJava. I am currently modifying an existing synchronous flow and making asynchronous using returned observables for the existing methods.
In one of the flows I make a remote call to receive an item from the DB. since the call to the database is asynchronous I am given back an observable. however, the item might not exist in the database in which case the value will be NULL.
if the value is NULL I need to go and make additional asynchronous calls to various other remote services and eventually return the observable response.
what I don't understand is how to implement such a flow with RxJava. here is a sample pseudo code:
void searchSomethingAsych(String key) {
Observable<SearchResult> result = doTheSearch(key);
}
Observable<SearchResult> doTheSearch(String key) {
Observable<SearchResult> resultFromDb = checkIfExistsInDb(key);
// THIS IS WHERE I AM NOT SURE HOW TO DO THIS
resultFromDb.subscribe((result)- > {
if(result == null){
// .. go get this from somewhere else
Observable<SearchResult> resultFromSomewhere = getSearchResultFromSomewhereElse(key);
// how do I return the 'resultFromSomewhere' ????
}
});
}
You can use Observable.flatmap(func), where func is function returning another Observable:
resultFromDb.flatMap((result) -> {
if(result == null){
return getSearchResultFromSomewhereElse(key);
} else {
return Observable.just(result)
});
I have a desire to add a property with a default value to a set of documents that I retrieve via a SELECT query if they contain no value.
I was thinking of this in two parts:
SELECT * FROM c article WHERE article.details.locale = 'en-us'
I'd like to find all articles where article.details.x does not exist.
Add the property, article.details.x = true
I was hoping this EXEC command could be supported via the Azure Portal so I don't have to create a migration tool to run this command once but I couldn't find this option in the portal. Is this possible?
You can use Azure Document DB Studio as a front end to creating and executing a stored procedure. It can be found here. It's pretty easy to setup and use.
I've mocked up a stored procedure based on your example:
function updateArticlesDetailsX() {
var collection = getContext().getCollection();
var collectionLink = collection.getSelfLink();
var response = getContext().getResponse();
var docCount = 0;
var counter = 0;
tryQueryAndUpdate();
function tryQueryAndUpdate(continuation) {
var query = {
query: "select * from root r where IS_DEFINED(r.details.x) != true"
};
var requestOptions = {
continuation: continuation
};
var isAccepted =
collection
.queryDocuments(collectionLink,
query,
requestOptions,
function queryCallback(err, documents, responseOptions) {
if (err) throw err;
if (documents.length > 0) {
// If at least one document is found, update it.
docCount = documents.length;
for (var i=0; i<docCount; i++){
tryUpdate(documents[i]);
}
response.setBody("Updated " + docCount + " documents");
}
else if (responseOptions.continuation) {
// Else if the query came back empty, but with a continuation token;
// repeat the query w/ the token.
tryQueryAndUpdate(responseOptions.continuation);
} else {
throw new Error("Document not found.");
}
});
if (!isAccepted) {
throw new Error("The stored procedure timed out");
}
}
function tryUpdate(document) {
//Optimistic concurrency control via HTTP ETag.
var requestOptions = { etag: document._etag };
//Update statement goes here:
document.details.x = "some new value";
var isAccepted = collection
.replaceDocument(document._self,
document,
requestOptions,
function replaceCallback(err, updatedDocument, responseOptions) {
if (err) throw err;
counter++;
});
// If we hit execution bounds - throw an exception.
if (!isAccepted) {
throw new Error("The stored procedure timed out");
}
}
}
I got the rough outline for this code from Andrew Liu on GitHub.
This outline should be close to what you need to do.
DocumentDB has no way in a single query to update a bunch of documents. However, the portal does have a Script Explorer that allows you to write and execute a stored procedure against a single collection. Here is an example sproc that combines a query with a replaceDocument command to update some documents that you could use as a starting point for writing your own. The one gotcha to keep in mind is that DocumentDB will not allow sprocs to run longer than 5 seconds (with some buffer). So you may have to run your sproc multiple times and keep track of what you've already done if it can't complete in one 5 second run. The use of IS_DEFINED(collection.field.subfield) != true (thanks #cnaegle) in your query followed up by a document replacement that defines that field (or removes that document) should allow you to run the sproc as many times as necessary.
If you didn't want to write a sproc, the easiest thing to do would be to export the database using the DocumentDB Data Migration tool. Import that into Excel to manipulate or write a script to do the manipulation. Then upload it again using the Data Migration tool.