Stream and Future in Dart - asynchronous

I've been using basic async/await for some time without many problems and I thought I understood how it worked. Can't say I'm an expert in it, but I understadn the gist of it. I just can't get my head around Streams though. Before today I thought I understood how they worked (basically ala Reactive Programming), but I can't get them to work in Dart.
I'm working on a persistance layer with the possibility of saving and retrieving (json) files. I've been using the fileManager example as a guideline.
import 'dart:io';
import 'dart:async';
import 'package:intl/intl.dart'; //date
import 'package:markdowneditor/model/note.dart';//Model
import 'package:path_provider/path_provider.dart';
import 'package:path/path.dart' as p;
import 'package:flutter/foundation.dart'; //log
import 'package:simple_permissions/simple_permissions.dart';//OS permissions
class FileManager {
static final FileManager _singleton = new FileManager._internal();
factory FileManager() {
return _singleton;
}
FileManager._internal();
Future<String> get _localPath async {
final directory = (await getApplicationDocumentsDirectory()).toString();
return p.join(directory, "notes"); //path takes strings and not Path objects
}
Future<File> writeNote(Note note) async {
var file = await _localPath;
file = p.join(
file,
DateFormat('kk:mm:ssEEEMMd').format(DateTime.now()) +
" " +
note.title); //add timestamp to title
// Write the file
SimplePermissions.requestPermission(Permission.WriteExternalStorage)
.then((value) {
if (value == PermissionStatus.authorized) {
return File(file).writeAsString('$note');
} else {
SimplePermissions.openSettings();
return null;
}
});
}
Future<List<Note>> getNotes() async {
//need file access permission on android. use https://pub.dartlang.org/packages/simple_permissions#-example-tab-
final file = await _localPath;
SimplePermissions.requestPermission(Permission.ReadExternalStorage)
.then((value) {
if (value == PermissionStatus.authorized) {
try {
Stream<FileSystemEntity> fileList =
Directory(file).list(recursive: false, followLinks: false);
// await for(FileSystemEntity s in fileList) { print(s); }
List<Note> array = [];
fileList.forEach((x) {
if (x is File) {
var res1 = ((x as File).readAsString()).then((value2) {
Note note = Note.fromJsonResponse(value2);
return note;
}).catchError((error) {
debugPrint('is not file content futurestring getNoteError: $x');
return null;
});
var array2 = res1.then((value3) {
array.add(value3);
return array;
});
//?
} else {
debugPrint('is not file getNoteError: $x');
}
});
// Add the file to the files array
//Return the Future<List<Note>>
return array;
} catch (e) {
debugPrint('getNoteError: $e');
// If encountering an error, return 0
return null;
}
} else {
SimplePermissions.openSettings();
return null;
}
});
}
}
Obviously as it is it won't work, but even trying to await the loop using the commented out parts raises an error.
In "getNotes", after checking the permissions I want to get an array of all the files in the directory, parse them as Note objects and return the resulting array.
I get the list of files:
Stream<FileSystemEntity> fileList =
Directory(file).list(recursive: false, followLinks: false);
And for each one of them in the stream I want to parse the file into an object and append it to an array to return at the end.
List<Note> array = [];
fileList.forEach((x) {
if (x is File) {
var res1 = ((x as File).readAsString()).then((value2) {
Note note = Note.fromJsonResponse(value2);
return note;
}).catchError((error) {
debugPrint('is not file content futurestring getNoteError: $x');
return null;
});
var array2 = res1.then((value3) {
array.add(value3);
return array;
});
//?
} else {
debugPrint('is not file getNoteError: $x');
}
});
// Add the file to the files array
//Return the Future<List<Note>>
return array;

Stream.forEach() returns a Future. Your last return statement runs immediately after the for-each call, but should await it.
await fileList.forEach((x) {
...
https://api.dartlang.org/stable/2.2.0/dart-async/Stream/forEach.html

Related

Why is the asnyc/await function not been waited?

I am doing an Image Upload feature with Cloudinary. I'm providing an array which may contains base64coded or uploaded image which is a url :
[
"https://res.cloudinary.com/\[userName\]/image/upload/v167xxxx4/luxxxfsgasxxxxxx7t9.jpg", "https://res.cloudinary.com/doeejabc9/image/upload/v1675361225/rf6adyht6jfx10vuzjva.jpg",
"data:image/jpeg;base64,/9j/4AAUSkZJRgABAQEBLAEsAA.......", "data:image/jpeg;base64,/9j/4AAUSkZJRgABAQEBLAEsAA......."
]
I'm using this function to upload the "un-uploaded", which returns the all uploaded version:
export async function uploadImage(el: string[]) {
const partition = el.reduce(
(result: string[][], element: string) => {
element.includes("data:image/")
? result[0].push(element)
: result[1].push(element);
return result;
},
[[], []]
);
for (let i = 0; i < partition[0].length; i++) {
const data = new FormData();
data.append("file", partition[0][i]);
data.append("upload_preset", "my_preset_name");
const res = await fetch(
"https://api.cloudinary.com/v1_1/userName/image/upload",
{
method: "POST",
body: data,
}
);
const file = await res.json();
partition[1].push(file.secure_url);
console.log(partition[1]);
}
return partition[1];
}
Then I will use the return value to update the state and call the api to update database:
const uploaded = await uploadImage(el[1])
console.log(uploaded);
setFinalVersionDoc({
...chosenDocument,
[chosenDocument[el[0]]]: uploaded,
});
However, it always updates the useState before the console.log(uploaded). I thought async/await would make sure the value is updated before moving on.
The GitHub repo is attached for better picture. The fragment is under EditModal in the 'component/document' folder:
https://github.com/anthonychan1211/cms
Thanks a lot!
I am hoping to make the upload happen before updating the state.
The function is correct, but you are trying to await the promise inside the callback function of a forEach, but await inside forEach doesn't work.
This doesn't work:
async function handleEdit() {
const entries = Object.entries(chosenDocument);
entries.forEach(async (el) => { // <------ the problem
if (Array.isArray(el[1])) {
const uploaded = await uploadImage(el[1]);
el[1].splice(0, el[1].length, uploaded);
}
});
[...]
}
If you want to have the same behaviour (forEach runs sequentially), you can use a for const of loop instead.
This works (sequentially)
(execution order guaranteed)
async function handleEdit() {
const entries = Object.entries(chosenDocument);
for (const el of entries) {
// await the promises 1,2,...,n in sequence
if (Array.isArray(el[1])) {
const uploaded = await uploadImage(el[1]);
el[1].splice(0, el[1].length, uploaded);
}
}
}
This also works (in parallel)
(execution order not guaranteed)
async function handleEdit() {
const entries = Object.entries(chosenDocument);
await Promise.all(entries.map(async (el) => {
// map returns an array of promises, and await Promise.all() then executes them all at the same time
if (Array.isArray(el[1])) {
const uploaded = await uploadImage(el[1]);
el[1].splice(0, el[1].length, uploaded);
}
}));
[...]
}
If the order in which your files are uploaded doesn't matter, picking the parallel method will be faster/better.

Get async results within .map function

I'm trying to get photos in map function from Nest.js server. This is slider component. I give an array of slides as input and show on within a map.
The thing is, I need to get real file from server (by res.sendFile..) while doing .map.
I searched and tried a few variants, this is the last, but still get errors.
what do I do wrong? Please, help
const Slider = async ({videos}) => {
async function getImageHandler(fileName) {
return getImageFromNestServer(fileName).then(async (response) => {
const fileType = await FileType.fromBuffer(response.data);
return `data:${fileType.mime};base64, ${ArrayBufferConverter.encode(response.data)}`;
})
}
let realImg = await Promise.all(
videos.map(async video => {
try {
video.fetchItem = await getImageHandler(video.content_preview_photo.filename)
return video;
} catch(err) {
throw err;
}
}
)
)
return (<MySlide>...</MySlide>)
}

Firebase storage download audio files in React Native

I'm using firebase/storage to set up audio file downloading/uploading. I have the audio file in my firestore storage already.
With the following code, I am able to get the download URL of the specific file:
import firebase from 'firebase/app';
import 'firebase/firestore';
import 'firebase/storage';
static async downloadMedia(mediaRef: string) {
try {
var storage = firebase.storage();
var pathReference = storage.ref(mediaRef);
const downloadUrl = await pathReference.getDownloadURL();
var xhr = new XMLHttpRequest();
xhr.responseType = 'blob';
xhr.onload = (event) => {
var blob = xhr.response;
};
xhr.open('GET', downloadUrl);
return downloadUrl;
} catch (e) {
switch (e.code) {
case 'storage/object-not-found':
console.warn('File does not exist.');
break;
case 'storage/unauthorized':
console.warn('Unauthorized.');
break;
case 'storage/canceled':
console.warn('Upload cancelled.');
break;
case 'storage/unknown':
console.warn('Unknown error.');
break;
}
}
}
However, I do not understand how to use the firebase library to download the file itself with the URL that it provides me.
Thanks.
Found a solution which doesn't involve me downloading the media but instead playing it directly with the download URL.
Using package 'expo-av'.
Hope this helps someone in my shoes!
export default function AudioPlay({ mediaDownloadUrl } : AudioPlayProps) {
const [sound, setSound] = React.useState<Audio.Sound | null>(null);
async function playSound() {
if (typeof mediaDownloadUrl !== 'string') return null;
try {
const { sound } = await Audio.Sound.createAsync(
{ uri: mediaDownloadUrl }
);
setSound(sound);
await sound.playAsync();
} catch (e) {
console.warn(e);
}
}
React.useEffect(() => {
return sound
? () => {
console.log('Unloading Sound');
sound.unloadAsync(); }
: undefined;
}, [sound]);
// ....

Read and write whole directories with tar archive (Standard Library)

This is a continuation of linked question.
It seems to me, that current implementation of the std/archive/tar.ts module only allows reads and writes per file and not for whole directories.
So far, my reference source are the test files, which only show case single file processing. But what if, for example, a directory ./my-dir/ with multiple files and a tar archive ./test.tar is given.
How can I then utilize append, extract & Co. to efficiently write ./my-dir/ to ./test.tar archive and read all file contents back from it?
You can archive a directory by using std/fs/walk
import { walk, walkSync } from "https://deno.land/std/fs/walk.ts";
import { Tar } from "https://deno.land/std/archive/tar.ts";
// Async
const tar = new Tar();
for await (const entry of walk("./dir-to-archive")) {
if (!entry.isFile) {
continue;
}
await tar.append(entry.path, {
filePath: entry.path,
});
}
const writer = await Deno.open("./out.tar", { write: true, create: true });
await Deno.copy(tar.getReader(), writer);
Untar implementation for folders/multiple files was broken, it was fixed by this PR and currently available in master using https://deno.land/std/archive/tar.ts
import { Untar } from "https://deno.land/std/archive/tar.ts";
import { ensureFile } from "https://deno.land/std/fs/ensure_file.ts";
import { ensureDir } from "https://deno.land/std/fs/ensure_dir.ts";
const reader = await Deno.open("./out.tar", { read: true });
const untar = new Untar(reader);
for await (const entry of untar) {
console.log(entry); // metadata
/*
fileName: "archive/deno.txt",
fileMode: 33204,
mtime: 1591657305,
uid: 0,
gid: 0,
size: 24400,
type: 'file'
*/
if (entry.type === "directory") {
await ensureDir(entry.fileName);
continue;
}
await ensureFile(entry.fileName);
const file = await Deno.open(entry.fileName, { write: true });
// <entry> is a reader
await Deno.copy(entry, file);
}
reader.close();
Update
Created a lib to allow transformations, including gzip/gunzip to create & read .tar.gz
gzip
import * as Transform from "https://deno.land/x/transform/mod.ts";
const { GzEncoder } = Transform.Transformers;
/** ... **/
const writer = await Deno.open("./out.tar.gz", { write: true, create: true });
await Transform.pipeline(tar.getReader(), new GzEncoder())
.to(writer);
writer.close();
gunzip
import * as Transform from "https://deno.land/x/transform/mod.ts";
const { GzDecoder } = Transform.Transformers;
/** ... **/
const reader = await Deno.open("./out.tar.gz", { read: true });
const untar = new Untar(
Transform.newReader(input, new GzDecoder())
);
for await (const entry of untar) {
console.log(entry);
}

put a firestore variable in a polymer template

Maybe I am over thinking it, but I can't figure out a way to put the results of a Firestore query into a Polymer 3 template. For example:
class MyPage extends PolymerElement {
constructor() {
super();
/* somehow set this.users to the firestore users query results */
}
static get properties() {
return {
users: {
type: String
}
}
}
static get template() {
return html`<div>[[users]]</div>`;
}
}
Using the following code, which does work correctly and print to the console:
var all_users;
const setsRef = firestore.collection("users");
setsRef.get().then(function(querySnapshot) {
var users = [];
querySnapshot.forEach(function(doc) {
users.push(doc.data().verb);
});
all_users = users.join(", ");
console.log("All users: ", all_users);
/* run a function here that sets this.users = all_users */
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
The problem is, I have to run a function from the Firestore results, while the other is a constructor of an object. I would preferably like to have all of my Firebase queries in an external js file, but this async thought process is confusing me.
Use one of the lifecycle methods to load the users:
https://www.polymer-project.org/3.0/docs/devguide/custom-elements#element-lifecycle
class MyPage extends PolymerElement {
constructor() {
super();
this.users = []
}
static get properties() {
return {
users: {
type: String
}
}
}
static get template() {
return html`<div>[[users]]</div>`;
}
async ready() {
super.ready()
try {
const querySnapshot = await firestore.collection("users").get()
this.users = querySnapshot.map(doc => doc.data().verb).join(", ");
} catch (error) {
console.log(error)
}
}
}
If you don't want to use one of the lifescycle methods, then you can fire your own custom event which your MyPage element can listen for: https://www.polymer-project.org/3.0/docs/devguide/events#custom-events
I'm not familiar with Polymer, so the above is untested and what I figure from reading the docs.

Resources