Unable to upload media file toFirebase emulator - firebase

Using Firebase tools 11.21.0 and FIREBASE_STORAGE_EMULATOR_HOST=localhost:9199 and maven dependency
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-storage</artifactId>
<version>2.17.1</version>
</dependency>
I started the firebase emulator. And tried a simple file store:
emulatorStorage = StorageOptions.newBuilder()
.setProjectId(projectId)
.setHost("http://localhost:9199")
.setCredentials(NoCredentials.getInstance())
.build()
.getService();
And tried to save a file:
byte[] compress = "test".getBytes();
Blob blob = emulatorStorage.create(
BlobInfo.newBuilder(index, filename)
.setContentType("text/plain")
.build()
,compress,
Storage.BlobTargetOption.doesNotExist());
but even with the content type set I get this every time:
com.google.cloud.storage.StorageException: Failed to parse multipart request body part. Missing content type.
at com.google.cloud.storage.StorageException.translate(StorageException.java:163)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:297)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:379)
at com.google.cloud.storage.StorageImpl.lambda$internalCreate$2(StorageImpl.java:208)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.storage.Retrying.run(Retrying.java:60)
at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1476)
at com.google.cloud.storage.StorageImpl.internalCreate(StorageImpl.java:205)
at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:151)
and through debug I know that it is talking to the local emulator:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
POST http://localhost:9199/upload/storage/v1/b/demo-project.appspot.com/o?ifGenerationMatch=0&projection=full&uploadType=multipart
{
"code" : 400,
"message" : "Failed to parse multipart request body part. Missing content type."
}
What am I missing in the save operation, is the content type wrong? Or is this likely to be a bug in the emulator or compatibility issue with cloud-storage libs?

I have found github issue you raised and followed steps from there with both firebase v11.21.0 And v11.19.0 but I can successfully upload video files using firebase emulators:start --project demo-project --debug.
As per our conversation above in comments, it seems like you have mistaken firebase-tools-linux as a command. the doc you followed to set up the forebase for linux will just have downloadable file name as firebase-tools-linux It is just file name it does not mean that for linux we have to use firebase-tools-linux as a command for linux machines.If you observe step 3 in the doc you shared, It is pointing to log in and test the CLI where we have to use firebase login only. Hence try with firebase emulators:start --project demo-project --debug command.
Steps I have taken
Step 1:
Cloned source code from the github you shared. Changed directory to firebase-emulator-debug.
Step 2:
Ran below command
`firebase emulators:start --project demo-project --debug`.
Step 3:
Successfully uploaded video file of 2.3 MB from emulator.
FYI, I have also used the linux machine only for the above steps.

As mentioned by #Gridcell Coder, The Cloud Storage for Firebase emulator only supports a very small subset of the Cloud API and it is intended to only be used via the firebase-admin package. Admin SDK is not yet supported for Cloud Storage for Firebase.

Related

Firebase CLI suddenly ignoring environment variables on Functions deployment

I have a Firebase code project with Functions meant to be deployed to multiple Firebase projects over multiple regions.
I used to set the deployment region like this:
return functions
.region(process.env.REGION)
// ...
and used this command to deploy:
$ REGION=us-central1 firebase deploy --only functions
it worked like a charm until recently. Now it seems to completely ignore REGION=us-central1 even if I export it before I run firebase deploy.
EDIT 2022-06-13 - Possible solution
I changed the code to dump the contents of process.env to a file during deployment. This is what I got:
{
"FIREBASE_CONFIG": "{\"projectId\":\"REDACTED\",\"storageBucket\":\"REDACTED.appspot.com\",\"locationId\":\"us-central\"}",
"GCLOUD_PROJECT": "REDACTED",
"CLOUD_RUNTIME_CONFIG": "{REDACTED}",
"__CF_USER_TEXT_ENCODING": "REDACTED"
}
so definitely is not the same list of variables I have in my local environment.
I could use the locationId from FIREBASE_CONFIG to get the target location, or CLOUD_RUNTIME_CONFIG (it contains the dump of the functions .config() object, so I could set the target there).
I also believe that I could use the .env and .env.{project alias or ID} files and their contents would be available in process.env at deployment time.
As per Osvaldo López's suggestion, here are some other details:
Running on MacOS
No errors are reported
No recent changes to the CLI
Any input would be very welcome! Thanks.

Deploy on Meteor galaxy server with bitbucket and deployment token as variable

Hello I want to use the automatic deploymen on bitbucket to the galaxy server with a deployment token.
For this reason I am creating a deployment token that is comitted in the repository.
https://galaxy-guide.meteor.com/deploy-guide.html#deployment-token
To strenghten the security I would like to use Repository variables in bitbucket pipelines:
https://confluence.atlassian.com/bitbucket/environment-variables-794502608.html
And to store the deployment token of meteor in the variables instead in file.
For the deployment we use in the command:
METEOR_SESSION_FILE=deployment_token.json
And my question is - Is there any way so that I use some variable(string) where the token is used like
METEOR_SESSION_DEPLOYMENT_TOKEN=$METEOR_TOKEN
instead to call it from a file?
Some research, after having the same problem, brought me to this article, which simply solves the problem that you can't feed meteor just the json in an env var in the following simple way:
By adding the json file content as an env var and then echoes it out into a file on deploy.
echo $METEOR_TOKEN_FILE > deploy_token.json
METEOR_SESSION_FILE=deploy_token.json
Thanks to this article I figured it out.
Save json settings as env variable and then in deployment procesS:
echo $DEPLOY_SESSION_FILE > deployment_token.json
METEOR_SESSION_FILE=deployment_token.json DEPLOY_HOSTNAME=galaxy.meteor.com meteor deploy --allow-superuser myApp-staging.meteorapp.com --settings config/staging/settings.json --owner username

Upload dSYMS to Firebase via Fastlane

I am struggling to upload dSYM files to Firebase via Fastlane. I have a lane that looks like the following:
desc "Fetch and upload dSYM files to Firebase Crashlytics"
lane :refresh_dsyms_firebase do |options|
download_dsyms(version: options[:version])
upload_symbols_to_crashlytics(gsp_path: "./App/GoogleService-Info.plist")
clean_build_artifacts
end
I confirmed that that is the correct path to the plist file, but when I try to run the lane at first I see the following:
[17:22:47]: invalid byte sequence in UTF-8
[17:22:47]: invalid byte sequence in UTF-8
[17:22:47]: invalid byte sequence in UTF-8
and then one of these for every dSYM file found:
[17:22:48]: Uploading '70DBE65E-227E-3754-89F2-EEFA6B8EEC2F.dSYM'...
[17:22:48]: Shell command exited with exit status instead of 0.
I am trying to determine exactly what I am missing from this process. Does anyone have ideas? I am fairly new to Fastlane, so definitely assume I could be missing something basic. (Although, that empty exit status is a bit weird).
fastlane 2.107.0
EDIT(June 7th: 2021):
I updated the answer from my own to one that was helpful to me at the time this was written.
There are many other great answers on this page on using Fastlane as well - please check them out
This may not be an option for most, but I just ended up fixing this by starting over. It may not be entirely obvious if you came over from Fabric, but I figured I would just rip off the band aid. My original setup was using the Fabric(Answers)/Firebase Crashlytics which is the Fabric->Firebase migration path, although subtle, the configuration between the two are slightly different and cause issues with upload_symbols_to_crashlytics
Remove support for Fabric answers, or replace with https://firebase.google.com/docs/analytics/ios/start
Remove the Fabric declaration in Info.plist
Modify your existing run script in BuildPhases: replace your existing runscript with "${PODS_ROOT}/Fabric/run" and add $(BUILT_PRODUCTS_DIR)/$(INFOPLIST_PATH) to input files
In you AppDelegate remove Fabric.with([Crashlytics.self]) and you can also kill the import Fabric as this is now covered by Firebase
Unlink fabric, re-onboard Firebase crashlytics, and select new integration.
desc "Upload any dsyms in the current directory to crashlytics"
lane :upload_dsyms do |options|
download_dsyms(version: version_number, build_number: build_number)
upload_symbols_to_crashlytics(gsp_path: "./App/Resources/GoogleService-Info.plist")
clean_build_artifacts
end
anyone interested in this can follow the thread here: https://github.com/fastlane/fastlane/issues/13096
TL;DR: When you call
upload_symbols_to_crashlytics(gsp_path: "./App/GoogleService-Info.plist")
It will call a binary from the installed Fabric pod called upload_symbols and will look something like this:
./Pods/Fabric/upload-symbols -a db4d085462b3cd8e3ac3b50f118e273f077497b0 -gsp ./App/GoogleService-Info.plist -p ios /private/var/folders/x1/x6nqt4997t38zr9x7zwz72kh0000gn/T/d30181115-8238-1fr38bo/D4CE43B9-9350-3FEE-9E71-9E31T39080CD.dSYM
You'll notice that it calls it using both the Fabric API key and the GoogleService-Info.plist path. I do not know why but this will cause it not to upload. You'll have to temporarily remove the fabric configuration information from your Info.plist file before running the fastlane lane. (remember to re-add the fabric configuration).
First, you need to use upload_symbols_to_crashlytics but before to use it you will need to download your dsyms from App Store Connect and to do that you should use download_dsyms with some parameters version and build_number and the Fastlane will ask you about your app_identifier so I advise you to use it to not interrupt the build until getting your answer.
desc "Upload any dsyms in the current directory to Crashlytics of firebase"
lane :upload_dsyms do |options|
version_number = get_version_number(target: "your_app_target")
build_number = get_build_number
download_dsyms(
app_identifier: "your_app_identifier",
version: version_number,
build_number: build_number
)
upload_symbols_to_crashlytics(gsp_path: "./your_app_name or your_app_target/another_directroy/GoogleService-Info.plist")
clean_build_artifacts
end
my app was
desc "Upload any dsyms in the current directory to Crashlytics of firebase"
lane :upload_dsyms do |options|
version_number = get_version_number(target: "Movies")
build_number = get_build_number
download_dsyms(
app_identifier: "com.vngrs.Movies.demo",
version: version_number,
build_number: build_number
)
upload_symbols_to_crashlytics(gsp_path: "./Movies/Resources/GoogleService-Info.plist")
clean_build_artifacts
end
Details
Xcode Version 11.3.1 (11C504)
Firebase tools 7.14.0
Fastlane 2.143.0
Solution
./fastlane/Pluginfile
gem 'fastlane-plugin-firebase_app_distribution'
Lane
before_all do
# Update fastlane
update_fastlane
# Update fastlane plugins
sh("fastlane update_plugins")
# Update firebase tools
sh("curl -sL firebase.tools | upgrade=true bash")
end
Usage 1.
Download dsyms from AppStore and upload to firebase
download_dsyms(version: '1.0', build_number: '1')
upload_symbols_to_crashlytics(gsp_path: "./App/Environment/production/GoogleService-Info-production.plist")
Usage 2.
Get dsyms from archive (after build) and upload to firebase
gym(
configuration: 'Release',
scheme: 'MyApp',
include_bitcode: true
)
upload_symbols_to_crashlytics(gsp_path: "./App/Environment/production/GoogleService-Info-production.plist")
Info
get_version_number
download_dsyms
sh
update_fastlane
upload_symbols_to_crashlytics
This worked for me
desc "Downlaod and Uplaod dSYMS to Firebase"
lane :uplaod_dsyms do
download_dsyms # This will download all version
upload_symbols_to_crashlytics(
gsp_path: "Path to your google plist",
binary_path: "./Pods/FirebaseCrashlytics/upload-symbols") # this goes to cocoapods of FirebaseCrashlytics
clean_build_artifacts # Delete the local dSYM files
end
This works for me.
1. Create a new lane on Fastfile
# Fastfile
default_platform(:ios)
# Constants
XCODE_PROJECT = "PROJECT.xcodeproj"
XCODE_WORKSPACE = "PROJECT.xcworkspace"
PRODUCTION_GSP_PATH = "./APP_ROOT/GoogleService-Info.plist"
platform :ios do
desc "Push a production build to TestFlight"
lane :upload_crashlytics_prod do
download_dsyms(
version: get_version_number(xcodeproj: XCODE_PROJECT),
build_number: get_build_number(xcodeproj: XCODE_PROJECT)
)
upload_symbols_to_crashlytics(gsp_path: PRODUCTION_GSP_PATH)
end
end
2. Now call from your shell script or terminal
fastlane upload_crashlytics_prod

Can't serve firebase functions

I am trying to follow this tutorial from Firebase Web Codelab, but I am facing a problem at step 5: after running the command firebase serve, I do not get the expected response
Listening at http://localhost:5000, but instead I get:
i functions: Preparing to emulate functions.
i hosting: Serving hosting files from: ./
+ hosting: Local server: http://localhost:5000
which is bad because functions are not being served, although ⚠ functions: Failed to emulate api is not prompted, it remains "preparing to emulate functions" ad infinitum.
More information: CLI Version: 3.18.4
Platform: win32
Node Version: v6.12.3 and I already tried going to ~/.config/configstore/ and deleting the #Google-Cloud folder in order to restart the whole process.
I'm pretty sure you are in the right track, the difference between the expected message (Listening at http://localhost:5000) and the message you get (hosting: Local server: http://localhost:5000) is because you are using the new version of the CLI (v3.18.4) but the codelab chapter is not updated to reflect the new change in the logging message.
You can see the difference in the source-code firebase-tools v2.2.1 where it displays
Listening at http://<HOST>:<PORT>
and in firebase-tools v3.18.4 it displays
hosting: Local server: http://<HOST>:<PORT>
And for the functions emulation, as you are in step5, you don't have yet functions, see functions in the codelab source code.

firebase storage cors. IOError: [Errno 2] No such file or directory: u'cors-json-file.json'

I'm familiar with firebase and fire storage, but new to G-Cloud-SDK. I found this topic https://groups.google.com/forum/#!msg/firebase-talk/oSPWMS7MSNA/RnvU6aqtFwAJ and tried to copy the results, but get "IOError: [Errno 2] No such file or directory: u'cors-json-file.json'" from the terminal.
I found a bit more information in the docs https://cloud.google.com/storage/docs/configuring-cors#configure-cors-gsutil and it describes the same process, and I get the same error.
I'm finding it hard to do any troubleshooting with Google Cloud SDK since I have already setup functions in Firebase.
gsutil cors set cors.json gs://example-bucket
Example: If the cors.json file in the d: drive then the path will be like
d:/folderName/cors_filename. The gs url is your bucket url.
I did not move my local directory to my 'cors-json-file.json' ...

Resources