I trying to getting data from FTP server's txt file by GCP Composer Tasks.
So I imported SFTPOperator package in code.
but error occurred:
ModuleNotFoundError: No module named 'airflow.providers.sftp'
then, I tried few ways:
Getting exception "No module named 'airflow.providers.sftp'"
Install apache-airflow-providers-sftp by composer pypi packages
but didn't work.ðŸ˜
My GCP Composer Environment is as below:
Image Version : composer-1.17.7-airflow-2.1.4
python version : 3
Network VPC-native : Enable
How can I use SFTPOperator ?
For this you will have to install sftp package, pip install 'apache-airflow[sftp]' . You can check the built-in and extras packages that airflow components have when installed (varies from version).
Once you have it installed you should be able to use SFTPOperator by importing the operator inside your DAG.
from airflow.providers.sftp.operators.sftp import SFTPOperation,SFTPOperator
with DAG(...) as dag:
upload_op = SFTPOperator(
task_id="test_sftp",
ssh_conn_id="ssh_default",
local_filepath="/tmp/file.txt",
remote_filepath="/tmp/tmp1/tmp2/file.txt",
operation=SFTPOperation.GET,
dag=dag
)
...
You can also find a mock tests on the airflow git hub project that can provide you some guidance, check this link.
UPDATE 17/08/2022: As commented by Diana, Composer has a documented way to install its components as mention on this link. Be advised to pick up the composer version your project uses as there is version1 and version2 guides.
Related
Within a mac environment I am trying to upgrade the Vue CLI v4.5.13 to 5.0.8 using yarn as a packet manager
enter image description here
After fetching the packages the terminal result is:
ted (end-of-life October 22nd 2023). This package's functionality is now found in the `#apollo/server` package. See https://www.apollographql.com/docs/apollo-server/previous-versions/ for more details.
[2/4] 🚚 Fetching packages...
error create-strapi-app#3.6.5: The engine "node" is incompatible with this module. Expected version ">=10.16.0 <=14.x.x". Got "16.17.1"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/global for documentation about this command.
Reading the documentation at https://classic.yarnpkg.com/en/docs/cli/global does not help.
I expect to update the VUE CLI regardless to project dependencies and I am missing the command that the command
yarn global upgrade #vue/cli
ignores all existent dependencies and simply updates...
yarn add #vue/cli
finished successfully, but that does not help, due the Vue CLI remains to v4.5.13
What does
Fetching packages...
error create-strapi-app#3.6.5: The engine "node" is incompatible with this module. Expected version ">=10.16.0 <=14.x.x". Got "16.17.1"
states?
Node is incompatible to the node module defined in a local strapi project, right?
How should I handle this and why?
Thank you very much, for your kind explanations...
Trying to update Vue CLI to the latest version and hitting dependency blocks on global layer
I am getting this error when I run "dart pub global activate flutterfire_cli" :
Activated flutterfire_cli 0.1.1+2. Package flutterfire_cli is
currently active at version 0.1.1+2. Resolving dependencies... The
package flutterfire_cli is already activated at newest available
version. To recompile executables, first-run dart pub global deactivate flutterfire_cli. Installed executable flutterfire.
Warning: Pub installs executables into
C:\Users\PC\AppData\Local\Pub\Cache\bin, which is not on your path.
You can fix that by adding that directory to your system's "Path"
environment variable. A web search for "configure windows path" will
show you how
I have added this to my user path but I am still getting this error.
I checked for solutionsto similar problems which suggested I restart the windows or vs code or terminal - I did all three but it is still not working .
I ran in cmd in that it did not give the warning but when I ran flutterfire configure . It gave this error:
"FirebaseCommandException: An error occured on the Firebase CLI when
attempting to run a command. COMMAND: firebase --version ERROR: The
FlutterFire CLI currently requires the official Firebase CLI to also
be installed, see
https://firebase.google.com/docs/cli#install_the_firebase_cli for how
to install it."
running firebase --version gave this error:
'firebase' is not recognized as an internal or external command,
operable program or batch file
This should not be happening as I did connect my app to firebase console.
What worked for me is instead of typing
flutterfire I use flutterfire.bat, e.g. flutterfire.bat configure, that way the command works in the git bash with windows.
You have to add Flutter SDK path to your system environment. The error is arising because your system is not able to find the folder where you have installed Flutter in your system.
You may refer to the Stackoverflow case for the steps on how to set path for the system environment.
Alternatively, you may also refer to the video link.
If you have windows 10. Do not use firebase-tools-instant-win.exe its a trap!
Add Paths to System Variables. Don't forget to change to your username in YOU-USER-NAME
C:\Users\YOU-USER-NAME\AppData\Local\Pub\Cache\bin
C:\Users\YOU-USER-NAME\AppData\Roaming\npm
Next you can execute anywhere (in cmd or firebase-tools-instant-win.exe)
firebase login
npm install -g firebase-tools
dart pub global activate flutterfire_cli
Next you open only cmd.exe command promt and go to the root of your project.
cd c:\PATH-TO-YOU-PROJECT
flutterfire configure
I'm working with Symfony 5 and I installed phpunit-bridge with the command included in the documentation : composer require --dev symfony/phpunit-bridge.
When I run phpunit with this command, php bin/phpunit, I have this error :
PHP Fatal error : Uncaught Error: Call to undefined method
PHPUnit\TextUI\TestRunner::doRun()
I'll try to guess.
You also have a bundle like liip/functional-test-bundle or other, which require PHPUnit in requirements (or even you require PHPUnit directly in your composer.json together with phpunit-bridge).
As a result you have two different version of PHPUnit installed in project, with two different api.
If installed liip/functional-test-bundle is your case and you do not want to remove it, you need to install by bridge same PHPUnit version as installed by Liip bundle dependency. You can set version through SYMFONY_PHPUNIT_VERSION env variable or directly in bin/phpunit file. Or you can redefine path to already installed PHPUnit version (in vendors/) through SYMFONY_PHPUNIT_DIR env or directly in bin/phpunit.
Note: you can not set SYMFONY_PHPUNIT_VERSION or SYMFONY_PHPUNIT_DIR through .env.* files, since bin/phpunit not read this files. So it should be real env vars.
I had a similar issue with symfony 5.1. I tried upgrading all dev packages related to testing but it didn't work.
Once I completely remove all dev (testing related) packages and reinstalling the symfony phpunit bridge package it worked.
You have to remove the phpunit file inside the root/bin directory
Steps to follow
Remove all dev test packages (Eg: composer remove symfony/browser-kit)
Remove if there are following packages : phpunit/phpunit, symfony/phpunit-bridge, symfony/css-selector, symfony/browser-kit
Install the brdge package again by running composer require --dev symfony/phpunit-bridge
I know how to package and then deploy meteor application. But recently for one project i'm stuck at an error which i couldn't resolve.
Steps I followed for package and deploy of my meteor app:
1. meteor build package
2. cd package
3. tar -xf inventoryTool.tar.gz
4. cd bundle/programs/server
5. npm install
6. cd ../..
7. PORT=<port> MONGO_URL=mongodb://127.0.0.1:27017/dbName ROOT_URL=http://<ip> node main.js
Here is the log for the error when i run the npm install(STEP 5) command.
Is there anything missing in my execution?. I'm not using the fibers package anywhere in my project. Does anyone have solution to this problem? Thanks in advance.
Why this happens (a lot)?
Your local version of node is v8.9.4. When using the build command, you will export your application and build the code against this exact node version. Your server environment will require this exact version, too.
An excerpt from the custom deployment section of the guide:
Depending on the version of Meteor you are using, you should install
the proper version of node using the appropriate installation process
for your platform. To find out which version of node you should use,
run meteor node -v in the development environment, or check the
.node_version.txt file within the bundle generated by meteor build.
Even if you don't use fibers explicitly it will be required to run your Meteor app on the server correctly.
So what to do?
In order to solve this, you need to
a) ensure that your local version of node exactly matches the version on the server
b) ensure that you build against the server's architecture (see build command)
To install a) the very specific node version on your server you have two options:
Option I. Use n, as described here. However this works only if your server environment uses node and not nodejs (which depends on how you installed nodejs on the server).
II. To install a specific nodejs version from the repositories, you may do the following:
$ cd /tmp
$ wget https://deb.nodesource.com/node_8.x/pool/main/n/nodejs/nodejs_8.9.4-1nodesource1_amd64.deb
$ apt install nodejs_8.9.4-1nodesource1_amd64.deb
If you are not sure, which of both are installed on your server, check node -v and nodejs -v. One of both will return a version. If your npm install still fails, check the error output and if it involves either node or nodejs and install the desired distribution using the options above.
To build b) against the architecture on your server, you should use the --architecture flag in your build command.
I've created package and published it, It is just a simple wrapper of "phantom"
I deployed it using
sudo meteor publih --create
then added it to my app
$> meteor show sasi513:phantom#1.0.0
Version 1.0.0 : phantom node Package wrapper
Architectures: os.linux.x86_32+web.browser+web.cordova
Maintained by sasi513.
EDIT
Then when I try to deploy it meteor deploy xxx.meteor.com
Errors prevented deploying:
While building the application:
error: Unable to download package builds for this architecture.
and I tried to publish-for-arch
sudo meteor publish-for-arch sasi513:phantom#1.0.0
sasi513:phantom: updating npm dependencies -- phantom...
Bundling build...
Creating package build...
Uploading build...
Publishing package build...
Error from package server
: Cannot override existing build [403]
What is wrong here?Anyone has idea about this
The documentation says:
You need to run publish-for-arch from a different architecture to
upload a different build.
Are you running the command on a different architecture? Or the same machine? Because running the command on the same machine wont work.[1] Based on your description, it sounds like you didn't try it from the other architecture you want to support.
[1] http://docs.meteor.com/#meteorpublishforarch