With latest airflow-code-editor version (>v.7.0.0). We couldn't load any files in code editor session.
Attached is the screen shot.
Env:
Airflow V1.10.11 (currently couldn't upgrade, will be upgrading EOY),
Docker version 20.10.2,
Running on an EC2 instance
Other packages installed
boto3
pytz
airflow-code-editor
hvac
paramiko
pyarrow
mysqlclient
imapclient
pysftp
s3fs
packaging
avro
pinned the versions from 7.0.0 till the latest available session
restarted the webserver after every version changes
Related
Trying to add SFTP connection to Apache Airflow run on the Composer.
Here is some packages' listing;
package-name
version
apache-airflow
2.2.5+composer
apache-airflow-providers-ftp
3.2.0
apache-airflow-providers-google
2022.10.17+composer
apache-airflow-providers-sftp
4.1.0
apache-airflow-providers-ssh
3.2.0
paramiko
2.12.0
sshtunnel
0.4.0
This is the message I get when hitting the Test button:
Any help would be appreciated.
This is an expected behavior on Composer 1 because in Composer 1 if DAG serialization is on, the Airflow webserver is using a base image that doesn’t include customer installed pypi packages.
Therefore, this functionality doesn’t work properly.
You may upgrade to Composer 2 so that everything will work properly.
However, if you want to stay on Composer 1, you may try changing the conn_type from ‘sftp’ to ‘ftp’.
There seems to be no proper documentation about upgrading airflow. The Upgrading Airflow to a newer version page only talks about upgrading the database. So what is the proper way of upgrading airflow?
Is it just upgrading the python packages to the newest versions? Or should I use the same venv and install the newer airflow version completely from scratch? Or is it something else altogether?
I'm guessing doing the database upgrade would be the final step followed by one of these steps.
I was also struggling with upgrading airflow for minor versions and didn't feel like I found a good answer in the docs. I think I have the right approach after looking back at how I installed airflow in the first place.
If you followed the guide to run airflow locally you'll want to change the value for AIRFLOW_VERSION in the commands to your desired version.
If you followed the guide to run airflow on docker, then you'll want to fetch the latest docker-compose.yaml. The command on the site always has the latest version. Then re-run docker compose up.
You can confirm you have the right version by running airflow version. I run airflow via docker so the docker steps work for me, I imagine the local steps should be about the same.
Adding to Vivian's answer -
I had installed airflow from PyPi and was upgrading from 2.2.4 to 2.3.0.
To upgrade airflow,
I installed the new version of airflow in the same virtual environment as 2.2.4 (using this).
Upgraded the database using airflow db upgrade. More details here.
You might have to manually upgrade providers using pip install packagename -U
After this, when I started Airflow, I got an error related to some missing conf. Airflow wanted the newest version of airflow.cfg, but I had the older version. To fix this,
Renamed my airflow.cfg to airflowbackup.cfg. This is done so that airflow will make a new airflow.cfg on start up when it sees that there is no config file.
Compared airflowbackup.cfg with the 2.2.4 config to find out all the fields I had changed.
Manually made those same changes in the newly made airflow.cfg
So I'm a brand new guy on this web-building stuff. i have a page on GitHub page that I generated from a Wordpress instalation exported to static files through a plugin. they are all static files. but when i try to use this same repository on cloudfare pages (i select the jerkyl framework because thats what i understand github uses) i get a build error
21:47:34.423 Success: Finished cloning repository files
21:47:34.697 Installing dependencies
21:47:34.710 Python version set to 2.7
21:47:38.959 v12.18.0 is already installed.
21:47:40.464 Now using node v12.18.0 (npm v6.14.4)
21:47:40.845 Started restoring cached build plugins
21:47:40.864 Finished restoring cached build plugins
21:47:41.647 Attempting ruby version 2.7.1, read from environment
21:47:48.133 Using ruby version 2.7.1
21:47:48.547 Using PHP version 5.6
21:47:48.733 5.2 is already installed.
21:47:48.766 Using Swift version 5.2
21:47:48.766 Installing Hugo 0.54.0
21:47:49.458 Hugo Static Site Generator v0.54.0-B1A82C61A/extended linux/amd64 BuildDate: 2019-02-01T10:04:38Z
21:47:49.463 Started restoring cached go cache
21:47:49.488 Finished restoring cached go cache
21:47:49.664 go version go1.14.4 linux/amd64
21:47:49.687 go version go1.14.4 linux/amd64
21:47:49.691 Installing missing commands
21:47:49.692 Verify run directory
21:47:49.692 Executing user command: jekyll build
21:47:49.699 /opt/build/bin/build: line 39: jekyll: command not found
21:47:49.707 Failed: build command exited with code: 127
am i missing something? am i supposed to use another framework?
Does the version of Node.js on my development machine / integrated development environment (IDE) machine have any effect on my ability to upgrade my Firebase Cloud Functions Node.js runtime to version 8?
I finally got my functions Node.js runtime upgraded to v8 but I had a difficult time doing it. I eventually installed node version manager, nvm, to nail my development machines Node version to 8.0.0 but then ran into the 8.0.0 issue w/ firebase-tools. "must use 8.1.0. 8.0.0 fails"
nvm install v8.1.0
nvm alias default 8.1.0
So, running Node.js v8.1.0 on my development machine seemed to be the breakthrough. I'd like to know but don'e have time to spend on recreating it all.
Trying to upgrade Nexus 2.1.4 Repo to 3.X. This is what i have done till now and what what went wrong!
-- installed the 3.0.2 version on the same machine in a different port (8090) Then opened the UI and tried doing the upgrade agent and there was no upgrade agent in 3.0.2.
-- so i installed 3.1 on the different port and came across a java exception error in nexus Repo UI while migrating stuff from 2.1.4 (installed on port 80). I found out that one need to be at-least on 3.2.4 V to migrate.
-- now i installed 3.6 on a different port (8082)in the same machine. couldn't start nexus service now. which i was able to for other nexus version in other ports.