Airflow has an upgradedb command that needs to be run when upgrading Airflow versions. I wonder if it's safe to run even if the version is the same
The way it works is in db.py they use the alembic command module to check the checked in files in the migrations directory https://github.com/apache/incubator-airflow/tree/master/airflow/migrations/versions, and only make the changes if the commit version differs. But these files only get changed/added once we change the version so the upgrade db step does nothing when it's the same version/whl.
Adding it as a default step since I've verified it's safe to do so.
Related
There seems to be no proper documentation about upgrading airflow. The Upgrading Airflow to a newer version page only talks about upgrading the database. So what is the proper way of upgrading airflow?
Is it just upgrading the python packages to the newest versions? Or should I use the same venv and install the newer airflow version completely from scratch? Or is it something else altogether?
I'm guessing doing the database upgrade would be the final step followed by one of these steps.
I was also struggling with upgrading airflow for minor versions and didn't feel like I found a good answer in the docs. I think I have the right approach after looking back at how I installed airflow in the first place.
If you followed the guide to run airflow locally you'll want to change the value for AIRFLOW_VERSION in the commands to your desired version.
If you followed the guide to run airflow on docker, then you'll want to fetch the latest docker-compose.yaml. The command on the site always has the latest version. Then re-run docker compose up.
You can confirm you have the right version by running airflow version. I run airflow via docker so the docker steps work for me, I imagine the local steps should be about the same.
Adding to Vivian's answer -
I had installed airflow from PyPi and was upgrading from 2.2.4 to 2.3.0.
To upgrade airflow,
I installed the new version of airflow in the same virtual environment as 2.2.4 (using this).
Upgraded the database using airflow db upgrade. More details here.
You might have to manually upgrade providers using pip install packagename -U
After this, when I started Airflow, I got an error related to some missing conf. Airflow wanted the newest version of airflow.cfg, but I had the older version. To fix this,
Renamed my airflow.cfg to airflowbackup.cfg. This is done so that airflow will make a new airflow.cfg on start up when it sees that there is no config file.
Compared airflowbackup.cfg with the 2.2.4 config to find out all the fields I had changed.
Manually made those same changes in the newly made airflow.cfg
I upgraded docker image to use airflow 1.10.14. Airflow is deployed with helm and I have an init-container which execute script to initialize airflow. The init script contain commands
...
airflow upgradedb
alembic upgrade heads
...
The upgrade failed so I need to rollback to previous deployed release version which contains the 1.10.10 version of airflow but it is now getting the alembic error. I tried to delete the row/record in the alembic_version table based on my search.
The error in scheduler container is this:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.DuplicateColumn) column "operator" of relation "task_instance" already exists
All the other pods are running fine (webserver and workers).
Any resolution/workaround to this issue?
Unless you are ok with scrapping your entire metadata DB (connections, variables, task runs, etc) I might opt to just push to 1.10.15 and see if the bug you encountered is resolved there. From my best understanding it is not possible to downgrade the DB after the upgrade has been done.
Suggesting upgrade to 1.10.15 based on if you remember encountering an issue similar to this user here. The CLI fix can be found here. If you found another issue with your 1.10.14 upgrade besides the one I noted for the CLI, it might be worth investigating a path to resolving that instead.
I am trying to update my Firebase CLI installation, in order to use the cloud functions for my project. I followed the getting-started guide, however for some reason, the CLI cannot detect the updated version.
I initialized a project of mine that I have already created via the Firebase console. During the project initialization, Firebase CLI mentioned that my CLI version should be updated, however the complete procedure went smoothly.
Once the initialization completed, I run for the first time (to update the CLI):
npm install -g firebase-tools
Once the updated finished, the command line reported:
firebase-tools#3.18.6
But when I run
firebase init functions
I get:
Error: CLI is out of date (on 3.0.1 , need at least 3.0.5)
I don't understand why this is happening, since the update reported that version 3.18.6 got installed Any hints would be great!
For reference, you can see a snapshot of the command line here.
It looks like you may have multiple versions of node installed in different locations. First, uninstall all versions of node that you may have previously installed. Make sure running node on the command line doesn't execute anything. Then, reinstall everything. After you've installed the Firebase CLI again, check its version with firebase --version.
I'm trying to host a meteor app that uses an old version of meteor.
Every time i try to start the app it will get somewhat through the process of installing the tool, and then i see a message such as:
Killednloading meteor-tool#1.1.3... -
(note how killed somehow overwrites the downloading part of the command line)
Is there a reliable way to install the meteor tool at a specific version?
EDIT:
The Meteor team added a release parameter to their download endpoint. Now you can simply specify the desired version:
curl "https://install.meteor.com/?release=1.3.3.1" | sh
For Windows, a version parameter exists for the choco installer:
choco install meteor --version 1.3.3.1
Original solution
You can use sed for that. Insert it in the middle of curl and sh:
curl https://install.meteor.com/ | sed 's/1.4/1.3.3.1/' | sh
That will replace the release 1.4 (current version) to 1.3.3.1
When you create a meteor app you can specify a release:
meteor create test --release x.y.z
And when you update a meteor app you can do the same:
meteor update --release x.y.z
#Jorge Issa's answer is good if you are installing Meteor from scratch, on a system that never had Meteor installed, however it's subject to change since versions change all the time, so you need to adapt the sed line.
If you have any version of Meteor already installed, as Michel Floyd mentioned, you can always create a project with a specific version by adding the --release flag.
meteor update --release xxxx works fine with you're actually upgrading, but downgrading is a different story.
My recommendation when it comes to upgrading and eventually downgrading, is to use version control (git).
Attempt upgrade and if all is fine, you're in good shape, if not and you want to downgrade, simply clear the file changes in your version control system and use meteor reset to clean your project and rebuild with the previous version.
!Note! meteor reset clears the local mongo database too, so be sure to back that up first if you're going to do that (check mongodump and mongorestore for that)
finally, if you're looking to clean up the clutter from the .meteor folder, you can delete the folder and then run meteor reset in your project: the meteor executable will detect you don't have the needed packages will re-download the packages for the version needed by your project. (This takes a while and if you have many project, can be cumbersome as you need to do this in each project, but if like me you are looking to clear some space, this works fine.)
Try:
meteor update --release x.y.z
Try
choco install meteor --x86 --params="'/RELEASE:1.5.4.4'"
I'm using travis to test my code. Recently the grunt tasks have started to fail without no change being made to anything involved with grunt. (The new commit which is tested contains just very minor changes in two PHP files.)
Here is the part of log from travis:
$ grunt build:app
Running "typings:default" (typings) task
Warning: Cannot find any-promise implementation nor global.Promise. You must install polyfill or call require("any-promise/register") with your preferred implementation, e.g. require("any-promise/register")("bluebird") on application load prior to any require("any-promise"). Use --force to continue.
Aborted due to warnings.
The command "grunt build:app" exited with 3.
I tried to search for that warning message but couldn't find anything useful.
One more thing: When I run grunt build:app locally on my pc it works just fine.
Thanks for your time :)
I had the same problem when I started using grunt-typings. Worked locally and didn't work on my CI server. Ended up fixing it by doing what the error message suggests:
npm install bluebird
npm install any-promise
In GruntFile.js:
require("any-promise/register")("bluebird");
Update your node.js version to >v0.12. To check your version of node.js use node -v. The documentation of any-promise explains:
Node.js versions prior to v0.12 may have contained buggy versions of the global Promise. For this reason, the global Promise is not loaded automatically for these old versions. If using any-promise in Node.js versions versions <= v0.12, the user should register a desired implementation.