I am using Airflow 2.2.4 installed on EKS via the official helm chart and uses KubernetesPodOperator.
Now I want to add SnowflakeOperator to my task. Following the guide here I created a custom airflow image like the below and hosted it in an ECR repo
FROM apache/airflow:2.2.4-python3.8
RUN pip install --no-cache-dir apache-airflow-providers-snowflake==3.1.0
I am then trying to customize the helm installation by passing the above ECR repo in values.yml file like below:
defaultAirflowRepository: xxx.dkr.ecr.ap-southeast-2.amazonaws.com/sophia/custom-airflow-image
images:
airflow:
repository: "xxx.dkr.ecr.ap-southeast-2.amazonaws.com/airflow-deploy/custom-airflow-image"
tag: airflow-app-deploy
pullPolicy: IfNotPresent
Here is the helm command I am using:
helm upgrade --install airflow apache-airflow/airflow --version v1.5.0 --namespace sophia-airflow --values /tmp/airflow/airflow.yaml
But for some reason, these values are not taken into consideration by the helm while installation. Any idea what I am doing wrong here?
Related
I am trying to install airflow in my kuber cluster(minikube, 3 node) with helm(almost same with original airflow helm chart, except postgresql).
Which node name is airflow-demo(kuber master), airflow-demo-m02, airflow-demo-m03.
However, when I install airflow, each components(pods) are created in seperate node. Which makes scheduler and db does not work(ex: airflow-webserver: airflow-demo-m02, airflow-postgresql-db: airflow-demo-m03 ...)
Am I able to install my helm package in only one node?
What I tried: helm install ... --set {nodeSelector."kubernetes.io/hostname"="airflow-demo-m02"}, helm install ... --set {nodeSelector="airflow-demo-m02"}, helm install ... --set nodeSelector.airflowtype=Master and all did not worked for me.
(nodeSelector.airflowtype is my custom label, airflow-demo-m02 is Master, and airflow-demo-m3 is worker)
I try to install Redis for tyk with my company's local jfrog repository but I am facing the following issue.
I am using Helm v3.8.2.
helm install redis tyk-helm/simple-redis --set image.repository=images.artifactory.mycompany.com/redis -n tyk
Error: unknown flag: --set image.repository
can you help me figure out how to solve this issue? I am new to using helm.
The error seems to be complaining of an unknown flag image.repository. Did you try running the command written here
helm install tyk-redis bitnami/redis -n tyk
Recently, I upgraded my Azure Kubernetes Cluster from 1.19.11 to 1.21.7.
Subsequently, I am upgrading nginx ingress on the cluster
[]$ helm list -A | grep nginx-ingress
nginx-ingress ingress-basic 1 2021-01-18 13:07:23.842072063 +0000 UTC deployed ingress-nginx-3.15.2 0.41.2
$ kubectl exec -it nginx-ingress-ingress-nginx-controller-85b8676f7d-8qjw5 -n ingress-basic sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/etc/nginx $ nginx -v
nginx version: nginx/1.19.4
/etc/nginx $
I want to upgrade nginx to 1.20.0 plus version. Do we need to specify the version in command below?
helm upgrade --install ingress-basic ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace nginx-ingress --create-namespace
kubectl describe pod nginx-ingress-ingress-nginx-controller-65b69d4895-d4cft -n ingress-basic | grep Image:
Image: demoacr.azurecr.io/nginx-ingress-controller:nginx-0.35.0-rancher2
To upgrade your ingress-nginx installation to a specific version, you can set the flag --version, as specified in Helm docs:
helm upgrade [RELEASE] [CHART] [flags]
--version string specify a version constraint for the chart version to use.
This constraint can be a specific tag (e.g. 1.1.1) or it may reference a valid range (e.g. ^2.0.0).
If this is not specified, the latest version is used
The command should look like this:
helm upgrade --install ingress-basic ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace nginx-ingress --create-namespace \
--version 1.20.0
It is worth mentioning, that you can also upgrade your nginx installation without Helm by changing the version of the image in your controller Deployment - just change the image tag to the version you wish to upgrade to - v1.20.0 . More details about it in Upgrade NGINX Ingress Controller.
helm install airflow-staging airflow-stable/airflow --version “7.14.0” --namespace staging --set airflow.config.AIRFLOW__KUBERNETES__GIT_BRANCH=master --set dags.git.ref=“https://github.com/pedaling/class101-airflow.git” --values staging-values.yaml
it gives this error --
Error: failed to download "airflow-stable/airflow" at version "“7.14.0”" (hint: running helm repo update may help)
so I type the helm repo update then it return --
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "airflow-stable" chart repository
Update Complete. ⎈Happy Helming!⎈
even I typed helm repo update it always return the same error like
Error: failed to download "airflow-stable/airflow" at version "“7.14.0”" (hint: running helm repo update may help)
what can I do?
To install the stable/airflow follow the steps from the repo README:
https://github.com/airflow-helm/charts/tree/main/charts/airflow
I have copied the relevant section below.
1 - Add the Repo:
helm repo add airflow-stable https://airflow-helm.github.io/charts
helm repo update
2 - Install the Chart:
⚠️ find values for CHART_VERSION under GitHub Releases
export RELEASE_NAME=my-airflow-cluster # set a name!
export NAMESPACE=my-airflow-namespace # set a namespace!
export CHART_VERSION=8.X.X # set a version!
export VALUES_FILE=./custom-values.yaml # set your values file path!
# Helm 3
helm install \
$RELEASE_NAME \
airflow-stable/airflow \
--namespace $NAMESPACE \
--version $CHART_VERSION \
--values $VALUES_FILE
# Helm 2
helm install \
airflow-stable/airflow \
--name $RELEASE_NAME \
--namespace $NAMESPACE \
--version $CHART_VERSION \
--values $VALUES_FILE
3 - Access the WebUI
⚠️ browse to http://localhost:8080 after running the following commands
export NAMESPACE=my-airflow-namespace # set a namespace!
export POD_NAME=$(kubectl get pods --namespace $NAMESPACE -l "component=web,app=airflow" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward --namespace $NAMESPACE $POD_NAME 8080:8080
NOTE:
default credentials -- user: admin - password: admin
I don't know the exact reason why that error is generated (maybe to old version) but I experienced the same by running
helm upgrade airflow -f values.yaml airflow-stable/airflow --version airflow-7.16.0
Solution:
Go to GitHub Releases
1. Find your release, mine for example is 7.16.0 and copy link to its .tgz
https://github.com/airflow-helm/charts/releases/download/airflow-7.16.0/airflow-7.16.0.tgz
2. Execute helm pull to your local directory
helm pull https://github.com/airflow-helm/charts/releases/download/airflow-7.16.0/airflow-7.16.0.tgz
3. And finaly upgrade using local chart
helm upgrade airflow -f values.yaml airflow-7.16.0.tgz
I am trying to install nginx ingress using helm version 3 on Google Cloud Terminal as follows :
curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 > get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh
and
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm install my-nginx stable/nginx-ingress --set rbac.create=true
I keep getting the error :
Error: This command needs 1 argument: chart name
Can you please help me?
From helmv3 docs:
https://helm.sh/docs/intro/install/
$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh
Can you also run helm version after the above steps. In the semantic helm version you should see 3.x.x something like that.
The command is just fine. Are you sure it's helmv3 and not helmv2. It shouldn't give that error because you are already providing the name for the chart.
Also can you try running the following command, it's just a test to see if the chart gets installed or does it throw an error. It will generate a random name for the chart and not my-nginx as you specified.
helm install --debug stable/nginx-ingress --set rbac.create=true --generate-name