I would like to know how can we run application E2E(UI or API) test after successful deployment of any micro services using ArgoCD.
Current Setup: I have CI pipeline setup with github-actions. Upon completion on CI build for any microservices, it updates the docker image version in helm values which resides in one of the github repo. This repo is than polled by ArgoCD for any change, and deploys in Kubernestes cluster if there is change exists.
Intent: I want to run the application E2E( UI & API ) test once argocd synced any micro-services deployment object defined in the Helm charts. But I am unsure what should be the trigger point in github actions for that. How E2E test github actions workflow will know argocd has deployed the microservices without any issue and service is ready to be consumed by the automated test.
Here is the full solution to the problem statement.
apiVersion: batch/v1
kind: Job
metadata:
name: api-test-trigger
annotations:
argocd.argoproj.io/hook: PostSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
template:
metadata:
labels:
name: api-test
spec:
containers:
- name: api-test
args:
- /bin/sh
- -ec
- "curl -X POST -H \"Accept: application/vnd.github.v3+json\" -H \"Authorization: token ${GITHUB_TOKEN}\" ${GITHUB_URL} -d '{\"ref\":\"main\"}'"
env:
- name: GITHUB_URL
value: "https://api.github.com/repos/<your org>/<your repo>/actions/workflows/<workflow id>/dispatches"
- name: GITHUB_TOKEN
value: <your PAT>
image: curlimages/curl
You can create the PAT from github settings and provide the PAT as a secret.
ArgoCD provides a feature called resource hooks. Hooks are ways to run scripts before, during, and after a sync operation. A use case for hooks from the official documentation:
Using a PostSync hook to run integration and health checks after a deployment.
Hooks can be any type of Kubernetes resource kind, but tend to be Pod, Job, or Argo Workflows.
Per the GitHub actions documentation, you can send a POST request to the Github API in PostSync hooks template to run the workflow run.
Related
Do you know why nginx is not starting using Knative?
No problem using standard deployment:
kubectl create deployment nginx --image=nginx
But using Knative service - there is issues:
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: service-web
namespace: default
spec:
template:
spec:
containers:
- image: nginx
ports:
- containerPort: 80
Kubectl get ksvc:
Unknown RevisionMissing
kubectl describe revision service-web:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning InternalError 4m55s (x3 over 4m55s) revision-controller failed to update deployment "service-webb-qb4km-deployment": Operation cannot be fulfilled on deployments.apps "service-web-qb4km-deployment": the object has been modified; please apply your changes to the latest version and try again
I'm assuming there's a typo in your error message (it doubles the b); if not I'd look at that first.
Do you have any sort of controller or mutating webhook operating on deployments? The message that the controller can't update the deployment because there is a newer version suggests that there is some sort of racing update going on.
I'm new to Kubernetes and wanted to use the NGINX Ingress Controller for the project I'm currently working on. I read some of the docs and watched some tutorials but I haven't really understood the:
installation process (should I use Helm, the git repo???)
how to properly configure the Ingress. For example, the Kubernetes docs say to use a nginx.conf file (https://kubernetes.io/docs/tasks/access-application-cluster/connecting-frontend-backend/#creating-the-frontend) which is never mentioned in the actual NGINX docs. They say to use ConfigMaps or annotations
Does anybody know of a blog post or tutorial that makes these things clear. Out of everything I've learned so far (both frontend and backend) developing and deploying to a cloud environment has got me lost. I've been stuck on a problem for a week and want to figure out it Ingress can help me.
Thanks!
Answering:
How should I install nginx-ingress
There is no one correct way to install nginx-ingress. Each way has its own advantages/disadvantages, each Kubernetes cluster could require different treatment (for example: cloud managed Kubernetes and minikube) and you will need to determine which option is best suited for you.
You can choose from running:
$ kubectl apply -f ...,
$ helm install ...,
terraform apply ... (helm provider),
etc.
How should I properly configure Ingress?
Citing the official documentation:
An API object that manages external access to the services in a cluster, typically HTTP.
-- Kubernetes.io: Docs: Concepts: Services networking: Ingress
Basically Ingress is a resource that tells your Ingress controller how it should handle specific HTTP/HTTPS traffic.
Speaking specifically about the nginx-ingress, it's entrypoint that your HTTP/HTTPS traffic should be sent to is a Service of type LoadBalancer named: ingress-nginx-controller (in a ingress-nginx namespace). In Docker with Kubernetes implementation it will bind to the localhost of your machine.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
spec:
ingressClassName: "nginx"
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx
port:
number: 80
The modified example from the documentation will tell your Ingress controller to pass the traffic with any Host and with path: / (every path) to a service named nginx on port 80.
The above configuration after applying will be reflected by ingress-nginx in the /etc/nginx/nginx.conf file.
A side note!
Take a look on how the part of nginx.conf looks like when you apply above definition:
location / {
set $namespace "default";
set $ingress_name "minimal-ingress";
set $service_name "nginx";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
On how your specific Ingress manifest should look like you'll need to consult the documentation of the software that you are trying to send your traffic to and ingress-nginx docs.
Addressing the part:
how to properly configure the Ingress. For example, the Kubernetes docs say to use a nginx.conf file (https://kubernetes.io/docs/tasks/access-application-cluster/connecting-frontend-backend/#creating-the-frontend) which is never mentioned in the actual NGINX docs. They say to use ConfigMaps or annotations.
You typically don't modify nginx.conf that the Ingress controller is using by yourself. You write an Ingress manifest and rest is taken by Ingress controller and Kubernetes. nginx.conf in the Pod responsible for routing (your Ingress controller) will reflect your Ingress manifests.
Configmaps and Annotations can be used to modify/alter the configuration of your Ingress controller. With the Configmap you can say to enable gzip2 compression and with annotation you can say to use a specific rewrite.
To make things clearer. The guide that is referenced here is a frontend Pod with nginx installed that passes the request to a backend. This example apart from using nginx and forwarding traffic is not connected with the actual Ingress. It will not acknowledge the Ingress resource and will not act accordingly to the manifest you've passed.
A side note!
Your traffic would be directed in a following manner (simplified):
Ingress controller -> frontend -> backend
This example speaking from personal perspective is more of a guide how to connect frontend and backend and not about Ingress.
Additional resources:
Stackoverflow.com: Questions: Ingress nginx how sto serve assets to application
Stackoverflow.com: Questions: How nginx ingress controller backend protocol annotation works
The guide that I wrote some time ago should help you with the idea on how you can configure basic Ingress (it could be little outdated):
Stackoverflow.com: Questions: How can I access nginx ingress on my local
The most straightforward process of installing nginx ingress controller (or any other for that matter) would be using helm. This would need basic understanding of helm and how to work with helm charts.
Here's the repo: https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx
Follow the instructions in there - quite straightforward if you use the default values. For the configuration, you can customize the chart too before installing. Look at the Readme to see how to get all the configurable options.
Hope this helps as a starting point.
I was following the instructions in this link https://docs.openshift.com/container-platform/4.2/networking/multiple-networks/configuring-macvlan.html
Here it says to edit the networks.operator.openshift.io CustomResource and add the additional network to it.
$ oc edit networks.operator.openshift.io cluster
apiVersion: operator.openshift.io/v1
kind: Network
metadata:
name: cluster
spec:
additionalNetworks:
...
I have done that and saved the CustomResource, but when I do oc get network-attachment-definitions, I get No resources found.
I have waited about 20 minutes after creating the CustomResource, but no CNO has been created.
Further down the page there is information for configuring macvlan cni plug-in. I am wondering if the configuration for the plug-in is a prerequisite and where do I configure the plug-in?
I have my some files which is in react lng. I am doing build from npm.I am doing locally this build.I have the build path.
I want to deploy this build to pods of the kubernetes.
how to write the deployment.yaml?
how to configure my nginx or httpd root folder which can publish my codes?
If first i have to make the docker image of that project file , then how?
First you have to create Dockerfile:
Egg. Dockerfile:
FROM golang
WORKDIR /go/src/github.com/habibridho/simple-go
ADD . ./
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix .
EXPOSE 8888
ENTRYPOINT ./simple-go
Build your image and try to run it.
$ docker build -t simple-go .
$ docker run -d -p 8888:8888 simple-go
Next step is transferring image to the server. You can use Docker Hub. You can push image to the repository and pull it from the server.
-- on local machine
$ docker tag simple-go habibridho/simple-go
$ docker push habibridho/simple-go
-- on server
$ docker pull habibridho/simple-go
You have to note that the default docker repository visibility is public, so if your project is private, you need to change the project visibility from the Docker Hub website.
Useful information about that process you can find here:
docker-images
Once we have the image in our server, we can run the app just like what we have done in our local machine by creating deployment.
The following is an example of a Deployment. It creates a ReplicaSet to bring up three your app Pods:
apiVersion: apps/v1
kind: Deployment
metadata:
name: your-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: your-app
template:
metadata:
labels:
app: your-app
spec:
containers:
- name: your-app
image: your-app:version
ports:
- containerPort: port
In this example:
A Deployment named your-deployment is created, indicated by the .metadata.name field.
The Deployment creates three replicated Pods, indicated by the replicas field.
The selector field defines how the Deployment finds which Pods to manage. In this case, you simply select a label that is defined in the Pod template (app: your-app).
However, more sophisticated selection rules are possible, as long as the Pod template itself satisfies the rule.
To create Deployment, run the following command:
$ kubectl create -f your_deployment_file_name.yaml
More information you can find here: kubernetes-deployment.
I have a web app hosted in the Google Cloud platform that sits behind a load balancer, which itself sits behind an ingress. The ingress is set up with an SSL certificate and accepts HTTPS connections as expected, with one problem: I cannot get it to redirect non-HTTPS connections to HTTPS. For example, if I connect to it with the URL http://foo.com or foo.com, it just goes to foo.com, instead of https://foo.com as I would expect. Connecting to https://foo.com explicitly produces the desired HTTPS connection.
I have tried every annotation and config imaginable, but it stubbornly refuses, although it shouldn't even be necessary since docs imply that the redirect is automatic if TLS is specified. Am I fundamentally misunderstanding how ingress resources work?
Update: Is it necessary to manually install nginx ingress on GCP? Now that I think about it, I've been taking its availability on the platform for granted, but after coming across information on how to install nginx ingress on the Google Container Engine, I realized the answer may be a lot simpler than I thought. Will investigate further.
Kubernetes version: 1.8.5-gke.0
Ingress YAML file:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: https-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/secure-backends: "true"
ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
tls:
- hosts:
- foo.com
secretName: tls-secret
rules:
- host: foo.com
http:
paths:
- path: /*
backend:
serviceName: foo-prod
servicePort: 80
kubectl describe ing https-ingress output
Name: https-ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (10.56.0.3:8080)
TLS:
tls-secret terminates foo.com
Rules:
Host Path Backends
---- ---- --------
foo.com
/* foo-prod:80 (<none>)
Annotations:
force-ssl-redirect: true
secure-backends: true
ssl-redirect: true
Events: <none>
The problem was indeed the fact that the Nginx Ingress is not standard on the Google Cloud Platform, and needs to be installed manually - doh!
However, I found installing it to be much more difficult than anticipated (especially because my needs pertained specifically to GCP), so I'm going to outline every step I took from start to finish in hopes of helping anyone else who uses that specific cloud and has that specific need, and finds generic guides to not quite fit the bill.
Get Cluster Credentials
This is a GCP specific step that tripped me up for a while - you're dealing with it if you get weird errors like
kubectl unable to connect to server: x509: certificate signed by unknown authority
when trying to run kubectl commands. Run this to set up your console:
gcloud container clusters get-credentials YOUR-K8s-CLUSTER-NAME --z YOUR-K8S-CLUSTER-ZONE
Install Helm
Helm by itself is not hard to install, and the directions can be found on GCP's own docs; what they neglect to mention, however, is that on new versions of K8s, RBAC configuration is required to allow Tiller to install things. Run the following after helm init:
kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
Install Nginx Ingress through Helm
Here's another step that tripped me up - rbac.create=true is necessary for the aforementioned RBAC factor.
helm install --name nginx-ingress-release stable/nginx-ingress --set rbac.create=true
Create your Ingress resource
This step is the simplest, and there are plenty of sample nginx ingress configs to tweak - see #JahongirRahmonov's example above. What you MUST keep in mind is that this step takes anywhere from half an hour to over an hour to set up - if you change the config and check again immediately, it won't be set up, but don't take that as implication that you messed something up! Wait for a while and see first.
It's hard to believe this is how much it takes just to redirect HTTP to HTTPS with Kubernetes right now, but I hope this guide helps anyone else stuck on such a seemingly simple and yet so critical need.
GCP has a default ingress controller which at the time of this writing cannot force https.
You need to explicitly manage an NGINX Ingress Controller.
See this article on how to do that on GCP.
Then add this annotation to your ingress:
kubernetes.io/ingress.allow-http: "false"
Hope it helps.