Creating, Running & Deploying a complete project in Kubernetes

When I was learning Kubernetes and Dockers, I understood all the important concepts but still wasn’t able to deploy a project. There are many things in Kubernetes which works together in order to run our deployment. Today in this article I will show you the step by step process of creating, running and deploying an application on Kubernetes.

I expect that you are already familiar with Kubernetes and Docker. If not then please go through this article first – Introduction to Kubernetes.

Introduction

A Kubernetes project requires these resources –

1. Deployment YAML
2. Services YAML
3. Configmaps
4. Secrets
5. Docker images of applications

In this article we are going to create a mongodb application using mongo database and mongo-express user interface. This article is inspired by the great youtube series on DevOps by TechWorld with Nana.

With these two applications you will be able to understand how to deploy a database as well as the connected web application.

A brief walkthrough

Let us discuss the setup details and how we are going to pursue this project –

• We are using mongoDB as database.
• Database should not be accessible outside our cluster. For example, browsers should not be able to access mongodb directly. So, we are going to create an internal service for mongoDB.
• We will use Mongo-Express to create a user interface for mongoDB data. So, express should be able to access mongoDB. Database url and credentials are used to create the connection.
• We will set up the db url and credentials as environment variables and use them in deployment file of mongo-express.
• We store the Credentials in Secrets file because they need to be secure. On the other hand we will store db url in configMaps file.
• To access mongo-express from browser we will create an external service.

The whole setup will look like this –

The final flow of the request will be –

Browser will send request to external service of mongo-express through url and port. It is forwarded to express pod. Pod will then access internal service of mongodb using db url and credentials. It is then pass to mongodb pod.

Let’s start creating project

First of all you should have minicube installed. You will need it to run all the kubernetes commands.

Create a project directory to store all the files. We will call it my-kubernetes-project. Suppose you created this directory on Desktop. Enter into it –

cd ~/Desktop/my-kubernetes-project

We will store all the files and scripts in this directory. Let’s start with creating the deployment file for mongodb.

Deployment.yaml for mongodb

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-deployment
  labels:
    app: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo

In this file, there are few points to pay attention to –

1. .kind represents the kind of file. We are creating a deployment.
2. .metadata.name indicates the name of deployment. So, mongodb-deployment is the name of our deployment.
3. .metadata.labels represents the label of deployment. Here we are using app=mongodb. This value is used by service to create connection with deployment.
4. .spec.template represents the configuration of pods.
5. .spec.replicas represents the number of pods to create. Here we are okay with 1 pod.
6. .spec.template.metadata.labels represents the label of pod.
7. .spec.template.spec represents the information about containers in the pod.
8. .spec.template.spec.containers.name represents the name of the container.
9. .spec.template.spec.containers.image represents the docker image.

So, we created the mongodb-deployment with label mongodb and having image mongo. If you want to check the mongo docker image then visit this – https://hub.docker.com/_/mongo.

This deployment file is not yet complete because we have not specified the port as well as the login credentials. MongoDB needs admin credentials for the database. So when the request comes, it will check if the user or api credentials match with the provided admin ones.

One more thing, we already specified that we should not store credentials in plain text. We need to store them in Secrets file. Next, we will create the Secrets file.

But, how will mongodb know about these credentials? I mean how are we going to provide these username and password to mongodb through deployment file?

Well, Mongo provided us 2 environment variables; one for username and another for password. We can find them on the docker website. Let’s check these environment variables as well as port.

From the above image you can see that the port is listed on mongo image page of docker website. Similarly, we can find the credentials environment variables too –

Mongo pod will get the username from MONGO_INITDB_ROOT_USERNAME and password from MONGO_INITDB_ROOT_PASSWORD.

Let’s use the port and environment variables in our deployment file –

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-deployment
  labels:
    app: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo
        ports:
        - containerPort: 27017
        env:
        - name: MONGO_INITDB_ROOT_USERNAME
          value:
        - name: MONGO_INITDB_ROOT_PASSWORD
          value:

We kept the values of these environment variables to be empty because it’s not safe to list credentials in deployment file. We are going to create Secrets file for it.

Secrets File for Mongo Credentials

Here is the yaml code for the secret file –

apiVersion: v1
kind: Secret
metadata:
    name: mongodb-secret
type: Opaque
data:
    mongo-root-username: 
    mongo-root-password: 

Few important points about this file –

1. .kind – We are using Secret as kind because this is a secret yaml file.
2. .metadata.name – Name of this secret deployment.
3. .type – There are multiple types like Opaque, TLS, basic-auth, ssh-auth, token etc. In Opaque you can define data according to you. But in other types you need to follow the protocol. They are used for specific types of secrets.

We have defined two keys – mongo-root-username and mongo-root-password. Values are empty for now but we will fill them soon.

The values of these keys are not stored in plain text. Else we need to store them in base64. I am using https://www.base64encode.org/ for encoding plain text to base64. Here are the values –

Key	Plain Text Value	Base64 Value
mongo-root-username	admin	YWRtaW4=
mongo-root-password	my5ecr3t9assw0r6	bXk1ZWNyM3Q5YXNzdzByNg==

So now the secrets file will look like this –

apiVersion: v1
kind: Secret
metadata:
    name: mongodb-secret
type: Opaque
data:
    mongo-root-username: YWRtaW4=
    mongo-root-password: bXk1ZWNyM3Q5YXNzdzByNg==

Our secret file is completed. We are ready to use the keys into our mongo deployment file. But before that, we need to run this file so that kubernetes system can get these values. Run this below command in terminal –

kubectl apply -f mongo-secret.yaml

Check this out in the screenshot of my terminal –

Now we are all set to reference the keys from this secret file to our deployment file –

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-deployment
  labels:
    app: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo
        ports:
        - containerPort: 27017
        env:
        - name: MONGO_INITDB_ROOT_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-username
        - name: MONGO_INITDB_ROOT_PASSWORD
          valueFrom: 
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-password

To reference key from secret file, we need to add .valueFrom.secretKeyRef for the env.

• .valueFrom.secretKeyRef.name refers to the .metadata.name property of secret file.
• .valueFrom.secretKeyRef.key refers to the key name.

So, the username & password values we provided in secret file is passed to the deployment which later will pass to the mongodb database using environment variables.

Our mongo deployment file is completed. It’s time to run it in kubernetes. Use this command –

kubectl apply -f mongo.yaml

From the above terminal image you can see that deployment is successfully applied. Also when you run –

kubectl get all

you can see the pod in running state. It could be in creating state for you as it takes some time to fetch the image and complete the setup. Check again after 5 minutes.

It’s time to create the internal service for mongodb.

Creating Internal Service for Mongodb Deployment

Services are required for communication among multiple applications. For example we need mongo-express to communicate to mongodb so we need a service. Similarly, browser needs to communicate with mongo-express, so we need a service for that too.

Internal service can’t be accessed outside cluster so it’s a good fit for things like database.

One more thing, you can either create a separate service file or you can include it in deployment file. The benefit of having a single file is that you don’t need to maintain multiple files as the service belongs to that deployment only. Here we are including the service in our mongo.yaml deployment file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-deployment
  labels:
    app: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo
        ports:
        - containerPort: 27017
        env:
        - name: MONGO_INITDB_ROOT_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-username
        - name: MONGO_INITDB_ROOT_PASSWORD
          valueFrom: 
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-password
---
apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
spec:
  selector:
    app: mongodb
  ports:
    - protocol: TCP
      port: 27017
      targetPort: 27017

This is the complete file with deployment + service. Let’s understand various important points of service in it –

• --- is used to separate deployment and service.
• .kind is Service here.
• .metadata.name represents the name of the service which is mongodb-service.
• .spec.selector is used to connect a service with the matching pod. Here we used app: mongodb which matches the .spec.template.metadata.labels in deployment file.
• .spec.ports is the most important part of service. It makes a connection between the port used by other application to connect with this one. So, mongo can run on 27017 port but mongo-express can call it using different port. All we need to do is specify here in service.
• .spec.ports.protocol is the protocol which will be TCP in most cases.
• .spec.ports.port is the service port. Other application will use this port to access mongodb.
• .spec.ports.targetPort is the container port. So, it must match with .spec.template.spec.containers.ports.containerPort in deployment file.

Since our service is created, we are ready to run it. Use apply command –

kubectl apply -f mongo.yaml

From the image you can see that the service is successfully created.

Creating Deployment for Mongo-Express

Mongo-Express is used to create a UI for accessing mongodb data. First let’s create the deployment file for it –

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo-express
  labels:
    app: mongo-express
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongo-express
  template:
    metadata:
      labels:
        app: mongo-express
    spec:
      containers:
      - name: mongo-express
        image: mongo-express

We have already explained the various keys of deployment file. Here we are fetching mongo-express image from docker.

Now we need to know few things –

1. Which port of mongo-express will service connect to?
2. How will mongo-express access mongodb or connect with the service of mongodb?

Every application needs to have a port on which service will connect. In mongodb we had 27017. Similarly mongo-express provides a port which is listed on docker website –

As shown in the image above, mongo-express listens to port 8081.

Now our second question, how will it connect with mongodb? Well we answered it in the beginning of the article that it requires url of mongodb database and credentials. We have already defined credentials in secrets file and we will use them. For url, we need a configmap file which we will create next.

How to pass these values? Again we have environment variables for it –

All the environment variables are shown in the above image. Out of these we are interested in –

• ME_CONFIG_MONGODB_ADMINUSERNAME – We will provide admin username in this.
• ME_CONFIG_MONGODB_ADMINPASSWORD – This is for admin password.
• ME_CONFIG_MONGODB_SERVER – This is for mongodb database url.

Now we have everything except database url. So, lets create a configmap file.

Creating configmap.yaml file

You should be clear why we are creating a configmap file. Actually we store those values here which are not secret and could be accessed by multiple deployments. A mongodb database could be accessed by mongo-express, php, nodejs, python or any other framework or languages in your package.

So, it’s not a good idea to include the url directly in the deployments of these frameworks. In future if you need to change the db then you will need to replace it from all the deployments. But with configmap you just need to make changes in this file only.

Let’s create the configmap.yaml file –

apiVersion: v1
kind: ConfigMap
metadata:
  name: mongodb-configmap
data:
  database_url: mongodb-service

The point of interest in this file is .data.database_url which is pointing to mongodb-service.

Let’s now complete our deployment file for mongo-express with port, credentials and url –

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo-express
  labels:
    app: mongo-express
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongo-express
  template:
    metadata:
      labels:
        app: mongo-express
    spec:
      containers:
      - name: mongo-express
        image: mongo-express
        ports:
        - containerPort: 8081
        env:
        - name: ME_CONFIG_MONGODB_ADMINUSERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-username
        - name: ME_CONFIG_MONGODB_ADMINPASSWORD
          valueFrom: 
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-password
        - name: ME_CONFIG_MONGODB_SERVER
          valueFrom: 
            configMapKeyRef:
              name: mongodb-configmap
              key: database_url

Now the last thing. We need an external service for browsers to connect with mongo-express. Let’s create it.

Creating External Service for Mongo-Express

External service is different than internal service because they are used to be accessed by applications outside the cluster. So, a web browser can contact an external service using ip-address and port.

Just like internal service of mongodb deployment, we are going to include this service in mongo-express deployment. Here is the complete code –

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo-express
  labels:
    app: mongo-express
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongo-express
  template:
    metadata:
      labels:
        app: mongo-express
    spec:
      containers:
      - name: mongo-express
        image: mongo-express
        ports:
        - containerPort: 8081
        env:
        - name: ME_CONFIG_MONGODB_ADMINUSERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-username
        - name: ME_CONFIG_MONGODB_ADMINPASSWORD
          valueFrom: 
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-password
        - name: ME_CONFIG_MONGODB_SERVER
          valueFrom: 
            configMapKeyRef:
              name: mongodb-configmap
              key: database_url
---
apiVersion: v1
kind: Service
metadata:
  name: mongo-express-service
spec:
  selector:
    app: mongo-express
  type: LoadBalancer  
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
      nodePort: 30000

This service is external because it has .spec.type as LoadBalancer. Also we added a new port here, nodePort. This port is used by browsers to connect to this service. It has to be between this range – 30000 to 32767.

Time to apply this deployment, service, and configmap in Kubernetes.

Running mongo-express deployment, service and configmap

First we will need to apply configmap because it is used in deployment file –

kubectl apply -f mongo-configmap.yaml

Now we will apply the mongo-express deployment file –

kubectl apply -f mongo-express.yaml

Testing

Let’s test our setup if everything is working fine. I want to check what services, pods, deployments, replicas etc. are running on my kubernetes cluster. Use the below command –

kubectl get all

The output will be as shown in the below image –

Note: If you are using Windows, then you will find External-IP as localhost in mongo-express-service, as shown in above image. THIS WILL NOT WORK. In order to make it work, we will need to change .spec.type as LoadBalancer to NodePort in external service.

So, for windows only, the mongo-express.yaml will change to this –

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo-express
  labels:
    app: mongo-express
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongo-express
  template:
    metadata:
      labels:
        app: mongo-express
    spec:
      containers:
      - name: mongo-express
        image: mongo-express
        ports:
        - containerPort: 8081
        env:
        - name: ME_CONFIG_MONGODB_ADMINUSERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-username
        - name: ME_CONFIG_MONGODB_ADMINPASSWORD
          valueFrom: 
            secretKeyRef:
              name: mongodb-secret
              key: mongo-root-password
        - name: ME_CONFIG_MONGODB_SERVER
          valueFrom: 
            configMapKeyRef:
              name: mongodb-configmap
              key: database_url
---
apiVersion: v1
kind: Service
metadata:
  name: mongo-express-service
spec:
  selector:
    app: mongo-express
  type: NodePort  
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
      nodePort: 30000

Now we can open our browser and search for the url – http://{External-IP}:30000

Since I am using windows, so External-IP is localhost. But if you are using Mac or Linux, then you will get an IP address.

You can see from above image that we have successfully started our mongo-express application in browser. Now we can create database, add tables, modify records etc.

Conclusion

In this article we learned a complete end to end creation and deployment of a project in Kubernetes. We saw how we can create deployment yaml files along with services, configmaps, secrets etc. Finally we run our application through browser. If you have any doubts in this whole process then feel free to ask from comments below.