Project story: Automate AEM deployments for a Swiss bank

Reading Time: 5 minutes

A large bank in St. Gallen, Switzerland had the need to improve the AEM deployment process for its various staging environments. It was one of my first projects for 47 North Labs and was settled to run for 6 months starting in October 2018. The following blog gives a short project overview.

Getting started

Starting to work for a new customer is always exciting to me because every company and team has a unique mindset and culture. Usually, it takes a few days or weeks to get to know the new teammates. But this time it was completely different, as I already worked with each of the three team members and their supervisor together in one before my previous company. It was nice to meet old colleagues and we had a very good start.

Deployment process before automation.
Source: www.dreamstime.com

Technology Stack

The technology stack was already defined and the servers ordered. But it took a while until the infrastructure was ready and for the time being, I worked on my local machine.

Jenkins was set as the central tool for build orchestration, deployments, and various DevOps tasks. All the pipeline source code is stored in GitLab and the main business application we’re dealing with is Adobe Experience Manager (AEM).

A relatively large amount of work was needed for the initial setup like enabling connectivity to the relevant systems, basic shared library, and getting to know the internal processes. Read more about Jenkins behind a corporate proxy as an example for this setup: https://www.47northlabs.com/knowledge-base/update-jenkins-plugins-behind-a-corporate-proxy/

Implemented Pipelines

The bank has two different AEM projects: one for the corporate website and another for their intranet. They require a slightly different deployment pipeline and both have three environments: development, staging, and production.

Besides the deployment pipelines, there are pipelines for copying content from the production to the development environment and restoring a complete production environment into the staging environment in order to have an exact copy and a good baseline for approvals.

Many auxiliary jobs like start/stop AEM + Dispatcher, checking the health of instances, fetch last backup time, and execute Groovy scripts are used in the deployment pipelines as well in an independent executable job.

An example of a Jenkins Pipeline
Source: https://jenkins.io
An example of a Gitlab Pipeline
Source: https://docs.gitlab.com/ee/ci/pipelines.html

Advantages of automation

The automation of the various processes brought faster deployments. But more important transparency and centralized logs about what exactly happened and higher quality as repetitive tasks are always executed.

One example is the backup check, which needed coordination and forced to long waiting times. Now an API is used and the automation pipeline has instant feedback about the last backup time and shows a note if a backup is missing. Before, such a step might have been skipped in order to save some time.

With each built pipeline, some more little and reusable helpers were introduced which made it then again easier and faster to create the next pipelines. Think of a construction kit.

Deployment process with automation.
Source: www.wikimedia.org

Project finished ➞ client is very happy

After several months of close collaboration, more and more pipelines have been implemented and are used to support the crucial deployment processes countless times.

I enjoyed building-up the AEM automation and believe it’s a very good aid for higher quality and further extensions.

After a warm welcome and six months of working together, it was time to say good-bye as the project had a fixed time span. The client’s team was very kind and gave me even some great presents to remember the exciting time in St. Gallen.

Present from client: Swiss beer, chocolate, bratwurst and biber

Deploying a Vue.js app on the Google Cloud Platform using GitLab AutoDeploy

Reading Time: 4 minutes

For a few weeks now, we are working on several internal projects. We are currently developing different products and services, which we want to release soon™️. We started from scratch, so we had the freedom to choose our tools, technologies and frameworks. We decided to deploy our application on a Kubernetes cluster on the Google Cloud. Here is a short howto, to automate the deployment process.

Getting started

First, we need an account on Google Cloud. When you register for the first time, they give you access to the clusters and $300 in credit.

  • Google Cloud account is required
  • Node.js (v10.x)
  • npm (v5.6.0)
  • Docker
  • Git & GitLab

We are using the GitLab AutoDeploy, Google Cloud, Vue.js and Docker to build this CI/CD.

Creating The Vue App

# let's create our workspace
mkdir vue-ci-app
cd vue-ci-app/

# install vue
npm install @vue/cli -g

# create the vue-app (select default settings)
vue create vue-app
cd vue-app/

# let's test out the app locally
npm run serve

We first create a folder and enter it, then we use npm to install the Vue command line interface and use that to create a bootstrapped Vue app. It should be accessible at http://localhost:8080/

Docker Config

FROM node:lts-alpine

# install simple http server for serving static content
RUN npm install -g http-server

# make the 'app' folder the current working directory
WORKDIR /app

# copy both 'package.json' and 'package-lock.json' (if available)
COPY package*.json ./

# install project dependencies
RUN npm install

# copy project files and folders to the current working directory (i.e. 'app' folder)
COPY . .

# build app for production with minification
RUN npm run build

EXPOSE 5000
CMD [ "http-server", "-p 5000", "dist" ]
  • From pulls the latest node from the public docker registry (Docker Hub)
  • Then we install http-server, a simple static serve
  • Afterwards, we make a directory where we will place the app
  • Copy our package.json local machine to the docker instance
  • After installing the dependencies and copying the dist of the app, we run a build which we serve using http-server
  • This is all done in a docker container

GitLab & Kubernetes

The last part of the deployment begins with setting up a Kubernetes cluster and enabling GitLab Autodeploy.

First, we need to go to our Project > Settings > CI/CD > Auto DevOps. Enable the default pipeline. This is the auto part that escapes the need for gitlab-ci.yml.

Then we need to add a cluster which means going to our GC account and setting up a Kubernetes cluster. We need to specify a name, environment scope, type of project, region, number of nodes, machine type, and whether it’s an RBAC-enabled cluster.

We need to go to GitLab, to the CI/CD page, and add a GitLab runner, this needs to be configured to run docker.

We need to set a base domain, and finally add our created Kubernetes cluster to the GitLab Autodeploy.

We have three jobs if all is set up and done, build and review phases where we have a build on the remote Kubernetes cluster and review where we can add linting and tests. Cleanup is a manual job that deletes the commit from the pipeline ready to be deployed again.

Deploy Spring Boot Application on Google Cloud with GitLab

Reading Time: 5 minutes

A lot of developers experience a painful process with their code being deployed on the environment. We, as a company, suffer the same thing so that we wanted to create something to make our life easier.

After internal discussions, we decided to make fully automated CI/CD process. We investigated and came up with decision for that purpose to implement Gitlab CI/CD with google cloud deployment.

Further in this blog you can see how we achieved that and how you can achieve the same.

Let’s start with setting up. 🙂

  • After that, we create simple rest controller for testing purposes.
package com.northlabs.gitlabdemo.rest;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class RootController {

    @GetMapping(value = "/")
    public String root() {
        return "Hello from Root";
    }

    @GetMapping(value = "/demo")
    public String demo() {
        return "Hello from Demo";
    }

}
  • Next step is to push the application to our GitLab repo.
  1. cd path_to_root_folder
  2. git init
  3. git remote add origin https://gitlab.com/47northlabs/47northlabs/product/playground/gitlab-demo.git
  4. git add .
  5. git commit -m “Initial commit”
  6. git push -u origin master

Now, after we have our application in GitLab repository, we can go to setup Google Cloud. But, before you start, be sure that you have a G-Suite account with enabled billing.

  • First step is to create a new project: in my case it is northlabsgitlab-demo.

Create project: northlabs-gitlab-demo
  • Now, let’s create our Kubernetes Cluster.

It will take some time after Kubernetes clusters are initialized so that GitLab will be able to create a cluster.

We are done with Google Cloud, so it’s time to setup Kubernetes in our GitLab repository.

  • First we add Kubernetes cluster.
Add Kubernetes Cluster
Sign in with Google
  • Next, we give a name to the cluster and select a project from our Google Cloud account: in my case it’s gitlab-demo.
  • The base domain name should be set up.
  • Installing Helm Tiller is required, and installing other applications is optional (I installed Ingress, Cert-Manager, Prometheus, and GitLab Runner).

Install Helm Tiller

Installed Ingress, Cert-Manager, Prometheus, and GitLab Runner

After installing the applications it’s IMPORTANT to update your DNS settings. Ingress IP address should be copied and added to your DNS configuration.
In my case, it looks like this:

Configure DNS

We are almost done. 🙂

  • The last thing that should be done is to enable Auto DevOps.
  • And to setup Auto DevOps.

Now take your coffee and watch your pipelines. 🙂
After couple of minutes your pipeline will finish and will look like this:

Now open the production pipeline and in the log, under notes section, check the URL of the application. In my case that is:

Application should be accessible at: http://47northlabs-47northlabs-product-playground-gitlab-demo.gitlab-demo.47northlabs.com

Open the URL in browser or postman.

https://47northlabs-47northlabs-product-playground-gitlab-demo.gitlab-demo.47northlabs.com
https://47northlabs-47northlabs-product-playground-gitlab-demo.gitlab-demo.47northlabs.com/demo
  • Let’s edit our code and push it to GitLab repository.
package com.northlabs.gitlabdemo.rest;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class RootController {

    @GetMapping(value = "/")
    public String root() {
        return "Hello from Root v1";
    }

    @GetMapping(value = "/demo")
    public String demo() {
        return "Hello from Demo v1";
    }
}

After the job is finished, if you check the same URL, you will see that the values are now changed.


https://47northlabs-47northlabs-product-playground-gitlab-demo.gitlab-demo.47northlabs.com

https://47northlabs-47northlabs-product-playground-gitlab-demo.gitlab-demo.47northlabs.com/demo

And we are done !!!

This is just a basic implementation of the GitLab Auto DevOps. In some of the next blogs we will show how to customize your pipeline, and how to add, remove or edit jobs.

Hackdayz #18: Git Repo Sync Tool

Reading Time: 3 minutes

Working as a consultant usually involves handling client and local repositories seamlessly and is often pretty simple as an individual.

When working in a distributed team environment where only a portion of the members have access to certain areas of the project, the situation becomes a bit tricky to handle. We identified this as a minor showstopper in our organization and during our internal hackathon Hackdayz18 we decided to make our lives easier.

Application overview and features

  • Ability to synchronize public and client repositories with a single button click
  • Persist user data and linked repositories
  • Visualized state of repositories
  • Link commits to different user
  • Modify commit message and squash all commits into one

Our Team

Nikola Gjeorgjiev (Frontend Engineer)
Antonie Zafirov (Software Engineer)
Fatih Korkmaz (Managing Partner)

Challenges and results

Initially what we had in mind was a tool that would read two repository URLs and with no further constraints squash all commits on one of the repositories, change the commit message and push the result on the second repository.

It turned out things weren’t that simple for the problem we were trying to solve.

Through trial and error, we managed to build a working demo of our tool in a short time frame that only need small tweaking in order to be used.

Workflow for the GitSync tool
Workflow for the GitSync tool

The final result was a small application that is able to persist user data and using given user credentials to read the state of repositories linked to the user. The final step is where all the magic happens and the content of the source repository is transferred into the destination repository.

There is still work to be done to get the application production ready and available to the team, but in the given timeframe we did our best, I am happy with our results.

Technology stack

  • Gitgraph.js – JavaScript library which visually presents Git branching, Git workflow or whatever Git tree you’d have in mind
  • GitLab API – Automating GitLab via a simple and powerful API
  • VueJS – front-end development framework
  • SpringBoot – back-end development framework

Conclusion

Even though we underestimated the problem we were facing, we pulled through and were able to deliver the base of the solution of the problem.

The hackathon was a valuable learning experience for the entire team and we’re looking forward to the next one!