Blog

Laravel + K8 & Terraform

Setting up Kubernet and Laravel or Laravel on Docker can be tedious. Hell, setting up Laravel on multiple servers can be a real pain. Been doing research on setting up a Laravel web server with a web app, database server, Redis server, storage server and for a long time. One preferably at Digital Ocean because I love them for their User Interface, innovation and developer friendly setup.

I have tried Laradock, LENDD, Stedding, more basic server setups and many others. All successful to some degree but never satisfactory. One of the main issues being dealing with automating Let’s Encrypt Certificates for random domains.

I am currently convinced that Kubernetes is the way to go. It is modern, scales automatically when need be and only uses resources when need be. It can also be managed really well these days with admin packages and Helm.

And I am convinced we should be able to set things up in one week. I however also want to automate all so we can do this again and again. For that I have not found great total packages yet.

New Git Branch

we create a new git branch to work on this in our Laravel package:

git checkout -b k8

https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging

Here we can add the needed files like for provisioning the Kubernetes cluster with Terraform.

Provisioning

We provision our Kubernetes cluster with a three nodes and auto scaling using Digital Ocean. This we want to automate and do the easiest way possible. We do not want to use a paid tool, we do not want to use Digital Ocean’s User Interface. We will be using command line tools such as doctl and Terraform.

Doctl

We set up Digital Ocean’s command line tool as we will be using this for various tasks and checkups. See documentation at: https://github.com/digitalocean/doctl

To install it use this on MacOS:

brew update
brew install doctl

Then, if you need to authenticate or re-authenticate do

doctl auth init

You may be asked to add your authentication token. You can generate one in Digital Ocean’s control panel under API.

Terraform

You can set up a cluster with Terraform. To install it for your OS use https://www.terraform.io/downloads.html .To upgrade just download the latest version and install it. For MacOS users you can also install Terraform with Homebrew:

brew install terraform

or to upgrade

brew upgrade terraform

And to check the version:

terraform version

I currently have Terraform v0.12.24

To check for current Kubernetes versions use doctl:

doctl kubernetes options versions
Slug           Kubernetes Version
1.16.6-do.2    1.16.6
1.15.9-do.2    1.15.9
1.14.10-do.2   1.14.10

Base Repository

Base alpha repo with infrastructure code is at https://github.com/Larastudio/laravel-k8/tree/master/infrastructure

This setup is not done yet! So use everything with great caution. It is based of Taito’s package https://github.com/TaitoUnited/terraform-digitalocean-kubernetes-infrastructure .

I am also experimenting with their command line tool and templates, but that is for another blog post. Not made up my mind whether I want to use their all in one package or this more finished setup package. Inclined to use this more ready made DO package .. for now.

Kubernetes Managed Cluster

Then adjust setup with autoscaling below accordingly. This so we can provision our own managed Kubernetes cluster at Digital Ocean:

/**
 * Copyright 2019 Taito United
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

 # https://github.com/TaitoUnited/terraform-digitalocean-kubernetes-infrastructure

resource "digitalocean_kubernetes_cluster" "kubernetes" {
  count   = var.kubernetes_name != "" ? 1 : 0

  name    = var.kubernetes_name
  region  = var.region
  version = var.kubernetes_version

  node_pool {
    name       = "worker-pool"
    size       = var.kubernetes_node_size
    # node_count = var.kubernetes_node_count
    auto_scale = var.kubernetes_autoscale
    min_nodes  = var.kubernetes_min_nodes
    max_nodes  = var.kubernetes_max_nodes
  }

  lifecycle {
    prevent_destroy = true
  }
}

The text above can be added to kubernetes.tf or the first terraform file we will be using.

NB To you can work with the latest 0.12 version Terraform Language server on Visual Studio Code run terraform: Enable Language Server

Provisioning Directory

We then create directory provisioning, add mentioned file here above to it and then do a 

terraform init

and once done successfully we see

➜ provisioning git:(k8) ✗ terraform init
Initializing the backend…
Initializing provider plugins…
Checking for available provider plugins…
Downloading plugin for provider "digitalocean" (terraform-providers/digitalocean) 1.17.0…
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "…" constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
provider.digitalocean: version = "~> 1.17"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

meaning all is well and this directory has been set up to have Terraform do its thing.

.gitignore

Do not forget to add infrastructure/.terraform to your .gitignore. In fact, we have this for Terraform alone in .gitignore

infrastructure/.terraform
infrastructure/*.tfvars
infrastructure/.tfstate
infrastructure/*.tfstate.backup

Terraform Test

To do a quick test run terraform plan. Currently we see

➜ provisioning git:(k8) ✗ terraform plan
Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
create
Terraform will perform the following actions:
# digitalocean_kubernetes_cluster.k8 will be created
resource "digitalocean_kubernetes_cluster" "k8" {
cluster_subnet = (known after apply)
created_at = (known after apply)
endpoint = (known after apply)
id = (known after apply)
ipv4_address = (known after apply)
kube_config = (sensitive value)
name = "k8"
region = "ams3"
service_subnet = (known after apply)
status = (known after apply)
updated_at = (known after apply)
version = "1.16.6-do.2"
vpc_uuid = (known after apply)
node_pool {
actual_node_count = (known after apply)
auto_scale = true
id = (known after apply)
max_nodes = 5
min_nodes = 1
name = "autoscale-worker-pool"
nodes = (known after apply)
size = "s-2vcpu-2gb"
}
}
Plan: 1 to add, 0 to change, 0 to destroy.

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Variables

Variables we set in terraform.tfvars:

do_token = ""
spaces_access_id = ""
spaces_secret_key = ""
region = ""
email = ""
state_bucket = ""
helm_enabled "true"
helm_nginx_ingress_classes = ""
helm_nginx_ingress_replica_counts = ""
kubernetes_name = ""
kubernetes_context ""
kubernetes_version = ""
kubernetes_node_size = ""
kubernetes_node_count = ""
mysql_instances = ""
mysql_node_sizes = ""
mysql_node_counts = ""
redis_instances = ""
redis_node_sizes = ""
# redis_node_counts = ""
kubernetes_min_nodes = 1
kubernetes_max_nodes = 3

NB Switched to auto scaling for nodes so commented out node_count.

Digital Ocean Managed Database

/**
* Copyright 2019 Taito United
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

resource "digitalocean_database_cluster" "mysql" {
count = length(var.mysql_instances)

name = var.mysql_instances[count.index]
engine = "mysql"
size = var.mysql_node_sizes[count.index]
region = var.region
node_count = var.mysql_node_counts[count.index]

lifecycle {
prevent_destroy = true
}
}

more to be added..

Redis Cluster

# https://www.terraform.io/docs/providers/do/r/database_cluster.html

resource "digitalocean_database_cluster" "redis-base" {
count = length(var.redis_instances)

name = var.redis_instances[count.index]
engine = "redis"
size = var.redis_node_sizes[count.index]
region = var.region
node_count = var.redis_node_counts[count.index]

lifecycle {
prevent_destroy = true
}
}

more to be added ..

Digital Ocean Spaces

To add a Digital Ocean Spaces Bucket for storing all our static data we use the following plan:

/**
* Copyright 2019 Taito United
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

resource "digitalocean_spaces_bucket" "state" {
count = var.state_bucket != "" ? 1 : 0
name = var.state_bucket
region = var.region
acl = "private"

lifecycle {
prevent_destroy = true
}
}

more to be added..

Current State of Affairs

current test run will show

terraform plan
Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
create
Terraform will perform the following actions:
# digitalocean_database_cluster.mysql-main will be created
resource "digitalocean_database_cluster" "mysql-main" {
database = (known after apply)
engine = "mysql"
host = (known after apply)
id = (known after apply)
name = "smt-mysql-cluster"
node_count = 1
password = (sensitive value)
port = (known after apply)
private_host = (known after apply)
private_network_uuid = (known after apply)
private_uri = (sensitive value)
region = "ams3"
size = "db-s-1vcpu-1gb"
uri = (sensitive value)
urn = (known after apply)
user = (known after apply)
version = "8"
}
digitalocean_kubernetes_cluster.k8 will be created
resource "digitalocean_kubernetes_cluster" "k8" {
cluster_subnet = (known after apply)
created_at = (known after apply)
endpoint = (known after apply)
id = (known after apply)
ipv4_address = (known after apply)
kube_config = (sensitive value)
name = "k8"
region = "ams3"
service_subnet = (known after apply)
status = (known after apply)
updated_at = (known after apply)
version = "1.16.6-do.2"
vpc_uuid = (known after apply)
node_pool {
actual_node_count = (known after apply)
auto_scale = true
id = (known after apply)
max_nodes = 5
min_nodes = 1
name = "autoscale-worker-pool"
nodes = (known after apply)
size = "s-2vcpu-2gb"
}
}
digitalocean_spaces_bucket.clients will be created
resource "digitalocean_spaces_bucket" "clients" {
acl = "private"
bucket_domain_name = (known after apply)
force_destroy = false
id = (known after apply)
name = "clients"
region = "ams3"
urn = (known after apply)
}
Plan: 3 to add, 0 to change, 0 to destroy.

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Docker and Helm

See laravel and Kubernetes blog post . Will merge terraform part in the repository I have been building there. Only will make some changes still as I will use a Digital Ocean Load Balancer as Ingress and using Resty Lua for automated LE SSL delivery.

Jasper Frumau

Jasper has been working with web frameworks and applications such as Laravel, Magento and his favorite CMS WordPress including Roots Trellis and Sage for more than a decade. He helps customers with web design and online marketing. Services provided are web design, ecommerce, SEO, content marketing. When Jasper is not coding, marketing a website, reading about the web or dreaming the internet of things he plays with his son, travels or run a few blocks.

Related Articles

Laradock SQLSTATE[HY000] [2002] Connection timed out

Laradock SQLSTATE[HY000] [2002] Connection timed out