Setting up Kubernet and Laravel or Laravel on Docker can be tedious. Hell, setting up Laravel on multiple servers can be a real pain. Been doing research on setting up a Laravel web server with a web app, database server, Redis server, storage server and for a long time. One preferably at Digital Ocean because I love them for their User Interface, innovation and developer friendly setup.
I have tried Laradock, LENDD, Stedding, more basic server setups and many others. All successful to some degree but never satisfactory. One of the main issues being dealing with automating Let’s Encrypt Certificates for random domains.
I am currently convinced that Kubernetes is the way to go. It is modern, scales automatically when need be and only uses resources when need be. It can also be managed really well these days with admin packages and Helm.
And I am convinced we should be able to set things up in one week. I however also want to automate all so we can do this again and again. For that I have not found great total packages yet.
New Git Branch
we create a new git branch to work on this in our Laravel package:
git checkout -b k8
https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging
Here we can add the needed files like for provisioning the Kubernetes cluster with Terraform.
Provisioning
We provision our Kubernetes cluster with a three nodes and auto scaling using Digital Ocean. This we want to automate and do the easiest way possible. We do not want to use a paid tool, we do not want to use Digital Ocean’s User Interface. We will be using command line tools such as doctl and Terraform.
Doctl
We set up Digital Ocean’s command line tool as we will be using this for various tasks and checkups. See documentation at: https://github.com/digitalocean/doctl
To install it use this on MacOS:
brew update
brew install doctlThen, if you need to authenticate or re-authenticate do
doctl auth initYou may be asked to add your authentication token. You can generate one in Digital Ocean’s control panel under API.
Terraform
You can set up a cluster with Terraform. To install it for your OS use https://www.terraform.io/downloads.html .To upgrade just download the latest version and install it. For MacOS users you can also install Terraform with Homebrew:
brew install terraform
or to upgrade
brew upgrade terraform
And to check the version:
terraform version
I currently have Terraform v0.12.24
To check for current Kubernetes versions use doctl:
doctl kubernetes options versions Slug Kubernetes Version 1.16.6-do.2 1.16.6 1.15.9-do.2 1.15.9 1.14.10-do.2 1.14.10
Base Repository
Base alpha repo with infrastructure code is at https://github.com/Larastudio/laravel-k8/tree/master/infrastructure
This setup is not done yet! So use everything with great caution. It is based of Taito’s package https://github.com/TaitoUnited/terraform-digitalocean-kubernetes-infrastructure .
I am also experimenting with their command line tool and templates, but that is for another blog post. Not made up my mind whether I want to use their all in one package or this more finished setup package. Inclined to use this more ready made DO package .. for now.
Kubernetes Managed Cluster
Then adjust setup with autoscaling below accordingly. This so we can provision our own managed Kubernetes cluster at Digital Ocean:
/**
* Copyright 2019 Taito United
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# https://github.com/TaitoUnited/terraform-digitalocean-kubernetes-infrastructure
resource "digitalocean_kubernetes_cluster" "kubernetes" {
count = var.kubernetes_name != "" ? 1 : 0
name = var.kubernetes_name
region = var.region
version = var.kubernetes_version
node_pool {
name = "worker-pool"
size = var.kubernetes_node_size
# node_count = var.kubernetes_node_count
auto_scale = var.kubernetes_autoscale
min_nodes = var.kubernetes_min_nodes
max_nodes = var.kubernetes_max_nodes
}
lifecycle {
prevent_destroy = true
}
}
- https://www.digitalocean.com/community/tutorials/how-to-use-terraform-with-digitalocean ,
- https://www.terraform.io/docs/providers/do/r/kubernetes_cluster.html,
- https://www.digitalocean.com/docs/kubernetes/
- https://www.digitalocean.com/docs/platform/availability-matrix/#other-product-availability
The text above can be added to kubernetes.tf or the first terraform file we will be using.
NB To you can work with the latest 0.12 version Terraform Language server on Visual Studio Code run terraform: Enable Language Server
Provisioning Directory
We then create directory provisioning, add mentioned file here above to it and then do a
terraform init
and once done successfully we see
➜ provisioning git:(k8) ✗ terraform init Initializing the backend… Initializing provider plugins… Checking for available provider plugins… Downloading plugin for provider "digitalocean" (terraform-providers/digitalocean) 1.17.0… The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "…" constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. provider.digitalocean: version = "~> 1.17" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.
meaning all is well and this directory has been set up to have Terraform do its thing.
.gitignore
Do not forget to add infrastructure/.terraform to your .gitignore. In fact, we have this for Terraform alone in .gitignore
infrastructure/.terraform infrastructure/*.tfvars infrastructure/.tfstate infrastructure/*.tfstate.backup
Terraform Test
To do a quick test run terraform plan. Currently we see
➜ provisioning git:(k8) ✗ terraform plan
Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
create
Terraform will perform the following actions:
# digitalocean_kubernetes_cluster.k8 will be created
resource "digitalocean_kubernetes_cluster" "k8" {
cluster_subnet = (known after apply)
created_at = (known after apply)
endpoint = (known after apply)
id = (known after apply)
ipv4_address = (known after apply)
kube_config = (sensitive value)
name = "k8"
region = "ams3"
service_subnet = (known after apply)
status = (known after apply)
updated_at = (known after apply)
version = "1.16.6-do.2"
vpc_uuid = (known after apply)
node_pool {
actual_node_count = (known after apply)
auto_scale = true
id = (known after apply)
max_nodes = 5
min_nodes = 1
name = "autoscale-worker-pool"
nodes = (known after apply)
size = "s-2vcpu-2gb"
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.Variables
Variables we set in terraform.tfvars:
do_token = "" spaces_access_id = "" spaces_secret_key = "" region = "" email = "" state_bucket = "" helm_enabled "true" helm_nginx_ingress_classes = "" helm_nginx_ingress_replica_counts = "" kubernetes_name = "" kubernetes_context "" kubernetes_version = "" kubernetes_node_size = "" kubernetes_node_count = "" mysql_instances = "" mysql_node_sizes = "" mysql_node_counts = "" redis_instances = "" redis_node_sizes = "" # redis_node_counts = "" kubernetes_min_nodes = 1 kubernetes_max_nodes = 3
NB Switched to auto scaling for nodes so commented out node_count.
Digital Ocean Managed Database
/**
* Copyright 2019 Taito United
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
resource "digitalocean_database_cluster" "mysql" {
count = length(var.mysql_instances)
name = var.mysql_instances[count.index]
engine = "mysql"
size = var.mysql_node_sizes[count.index]
region = var.region
node_count = var.mysql_node_counts[count.index]
lifecycle {
prevent_destroy = true
}
}
more to be added..
Redis Cluster
# https://www.terraform.io/docs/providers/do/r/database_cluster.html
resource "digitalocean_database_cluster" "redis-base" {
count = length(var.redis_instances)
name = var.redis_instances[count.index]
engine = "redis"
size = var.redis_node_sizes[count.index]
region = var.region
node_count = var.redis_node_counts[count.index]
lifecycle {
prevent_destroy = true
}
}more to be added ..
Digital Ocean Spaces
To add a Digital Ocean Spaces Bucket for storing all our static data we use the following plan:
/**
* Copyright 2019 Taito United
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
resource "digitalocean_spaces_bucket" "state" {
count = var.state_bucket != "" ? 1 : 0
name = var.state_bucket
region = var.region
acl = "private"
lifecycle {
prevent_destroy = true
}
}
more to be added..
Current State of Affairs
current test run will show
terraform plan
Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
create
Terraform will perform the following actions:
# digitalocean_database_cluster.mysql-main will be created
resource "digitalocean_database_cluster" "mysql-main" {
database = (known after apply)
engine = "mysql"
host = (known after apply)
id = (known after apply)
name = "smt-mysql-cluster"
node_count = 1
password = (sensitive value)
port = (known after apply)
private_host = (known after apply)
private_network_uuid = (known after apply)
private_uri = (sensitive value)
region = "ams3"
size = "db-s-1vcpu-1gb"
uri = (sensitive value)
urn = (known after apply)
user = (known after apply)
version = "8"
}
digitalocean_kubernetes_cluster.k8 will be created
resource "digitalocean_kubernetes_cluster" "k8" {
cluster_subnet = (known after apply)
created_at = (known after apply)
endpoint = (known after apply)
id = (known after apply)
ipv4_address = (known after apply)
kube_config = (sensitive value)
name = "k8"
region = "ams3"
service_subnet = (known after apply)
status = (known after apply)
updated_at = (known after apply)
version = "1.16.6-do.2"
vpc_uuid = (known after apply)
node_pool {
actual_node_count = (known after apply)
auto_scale = true
id = (known after apply)
max_nodes = 5
min_nodes = 1
name = "autoscale-worker-pool"
nodes = (known after apply)
size = "s-2vcpu-2gb"
}
}
digitalocean_spaces_bucket.clients will be created
resource "digitalocean_spaces_bucket" "clients" {
acl = "private"
bucket_domain_name = (known after apply)
force_destroy = false
id = (known after apply)
name = "clients"
region = "ams3"
urn = (known after apply)
}
Plan: 3 to add, 0 to change, 0 to destroy.
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.Docker and Helm
See laravel and Kubernetes blog post . Will merge terraform part in the repository I have been building there. Only will make some changes still as I will use a Digital Ocean Load Balancer as Ingress and using Resty Lua for automated LE SSL delivery.
