About David Peach

this is me

I’m not a “devops” person, but I’m learning

By trade I am a PHP developer. I’ve never done devops in a professional setting. However, for a while I have had a strange fascination with various continuous integration and deployment strategies I’ve seen at many of my places of work.

I’ve seen some very complicated setups over the years, which has created a mental block for me to really dig in and understand setting up integration and deployment workflows.

But in my current role at Geomiq, I had the opportunity of being shown a possible setup — specifically using Kubernetes. And that was sort of a gateway drug, which finally led me to getting a working workflow up and running.

I now want to start sharing what I have learnt and build out a fully-fledged deployment workflow. Not sure how many posts it will take, or what structure it will take, but my aim is to make devops and CI/CD as approachable as possible.

Getting started with Terraform

Terraform is a program that can be used to build your cloud-based infrastructure based off of configuration files that you write. It’s a part of what is referred to as “Infrastructure as code (Iac)”.

Instead of going into various cloud provider UI dashboards and clicking around to build your resources, Terraform can do all that provisioning for you. It uses the cloud provider APIs behind the scenes — you just write exactly the infrastructure that you want to end up with at the end.

In this guide, we will provision a simple Digital Ocean Server (a Droplet in Digital Ocean parlance) using Terraform from our local terminal.

If you don’t yet have a Digital Ocean account, feel free to use my referral link to set one up. With that link you’ll get $200 in credit to use over 60 days.

Setting up Terraform in 4 steps

1 :: Install terraform

Terraform is available to install from pretty much all package repositories out there.

Installing it should be as simple as running a one-line command in your terminal.

2 :: Configure any required cloud provider API tokens

In order to let the Terraform program make changes to your cloud provider account, you will need to set up API tokens and tell Terraform where to find them.

In this post I’ll only be setting up a single one for Digital Ocean.

3 :: Write your main.tf configuration file

A single main.tf file will be enough to get you something working.

Add all of your needed resources / infrastructure in it.

4 :: Run the apply command

By running the terraform apply command against your main.tf file, you can turn your empty cloud infrastructure into a working setup.

Step 1 :: Install Terraform

Terraform’s documentation details the numerous ways of getting it installed across operating systems.

I use Arch Linux and so install it like so:

Bash
sudo pacman -Sy terraform

You can check it is installed and discoverable on your system by checking the version you have installed:

Bash
terraform -v

# My Output
Terraform v1.6.4
on linux_amd64

Now create an empty directory, which will be your “terraform project”. It doesn’t matter what you call the folder.

Then inside that file create a file called main.tf. We’ll come back to this file a little later.

Step 2 :: Configure any required cloud provider API tokens

Head to your Digital Ocean API Tokens dashboard and click “Generate New Token”. Give it a name, choose an expiry and make sure you click the “write” permission option. Click “generate token”.

There are a number of ways we can tell Terraform what our Digital Ocean API Token is:

  • Obviously we could hard code it for the purposes of just getting it running while learning, though I wouldn’t recommend this approach even in testing.
  • Another is to use Terraform-specific environment variables set on your system. This has been my approach in the past. However, I came to realize how unsafe this was as every program you install has the potential to read from your environment variable.
  • A third way is to pass it as a parameter when calling the apply command.

I will be opting for that third option, but I don’t want to have that token saved in my history or have to pass it in everytime I want to run a Terraform command.

So my solution is to write a small wrapper bash script that will read the contents of a file in my home directory (with my token in) and pass it as an argument to the Terraform apply command.

Creating a wrapper bash script to safely pass secret token to command

Create a file in your home directory called “terraform-test”. You can call it anything, just remember to reference it correctly when using it later in the guide.

Inside that file, paste only the API token that you got from your Digital Ocean API dashboard. Then save the file and close it.

Open a new file in the root of your Terraform project and add the following contents:

Bash
#!/usr/bin/bash

terraform "$@" -var "do_token=$(cat ~/terraform-test)"

Save that file as “myterraformwrapper”. (You can call it whatever you want, I use “myterraformwrapper” as an example).

Also make sure to give it executable permissions by running the following command against it:

Bash
chmod o+x myterraformwrapper

The myterraformwrapper script does the following:

  1. Calls the terraform command.
  2. Any arguments you pass to myterraformwrapper get put in the place of "$@"
  3. Appends on to the command, the -var flag and sets the do_token parameter to the contents of the terraform-test file you created previously.

This means:

Bash
./myterraformwrapper apply

… behind the scenes, becomes…

Bash
terraform apply -var "do_token=CONTENTS_OF_YOUR_DO_TOKEN"

This means that you are not having to keep passing your Digital Ocean token in for every command, and you wont end up accidentally leaking the token inside your shell’s env variables.

We will use that file later in this guide.

Step 3 :: Write your main.tf configuration file

For this example, everything will be kept in a single file called main.tf. When you start working on bigger infrastructure plans, there is nothing stopping you from splitting out your configuration into multiple, single-purpose files.

YAML
terraform {
    required_providers {
        digitalocean = {
            source = "digitalocean/digitalocean"
            version = "~> 2.0"
        }
    }
}

variable "do_token" {}

provider "digitalocean" {
  token = var.do_token
}

resource "digitalocean_droplet" "droplet" {
  image    = "ubuntu-22-04-x64"
  name     = "terraform-test"
  region   = "lon1"
  size     = "s-1vcpu-1gb"
}

terraform block

At the top of the file is the terraform block. This sets up the various providers that we want to work with for building out our infrastructure. In this example we only need the digital ocean one.

variable declarations

Variable declarations can be used to keep sensitive information out of out configuration — and thus source control later, as well as making our configuration more reusable.

Each of the variables that our configuration needs to run must be defined as a variable like above. You can define variables in a few different ways, but here I have opted for the simplest.

We can see that all our configuration needs is a do_token value passed to it.

provider setups

Each of the providers that we declare in our terraform block will probably need some kind of setup — such as an api token like our Digital Ocean example.

For us we can see that the setting up of Digital Ocean’s provider needs only a token, which we are passing it from the variable that we will pass in via the cli command.

resource declarations

We then declare the “resources” that we want Terraform to create for us in our Digital Ocean account. In this case we just want it to create a single small droplet as a proof of concept.

The values I have passed to the digitalocean_droplet resource, would be great examples of where to use variables, potentially even with default placeholder values.

I have hard coded the values here for brevity.

Step 4 :: Run the apply command

Before running apply for the first time, we first need to initialize the project:

Bash
terraform init
# You should see some feedback starting with this:
Terraform has been successfully initialized!

You can also run terraform plan before the apply command to see what Terraform will be provisioning for you. However, when running terraform apply, it shows you the plan and asks for explicit confirmation before building anything. So I rarely use plan.

If you run terraform apply, it will prompt you for any variables that your main.tf requires — in our case the do_token variable. We could type it / paste it in every time we want to run a command. But a more elegant solution would be to use that custom bash script we created earlier.

Assuming that bash script is in our current directory — the Terraform project folder — run the following:

Bash
./myterraformwrapper apply

This should display to you what it is planning to provision in your Digital Ocean account — a single Droplet.

Type the word “yes” and hit enter.

You should now see it giving you a status update every 10 seconds, ending in confirmation of the droplet being created.

If you hard back over to your Digital Ocean account dashboard, you should see that new droplet sitting there.

Step 5 :: Bonus: destroying resources.

Just as Terraform can be used to create those resources, it can also be used to destroy them too. It goes without saying that you should always be mindful of just what you are destroying, but in this example we are just playing with a test droplet.

Run the following to destroy your newly-created droplet:

Bash
./myterraformwrapper destroy

Again, it will first show you what it is planning to change in your account — the destruction of that single droplet.

Type “yes” and hit enter to accept.

Next Steps

I love playing with Terraform, and will be sharing anything that I learn along my journey on my website.

You could start working through Terraform’s documentation to get a taste of what it can do for you.

You can even take a look at its excellent registry to see all of the providers that are available. Maybe even dig deep into the Digital Ocean provider documentation and see all of the available resources you could play with.

Just be careful how much you are creating and when testing don’t forget to run the destroy command when you’re done. The whole point of storing your infrastructure as code is that it is dead simple to provision and destroy it all.

Just don’t get leaving test resources up and potentially running yourself a huge bill.

Using docker and docker compose for my Homelab

I’ve seen some very elaborate homelab set-ups online but wanted to get the easiest possible implementation I could, within my current skill set.

As I have quite a lot of experience with using docker for development in my day to day work, I thought I’d just try using docker compose to setup my homelab service

What is docker?

Docker is a piece of software that allows you to package up your services / apps in to “containers”, along with any dependencies that they need to run.

What this means for you, is that you can define all of the things you need to make your specific app work in a configuration file, called a Dockerfile. When the container is then built, it builds it with all of the dependencies that you specify.

This is opposed to the older way of setting up a service / app /website, by installing the required dependencies manually on the host server itself.

By setting up services using docker (and its companion tool docker compose) You remove the need to install manual dependencies yourself.

Not only that, but if different services that you install require different versions of the same dependencies, containers keep those different versions separate.

Installing the docker tools

I use the guide for ubuntu on the official docker website.

Once docker and docker compose are installed on the server, I can then use a single configuration file for each of the services I want to put into my Home Lab. This means I don’t need to worry about the dependencies that those services need to work — because they are in their own containers, they are self-contained and need nothing to be added to the host system.

There are services that can help you manage docker too. But that was one step too far outside of my comfort zone for what I want to get working right now.

I will, however, be installing a service called “Portainer”, detailed in my next Home Lab post, which gives you a UI in which to look at the docker services you have running.

Homelab initial setup

I have gone with Ubuntu Server 22.04 LTS for my Homelab’s operating system.

Most of the videos I’ve seen for Homelab-related guides and reviews tend to revolve around Proxmox and/or TrueNAS. I have no experience with either of those, but I do have experience with Docker, so I am opting to go with straight up docker — at least for now.

Setting up the Operating system

I’m using a Linux-based system and so instructions are based on this.

Step 1: Download the Ubuntu Server iso image

Head here to download your preferred version of Ubuntu Server. I chose the latest LTS version at the time of writing (22.04)

Step 2: Create a bootable USB stick with the iso image you downloaded.

Once downloaded, insert and a usb stick to install the Ubuntu Server iso on to.

Firstly, check where your USB stick is on your filesystem. For that, I use fdisk:

Bash
sudo fdisk -l

Assuming the USB stick is located at “/dev/sdb“, I use the dd command to create my bootable USB (please check and double check where your USB is mounted on your system):

Bash
sudo dd bs=4M if=/path/to/Ubuntu-Server-22-04.iso of=/dev/sdb status=progress oflag=sync

Step 3: Insert and boot to the bootable USB stick into the Homelab computer

Boot the computer that you’re using for your server, using the USB stick as a temporary boot device.

Step 4: Install the operating system

Follow the steps that the set up guide gives you.

As an aside, I set my server ssd drive up with the “LVM” option. This has helped immensely this week, as I have added a second drive and doubled my capacity to 440GB.

Step 5: install and enable ssh remote access

I can’t remember if ssh came installed or enabled, but you can install openssh and then enable the sshd service.

You can then connect to the server from a device on your network with:

Bash
ssh username@192.168.0.77

This assumes your server’s IP address is 192.168.0.77. Chances are very high it’ll be a different number (although the 192.168.0 section may be correct.

Everything else done remotely

I have an external keyboard in case I ever need to plug in to my server. However, now I have ssh enabled, I tend to just connect from my laptop using the ssh command show just above.

Status

Started to re-watch Breaking Bad. This’ll be my second time viewing. Think I’m gonna try and share my favourite shot from each episode.

Setting up mine, and my family’s, Homelab

I’ve opted for what I believe is the easiest, and cheapest, method of setting up my Homelab.

I’m using my old work PC which has the following spec:

  • Quad core processor — i7, I think.
  • 16gb of RAM
  • 440GB ssd storage (2x 220gb in an LVM setup)
  • A USB plug-in network adapter (really want to upgrade to an internal one though)

My Homelab Goals

My homelab goals are centered around two fundamental tenets: lower cost for online services and privacy.

I want to be:

  • Hosting my own personal media backups: All my personal photos and videos I want stored in my own installation of Nextcloud. Along with those I want to also utilize its organizational apps too: calendar; todos; project planning; contacts.
  • Hosting my own music collection: despite hating everything Google stands for, I do enjoy using its Youtube Music service. However, I have many CDs (yes, CDs) in the loft and don’t like the idea of essentially renting access to music. Plus it would be nice to streaming music to offline smart speakers (i.e. not Alexa; Google Speaker; et al.)
  • Hosting old DVD films: I have lots of DVDs in the loft and would like to be able to watch them (without having to buy a new DVD player)
  • Learning more about networking: configuring my own network is enjoyable to me and is something I want to increase my knowledge in. Hosting my own services for my family and myself is a great way to do this.
  • Teach my Son how to own and control his own digital identity (he’s 7 months old): I want my Son to be armed with the knowledge of modern day digital existence and the privacy nightmares that engulf 95% of the web. And I want Him to have the knowledge and ability to be able to control his own data and identity, should He wish to when he’s older.

Documenting my journey

I will be documenting my Homelab journey as best as I can, and will tag all of these posts with the category of Homelab.

Gutted that I’m now all up to date with Taskmaster. Only discovered it a month or so ago and been binging it.

I’m now running pi-hole through my Raspberry Pi 2b.

It’s both amazing and depressing just how many trackers are being blocked by it. I even noticed a regular ping being made to an Amazon endpoint exactly every 10 minutes.

I will try and write up my set up soon, which is a mix of setting up the Raspberry Pi and configuring my home router.


I’ve also managed to finally get a home server running again – using Ubuntu Server LTS.

My plan on my server is to just install services I want to self-host using docker. Docker being the only program I’ve installed on the machine itself.

So far I have installed the following:

  • Home Assistant — On initial playing with this I have decided that it’s incredible. Connected to my LG TV and lets me control it from the app / laptop.
  • Portainer — A graphical way to interact with my docker containers on the server.

Status

I have decided to get back into tinkering with my Raspberry Pi.

I will be blogging my journey as I stumble through my initial playing, through to building out my first proper homelab.

This first Raspberry Pi (model 2b) will be initially used as both a wireguard VPN server and a local DNS server.

The God Slayer by Otep

Front cover of The God Slayer album by Otep

I’ve loved Otep’s music since discovering the album “Sevas Tra” — with that insane album cover being the thing that brought me in.

Earlier today (yesterday) I listened to the recently-released The God Slayer, made up of half original songs and half covers.

Loved it, although Sevas Tra has been — and remains — my favourite of Otep’s.

My favourite songs from my first listen of the album are definitely the covers of Eminem’s “The way I Am” and the Beach Boys’ “California Girls”.

Average Semi-detached house prices in UK by county – Statistical Analysis using R

This is my first data visualization attempt and uses data from HM Land Registry to show to average cost of a semi-detached house in four counties across the past ten years.

You can see the full repository for the project on Github.

The Code

Here I have included the code at the time of writing this post. The git repository code may now differ slightly.

library("tidyverse")

regions  <- c(
  "Derbyshire",
  "Leicestershire",
  "Staffordshire",
  "Warwickshire"
)

data  <- read.csv("props.csv")

data %>%
  filter(Region_Name %in% regions) %>%
  filter(Date > "2013-01-01") %>%
  ggplot(aes(
    Date,
    Semi_Detached_Average_Price
  )) +
  geom_point(aes(color = Region_Name), size = 3) +
  theme_bw() +
  theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1)) +
  labs(
    title = "Average Semi-detached house prices per county",
    x = "Month and Year",
    y = "Average Price",
    color = "County"
  )

ggsave(
  "semi-detached-house-prices-derby-leicester-staffs-warwickshire.png",
  width = 4096,
  height = 2160,
  unit = "px"
)

The Graph

Graph to show increasing semi-detached house prices by county.

Observations

Warwickshire has been the most expensive county to buy a semi-detached house out of the four counties observed.

Derbyshire has been the least expensive county to buy a semi-detached house out of the four counties observed.

The shapes of the line formed seem consistent across the counties; the rate of price increase seems similar between them.

A lot can happen over ten years.

Using a single file neovim configuration file

When I first moved my Neovim configuration over to using lua, as opposed to the more traditional vimscript, I thought I was clever separating it up into many files and includes.

Turns out that it became annoying to edit my configuration. Not difficult; just faffy.

So I decided to just stick it all into a single init.lua file. And now its much nicer to work with in my opinion.

View my Neovim init.lua file on Github.

Don’t stop building

I really enjoy building scripts for my own workflow.

I wish I had the skills to build things in the real world, but until then I’ll keep building stuff in the digital space only.

Although I love working with PHP and Laravel, it is Bash that has re-ignited a passion in me to just build stuff without thinking its got to work towards being some kind of “profitable” side project.

Don’t. Stop. Building.

Lupo static site generator

What is Lupo?

Lupo is a simple static site generator, written in Bash.

I built it for myself to publish to a simple website of my own directly from the command line.

It was inspired by Rob Muhlestein and his approach to the Zettelkasten method.

Installation

Running through the following set of commands will install the lupo bash script to the following location on your system: $HOME/.local/bin/lupo

If you add the $HOME/.local/bin directory to your $PATH, then you can execute the lupo command from anywhere.

I chosen that directory as it seems to be a pretty standard location for user-specific scripts to live.

Bash
git clone https://github.com/davidpeach/lupo
cd ./lupo
./install
cd ..
rm -rf ./lupo

Anatomy of a Lupo website

The structure of a newly-initialized Lupo website project is as follows:

Bash
.
./html/
./src/
./src/style.css
./templates/
./tmp/

All of your website source code lives within the ./src directory. This is where you structure your website however you want it to be structured in the final html.

You can write your pages / posts in markdown and lupo will convert them when building.

When building it into the final html, lupo will copy the structure of your ./src directory into your ./html directory, converting any markdown files (any files ending in .md) into html files.

Any JavaScript or CSS files are left alone and copied over in the same directory relative to the ./html root.

Starting a lupo website

Create a directory that you want to be your website project, and initialize it as a Lupo project:

Bash
mkdir ./my website
cd ./my-website
lupo init

The init command will create the required directories, including a file located at $HOME/.config/lupo/config.

You don’t need to worry about the config file just yet.

Create your homepage file and add some text to it:

Bash
touch ./src/index.md
echo "Hello World" > ./src/index.md

Now just run the build command to generate the final html:

Bash
lupo build

You should now have two files in your ./html directory: an index.html file and a style.css file.

The index.html was converted from your ./src/index.md file and moved into the root of the ./html directory. The style.css file was copied over verbatim to the html directory.

Viewing your site locally

Lupo doesn’t currently have a way to launch a local webserver, but you could open a browser and point the address bar to the root of your project ./html folder.

I use an nginx docker image to preview my site locally, and will build in this functionality into lupo soon.

Page metadata

Each markdown page that you create, can have an option metadata section at the top of the page. This is known as “frontmatter”. Here is an example you could add to the top of your ./src/index.md file:

Markdown
---
title: My Super Homepage
---

Here is the normal page content

That will set the page’s title to “My Super Homepage”. This will also make the %title% variable available in your template files. (More on templates further down the page)

If you re-run the lupo build command, and look again at your homepage, you should now see an <h1> tag withyou title inside.

The Index page

You can generate an index of all of your pages with the index command:

Bash
lupo index

lupo build

Once you’ve built the website after running index, you will see a file at ./html/index/index.html. This is a simple index / archive of all of the pages on your website.

For pages with a title set in their metadata block, that title will be used in the index listing. For any pages without a title set, the uri to the page will be used instead.

@todo ADD SEARCH to source and add to docs here.

Tag index pages

Within your page metadata block, you can also define a list of “tags” like so:

Markdown
---
title: My Super Page
tags:
    - tagone
    - tagtwo
    - anotherone
---

The page content.

When you run the lupo index command, it will also go through all of your pages and use the tags to generate “tag index pages”.

These are located at the following location/uri: ./html/tags/tagname/index.html.

These tag index pages will list all pages that contain that index’s tag.

Customizing your website

Lupo is very basic and doesn’t offer that much in the way of customization. And that is intentional – I built it as a simple tool for me and just wanted to share it with anyone else that may be interested.

That being said, there are currently two template files within the ./templates directory:

Bash
./templates/default.template.html
./templates/tags.template.html

tags.template.html is used when generating the “tag index” pages and the main “index” page.

default.template.html is used for all other pages.

I am planning to add some flexibility to this in the near future and will update this page when added.

You are free to customize the templates as you want. And of course you can go wild with your CSS.

I’m also considering adding an opt-in css compile step to enable the use of something like sass.

New post helper

To help with the boilerplate of add a new “post”, I add the following command:

Bash
lupo post

When ran, it will ask you for a title. Once answered, it will generate the post src file and pre-fill the metadata block with that title and the current date and timestamp.

The post will be created at the following location:

Bash
./src/{year}/{month}/{date}/{timestamp}/{url-friendly-title}

# For example:
./src/2023/08/30/1693385086/lupo-static-site-generator/index.html

Page edit helper

At present, this requires you to have fzf installed. I am looking to try and replace that dependancy with the find command.

To help find a page you want to edit, you can run the following command:

Bash
lupo edit

This will open up a fuzzy search finder where you can type to search for the page you want to edit.

The results will narrow down as you type.

When you press enter, it will attmept to open that source page in your system’s default editor. Defined in your $EDITOR environment variable.

Automatic rebuild on save

This requires you to have inotifywait installed.

Sometimes you will be working on a longer-form page or post, and want to refresh the browser to see your changes as you write it.

It quickly becomes tedious to have to keep running lupo build to see those changes.

So running the following command will “watch” you ./src directory for any changes, and rebuild any file that is altered in any way. It will only rebuild that single file; not the entire project.

Deploying to a server

This requires you to have rsync installed.

This assumes that you have a server setup and ready to host a static html website.

I covered how I set up my own server in This Terraform post and This Ansible post.

All that lupo needs to be able to deploy your site, is for you to add the required settings in your config file at $HOME/.config/lupo/config

  • remote_user – This is the user who owns the directory where the html files will be sent to.
  • ssh_identity_key – This is the path to the private key file on your computer that pairs with the public key on your remote server.
  • domain_name – The domain name pointing to your server.
  • remote_directory – The full path to the directory where your html files are served from on your server.

For example:

Bash
remote_user: david
ssh_identity_key: ~/.ssh/id_rsa
domain_name: example.com
remote_directory: /var/www/example.com

Then run the following command:

Bash
lupo push

With any luck you should see the feedback for the files pushed to your remote server.

Assuming you have set up you domain name to point to your server correctly, you should be able to visit you website in a browser and see your newly-deployed website.

Going live

This is an experimental feature

If you’ve got the lupo watch and lupo push commands working, then the live command should also work:

Bash
lupo live

This will watch your project for changes, and recompile each updated page and push it to your server as it is saved.

The feedback is a bit verbose currently and the logic needs making a bit smarter. But it does currently work in its initial form.