As I have a static blog page build on Hugo I wanted to leverage Twitter to allow discussions. The challenge here was that to embed a tweet which is relevant I would always have to create a tweet manually and then embed it later on the blog. This sounded to cumbersome for me so I though about how I could automate it.

I use github to store & backup my page. Therefore I thought my workflow ideally looked somewhat like this:

Every time I push to master or a specific branch a workflow should be trigger which does the following
- validate if new posts are there
  - if new posts then create tweet per post and adapt file
- execute hugo and deploy to firebase
  - hugo embeds the new tweet in the discuss section on the post
  - firebase deploy is executed with newly generated files

Before I started I checked several possibilities on how to achieve this. After looking into several options like google’s cloud build or Github workflows I decided to go for Github workflows. They are directly available on Github.org and seemed to be the easiest way to set up my required workflow.

Below an explanation how I did it including descriptions of the individual used components.

Github

Github allows a certain degree of automation. To achieve this you can configure workflows. Workflows consist of jobs. Jobs contain of steps. Each step can either be a command (echo, run a script) or trigger an action.

A workflow is triggered by events. An event could be a push, merge, and so on. More details can be taken from the Github events page.

To define a workflow you need to create a new folder in your repo, .github/workflows. In this folder you then create .yml or .yaml files to define the worklow and steps.

Hello Wolrd example:

name: Greet Everyone
# This workflow is triggered on pushes to the repository.
on: [push]

jobs:
  build:
    # Job name is Greeting
    name: Greeting
    #event -> filtering on branch
    on:
      push:
        - master
    # file paths to consider in the event. Optional; defaults to all.
    paths:
      - 'test/*'
    # This job runs on Linux
    runs-on: ubuntu-latest
    steps:
      # This step uses GitHub's hello-world-javascript-action: https://github.com/actions/hello-world-javascript-action
      - name: Hello world
        uses: actions/hello-world-javascript-action@v1
        with:
          who-to-greet: 'Mona the Octocat'
        id: hello
      # This step prints an output (time) from the previous step's action.
      - name: Echo the greeting's time
        run: echo 'The time was ${{ steps.hello.outputs.time }}.'

To execute worklow steps you need to choose a so called runner. There are default runners available on Git or you can host your own runner (remotely). A runner is basically a VM, probably containerized, which allows you to call scripts and execute shell commands. Each virtual machine has the same hardware resources available.

2-core CPU
7 GB of RAM memory
14 GB of SSD disk space

The VMs have 3 very important directories:

home => Environment Variable: HOME Contains user-related data. For example, this directory could contain credentials from a login attempt.
workspace => Environment Variable: GITHUB_WORKSPACE Actions and shell commands execute in this directory. An action can modify the contents of this directory, which subsequent actions can access.
workflow/event.json => Environment Variable: GITHUB_EVENT_PATH The POST payload of the webhook event that triggered the workflow. GitHub rewrites this each time an action executes to isolate file content between actions.

The VMs are hosted on azure. More details here

Full details on Workflows are available on the Github docu page.

Building workflows in Github

In order to build your own workflow you need to identify actions and steps you want to execute.

You can create actions by writing custom code that interacts with your repository in any way you’d like.

You can build docker container based actions or JavaScript based actions. The benefit of docker is that you can customize the operating system and tools. However because of the latency to build and retrieve the container, a docker based action is slower than a JavaScript action.

JavaScript actions can run directly on a runner machine. This makes the implementation a little bit easier and straight forward in my opinion. For details on how to create javascript based actions have a look here.

Apart from this you can execute shell commands in the runner vm.

After looking into the options I decided to put together a mixture of pre-defined actions, some direct shell commands and to execute a python script I build for my purposes.

After reading through the documentation I went through the following steps to get to my plan:

Define the workflows and the jobs I want to execute within them For my use case I defined 2 workflows. One workflow is triggered if commits are pushed to a branch. The second workflow which builds my page is triggered on push to the master repo.
Define per workflow the jobs A job is a logical definition of actions or steps which are belonging together. In my first workflow I defined two jobs, namely scanFilesAndTweet and mergeUpdateToMasterCommit.
- scanFilesAndTweet - Is a job for scanning posts, creating tweets and updating the files
- mergeUpdateToMasterCommit - Is a job for running the hugo build and pushing the new generated build files to the master branch of the repo
The second workflow contains only one jobs responsible for deploying the newly created build on my firebase hosting.
Define the steps within each job In the end I ended up with many steps per job, therefore I will simply link to my workflow github entry. Have a look at the yaml files if you are interested. You can find the code here
Create workflow/.yaml I created two workflow files: createTweetsAndHugoBuild.yml and deployToFirebase.yml

Github Secrets

In order to run the actions/steps I required, I needed to add secrets or environment variables which allowed for example access to my deploy landscape. Therefore I decide to use github secrets to store the api credentials and keys. Those secrets are available during github workflow runs. Details can be found here

To create secrets for a repository select to the Settings tab of your repo. In the left sidebar click “Add a new secret” and enter the name and value of the secret. Then click add.

To access secrets use the syntax ${{ secrets.<SECRET_NAME> }}.

Creating my workflow - Step by step

First I created an empty git repo called https://github.com/jonny74889/gitHugoTwitterPush. Then I created the folders .github/workflows/.

For scanning the new files I created the script main.py in the workflows folder. (This could also reside somewhere else. You can pass the path during the job step description).

Furthermore I created the workflow yaml which describes the steps and jobs. The file has to be located in ‘workflows/'.

Here a snippet of the file content - createTweetsAndHugoBuild.yml:

name: Create Tweets update Posts and build Hugo
# This workflow is triggered on pushes to the repository.
on:
  push:
    branches:
      - 'update'

jobs:
  scanFilesAndTweet:
    # This job runs on Linux
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.5]
    steps:
    - uses: actions/checkout@v2
      with:
        persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
        ref: update
    - uses: actions/setup-python@v2 #load right python version
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install requirements
      run: pip install -r ./.github/workflows/requirements.txt
    - name: Run a one-line script
      run: python ./.github/workflows/main.py -c ${{secrets.CONSUMER_KEY}} -s ${{secrets.CONSUMER_SECRET}} -t ${{secrets.ACCESS_TOKEN}} -o ${{secrets.ACCESS_TOKEN_SECRET}} #important to call python from current folder, avoids issues with pathfinding

    #####run hugo build command ####
    - name: Hugo Action
      uses: srt32/hugo-action@master
      #args: <Hugo args> #optional - i do not need it just build

    ####commit changes to update ####
    - name: git config
      run: git config --local user.email "action@github.com"
    - name: git config
      run: git config --local user.name "GitHub Action"
    - name: git add --all
      run: git add --all
    - name: status
      run: git status
    - name: git commit
      run: git commit -m "Updated posts with Tweet handles" -a
    - name: Push changes
      uses: ad-m/github-push-action@master
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        branch: 'update'

  ####### Merge master & update and push to master
  mergeUpdateToMasterCommit:
      # This job runs on Linux
      runs-on: ubuntu-latest
      steps:
      - uses: actions/checkout@v2 #checkout master
        with:
          persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
          fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
          ref: master
      - name: git config
        run: git config --local user.email "action@github.com"
      - name: git config
        run: git config --local user.name "GitHub Action"
      - name: git merge origin/update
        run: git merge origin/update
      - name: status
        run: git status
      - name: Push changes
        uses: ad-m/github-push-action@master
        with:
          github_token: ${{ secrets.PERSONAL_TOKEN }}
          branch: 'master'

The second workflow file I created was deployToFirebase.yml. In this workflow I just call an existing action to deploy the new build to firebase. Here is the content of the file:

name: Deploy to firebase
# This workflow is triggered on pushes to the repository.
on:
  push:
    branches:
      - 'master'

jobs:
  hugoDeployToFirebase: #maybe just add this on master push
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2 #checkout master
      with:
        persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
        ref: master
    - name: Deploy to Firebase
      uses: w9jds/firebase-action@master
      with:
        args: deploy --only hosting
      env:
        FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
        PROJECT_ID: theprogress-83b4d

Reading new posts and creating tweets

Reading the new posts and creating tweets will be done via the above mentioned python script. Furthermore the script will create tweets and adapt the origin files in order to embed twitter comments.

The script as well as the required package requirements.txt are located in the path .github/workflows.

The script looks in the repo for the folder content/posts/. Content is a standard folder created by Hugo to handle content pages. Posts is a sub-section which I created. This subdirectory contains all my posts. I do not want to create tweets for other pages on my side.

Now in order to execute the script as a job you simply need to configure and describe the steps as you would do in your local environment in the workflow yaml.

Embedding tweets on your post page to allow discussions

Thanks to this post by Janne Kemppainen I got the idea and could re-use a lot.

I adopted my default archetype which generates the meta data every time a new post is created. The details can be taken from below.

In order to embed a tweet you need to adapt your single.html template. The file might be called differently depending on the Hugo template or your own implementation. In the end you need to modify the blog/post page where you want to show the tweet.

In my case I added the following code after the content section:

<!-- Add comment section for tweets -->
{{ if .Params.tweet }}
  <div class="container">
    <div class="section">
      <h2 class="title is-4">Discuss on Twitter</h2>
      {{ partial "widgets/twitter-embed.html" . }}
    </div>
  </div>
{{ end }}

Furthermore I created the twitter-embed.html. Here I had to adapt the code from Janne as I received an error when trying to retrieve the tweet.

My twitter-embed.html looks like this:

{{ with .Params.tweet }}

  {{- $url := printf "https://api.twitter.com/1.1/statuses/oembed.json?id=%v" . -}}
  {{- $json := getJSON $url -}}
  {{ $json.html | safeHTML }}

{{ end }}

Adapting the hugo archetypes to add tweet frontmatter

The python script needs to know what content to post to twitter, furthermore the hugo blog post requires a metadata / frontmatter entry which will contain the tweet ID. Otherwise Hugo will not be able to embed a twitter post.

Therefore I adapted the /archetypes/default.md with the following parameters (those parameters will be added if I create a new post via hugo new <filename>). If a file is manually created I need to add this section manually.

The content of my archetype file looks like this:

---
title: "{{ replace .Name "-" " " | title }}"
date: {{ .Date }}
featured_image: /images/...
tags: [""]
hidden: false
tweet: 1280919243928862721
twitterContentBegin: {"text": "{{ replace .Name "-" " " | title }}", "hashtags": ["#theprogress", "#number2"], "url": "https://theprogress.site/{{ .Name }}" }

---

Important here is the section tweet and twitterContentBegin as well as **. My python script is looking for this text information in order to generate the tweet.

For details on the script have a look at the blog entry here

After testing the script successfully and entering the steps as described above in the workflow it is now time to look into how to trigger the hugo build and deploy to firebase.

Using actions to run hugo and deploy to firebase

For running Hugo and Deploying to Firebase I checked if an action is already available. You can find existing actions here. For my case I identified two good examples. After thinking about creating my own docker image, I decided to first test the existing ones to save me this step.

The actions I used are:

Firebase CLI wrapper
- To use this action I created a Github secret with the name FIREBASE_TOKEN
- To get the token execute firebase login:ci
Hugo execution

Then I adapted the workflow files deployToFirebase.yml and createTweetsAndHugoBuild.yml to add the actions at right place in the logic.

Setting up my Github repo to handle workflow automation

I already briefly mentioned github secrets. In order to make my automation work and not paste secrets or passwords in the code I created 5 github secrets which are safely retrieved during the workflow execution. 4 secrets are related to the twitter API to verify my account and allow me to create Twitter tweets (Names: ACCESS_TOKEN, ACCESS_TOKEN_SECRET, CONSUMER_KEY, CONSUMER_SECRET). The last secret I added is called FIREBASE_TOKEN. This token is necessary to give the firebase cli authorization for my project.

Once the secrets were configured the only remaining step was to push the .github/ folder to the repository for which I wanted to enable the workflow. Once a workflow file is uploaded to Github, each activity will let Github check if the workflow needs to be executed. Before you test this with your production repo I really recommend to test it on a less important one.

Finally, it works

After several tests and bug fixes the workflow now finally works.

Every time I write a new blog post (like this one for instance), the metadata contains metadata for twitter. I usually adapt it to have the best possible text for my tweet related to the blog entry. Once I am happy with my new post, I commit it and push it to my remote branch called update on Github. This now triggers the automation/workflow. A tweet is generated for each new post (which has the right metadata). This tweet is then embedded on the post itself and as a last step the new version of my homepage is deployed to firestore.

Let me know what you think about this workflow. Let’s discuss it on twitter ;)

The Progress

How to use Twitter as comment section in your static blog and automatically create a tweet per post