Category: Docker

[Docker] Autobuild Docker Images from Github Repos you don't own

With Docker Hub it is dead simple to just link your Github Account to Docker, choose an Github Repo which contains your Dockerfiles and Apps - and build them as soon as you push new code. But as soon as you're not the owner of the repo (i.e. you're a fan and want to provide Dockerbuilds for Docker Hub) - things get messy. But actually, it works quite easy: Just apply some IFTTT - and you're set ;).

Ok, a little bit more detail: If this then that is a nice Website which lets you create some small but powerful Applets. I.e.: If the weather is cloudy, sent me an email. Things like that. Just sign up on https://ifttt.com, its free! 🙂

After that, just go over to https://hub.docker.com. Link your Github Account with Docker Hub and create and Auto Build Repo on Github with the Dockerfile of the project you want to build. In my case, the program I'd like is developed on Github itself, so my Dockerfile contains some instructions to just check out that repo, build the program and.. done :)! After this is working, go to the "Build Settings" Dialog of your Docker Hub Repo and go to the "Build Triggers" area. Activate the Build Triggers and Copy Down the Trigger URL. Every time the URL is used, your Repo is going to be build a new :).

The only thing we now need, is to know when a new commit is made. And Github makes that very easy: Go to your wanted Github Repo (I will use pklaus awesome brother_ql as example: https://github.com/pklaus/brother_ql), click on commits in the branch you like to use (i.e. master in my case) - and you will be redirected to the page showing you all commits: https://github.com/pklaus/brother_ql/commits/master. Just add an .atom after that URL, and voilá: You got yourself an RSS Feed of that commit stream ;)!. ( https://github.com/pklaus/brother_ql/commits/master.atom )

Copy that link as well, we're going to combine both links via the "magic" of IFTTT ;): Click on "my Applets", "New Applet" and on the "+ this" part. Select the Feed Service, and "New Feed item". Paste the collected Atom/RSS Feed URL in the Feed URL field and click "Create Trigger" (https://github.com/pklaus/brother_ql/commits/master.atom).

After that, click on the "+ that" part and search for "Maker" - and choose it as action service - the action should be "Make a web request". Now enter the Build trigger URL from Docker Hub you noted down, choose the "POST" Method, "application/json" as Content Type and enter "build": true as Body. After that, click save.

Activate your newly made Applet - and you're done: As soon as someone pushes a new commit, IFTTT will see that and start an rebuild of your Docker Hub Repo :).

(Yeah, that is a really hacked-together part, but it works ;))

[Docker] Backup / Restore Images

Images should be a kind of "throw-away" format: You got your Dockerfile, and as soon as you need to run a specific container, you should just built it from scratch: All the latest security updates, software versions - great. However, sometimes you got a bit of specialized stuff which you want to backup for later use.

I found a good answer to that topic: http://stackoverflow.com/questions/26707542/how-to-backup-restore-docker-image-for-deployment


Backup:
docker save myusername/myproject:latest | gzip -c > myproject_img_bak20141103.tgz

Restore:
gunzip -c myproject_img_bak20141103.tgz | docker load

[Sonarqube] Use Sonarqube in IntelliJ / PhpStorm / etc.

Sonarqube is cool, as the generated results can not only be viewed via the web interface, but also via an plugin in IntelliJ or other Jetbrains IDEs. The installation is quite simple, you just need to download the latest plugin on this website: https://plugins.jetbrains.com/plugin/7238 and install it in the way described here: https://github.com/sonar-intellij-plugin/sonar-intellij-plugin. After that, you can easily hunt down and eliminate those errors :)!

[Docker] Sonarqube with Gogs/Drone to check software quality

Sonarqube is an really useful tool to check for different kind of errors in all kinds of programming languages. If you want to check out Sonarqube in a livedemo, just go over to https://sonarqube.com/. In the last two parts of my Docker series we created an gogs Git Repo as docker container and then went over to integrating gogs with drone (Version 0.4), an CI Build Server - also as Docker container. Today, we want to install Sonarqube as a tool to check upon our software quality, so that - as soon as we push an update to our gogs project, drone will execute an analysis using Sonarqube - which will give us information about different kind of errors and code quality.

# We will start by creating the needed folders:
sudo mkdir /var/sonarqube
sudo mkdir /var/sonarqube/data
sudo mkdir /var/sonarqube/extensions
sudo chown -R yourusername:yourusername /var/sonarqube
# Then we change our drone docker-compose.yml, so that drone and sonarqube will be started at the same time

sonarqube:
  restart: unless-stopped
  image: sonarqube:lts-alpine
  volumes:
    - /var/sonarqube/data:/opt/sonarqube/data
    - /var/sonarqube/extensions:/opt/sonarqube/extensions
  environment:
    - SONARQUBE_HOME=/opt/sonarqube
    - SONARQUBE_JDBC_USERNAME=sonar
    - SONARQUBE_JDBC_PASSWORD=sonar
    - SONARQUBE_JDBC_URL=
  ports:
    - "9000:9000"
    - "9092:9092"
drone:
  restart: unless-stopped
  image: drone/drone:0.4.2
  volumes:
    - /var/drone:/var/lib/drone
    - /var/run/docker.sock:/var/run/docker.sock
  env_file:
    - ./dronerc
  ports:
    - "8000:8000"
  links:
    - sonarqube

After that, we can start the service with docker-compose up -d and see Sonarqube on http://IPADDRESS:9000 (needs some time to load...).
To do an check of i.e. an Java Project, we need to write an new pom.xml file:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example.appexample</groupId>
  <artifactId>appexample</artifactId>
  <version>1.0</version>

  <name>phpTest</name>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <sonar.language>java</sonar.language>
    <sonar.sources>src</sonar.sources>
    <sonar.exclusions>src/test/test.php, src/test/test/*</sonar.exclusions>
  </properties>

</project>

and we need a new .drone.yml

cache:
  mount:
    - /drone/.m2
build:
  main:
    image: maven:3-jdk-8-onbuild
    commands:
      - mvn sonar:sonar -Dsonar.host.url=http://IPOFTHESONARQUBESERVER:9000 -Dmaven.repo.local=/drone/.m2
      - echo "Sonarqube has been completed."
      - mvn clean install -Pcoverage -Dmaven.repo.local=/drone/.m2
      - mvn package -Dmaven.repo.local=/drone/.m2
      - mvn test -Dmaven.repo.local=/drone/.m2
      - echo "Build has been completed."
debug: true

And thats it :). Login into drone, activate your repo as valid CI repo and after that - with every push to that gogs repo, a new sonarqube analysis should be performed.

[Docker] CI with Drone and Gogs

Now that we have a running Gogs installation for our source code (see here) - we can use Drone to build working software from those repos. Drone is an Continious Integration System, that comes in two flavors: Drone.io - which is the hosted service you could use - or Drone - as self-hosted service. We want to use the later one. To install drone in our existing Docker setup (with gogs already installed) we need to complete following steps:

# Create docker-compose.yml in ~/drone/
cd ~
mkdir drone
cd drone
vi docker-compose.yml
# Copy this content into your docker-compose.yml file

drone:
  restart: unless-stopped
  image: drone/drone:0.4.2
  volumes:
    - /var/drone:/var/lib/drone
    - /var/run/docker.sock:/var/run/docker.sock
  env_file:
    - ./dronerc
  ports:
    - "8000:8000"

# Create the dronerc file
vi dronerc
# Copy this content into your dronerc file
# replace yourserverip with the ip or dns name of your server, i.e. example.com
# replace the gogsport with the port of the http port of the gogs installation, i.e. was 3000 in our example

REMOTE_DRIVER=gogs
REMOTE_CONFIG=https://yourserverip:gogsport?open=false
DEBUG=true

# Create the needed folders
sudo mkdir /var/drone
sudo chown -R yourusername:yourusername /var/drone/

After that, an docker-compose up will start the service, you can end it via STRG + C and really start it in deattached mode with docker-compose up -d
You can then go to http://yourserverip:8000 and log in into drone with your gogs login and allow access.

The current readme for drone can be found on http://readme.drone.io/usage/overview/ and you'll need to include a .drone.yml in your repos to really build something.

In my example, I used this .drone.yml

cache:
  mount:
    - /drone/.m2

build:
  main:
    image: maven:3-jdk-8-onbuild
    commands:
      - mvn clean install -Pcoverage -Dmaven.repo.local=/drone/.m2
      - mvn package -Dmaven.repo.local=/drone/.m2
      - mvn test -Dmaven.repo.local=/drone/.m2
      - echo "Build has been completed."

debug: true

and this pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.mycompany.app</groupId> 
  <artifactId>WhereIsMyPi</artifactId>
  <version>1.0</version>
  <packaging>jar</packaging>
 
  <name>Where is my Pi</name>
  <url>http://www.nico-maas.de</url>

  <build>
    <sourceDirectory>src</sourceDirectory>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.5</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>
      <plugin>
       <!-- Build an executable JAR -->
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-jar-plugin</artifactId>
       <version>2.4</version>
       <configuration>
        <archive>
          <manifest>
            <addClasspath>true</addClasspath>
            <classpathPrefix>lib/</classpathPrefix>
            <mainClass>WMP</mainClass>
          </manifest>
        </archive>
       </configuration>
      </plugin>
    </plugins>
  </build> 

</project>

to build my "WhereIsMyPi" project :).
Happy building!

[Docker] Keep Docker Container up-to-date with Watchtower

If you're using Docker, you know you will need to update these containers from time to time by hand. Mostly with an docker pull repo/DockerContainerName and an docker-compose up -d. If you want to automate this, you can now use Watchtower: https://github.com/CenturyLinkLabs/watchtower

Using it, is very easy. Just run following command to let all your containers automatically update via watchtower:

docker run -d \
--name watchtower \
-v /var/run/docker.sock:/var/run/docker.sock \
centurylink/watchtower

If you want choosen containers to be updated, include the names of the running containers as arguments, i.e.:


docker run -d \
--name watchtower \
-v /var/run/docker.sock:/var/run/docker.sock \
centurylink/watchtower gogs_gogs_1 drone_drone_1

This Info was brought to you by Christopher Perrin @ https://blog.screebo.net/ ;).

[Docker] Install Gogs as Docker container

Gogs (Go Git Service) is an awesome Github/Gitlab like solution, completly written in Go(lang) - which makes it blazing fast - and lightweight.
And as there is even an Docker Container of Gogs available - I thought - why not using this to finally move from my SSH-only-Git to an "real" Gitservice :).

We are going to use docker-compose in this example - so I assume you have installed this as shown in my last guide on Docker.

# Create docker-compose.yml in ~/gogs/
cd ~
mkdir gogs
cd gogs
vi docker-compose.yml
# Copy this content into your docker-compose.yml file

gogs:
  restart: unless-stopped
  image: gogs/gogs
  volumes:
    - /var/gogs:/data
  ports:
    - "10022:22"
    - "3000:3000"

After that save the file and issue docker-compose up in your terminal. Docker will start to pull the gogs image from the hub and launching gogs. All your gogs files will be saved on your local drive in /var/gogs. You can find the overview of the file structure here.
After Docker is ready - launching your favorite browser and go to http://127.0.0.1:3000.

Now its time to configure gogs. Please bear in mind the important information of the Gogs Guide regarding Settings in a Docker installation.

Regarding this, we will use following settings:
As Database type, we choose SQLite3. As path, already data/gogs.db is choosen, which is important: A Docker Container is non-persistent - so if you restart that container, all files not saved in a mounted directory (like your /var/gogs, which is mounted as data in the docker session of gogs...) will be lost.
As name, we choose something catchy like PiGit or so - as you wish.
We won't touch the repo path (/data/git/gogs-repositories), nor the user (git). As Domain, we could choose our http://mydnshost.com.
As SSH Port, we choose 10022. HTTP Port remains as 3000 and the application domain should be something like http://mydnshost.com:3000 - while 3000 is the exposed HTTP port - so someone can actually access your Gogs service.

You can also configure your mail server - as you wish.

Regarding the server and additional settings, I did choose the Offline Mode, but enabled Gravatar, disabled User Registration and Captchas and I did enable "Need to be registred to see contents" - as I want my Gogs Server to be actually reachable from the net - but I only want to create user accounts by hand - and not have my server filed with stuff of foreign people.

The last step is to create an Admin User Account. I would recommend to do this "now". After that, click "Install Gogs". And then - you can login :)!

The cool thing about gogs: You can even migrate from other Git Services to your new Gogs Server via the "Migration" Tab (+ next to your User Icon after you have logged in). Please bear in mind, that only http/https and local paths do work for that.

If you create a new repo, you should always check the "Init the repo", so that you can directly clone and use it.

Regardings cloning: You can access your repo via the webpage (ip:3000) or ssh (ip:10022). To use ssh, you need to insert your public key in the "SSH Key" tab in your gogs settings.

If you expose the 3000 and 10022 ports via your Firewall/Router, you can access gogs from everywhere - or you just use VPN to get into your network.

Bonus: Making Gogs Secure with Letsencrypt
If you already have an Letsenrcypt certificate for your server / pc, you can easily get gogs to use that: Just go to /var/gogs/gogs/data and copy your fullchain.pem and privkey.pem from your letsencrypt folder ( /etc/letsencrypt/live/[yourdomain]/ ) and give your user access to it via chown.
After that, go to /var/gogs/gogs/conf open app.ini and add following settings under [server]:
PROTOCOL = https
CERT_FILE = data/fullchain.pem
KEY_FILE = data/privkey.pem

If there should be another entry like PROTOCOL = http, just delete it. Save that file, go back to your open docker terminal with gogs running, CTRL + C and enter docker-compose up -d. With that, it will restart in detached mode. And more important: Your service will automatically start on every reboot of your system.

If you ever would need to stop gogs, just go again to your docker-compose file, i.e. cd ~/gogs/ and enter docker-compose stop.
You can also watch what your docker container is doing with the command docker ps

Happy coding!

[Docker] OpenWRT Images for x86, x64, Raspberry Pi and Raspberry Pi 2

As some of you know, I am trying to learn to use Docker.
I love the simplicity of this tool and the fact that a lot of my Appliances could be built and mainted more efficiently with the use of it.
So I thought "Well, I should at least try to create some useful Images for the Docker Registry / Hub" - and so I came up with the Github Repo of x-drum which I could not help - but fork ;). So, x-drum showed an easy way to build x86 Images for 14.07 and 15.05 OpenWRT.
And I thought "well, lets extend that". So now, we also got 12.09 OpenWRT x86, as well as trunk x86... and while I was doing some research, I slapped the x64 Versions for trunk and 15.05 on as well :).

But wait - somethings missing - yeah: We need some Raspberry Pi Stuff ;):
The guys over at Hypriot did an really awesome job with creating Hypriot OS - basically an bootable Image for RPi1/2 to use a recent Version of Docker :). But - truth been told - they already created some special RPi (ARM) Images on the Docker Hub - but... it would be nice to get some more base images to play with...

So I basically used Hypriot OS on a RPi 1 to create the OpenWRT 12.09, 14.07, 15.05 and Trunk Docker Images - and on a RPi 2 to create OpenWRT 15.05 and Trunk Docker Images.
The RPi 1 Images are also usable on a RPi 2 - so I recommend to use them. RPi 2 Images are only usable on a RPi 2 :).

Everything can be grabbed from my Github Repo: https://github.com/nmaas87/docker-openwrt or directly on Docker Hub.
The x86/x64 Images can be found on https://hub.docker.com/r/nmaas87/docker-openwrt/, while the RPi 1 and RPi 2 Images are here: https://hub.docker.com/r/nmaas87/rpi-openwrt/.

Have fun :)!