Log Rotation in a Nutshell

The basic idea of log rotation is explained in the following five steps.

  1. You have a program that sends its logs to /var/log/foo.log.
  2. It’s starting to grow to an uncomfortable size.
  3. When it gets too big you rotate it by renaming the file something like /var/log/foo.log.1.
  4. You create a new file called /var/log/foo.log (where your program sends its log to).
  5. You can decide what you want to do with /var/log/foo.log.1. Maybe you compress it or copy it to another server and then delete it.

Many modern Linux distributions the logrotate utility to manage log rotation. By default, it is configured to run daily using cron.

$ cat /etc/logrotate.conf
# see "man logrotate" for details

include /etc/logrotate.d

man logrotate has detailed explanations of the configuration options. It’s suggested that you make a config file in /etc/logrotate.d for each log you want to be managed. Here is an example.

/var/log/myapp {
    rotate 24

This configuration will watch /var/log/myapp, rotate the log once a month, compress the rotated log, and create a new empty log after rotation.

That’s pretty much all there is to it.

Chrome Shortcuts

Here are some helpful shortcuts for getting around the Web in Google Chrome.

Cmd+lSelect the address bar
Ctl+eTake the cursor to the end of a line
Ctl+aTake the cursor to the beginning of a line
Alt+Shift+ArrowSelect full words in the address bar
Alt+Cmd+ArrowMove to the previous or next tab
Cmd+wClose a tab
Cmd+Shift+tBring back a tab you accidentally closed

docker run -it: TTYs and Stdin

I was working on a project that involved running a docker command as a systemd service. The docker command looked something like the following.

docker run -it \
  foobar python3 -c 'import this; print(this)'

When I tried to start the systemd service I saw the following error.

Jan 27 18:37:37 host program[123]: the input device is not a TTY

After some research, I found other people who had the same problem. Their solution was to remove the -t option from the docker command. I tried it, and it worked. I submitted a pull request and my teammate asked what the difference between the two options was. I was a little embarrassed that I couldn’t explain it to him.

The documentation says

-i, –interactive Keep STDIN open even if not attached

docker run –help


-t, –tty Allocate a pseudo-TTY

docker run –help

I learn best by doing. So I created a simple Dockerfile to test out these different options.

FROM python:3-slim
CMD python

Then I built the image.

docker build -t dummy .

I ran the container in four different ways.

$ docker run dummy
$ docker run -i dummy
# .....waits for me to type something
# pressed Ctrl-D to signal End of Transmission
$ docker run -t dummy
Python 3.8.1 (default, Jan  3 2020, 22:55:55)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> ^C^Fexit
# ... help! I cannot escape
# because a psuedo-terminal has not been allocated
$ docker run -it dummy
Python 3.8.1 (default, Jan  3 2020, 22:55:55)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
# I pressed Ctrl-C to kill the process


-i by itself tells docker to connect the terminal stdin to the container’s stdin.

$ docker run -i alpine sh
ls /var

Notice there is no shell prompt. There is no pseudo-terminal, the alpine container is just using the stdin of its parent process. This is not user-friendly for interactive shell sessions inside containers.

-t tells the docker process that input is coming not only from stdin, but specifically a terminal device (aka your shell session), so it allocates a pseudo-terminal for the container.

$ docker run -it alpine sh
/ # ls /var
cache  empty  lib    local  lock   log    mail   opt    run    spool  tmp
/ #

If you remove -t from the docker run command, you are telling docker that you don’t need a pseudo-terminal allocated for the container. You will still be able to type commands into it (for example, Ctrl-C will be read by the container if you want to kill it), but you won’t get a shell prompt.

This is a good post that helped me understand what’s going on.

Glossary: Provision

To supply with food, drink, or equipment, especially for a journey.

Provision (verb)

Provision comes from the word “provide”, which means to attend to.

I remember being re-introduced to the word provision when I started using Vagrant. There is an option you can use when you start a virtual machine, --provision, that will automatically install software and alter configurations according to what is written in your Vagrantfile.

For example, this Vagrant file:

Vagrant.configure("2") do |config|
  config.vm.provision "shell",
    inline: "sudo apt-get install -y nginx"

Uses config.vm.provision "shell" to run a shell command that installs Nginx before the virtual machine begins its journey.

You could say, I am provisioning my webserver with Nginx ("sudo apt-get install -y nginx").

You take a bare Ubuntu virtual machine and provision it with Nginx and some configuration changes in order for it to be a functioning web server.

An important thing to understand is that the provisioning process should be automated. As a DevOps engineer, you probably don’t want to manually do sudo apt-get install -y nginx and write the configuration files each time you create a virtual machine.

It is better to automate that process by using a tool such as Vagrant, Ansible, or at least a script. You can write your Vagrantfile, Ansible playbook, or script in a repository so it is version-controlled and documented, and you can use Ansible, for example, to run the provisioning script across multiple machines at the same time.

Like the original quote at the top of this post says, provisioning is supplying a machine with software and configuration settings it needs in order to perform its function.