Useful Docker oneliners

Display image virtual size in human-readable format:

sudo docker inspect <image> | grep -i VirtualSize | awk '{$2/=1000000; printf "%.2f MB\n",$2}'

Checking what command has produced specific Docker image layer:

docker inspect --format '{{ ((index .ContainerConfig.Cmd ) 0) }}' <layer_id>

Useful Docker-related links:

https://imagelayers.io/
https://github.com/larsks/dockerize

Show amount of open files per process

This one also displays process name, pid and number of open files:
lsof | awk '{ print $2 " " $1; }' | sort -rn | uniq -c | sort -rn | head -20

Backup Jenkins to S3

This way we will backup all except for SCM folders and jars. All Jenkins configs, jobs, logs and builds will be preserved, only built artifacts will be skipped

Backup command:
export GZIP=-9 && tar czfh - /var/lib/jenkins/ --exclude-vcs --exclude "archive" --exclude "target" --exclude "/var/lib/jenkins/docker" --exclude "/var/lib/jenkins/.m2/repository" | /usr/local/bin/aws s3 --region us-west-2 cp - s3://<bucket_name>/jenkins-backup.tar

 Restore command:
aws s3 cp s3://confyrm-jenkins-backups/jenkins-backup.tar - | tar -C / -zxf -

Example Logstash config to parse Java / Scala multiline logs (e.g. stacktraces) into ES

Java/Scala stack traces are multiline and usually it have the message starting from

Any line which isn't starting with '[' will be joined into previous one having '[' at the beginning

E.g. this works with Logstash 1.4.0+:

if [type] == "app_logs" {
    multiline {
      pattern => "^[^\[]"
      what => "previous"
    }
    grok {
      match => { "message" => "\[(?<app_log_timestamp>.+)] \[%{WORD:app_name}\] \[(?<thread_name>.+)\] \[(?<class_name>.+)\] \[(?<marker>[a-zA-Z]*)\] \[(?<transaction_id>.*)\] \[%{WORD:log_level}\]: ?%{GREEDYDATA:msg}" }
    }
    date {
      match => ["app_log_timestamp", "MM/dd HH:mm:ss:SSS", "ISO8601"]
      target => "@timestamp"
      add_tag => [ "timestamp_updated_w_log_value" ]
      remove_field => [ "app_log_timestamp" ]
    }
}

This works for all Java multiline logs, the only rule is to not start multiline log newlines from '['.

Useful link: http://logstash.net/docs/1.4.0.rc1/filters/multiline

Also starting from Logstash 1.2 there is a 'multiline' codec (http://logstash.net/docs/1.2.2/codecs/multiline). But I didn't get it work properly with Logstash 1.4. What have I tried:

input {
  file {
      codec => multiline {
        pattern => "^\s"
        what => "previous"
      }
    ..... file path and so on
  }
}

What issue did I met with 'multiline' codec: Java stacktraces were parsed without the very first line. E.g. in ES I was getting:

java.lang.RuntimeException: Exception while executing statement : An I/O error occurred while sending to the backend. errorCode: 0, sqlState: 08006 at ... [other stacktrace lines omitted]

Instead of expected:

[06/24 16:43:51:393] [app_name] [pool-99-thread-999] [ClassName] [smth0] [bar] [ERROR]: Cannot load XXX java.lang.RuntimeException: Exception while executing statement : An I/O error occurred while sending to the backend. errorCode: 0, sqlState: 08006 at ... [other stacktrace lines omitted]

Script for Jenkins 'Total build status' job

Script for Jenkins 'Total build status' job which displays all failed jobs with last changes, committers and links to job's console. It will fail if at least one other Jenkins job is failed.

Usage: 
- Create Jenkins Freestyle job
- Add this script as "Execute system Groovy script" build step 

Docker: check the memory used by docker containers at specific host

for line in `docker ps | awk '{print $1}' | grep -v CONTAINER`; do echo "" && docker ps | grep $line | awk '{printf $NF" "}' && echo -e "\nmemory usage: $(( `cat /sys/fs/cgroup/memory/docker/$line*/memory.usage_in_bytes` / 1024 / 1024 ))MB" && echo max memory usage: $(( `cat /sys/fs/cgroup/memory/docker/$line*/memory.max_usage_in_bytes` / 1024 / 1024 ))MB ; done

How to 'tail -f ' Docker logs

sudo docker logs -tf --tail 0 <container_name_or_id>