Let's do it the hard way

02 Mar 2015 on AWS, Salt

jekyll

I'm playing a recital later this month and put together a quick site using the static site generator Jekyll. Aside from the killer performance and easy deployment and backup you get with a static site, the best part of going without a database is security. While escaping variables is always a nagging voice in the back of your head when coding a web app - it's really nice to not even have to think about it.

As usual I scripted out the deployment fully using salt, and added google pagespeed module to nginx to get great performance.

The sls file (assumes Debian):

locales:
  pkg:
    - installed

{% if grains['os'] == 'Debian' %}
locales-all:
  pkg:
    - installed
{% endif %}

git:
  pkg:
    - installed

/var/www:
  file.directory:
    - user: root
    - group: root
    - file_mode: 755
    - makedirs: True
    - recurse:
      - user
      - group

libsqlite3-dev:
  pkg:
    - installed

curl:
  pkg:
    - installed

unzip:
  pkg:
    - installed

build-essential:
  pkg:
    - installed

zlib1g-dev:
  pkg:
    - installed

libpcre3:
  pkg:
    - installed

libpcre3-dev:
  pkg:
    - installed

make:
  pkg:
    - installed

gcc:
  pkg:
    - installed

g++:
  pkg:
    - installed

nginx_init:
  file.managed:
    - name: /etc/init.d/nginx
    - source: salt://conf/nginx/nginx
    - user: root
    - group: root
    - mode: 755

nginx_update_rcd:
  cmd.run:
    - name: update-rc.d nginx defaults

nginx_with_ps:
  cmd.run:
    - name: bash /opt/stack/scripts/install/nginx.sh && touch /usr/src/nginxinstalled
    - unless: test -f /usr/src/nginxinstalled
    - creates: /usr/src/nginxinstalled
    - require:
      - pkg: build-essential
      - pkg: zlib1g-dev
      - pkg: libpcre3
      - pkg: libpcre3-dev
      - pkg: unzip
      - git: stack.git

/var/ngx_pagespeed_cache:
  file.directory:
    - user: nobody
    - group: nogroup
    - mode: 755
    - makedirs: True

nginx_conf:
  file.managed:
    - name: /usr/local/nginx/conf/nginx.conf
    - source: salt://conf/nginx/nginx.conf
    - require:
      - cmd: nginx_with_ps

nginx:
  service:
    - running
    - enable: True
    - watch:
      - file: nginx_conf
    - require:
      - file: nginx_conf
      - file: /var/ngx_pagespeed_cache
      # temporary
      - file: /var/www/recital

ruby1.9.1:
  pkg:
    - installed

ruby1.9.1-dev:
  pkg:
    - installed

rubygems:
  pkg:
    - installed

jekyll:
  gem:
    - installed
    - require:
      - pkg: rubygems
      - pkg: ruby1.9.1
      - pkg: ruby1.9.1-dev

/var/www/recital:
  file.directory:
    - user: root
    - group: www-data
    - dir_mode: 755
    - file_mode: 644
    - makedirs: True
    - recurse:
      - user
      - group
      - mode

recital.git:
 git.latest:
    - name: https://www.emikek.com/cloud/recital.git
    - rev: master
    - target: /opt/recital
    - require:
      - pkg: git

/usr/local/bin/jekyll build -d /var/www/recital:
  cmd.run:
    - cwd: /opt/recital
    - require:
      - gem: jekyll
      - file: /var/www/recital
      - git: recital.git

The nginx sysv init file:

#!/bin/sh

### BEGIN INIT INFO
# Provides:          nginx
# Required-Start:    $local_fs $remote_fs $network $syslog
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the nginx web server
# Description:       starts nginx using start-stop-daemon
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/nginx/sbin/nginx
NAME=nginx
DESC=nginx
ULIMIT="-n 4096"

test -x $DAEMON || exit 0

set -e

. /lib/lsb/init-functions

test_nginx_config() {
        if $DAEMON -t $DAEMON_OPTS >/dev/null 2>&1; then
                return 0
        else
                $DAEMON -t $DAEMON_OPTS
                return $?
        fi
}

case "$1" in
        start)
                echo -n "Starting $DESC: "
                test_nginx_config
                # Check if the ULIMIT is set in /etc/default/nginx
                if [ -n "$ULIMIT" ]; then
                        # Set the ulimits
                        ulimit $ULIMIT
                fi
                start-stop-daemon --start --quiet --pidfile /var/run/$NAME.pid \
                    --exec $DAEMON -- $DAEMON_OPTS || true
                echo "$NAME."
                ;;

        stop)
                echo -n "Stopping $DESC: "
                start-stop-daemon --stop --quiet --pidfile /var/run/$NAME.pid \
                    --exec $DAEMON || true
                echo "$NAME."
                ;;

        restart|force-reload)
                echo -n "Restarting $DESC: "
                start-stop-daemon --stop --quiet --pidfile \
                    /var/run/$NAME.pid --exec $DAEMON || true
                sleep 1
                test_nginx_config
                # Check if the ULIMIT is set in /etc/default/nginx
                if [ -n "$ULIMIT" ]; then
                        # Set the ulimits
                        ulimit $ULIMIT
                fi
                start-stop-daemon --start --quiet --pidfile \
                    /var/run/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS || true
                echo "$NAME."
                ;;

        reload)
                echo -n "Reloading $DESC configuration: "
                test_nginx_config
                start-stop-daemon --stop --signal HUP --quiet --pidfile /var/run/$NAME.pid \
                    --exec $DAEMON || true
                echo "$NAME."
                ;;

        configtest|testconfig)
                echo -n "Testing $DESC configuration: "
                if test_nginx_config; then
                        echo "$NAME."
                else
                        exit $?
                fi
                ;;

        status)
                status_of_proc -p /var/run/$NAME.pid "$DAEMON" nginx && exit 0 || exit $?
                ;;
        *)
                echo "Usage: $NAME {start|stop|restart|reload|force-reload|status|configtest}" >&2
                exit 1
                ;;
esac

exit 0

The nginx.sh install script:

pushd /usr/src

wget http://nginx.org/download/nginx-1.7.10.tar.gz
mv nginx-1.7.10.tar.gz nginx.tar.gz

NPS_VERSION=1.9.32.3
wget https://github.com/pagespeed/ngx_pagespeed/archive/release-${NPS_VERSION}-beta.zip
mv release-${NPS_VERSION}-beta.zip pagespeed.zip



unzip pagespeed.zip
tar -xvzf nginx.tar.gz
mv ngx_pagespeed-release* ngx_pagespeed
cd ngx_pagespeed
wget https://dl.google.com/dl/page-speed/psol/1.9.32.3.tar.gz
tar -xzvf 1.9.32.3.tar.gz # expands to psol/
cd ..

mv nginx-* nginx
cd nginx
./configure --add-module=/usr/src/ngx_pagespeed --with-http_realip_module
make
make install

popd

And finally, nginx.conf:

#user  nobody;
worker_processes  1;

pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;

    keepalive_timeout  65;

    gzip  on;

    real_ip_header X-Forwarded-For;
    set_real_ip_from 0.0.0.0/0;


    server {
        listen      80;
        server_name vivo.nyc  www.vivo.nyc;

        root /var/www/recital;

        pagespeed on;

        pagespeed FileCachePath /var/ngx_pagespeed_cache;
        pagespeed EnableFilters extend_cache;
        pagespeed DisableFilters rewrite_images;
        pagespeed AvoidRenamingIntrospectiveJavascript off;
        pagespeed MaxCombinedJsBytes 160000;

        location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
          add_header "" "";
        }
        location ~ "^/pagespeed_static/" { }
        location ~ "^/ngx_pagespeed_beacon$" { }

    }

}

This is part of a hybrid salt-cloud and cloudformation deployment, without going into all the details of it this line will set the DNS (provided it's hosted on route 53 and you have the aws console tools installed and have your site behind an elastic load balancer):

ELBDATA="`aws elb describe-load-balancers --region your-region-here`"
LBDNSNAME=`echo "$ELBDATA" | python -c 'import json,sys;obj=json.load(sys.stdin);print obj["LoadBalancerDescriptions"][0]["CanonicalHostedZoneName"]'`
ZONEID=`echo "$ELBDATA" | python -c 'import json,sys;obj=json.load(sys.stdin);print obj["LoadBalancerDescriptions"][0]["CanonicalHostedZoneNameID"]'`

aws route53 change-resource-record-sets --hosted-zone-id <REDACTED> --change-batch "`echo "HostName=your.hostname.com&LBDnsName=$LBDNSNAME.&ZoneID=$ZONEID" | /usr/local/bin/jinja2 --format=querystring route53-alias.json`"

The route53-alias.json file looks like this:

      {
        "Changes": [
          {
            "Action": "UPSERT",
            "ResourceRecordSet": {
              "Name": "{{ HostName }}",
              "Type": "A",
              "AliasTarget": {
                "HostedZoneId": "{{ ZoneID }}",
                "DNSName": "{{ LBDnsName }}",
                "EvaluateTargetHealth": false
              }
            }
          }
        ]
      }

Leave a comment -

Elliot

https://www.emikek.com

I'm a .NET, PHP, Python, and Javascript developer. Also Project Manager, Scrum Master, and Juilliard trained violist. You cannot connect with me on Facebook, Twitter, or LinkedIn so don't ask.