vanad.ishttps://vanad.is/2018-06-13T00:00:00+00:00"What is my IP-address?" with Varnish 4+2018-06-13T00:00:00+00:002018-06-13T00:00:00+00:00Freja Borgingertag:vanad.is,2018-06-13:/what-is-my-ip-address-with-varnish-4.html<p>Ever thought sites like <a href="ifconfig.me">ifconfig.me</a> and similar are too slow from the command line? Here I'll show you how easy it is to setup something similar to those services - except it is extremely fast because of Varnish.</p> <p>The receiver part:</p> <div class="highlight"><pre><span></span>sub vcl_recv { if (req.http.host == &quot;ip.vanad.is …</pre></div><p>Ever thought sites like <a href="ifconfig.me">ifconfig.me</a> and similar are too slow from the command line? Here I'll show you how easy it is to setup something similar to those services - except it is extremely fast because of Varnish.</p> <p>The receiver part:</p> <div class="highlight"><pre><span></span>sub vcl_recv { if (req.http.host == &quot;ip.vanad.is&quot;) { return(synth(800)); } } </pre></div> <p>The synthetic part:</p> <div class="highlight"><pre><span></span>sub vcl_synth { if (resp.status == 800) { set resp.status = 200; synthetic(client.ip); return (deliver); } } </pre></div> <p>Go to <a href="http://ip.vanad.is">http://ip.vanad.is</a> or test it out from the terminal:</p> <div class="highlight"><pre><span></span>curl ip.vanad.is </pre></div>Tunnel UNIX-socket with SSH or HAProxy2017-07-11T00:00:00+00:002017-07-11T00:00:00+00:00Freja Borgingertag:vanad.is,2017-07-11:/tunnel-unix-socket-with-ssh-or-haproxy.html<p>Forward PHP-application's MySQL-connections towards localhost to a remote server, edit <code>~/.ssh/config</code>:</p> <div class="highlight"><pre><span></span>Host my-remote-database-server LocalForward /var/run/mysqld/mysqld.sock 127.0.0.1:3306 </pre></div> <p>Do the same, with HAProxy:</p> <div class="highlight"><pre><span></span>global log /dev/log local0 notice user haproxy group haproxy stats socket /var/run/haproxy.stat level admin defaults log …</pre></div><p>Forward PHP-application's MySQL-connections towards localhost to a remote server, edit <code>~/.ssh/config</code>:</p> <div class="highlight"><pre><span></span>Host my-remote-database-server LocalForward /var/run/mysqld/mysqld.sock 127.0.0.1:3306 </pre></div> <p>Do the same, with HAProxy:</p> <div class="highlight"><pre><span></span>global log /dev/log local0 notice user haproxy group haproxy stats socket /var/run/haproxy.stat level admin defaults log global retries 2 timeout connect 3000 timeout server 5000 timeout client 5000 listen socket bind /var/run/mysqld/mysqld.sock mode 666 option mysql-check user HAPROXY balance first server mysql-offsite-1 10.0.0.1:3306 check maxconn 64 server mysql-offsite-2 10.0.0.2:3306 check maxconn 64 server mysql-offsite-3 10.0.0.3:3306 check maxconn 64 </pre></div> <blockquote> <p>Why not just use the remote hostname or IP in the PHP-applications?</p> </blockquote> <p>Imagine you have 100 websites, and all of those connect to a specific IP, localhost. Then you need to move out the database server from localhost to somewhere else. This would require changing all the websites configurations...</p> <p>Another reason, especially with HAProxy, is that you can handle the load-balancing on the webserver instead of inside the PHP-application. Most PHP-applications don't support load-balanced database servers anyway.</p>Send system cronjob output via XMPP2017-07-03T00:00:00+00:002017-07-03T00:00:00+00:00Freja Borgingertag:vanad.is,2017-07-03:/send-system-cronjob-output-via-xmpp.html<p>This is useful if you either lack a complete e-mail infrastructure for your server, or if you don't need a full-fledged e-mail server setup with domain names and all that.</p> <p>I personally use it to get the information as fast as possible, rather than wait for the next polling of …</p><p>This is useful if you either lack a complete e-mail infrastructure for your server, or if you don't need a full-fledged e-mail server setup with domain names and all that.</p> <p>I personally use it to get the information as fast as possible, rather than wait for the next polling of my e-mail client on my cellphone.</p> <p>First we install sendxmpp. This is a command-line client to send messages to XMPP-accounts or chatrooms.</p> <div class="highlight"><pre><span></span>apt-get install sendxmpp </pre></div> <p>It's possible to pass parameters like username and possword directly to sendxmpp, but that would reveal the password to other users if they view the processlist. So to prevent it, we make the file <code>/etc/sendxmpprc</code> for sendxmpp with the following content:</p> <div class="highlight"><pre><span></span>user@domain.tld password </pre></div> <p>Then we adjust the ownership of it:</p> <div class="highlight"><pre><span></span>chmod 600 /etc/sendxmpprc chown nobody:nogroup /etc/sendxmpprc </pre></div> <p>To make the server's mail transfer agent, in this case "postfix", we edit <code>/etc/aliases</code> and add the following, adjusted to your needs:</p> <div class="highlight"><pre><span></span>root: &quot;|sendxmpp -f /etc/sendxmpprc user@domain.tld&quot; </pre></div> <p>... or if you also want an e-mail to be sent to remoteuser@mailprovider.tld:</p> <div class="highlight"><pre><span></span>root: &quot;|sendxmpp -f /etc/sendxmpprc user@domain.tld&quot;,localuser,remoteuser@mailprovider.tld </pre></div> <p>Then make postfix re-read the changes</p> <div class="highlight"><pre><span></span>newaliases </pre></div> <p>Send a test-message with sendxmpp</p> <div class="highlight"><pre><span></span>su -s /bin/bash nobody echo test | sendxmpp -f /etc/sendxmpprc exit </pre></div> <p>Send a test-message with mail, which will pipe to sendxmpp:</p> <div class="highlight"><pre><span></span>echo test | mail -s user@domain.tld test </pre></div> <p>Here's an example of how it looks when receiving an e-mail about unattended upgrades via XMPP on SailfishOS:</p> <p><img alt="A screenshot from SailfishOS" src="https://vanad.is/xmpp.png"></p>How to host Etherpad2017-06-27T00:00:00+00:002017-06-27T00:00:00+00:00Freja Borgingertag:vanad.is,2017-06-27:/how-to-host-etherpad.html<p>This is what Wikipedia says about it</p> <blockquote> <p>Etherpad is an open source, web-based collaborative real-time editor, allowing authors to simultaneously edit a text document, and see all of the participants' edits in real-time, with the ability to display each author's text in their own color. </p> </blockquote> <p>The good things:</p> <ul> <li>Self-hosted</li> <li>Open …</li></ul><p>This is what Wikipedia says about it</p> <blockquote> <p>Etherpad is an open source, web-based collaborative real-time editor, allowing authors to simultaneously edit a text document, and see all of the participants' edits in real-time, with the ability to display each author's text in their own color. </p> </blockquote> <p>The good things:</p> <ul> <li>Self-hosted</li> <li>Open source</li> <li>Plugins</li> </ul> <p>The bad things:</p> <ul> <li>Mostly single-threaded (100% CPU usage at one core when a user is loading a pad)</li> </ul> <h4>Installation</h4> <p>First we get the current stable and LTS of NodeJS, which is 6.x at the time of writing. For Debian that means we get the key for the <a href="https://nodesource.com/">Nodesource</a> repository and import it:</p> <div class="highlight"><pre><span></span>wget https://deb.nodesource.com/gpgkey/nodesource.gpg.key apt-key add nodesource.gpg.key </pre></div> <p>Then we add the repository:</p> <div class="highlight"><pre><span></span>VERSION=node_6.x DISTRO=&quot;$(lsb_release -s -c)&quot; echo &quot;deb https://deb.nodesource.com/$VERSION $DISTRO main&quot; | tee /etc/apt/sources.list.d/nodesource.list echo &quot;deb-src https://deb.nodesource.com/$VERSION $DISTRO main&quot; | tee -a /etc/apt/sources.list.d/nodesource.list </pre></div> <p>Then we refresh the repositories and install NodeJS:</p> <div class="highlight"><pre><span></span>apt-get update apt-get install nodejs </pre></div> <p>Following the <a href="https://github.com/ether/etherpad-lite#installation">official guide</a> for Etherpad installation on GNU/Linux at GitHub we do:</p> <div class="highlight"><pre><span></span>apt-get install gzip git curl python libssl-dev pkg-config build-essential </pre></div> <p>Add a new user which will run Etherpad and login as that user:</p> <div class="highlight"><pre><span></span>useradd -s /bin/bash -m -U etherpad su etherpad </pre></div> <p>Get Etherpad and execute it:</p> <div class="highlight"><pre><span></span>git clone git://github.com/ether/etherpad-lite.git etherpad-lite/bin/run.sh </pre></div> <p>Get something to drink. It will take awhile to bootstrap Etherpad...</p> <p>When it's done then browse to <a href="http://localhost:9001">http://localhost:9001</a> and test it out.</p> <p>If you like it and want to use it in production, then shut it down <code>Ctrl+c</code>, and proceed.</p> <h4>Autostart</h4> <p>We want Etherpad to start automatically at boot. Open and edit the crontab for the user running Etherpad</p> <div class="highlight"><pre><span></span>crontab -u etherpad -e </pre></div> <p>And write the following into it to make it run at boot:</p> <div class="highlight"><pre><span></span>@reboot $HOME/etherpad-lite/bin/run.sh </pre></div> <h4>Dedicated database</h4> <p>For Debian 8, the quickest pick is MySQL. Execute:</p> <div class="highlight"><pre><span></span>apt-get install mysql-server </pre></div> <p>I leave the password blank / autogenerated, because I use the maintenance account anyway.</p> <p>Login via the maintenance account:</p> <div class="highlight"><pre><span></span>mysql --defaults-file=/etc/mysql/debian.cnf </pre></div> <p>Create the database and give it a password:</p> <div class="highlight"><pre><span></span>mysql&gt; create database etherpad; mysql&gt; grant all on etherpad.* to etherpad@localhost identified by &#39;etherpad&#39;; mysql&gt; flush privileges; </pre></div> <p>Edit the configuration <code>settings.json</code> to suit your needs. This file is created when you first run Etherpad.</p> <p>Add the database settings</p> <div class="highlight"><pre><span></span>&quot;dbType&quot; : &quot;mysql&quot;, &quot;dbSettings&quot; : { &quot;user&quot; : &quot;etherpad&quot;, &quot;host&quot; : &quot;localhost&quot;, &quot;password&quot;: &quot;etherpad&quot;, &quot;database&quot;: &quot;etherpad&quot;, &quot;charset&quot; : &quot;utf8mb4&quot; }, </pre></div> <p>Make sure the other database settings are commented out or removed.</p> <h4>Admin account</h4> <p>Open up the same file <code>settings.json</code> and comment out the authorization parts so they become active. Then change the admin password, like so:</p> <div class="highlight"><pre><span></span>&quot;users&quot;: { &quot;admin&quot;: { &quot;password&quot;: &quot;secretpassword&quot;, &quot;is_admin&quot;: true }, }, </pre></div> <p>Then start up Etherpad again as a non-privileged user:</p> <div class="highlight"><pre><span></span>su -s /bin/bash etherpad etherpad-lite/bin/run.sh </pre></div> <p>After that it's up to you to tweak the configuration according to your needs.</p>Mail on SSH-login2017-06-20T00:00:00+00:002017-06-20T00:00:00+00:00Freja Borgingertag:vanad.is,2017-06-20:/mail-on-ssh-login.html<p>First we need a way to send an e-mail. This package provides the "mail" command.</p> <div class="highlight"><pre><span></span>apt-get install mailutils </pre></div> <p>As root, create the file <code>/etc/ssh/sshrc</code> with the following content:</p> <div class="highlight"><pre><span></span>exec 1&gt;<span class="err">&amp;</span>2 env | mail -s &quot;LOGIN <span class="nv">$USER</span>@$(hostname) from <span class="cp">${</span><span class="n">SSH_CONNECTION</span><span class="o">%%</span> <span class="o">*</span><span class="cp">}</span>&quot; root </pre></div> <p>From now on the user "root" will receive …</p><p>First we need a way to send an e-mail. This package provides the "mail" command.</p> <div class="highlight"><pre><span></span>apt-get install mailutils </pre></div> <p>As root, create the file <code>/etc/ssh/sshrc</code> with the following content:</p> <div class="highlight"><pre><span></span>exec 1&gt;<span class="err">&amp;</span>2 env | mail -s &quot;LOGIN <span class="nv">$USER</span>@$(hostname) from <span class="cp">${</span><span class="n">SSH_CONNECTION</span><span class="o">%%</span> <span class="o">*</span><span class="cp">}</span>&quot; root </pre></div> <p>From now on the user "root" will receive an e-mail with the user who logged in and their IP-address.</p> <p>This is only a practical way to keep track of users logging in to a shell. It's not a way to audit every login because it's easy to bypass, for example by logging in without starting a shell. For auditing it's better to watch the logs.</p>Accelerating Wordpress with Varnish 42017-06-15T00:00:00+00:002017-06-15T00:00:00+00:00Freja Borgingertag:vanad.is,2017-06-15:/accelerating-wordpress-with-varnish-4.html<p>The receiver part:</p> <div class="highlight"><pre><span></span>sub vcl_recv { # Wordpress - cache everything except... if (!(req.url ~ &quot;wp-admin|xmlrpc\.php|wp-(comments-post|login|activate|mail)\.php|&amp;preview=true&quot;)) { unset req.http.cookie; return (hash); } # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you …</pre></div><p>The receiver part:</p> <div class="highlight"><pre><span></span>sub vcl_recv { # Wordpress - cache everything except... if (!(req.url ~ &quot;wp-admin|xmlrpc\.php|wp-(comments-post|login|activate|mail)\.php|&amp;preview=true&quot;)) { unset req.http.cookie; return (hash); } # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don&#39;t need, # rewriting the request, etc. } </pre></div> <p>The backend part:</p> <div class="highlight"><pre><span></span>sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. # Wordpress - delete unnecessary before storing in cache if (!(bereq.url ~ &quot;wp-admin|xmlrpc\.php|wp-(comments-post|login|activate|mail)\.php|&amp;preview=true&quot;)) { unset beresp.http.set-cookie; unset beresp.http.cookie; unset beresp.http.expires; unset beresp.http.cache-control; unset beresp.http.pragma; set beresp.ttl = 1m; set beresp.grace = 1w; } } </pre></div> <p>This parts makes the client retry if they got an <em>503 Service Unavailable</em>. This is practical if one backend suddenly went down.</p> <div class="highlight"><pre><span></span>sub vcl_backend_error { if (beresp.status == 503) { return (retry); } } </pre></div> <p>If you use SSL, this is almost required. Otherwise the default works well enough.</p> <div class="highlight"><pre><span></span># Split plain and encrypted objects sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } # Use special internal SSL hash for https content # X-Forwarded-Proto is set to https by Pound if (req.http.X-Forwarded-Proto ~ &quot;https&quot;) { hash_data(req.http.X-Forwarded-Proto); } return (lookup); } </pre></div> <p>Without this we won't be able to cache completely.</p> <div class="highlight"><pre><span></span>sub vcl_hit { set req.http.grace = &quot;full&quot;; } </pre></div>Automatic upgrades in Debian2017-06-15T00:00:00+00:002017-06-15T00:00:00+00:00Freja Borgingertag:vanad.is,2017-06-15:/automatic-upgrades-in-debian.html<p>First we install the required packages. As root do:</p> <div class="highlight"><pre><span></span>apt-get install unattended-upgrades </pre></div> <p>Create the file: <code>/etc/apt/apt.conf.d/20auto-upgrades</code> and add the following content to it:</p> <div class="highlight"><pre><span></span><span class="nt">APT</span><span class="p">::</span><span class="nd">Periodic</span><span class="p">::</span><span class="nd">Update-Package-Lists</span> <span class="s2">&quot;1&quot;</span><span class="o">;</span> <span class="nt">APT</span><span class="p">::</span><span class="nd">Periodic</span><span class="p">::</span><span class="nd">Unattended-Upgrade</span> <span class="s2">&quot;1&quot;</span><span class="o">;</span> </pre></div> <p>Replace the file <code>/etc/apt/apt.conf.d/50unattended-upgrades</code> and add the following to …</p><p>First we install the required packages. As root do:</p> <div class="highlight"><pre><span></span>apt-get install unattended-upgrades </pre></div> <p>Create the file: <code>/etc/apt/apt.conf.d/20auto-upgrades</code> and add the following content to it:</p> <div class="highlight"><pre><span></span><span class="nt">APT</span><span class="p">::</span><span class="nd">Periodic</span><span class="p">::</span><span class="nd">Update-Package-Lists</span> <span class="s2">&quot;1&quot;</span><span class="o">;</span> <span class="nt">APT</span><span class="p">::</span><span class="nd">Periodic</span><span class="p">::</span><span class="nd">Unattended-Upgrade</span> <span class="s2">&quot;1&quot;</span><span class="o">;</span> </pre></div> <p>Replace the file <code>/etc/apt/apt.conf.d/50unattended-upgrades</code> and add the following to it to upgrade everything. Here you can also add excludes if there are packages you don't want upgraded automatically.</p> <div class="highlight"><pre><span></span><span class="nt">Unattended-Upgrade</span><span class="p">::</span><span class="nd">Origins-Pattern</span> <span class="p">{</span> <span class="err">&quot;o=*&quot;</span><span class="p">;</span> <span class="p">}</span><span class="o">;</span> <span class="nt">Unattended-Upgrade</span><span class="p">::</span><span class="nd">Automatic-Reboot</span> <span class="s2">&quot;false&quot;</span><span class="o">;</span> <span class="nt">Unattended-Upgrade</span><span class="p">::</span><span class="nd">Mail</span> <span class="s2">&quot;root&quot;</span><span class="o">;</span> </pre></div> <p>That's it! Now you'll receive an e-mail every time your system upgrades itself.</p>