Rage Against the Shell – Page 2 – Linux tips and other things…

Boot on LVM root partition in Raspbian

Posted on March 8, 2017 - March 21, 2018 by Mr. Reboot

Tested in Raspberrypi 2 Model B / Raspbian Jessie

It’s possible booting on a LVM root partition using a external USB disk instead a SD card in Raspberrypi. Steps are the next (create lvm partitions is out of this post):

Check if your raspberrypi supports booting from initrd, is necessary for activate LVM on system boot:

~ $ zcat /proc/config.gz | grep INITRD
CONFIG_BLK_DEV_INITRD=y

If don’t exist /proc/config.gz load configs before:

~ $ modprobe configs

If result is “y” (usually) then create initrd file:

~ $ mkinitramfs -o /boot/initramfs.gz

In /boot/cmdline.txt file change the root partition replacing:

~ $ root=/dev/mmcblk0p2

with lvm disk using mapper designation, supposing vg0 as lvm group and lv01 as lvm volume:

~ $ root=/dev/mapper/vg0-lv01

You can add rootdelay=5 because interfaces can take a while to appear. At last add:

~ $ initramfs initramfs.gz

at the end of file /boot/config.txt. Finally reboot, with a little luck your rapsberrypi should boot on the new lvm root partition.

Check opendkim keys

Posted on February 4, 2017 - February 2, 2017 by Mr. Reboot

For checking opendkim private and public keys you can use opendkim-testkey:

~ $ opendkim-testkey -d mydomain.com -s default -k /etc/opendkim/keys/mydomain.com/key.private -vvv
opendkim-testkey: key loaded from /etc/opendkim/keys/mydomain.com/key.private
opendkim-testkey: checking key 'default._domainkey.mydomain.com'
opendkim-testkey: key not secure
opendkim-testkey: key OK

-d: domain
-s: selector, in this case “default”
-k: local path to the private key
-vvv: extra verbose info

Message key not secure is just due that the domain has not DNSSEC configured.

Custom errors in apache + php-fpm

Posted on December 13, 2016 by Mr. Reboot

Tested in Apache 2.4 / PHP-FPM 5.6

By default php-fpm shows “File not found” when someone try to request a php file that doesn’t exist. If you use php-fpm in apache with mod_proxy for using custom errors just configure this in apache2.conf:

ProxyErrorOverride on

Custom errors in nginx + php-fpm

Posted on December 13, 2016 by Mr. Reboot

Tested in Nginx 1.10 / PHP-FPM 5.6

By default php-fpm shows “File not found” when someone try to request a php file that doesn’t exist. If you use php-fpm in nginx with fastcgi for using custom errors just configure this in nginx.conf:

fastcgi_intercept_errors on;

Redirect http to https on the same port in nginx

Posted on November 19, 2016 by Mr. Reboot

Tested in Ubuntu 14 / Nginx 1.11

If you use a custom port to serve SSL and you want to force a secure connection redirecting from http to https, a solution is use error_page directive with code 497:

server {

        listen 8080 ssl;

        root /usr/share/nginx/mydomain;
        index index.html index.php;

        ssl_certificate /etc/ssl/certs/mydomain.com.crt;
        ssl_certificate_key /etc/ssl/private/mydomain.com.key;

        error_page 497  https://$host:$server_port$request_uri;

        ...

}

Don’t forget to reload nginx:

~ $ /etc/init.d/nginx reload

Varnish SSL redirect

Posted on November 6, 2016 - November 6, 2016 by Mr. Reboot

Tested in Ubuntu 14 / Debian 8 / Varnish 4.1

If you have configured nginx as SSL proxy for varnish, you could be interested in redirecting requests from HTTP to HTTPS. We are going to suppose this configuration, so first in VCL recv, add this code:

sub vcl_recv {
    ...
    if (req.http.X-Forwarded-Proto !~ "(?i)https") {
        return (synth(750, ""));
    }
    ...
}

And then in VCL synth:

sub vcl_synth {
    ...
    if (resp.status == 750) {
        set resp.status = 301;
        set resp.http.Location = "https://domain.com" + req.url;
    }
    ...
}

Finally reload varnish:

~ $ /etc/init.d/varnish reload

Mysql failover in Dovecot

Posted on November 5, 2016 - December 6, 2017 by Mr. Reboot

Tested in Debian 8 / Dovecot 2.2

In a environment with Dovecot and Mysql as backend, if you have several mysql servers in replication mode (cluster, master-master, master-slave …), you can configure dovecot to connect to these servers, so if one of them falls, dovecot will try to connect the next available one.

This is made in the configuration file of the mysql connection, in the connect parameter, host var:

~ $ cd /etc/dovecot
~ $ cat dovecot-sql.conf.ext
...
driver = mysql
connect = host=server1 host=server2 host=server3 dbname=mysql-db user=mysql-user password=mysql-password
...

Mysql failover in Postfix

Posted on November 4, 2016 - December 6, 2017 by Mr. Reboot

Tested in Debian 8 / Postfix 2.11

In a environment with Postfix and Mysql as backend, if you have several mysql servers in replication mode (cluster, master-master, master-slave …), you can configure postfix to connect to these servers, so if one of them falls, postfix will try to connect the next available one.

This is made in the configuration file of the mysql connection, in the hosts parameter:

~ $ cd /etc/postfix
~ $ cat mysql-users.cf
user = mysql-user
password = mysql-password
dbname = mysql-db
table = users
hosts = server1 server2 server3
query = select maildir from users where username='%s'

If you have configuration files for alias, domains, or more, you will have to make changes in these files too.

Varnish + SSL + WordPress

Posted on October 23, 2016 - October 23, 2016 by Mr. Reboot

Tested in Ubuntu 14 / Varnish 4 / Nginx 1.11 / Apache 2.4

Using nginx as a proxy is the easiest and powerfull method to use SSL on a Varnish scenario, all incoming SSL traffic on 443 port will be redirected by nginx to varnish on port 80. Schema would be this:

Nginx(ssl) -> Varnish(caching) -> Apache|Nginx(backend) -> WordPress(app)

We assume that varnish is runnig and caching requests to the backend, so let’s go to install nginx:

~ $ apt-get install nginx

Now you have to create a virtual host file with the SSL and proxy parameters:

server {
        listen 443 ssl;

        server_name domain.com;
        ssl_certificate /etc/ssl/certs/domain.com.pem;
        ssl_certificate_key /etc/ssl/private/domain.com.key;

        location / {
                proxy_pass http://127.0.0.1:80;
                proxy_redirect off;
                proxy_set_header X-Real-IP  $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto https;
                proxy_set_header X-Forwarded-Port 443;
                proxy_set_header Host $host;
        }

}

Be sure nginx load this file, you can create it in /etc/nginx/conf.d directory with *.conf extension. Add this, if not exist, to the end of /etc/nginx/nginx.conf file inside the http block:

include /etc/nginx/conf.d/*.conf;

You can install a Let’s encrypt certificate or generate one self-signed.

Now restart nginx:

~ $ /etc/init.d/nginx restart

If you try to load domain.com via https probably you will see errors on load style sheets, images, even on secondary pages. This happens because wodpress doesn’t know that the connection is HTTPS, and internally try to serve content in plain HTTP.

To solve it, you have to tell the backend that changes to HTTPS if connection is originated in HTTPS.

Nginx as backend

Configure the HTTPS fastcgi parameter:

~ $ echo "fastcgi_param HTTPS $wordpress_https;" >> /etc/nginx/fastcgi_params

In /etc/nginx/nginx.conf add this to the http block:

map $http_x_forwarded_proto $wordpress_https {
       https on;
}

And restart nginx:

~ $ /etc/init.d/nginx restart

Apache as backend

Be sure apache has loaded mod_setenvif module first. Then create the config file domain.com.conf in /etc/apache2/conf-available/ with the content:

SetEnvIf X-Forwarded-Proto "^https$" HTTPS=on

Create the symlink and restart apache:

~ $ ln -s /etc/apache2/conf-available/domain.com.conf /etc/apache2/conf-enable/domain.com.conf
~ $ /etc/init.d/apache2 restart

Now your wordpress should be loading correctly.

Error: page xxxx log sequence number yyyy is in the future

Posted on October 21, 2016 - October 23, 2016 by Mr. Reboot

Tested in Debian 8 / MySQL 5.5

If you’re getting this error on mysql error log:

InnoDB: Error: page 4352 log sequence number 12151412585
InnoDB: is in the future! Current system log sequence number 8204.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files

It’s due to a sequence mismatch in the InnoDB log, so it’s probable mysql doesn’t start. One thing you can do is restore a backup but, what happens if there is no backup?… Well, you can try to recover mysql using innodb-force-recovery.

First add this to the [mysqld] section in the my.cnf file:

innodb-force-recovery = 6

And then restart mysql:

~ $ /etc/init.d/mysql start

In the mysql error log you should get something like this:

InnoDB: Waiting for the background threads to start
InnoDB: 5.5.52 started; log sequence number 0
InnoDB: !!! innodb_force_recovery is set to 6 !!!
[Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
[Note]   - '127.0.0.1' resolves to '127.0.0.1';
[Note] Server socket created on IP: '127.0.0.1'.
[Note] Event Scheduler: Loaded 0 events
[Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.52'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

If not, you are in trouble, look for a backup wherever… If yes, the next step is make a backup of all databases:

~ $ mysqldump -u root --all-databases > dump.sql

If dump ends successfully you are lucky. Now stop mysql:

~ $ /etc/init.d/mysql stop

Comment the innodb-force-recovery line from my.cnf file, and move ibdata*, ib_logfile* and all database folders, except mysql, to a temp dir:

~ $ cd /var/lib/mysql
~ $ mv ibdata* ib_logfile* database1 database2 (...) /tmp/

Start mysql, it will create new ibdata and ib_logfiles:

~ $ /etc/init.d/mysql start

In the error log:

InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
InnoDB: Waiting for the background threads to start
InnoDB: 5.5.52 started; log sequence number 0
[Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
[Note]   - '127.0.0.1' resolves to '127.0.0.1';
[Note] Server socket created on IP: '127.0.0.1'.
[Note] Event Scheduler: Loaded 0 events
[Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.52'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

Mysql is working fine in a clean eviroment, the last step is restore the prior dump:

~ $ mysql -u root < dump.sql

At this time you should have all mysql databases, working like they were before crash.