Screenshot from 2016-07-23 15-47-36

MySQL 5.7 Active-Active replication on Ubuntu 16 Lts


For this recipe we got two identical (virtual) nodes, Both Ubuntu 16Lts, Quad CPU, 16G ram, OS on a 16G vda1, and a 2T data partition on vda2
Two network interfaces, 1st is public (192.168.32.x/24) and second in the replication network on 10Gbe (192.168.33.0/24)

DB01:
eth0 192.168.32.13
eth1 192.168.33.14
DB02:
eth0 192.168.32.23
eth1 192.168.33.23

First, set up both nodes for replication, Edit /etc/mysql/mysql.conf.d/mysqld.cnf
<snip>
bind-address            = 0.0.0.0
server-id               = 10 (20 for DB02)
log_slave_updates       = 1
log_bin                 = /var/log/mysql/mysql-bin.log
log_bin_index           = /var/log/mysql/mysql-bin.log.index
relay_log               = /var/log/mysql/mysql-relay-bin
relay_log_index         = /var/log/mysql/mysql-relay-bin.index
log-error = /var/log/mysql/error.log
auto_increment_increment = 2
auto_increment_offset = 1
expire_logs_days        = 10
max_binlog_size         = 100M
</snip>

Then restart Mysql: service mysql restart

On DB01 do mysql -u root -p and login

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000019 |      154 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

The file and position are needed for the master configuration on DB02, this is no longer done in the my.cnf file but in the DB, so open up mysql on DB02 and configure:

change master to master_host='192.168.33.13', master_user='slaveuser', master_password='somesupersecretpassword', master_log_file='mysql-bin.000019', master_log_pos=154, master_port=3306
grant replication slave on *.* to slaveuser@'db01.yourdomain.com' identified by 'somesupersecretpassword';
grant replication slave on *.* to slaveuser@'192.168.33.13' identified by 'somesupersecretpassword';
flush privileges;
start slave;
slave status \G;

Now you have one way replication, to make it two way, do a show master status on DB02, and do the same configuration on DB01, replacing .13 with .23 and db01 with db02 and of course the file and position values where applicable.

Now create some databases and see them appear on the other side.

Note, this does not replicate existing databases, this is meant for a CLEAN server. If you have existing databases,  you will first need to backup/restore them to the replica node before setting up the replication. If you do any action on a pre-existing DB the replication will halt, when this happens, just do a show master on the source and get the file and position values, then reset the replication on the slave DB (first stop the replication slave with stop slave, start it again when done.

Screenshot from 2016-03-26 18:03:54

InfluxDB IfHCInOctets Query


Some basic SQL for InfluxDB to fetch the SNMP ifHCInOctets from a table and show them as per minute Mbit/S, hope it helps you make beautiful graphs

select (8*derivative(mean(ifHCInOctets)) / 60)/1024 as value  from ifHCInOctets where time > now() - 1d and host = '172.16.1.13' and instance = 'pppoe0' GROUP BY time(1m)

172.16.1.13 is some router of course (we poll VyOS devices) and ‘pppoe0’ is an interface (could be eth0,eth1 etc etc)

Cheers

J

Screenshot from 2016-03-26 18:03:54

Remote SSH Parallel processing


Ever need to update a whole bunch of (Ubuntu) hosts and got tired of manual processing. Why not hack a little script to do it for you.

First we need a list for servers in a text file. Mine is called servers.list and contains the hostnames of the hosts that need to be processed, like so;

amber.integrative.it
angela.integrative.it
christel.integrative.it
linda.integrative.it
...
..
.

For authentication I use SSH/RSA, so my admin station has privs on every host. If you have not set that up, do an ssh-copy-id root@hostname.something to copy your Key over to the hosts (yes I use root here, bad bad bad..)

Then this simple script will do the magic for you, mine is called do-servers.sh (don’t forget to chmod +x the script)

#!/bin/bash
command="$1"
while read -u999 server; do
 (echo + Processing $server - $command;ssh root@$server $command;echo - Done $server - $command) &
done 999< servers.list
wait

(the while loop uses another file handle to keep out of the way of the stdout of ssh, we block the whole ssh command with () and execute in the background with &, finally the wait at the end does what it says, wait for everything to be done.

Now all that remains is to call the script with something like ./do-servers.sh ‘apt-get disk-upgrade -y’ (put your command between parentheses to make it a single parameter.

Cest Ca..

 

Alfresco 5 – Zimbra 8


How to link Alfresco 5 to a Zimra 8.6 LDAP.

First set your authentication_chain and sync properties in

./tomcat/shared/classes/alfresco-global.properties

Replace all the bold domain/password stuff with your own of course, and use the admin account of zimbra to connect into the OpenLdap part.

Like so:

### Use Alfresco authentication for admin accounts and LDAP for users ###
authentication.chain=alfrescoNtlm1:alfrescoNtlm,ldap1:ldap
## For DEV, set synchronizeChangesOnly to false for FULL SYNC
synchronization.synchronizeChangesOnly= true 
## Set up regular synchronization with the LDAP server ##
synchronization.syncWhenMissingPeopleLogIn=true
synchronization.syncOnStartup=true
synchronization.import.cron=0 */15 * * * ?

Then create an ldap-authentication.properties File in:

./tomcat/shared/classes/alfresco/extension/subsystems/Authentication/ldap/ldap1

# LDAP Settings for OpenLDAP sync and auth

ldap.authentication.active=true
ldap.authentication.allowGuestLogin=false
ldap.authentication.userNameFormat=uid=%s,ou=people,dc=integrative,dc=it

# The LDAP context factory to use
#ldap.authentication.java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory

# The URL to connect to the LDAP server 
ldap.authentication.java.naming.provider.url=ldap://1.2.3.4:389

# The authentication mechanism to use for password validation
ldap.authentication.java.naming.security.authentication=simple

# Escape commas entered by the user at bind time Useful when using simple authentication and the CN is part of the DN and contains commas

ldap.authentication.escapeCommasInBind=false
ldap.authentication.escapeCommasInUid=false

# Comma separated list of user names who should be considered administrators by default
ldap.authentication.defaultAdministratorUserNames=admin

# Enable FTP authentication using LDAP
ldap.authentication.authenticateFTP=true

ldap.synchronization.active=true
ldap.synchronization.java.naming.security.authentication=simple
ldap.synchronization.java.naming.security.principal=uid=admin,ou=people,dc=root,dc=domain
ldap.synchronization.java.naming.security.credentials=yourpassword

ldap.synchronization.queryBatchSize=50
ldap.synchronization.attributeBatchSize=0

# The query to select all objects that represent the groups to import.

ldap.synchronization.groupQuery=(objectclass\=zimbraDistributionList)
ldap.synchronization.groupDifferentialQuery=(&(objectclass\=zimbraDistributionList)(!(modifyTimestamp< \={0})))
ldap.synchronization.groupSearchBase=ou\=people,dc\=integrative,dc\=it

ldap.synchronization.personQuery=(objectClass\=organizationalPerson)
ldap.synchronization.personDifferentialQuery=(&(objectclass\=organizationalPerson)(!(modifyTimestamp<\={0})))
ldap.synchronization.userSearchBase=ou\=people,dc\=integrative,dc\=it

# The name of the operational attribute recording the last update time for a group or user.
ldap.synchronization.personType=organizationalPerson
ldap.synchronization.modifyTimestampAttributeName=modifyTimestamp
ldap.synchronization.timestampFormat=yyyyMMddHHmmss'Z'
ldap.synchronization.userIdAttributeName=uid
ldap.synchronization.userFirstNameAttributeName=givenName
ldap.synchronization.userLastNameAttributeName=sn
ldap.synchronization.userEmailAttributeName=mail
ldap.synchronization.userOrganizationalIdAttributeName=ou
ldap.synchronization.defaultHomeFolderProvider=userHomesHomeFolderProvider

ldap.synchronization.groupIdAttributeName=mail
ldap.synchronization.groupDisplayNameAttributeName=mail
ldap.synchronization.groupType=zimbraDistributionList
ldap.synchronization.personType=zimbraMailRecipient
ldap.synchronization.groupMemberAttributeName=zimbraMailForwardingAddress

ldap.synchronization.enableProgressEstimation=true
ldap.authentication.java.naming.read.timeout=0

That;s all, reload or reboot and log in

HP/ILO troubles


The wonderful security updates in Firefox (and Chrome) will give you the dreaded SSL_ERROR_BAD_MAC_ALERT. So no more ILO for you😦

How to mitigate that?, well first do as told in the article above, go to about:config in firefox and set the security.tls.version.fallback-limit to 1.

How you can access the ILO and update the firmware to 2.29 for ILO2 (or higher), as instructed here, you can fetch the ILO2 firmware  Here

HP has a habit of only updating the Windoze firmware binaries, forgetting that most of the internet runs on Linux, but anyway just download the EXE and extract the ilo229.bin from the file.

If you don’t like the clickerdyclick web interface, or have more than 2 servers to update, it makes sense to put your binary on some web server  in your environment and just ssh into your ILO and update from there, the magic is done like so:

load -source http://192.168.1.11/iso/HP/Firmware/ilo2_229.bin /map1/firmware1

(192.168.1.11/iso…. is our internal web server of course, replace with your own)

O and don’t forget to set your FireFox security.tls.version.fallback-limit back to 3🙂

Ces’t Ca..

VyOS Backup


Want to make backups of your VyOS router/firewall, This little script might help, It takes the config and converts it into set commands for easy restore on another box. We push it to an RSYNC on a ZFS/Nexenta server, but you put it anywhere as you like. Schedule it through Cron or better through the system task scheduler.

Don’t forget to use the commit archive to record your changes for the audit trails, like so :

set system config-management commit-archive location 'scp://admin:<password>@x.x.x.x/volumes/pool1/backup/vyos'

VyOS backup.sh Script: (store in /config/scripts/backup/ and do not forget to make it executable : chmod +x /config/scripts/backup/backup.sh)

# Vyos (1.6) Backup Script (jkool@integrative.it)
# Fetch me with scp root@x.x.x.x:/volumes/pool1/backup/vyos/backup.sh /config/scripts/backup/backup.sh
# Keep 5 versions local 
#
# Schedule with:
#
# set system task-scheduler task backup executable path '/config/scripts/backup/backup.sh'
# set system task-scheduler task backup interval '8h'

h=$(hostname)
d=$(date +"%Y%m%d%H%M")
dest=192.168.1.200::pool1_backup/vyos
scripts=/config/scripts/backup

cd $scripts

tar -czf $scripts/backup-auth-$h-$d.tar.gz /config/auth
/opt/vyatta/sbin/vyatta-config-gen-sets.pl > $scripts"/backup-config-"$h"-"$d".txt"

ls -F backup-config-$h*.txt | head -n -5 | xargs rm
ls -F backup-auth-$h*.tar.gz | head -n -5 | xargs rm

rsync $scripts/backup-config-$h-$d.txt $dest/$h
rsync $scripts/backup-auth-$h-$d.tar.gz $dest/$h



Alfresco 5.0.d + OpenLdap


Can’t believe I once hired an Indy IT mercenary to configure this for MS/AD, sad really. Anyways was bored today and had some time so dug up my DEV server for Alfresco and found it was still using an internal database for authentication. So I took the trouble of configuring it to use the OpenLdap server we now use for all Authentications down here.

So how to get to it, firstly out Alfresco sits in /opt/alfresco (that might be different for you) from here make a folder for the ldap class like so:

mkdir -P /opt/alfresco/tomcat/shared/classes/alfresco/extension/subsystems/Authentication/ldap/ldap1

Now if you want to go from a template get the 5.0.d binary from GitHub and extract the ldap-authentication.properties to work on, like so:

wget https://github.com/Alfresco/community-edition/archive/V5.0.d.tar.gz
 tar -zxvf V5.0.d.tar.gz -C ~/tmp --wildcards --no-anchored 'ldap-authentication.properties'
 find tmp -type f -exec mv -i {} . \;
You can also just copy and paste this below to modify to your liking
 
 # LDAP Settings for OpenLDAP sync and auth
 ldap.authentication.active=true
 ldap.authentication.allowGuestLogin=false
 ldap.authentication.userNameFormat=uid=%s,ou=users,dc=integrative,dc=it
 # The LDAP context factory to use
 #ldap.authentication.java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory
 # The URL to connect to the LDAP server
 ldap.authentication.java.naming.provider.url=ldap://ldap.legal-it.net:389
 # The authentication mechanism to use for password validation
 ldap.authentication.java.naming.security.authentication=simple
 # Escape commas entered by the user at bind time Useful when using simple authentication and the CN is part of the DN and contains commas
 ldap.authentication.escapeCommasInBind=false
 ldap.authentication.escapeCommasInUid=false
 # Comma separated list of user names who should be considered administrators by default
 ldap.authentication.defaultAdministratorUserNames=admin
 # Enable FTP authentication using LDAP
 ldap.authentication.authenticateFTP=true
 ldap.synchronization.active=true
 ldap.synchronization.java.naming.security.authentication=simple
 ldap.synchronization.java.naming.security.principal=cn\=admin,dc\=integrative,dc\=it
 ldap.synchronization.java.naming.security.credentials=supersecretpassword
 ldap.synchronization.queryBatchSize=50
 ldap.synchronization.attributeBatchSize=0
 # The query to select all objects that represent the groups to import.
 ldap.synchronization.groupSearchBase=ou\=groups,dc\=integrative,dc\=it
 ldap.synchronization.groupQuery=(objectclass\=groupOfNames)
 ldap.synchronization.groupDifferentialQuery=(&(objectclass\==groupOfNames)(!(modifyTimestamp<\={0})))
 ldap.synchronization.userSearchBase=ou\=users,dc\=integrative,dc\=it
 ldap.synchronization.personQuery=(objectclass\=inetOrgPerson)
 ldap.synchronization.personDifferentialQuery=(&(objectclass\=inetOrgPerson)(!(modifyTimestamp<\={0})))
 # The name of the operational attribute recording the last update time for a group or user.
 ldap.synchronization.modifyTimestampAttributeName=modifyTimestamp
 ldap.synchronization.timestampFormat=yyyyMMddHHmmss'Z'
 ldap.synchronization.userIdAttributeName=uid
 ldap.synchronization.userFirstNameAttributeName=givenName
 ldap.synchronization.userLastNameAttributeName=sn
 ldap.synchronization.userEmailAttributeName=mail
 ldap.synchronization.userOrganizationalIdAttributeName=o
 ldap.synchronization.defaultHomeFolderProvider=largeHomeFolderProvider
 ldap.synchronization.groupIdAttributeName=cn
 ldap.synchronization.groupDisplayNameAttributeName=description
 ldap.synchronization.personType=inetOrgPerson
 ldap.synchronization.groupType==groupOfNames
 ldap.synchronization.groupMemberAttributeName=member
 ldap.synchronization.enableProgressEstimation=true
 ldap.authentication.java.naming.read.timeout=0

The path for this file should be

/opt/alfresco/tomcat/shared/classes/alfresco/extension/subsystems/Authentication/ldap/ldap1/ldap-authentication.properties

Now edit your alfresco global properties to call the new class for authentication and sync, just add the following to the top

### Use Alfresco authentication for admin accounts and LDAP for users ###
authentication.chain=alfrescoNtlm1:alfrescoNtlm,ldap1:ldap
## For DEV, set synchronizeChangesOnly to false for FULL SYNC
synchronization.synchronizeChangesOnly=true
## Set up regular synchronization with the LDAP server ##
synchronization.syncWhenMissingPeopleLogIn=true
synchronization.syncOnStartup=true
synchronization.import.cron=0 */15 * * * ?

And ce’st ca., you will be synchronizing with LDAP every 15 mins or on startup, so maybe a good time to restart the alfresco server and see if it works.