DNSSEC with BIND 9.10 and native PKCS#11

DNSSEC with BIND and native PKCS#11 support (BIND & SoftHSM)

Bind 9.10.0-P1 supports the native PKCS#11 mode, instead of the openssl based PKCS#11. You can either compile it with (./configure --enable-native-pkcs11 \
--with-pkcs11=provider-library-path
) or install prebuilt packages.

Upon writing this blog, Fedora 23, has built-in bind-9.10.3-7.P2 and SoftHSM (Software based HSM)

SoftHSM is an implementation of a cryptographic store accessible through a PKCS #11 interface

Install the required packages

# dnf install bind-chroot bind-pkcs11 softhsm

bind-chroot-32:9.10.3-7.P2.fc23.x86_64
bind-pkcs11-9.10.3-7.P2.fc23.x86_64
softhsm-2.0.0rc1-3.fc23.x86_64

Initialize the SoftHSM repository
# softhsm2-util --init-token 0 --slot 0 --label softhsm
enter the user and security pin

Generate the keys (Key Signing Key and Zone Signing Key)

You may use the algorithm and key size depends on your requirement.
# pkcs11-keygen -a RSASHA256 -b 2048 -l sample_ksk
Enter Pin:
# pkcs11-keygen -a RSASHA256 -b 2048 -l sample_zsk
Enter Pin:

# pkcs11-list
Enter Pin:
object[0]: handle 2 class 2 label[12] 'sample_ksk' id[0]
object[1]: handle 3 class 2 label[12] 'sample_ksk' id[0]
object[2]: handle 4 class 3 label[12] 'sample_zsk' id[0]
object[3]: handle 5 class 3 label[12] 'sample_zsk' id[0]

Create a pair of BIND9 key files using dnssec-keyfromlabel-pkcs11 utility, since we are using pkcs#11 backend the label must be pkcs#11 uri format. Don’t know how safe it is to store the pin on the file system, but yes we have to create a text file with the HSM pin. Not sure if the dnssec-keyfromlabel can prompt for the pin.

# dnssec-keyfromlabel-pkcs11 -a RSASHA256 -f KSK -l 'pkcs11:object=sample_ksk;pin-source=/etc/token_pin' example.com
Kexample.com.+005+46938.key
# dnssec-keyfromlabel-pkcs11 -a RSASHA256 -l 'pkcs11:object=sample_zsk;pin-source=/etc/token_pin' example.com
Kexample.com.+005+46939.key

The resulting files can be used to sign the zone, as per the BIND documentation – “Unlike the normal K* files, which contain both public and private key data, these files will contain only the public key data, plus an identifier for the private key which remains stored within the HSM. Signing with the private key takes place inside the HSM.”

Include the keys in zone file or specify the key path on the named configuration.

echo "$INCLUDE Kexample.com.+005+46938.key" >> example.com.zone
echo "$INCLUDE Kexample.com.+005+46939.key" >> example.com.zone

Signing the zones
# dnssec-signzone-pkcs11 example.com
Verifying the zone using the following algorithms: RSASHA2.
Zone fully signed:
Algorithm: RSASHA2: KSKs: 1 active, 0 stand-by, 0 revoked
ZSKs: 1 active, 0 stand-by, 0 revoked

# head example.com.signed
; File written on Mon Jan 18 16:02:19 2016
; dnssec_signzone version 9.10.3-P2-RedHat-9.10.3-7.P2.fc23

Reference: BIND 9 Administrator Reference Manual
https://ftp.isc.org/isc/bind/cur/9.10/doc/arm/Bv9ARM.ch04.html
SoftHSM documentation

Monitor VMware ESXi hardware without root (Nagios)

Download and configure the plugin: http://exchange.nagios.org/directory/Plugins/Operating-Systems/*-Virtual-Environments/VMWare/check_esxi_hardware-2Epy/

– Create a new user in ESXi with no access privilege, you need to login to the ESXi directly to do that.

user

esxi_access

– Enable SSH, and add nagios user to root group:
# vi /etc/group
root:x:0:root,nagios

– Check from the command line, if it works
./check_esxi_hardware.py --host https://esxihost:5989 --user file:credentials.txt --pass file:credentials.txt
OK - Server: Cisco Systems Inc.....

– Configure the credentials files to use the nagios user credentials.

Setup GeoIP (PECL) for piwik geolocation and updating old visits

GeoIP is the recommended way to accurately determine the location of the visitor, by default geolocation settings may provide in accurate result.

To enable GeoIP(PECL) from redhat/centos machines:
# yum install php-pecl-geoip
#apachectl restart
# php -m | grep -i geo
geoip

From Piwik, Settings –> Geolocation –> GeoIP (PECL)

To reindex the old visits:
# cd misc/others
# php ./geoipUpdateRows.php
[note] Found working provider: geoip_pecl
90094 rows to process in piwik_log_visit and piwik_log_conversion....
.
.
.
100% done!

Send attachments from command line with mutt

To send e-mails from command line with attachments using mutt.

Set the from address with EMAIL=
-s – Subject
-a – attachment file
recipient name
-c – for CC
-b – for BCC
create a text file (eg: /tmp/testmessage) , with the body of the message.

EMAIL="foo@bar" mutt -s "Subject" -a test.doc foo1@bar -c foo2@bar < /tmp/testmessage

Could not connect to vmware console https://vcenter_address:7331/

This usually happens from the vSphere web client while opening a console session with virtual machine.

and the log (/var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log) shows something like:

[ERROR] Thread-42 System.err
INFO:oejsh.ContextHandler:started o.e.j.w.WebApp Context{/console,file:/tmp/jetty-0.0.0.0-7331-console.war-_console-any-/webapp/},/usr/lib/vmware-vsphere-client/server/work/tmp/console-distro/webapps/console.war

To fix this set the environment variable VMWARE_JAVA_HOME to proper path:

– SSH to vcenter
# vi /usr/lib/vmware-vsphere-client/server/wrapper/conf/wrapper.conf

– Under Environment variables add:
set.default.VMWARE_JAVA_HOME=/usr/java/jre-vmware

– Restart vsphere-client
# /etc/init.d/vsphere-client restart
Stopping VMware vSphere Web Client...
Stopped VMware vSphere Web Client.
Starting VMware vSphere Web Client...
Intializing registration provider...
Getting SSL certificates
Service with name was updated.
Return code is: Success
Waiting for VMware vSphere Web Client......
running: PID:

Reference: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2060604

Create bootable USB on OSX

– Identify the disk number for the USB disk inserted, usually you can find it from the “Name” and “Size” field.
Below eg: , We inserted a USB with 4.1 GB size (so as the identified: “disk2”.)
Open terminal and execute the following commands:

sh-3.2$ diskutil list
/dev/disk0
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *120.0 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_CoreStorage 119.0 GB disk0s2
3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1
#: TYPE NAME SIZE IDENTIFIER
0: Apple_HFS Macintosh HD *118.7 GB disk2
/dev/disk2
#: TYPE NAME SIZE IDENTIFIER
0: UNTITLED *4.1 GB disk2

– Unmount the disk
sh-3.2$ diskutil unmountDisk /dev/disk3
Unmount of all volumes on disk3 was successful

– write the ISO file to USB using dd command
sh-3.2$ sudo dd if=Downloads/ubuntu-12.04.3-desktop-i386.iso of=/dev/disk2 bs=1m
Password:
707+0 records in
707+0 records out
741343232 bytes transferred in 139.059398 secs (5331126 bytes/sec)

– Unmount the disk
sh-3.2$ diskutil eject /dev/disk3
Disk /dev/disk3 ejected

Convert Linux Physical Server to VMware virtual machine

Download and install vCenter Converter on a windows machine.
http://www.vmware.com/products/converter
Unfortunately this tool does not have a Linux / MAC version.

vCenter Converter

vCenter Converter

In case you see an error: “Permission to perform this operation was denied”, right click and run the program as Administrator.

Permission to perform this operation was denied
Provide the source and destination information, the source is the physical server to be converted and the destination vCenter.

vCenter Converter

vCenter Converter

vCenter Converter

vCenter Converter

Follow the steps, to do the conversion , a temporary OS will be started on the destination, by default it try to get an IP address from the DHCP server so that it can connect to the source machine and fetch the files required. But in case if you don’t have DHCP server you might see error like: “Unable to obtain the IP address of the helper virtual machine” . Fix this issue by setting up a static IP to the helper virtual machine during the conversion setup. Basically the helper VM IP should be able to communicate with the source machine which needs to be migrated

vCenter Converter Static IP

vCenter Converter Static IP

Proceed with the conversion, the duration will be based on the size of the VM and the connectivity if it belongs to another site/LAN.

You may need to change the network configuration (eg: HWADDR) and the MAC address mapping (/etc/udev/rules.d) to get it connected.

ESXi host fails with a purple diagnostic screen PSOD

This happened while converting KVM VMs to VMware and power them on (method used: http://arunnsblog.com/2013/06/10/migrate-kvm-virtual-machines-to-vmware-esxi/) . It works for a while but then the ESXi crashes with PSOD.

Version : 5.1.0-799733

There were two sort of PSOD messages observed:
1) Crashed while the VM was running

 VMware NOT_IMPLEMENTED bora/vmkernel/sched/memsched.c:17724
 Code start: 0x41802b200000 VMK uptime: 10:19:25:27.335
 cpu4:8243)0x412200cdbaf0:[0x41802b27abff]PanicvPanicInt@vmkernel#nover+0x56 stack: 0x3000000008
 cpu4:8243)0x412200cdbbd0:[0x41802b27b4a7]Panic@vmkernel#nover+0xae stack: 0x100000000000000
 cpu4:8243)0x412200cdbc50:[0x41802b3d88eb]MemSched_WorldCleanup@vmkernel#nover+0x426 stack: 0x4100018a4fb0
 cpu4:8243)0x412200cdbef0:[0x41802b3033b8]WorldCleanup@vmkernel#nover+0x1cb stack: 0x4700cdbf40
 cpu4:8243)0x412200cdbf60:[0x41802b303829]WorldReap@vmkernel#nover+0x318 stack: 0x0
 cpu4:8243)0x412200cdbff0:[0x41802b2483c8]helpFunc@vmkernel#nover+0x517 stack: 0x0
 cpu4:8243)0x412200cdbff8:[0x0] stack: 0x0
 cpu4:8243)base fs=0x0 gs=0x418041000000 Kgs=0x0
VMWare_ESXi_PSOD
VMWare_ESXi_PSOD

2) Crashed during ESXi reboot.

#PF Exception 14 in world 8243:helper13-1 IP 0x41802b880a1e addr 0x410401503020
VMWare_ESXi_PSOD
VMWare_ESXi_PSOD

This seems to be a known issue in VMware ESXi 5.1 and is resolved in patch ESXi510-201212401-BG (Build 914609).
Ref: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2038767

To work around this issue, SSH to the ESXi host and increase the MinZeroCopyBufferLength to 512.

# esxcli system settings advanced set -o /BufferCache/MinZeroCopyBufferLength -i 512

To verify that the setting has been updated, run this command:

# esxcli system settings advanced list --option /BufferCache/MinZeroCopyBufferLength
Before and after change
Before and after change

 

 

Migrate KVM virtual machines to VMware ESXi

– Shutdown the KVM guest
– convert the QCOW2 or RAW format to VMDK format

# qemu-img convert image.img -O vmdk image.vmdk

– Upload this image to datastore

– Create a new virtual machine with this disk image

– There might be issues with network interface mapping, fix the network mapping at /etc/udev/rules.d/70-persistent-net.rules