Saturday, December 12, 2020

How2 change Datadir path under MariaDB 10.3 running on CentOS 8 server

This tutorial will guide with the steps to change datadir in MariaDB and also provide with rollback steps in case it does not work as expected.

Configuration steps are as follows


    a) Plan for the server downtime and shutdown MariaDB.

            # systemctl stop mariadb

        Reconfirm the status of MariaDB is properly shutdown.

            # systemctl status mariadb


    b) Copy existing data to new datadir (in our case we will use /mnt/data), but replace this path as per your new datadir path.

            # rsync -av /var/lib/mysql /mnt/data

        Optional step : Above rsync command will copy all data files keeping ownership intact, but to ensure all directories and datafiles have correct ownership execute following command

           
# chown -R mysql:mysql /mnt/data/


    c) Rename original datadir so as to have a rollback plan in place and also to avoid confusion.

            # mv /var/lib/mysql /var/lib/mysql.old


    d) Update config files with new datadir path

        Update server config file

            
# vi /etc/my.cnf.d/mariadb-server.cnf

        in [mysqld] section, comment out original datadir and socket lines as it can be used to roll back changes

            
#datadir=/var/lib/mysql
            #socket=/var/lib/mysql/mysql.sock

        and add new values

            datadir=/mnt/data/mysql
            socket=/mnt/data/mysql/mysql.sock
 
        and save the file and proceed to updated client config file

            # vi /etc/my.cnf.d/client.cnf

        under client-mariadb section add this line

            [client-mariadb]
            
socket=/mnt/data/mysql/mysql.sock

        and save the file


    e) Start MariDB server

           
# systemctl start mariadb

        and check if mariadb is working fine

            # systemctl status mariadb

        Incase there is any error then you can try to debug the same or roll back steps as mentioned in section c & d and restart MariaDB and you will be back to original state


    f) To check the status, login into MariaDB console

            # mysql -uroot -p

        check if datadir is set correctly
        
           
> select @@datadir;

        and response should be as follows

            +-------------------+
            | @@datadir         |
            
+-------------------+
            | /mnt/data/mysql/  |
           
+-------------------+


    g) To check everything is working fine, create a database

            
> create database test;

        and if created successfully then proceed to clean up the test database. 

            > drop database test;

        and exit out of mariadb.


    h) After ensuring all the data is successfully copied and new Datadir path is working fine, delete the /var/lib/mysql.old folder to save on hdd space.

            # rm -rf /var/lib/mysql.old

With this we are done with moving Datadir path under MariaDB 10.3 and I hope this post help you with the required steps.

Thursday, March 12, 2020

How2 deploy Redis 5 Cluster on CentOS 8

This tutorial will guide you with the steps to deploy Redis 5.0 master nodes cluster across 3 nos of CentOS 8 servers.

Disclaimer :

a) There are multiple ways to deploy Redis cluster
b) This documents attempts to quickly get started and cover basic commands that are required to get cluster running
c) I am also learning and not an expert, so help me improvise :)


Assumptions :

1) We have 3 different servers / VMs with their own IPs to deploy cluster
2) User need to have root access to all servers and user would do su to get root access. And as a policy direct ssh root login should be avoided.

Process :

1] First we need to check if firewall is enabled on the server and atleast running with bare minimum rules

    # systemctl status firewalld

and if not running we start it immediately and set it to start by default when server reboots

    # systemctl enable --now firewalld


2] We need to install EPEL repository on CentOS 8 server and update packages

    # dnf install epel-release && dnf update

optionally we can restart the server and at times it would be kernel update that would require restart

    # reboot


3] Now install few basic tools that we would need for debugging or testing

    # dnf install telnet bind-utils

you should be able to use telnet command to debug open port like

    # telnet localhost 6379

or for inter server connectivity

    # telnet SERVER_1_IP 6379


4] We do some basic OS level tweek for improving Redis performance by editing /etc/sysctl.conf file by adding following line at the end of file

    vm.overcommit_memory = 1

and reboot the server for setting to get effect

or

to enable it without reboot then run

    # sysctl vm.overcommit_memory=1


5] 
Disable transparent huge pages features to improve on memory usage and latency.

    # echo never > /sys/kernel/mm/transparent_hugepage/enabled


6] Install Redis on all 3 servers

    # dnf install @redis

once installed, we start Redis server and also enable Redis services to auto start on reboot

    # systemctl enable --now redis


7] Lets check if Redis is running fine

    # redis-cli ping

    and get PONG in response


8] Now configure redis.conf file for cluster

    # vi /etc/redis.conf

search for following lines and update 

a) Change Redis server listening IP address to servers ip address

    bind 127.0.0.1

to

    bind SERVER_IP_ADDRESS

optionally

    bind SERVER_IP_ADDRESS 127.0.0.1

Please note do not set 127.0.0.1 as first ip otherwise your cluster would fail


b) This is optional step, change port number if required to be running on different port. Please note, any good security tool can identify service running on any port, so don't bother to change port no for security reason

    port 6379

to

    port NEW_PORT_NO


c) Correct the path of the pidfile accordingly

    pidfile /var/run/redis_6379.pid

to

    pidfile /var/run/redis/redis_6379.pid


d) Enable cluster

    # cluster-enabled yes

to

    cluster-enabled yes


e) Enable cluster config file that would be system configured

    # cluster-config-file nodes-6379.conf

to

    cluster-config-file nodes-6379.conf


f) Set node timeout to 5 seconds


    # cluster-node-timeout 15000


to


    cluster-node-timeout 5000



g) Enable Append-only logs of all write operations performed by the server (AOF)

    appendonly no

to

    appendonly yes


h) Enable security by setting authentication password

    # requirepass foobared

with random password string

    requirepass LONG_RANDOM_STRING_WITH_NUMBERS_AND_SPECIALKEYS

and if this is set then user has to authenticate before accessing data

$ redis-cli -c -p 6379
127.0.0.1:6379> keys *
(error) NOAUTH Authentication required.
127.0.0.1:7531> auth LONG_RANDOM_STRING_WITH_NUMBERS_AND_SPECIALKEYS
OK


9] Now configure firewalld on all Redis servers to run cluster by opening 6379 and 16379 ports between all servers for running cluster and opening client IPs

    # firewall-cmd --new-zone=redis --permanent
    # firewall-cmd --zone=redis --add-port=6379/tcp --permanent
    # firewall-cmd --zone=redis --add-port=16379/tcp --permanent

and for following skip your own server ip rule i.e on Redis server 1, you don't need to add open rule for Redis server 1 ip

    # firewall-cmd --zone=redis --add-source=SERVER_1_IP --permanent
    # firewall-cmd --zone=redis --add-source=SERVER_2_IP --permanent
    # firewall-cmd --zone=redis --add-source=SERVER_3_IP --permanent
    # firewall-cmd --reload
    # firewall-cmd --runtime-to-permanent

to check firewalld rules run

    # firewall-cmd --list-all --zone=redis
    # firewall-cmd --list-all-zones


10] Finally lets create Redis Cluster by running following command on one of the server

    # redis-cli --cluster create SERVER_1_IP:6379 SERVER_2_IP:6379 SERVER_3_IP:6379

and you would see something similar. Also you will be asked to confirm the config by typing 'yes'

    Performing hash slots allocation on 3 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    M: SERVERSTRING9dcc5 SERVER_1_IP:6379
     slots:[0-5460] (5461 slots) master
    M: SERVERSTRING42932 SERVER_2_IP:6379
     slots:[5461-10922] (5462 slots) master
    M: SERVERSTRING7174c SERVER_3_IP:6379
     slots:[10923-16383] (5461 slots) master
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    .
    >>> Performing Cluster Check (using node SERVER_1_IP:6379)
    M: SERVERSTRING9dcc5 SERVER_1_IP:6379
     slots:[0-5460] (5461 slots) master
    M: SERVERSTRING7174c SERVER_3_IP:6379
     slots:[10923-16383] (5461 slots) master
    M: SERVERSTRING42932 SERVER_2_IP:6379
     slots:[5461-10922] (5462 slots) master
   [OK] All nodes agree about slots configuration.
   >>> Check for open slots...
   >>> Check slots coverage...
   [OK] All 16384 slots covered.


11] Now we are done with deploying Redis cluster and we can run few basic commands to check the status by running redis-cli with -c parameter

    127.0.0.1:6379> cluster nodes

should see

    SERVERSTRING9dcc5 SERVER_1_IP:6379@16379 master - 0 9999999998108 1 connected 0-5460
    SERVERSTRING7174c SERVER_3_IP:6379@16379 myself,master - 0 9999999997000 3 connected 10923-16383
    SERVERSTRING42932 SERVER_2_IP:6379@16379 master - 0 9999999999000 2 connected 5461-10922


and

    127.0.0.1:6379> cluster info
    cluster_state:ok
    cluster_slots_assigned:16384
    cluster_slots_ok:16384
    cluster_slots_pfail:0
    cluster_slots_fail:0
    cluster_known_nodes:3
    cluster_size:3
    cluster_current_epoch:3
    cluster_my_epoch:3
    cluster_stats_messages_ping_sent:4309
    cluster_stats_messages_pong_sent:4259
    cluster_stats_messages_meet_sent:1
    cluster_stats_messages_sent:8569
    cluster_stats_messages_ping_received:4258
    cluster_stats_messages_pong_received:4309
    cluster_stats_messages_meet_received:1
    cluster_stats_messages_received:8568

Thanks and do share your feedback below.

Saturday, August 18, 2018

How2 install and configure MongoDB 4 on CentOS 7

1] To get started we first need add a yum repository

    # vi /etc/yum.repos.d/mongodb-org-4.0.repo

    add the following and save the file

    [mongodb-org-4.0]
    name=MongoDB Repository
    baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.0/x86_64/
    gpgcheck=1
    enabled=1
    gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc


2] Now proceed to install MongoDB

    # yum install mongodb-org

3] Lets configure MongoDB for security and take care of MongoDB warning.

    # vi /etc/mongod.conf

    and to enable security search for

    #security:

    and change it to

    security:
      authorization: enabled


    and save the file

4] We need to disable hugepage which affects the performance on MongoDB and also take care of MongoDB warnings. It is enabled by default in Centos7 and there are various ways to disable it and below is one of them

    # vi /etc/rc.local

    and append following lines at the end of the file and save the file

  if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
      echo never > /sys/kernel/mm/transparent_hugepage/enabled
  fi
  if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
      echo never > /sys/kernel/mm/transparent_hugepage/defrag
  fi


    for it to work we need to make rc.local executable and reboot the server

  # chmod +x /etc/rc.d/rc.local
  # reboot


5]  Login again to start and enable auto start for MongoDB

  # systemctl start mongod && systemctl enable mongod
  # systemctl status mongod


6] Now lets go ahead and connect to MongoDB and create admin user and database to store credential details. Please ensure secure password is used for admin.

    $ mongo
  > use admin
  > db.createUser({user: "admin", pwd: "Password", roles:[{role: "userAdminAnyDatabase", db: "admin"}]})
  > quit()


    and login with admin credentials

    $ mongo -u adminm -p --authenticationDatabase admin

    Now proceed to use MongoDB to create databases, collections etc.

Saturday, April 16, 2016

HowTo deploy Icinga2 Monitoring Server with Icinga Web 2 and Icingacli on CentOS 7.x 64 bit

To start with we require server ready with CentOS 7.x 64 bit

1) For Icinga2 installation we require server root access

$ sudo su -

now enter password to get the root prompt

#


2) Ensure SELINUX is disabled and to check the status run following command

# sestatus

and to disable read

Howto Disable SELINUX on CentOS 5.x / RHEL 5.x



3) Add EPEL repository as it is required for deploying Icinga2

# yum install epel-release


4) Add Icinga2 repository

# rpm --import http://packages.icinga.org/icinga.key
# curl -o /etc/yum.repos.d/ICINGA-release.repo http://packages.icinga.org/epel/ICINGA-release.repo
# yum makecache


5) Deploy Apache2 web server

# yum install httpd

to start and enable Apache2 server run following command

# systemctl start httpd && systemctl enable httpd

and to check the status of the server

# systemctl status httpd

open http://SERVERIPADDRESS/ url in the browser and you should get default apache2 page

create index.html file as this will take care of http warning initially displayed by Icinga2

# touch /var/www/html/index.html


6) Deploy MariaDB database server

# yum install mariadb-server mariadb

to start and enable MariaDB server run following command

# systemctl start mariadb && systemctl enable mariadb

and check the status of the server

# systemctl status mariadb

now lets secure Mariadb server

# mysql_secure_installation

Enter current password for root (enter for none):  (click on enter as no password is set by default)

Set root password? [Y/n] (click on enter to set root password)

New password: (type new root password)
Re-enter new password: (reenter new root password)

Remove anonymous users? [Y/n] (click on enter to remove all anonymous users)

Disallow root login remotely? [Y/n] (click on enter to disallow root login remotely)

Remove test database and access to it? [Y/n] (click on enter to delete test database)

Reload privilege tables now? [Y/n] (click on enter to reload user privileges)


7) Deploy PHP

# yum install php php-mysql php-ldap

update timezone in php.ini

# vi /etc/php.ini

search for [date] section and set timezone as per your location and save the file

date.timezone = "Asia/Kolkata"

restart apache2 server

# systemctl restart httpd

and check the status of the server

# systemctl status httpd


8) Deploy Icinga2

# yum install icinga2

to start and enable Icinga2 server run following command

# systemctl start icinga2 && systemctl enable icinga2


and check the status of the server

# systemctl status icinga2

by default all config files are stored in /etc/icinga2 directory

and by default Icinga2 enables three features and it can be checked by running following command

# icinga2 feature list


9)  Install Nagios plugins

# yum install nagios-plugins-all

by default plugins are installed in /usr/lib64/nagios/plugins/ directory and if plugin path is to be changed then it has to be updated for "const PluginDir" parameter in /etc/icinga2/constants.conf file

restart Icinga2 server

# systemctl restart icinga2


10) Configure IDO Mysql module

# yum install icinga2-ido-mysql

login into mariadb

# mysql -u root -p (at the password prompt provide root password)

at the prompt run following commands

> CREATE DATABASE icinga;

> GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON icinga.* TO 'icinga'@'localhost' IDENTIFIED BY 'icinga';

> use icinga;

to import database schema run following command

> \. /usr/share/icinga2-ido-mysql/schema/mysql.sql

> exit

now lets update ido-mysql.conf file as follows

# vi /etc/icinga2/features-available/ido-mysql.conf

Edit IdoMysqlConnection section as follows

object IdoMysqlConnection "ido-mysql" {
  user = "icinga"
  password = "icinga"
  host = "localhost"
  port = 3306
  database = "icinga"
  table_prefix = "icinga_"
  instance_name = "icinga2"
  instance_description = "icinga2 instance"
}


11) Enable web interface feature to send commands to Icinga2

# icinga2 feature enable command

check if command is added in Enabled feature list

# icinga2 feature list

restart icinga2 to take effect

# systemctl restart icinga2


12) Add apache user to icingacmd group

# usermod -a -G icingacmd apache

to check if apache user is added to the group run following command

# id apache

and output would look similar to

uid=48(apache) gid=48(apache) groups=48(apache),992(icingacmd)


13) Deploy Icinga Web 2 and Icingacli

# yum install icingaweb2 icingacli


restart icinga2 to take effect


# systemctl restart icinga2

create database for Icinga web 2

# mysql -u root -p (at the password prompt provide mariadb root password)

at the prompt run following commands

> CREATE DATABASE icingaweb2;

> GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON icingaweb2.* TO 'icingaweb2'@'localhost' IDENTIFIED BY 'icingaweb2';

> exit


restart apache2 server

# systemctl restart httpd

and check the status of the server

# systemctl status httpd


14) Now generate token to proceed with the installation using browser

icingacli setup token create

to recheck generated token run following command


# icingacli setup token show

open http://SERVERIPADDRESS/icingaweb2/setup URL in the browser to proceed with the installation

On 'Setup' page provide setup token and click on "Next"

On 'Modules' page click on "Next"

On 'Requirement' page ensure every option is green, scroll to bottom of the page and click on "Next"

On 'Authentication' page the default Authentication Type is set to Database, click on "Next"

On 'Database Resource' page provide details as follows

Resource Name : icingaweb2
Database Type : MySQL
Host          : localhost
Port          : 3306
Database Name : icingaweb2
Username      : icingaweb2
Password      : icingaweb2

Click on "Validate Configuration" and once successfully validated click on "Next"

On 'Database Setup' page for Username and Password provide MariaDB administrative username i.e root and its configured password and click on "Next"

On 'Authentication Backend' page keep default value and click on "Next"

On 'Administration' page create Icinga Web2 administrative user i.e admin, and set its password now click on "Next"

On 'Application Configuration' page keep default value and click on "Next"

On 'Configuration' page it will show all configuration provided earlier and would mention that Icinga Web 2 is successfully installed and click on "Next"

On 'Welcome' page click on "Next"

On 'Monitoring Backend' page keep default values and click on "Next"

On 'Monitoring IDO Resource'page provide details as follows

Resource Name : icingaido
Database Type : MySql
Host          : localhost
Port          : 3306
Database Name : icinga
Username      : icinga
Password      : icinga

Click on "Validate Configuration" and once successfully validated click on "Next"

On 'Command Transport' page keep default values and click on "Next"

On 'Monitoring Security' page keep default values and click on "Next"

On 'Finish' page click on "Finish"

On 'Congratulations' page on the the right hand side, click on 'Login to Icinga Web 2'. On login page provide Icinga Web2 admin user name and password to go to Icinga Dashboard page.

With this you have successfully deployed Icinga2 Server along with Icinga Web2 and Icingacli module. Now you can proceed with configuring servers to be monitored that will be covered in the next blog post.

Tuesday, February 17, 2015

Howto disable / enable autostart of MariaDB 5.x & 10.x at system startup on Ubuntu 14.04.1 64bit

To disable autostart of MariaDB during system boot / startup do as follows

$ sudo update-rc.d -f mysql disable


and to enable it again do

$ sudo update-rc.d mysql enable

or

$ sudo update-rc.d mysql default

Monday, February 16, 2015

Howto change default Python version 2.7 to 3.4 in Ubuntu 14.04.1 LTS 64bit

Ubuntu 14.04 LTS comes with both Python version 2.7 and 3.4. The default Python version is 2.7 and update-alternatives is not configured to switch Python versions

$ sudo update-alternatives --config python
update-alternatives: error: no alternatives for python

1. So to configure alternatives for Python 2.7 & 3.4 we need to configure update-alternatives as follows

$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1
update-alternatives: using /usr/bin/python2.7 to provide /usr/bin/python (python) in auto mode

$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.4 2
update-alternatives: using /usr/bin/python3.4 to provide /usr/bin/python (python) in auto mode

2. And now you can switch between Python versions

$ sudo update-alternatives --config python
[sudo] password for linuxuser: 
There are 2 choices for the alternative python (providing /usr/bin/python).

  Selection    Path                Priority   Status
------------------------------------------------------------
  0            /usr/bin/python3.4   2         auto mode
* 1            /usr/bin/python2.7   1         manual mode
  2            /usr/bin/python3.4   2         manual mode

Press enter to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/bin/python3.4 to provide /usr/bin/python (python) in manual mode

Tuesday, January 28, 2014

Howto access data from MS Access .MBD file on Ubuntu 12.04.4 64 bit

There are very few tools / guides available to access data from MS Access .mdb file on GNU/Linux and following steps would help you quickly install MDB tools and access data from MS Access .mdb file on your system.

1) Install MDB tools

$ sudo apt-get install mdbtools

2) Install MDB graphical interface

$ sudo apt-get install mdbtools-gmdb

3) Now from command prompt run

$ gmdb2

and select .mdb file to access tables, queries, forms, reports macros & modules.

4) Also you can use other mdb tools commands prefixed with 'mdb-' from command prompt