Automating MYSQL Backups to Dropbox in Ubuntu – 2 simple steps

You can easily automate daily backups of your MYSQL database and other files, to your Dropbox account in 2 simple steps.
The scare happened a few weeks ago. I did a regular Ubuntu OS update and a few MYSQL packages also upgraded. Rebooted the machine and viola! MYSQL wont start and I’d no backup. (Stupid yet true!). With great difficulty and lots of help from pros, I managed to restore the database with minimal data loss.¬†I was extremely lucky but a risk not worth taking. So here’s a very easy method.

Step 1: Install Dropbox

You can install Dropbox (headless) using a single command. (If you don’t have a dropbox account, sign up here)

32-bit:¬†cd ~ && wget -O – “https://www.dropbox.com/download?plat=lnx.x86” | tar xzf –
64-bit:¬†cd ~ && wget -O – “https://www.dropbox.com/download?plat=lnx.x86_64” | tar xzf –

Next, run the Dropbox daemon from the newly created .dropbox-dist folder using

~/.dropbox-dist/dropboxd

If you’re running Dropbox on your server for the first time, you’ll be asked to copy and paste a link in a working browser to create a new account or add your server to an existing account. Once you do, your Dropbox folder will be created in your home directory.

Step 2: Create the MYSQL Dump & add a Cronjob

It is quite easy to create a backup of your MYSQL database. I created mine as followsL

$ mysqldump -u’username’ -p’password’ database_name > database_name.sql

This will create a .sql file in the Dropbox folder, which will automatically get synced with your dropbox account. THAT’S IT!
To automate $ crontab -e

Add the line: 0 6 * * *¬†mysqldump -u’username’ -p’password’ database_name > database_name.sql

Now every morning at 6 am, the cron job will run, creating a backup file of my database and get synced with dropbox.
Note: This will replace the file everyday and you will only have the most recent backup.
If you want a daily backup append the file name as

mysqldump -u’username’ -p’password’ database_name > database_name-$(date “+%d-%m-%Y”).sql

Other cool things you can do
So now that MYSQL backup is taken care of, I also wanted to back up some code and other files. I simply created symlinks and Dropbox works like a charm

$ ln -s /home/code ~/Dropbox
$ln -s /var/www/html ~/Dropbox

Again, this command will create the most recent copy of the files in dropbox. If you want to automate a daily backup, you can add another cronjob like above. Example below:

0 6 * * * zip -r ~/Dropbox/html.zip /var/www/html

Foreign Direct Investment in India – Visualization

Foreign direct investment (FDI) is a major source of non-debt financial resource for the economic development of India. FDI can be in the form of either equity or debt. Recently the data of the last 15 years of FDI Equity Inflows was published on data.gov.in. Here are a few interesting insights & visualizations. (All values in millions of USD)
In the last 15 years, the amount inflows has steadily or even exponentially grown, especially after 2005.

Services Sector which includes — finance, banking, insurance, non Finance/business, outsourcing, R&D, Courier, Technoloy Testing and Analysis, and¬†others — received the highest FDI in equity over the last 15 years.

Though cumulatively the FDI inflows have grown, not all years have seen an increase. 2015-16 is the highest ever.

FDI was received under 63 different sectors, over this period. With 61 sectors receiving FDI, 2007-08, 2014-15 was the year with the highest number of sectors receiving investment.

IN 2001, only 40 sectors received FDI. Over the years, the number of sectors have grown and the last year, it stood at 58.

Not all years saw the same amount of investment. In some years, there was a dip of more than 50% and in some other years, there was an increase of 50%.

Can Trees Talk To Each Other?

In her fascinating talk about her 30+ years of research in Canadian forests, Suzanne Simard shares an astounding discovery ‚ÄĒ trees talk, often and over vast distances.
Through a vast underground network, trees communicate with each other. How exactly does this happen in the forest?
To quote Suzzane

You see their reproductive organs when you walk through the forest. They’re the mushrooms. The mushrooms, though, are just the tip of the iceberg,because coming out of those stems are fungal threads that form a mycelium, and that mycelium infects and colonizes the roots of all the trees and plants. And where the fungal cells interact with the root cells,there’s a trade of carbon for nutrients, and that fungus gets those nutrients by growing through the soiland coating every soil particle. The web is so dense that there can be hundreds of kilometers of mycelium under a single footstep. And not only that, that mycelium connects different individuals in the forest, individuals not only of the same species but between species, like birch and fir, and it works kind of like the Internet.”

Wondering if Steven Spielberg happened to reach her research before making Avatar. Do watch it:

Last 9 Tamilnadu assembly elections explained in 9 charts

9 interesting patterns emerge when the 2016 TamilNadu assembly elections data is analysed and compared with that from the last 30 years.
In the last 32 years, an incumbent government has never won an election. Though the pattern has been continuing, the difference has narrowed down.
Number_of_seats
Though women formed almost 50% of the electorate, the number of women voting were marginally less than men.
The number of people voting has been consistently going up in the last 15 years. 78% of the electorate voted in the 2011 election. The election commission is gunning for 100% polling in the 2016 elections.

Barring 1996 elections, the DMK alliance has never got more than 50% of the vote share. In fact, in the 2006 election, DMK received lesser votes than AIADMK though they won more seats.
Percentage_of_votes
Regional parties get the lions share of votes and win. The trend is only becoming stronger in the recent years with parties like PMK and DMDK beginning to make a dent.
Number_of_seats_regional
The national parties have been trying to make a comeback in recent years but collectively they won less than 10% of the seats in the last election. BJP has never won more than 4 seats in the last 9 elections.
Corresponding to the number of seats, the vote share of the national parties have also been on a falling trend. They are trying to make a comeback in the recent years.
82% of all the candidates who contested in the last election fortfeited their deposit. 1989 elections saw the highest number of candidates (3046).

Though women form nearly 50% of the electorate, their representation in the assembly is very less. 1991 saw the most number of women (32) being elected to the assembly.
 

Free VPS/Cloud Servers

Digital Ocean, Amazon Web Services (AWS), Google Cloud Platform, Microsoft Auzre, RackSpace and almost every major cloud computing company gives free/trial plans. While it is business strategy, it may be useful for you if you are looking at something for a short span and not willing to pay a bomb for it. In this post, I am only compiling a set of such plans and would add/edit as things evolve.

Company Free Plan Offered Duration
Digital Ocean  $10 Free Credit Till you exhaust the credit
Amazon Web Services Enough to run a website for free for a year. (https://aws.amazon.com/free/)  1 Year
Microsoft Azure $200 (~Rs. 12,000) credit (https://azure.microsoft.com/en-in/offers/ms-azr-0044p/) 1 month
Google Cloud $300 (https://cloud.google.com/free-trial/) 2 months
RackSpace
$600 free credit Р $50/month for 12 months to use toward the cloud services.
1 year

I’m a big fan of digital ocean but I equally like AWS. I’m currently experimenting with Azure and planning to try the Google Cloud soon. I hope to write detailed tutorials for each.
1) Getting started on digital ocean.
 

How to setup a Raspberry Pi & WiFi without a monitor (Headless) using a Macbook

In this article, I’m going to share my experince of setting up a Raspberry Pi headless (without a monitor) using a Macbook.
In this 3 step tutorial, we will go through how to prepare your Raspberry Pi, setting it up through a wired connection and finally setting up the WiFi connection.
Things you will need:

  1. Raspberry Pi
  2. USB WiFi Adapter
  3. Micro SD Card
  4. Micro USB Cable
  5. Ethernet Cable

Software to download

  • Raspbian Operating System
  • Pifiller

Assumptions

  • You have a Wifi router with Ethernet ports
  • You have basic working knowledge of Linux (Debian).

Step 1 – Preparation
Open PiFiller and it will ask you to select the OS image. Choose the .img file.
When prompted, insert the SD card (using an adaptor) and proceed.
The content of the card (if any) would be erased the Raspiban files would be copied.
After trying out a couple of different ways to do this, I can now vouch that PiFiller is the best way to install Raspiban OS on a SD card.
Step 2 – Booting the PI and accessing it via SSHRaspberry Pi connected to ethernet and power
Insert the SD Card into the slot
Plug in the Ethernet Cable and connect it to the Router
Power up using the Pi
Before you can access your Raspberry Pi, you need its IP address. Below are the steps to get the IP address of your Raspberry PI and log in via SSH

  1. Log in into your router and check out what IP addresses it is giving out. Mine gives out IP addresses starting from 192.168.0.100. You will find this under DHCP settings of your router.
  2. Open up a terminal window and type nmap -sn 192.168.0.100/24
  3. From the list you should be able to identify the IP address that corresponds to your RaspberryPi
  4. type ssh [email protected]{raspberry_pi_ip_address}
  5. When prompted for password, enter ‘raspberry’

Your screen should look something like this. I’ve highlighted the commands you will be running.

[email protected]:~$ ssh [email protected]
The authenticity of host ‘192.168.0.107 (192.168.0.107)’ can’t be established.
ECDSA key fingerprint is fc:13:a8:50:4d:ae:d8:7c:6d:3c:0b:3b:b1:58:d8:25.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘192.168.0.107’ (ECDSA) to the list of known hosts.
[email protected]’s password:
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Oct 30 05:17:06 2015
[email protected] ~ $

login
Step 3 – Setting Up Wifi
If you have completed step 2, then you are logged into your Raspberry Pi. Now plug in your USB dongle run the following command.

[email protected] / $ sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

There may be a couple of lines in that file. Add the following

network={
ssid=”Your WiFi Connections’ Name”
psk=”Your WiFi Password”
}

Save the file. Now run the command ifconfig. If everything is okay, you should see the IP address assinged to your Raspberry Pi under wlan0.
Raspberry Pi ip addres of Wifi adapter
That’s it. Restart your Raspberry Pi and remove the ethernet cable. Before you begin exploring, don’t forget to run apt-get update and apt-get upgrade.

Setting Up Your Website and Email on Digital Ocean

This blog is about how I went about setting up my website and email on Digital Ocean. It is a continuation from my previous article about the decision to move from shared hosting to a VPS.
My requirements are:

  1. Run my personal website
  2. Run my own email server
Note: The following guide assumes that you have basic understanding of VPS, Ubuntu, Webservers, Database, Webhosting etc
Step 1 : Create A Digital Ocean Droplet
You have to select the size of your Droplet, the physical location and an operating system. I chose the 1GB package, Singapore as the location and Ubuntu as my Operating System. Click here to get $10 free credit from Digital Ocean. It is enough to let your try out for a month.
Digital ocean lets you one click install applications like – LAMP/LEMP Stack, WordPress on Ubuntu etc.
DO_Droplet
DO_Region
DO_OS
I did not choose the preconfigured stacks because sometimes it creates issues while installing your applications.
Once your Droplet is ready, the IP address, root password is mailed to you.

Step 2 : Install Webmin and Virtualmin

Webmin is a web-based interface for system administration for Unix. Using any modern browser you can setup user accounts, Apache, DNS, file sharing and so on. Webmin consists of a simple web server, and a number of CGI programs which directly update system files. The web server and all CGI programs are written in Perl.
Installing webmin is pretty straightforward. Both Digital Ocean and Webmin has good guides on how to do this. In short, you add the repository and install webmin.
Edit the /etc/apt/sources.list file using
sudo nano /etc/apt/sources.list
Add the following repos
deb http://download.webmin.com/download/repository sarge contrib
deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib
DO_source.list_
Download and install the GPG key with which the repository is signed
wget http://www.webmin.com/jcameron-key.asc
apt-key add jcameron-key.asc
Install Webmin using the following commands
apt-get update
apt-get install webmin
Once installed, Webmin is accessible at http://your_ip_address:10000 . You can log in with as root or any other user with root previliges and access webmin.
DO_Webmin_Login
Webmin alone, however, was not enough. I needed Virtualmin on top of Webmin.
Virtualmin is a web hosting control panel based on Webmin. You can manage your virtual domains, mailboxes, databases, applications, and the entire server, from the Virutalmin interface.
You can download the installer script using
wget http://software.virtualmin.com/gpl/scripts/install.sh -O /root/virtualmin-install.sh and
install it using
sh /root/virtualmin-install.sh
Once installed, Virtualmin can be accesssed at http://your_ip_address:10000.
The Post-Installation Wizard will run and it is self-explanatory.

Step 3 : Creating your Virtual Server

Login into Webmin and click Create Virtual Server
DO_Create_Virtual_Server_Virtualmin
Fill in all the relevant fields and enable the features you might need and create the server.
Virtualmin is pretty solid. It creates your home directory, DNS entries etc as you create the Virtual Server.
Step 4 : Update your DNS entries.
This has to be updated at two places
  • With your Domain provider
  • In the Digital Ocean control panel

Step 5 : Install add-ons

I use Roundcube for emails, Phpmyadmin to managed the MySql databases and Virutalmin makes it easy. Click on install Scripts. Choose your script and the location where you want it installed.
DO_roundcube
If you had done the above, then your email will be accessbile at
http://your_ip_address/roundcube
You can create users/email accounts from Virutalmin and log in via Roundcube.

Step 6 : Secure Your Server

This is a tricky part and there are a bunch of things one can do. The most basic ones include
  1. Disable remote login for root
  2. Secure Mysql and Phpmyadmin
  3. Implementing intrusion detection
  4. Use SFTP instead of FTP
 This comprehensive article on Linux server safety by Digital Ocean is a great place to start.
https://www.digitalocean.com/community/tutorials/an-introduction-to-securing-your-linux-vps