Setting Up Your Website and Email on Digital Ocean

This blog is about how I went about setting up my website and email on Digital Ocean. It is a continuation from my previous article about the decision to move from shared hosting to a VPS.
My requirements are:

  1. Run my personal website
  2. Run my own email server
Note: The following guide assumes that you have basic understanding of VPS, Ubuntu, Webservers, Database, Webhosting etc
Step 1 : Create A Digital Ocean Droplet
You have to select the size of your Droplet, the physical location and an operating system. I chose the 1GB package, Singapore as the location and Ubuntu as my Operating System. Click here to get $10 free credit from Digital Ocean. It is enough to let your try out for a month.
Digital ocean lets you one click install applications like – LAMP/LEMP Stack, WordPress on Ubuntu etc.
DO_Droplet
DO_Region
DO_OS
I did not choose the preconfigured stacks because sometimes it creates issues while installing your applications.
Once your Droplet is ready, the IP address, root password is mailed to you.

Step 2 : Install Webmin and Virtualmin

Webmin is a web-based interface for system administration for Unix. Using any modern browser you can setup user accounts, Apache, DNS, file sharing and so on. Webmin consists of a simple web server, and a number of CGI programs which directly update system files. The web server and all CGI programs are written in Perl.
Installing webmin is pretty straightforward. Both Digital Ocean and Webmin has good guides on how to do this. In short, you add the repository and install webmin.
Edit the /etc/apt/sources.list file using
sudo nano /etc/apt/sources.list
Add the following repos
deb http://download.webmin.com/download/repository sarge contrib
deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib
DO_source.list_
Download and install the GPG key with which the repository is signed
wget http://www.webmin.com/jcameron-key.asc
apt-key add jcameron-key.asc
Install Webmin using the following commands
apt-get update
apt-get install webmin
Once installed, Webmin is accessible at http://your_ip_address:10000 . You can log in with as root or any other user with root previliges and access webmin.
DO_Webmin_Login
Webmin alone, however, was not enough. I needed Virtualmin on top of Webmin.
Virtualmin is a web hosting control panel based on Webmin. You can manage your virtual domains, mailboxes, databases, applications, and the entire server, from the Virutalmin interface.
You can download the installer script using
wget http://software.virtualmin.com/gpl/scripts/install.sh -O /root/virtualmin-install.sh and
install it using
sh /root/virtualmin-install.sh
Once installed, Virtualmin can be accesssed at http://your_ip_address:10000.
The Post-Installation Wizard will run and it is self-explanatory.

Step 3 : Creating your Virtual Server

Login into Webmin and click Create Virtual Server
DO_Create_Virtual_Server_Virtualmin
Fill in all the relevant fields and enable the features you might need and create the server.
Virtualmin is pretty solid. It creates your home directory, DNS entries etc as you create the Virtual Server.
Step 4 : Update your DNS entries.
This has to be updated at two places
  • With your Domain provider
  • In the Digital Ocean control panel

Step 5 : Install add-ons

I use Roundcube for emails, Phpmyadmin to managed the MySql databases and Virutalmin makes it easy. Click on install Scripts. Choose your script and the location where you want it installed.
DO_roundcube
If you had done the above, then your email will be accessbile at
http://your_ip_address/roundcube
You can create users/email accounts from Virutalmin and log in via Roundcube.

Step 6 : Secure Your Server

This is a tricky part and there are a bunch of things one can do. The most basic ones include
  1. Disable remote login for root
  2. Secure Mysql and Phpmyadmin
  3. Implementing intrusion detection
  4. Use SFTP instead of FTP
 This comprehensive article on Linux server safety by Digital Ocean is a great place to start.
https://www.digitalocean.com/community/tutorials/an-introduction-to-securing-your-linux-vps

Moving From Hostgator To Digital Ocean

HG to DOI recently moved from Hostgator to Digital Ocean and this how/why I did it.
This is something I have been wanting to do for quite sometime. I tried a couple of years back to shift to a VPS but reverted to a managed host because VPS was expensive and it was time consuming.
With the likes of Digital Ocean, Linode and the grand old Amazon AWS offering more and better (read easier) options now, I decided to make the jump, yet again.
Also, I should mention that there is nothing specifically wrong with Hostgator. They have good hosting services with excellent customer service. But I was about hit the limits of their shared hosting and their VPS offerings are quite expensive.
So here is what I needed :

  1. Run a personal website plus a few test websites.
  2. Run an email server that will take care of my personal email
  3. Have some kind of a personal data backup system 
  4. Run experiments (Twitter client, scraping data, RSS aggregator etc)

Why Digital Ocean?
Two simple reasons :

  1. Their extensive documentation. For someone like me who doesn’t have a lot of experience of dealing with Ubuntu servers, their guides are quite helpful. They are not 100% complete but it is a GREAT starting point.
  2. Easy to use – You can create and deply a droplet with the OS (along with different stacks if you want) within seconds.

What Did I do ?
First off, I experimented with some flavours of Linux (Ubuntu/CentOS), a range of Control Panels (Ajenti/Zpanel/ISPConfig/Centos’ Control Panel) and some server monitoring tools like Cacti.¬†
The $10 free credit from Digital Ocean was put to full use. I deployed, destroyed and delpoyed again, more than a dozen times, before settling for the following. 

  1. Ubuntu 14.04 with the AMP Stack
  2. Webmin (with Virtualmin) for Control Panel (This best suited my needs, though I liked Ajenti too)
  3. Postfix/Dovecot Email set up
  4. Periodical backups (of the websites and email) to Amazon S3
  5. Cloudflare  for DNS
  6. Server monitoring using New Relic (experimental – I am not sure if I actually need this but subscribed to a free account)

What’s next?
I am hearing that apache isn’t ideal and I should shift to Nginix. But for now I am sticking to this before I learn more about Nginx.¬†
I’ve also been reading about how a control panel is another level of complexity and I should stick to the command line. But this is sometime away. Right now the Control Panel lets me enable/disable services easily. So will use it for sometime.
To be continued…. Next up – How Did I Set Up The Server, the Control Panel, Move The Data, Set up Back ups and set up the monitoring.

Twitter Share Link Creator

A share link creator is an ubiquitous tool, especially for those working in digital marketing.
There are a couple of websites that have scripts that help generate these links. A google search will throw a bunch of websites that help you generate such links and the one I like the most is http://mashe.hawksey.info/sharegen.php (largely for its simplicity). However, there is a problem with these websites.
They don’t generate the perfect tweet you want, one that includes the text, a hashtag and an URL. Most these services don’t generate a tweet link with a hashtag and the ones that support hashtags don’t append the URL.
So, I decided dig this a bit more to figure out the best way to generate the twitter share links, that can support hashtags and URLs as well.
The solution I found was very simple. The best way is to construct the URL yourself.
Below is the structure

https://twitter.com/intent/tweet?
url=
text=
hashtags=

A sample tweet would be

https://twitter.com/intent/tweet?
url=www.surendran.info/twittershare
text=How to generate a twitter share url with
hashtags=hashtag,twitter,social media

The final URL will look like
https://twitter.com/intent/tweet?url=http://www.surendran.info/twittershare&text=How to generate a twitter share url with hashtags?&hashtags=twitter,socialmedia,hashtags
To make tracking easier, you can use bitly. The short link of the above long url would be something like http://j.mp/19SajuU
However, this method isn’t 100% perfect either. Some challenges I identified in this include

  • You cannot alter the structure of the tweet. It remains text, url, hashtags
  • You cannot insert hashtags in the text
  • If you add special characters in the text it doesn’t work. Just ‘?’ works.
  • You have to stick to the syntax. Things like spaces, a extra ‘/’ at the end of the URL etc will result in a wrong URL or a truncated tweet.

Lemme know if this works and if you have more ideas to generate better share links for twitter.
PS: The share URL makes use of the Twitter web intent. In addition to the URL and hashtags, it supports a couple of other parameters too. Read more here

Independent MPs in the LokSabha

In the last 60 years, India has seen 4,665 MPs being elected to the Lok Sabha from over 140 political parties.  4242 of them were men and 316  women.
Information about our Members of the Parliament both in the Lok Sabha and the Rajya Sabha, was recently uploaded in the Data Portal of India.  I consolidated all the basic information of the Members of the Parliament into one single .CSV file and converted it into a Mysql database (both of them can be found here).  I then ran specific queries to get the data I wanted to analyse and used google fusion tables to visualise them. Below are the first three visualisations.
1) Independent Members of the Parliament in the Lok Sabha from different states.
2) Independent Members of the Parliament organised by the sessions of Lok Sabha
3) Male/Female ratio in the Lok Sabha
*Some constituencies have changed and now are in different states as compared to their original state. I’ve counted them according to the present state where the constituency falls.
I plan to create more such visulaisations out of this dataset and add more data to this dataset too. If you want a specific visulaisation or data to be added, let me know and I will try to do it.
Independent Members of the Parliament – Statewise


Independent Members of the Parliament – Lok Sabha wise


Members of the Parliament – Male/Female

Male
4242
Female
316
NA
7

 

Is the world population growing?

Is the world population growing? If yes, how long will it grow like this and does religion have a role to play in this?
In this amazing video (posted a year ago), Hans Rosling explains where the world population is headed and what role does religion play in it. This is a fantasitic example of what we can do with open/public data and how it can give insights unlike never before.

You can find more about Hans Rosling and his work here

3 reasons why a video goes viral on Youtube

Kevin Allocca, YouTube's trends manager, shares the 3 reasons why a video goes viral. In this video he takes the example of three videos and goes on to explain how, tastemakers, communities of participation and unexpectedness are the reasons on why the videos went viral online.

Another interesting thing to note in his talk is the role played by twitter. Some of these videos were quite old by the time they went viral and how a tweet from a tastemaker resulted in the video going viral. Do watch this interesting talk.