Big Data

The book I read to research this post was Big Data For Dummies by Judith Hurwitz et al which is a very good book which I read at http://safaribooksonline.com . Big Data is still in its relative infancy and consists of two types of information, machine generated and human generated. Machine generated might be weather data or sensor data from an aircraft where as human data might be something a person said about your product on facebook or search information on a your website. The biggest problem isn't storing all this information but accessing relavant data hidden amongst it and doing something with the data. At one time companies used databases that had to be written specifically for what they wanted to do. Then came relational databases like Oracle & Microsoft SQL Server. These could be adapted using languages like SQL to do lots of things. Now we are going into another era where most information will be stored either on a hybrid cloud or cloud provider. Companies will have virtual servers where 1 or 2 servers will have to do the job of several servers and their massive hard drives will be partitioned to run several server roles possibly with several operating systems. When people go on the internet the computer will be a virtualized workstation with limited permissions to limit the damage a virus can do. Companies will use big data organisations Amazon Web Services to store huge amounts of data but at a fraction of the cost an equivalent server would cost. There are a lot of open source databases like Hbase & PostGRE  where generally they are free to use but in some cases you can buy a technical support package. There are some big players in all this like IBM, SAS & Oracle. You have to shop around for what is best for your requirements. Amazon Web Services are regarded as one of the cheapest and best and are the biggest cloud provider in the world. Of course the hardest part is integrating everything so it works. There are some packages and services which are specific to a particular type of business like SPSS for Psychology Statistics. I thoroughly enjoyed reading this book which is on a very interesting subject which as time goes on is going to get more and more important to businesses and organisations.