Especially with the rise of social media, cell phones and online shopping there is a lot of data out there about people, what the do and where they do it. A lot of people are scared about having that data about themselves out there, being controlled, sold and manipulated by organizations.
In my opinion all of that data is actually pretty harmless primarily because I don’t believe others have a lot to gain to know all that information about me, I’m not that interesting. Even if this wasn’t the case, organizations that have that data about doesn’t scare me, what does scare me is the organizations that take that data and convert it to Information.
Data is just entries in a database, even if it is a history of where I have been over the last 10 days (I’ll give you a hint, somewhere in Monterey, CA). But when that data is combined with other data, such as the kind of car I drive, who my relatives are, where I buy groceries, etc. that is where it gets valuable to companies.
But the process for converting all of that data into information is not an easy one. it’s something I’m interested in and want to learn more about. The type of people who do this are considered Data Analysts, Data Scientists and a variety of others. And, from what I hear, it’s not an exact science, it’s a lot of guesswork, trying to combine data from several different sources and doing a lot of data cleanup. Still, I’m interested in going down that road.
My initial experience with this was when I did ETL (Extract, Transform & Load) on the database from a large application I was managing just because I could. The application was the LMS (Learning Management System) at the Naval Postgraduate School. It being a transactional system, I used an open source tool called Kettle, a tool in the Pentaho suite, to extract data from the transactional database into a reporting database. From there I was able to use Report Designer to generate reports that management needed.
I really enjoyed the project because I created an achievable goal for myself and I achieved that goal while learning new things along the way. Although I try to do that in many aspects of my work and personal life, for some reason I really enjoyed data manipulation and would like to be able to do more of that moving forward.