Back to the basics

Photo Alban Gonzalez @ Flickr

As I have written before (There are no shortcuts) sometimes is good to return to the basics and take it as reflection and improvement of your knowledge. All the programming languages and tools have many options how it could be extended and used together with libraries, packages, and plugins. But do you know how it works inside, what's under the hood? How to setup properly the parameters to get the best results from such tools?

What I want to do today is continue with this idea, but not on the philosophical level as last time, but with concrete examples. I give you some hints which I have tried by myself.

Machine learning

What could you do with machine learning? Whenever you install Python or R language you get such libraries like NLTK or Scikit-learn for Python or e1071. But do you see under the hood? Do you know how it's implemented inside or it's just a black-box for you? Do you want to know? So, what you can do:

• Start with the implementation of the simplest machine learning methods by yourself. For instance, k-Nearest Neighborhoods is the really good example and you can do it in half an hour from scratch or Decision Tree method or even Random Forest method. 
• Take a look how it's implemented in existing code, for example, you can take a look into NLTK Python code and learn how they invent particular process

Natural Language Processing (NLP)

The machine learning methods used for NLP are the similar one which you know already from the previous study of basics (kNN, Decision Tree, Random Forest) or they are quite extensive to implement them from scratch. The same it is with n-grams and textual document models, so what to do then:

• You will not most likely implement word2vec, but what you can implement simply is bag-of-words or with little bit more effort is possible to implement tf-idf.
• This code is also accessible in NLTK library where you can take a look and for instance, find that tf-idf is implemented with smoothing.
• With n-grams when you implement it (see my post Computing text conditional entropy with uni- and bi-grams in R and Python) you can play further and calculate pointwise mutual information and find the collocations and associations between words with that.

I will return to this topic whenever I have more experience to share, but if you will search you will definitely find similar sources about other topics of your interests such as Big Data, Visualization, Statistics, etc.

Continuous and discrete time series visualized together

The second publication for Gigascience is nearly finished and the deadline for third publication is approaching really quickly so I needed to start with data analysis. Because I had just a few ideas about how to analyze experiment data I asked the colleague from University about advice and sent her data. It was a few months ago.

Recently I got back from her the ideas and notes and also the recommendation for the book about time series analysis. As usual, it's good to start with several visualizations and from simple things to complex. So, I want to share some notes about it.

Data

As the output from two parts of the experiment were recorded two pairs of datasets for:

• Experiment #1 with Fitbit Charge HR
• 1029 tweets with average of 20.56 tweets per day, 
• 411 799 records of HR with frequency of 6 - 7 records per minute
• Experiment #2 with Peak Basis
• 1017 tweets with average of 20.32 tweets per day,
• 69 909 records of HR with frequency of 1 record per minute

All the tweets are just a records of sentiment and they are evaluated by hashtags #p for positive and #n for negative sentiment by experiment participant. As a part of publication, the sentiment will be extracted via machine learning methods and compared to human evaluation.

Visualization

I took data from experiment #1 since there are more records of HR and did following data wrangling:

• Tweets respectively evaluated sentiment needs to be extended to not represent just a point in time, but the whole window. For this, we need to define "breaking point" between two consecutive going sentiments. It's simply in the middle. This is visualized in the following figure as the gray line with the dots on it where positive sentiment = 150 of HR bpm and negative sentiment = 0 of HR bpm (scale adjustment was done, otherwise sentiment is represented by +1 and -1). Dot's here representing true records, the line is an extrapolation to the time window.
• Heart rate is drawn with another gray line representing rapid changes over time. I have applied to it simple moving average (SMA) method to get a much smoother line for further analysis. And also cut this SMA line by extrapolated windows of sentiment where the red part represents negative sentiment and blue part positive sentiment.

Here is the result:

It looks really promising, but that's all so far. I have few ideas how to continue, but since this is a part of my third publication I will publish it first and then link the publication itself and also complete code on Github. Sharing this was just about the idea how to start with the visualization of continuous and discrete time series together.

Every Data Scientist must only pay taxes and die, rest is just optional...

Picture from cacm.acm.org

I am just wondering. There are many articles about what every Data Scientist MUST know and do to be real Data Scientist. There are many articles about MUST not do as Data Scientist. What real Data Scientist MUST read and so on and so on. And honestly I don't care. Why is that so?

Research

Let's take the last one must: "What Data Scientist must read" list. I just briefly took several results from Google, here is the list of 10 of them:

• https://www.quora.com/What-is-a-list-of-books-that-every-data-scientist-should-read
• http://bigdata-madesimple.com/20-must-read-books-for-budding-data-scientists
• http://www.analyticsvidhya.com/blog/2015/06/infographic-read-books-analytics-data-science
• http://www.datasciencecentral.com/profiles/blogs/15-books-every-data-scientist-should-read
• http://www.mastersindatascience.org/blog/7-must-read-books-for-the-budding-data-scientist
• http://www.forbes.com/sites/gilpress/2015/03/31/3-recent-books-for-budding-data-scientists-and-business-executives-involved-with-big-data-analytics
• https://www.umbel.com/blog/big-data/must-read-big-data-books
• http://analyticstraining.com/2014/five-must-reads-for-data-scientists
• https://www.quora.com/What-are-the-must-reads-for-data-science-and-machine-learning
• http://www.analyticsvidhya.com/blog/2015/06/infographic-read-books-analytics-data-science

And what is this quick research good for? From this circa 80 books and list of several articles you get list of subjective chosen resources which lead practically to nowhere. Just several books repeating like famous Nate Silvers Signal and Noise and of course some R or other Cookbooks. So, what is conclusion?

Conclusion

This lists of books which someone else read leads me always to Vincent Granville's article Fake data science. And what you need to take from it? Pick any book you need for your field of expertise. And what should be your field of expertise? Choose some project, doesn't matter if your personal one, school or for instance from Kaggle.com. And follow up approaches which you need to for goal achievement, then pick book, course to support your path towards this goal. And by real work, life experience you will sooner or later become Data Scientist.

Course: Exploratory Data Analysis

What do you imagine under Exploratory Data Analysis?

After a while I decided to continue with one of the first specializations on Coursera which I started more than year ago. This Data Science specialization contains 4th course with the same name as the topic which I was asking in opening question: Exploratory Data Analysis.

I was wondering how much effort I need to include and how much knowledge I can get. After 2 years (yes, anniversary of this blog is 18th of December 2015) with Data Science study path and more than 1 and half year with intensive usage of R language and couple of statistics and R courses. What can surprised me, right?

Nothing and something. I mean this both in positive way and will tell you why.

Computing text conditional entropy with uni- and bi-grams in R and Python

During my first semester of PhD study I have implemented solution for computing conditional entropy over text where each word (including interpunction) was on separate line. I would like to share this example, because it is simple and good for get basics (information theory, R and Python programming).

Conditional entropy is defined as (source is Wikipedia):

Just note, this computation determine the conditional entropy of the word distribution in a text given the previous word.

According line 4 in previous formula, I have to compute p(x,y), which is the probability that at any position in the text you will find the word x followed immediately by the word y, and p(y|x), which is the probability that if word x occurs in the text then word y will follow.

Or according line 5 in formula, I can use probability p(x,y) twice and calculate p(x) which is the probability of single word appearance in the text.

Big Data and Data Science study - presentation

During our Friday 26th of September company meeting I talked about topic in title with presentation. I think this presentation could be useful not only for audience which was there but for anyone.

Here is description what is content of presentation about:
Big Data and Data Science study with subtitle "study materials and online courses" is little bit more over 40 slides presentation about 10 domains of Data Science covered by online free and paid MOOC courses, study materials and free books.

Based on my almost year of study, investigate and collect of materials, tutorials, courses, books, links, etc. I have prepared distillation of the best in this short presentation.

Of course list is not full, because there is always something new, undiscovered and better than before. But it contains the most important information for those who want to start or don't know where exactly follow up when they already begun.

Follow up to the Speaker Deck site and you can download presentation as PDF which is quite useful when you consider functional links over all presentation.

Interviews to Data Scientist role

After almost year of self study of Data Science courses via Coursera, DSE program and another MOOC pages I had chance to go to Data Scientist role interviews. So far two.

First one was a couple weeks ago I was in interview to the Researcher / Data Scientist position. Mainly about prediction models, math, statistics and data mining/machine learning. Mostly in R connected with Hadoop and some other mathematical tools. This was the first for such position.

Second one was a couple days ago and it was interview to the purely Data Scientist position. Including all data science work from many different sources with project driven or data driven approach, it depends. This was most recent experience and I hope not a last one.

My study list for Summer time 2014

I was thinking what to study during Summer time. I need to improve my insight into Big Data, especially Hadoop and its fundamentals like HDFS, HBase and MapReduce. At least from non-Java developer point of view. But I would like to start with Java development (or another programming language) perspective too, some intro would be good.

I have also in my study plans two courses about Machine Learning and Data Science on Coursera which I already subscribed to. When I am currently taking a break from courses in Data Science Specialization.

And last but not least I need to finish mini project in R for DSE 501 and follow up with next one DSE 502.

If anyone wants to join me, for Coursera courses for example. Sign in, links to the courses or another staff are below and let me know in comments. We can make a study group.

Course: DSE 501 - Machine Learning with R

After I went successfully through DSE 400, which is in detail described here. I followed up with DSE 501 Machine Learning with R. I was really eager to learn Machine Learning since beginning of my Data Science study, so here was an opportunity and I took it.

First at all I need to say, I really enjoyed it, even though I haven't finished it yet. Last thing which is still on my table even though I am working on it really hard is final project, the 6th assignment. You are choosing it by your self and define proposal by your self, so it's up to you, how difficult it would be. But, I am going backwards. Let's start from beginning.

Reading: Free online resources June 2014

I need to say, my last obsession is NLP. After I came through basics of ML/DM techniques I challenge my self in NLP Kaggle.com contest about Sentiment analysis. So, if you are interested about free (as usual of course) basic sources  for this topic, be my guest:

• Natural Language Annotation for Machine Learning by James Pustejovsky, Amber Stubbs
• Natural Language Processing with Python by Steven Bird, Ewan Klein, Edward Loper
• Python Text Processing with NLTK 2.0 Cookbook by Jacob Perkins
• Taming Text by Grant S. Ingersoll, Thomas S. Morton, Andrew L. Farris
• and many others...

Unfortunately I have found that 1st one is not completely available, just first 4 chapters, but as intro it is good enough.

Course: DSE 400 - Fast Track to Data Science

After I described the whole DSE Program it self here. I would like to continue with specific parts which I came through. Currently I study 2nd course DSE 501 - Machine Learning with R which I will describe in some next blog posts, but let's start first with initial course which was DSE 400 - Fast Track to Data Science.

As I wrote already in my previous blog post Data Science Enablement Program is really different than other known courses which you know from Coursera or other MOOC websites. Instead looking at video courses and doing quizzes and course project your need to be personally involved and discuss weekly topics and also do (at least) weekly assignments which corresponds to current week topic. Each week you got a lot of study materials which could help you to achieve assignment and get reasonable knowledge about the topic and also lot of following practice suggestions with reasonable questions or tasks which move you forward through the topic.

Course: Data Scientist Enablement Program (DSE)

Data Scientist Enablement Program (DSE) is something which I have found by accident. At the beginning of year I was looking for new courses or some complex study with topics about Data Science and Big Data and have been really keen to learn something new. Then I found form which gave me access to SONO to DSE.

This program and its all 4 modules (so far I have done 1st one, so I am just assuming based on the current experience) are not classic course with video tutorial, quizzes and labs with question form and score. It is almost like in school, you can participate or you can not, you can read and watch materials which you got, but nobody checks if you have done it or not. Only one thing you need to do, is submission of assignment. And even this action is not time limited, so there is no penalty for late submissions, only recommendation to do it ASAP.

Moreover, you will get certificate corresponding to your maturity level in this study and based on your composite score which is computed based on your social engagement, activities, projects, collaboration, etc in all 4 modules.

Study: Apply to Berkeley School of Information MIDS Program

It was little bit easygoing decision. When I saw offer to apply to Berkeley School of Information Master of Information and Data Science (MIDS) Program I tried it. It is completely online study. And here is my experience with how it went. In front of this post I need to say I haven't finish it and I won't do it. The main reason is: for me it is impossible to finance it. Before you start, please have a look over financial part too. You can find tuition-fee described in detail in separate article. I haven't done that, I expect it is expensive, but not so much, of course it is Berkeley :-), so I have been surprised in the middle of application.

Course: Data Science Specialization on Coursera

Surprise came to my mailbox today morning. Coursera announced new type of study: Specialization.

Specializations are new multi-course programs. In case of Data Science Specialization you will take 9 separate courses with signature track for $49 and earn certificate. And complete track you can finish with capstone project which is another $49. So, if you wanna be certified for Specialization you need finish 9 certified courses and do final project. In total you will pay $490, but it is in 10 separated payments. Great thing is all of these courses are under patronage of Johns Hopkins University which I hope ensures quality. I need to say that based on my experience with Computing for Data Analysis course which is done by this university as well, it has really good quality!

Looking for first hands on experience

Last week I was quite busy with MongoDB course. And even though I have plenty work with final exam which contains 10 separate works I am little bit fed up of databases at all. No matter if NoSQL or SQL which I work with really intensively in last week before Christmas.

I moved my sight to the programming language and was looking for some practical hands on experience. I have been lucky and found Data Mining: Discovering and Visualizing Patterns with Python, when I was looking for new articles and resources about data science. When I found this I was so excited! It's like kill many data science skills with one stone. I can play with Python instead of the database, work with new data and visualize it. Good, let's see how it works.

What, where, why and how to study Data Science

I just jumped into the water. Just like that. I haven't so much thought about that. I knew some basics about Big Data, but about Data Science not. It changed when I passed through one commercial enterprise Big Data solution introduction. It also contains Data Science role description and its contribution to development process.

Next step was reading many web discussion about it. And then finally read really good article Fake Data Science from Vincent Granville published in February 2013. It formed my opinion and it turned my way little bit different way than I was at first time. Simply said from more technical point of view to more business point of view. And it make sense if you would start to look on Data Science from right angle. What is result or what result do you expect (what's are business needs) and what path do you need to use to accomplish it?

The Beginning, actually The Inception

When it starts, when I realized that is more then enough with stay on spot and not moving forward in my knowledge, experience, learning, working role,..., career? Its few months ago, when I was working again many extra hours and developed solution which was interesting, but not satisfying me. From work point of view of course.

I am struggling many months, even few years with buzz word Big Data, I was laughing when I heard first time Data Scientist role name... I omit to learn anything about it, even to read something about it. But, you know, when you are fight against something the best think you can do is to accept it, especially if would not destruct you.

First I wanted to be Business Analyst. It is to hard for me to understand from scratch business which I have never studied in university. I am technical guy. I am learning business, but slowly, based on practical experience. Then I realized this is solution, to become someone else than ETL/DWH/BI developer. It was like real beginning, no actually it was inception.

And here I am, on the beginning of journey to become Data Scientist, never mind how noble it sounds and how abused role name it is.