Chasing own tail

Photo Taro the Shiba Inu @ Flickr

It's a depressive moment in everyone's life, chasing own tail. What this about in my case? When I want to move in career towards data science role. Not a big data, i.e., working with Hadoop, but real data science which is about statistics, data mining, data analysis, looking for patterns, feature extraction and so on. The issue is the  experience on some project.

Yes you most likely know it's this pattern:

1. There is no job without project experience
2. You can't get project experience without job

Oh, wait maybe I can break this infinite loop. How?

Back to the basics

Photo Alban Gonzalez @ Flickr

As I have written before (There are no shortcuts) sometimes is good to return to the basics and take it as reflection and improvement of your knowledge. All the programming languages and tools have many options how it could be extended and used together with libraries, packages, and plugins. But do you know how it works inside, what's under the hood? How to setup properly the parameters to get the best results from such tools?

What I want to do today is continue with this idea, but not on the philosophical level as last time, but with concrete examples. I give you some hints which I have tried by myself.

Machine learning

What could you do with machine learning? Whenever you install Python or R language you get such libraries like NLTK or Scikit-learn for Python or e1071. But do you see under the hood? Do you know how it's implemented inside or it's just a black-box for you? Do you want to know? So, what you can do:

• Start with the implementation of the simplest machine learning methods by yourself. For instance, k-Nearest Neighborhoods is the really good example and you can do it in half an hour from scratch or Decision Tree method or even Random Forest method. 
• Take a look how it's implemented in existing code, for example, you can take a look into NLTK Python code and learn how they invent particular process

Natural Language Processing (NLP)

The machine learning methods used for NLP are the similar one which you know already from the previous study of basics (kNN, Decision Tree, Random Forest) or they are quite extensive to implement them from scratch. The same it is with n-grams and textual document models, so what to do then:

• You will not most likely implement word2vec, but what you can implement simply is bag-of-words or with little bit more effort is possible to implement tf-idf.
• This code is also accessible in NLTK library where you can take a look and for instance, find that tf-idf is implemented with smoothing.
• With n-grams when you implement it (see my post Computing text conditional entropy with uni- and bi-grams in R and Python) you can play further and calculate pointwise mutual information and find the collocations and associations between words with that.

I will return to this topic whenever I have more experience to share, but if you will search you will definitely find similar sources about other topics of your interests such as Big Data, Visualization, Statistics, etc.

Luxury of the study compare to the work

On the way back from work to home, I just realized the luxury which I have during the study. I don't need to deliver a final and functional product I am not tight up with deadlines and budget. I am not technology locked up based on customers decision.

I can switch the technologies as I need and wish. I can play with them, experiment with them. Commit the output to versioning control repository or not. Freely publish the data, code, share the ideas on the blog and finally write it all on paper wrap it up by conference or magazines template and publish.

And when I realized it I want to share my approach with my last two publications, because they are connected and it would be end-to-end process description with two publications and several Github repositories as output.

One picture is better than 1000 words so let's imagine the following situation about what I have done during last year:

Continuous and discrete time series visualized together

The second publication for Gigascience is nearly finished and the deadline for third publication is approaching really quickly so I needed to start with data analysis. Because I had just a few ideas about how to analyze experiment data I asked the colleague from University about advice and sent her data. It was a few months ago.

Recently I got back from her the ideas and notes and also the recommendation for the book about time series analysis. As usual, it's good to start with several visualizations and from simple things to complex. So, I want to share some notes about it.

Data

As the output from two parts of the experiment were recorded two pairs of datasets for:

• Experiment #1 with Fitbit Charge HR
• 1029 tweets with average of 20.56 tweets per day, 
• 411 799 records of HR with frequency of 6 - 7 records per minute
• Experiment #2 with Peak Basis
• 1017 tweets with average of 20.32 tweets per day,
• 69 909 records of HR with frequency of 1 record per minute

All the tweets are just a records of sentiment and they are evaluated by hashtags #p for positive and #n for negative sentiment by experiment participant. As a part of publication, the sentiment will be extracted via machine learning methods and compared to human evaluation.

Visualization

I took data from experiment #1 since there are more records of HR and did following data wrangling:

• Tweets respectively evaluated sentiment needs to be extended to not represent just a point in time, but the whole window. For this, we need to define "breaking point" between two consecutive going sentiments. It's simply in the middle. This is visualized in the following figure as the gray line with the dots on it where positive sentiment = 150 of HR bpm and negative sentiment = 0 of HR bpm (scale adjustment was done, otherwise sentiment is represented by +1 and -1). Dot's here representing true records, the line is an extrapolation to the time window.
• Heart rate is drawn with another gray line representing rapid changes over time. I have applied to it simple moving average (SMA) method to get a much smoother line for further analysis. And also cut this SMA line by extrapolated windows of sentiment where the red part represents negative sentiment and blue part positive sentiment.

Here is the result:

It looks really promising, but that's all so far. I have few ideas how to continue, but since this is a part of my third publication I will publish it first and then link the publication itself and also complete code on Github. Sharing this was just about the idea how to start with the visualization of continuous and discrete time series together.

There are no shortcuts

During last 3 months I have realized it is really difficult to progress with something when you don't have enough knowledge and you are trying to go ahead without proper basics and good background, because you don't have time to focus on them, rather jump directly to wild water and swim. What a mistake.

I was dealing with many things simultaneously. I needed to start with second publication for my PhD study, but I worked on second experiment which needed to be included into this publication. And of course I studied two difficult and time consuming courses in the same time on Coursera and elsewhere. All of this together with big workload in work and push from my company to learn German. To much to process, to much work to do.

Problem is, nobody can control dreams you have.
There is a reason why Magical Realism has been born in Columbia.
It's a country wheres dreams and reality are conflated.
Where in the head, people fly high as Icarus.
But even Magical Realism has it's limits
and when you get to close to the sun...
your dreams may melt away.

- Narcos

I haven't fly so high as Icarus yet. I haven't fall down and let my dreams melt away yet. But I have been close. My PhD was running away from me. I haven't finish courses.

So, there are no shortcuts. Basic need was to organize and finish all of those tasks one by one with proper priority and timing. This helped me out of the bad mantra I am too busy for basics, but I cannot manage the complex things because of lack of basics.

And I returned to basics (learn particular technique or setup, install and explore new technology) at least for once a week instead of work on my PhD or watching lot a videos from MOOC courses I am focusing on small parts with hands on experience.

Heart rate and sentiment experiment design

Photo GrejGuide.dk @ Flickr

After I finished first experiment and publish article in HEALTHINF 2016 conference this year in Rome I started thinking about next experiment design.

As I mentioned in previous paper improvements I tried to get a lesson from previous mistakes and improve a lot. First, steps are increasing during the day and thus they are not so much independent. Better would be to use heart rate because is totally independent. Second, I can improve my records about sentiment in timing and evaluation. And last but least important is sentiment extraction, instead of supervised learning used in previous work I would like to used unsupervised classification.

So, let's get to details.

Experiment Design

We are still looking for relation between soft data (sentiment) and hard data (measurand). In first experiment it was text recorded via twitter and footsteps. This time it's again text recorded via twitter, but instead footsteps it's heart rate which is more idenpendent.

What's are the main objectives:

• One month experiment (30 days)
• 20 tweets per day (600 tweets minimum)
• Continuous heart rate measurement (24/7)
• Effective time for heart rate measurement between 7 and 23, i. e. 16 hours a day, rest used for wristband charging
• No sleep activity monitoring
• Steps monitoring? Perhaps.

Result of fitness band experiment

As I am progressing with my PhD study I haven't been able to write down any article because I have been busy. And not only PhD made me busy for whole time since January this year, I have to care about work and family.

Nevertheless I should point out results from Introduction fitness band experiment when I wrote about it one and half year ago and never continued.

What was the experiment about

To find relation between sentiment (represented by text recorded in twitter just for practical reasons) and human activity represented by footsteps. Practically about finding link between soft data - sentiment - and hard, measured data.

Reading data: Github implementations

Data and it's processing and analysis code I will publish another time, because I don't have them yet at GitHub.

Twitter data reading implementation

What I have is implementation or reading tweets from Twitter API through tweepy. Which is refactored original version. The reason of refactoring is that I work on second version of fitness band experiment.

Jawbone data reading implementation

What I also have is implementation of reading data from Jawbone API through many different libraries. It's small mess which I need to clean up later when I will use Jawbone again.

Story telling

Long story short

All the data has been extracted, processed and was defined hypothesis which wasn't rejected. Unfortunately, rejection of null hypothesis in favor of alternative was expected and it doesn't happened. That's result and that's the long story short. The whole article presented on HEALTHINF 2016 conference in February this year (2016) is possible to get here:



You can also follow up with the whole story in following chapter, if your are not interested in paper it self right now.

Every Data Scientist must only pay taxes and die, rest is just optional...

Picture from cacm.acm.org

I am just wondering. There are many articles about what every Data Scientist MUST know and do to be real Data Scientist. There are many articles about MUST not do as Data Scientist. What real Data Scientist MUST read and so on and so on. And honestly I don't care. Why is that so?

Research

Let's take the last one must: "What Data Scientist must read" list. I just briefly took several results from Google, here is the list of 10 of them:

• https://www.quora.com/What-is-a-list-of-books-that-every-data-scientist-should-read
• http://bigdata-madesimple.com/20-must-read-books-for-budding-data-scientists
• http://www.analyticsvidhya.com/blog/2015/06/infographic-read-books-analytics-data-science
• http://www.datasciencecentral.com/profiles/blogs/15-books-every-data-scientist-should-read
• http://www.mastersindatascience.org/blog/7-must-read-books-for-the-budding-data-scientist
• http://www.forbes.com/sites/gilpress/2015/03/31/3-recent-books-for-budding-data-scientists-and-business-executives-involved-with-big-data-analytics
• https://www.umbel.com/blog/big-data/must-read-big-data-books
• http://analyticstraining.com/2014/five-must-reads-for-data-scientists
• https://www.quora.com/What-are-the-must-reads-for-data-science-and-machine-learning
• http://www.analyticsvidhya.com/blog/2015/06/infographic-read-books-analytics-data-science

And what is this quick research good for? From this circa 80 books and list of several articles you get list of subjective chosen resources which lead practically to nowhere. Just several books repeating like famous Nate Silvers Signal and Noise and of course some R or other Cookbooks. So, what is conclusion?

Conclusion

This lists of books which someone else read leads me always to Vincent Granville's article Fake data science. And what you need to take from it? Pick any book you need for your field of expertise. And what should be your field of expertise? Choose some project, doesn't matter if your personal one, school or for instance from Kaggle.com. And follow up approaches which you need to for goal achievement, then pick book, course to support your path towards this goal. And by real work, life experience you will sooner or later become Data Scientist.

Bunnies, Dragons and the 'Normal' World: Central Limit Theorem

Statistics is difficult field, but with this video you can see under the hood of one of many things from statistics theory and it's fun.

So, enjoy it as I enjoyed it. And learn something.

Course: Exploratory Data Analysis

What do you imagine under Exploratory Data Analysis?

After a while I decided to continue with one of the first specializations on Coursera which I started more than year ago. This Data Science specialization contains 4th course with the same name as the topic which I was asking in opening question: Exploratory Data Analysis.

I was wondering how much effort I need to include and how much knowledge I can get. After 2 years (yes, anniversary of this blog is 18th of December 2015) with Data Science study path and more than 1 and half year with intensive usage of R language and couple of statistics and R courses. What can surprised me, right?

Nothing and something. I mean this both in positive way and will tell you why.

Computing text conditional entropy with uni- and bi-grams in R and Python

During my first semester of PhD study I have implemented solution for computing conditional entropy over text where each word (including interpunction) was on separate line. I would like to share this example, because it is simple and good for get basics (information theory, R and Python programming).

Conditional entropy is defined as (source is Wikipedia):

Just note, this computation determine the conditional entropy of the word distribution in a text given the previous word.

According line 4 in previous formula, I have to compute p(x,y), which is the probability that at any position in the text you will find the word x followed immediately by the word y, and p(y|x), which is the probability that if word x occurs in the text then word y will follow.

Or according line 5 in formula, I can use probability p(x,y) twice and calculate p(x) which is the probability of single word appearance in the text.

Course: Data Mining Specialization on Coursera

When Specializations started last year on Coursera I joined to Data Science Specialization and was really keen to do as much as I can. I did 3 courses in row so far and then have been stopped with my own activity about PhD study and also little bit disappointed by courses quality which really differ course by course.

Anyway with new year, and I think it was announced during November last year, I joined to first course under Data Mining Specialization - Pattern Discovery in Data Mining. With expectation of better quality than previous courses of Data Science Specialization and more information which I can learn from those courses. It already started from 9th of February, so lets have a more detailed look over it and I will tell you my first impression from it.

Data Mining Specialization is prepared and lead by people of University of Illinois at Urbana-Champaign and contains following courses which you can do in any order (they are independent):

1. Pattern Discovery in Data Mining
2. Text Retrieval and Search Engines
3. Cluster Analysis in Data Mining
4. Text Mining and Analytics
5. Data Visualization

So far, each course have just 1 term offered (Feb 9th, Mar 16th, Apr 27th, Jun 8th, Jul 20th) so you can follow up order above month by month. Either for free or with verification payment of $49 (which is the same as in Data Science Specialization courses) which leads to certificate.

So, I am little bit disappointed that I cannot start with Text Mining, which is most interesting for me and need to wait for it 3 months, but it needs some time to be fully ready, I suppose.

At the beginning of Pattern Discovery course I need to say they were some lacks about organization and course preparation, but it gets better and better and after 2 weeks of study (you have always some days in advance for each week to finalize quiz) it is interesting study, but so far it seems to me, that I am missing some more details, some tough study going deep into problematic. Maybe it is given just by passing the quizzes and missing some implementation assignments.

Anyway I can strongly recommend to you to test it for free (you have couple of weeks before you need to pay for verified certification path) and then decide if you like it or not. Based on my experiences each teacher and each university make different courses. Different approach, presentation, tasks, quizzes and so on, so you need to get used to it.

Killing me softly

The title say everything. This killing combination come with project in Sweden together with previously started PhD study and my spare time study during evenings and sleepless nights, but lets take it from beginning.

At the beginning there was sustainable project with daily routine and spring. Part of the year with the most energy. So, I applied and got into PhD study. Euphoria was the mood which I had directly when I got letter with acceptation.

Then I started study and not realized how much things I need to go trough so I started with two subjects in one semester. So far so good. Lot of study materials, I can't say no. But, really interesting topics (I will blog about it sometimes later).

"Winter is coming."

- the motto of House Stark

And then I started to work on project in Sweden. And this started to be more interesting. I am not able to study on daily basis, so I am little bit behind schedule, but I am really tired from winter and dark (at 7 AM is not daylight yet and at 2 PM is no longer daylight).

So, even when I put into it big discipline and lot of effort I have more things to do. And I hope I will handle it at least partially or with some postponements. And will finalize first semester somehow reasonable.

Introduction to fitness band experiment

This started as stupid idea. It was influenced by someone's project in Coursera course (Data Analysis and Statistical Inference), when I did volunteer evaluation. The other student analyzed his own data from Fitbit Flex . I almost forgot on it. Then later on in another Coursera course (Getting and Cleaning Data from Data Science Specialization) we did data processing in R and this data contained also information from accelerometer and some experiment with couple of human subjects (see Human Activity Recognition Using Smartphones Data Set).

Then continued when I was discussing my dissertation thesis topic with my supervisor and we came to the experiment with fitness bands and sentiment written into twitter.

Big Data and Data Science study - presentation

During our Friday 26th of September company meeting I talked about topic in title with presentation. I think this presentation could be useful not only for audience which was there but for anyone.

Here is description what is content of presentation about:
Big Data and Data Science study with subtitle "study materials and online courses" is little bit more over 40 slides presentation about 10 domains of Data Science covered by online free and paid MOOC courses, study materials and free books.

Based on my almost year of study, investigate and collect of materials, tutorials, courses, books, links, etc. I have prepared distillation of the best in this short presentation.

Of course list is not full, because there is always something new, undiscovered and better than before. But it contains the most important information for those who want to start or don't know where exactly follow up when they already begun.

Follow up to the Speaker Deck site and you can download presentation as PDF which is quite useful when you consider functional links over all presentation.

Data Science and PhD study

When I was applying for the PhD study I was thinking about it as opportunity to have chance to use all techniques, programming languages, methods, processes and technologies in practical way and delivery something reasonable which supports my professional growth and development. And it could stands as proof that I am really keen to learn and improve in this field and I have passion to do Data Science.

I was thinking about it again when I have read this article The Modern Data Nerd Isn’t as Nerdy as You Think on Wired about Data Nerds who are currently in charge of Data Science departments or in similar role and apply their experiences and knowledge and usually they do not have PhD. More over they do not have master degree, just bachelor degree. Or in case they have any kind of degree it is not from field close to Data Science.

Interviews to Data Scientist role

After almost year of self study of Data Science courses via Coursera, DSE program and another MOOC pages I had chance to go to Data Scientist role interviews. So far two.

First one was a couple weeks ago I was in interview to the Researcher / Data Scientist position. Mainly about prediction models, math, statistics and data mining/machine learning. Mostly in R connected with Hadoop and some other mathematical tools. This was the first for such position.

Second one was a couple days ago and it was interview to the purely Data Scientist position. Including all data science work from many different sources with project driven or data driven approach, it depends. This was most recent experience and I hope not a last one.

My study list for Summer time 2014

I was thinking what to study during Summer time. I need to improve my insight into Big Data, especially Hadoop and its fundamentals like HDFS, HBase and MapReduce. At least from non-Java developer point of view. But I would like to start with Java development (or another programming language) perspective too, some intro would be good.

I have also in my study plans two courses about Machine Learning and Data Science on Coursera which I already subscribed to. When I am currently taking a break from courses in Data Science Specialization.

And last but not least I need to finish mini project in R for DSE 501 and follow up with next one DSE 502.

If anyone wants to join me, for Coursera courses for example. Sign in, links to the courses or another staff are below and let me know in comments. We can make a study group.

Course: DSE 501 - Machine Learning with R

After I went successfully through DSE 400, which is in detail described here. I followed up with DSE 501 Machine Learning with R. I was really eager to learn Machine Learning since beginning of my Data Science study, so here was an opportunity and I took it.

First at all I need to say, I really enjoyed it, even though I haven't finished it yet. Last thing which is still on my table even though I am working on it really hard is final project, the 6th assignment. You are choosing it by your self and define proposal by your self, so it's up to you, how difficult it would be. But, I am going backwards. Let's start from beginning.

Reading: Free online resources June 2014

I need to say, my last obsession is NLP. After I came through basics of ML/DM techniques I challenge my self in NLP Kaggle.com contest about Sentiment analysis. So, if you are interested about free (as usual of course) basic sources  for this topic, be my guest:

• Natural Language Annotation for Machine Learning by James Pustejovsky, Amber Stubbs
• Natural Language Processing with Python by Steven Bird, Ewan Klein, Edward Loper
• Python Text Processing with NLTK 2.0 Cookbook by Jacob Perkins
• Taming Text by Grant S. Ingersoll, Thomas S. Morton, Andrew L. Farris
• and many others...

Unfortunately I have found that 1st one is not completely available, just first 4 chapters, but as intro it is good enough.