This blog is about the presentation I made for my dissertation in the NLP – ML domain. Sentiment Analysis is easy for English. It’s a little but not much difficult for other languages such as Hindi or Marathi. It gets difficult when the script used for the other languages is Roman. What makes it most difficult is when that transliterated text is a mixture of multiple languages. In our case we are considering two combinations of English-Hindi and English-Marathi. This mixture of languages is called code-mix. Usually one of the language (usually Roman script) is used for the textual representation. How to analyse the sentiments in this situation. Ordinary Sentiment Analysis The steps involved in the usual analysis is as[…]

The why? Although, there have been great products that existing since a long time, which does exactly and more than what this project is trying to accomplish, namely – Celery. However, the basic premise of this effort is again based on a very simple idea, which happens to be the mainstay of python community and that is – KISS. Most of the time, a project begins humbly with small code based and small set of functionality and over time, tries to do a lot of things and while doing so, ends up accumulating huge sets of features that are overwhelming for a beginners to start with. Therefore, keeping the same problem in focus, I decided to make a celery like[…]

To begin, I would first like to answer the obvious question: Why would anyone want to use lua when already using python? The answer is simple, legacy code. This experiment arose from the evaluation of possible candidates to replace an existing c/c++ system with limited throughput in terms of request per second with something like python or erlang. In this post, we will be focussing on python, since it was an experiment borne out of my own curiosity. As for the usage of lua, let’s say there was a lot of business logic already written in it and the requirement is to reuse that code base. I chose to use LuaJIT, because it already proved to be fast enough for[…]

Lately, I had a task of creating a workflow manager and after much testing and feature evaluation of products such as Luigi, TaskFlow, Azkaban, etc. I settled on doing it with plain celery. I chose to use canvas functionality offered by celery and was floored by what I found. It was easy to create workflows using chain and group feature of celery and it was also very performant system. The configuration for system that was finalized had rabbitmq as broker and mongodb as result backend. Since the use case was to have completely asynchronous flows, there was hardly any need for result backend except for celery’s own use when handing subtask level errors. Since what we were building itself was[…]