Project Daytona

It's been a while since I've been trying out Apache Mahout - a library of scalable Machine Learning algorithms. And it was only a matter of time, I thought, before the other biggies would come up with their own solutions to the problem of Machine Learning on Big Data. I came across Microsoft's Project Daytona yesterday. Here's a quick introduction -

Project Daytona is the new iterative MapReduce runtime developed my Microsoft to run on Windows Azure. It supports a range of Data Analytics and Machine Learning algorithms and its scalability makes it suitable for analyzing Distributed Data. The project was developed to answer the ever increasing demand for simple tools for analyzing large and growing data collections in various fields of research.

How to Use Daytona?

§ Develop your Data Analytics algorithms. These algorithms are authored as sets of Map-Reduce tasks, but you don't require detailed knowledge of Distributed Computing or Software Development on Windows Azure for this step.

§ Upload your data and library of Data Analytics routines into Windows Azure

§ Deploy the Daytona runtime to Windows Azure (also configure the necessary details for the deployment)

§ Launch the Data Analytics algorithms

Why Daytona?

§ The scheduling, network communications scheduling, and the fault tolerance logic of Project Daytona has been tuned for the cloud, specifically for Windows Azure

§ Daytona is designed for cloud storage services and does not even require a Distributed File System (see more)

§ It is horizontally scalable and elastic i.e. you can add more virtual machines to the deployment and Daytona will take care of the rest

§ Daytona offers support for iterative algorithms and sequential answers, often encountered in Data Analytics and Machine Learning. Thus, it is optimized for Data Analytics.

Excel DataScope Example (see more)

Excel DataScope is an example of Project Daytona's attempt to provide Data Analytics as a service on Windows Azure. The vision here is to provide the ability to upload data and select from an extensible library of models for data analysis. Users upload the data in their Excel spreadsheet to the DataScope service. Then they select an analysis model from DataScope's research ribbon to run on the data. Project Daytona then scales out the model processing by using possibly hundreds of CPU cores and finally returns the results to the Excel client.


Although this is exciting news for all Distributed Machine Learning lovers, how successful this attempt turns out to be will depend on Microsoft's ability to create an active and vibrant user community.



(Reference: Microsoft Research website)