Saturday, August 2, 2014

Big Data: Map Reduce Development

So far we have covered the following topics in the big data. You can click on the hyperlink and go to a specific topic.

-Managing HDFS

In this blog, we cover the topic, Map Reduce development. I am not a programmer, but an architect. I cant take you deep into programming as myself dont know it. What i can showing you the programming architecture and thats below.

MAP REDUCE DEVELOPMENT OVERVIEW
Choose IDE
-Eclipse
-Net beans

Configure IDE
-Create project
-Add hadoop JARs

Code it up! ----------> Write Unit Tests--> Compile to jar
                                 Debug Locally             - Run on Cluster
                                - Mockito for unit tests   -View Results and counters
                               -Debug in IDE(On Standalone) -Optimize jobs
                                on subsets of data

- Implement Mapper
- Implement Reducer
- Implement Driver



No comments:

Post a Comment

Featured Post

Amazon Route 53

Amazon Route 53 is a highly available and scalable Domain Name System (DNS) web service.Route 53  perform three main functions in any...