- As a part of this project a Hadoop cluster would be built to demonstrate EDH capabilities to ingest, process and deliver the data for analytical use.
- The goal of EDH Phase 2 is limited to Batch Ingesting of data into Hadoop with daily frequency.
- The BI tool selected for Visualization would have the necessary compatibility with the technologies selected for Data Storage and Retrieval as a part of EDH Phase 2.D
- Data Hub on modern data platform using open source.
- Setup a data acquisition process to bring data on a configured frequency.
- Parse, clean and validate data.
- Ingest data from flat file to hadoop using pig.
- Ingest data from traditional database system like oracle, teradata, mysql to hadoop using sqoop.
- Expose this data using hive so that analyst can access this data for further analytics.
- Develop a dashboard (light weight web interface) for displaying the KPIs based on the data acquired.
- Load delta feeds using data ingestion framework and schedule it by oozie.
- The application framework will be designed in such a way to be able to easily support real-time use case in the next phase of the project.
- We delivered the online solution – Business Analysis Tool Using it.
- Client now has automated their data ingestion process.
- Client able to generate the combine report from all the subsystems.
- Client able to analyze data 10 to 15 times faster than tradition DWH.
Inventory Management System
- Driven by highly competitive market conditions and the need to modernize and enhance the EDW Architecture, client aims to leverage Hadoop Big Data Technologies for establishing an Enterprise-wide Data Hub(EDH).