Skip to main content

Enterprise Data Management (Data Warehouse/ ETL)

 A data warehouse, also known as an enterprise data warehouse (EDW), is a system that stores and analyzes data from multiple sources.

Data warehouses are used for reporting and data analysis, and are considered a core component of business intelligence.

Case study: 

Trends, an affordable fashion & lifestyle retailer which operates more than 2,300 stores in over 1,100 towns and cities across India.

They might have goods stored in different warehouse, They might have enterprise application to track the goods in warehouse and analyze them for improving their business.


A data warehouse has four main components: refer above diagram
  1. Central database:
  2. ETL (extract, transform, load) tools
  3. Metadata (Data sources)
  4. Access tools 
Some examples of data warehouses include:
  1. Snowflake
  2. Google BigQuery
  3. Amazon Redshift
  4. Azure Synapse Analytics
  5. IBM Db2 Warehouse
  6. Firebolt 

Benefits of Data Warehouse:
  1. Understand business trends and make better forecasting decisions.
  2. Data Warehouses are designed to perform well enormous amounts of data.
  3. The structure of data warehouses is more accessible for end-users to navigate, understand, and query.
  4. Queries that would be complex in many normalized databases could be easier to build and maintain in data warehouses.
  5. Data warehousing is an efficient method to manage demand for lots of information from lots of users.
  6. Data warehousing provide the capabilities to analyze a large amount of historical data.

Enterprise Data Management ( EDM ) governance and control:

Enterprise data management (EDM) is the process of defining, integrating, and retrieving data for internal applications and external communication. 

EDM also involves managing the people who access the data, ensuring that they have the right information and follow the organization's standards for storing data. 

EDM focuses on creating accurate, consistent, and transparent content. 


EDM Architecture:

Data Management Maturity Model:


Extract, transform, and load (ETL) is the process of combining data from multiple sources into a large, central repository called a data warehouse. ETL uses a set of business rules to clean and organize raw data and prepare it for storage, data analytics, and machine learning (ML).
Here are some advantages of ETL:
  1. Data governance: ETL fosters data democracy, which broadens data accessibility and makes it available to all the stakeholders to analyze it and use it for business
  2. Reduce delivery time: ETL can reduce delivery time
  3. Reduce unnecessary expenses: ETL can reduce unnecessary expenses
  4. Automate complex processes: ETL can automate complex processes
  5. Validate data before migration: ETL can validate data before migration 
Amazon Redshift:





Comments

Post a Comment

Popular posts from this blog

New way of product development

Today is the era of fast-paced world and competitive world. Companies are realizing that the old sequential approach to developing new products won’t get the job done and product can’t be reached to market when compared to competitors. The 4 stages of product development are as follows – R&D, Growth, Maturation, and Decline. Instead of sequential approach, companies are using holistic approach – as in rugby game, the ball gets passed within the team as it moves as a unit up the field. This holistic approach has six characteristics: 1)     Build-in-instability 2)   Self-organizing project teams 3)   Overlapping development phases 4)   Multi-learning 5)   Subtle (very clear and strong) control 6)   Organizational change to explore and learning The above six characteristics forming a fast and flexible process for new product development with advantage of act as a change agent, creative, market driven ideas, flexi...
Post a Comment
Read more

Product Manager vs Product Owner

Both the product manager and the product owner work towards a common goal, to build and improve products that create meaningful value for customers and all stakeholders within the company. This usually happens by delivering and optimizing product features. Product Manager Product Owner The product manager discovers what users need, prioritizes what to build next, and rallies the team around a product roadmap. The product owner is responsible for maximizing the value of the product by creating and managing the product backlog. This person creates user stories for the development team and communicates the voice of the customer in the Scrum process.      Product Manager and Product Owner's work on below vacuum. Product manager focus on: Business Strategy Long term Product Vision Long term Product Strategy Product Roadmap Alignment with Product Owner Product owner focus on: Release Plan (Product Backlog ie: ...
Post a Comment
Read more

Delivering a project within budget

 Here are some tips for delivering a project within budget: Set a realistic budget Define the project's scope and necessary resources, and create a budget that's realistic. Cost estimate Segment the project into smaller tasks and milestones to plan how to use resources and provide clarity. Divide the project plan Break down the project into tasks to avoid late deliverables and over-budget projects. Monitor progress Regularly track the project's progress to identify and prevent cost overruns. Use progress reports to compare actual costs to the budget. Anticipate and revise changes Communicate with stakeholders to identify and assess risks, and assign owners to each risk. Consider different scenarios Estimation can be difficult for complex projects with many potential outcomes. Tracking: Tracking time spent on tasks, Tracking expenses per project, and Using project management software. Use Historical Data Your project is likely not the first to try and accomplish a specific o...
Post a Comment
Read more