Introduction to Data Management

• What Is Data Management?

• Data Management Challenges

  • Update vs. Query Structured Data
  • Emergence of Big Data Un-Structured and Semi-Structured
  • Business Intelligence vs. Data Analysis
  • Machine Learning

• Responding to Challenges

• Technology

• Architecture

  • Classic Data Warehouse

  • Data Lake

  • Data Vault

  • Data Lakehouse

  • Data Pipeline

  • Data Fabric Conceptual Architecture

  • Data Mesh Architectural Principles

• Governance

• Data Quality

  • Proactive Data Quality Improvement

  • Reactive Data Quality Improvement

Defining Business Needs

• Defining Business Needs

• Business Drivers

  • Business Intelligence

  • Data Analytics

  • Data Source Availability

• User and Data Source Driven Requirements

• Identifying Business Activities

       -  Traditional Approach
  
           - Enterprise Architecture
  
           - Survey Business Processes
  
           - Map Data to Business Processes
  
       -  Agile Approach
  
           - Jobs To Be Done
  
               - Hierarchy of Jobs
  
               - Mission
  
               - Capabilities
  
           - Organisation Structure
  
           - Capabilities and the Organisation Structure

• Defining Business Outcomes

       -  Identifying Business Outcomes
  
       -  Drawing Concept Maps

• Measuring Performance

       -  Identifying Performance Measures
  
       -  Leading and Lagging Measures

• Step-By-Step Approach

       -  Step 1: Define the Business Area Mission
  
       -  Step 2: Brainstorm Jobs To Be Done
  
       -  Step 3: Group Into Capabilities
  
       -  Step 4: Identify Data Marts
  
       -  Step 5: Develop a Data Mart Concept Map
  
       -  Step 6: Define Performance Measures

Modelling Data

• Comparing Data Models

       -  Data Storage Models
  
       -  Normalised Data Models
  
       -  Star Schemas

• Data Modelling Principles

       -  Classification
  
       -  Abstraction

• Representation

• Concepts

        -  Reification
  
        -  Entities
  
        -  Attributes
  
        -  Relationships
  
        -  Cardinality (Multiplicity)

• Normalised Data Models

       -  Roles As Attributes
  
       -  Roles As Entities
  
       -  Multiple Roles
  
       -  Resolving Many-To-Many Relationships

• Dimensional Data Models

       -  Star Schema
  
       -  Dimensions
  
           - Role Dimensions
  
           - Roles as Views
  
           - Type Dimensions
  
       -  Relationship Between Normalised and Dimensional Models

• Dimensions

       -  High Quality Verbose Attributes
  
       -  De-Normalised Roles
  
       -  Time Dimension
  
       -  Location Dimension
  
       -  Degenerate Dimensions
  
       -  Optimising Large Dimensions
  
       -  Time Mini-Dimension
  
       -  Junk Dimensions
  
       -  Slowly Changing Dimensions
  
       -  Multi-Valued Facts
  
       -  Snowflake Schema

• Facts

       -  Fact Granularity
  
           - Time
  
           - People, Things (and Places)
  
       -  Fact Aggregation
  
       -  Fact-Less Facts
  
       -  Facts as Dimensions
  
       -  Conformed Measures
  
       -  Conformed Dimensions
  
       -  Line of Business Facts
  
       -  Drilling Down