Artificial Intelligence and Big Data Science [Q1, 2019]

Introduction

• From Big Data to AI via Data Science
  • Data-driven Decision Making for Data-driven Organizations
  • What are Big Data, Data Science and AI?
  • How do Big Data, Data Science and AI support Data-driven Decision Making?
  • Who is using Big Data, Data Science and AI?
• Success story: The Big Data Science & AI Behind Netflix’s “House of Cards”
  

Diving in Big Data

• Big Data Vertical vs. horizontal scalability [slides][drawing]
• Scaling storage horizontally with K-V pairs [slides][drawings]
• Scaling processing horizontally with MapReduce [slides][drawing]
• Achieving usability implementing a SQL interfaces on horizontally scalable solutions [slides]

Diving into AI

• solving problems the AI way [drawings]
  • coding vs. machine learning/AI
  • Gradient Descent as a general procedure
• Addressing dynamic pricing with linear regression (supervised machine learning) [drawings]
  • what is the relationship is not linear? Polinomial fitting vs. axis transforamtion (a circle is a line in x² X y²)
• Detecting spam emails using Naive Bayes Algorithm [drawings]
• Recommending Apps based on Decision Trees [drawings][animation]
• Deciding to accept students at a university based on Logistic Regression and Gradient Descent with Log-loss function [drawings]
  When a line is not enough … or the kernel trick of Support Vector Machines [drawings]
• Placing shops given distribution of buyers with desired segment solution using k-means (unsupervised machine learning) [drawings]
  • does this always work?!? no … same data -> different cluster algorithm -> different clusters
• From a single neuron to deep learning visually using tensorflow playgroundhttps://playground.tensorflow.org
  • single neuron classification of two groups that can be separated by a single line
  • single neuron classification of two groups that can be separated by a circle in x² X y²
• classification of two groups that can be separated by a circle using multiple neurons (each tracing a line)
  • basic implementation using sigmoid attivation function (converges in 300 epochs)
  • optimized implementation using ReLU attivation function (converges in 30 epochs)
• complex classification
  • a sigle layer of neurons hardly works (even if in theory enough neurons do)
  • multiple layers (deep learning) easily solves the problem

Final discussion

• beware confirmation bias
• Ethics and legal implications