Python With Machine Learning

Attend free DEMO today

GET REAL TIME EXPERIENCE

COurse COntent

Python Introduction

  • * What is Python?
  • * Why Python now?
  • * How is the Job Market for Python Developers
  • * Installation and documentation

Python Basics

  • * Keywords and Identifier
  • * Statement, Indentation and Comments
  • * Variables and Datatypes
  • * Native Datatypes
  • * Input, Output and Import
  • * Relational / Logical Operators

Python Functions

  • * Running Python as a calculator
  • * Python Functions
    • * String Functions
    • * Date Functions
    • * Numeric Functions
    • * Loading commands from the library
    • * User Defined functions

Python commands

  • * Numbers and other data type function
  • * Strings
  • * Lists
    • * Length of a list; empty list
    • * Sub lists (slicing)
    • * Joining two lists
    • * List methods
  • * Range function
  • * Boolean values
  • * Expressions
  • * Variables and assignment
  • * Decisions
  • * Loops
    • * For loop
    • * While loop
    • * Else in loops
    • * Break, continue, and pass

Data structures

  • * More on lists
  • * The del statement
  • * Tuples and sequences
  • * Sets
  • * Dictionaries
  • * Modules
  • * Standard modules
  • * The dir() Function
  • * Packages

Files and Input and Output Operations on Files

  • * Fancier Output Formatting
  • * Reading and Writing Files
  • * File Operations
    • * Appending
    • * Sorting
    • * Merging
    • * Python Directory and Files Management

Exceptions and Error Handling

  • * Built-in Exceptions
  • * Exception Handling - Try, Except and Finally
  • * User-Defined Exception
  • * Error Handling scenarios

Python Object & Class

  • * Namespace and Scope
  • * Inheritance
  • * Multiple Inheritance
  • * Operator Overloading

Additional Topics

  • * Iterators
  • * Generators
  • * Closures
  • * Decorators

Numpy

  • * Introduction
  • * Fundamental package for scientific computing with Python
  • * N-dimensional array object
  • * Linear algebra, Fourier transform, random number capabilities
  • * Numerical Operations on Numpy Arrays
  • * Concatenating, Flattening and Adding Dimensions
  • * Python, Numpy and Probability
  • * Synthetical Test Data With Python
  • * Numpy: Boolean Indexing
  • * Matrix Arithmetics under Numpy and Python

Introduction into Pandas

  • * Defining Data Frames
  • * Operations using Data Frames
  • * Adding/deleting columns - Index operations
  • * Stack/Unstack/Transpose functions
  • * GroupBy function
  • * Converting between different kinds of formats

Web Scraping with Python

  • * Reading data from public websites
  • * Identifying websites DOM structures
  • * Using BeautifulSoup
  • * Writing web data to local files
  • * Handling errors from web data reads

Project outline

In this project you will learn how to read data from any public website and then store data into CSV file format. We will create a programmatic flow to ensure that the data appended to CSV file and then we create various Data Frames to generate analytical results. In this entire process we will use all the python features including classes, modules and libraries including (CSV, OS, GLOB, BS4, BeautifulSoup, Pandas, Numpy and many other modules). We will implement several filter conditions and data cleansing functions to ensure the quality. We will implement several String, Date functions.

COurse COntent

Introduction to Big Data and Hadoop

  1. What is Big Data?
  2. OVERVIEW of BIG DATA
  3. What are the challenges for processing big data?
  4. From where BIG DATA is generated?
  5. Big Data Job opportunities
  6. How many years’ experience you need for BIG DATA Jobs?
  7. What is Hadoop?
  8. Why Hadoop?
  9. Why HADOOP is good for Beginners / Freshers
  10. Why HADOOP is good for NON PROGRAMMERS
  11. Today’s Challenges in existing technologies
  12. Advantages and Challenges of Hadoop

HDFS (Hadoop Distributed File System)

  1. HDFS Architecture
  2. HDFS Overview & Features
  3. HDFS components and its functionalities
  4. Name Node(NN)
  5. Secondary Name Node
  6. Data Node(DN)
  7. Job Tracker (JT)
  8. Task Tracker(TT)
  9. How to start hadoop services
  10. Use of JPS
  11. HDFS Permissions
  12. Accessing HDFS files and folders

MAPREDUCE

  1. What is MapReduce?
  2. Why MapReduce?
  3. Features
  4. Architecture
  5. Many ways to MapReduce
  6. Using Raw Java code
  7. Hadoop Streaming
  8. Hive and/or Pig
  9. Five stages of Map Reduce
  10. Prepare Map() input
  11. Run Map() code
  12. Shuffle
  13. Run Reduce() code
  14. Produce final output
  15. Data Flow
  16. Input Reader
  17. Map Function
  18. Partition Function
  19. Comparison Function
  20. Reduce Function
  21. Output Writer
  22. Mapper and Reducer Class
  23. Challenges with Reducer

HIVE

  1. What is Hive?
  2. Why Hive?
  3. Features
  4. Architecture
  5. How Hive interacts with Hadoop framework
  6. Metastore
  7. Embedded Mode
  8. Local Mode
  9. Remote Mode
  10. Hive - Data Types
  11. Integral Types
  12. String Types
  13. Timestamp
  14. Dates
  15. Decimals
  16. Union Types
  17. Hive - Built-in Operators
  18. Relational Operators
  19. Arithmetic Operators
  20. Logical Operators
  21. Hive Data Management
  22. Creation of tables Internal vs. External tables
  23. Loading Data into Hive
  24. Altering Hive Tables
  25. Modes of Execution in hive
  26. Local Mode
  27. HDFS Mode/MR Mode/Clustered Mode
  28. Execution Mechanisms
  29. Hive Shell
  30. Script
  31. Hive Optimization
  32. Partitioning
  33. Bucketing
  34. Hive Advantages
  35. Hive Limitations
  36. Issues related to hive
  37. Tips for efficient HIVE Queries
  38. Reasons Why Apache Hive 2.0 Matters to Your Big Data Strategy
  39. Ways To Make Your Hive Queries Run Faster

PIG

  1. What is PIG?
  2. Why PIG?
  3. Pig Installation
  4. Features
  5. Architecture
  6. How Pig Works
  7. Apache Pig Vs Map Reduce Vs Hive
  8. Data types
  9. Modes of Execution in Pig
  10. Local Mode
  11. HDFS Mode/MR Mode/Clustered Mode
  12. Execution Mechanisms
  13. Grunt Shell
  14. Script
  15. Writing your first PIG script
  16. UDFs in Pig
  17. Eval Functions
  18. Filter Functions
  19. Aggregate Functions
  20. Limitations
  21. How to execute a pig program
  22. Issues related to PIG

SQOOP

  1. Sqoop Overview
  2. What is sqoop?
  3. Why Sqoop?
  4. Features
  5. Sqoop Installation
  6. Architecture
  7. Sqoop Commands and Examples
  8. SQOOP Import
  9. SQOOP Export
  10. SQOOP Incremental load
  11. SQOOP parallel load with split file mechanism while Import and Export
  12. SQOOP using MYSQL
  13. SQOOP to load into Web server Database

FLUME

  1. Apache FLUME Overview
  2. What is Flume?
  3. Why Flume?
  4. Features
  5. Flume Installation
  6. Architecture
  7. Flume agent usage
  8. Complex flows
  9. Reliability
  10. Recoverability
  11. Setting up an agent
  12. Configuring individual components
  13. Wiring the pieces together
  14. Starting an agent

INSTALLATIONS

  1. Hadoop Single node Installation
  2. Pig Installation
  3. Hive Installation
  4. Sqoop Installation
  5. Flume Installation


COurse COntent

Introduction to Blockchain

  • What is Blockchain?
  • History of Blockchain
  • Explaining Distributed Ledger
  • Blockchain ecosystem
  • Explaining Distributed Ledger

Types of Blockchain

  • Private/Consortium/Permission-less
  • Public/Permissioned implementation difference
  • What Blockchain has to offer across Industry?
  • Companies currently using Blockchain
  • Overview of what we are going to study in this course

Key Concepts of the Blockchain

  • Mining -Mining algorithm
  • Node, peer and block explanation
  • Merkle tree and Blockchain
  • Consensus Mechanisms- proof of work , proof of stake
  • How Bitcoin Blockchain works?
  • What is Transaction?

Introduction to Ethereum

  • Ethereum : Blockchain with smart contract
  • What is Ether?
  • Bitcoin vs Ethereum Blockchain
  • What is Ethereum wallet?
  • What is Smart Contract?
  • Ethereum clients
  • Geth Introduction
  • Setting up Private Blockchain using Geth

Learn Solidity

  • Introduction to solidity
  • Hands on solidity
  • Understand and implement different usecases
  • Implement and deploy smart contract on Blockchain

Implement Dapp

  • Setting up the environment
  • Tools to install – Truffle , Metamask ,Testrpc
  • Implement and deploy your first Dapp
  • Different usecases for implementation of Dapp

Future Scope

  • Talk about the future of the Blockchain
  • What is Hyperledger?
  • What is Hashgraph?
  • Discussion on current research on Blockchain
  • Understand current industry challenges and needs
  • Conclude the course


COurse COntent

PYTHON

Python Content is same as Above

ALGORITHMS


Descriptive and Inferential Statistics


* Samples and Populations (Day 1)

* Sample Statistics

* Estimations of Population Parameters

* Random and Non-random Sampling

* Sampling Distributions

* The Central limit Theorem

* Degree of Freedom

* Percentiles and Quartiles

* Measures of Central Tendency (Day 2)

* Mean

* Median

* Mode

* Measures of Variability/Dispersions

* Range

* IQR

* Variance

* Standard Deviation

* Probability Distributions (Day 3)

* Events, Sample Space and Probabilities

* Conditional Probabilities

* Independence of Events

* Bayes’ Theorem

* Random Variable

* The Normal Distributions (Day 4)

* The Comparison of Two Populations

* Analysis of Variance

* ANOVA Computations

* Two-way ANOVA

Data Wrangling

* What is Data Wrangling?

* Acquiring Data

* Common Data Formats

* What are Relational Databases?

* Introduction to Databases Schemas

* API’s

* Data in JSON Format

* How to Access an API efficiently

* Missing Values

* Easy Imputation

* Impute using Linear Regression

* Tip of the Imputation Iceberg

Text Mining

* Sentiment Analysis

* User Behavior Analysis

* Topic Categorization

* Topic Ranking

Data Exploration and Dimension Reduction

* Data Summaries

* Covariance, Correlation, and Distances

* Missing Values Handling

* Outliers Handling

* Principal Component Analysis

* Exploratory Factor Analysis


MACHINE LEARNING


Machine Learning: Introduction and Concepts

* Differentiating algorithmic and model based frameworks

* Regression

* Ordinary Least Squares

* Ridge Regression

* Lasso Regression

* K Nearest Neighbors Regression & Classification

Supervised Learning with Regression and Classification

* Bias-Variance Dichotomy

* Model Validation Approaches

* Training Set

* Validation Set

* Test Set

* Cross-Validation

* Logistic Regression

* Linear Discriminant Analysis

* Quadratic Discriminant Analysis

* Forecasting (Time-Series Modelling )

* Trend and Seasonal Analysis

* Different Smoothing Techniques

* RIMA Modelling

* ETS Modelling

Unsupervised Learning

* Clustering

* Hierarchical (Agglomerative) Clustering

* Non-Hierarchical Clustering: The k-Means Algorithm

* Associative Rule Mining

* Apriori Algorithms

* Frequent Item-sets

* Support

* Confidence

* Lift Ratio

* Discovering Association Rules

Machine Learning Techniques Using R Part-1

* Machine Learning Overview

* Machine Learning Common Use Cases

* Clustering, Similarity Metrics

* Distance Measure Types: Euclidean, Cosine Measures, Creating predictive models.

Machine Learning Techniques Using R Part-2

* Understanding K-Means Clustering

* Understanding TF-IDF and Cosine Similarity and their application to Vector Space Model

* Implementing Association rule mining in R.

Machine Learning Techniques Using R Part-3

* Understanding Process flow of Supervised Learning Techniques

* Decision Tree Classifier

* How to build Decision trees

* Random Forest Classifier

* What is Random Forests

* Features of Random Forest

* Out of Box Error Estimate and Variable Importance

* Naive Bayes Classifier.

COurse COntent

SPARK - Basic & Advanced

SPARK with SCALA

  • * Scala overview
  • * Introducing Scala
  • * Journey – Java to Scala
  • * What is Scala?
  • * Why Scala?
  • * Scala Installation
  • * Execution Mechanisms
  • * Scala in interactive mode
  • * Scala Scripts
  • * Scala Advanced programming
  • * Defining Functions

SPARK - Python

  • * SPARK - Python overview
  • * What is Python?
  • * Why Python?
  • * How to use PYSPARK?
  • * Advantages of Python in SPARK
  • * Web scraping using Python
  • * Handling high volume unstructured data using Python
  • * Generating huge volumes (10 crore + records) using Python

SPARK

  • * Spark overview
  • * What is Spark?
  • * Why Spark?
  • * Linking with Spark
  • * Initializing Spark
  • * Using the Shell
  • * Spark Installation
  • * Resilient Distributed Datasets (RDDs)
  • * Parallelized Collections
  • * External Datasets
  • * RDD Operations
  • * Basics, Passing Functions to Spark
  • * Working with Key-Value Pairs
  • * Transformations
  • * RDD Persistence
  • * Which Storage Level to Choose?
  • * Removing Data
  • * Shared Variables
  • * Broadcast Variables
  • * Accumulators
  • * SPARK Streaming
  • * What is Spark Streaming
  • * Why we need Spark Streaming
  • * Differences between SPARK Streaming Vs Flume Vs Kafka
  • * Dynamic memory allocation of Streaming
  • * Configuration and settings for Streaming path and sources
  • * Spark SQL
  • * Why Spark SQL
  • * Integrating SPARK SQL with RDBMS
  • * Performance considerations using SPARK SQL
  • * Integrating unstructured sources to SPARK SQL
  • * RDD usage in SPARK SQL
  • * SPARK GRAPHX
  • * What is Spark GraphX
  • * How to create connected Data Structures
  • * Develop your first DATA GRAPGH
  • * Using RDD's and SPARK SQL as input
  • * Handling complex DATA GRAPHS
  • * SPARK R
  • * What is R?
  • * How to work on scientific calculations using SPARK R
  • * How to execute Statistical operations using SPARK R

Project Work : To differentiate between SPARK execution Vs Hadoop execution SPARK

COurse COntent

Introduction to DevOps

  • What is DevOps?
  • History of DevOps
  • Dev and Ops
  • DevOps definitions, Software Development Life Cycle and main objectives of the DevOps
  • Infrastructure As A Code
  • Prerequisites for DevOps
  • Tools (Jenkins, Chef, Docker, Vagrant and so on.)
  • Continuous Integration and Development

Linux Concepts

  • Linux Installation
  • User Management
  • Package Management
  • Networking

Automation Concepts

  • OS Basics
  • Scripting Introduction
  • Learn Shell Scripting
  • Database Concepts
  • Shell Variable, Decision Making and Shell Test Conditions
  • Shell Loops, Re-directors, Exit status

Revision Controls System

  • Subversion Controls/Git
  • Working with local repositories, remote repositories
  • branching
  • merging
  • cloning
  • fetch/pull
  • Installation of Git Server

Configuration Management

  • Chef/Puppet/Ansible Introduction
  • Chef server Hands-on, workstation setup, Chef Distribution Kit and Concepts

Environments

  • Attributes
  • Resources
  • Cookbook
  • Run list
  • Recipes
  • Supermarket

Build Automation

  • Introduction with Maven
  • Maven structure and Phases
  • Installation of Maven
  • Configuration
  • jar/war project structure

Tomcat Web Server

  • Installation and Configuration
  • Tomcat Manager
  • Application Management
  • App Deployment Methods

Nexus Artifacts/Proxy Tool

  • Introduction to Nexus
  • Installation and Configuration
  • Repository Management
  • Proxy Management
  • Integration with Maven

Jenkins Framework

  • Introduction to Jenkins
  • Jenkins Installation
  • User Profile and Management
  • Security and Plugins Management
  • Builds Setup
  • Integration with Git, Maven, Tomcat

LAMP Setup

Apache/HTTPD Web Service

  • Installation of Apache
  • Configuration of Apache
  • Static Pages
  • Dynamic Pages
  • PHP Integration

MySQL Database

  • Working with Database
  • Introduction to MySQL Database
  • Configuration
  • User management
  • Permission management
  • Creating Database
  • Data insertion/update
  • MySQL Data Backup, Hands-on and MySQL GUI Tools

Installation of WordPress with LAMP

  • Vagrant
  • Introduction to Vagrant
  • Vagrant Terminologies
  • Installation of Vagrant
  • Vagrant Proxy Project and hands-on

Working with Docker

  • Introduction to Docker
  • Docker Terminologies
  • Installation of Docker
  • Docker image creation and Docker hands-on

System Monitoring

  • Introduction to Nagios
  • Concepts behind Nagios
  • Nagios Installation
  • Hands-on

DevOps Project Work

  • Project LAMP Setup
  • Web layer
  • DB Layer
  • App Layer
  • Monitoring