Outlier detection and removal: z score, standard deviation | Feature engineering tutorial python # 3
If we have a dataset that follows normal distribution than we can use 3 or more standard deviation to spot outliers in the dataset. Many times these are legitimate values and it really depends on the situation if you want to remove them or not. But removing outliers can significantly increase the statistical power of machine learning model hence it is recommended that you treat outliers before building a model. Z score indicates how many standard deviation away a given sample is. We are going to go through all this theory and write python code to remove outliers from heights dataset that I have taken it from kaggle.
Link for kaggle dataset: https://www.kaggle.com/mustafaali96/weight-height
Code & Exercise: https://github.com/codebasics/py/blob/master/ML/FeatureEngineering/2_outliers_z_score/2_outliers_z_score.ipynb
CSV file for exercise: https://github.com/codebasics/py/tree/master/ML/FeatureEngineering/2_outliers_z_score/Exercise
Topics
00:00 Introduction
00:20 Exploratory analysis on a kaggle dataset
01:14 Plot histogram and bell curve
06:30 Use 3 standard deviation to remove outliers
12:14 Use Z score to remove outliers
17:39 Exercise
Website: http://codebasicshub.com/
Facebook: https://www.facebook.com/codebasicshub
Twitter: https://twitter.com/codebasicshub
Видео Outlier detection and removal: z score, standard deviation | Feature engineering tutorial python # 3 канала codebasics
Link for kaggle dataset: https://www.kaggle.com/mustafaali96/weight-height
Code & Exercise: https://github.com/codebasics/py/blob/master/ML/FeatureEngineering/2_outliers_z_score/2_outliers_z_score.ipynb
CSV file for exercise: https://github.com/codebasics/py/tree/master/ML/FeatureEngineering/2_outliers_z_score/Exercise
Topics
00:00 Introduction
00:20 Exploratory analysis on a kaggle dataset
01:14 Plot histogram and bell curve
06:30 Use 3 standard deviation to remove outliers
12:14 Use Z score to remove outliers
17:39 Exercise
Website: http://codebasicshub.com/
Facebook: https://www.facebook.com/codebasicshub
Twitter: https://twitter.com/codebasicshub
Видео Outlier detection and removal: z score, standard deviation | Feature engineering tutorial python # 3 канала codebasics
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Outlier detection and removal using percentile | Feature engineering tutorial python # 2Machine Learning Tutorial Python - 17: L1 and L2 Regularization | Lasso, Ridge RegressionStatistics-Finding Outliers in Dataset using Z- score and IQRZ-Scores, Standardization, and the Standard Normal Distribution (5.3)How to Find Outliers with ExcelStep by step roadmap to learn data science in 6 months | Complete data science roadmapTutorial 24-Z Score Statistics Data ScienceTutorial 32- All About P Value,T test,Chi Square Test, Anova Test and When to Use What?Machine Learning Tutorial Python 12 - K Fold Cross ValidationDifferent Types of Feature Engineering Encoding TechniquesOutlier detection and removal using IQR | Feature engineering tutorial python # 4How to remove outliers in Python? | For multiple columns | Step by step ♥Machine Learning Tutorial Python - 13: K Means Clustering AlgorithmTutorial | Anomaly Detection | Local Outlier Factor | LOF AlgorithmStandardization Vs Normalization- Feature ScalingMachine Learning Tutorial Python - 2: Linear Regression Single VariableHandling imbalanced dataset in machine learning | Deep Learning Tutorial 21 (Tensorflow2.0 & Python)Tutorial 2- Feature Selection-How To Drop Features Using Pearson CorrelationPython Machine Learning Tutorial (Data Science)