Feature Selection in Machine Learning

Today we will see what is Feature Selection and Various Methods used in this process.

Description

• Feature selection is a process where from all the features only the optimal subset of features are selected for the learning algorithm(model as well we call sometimes).

• Different methods are used to select best and required features for the model.

• Feature Selection is needed for :

  • Easy to interrupt or understand

  • Optimising training times

  • Reduces overfitting

  • Removing highly correlated variables

• Feature Selection Methods are below :

  • Filter : using features

    • Variance

    • Correlation

    • Univariate Selection

      • Mutual Information

      • Chi Square

      • F-Test

      • ROC_AUC or RMSE

  • Wrapper : using predictive ML

    • Step backward selection

    • Step forward selection

    • Exhaustive search selection

  • Embedded

    • Linear

    • LASSO

    • Tree Importance

      
      

Filter

• Filter methods are generally used as a preprocessing step

• The selection of features is independent of any machine learning algorithms. 

• Instead, features are selected on the basis of their scores in various statistical tests for their correlation with the outcome variable. 

• Visualization

Filter - Variance 

• Remove columns which as constant values, we can check wth standard deviation with 0

• Remove columns which have standard deviation of 0.01, like almost constant. 

  • But need to be careful while doing this. 

  • Also known as quasi constant features.

  • Eg: we have feature called A have values 1 with frequency 9999 and 0 with frequency1

• Remove columns which are same or remove duplicate columns

  
  

Filter – Correlation

• Remove columns which are highly correlated within independent variables

• Process : 

  • Finding highly correlated column groups

  • Find best column, which contributes more in target (Y) prediction

    
    

Filter – Univariate Selection

Mutual Information

• Measures how much each independent values contributes in the correct prediction on Y

• Works with only the numeric data

• Used with the continous values in X

• Feature Selection Methods :

  • Regression : sklearn.feature_selection.mutual_info_regression
  • Classification : sklearn.feature_selection.mutual_info_classif

• Best Feature Selection Methods :

• sklearn.feature_selection.SelectKBest
• sklearn.feature_selection.SelectPercentile

• Higher the mutual info score, more significant the features

• Caution: Rarely Used in Real Life

  
  

Chi – Square

• Measures how much each independent values contributes in the correct prediction on Y

• Uses Fisher Score

• Should be used to evaluate categorical variables in the classification task

• Feature Selection Methods :

  • Classification : sklearn.feature_selection.chi2

• Best Feature Selection Methods :

• sklearn.feature_selection.SelectKBest
• sklearn.feature_selection.SelectPercentile

• Here, lower the p-values more significant the features

• Note: Fisher score or any univariate selection model, with very big dataset shows small p value, therefore looking the low p-value we cannot conclude these are highly predicted features. But in reality this is because of large sample size

• Caution: Rarely Used in Real Life

  
  

F Test

• Measures how much each independent values contributes in the correct prediction on Y

• Works by selecting the best features based on univariate statistical test (ANOVA)

• Used with the continous values in X

• F-Test Method

  • Assumptions :

    • Normal distribution

    • Linear relationship between the features and the target

  • So while using this we need to think on the assumptions as well

• Feature Selection Methods :

  • Regression : sklearn.feature_selection.f_regression

  • Classification : sklearn.feature_selection.f_classif

• Best Feature Selection Methods :

  • sklearn.feature_selection.SelectKBest

  • sklearn.feature_selection.SelectPercentile

• Here, lower the p-values more significant the features

• Caution: Rarely Used in Real Life, but can be used for investigation

  
  

ROC_AUC or RMSE

• Uses ML to define most significant features

• But this is Time and Memory Consuming

• Features are selected using ROC_AUC or MSE

• And the highest ranked features are selected

• Get the numerical features, each X feature is used to fit for prediction Y value

• ROC_AUC :

  • Features with the values > 0.5 is selected, as considered more significant

  • Less then that would be the features would be random

• RMSE

  • Smaller the RMSE are more significant

Wrapper Methods

• Is also known as Greedy Search

• As this scans all the possible combination

• Often not feasible due to number of features are high in the dataset

• UPSIDE : Provides best features subset

• DOWNSIDE : Extremely computable expensive, ML Model Specific feature selected

• Combined with CV, will overcome the overfitting also

• Visualization

  

  
  

Step Forward

• Begins with one feature at a time and keeps adding one by one

• Algorithm

  • Select each features at a time and get the best one using accuracy

  • Later add the second feature with all the feature space and together calculate the accuracy

  • From the best combination accuracy, select the second feature

  • Continue this until all the k number of features are selected or no further improvement is observed 

• Method : mlxtend

  
  

Step Backward

• Opposite of Step Forward

• Evaluates all the features first, and removes one feature and evaluate algorithm performance

• Algorithm

  • Select all features at a time and get the accuracy

  • Later remove one feature, from all the feature space

  • Calculate the accuracy, and remove the least significant feature at each iteration which improves the performance of the model.

  • Continue this until all the k number of features are selected or no improvement is observed 

• Method : mlxtend

  
  

Exhaustive

• Try all the possible combinations, like tries all the single, group (combination) features

• PROS: Computationally expensive, at times unfeasible

• CONS: Provides best subset

• Algorithm

  • all possible combinations of 1 feature

  • all possible combinations of 2 feature

  • all possible combinations of n feature

  • and selects the one that results the best in performance

• Method : mlxtend

Embedded

• Embedded methods combine the qualities’ of filter and wrapper methods

• It’s implemented by algorithms that have their own built-in feature selection methods

• Visualization

Regularization

• In regression we always try to find Low Cost Funtion Score, for the best fitted line, with less errors in Training Set

• Regression Coefficient :

  • y = β0 +  β1X1 + β2X2 + .. + βnXn

• The coefficients are predictors the Bs are directly proportional how the features contributes to the final value y

• Regularization consists of adding penalty to the different parameters

• Hence the model is less likely to fit the noise of the training data and improving the generalization

• In general the below models are used when the Simple Regression model predicts overfitting results, so these techniques are used to reduce model complexity and prevent over fitting

• Types of Linear Model

  • L1 Regularization : Lasso

  • L2 Regularization : Ridge

  • L1|L2 Regularization : Elasticnet

• Assumptions :

  • Linear relationship between X and Y

  • Xs are independent

  • Xs are not correlated to other independent feature

  • Xs are normally distributed

  • Xs should be scaled as the coefficients are directly influenced by the scale of X

  • So to select important features its very much important all the X should be in the Same Scale

    
    

Lasso Reguralization

• Adding penalty to the coefficient

  • penalty is lambda

• This is done to avoid overfitting

• Formula : 

  

• Formula to find Coefficient:

  • 1/2m * ∑(y – ypred)ˆ2 = ႓ ∑ϴ ˆ1

  • m : number of observations

  • y : observed

  • ypred : predicted output

  • ႓ : lamda regularization parameter, penalty

  • ϴ : coefficient, its power one to make it always positive

• Note: when λ → 0 , the cost function becomes similar to the linear regression cost function

• So lower the penalty it will resemble the linear regression model

• The higher the penalty the bigger the generalization

• This ML model, tries to minimize the difference between y and ypred, works to reduce all the three including the regularization component

• Here the ϴ is at times moved to 0 so some features could be removed, as the formula doesnt takes the Square of Cofficient only the magnitude is taken into consideration

• Hence this is used for Feature Selection, not for model optimization

• Model Optimization : Reduce over fitting

• Method: 

  • sklearn.linear_model.Lasso

• Increase in the penalty will remove the features, so we need to keep this in mind

  
  

Ridge Reguralization

• Works same as Lasso, in Ridge the ϴ may be changed to close to 0 but never the ϴ is never turned to 0, as the coefficient is always squared

• Hence Not suitable for feature selection, but used for model optimization

• Model Optimization is done to reduce over fitting

• Formula :

Tree Based

Decision Tree

• Most popular way to use in the feature selections

• Highly accurate in training set

• Provides good generalization

• Helps in bringing Interpretobility

• Algorithm

  • First Node: Highest Impurity

  • Impurity decreases further each level

  • Until the impurity is 0

• More the feature decreases the impurity, the more the the feature importance

  
  

Random Forest

• Contains several decision trees

• Decrease of impurity for each feature is averaged accross trees

• Algorithm

  • Build random forests

  • Calculate feature importance

  • Remove least important feature

  • Repeat till condition is met

• Less of real life use, although doesn't give more benifit in removing feature

• Limitation

  • If correlated are not removed both the variable will show equal low importance

  • If a feature removed is correlated to another feature in the dataset, by removing the correlated features the tree importance of the feature will be revealed

  • Highly baised with the categorical variables

Reference and Credits

https://www.analyticsvidhya.com/blog/2016/12/introduction-to-feature-selection-methods-with-an-example-or-how-to-select-the-right-variables/

https://www.datacamp.com/community/tutorials/feature-selection-python

https://towardsdatascience.com/feature-importance-and-forward-feature-selection-752638849962

https://towardsdatascience.com/ridge-and-lasso-regression-a-complete-guide-with-python-scikit-learn-e20e34bcbf0b

https://towardsdatascience.com/of-suppandi-regularization-and-lasso-the-feature-selector-2a09acfdbc1b

https://miro.medium.com/max/1400/1*MO2JGos1gZnyfQueFrzzsw.png

Internet Security

Introduction

In today's digital world most of activities rely on the internet. Tons of data and sensitive information is shared over internet each millisecond. Internet is having high risk of intrusion by hackers, frauds and cybercriminals, internet security is a top priority for individuals and businesses alike.

Internet Security is the branch of Computer Security. Internet Security consists of a range of security tactics for protecting activities and transactions conducted online over the internet. 

Today we will see some of the threats and tactics to secure. To start lets see who are the Threat Actors.

Threat Actors

Figure 1: Threat Actors

A threat actor or malicious actor is either a person or a group of people that take part in an action that is intended to cause harm to the cyber realm including: computers, devices, systems, or networks.

Categories of threat actors

• Open Source Intelligence

  • Information from websites, such as CEO of the company

  • Similarly many post information from social media such as Facebook and Twitter

• Script Kiddie

  • Uses existing computer scripts or code to launch attacks

  • Motivations vary, but typically boredom

• Organised Crime

  • Cyber criminals have two main objectives. First, they want to take infiltrate a system to access valuable data or items.

  • Second, they want to ensure that they avoid legal consequence after infiltrating a system. 

  • Cyber criminal can be broken down into three sub-groups: mass scammers/automated hackers, criminal infrastructure providers, and big game hunters

• Hactivist

  • Launches attacks as part of activist movement or a further cause, usually not for own benefit but instead increase the awareness

• Insider

  • Anyone who is legitimate to access an organisation's internal resource. Issues, loss of confidentiality, internal resource, availability of organisation's assets

• Competitors

  • Outside organisations seeking to commit corporate espionage for financial or market gain

Example - APT10

• About

  • APT10 is a Chinese group that has been around since early 2009. Their primary mission seems to be targeting defence contractors around the world. Some other names they go by are Potassium and Red Apollo.

• Techniques

  • Some techniques used for recon and initial compromise include advanced spear phishing from a known email domain (spoofed), and creating fake video game advertising emails as well.

• Tools

  • Popular tools used by APT10 include Haymaker backdoor, Sungridge, and the Bug Juice backdoor

• Targets

  • Main targets for APT 10 are any government entities, defence contractors, healthcare and U.S. Lobby groups, agencies or groups in major countries including the US, France and Germany being of the largest.

Below are some of the Malware Types explained.

Virus

Figure 2: Virus

• Virus is a malicious code that attach itself to the host application, when the application runs the malicious code also runs
• Virus is abbreviation of Vital Information Resources Under Siege

• Virus are named after human, as it propagates device to device

• It doesn't needs who click anywhere but it needs to be executed as a program

• Virus won't show up or cause damage immediately, because it takes time to replicate first

• The user often executes the virus though its not intentional, but often times OS system will automatically execute after user interaction, eg: usb drive

• Virus types: 

  • Program virus

  • Boot Sector virus

  • Script virus

  • Macro virus

• Though anti-virus is very common, thousands of new virus is released every week

• Have anti-virus updated always to be secure.

Worms

Figure 3: Worms

• Worms is a self replication malware that travels throughout the network without any assistance of a host application or user interaction

• A worm reside in memory and can use different transport protocols to travel over the network

• Each infected systems tries to locate and infect other systems on the network, and network performance can slow to crawl

• Devices infected with worms will show symptoms of erratic behaviour and slow response times.

• Any code designed to do more than spread the worm is typically referred to as the "payload". Typical malicious payloads might delete files on a host system (e.g., the ExploreZip worm)

• If the worms are well known we can remove it from the the firewalls and IDS | ISP (Intrusion Prevention/Detection System) can mitigate many worms infections, but doesn't help much once the worms get inside

• As said getting rid of worms isn’t impossible, but may result in a clean wipe of the device and its files.

Trojan

Figure 4: Trojan

• Software that pretends to be something else, so it can conquer your computer

• Trojan doesn't really cares about replication

• Circumvents your existing security

• Once its inside it has free reign, and it opens back door

• Trojan horse can come as pirated software, as useful utility, game or something else the user wants to download

• Another method of Trojan is Rogueware

• Rogueware masquerades as a free antivirus program

• When a user visits a site, a message on the web page or a pop-up appears indicating it detected malware on the user's system

• The user is encouraged to download and install the free antivirus software

• Never download or install software from a source you don't trust completely.

• Never open an attachment or run a program sent to you in an email from someone you don't know.

• Keep all software on your computer up to date with the latest patches.

Ransomware

Figure 5: Ransomware

• A specific type of Trojan is Ransomware, attackers encrypt the users data or take control of the computer and lock out the user

• Then they demand that the user pays a ransom to regain access to the data or the computer

• Criminals often deliver ransomware via drive by downloads or embedded in other software delivered via email

• Attackers originally targeted the individuals with ransomware

• However they have increasing been targeting organisations demanding larger and larger ransom

• Crypto Malware, new generation ransomware

• After the pandemic the increased trend of cyber attacks that began in 2020 showed no signs of slowing down.

• Example – JBS Foods: 

  • High-profile ransomware attack took place this May on JBS Foods, one of the biggest meat processing companies in the world. 

  • The same Russia-based hacking group that attacked Acer, REvil, is thought to be behind the attack. 

  • Although there weren't any major food shortages as a result of the attack, government officials told consumers not to panic buy meat in response. 

  • On June 10^th 2021, it was confirmed that JSB paid the $11 million ransom demand after consulting with cybersecurity experts. 

  • This massive payment in bitcoin is one of the largest ransomware payments of all time.

  • Reference: https://illinois.touro.edu/news/the-10-biggest-ransomware-attacks-of-2021.php

Keylogger

Figure 6: Keylogger

• Keyloggers capture a user's keystrokes and stores them in a file

• This file can be automatically sent to an attackers or manually retrieved depending on the keylogger

• Keylogger is a software

• Your key strokes are in a saved in a clear text

• Other data logging, could be also done, such as clipboard, screen shot, instant messaging, search engine

• Keylogger is usually installed with the any other malware

• Recommended you to block unauthorised communication, through firewall and monitoring

Spyware

Figure 7: Spyware

• Spyware monitors a user's computer and often includes a keylogger

• Malware that spies on you, advertising, identity theft, affiliate fraud

• Spyware is a software installed on the user's systems without their awareness or consent

• Its purpose is often to monitor the user's computer and user's activity

• Spyware takes user's computer information to third party, its private data, it would result in loss of confidentiality

• Poor performance, installing additional software within the browser, changing the user's home page

• Can trick you into installing, peer to peer, fake security software

Adware

Figure 8: Adware

• Biggest ways to make money is through ads, the bad guys target's users by installing adware by making your computer

• Showing popups after popups

• May also cause performance issues

• Internet marketers, have become very sophisticated and use a combination of web and user behaviour analytic, track users habits for the purpose of targeting advertisements

• As this became more malicious by gathering more information of use that people starting calling as spyware

• Removing of adware is very challenging as its difficult to trust who would remove, because the authors would have removal software but that would end up having more adware to be installed

• Adware also applies to software that is free but includes ads, not like paid versions

Bot and Botnets

Figure 9: Botnet

• Bots are software robots

• Botnet combines the words robot and network

• Both are often called zombies and they will do the bidding of whoever controls the botnet

• Bot herders are criminals who manage botnets, they attempt to infect as many computer and control from one or many servers running command control software

• A day of life of bot, sit around and wait for instructions

• Most computer join a botnet through malware infection

• Eample: Botnet – Mirai, 2016, launched an attack against DNS server, including simple devices such as digital cameras, printers that are connected to the internet, and the bot herders directed the device to repeatedly query DNS servers in protracted distributed denial of servers (DDOS) attack

Social Engineering

Figure 10: Social Engineering

• Social engineering is the practice of using social tactics to gain information or performing actions they wouldn't normally take.

• These attacks can occur in person, over the phone, while surfing the internet and via email

• Using flatter and conning

• Assuming a position of authority

• Encouraging someone to perform a risky action 

• Encouraging someone to reveal sensitive information

• Impersonating someone, such as an authorised technician

• Tailgating or closely following authorised personnel without providing credentials

Impersonation

Figure 11: Impersonation

• Some social engineers often attempt to impersonate others

• The goal is to convince an authorised user to provide some information or help the attacker defeat a security control

Dumpster Diving

Figure 12: Dumpster Diving

• Is a practice of searching through the trash to gain information from the discarded documents

• Many organisation shred or burn instead of throwing to away

Shoulder Surfing

Figure 13: Shoulder Surfing

• Shoulder surfing is simply looking over the shoulder of some to gain information

• When a user is at computer or remotely using camera

• Prevention, position of monitors to that unauthorised personal cannot see them

• Screen Filters help to prevent by obscuring the view for people unless they are directly from of the monitors

Phishing

Figure 14: Phishing

• Spam is unwanted email, phishing is malicious spam

• Attackers attempt to trick users into revealing sensitive or personal info or clicking on a link

• Beware of emails from friends, impersonating friends using social media

• Phishing to install malware, attach email with malware and sent across

• Could be to validate your email address

• Phishing:

  • Emails are typically sent by a known contact or organisation including malware

• Spear Phishing

  • Same goal, but the attackers first gather information is used to personalise the spear phishing attack

• Whaling

  • Same personalised technique is used but to attack for specifically high profile targets

Vishing

Figure 15: Vishing

• Vishing is the type of cyber attack in which voice communication is used for stealing the confidential data from group of people. 

• In vishing, attacker tricks the target to give the sensitive information through the voice call pretending to be an employee from the related and trusted firm.

• Example –

  • Asking for bank transaction OTP from users.

  • Asking for UPI PIN from users.

• Don't Share Personal Information Over the Phone

• Don't Answer Calls From Unknown Numbers

• Ask for Proof of Identity Report

Malware Spread

Figure 16: Malware Spread

• So far we have seen many types of Malwares.

• Malware spreads in computer when downloaded or install an infected software.

• Could be through ad or suspicious popup ads.

• They also enter your computer through an email or a link. 

• Once malware enters the computer, it attaches itself to different files and overwrites the data.

• As malware travels within the network, it infects the computer it moves into.

• Advisable to have anti-virus up to date, and configured required firewalls.

Security Measures

As said prevention better than cure, very much applies to cyber security. Below are few measures should aim to prevent risks from various sources. 

• Awareness

• Education

• Anti Virus

• Backups

• Patching

• Strong Passwords

• Spam Filters

• Firewalls

• Encryption

• Hashing

• Digital Signature

• Monitoring

Reference:

• Different links from https://en.wikipedia.org/

• https://www.mcafee.com/enterprise/en-in/security-awareness/cybersecurity/what-is-internet-security.html

• Credits to the different sources from there taken these images

Design Patterns - An Introduction

About Design Pattern

Design pattern is just like cookie cutter, a general templated solution for common or similar problem in a software product. Each pattern can be said as template solution, which can solve a particular design problem in the software code.

You may wonder this is what we have been calling Algorithm, did something change? No Algorithm and Design Pattern are two different concept for solving a particular problem. Algorithm is exact steps need to be followed to get the desired results, where as Design Pattern is best practices to solve common problems when designing a software application.

An analogy to an algorithm is like following maps to reach a destination, but with design pattern is more like choice of transport used, for shorter distance walk, then cycle, car, if in water then boat/ship or via air and so on. As design pattern allows developer to provides suggestion and patterns, but implementation is up to him/her. 

The choice of programming language is very important because it decides the patterns could be used.

Need of Design Patterns

• Object Oriented Programs (OOP) are made of objects, is hard when decomposing a system into objects. As there are may factors come into picture such as encapsulation, granularity, dependency, flexibility, performance, evolution, readability, reusability, security and so on. These have be addressed over a period of time and built as design patterns.

• Instead of each time reinventing the same wheel, we can use the proven patterns in the development process. 

• Development time saved, as already these patterns have the template which could be used but making required changes.

• As these patterns are proven solution, tested in various scenarios will help us to overcome the edge cases which could occur later after the implementation or productionized. Avoids burning the fingers again.

• Ease to communicate the idea and talk in the same language.

• As these patterns where not just built overnight, each pattern would have gone through various iteration. Makes its more reliable to use.

Design Principles

Before we discuss about design patterns its also very important to know about the Design Principles as it helps us in creating more maintainable, understandable, and flexible software.

The SOLID principles were introduced by Robert C. Martin in his 2000 paper “Design Principles and Design Patterns.”, later Michael Feathers built up and created the SOLID Acronym.

SOLID stands for:

• S - Single-responsibility Principle

  • Class should hold only one responsibility, so when there is new change it should be only for the single functionality.

• O - Open-closed Principle

  • Class should be open for extension but closed for modification. This allows the class to be stable avoiding new bugs.

• L - Liskov Substitution Principle

  • Allowing the subclass to independently working even when the parent class is not available or can be said subclass would be able to replace parent class. Hence the dependency of the parent class can be vanished if the child can operate fully without altering the desirable properties of the program.

• I - Interface Segregation Principle

  • Larger interfaces should be split into smaller ones, allowing the class to use only the required ones.

• D - Dependency Inversion Principle

  • Instead of high level modules depending on low level modules, both should be be implemented on abstractions.

Classification of Design Patterns

Originally Design patterns were categorized into 3 sub classifications based on the problem it solves. But to my wonder could find 4 types in wiki, may be later it got added. As I find all the 4 are valid sub classification, lets see what is holds.

Creational Patterns: Provides capability to create objects based on the required criteria or flexibility to reuse or certain constrained. In simple term creational patterns concern the process of the object creation. Below are few patterns:

• Singleton

  • Ensures a class only has one instance at any given time and provides a global access to the object.

  • Eg: When you write in stdout, its the same for all the packages or modules. There cannot be multiple file descriptor for a single stdout.

• Abstract factory

  • Provides an interface for creating similar or same family of classes without the class definition.

  • Eg: When designing class for car, bike, auto we can have abstraction of Vehicle having methods like no of wheels, speed, and so on.

• Factory

  • Defines a family of classes and let the factory method decide which class needs to be instantiated.

  • Eg: When designing transport service, there would be car, bike or auto services available. The book any option will act as a Factory class which will book either of this service based on the destination.

• Builder

  • Break down the complex structure into simple objects and building the complex object by series of step by step process.

  • Eg: Assume you are building coffee vending machine, first option would be selecting water or milk, then adding coffee or tea, and then adding sugar or not. Like wise step by step adding small objects to build a product.

• Dependency Injection

  • Makes the class independent and decoupled so it can be for their integration and testing. 

  • Eg: When you build a system to book tickets, it could be easily decoupled either for booking movies or bus tickets the system varies. The same class can be used for both the system depending on where its plugged. 

• Lazy initialization

  • Delays the construction of the values until the data is actually called or needed.

  • Eg: When you define arrays in python it will not know and not allocate memory in huge sizes. Once you start using linearly the memory will be allocated.

• Multiton

  • Updated version of singleton, instead of holding one instance Multiton holds group of instances created and reserved to access globally.

  • Eg: Lets say holding a dictionary of output methods, for error print in stderr else in stdout

• Object pool

  • Object pool is a container which contains some amount of objects where the cost of initializing the class is high. So, when an object is needed its taken from the pool, and later after using it is pushed back to available in the pool. So it may be unavailable until it is put back.

  • Eg: Let's say you want to create array of database connections and reserve it in the array and use them when needed and return them back to the pool once used.

• Prototype

  • Creates the new object by copying the existing object rather creating the object from scratch, allowing to hide the complexity of making new instances.

  • Eg: When you build car of specific type and want to replicate it, using prototype you can make a exact copy of the object with its values.

Structural Patterns: Deals with the composition of classes or objects. As how to assemble the objects and classes into larger structures which creates new functionality while still flexible to extend and efficient to use. Below are few patterns:

• Adapter

  • Create an interface to enable communication between the two or more incompatible classes. Adapter lets the classes work together that couldn't work otherwise because of the incompatible interfaces.

  • Eg: Mobile phone chargers when you are in India you use a different style plug but whereas in US you have different plug option, does that mean need to replace mobile. No we can use an adapter that fits in different countries accordingly.

• Bridge

  • Split the larger or complex classes of similar types into two separate hierarchies, by decoupling the abstraction from its implementation so both of them independently vary.

  • Eg: Let say you want to build a Vehicle it could be car or bike. I have parent class as Vehicle and sub classes as Car and Bike. Now I have requirement introducing Petrol or Electric. It would hard to have these combinations like Car Petrol and Car Electric or Bike Petrol and Bike Electric. Rather here I have another Interface called RunsOn where it can be extended to Petrol and Electric. Finally in the Vehicle class RunsOn is the object. 

• Composite

  • Compose the objects like a tree structure building hierarchy then we can work with these object as individually 

  • Eg: We would want to build approval work flow, first level will be immediate manager and next would be boss's boss and so on. Finally we will have CEO. Here the hierarchy is built. 

• Decorator

  • Placing object inside wrapper objects this helps to enable additional/extending features on the wrapper layers of the object.

  • Eg: Authentication is a good example, when you are in office logging in it will ask for username and password. The same application when accessed from outside will ask for username and password and MFA. Later if you typed password wrongly twice it will be wrapping to security questions. So you could see the layers are built on top.

• Extension Object

  • Anticipate the object interface needs to be extended in future. Additional interfaces can be added.

  • Eg: Lets say I need to build an app for Android, but would create interface called platform. As later we can have this running in other OS.

• Facade

  • Provides the simple high level interface for a complex system making it easier to use.

  • Eg: Making an order in Amazon, later all the other tasks right from dealing with vendor to delivery is taken care.

• Flyweight

  • Sharing the common objects in different contexts, in-order to save space supporting large number of objects saving and using efficiently.

  • Eg: Moving to example of building car or bike. I would need spare parts for each many instead of saving each of its parts in different parts occupying ram. Initialize them in common and use the appropriately.

• Module

  • Encapsulation of both the private and public methods and variable in single object. Protecting the states and behaviour of being accessed from other classes.

  • Eg: Let say you want to build House class, you will have locker as private where it cannot be accessed from outside.

• Proxy

  • Providing the surrogate or placeholder for another object to control access, in simple terms ability to perform an action on behalf of someone.

  • Eg: When you forward invite on behalf of someone. On a lighter no when you give proxy attendance of your friend who skipped the class.

Behavioural Patterns: Provides effective communication and assignment of responsibilities between objects. It defines the ways in which classes or objects interact and distribute responsibilities. Below are few patterns:

• Chain of responsibility

  • In this pattern the responsibilities are handled in a chain format, where each handler gets the request and see if it can handle if not pass it to the next handler.

  • Eg: When you click a video, it will first look into the local cache if available. Else the request will go the CDN. If not found will reach the application server. If not found finally get it from the db. Here chain of responsibility is performed.

• Command

  • Calling an object by passing parameters to it, allowing the class to work on the parameters and result. This allows to serve different requests, queue or log requests and support undoable operations. The Execute method of the command knows how to bring all the pieces together to get the work done.

  • Eg: Unix command is the good example for this, lets take “ls” command it takes various options and displays the output based on the options passed.

• Interpreter

  • While given a language with a defined grammar, an interpreter class uses to define this and provide a interpreter to make meaning out of the sentences in the language. 

  • Eg: SQL query is defined a particular format and we need a parser to make meaning out of the query.

• Iterator

  • Provides a way to access the elements of an collection object, traversing sequentially without exposing the underlying implementation of the class.

  • Eg: List, stack, tree, queue are some of the example of the Iterator class.

• Mediator

  • Reduces the chaotic interaction between objects. Mediator works as a middle man to collaborate between objects. Promotes loose coupling by keeping objects only referring the mediator class.

  • Eg: Load balancer could be said as middle man, where the different clients do not interact with different servers sitting. Each talks to the load balancer, and load balancer will talk to the server and reply back the response to the client.

• Memento

  • Storing the state of object without compromising on encapsulation or details of the class implementation.

  • Eg: Undo operation while performing in the word edit.

• Observer

  • Subscribe-Publish can be also called to this pattern. Here there can be multiple object subscribing to the parent class and the publish will send the messages or changes to all the subscribed dependents. 

  • Eg: You have an output which needs to be sent as email, stored in db and viewed in html. Here you can have one output class as publisher and all others as subscribers. When the output is generated, it passes to the subscribed list.

• Servant

  • Servant is used for providing some behaviour to a group of classes. Instead of defining that behaviour in each class or in the common parent class - it is defined once in the Servant.

  • Eg: In a mobile we have several features related to Bluetooth, we can isolate this class and call in the master class. So the servant class features will be called when required, without overloading the master phone class.

• State

  • Enables the object to change when the internal state of the class changes. The object will appear as the change happened in class.

  • Eg: When a phone is locked it can only perform certain actions, but where as when the phone is unlocked it can perform different set of actions. The internal state of phone decides the course of actions.

• Strategy

  • Defines a family of algorithms and encapsulate them in one. While client using the strategy decides which class to be used.

  • Eg: Coming back to transport service, you have car, bike and auto grouped, as a user you can decide based on the budget and time which to pick.

• Template

  • Defines the skeleton of an algorithm in the parent class, allowing the child classes override specific steps of the algorithm without changing its structure.

  • Eg: Building different models of Tata car, well the primary structure remains the same but only few customization is done based on the model.

• Visitor

  • Enabling a operation performed on the object structure without changing the class of elements on which it operates.

  • Eg: Imagine you are car seller, when you visit professionals you would prefer selling sedan, for teens would prefer sports and kids battery cars. Ideally you are selling but based on the object structure it operates.

Concurrency Patterns: Provides effective patterns when dealing with either multiple threads or multiple computations happening in the same time.

• Active Object

  • Active Object pattern separates invocation of the method from its execution. That means the requests called from the client but not been executed directly, rather it is added to the queue maintained in the different object (Active Object). These requests will be running in its own thread of control but invoking asynchronously by picking up from the queue. This pattern is recommended when avoiding race condition and execution of the request in queue handled by the scheduler.

  • Eg: When you attach multiple files in the gmail, each will be running async.

• Balking

  • The word balk means hesitate or unwilling to accept, which actually matches with the pattern type. In Balking pattern the execution of the object occurs only if its present in particular state, else it will balk or in simple words throw an exception saying the state of the object is not matched hence balking. The state is maintained internally.

  • Eg: Designing a running car, when the car is not started but pressed acceleration it should balk. Similarly if the car is in acceleration mode but also pressing the break, it should again balk.

• Double checked locking

  • We have seen Singleton before, where an object can be initialised only once. But assume we are running in threads, there is a possibility of the multiple object created by each thread. To avoid using double checked locking pattern where the object is check twice before the performing the operation. 

  • Eg: Continuing with our Singleton pattern, to have singleton pattern implemented we create a global variable and verify it is null just then create an object. Now when this is running in threads, its execution is not under user control rather it is under kernel control. Let's see where we can have problem, the first thread checking if the value is null, yes its null. Now the execution moves to second thread it checks if the value is null, yes its null. Now the execution moves to first thread the object is created, similarly it happens with second thread. So our object is no more called once, well its been created twice.  In double checking the global variable is checked if its null, yes then perform lock and then again check if the object is null if yes then create the instance of the class.

• Guarded suspension

  • When running in threads we would need to check if the lock is acquired as well precondition is met before performing the operation. In this pattern the object method call is suspended until the precondition is met, we can say it acts as guard.

  • Eg: Let's say we are building an app where it needs to access the photos folder, but it cannot until we have go ahead from the user. So here the precondition would be acceptance of the user before performing the operation.

• Monitor Object

  • Monitor pattern controls the concurrent access of the method of the object. Let say concurrently two or more threads are calling a method at a same time, only one thread can execute this particular method

  • Eg: Assume we are having multiple threads writing in same files causing mixed up of the information, can use this pattern, allowing one thread to write at a time. 

• Thread pool

  • In this pattern we maintain pool of threads, waiting for the concurrent execution. This allows us to speed of the process, avoiding creation and destruction of threads for short-lived tasks.

  • Eg: Creating a pool of threads to process quick user queries, where the performance is important and usually it would be single response, such as status of ticket and so on. 

    
    

Use a Design Pattern

• Select the design pattern based on the requirement, study the patterns and it similar patterns and drill down patterns which fits the best.

• Understand the cons and pros of the selected design pattern

• Get the template code 

• Build on top of the template by defining the classes based on the application requirement

Challenges using Design Patterns

• Every solution has or creates new problems or challenges, the same goes to design patterns.

• Not every design pattern can be used or needed in each programming languages

• Modern programming languages may have simple approach to resolve the solution

• For not all problems design patterns can be used

  
  

Conclusion

We have seen what is design pattern and design principles. Classification and subtypes of the design patterns is covered. Finally the pros and cons of using a design pattern. These are not the complete list of design patterns we have many more and many more are yet to be discovered. We have not covered distributed system design patterns here, Leader and Follower is one type of de.