Information Security - Governance, Risk, and Compliance

  

Security is an ongoing process, it needs to present in every phase in an organization. Risk is impossible to avoid, but we can always take necessary precautions by preventing if not by minimize the impact and limit the damage. This process is known as security controls. Managing risk is an important aspect for any organization, it requires strong governance by understand the goals, critical functions performed by an organization. Meeting the compliance standards of laws, policies and regulations is very essential to avoid fines, incarceration or loss of business or employment, all this includes reputational damage. Scope of laws covers the state, territory, national or international based on organization. Today we will see highlights from the shoes of security consultant.

Information Security

Information should be protected in all the terms on rest, in transit and while processing. We have below properties to call when the data is secure from both hardware or software.

• Confidentiality
• Only required party should know the data
• Integrity
• No data is tampered, against the will
• Availability
• Information accessible when required
• Non Repudiation
• Cannot deny on the action performed

Risk Management Components

• Policy: Proposes principle of actions on which an organization is built, using to make decisions.
• Standard: Set of requirements which must be mandatory to be adhered by everyone.
• Guideline: Provides recommendations or suggestions and best practices.
• Procedure: Provides step by step instructions on how the policy, standard or guideline needs to be implemented. Also known as Standard Operating Procedure - SOP but tend to have more specific details on the instructions.

Credit: https://www.ccexpert.us/firewall-fundamentals/the-difference-between-policies-standards-guidelines-and-procedures.html

Security Framework and Standards 

Security framework is list of actions takes to mitigate risk. The use of the framework allows the organization to set benchmarks in terms of security, define tasks, prioritize projects and allocate required funds to achieve. Organizations follow these frameworks based on the org's location, size, type of industry to ensure legal compliance and strengthen org's security. It is important to build security framework personalised for each organization as each is unique but following the best practices from the industry standards it good to start.  

Below are few secure frameworks for both organisation and cloud.

• NIST
• NIST is abbreviated as National Institute of Standards Technology.
• Developed for US and US government, but can be applicable for other countries as well.
• NIST Risk Management Framework (RMF), specifically used for federal agencies, and mandatory for US federal agencies. 
• Six step process:
1. Categorize or define environment
2. Select appropriate controls
3. Implement controls
4. Assess controls
5. Authorize a system
6. Monitor for ongoing compliance
• NIST is responsible for issuing Federal Information Processing Standards (FIPS).
• NIST is also responsible for issuing advisory guides called as Special Publications.
• NIST provides a list of checklists and benchmarks for variety of operating system and applications called National Checklist Program (NCP)
• NIST Cyber Security Framework (CSF), is responsible to mitigate cybersecurity risks. CSF is can be classified as five functions, which is developed by the NIST.
• Identity
• Develop security systems to mitigate risk, threats, vulnerabilities
• Protect
• Build secure systems at every phase
• Detect
• Perform ongoing monitoring which are capable to identify and protect against  different and new threats
• Respond
• Action against the detected risk, threats, vulnerabilities
• Recover
• Restore system or data if unable to prevent the attacks
• NIST GGSS
• NIST Guide to General Server Security addresses general security issues related to typical servers.
• ENISA
• European Union Agency for Network and Information Security - ENISA is similar to NIST that focuses on information security enterprise for the EU.
• ISO/IEC
• Common Security Framework ISO/IEC
• ISO - International Standard Organization
• IEC - International Electrotechnical Commission
• ISO 21K
• ISO 21K is cyber security framework
• ISO 27K
• ISO 27K contains overall series of information security standards
• ISO 27001 information security management standards
• ISO 27002 provides best practices for information security controls
• ISO 27017 and 27018 references cloud security
• ISO 27701 focus on personal data and privacy
• ISO 31K
• ISO/IEC 31K is an international standard providing an overall framework for enterprise risk management practices - ERM, includes performing risk assessments
• CSA
• CSA is abbreviated as Cloud Security Alliance
• CSA is a non-profit organization to assist cloud service providers - CSP for providing secure platforms
• CCM - Cloud Controls Matrix, security guidance for using cloud platform and best practices to be followed
• CCM list of specific controls should be adopted by the CSPs.
• COBIT
• Control Objectives for Information and Related Technology - COBIT
• A set of best practices for IT management.
• COSO
• The Committee of Sponsoring Organizations (COSO) of the Treadway Commission for enterprise governance and risk management to reduce fraud in organizations.
• HITRUST CSF
• Health Information Trust Alliance Common Security Framework for healthcare information.
• SSAE
• The Statement on Standards for Attestation Engagements - SSAE
• Audit specifications developed by American Institute of Certified Public Accountants AICPA for the CSPs.
• SOC2 - Service Organization Controls, results in reports which evaluates the internal controls implemented; practices around confidentially, integrity, availability and privacy; by the service providers to ensure compliance with Trust Service Criteria - TSC where storing or processing customer data. Includes evaluating the firewalls, intrusion detection, multi factor authentication.
• SOC3 - a less detailed report of SOC2 for everyones knowledge.
• Type I audit
• Tests controls in place at a particular point in time.
• Type II
• Tests controls over a period of at least six consecutive months.
• CIS
• Centre of Internet Security, non profit institute developed by SANS Institution,  maintaining the security controls.
• CIS CSC - Critical Security Controls for effective cyber defence. It is designed to improve cyber defences making it implementable for IT professionals.
• CIS RAM - Risk Assessment Method, can be used to perform an overall emulation of security.
• CIS CAT - Configuration Access Tool, can be used with automated vulnerability scanners to test compliance against the benchmarks
• STIG
• Security Technical Implementation Guides developed by Department of Defence Cyber Exchange, hardening guide for hardware and software
• OWASP
• Open Web Application Security Project - OWASP, is a non-profile community publishes best security practices and have built tools Zed Attack Proxy/Juice Shop to help to investigate security issues. 
• OCTAVE
• Operational Critical Threat, Asset and Vulnerability Evaluation - OCTAVE
• For educational institution.

Security Job Roles

The role of a security engineer differs from organization to organization, also based on the size and nature of the business. 

• CIO, CEO, CTO
• Liable for external and internal security
• Director of Security, Chief Security Officer, Chief Information Security Officer
• Overall responsibility for security
• Managers
• Responsible for the domain or section of the business/infra
• Information Security System Officer, Specialist
• Responsible for implementation, maintaining and monitoring
• Every Employee
• Responsible of the actions and needs to comply with the corporate policy

Information Security Business Units

Units within the organization to represent security function.

Security Operation Centre (SOC)

• A location where the security professional monitor and protect critical information across organization.
• The physical entry and exit is usually for restricted for all, allowing only required employees.

DevSecOps

• Traditionally the software development involves only in developing code, with increasing cloud and container driven architecture there is more collaboration between developers and system administrators leading to DevOps.
• DevSecOps has extended the boundary to security specialities, making security as primary consideration in the development and deployment.
• DevSecOps comprises of Development, Security and Operations making the code move production faster and secure.

Credits: https://gigaom.com/2020/12/11/security-by-design-why-devsecops-is-so-important/

Incident Response

• Dedicated team responsible for handling security incidents.
• Acts as the single point of contact as a independent business unit, but at times handled as part of SOC.
• Teams are called as:
• CIRT - Cyber Incident Response Team
• CERT - Computer Emergency Response Team
• CSIRT - Computer Security Incident Response Team

Security Control Categories

Security is met by implementation of the security controls, handling the properties of confidentiality, integrity, availability and non-repudiation. A control is the defence or countermeasure put in place to manage risk. 

Below are three major categories controls are classified into to protect data and systems. By adding multiple layers of control types, we are building a stronger system to avoid intrusion. This technique is called as layered defensive strategy or defence in depth.

Technical

• The control implemented as a system which could be hardware or software.
• Eg: firewall, anti-virus, logging, encryption, data classification system, access controls, security patches and updates, change the default authentications, hardening of systems - web application, os, application server, networking infrastructure devices

Operational

• The control implemented by people, forms outer line of defence includes physical controls and organization controls. 
• Eg: security guards, training programs, protecting storage systems including backup systems, surveillance systems 

Managerial

• The control gives oversight of the information system and address system design and implementation
• This is also called as Administration control
• Eg: risk identification, tools to evaluate or suggest security fixes, security awareness trainings, personnel background checks, change management process, security policies and procedures, 

Security Control Function Types

The security controls can further be classified in types based on the function they perform. We would ideally need to have prevention as its better than cure, but in reality we should be ready and have controls in place to take address all the mishaps.

Preventive

• The precaution taken to avoid an attack, physically control access in virtual world.
• Eg: ACL, firewalls, anti-virus, anti-malware - prevention from malicious content, control access including physical, door lock, security guard, IPS intrusion prevention system, firewalls, anti-malware.

Detective

• Identify the attack during the progress of an attack, here it may not prevent the attack.
• Eg: Log monitoring, motion detector, IDS, checksum, CCTV, alarms, security reviews and audit, mandatory vacation, rotation of duties, anti-malware - detection of malicious content in the existing system.

Corrective

• Correction or recovery made after an attack to mitigate the damage.
• Eg: Restore, patch management, backups can mitigate the attack, backup sites can provide control over natural or man made disaster, add IPS can block the attacker in future, anti-malware - fix infected files, vulnerability mitigation.

Physical

• Physical controls placed to avoid attacks in real world.
• Eg: Alarms, gateways, locks, security cameras, fences.

Deterrent

• Psychologically discourage the attacker to perform attacks, but may not directly prevent access.
• Deterrent, does not stop unauthorized access
• Eg: Warning boards, legal penalties, lights.

Compensating

• Controls which serves as the substitute if any of the primary controls are not in place, this does not prevent an attack but restores back to normal.
• Eg: Restore from the backups, hot-site, backup power systems.

Security Regulations, Laws and Standards

Organizations needs to follow compliance guidelines, based on the scope of the business which needs to cover international, national, territory and state laws. Compliance means meeting the standards of laws, policies and regulations. Laws associated to security has been created if breached will be held for fines, loss of employment, civil or criminal liabilities. 

• SOX
• Sarbanes Oxley Act, developed by US governs the financial and accounting disclosure information to protect investors from fraudulent financial reporting by corporations, mandates the implementation for risk assessments, internal and audit controls.
• CSA
• Computer Security Act developed in 1987 requires federal agencies to develop security policies for confidential information. 
• GDPR
• General Data Protection Regulations, developed by European Union for handling confidential data under data privacy act.
• User can control where the data goes, and have right to be forgotten.
• Each sites have privacy policy tells all in detail about the data gathered, data usage, data stored and user's privacy right.
• Privacy of individuals are protected which includes, Name, address, photo, mail address, bank details, identify proofs - PAN/Aadhar/SSN, medical information, IP address and so on.
• HIPAA
• Health Insurance Portability and Accountability Act, regulations for health organizations.
• PCI DSS
• Payment Card Industry Data Security Standard (PCI DSS) a standard for protecting credit cards in order to reduce fraud.
• Defines the safe handling and storage of financial information.
• Need to comply with PCI DSS if dealing with credit card in businesses.
• Six control objectives:
• Secure network
• Secure end-to-end card details
• Always patched
• Strong Access control measures
• Regular audit and monitor
• Maintain information security policy
• GLBA
• Gramm–Leach–Bliley Act (GLBA) establishes the privacy rules for financial services.
• CCPA
• California Consumer Privacy Act (CCPA), personal data regulations.
• FISMA
• Federal Information Security Act was introduced in 2002 to govern the data processed by federal agencies.
• PIPEDA
• Is a Canadian law that governs the collection and the use of personal information.

Security Policies

AUP - Acceptance Use Policy

• AUP provides a detailed documentation on how the company assets should be used, such mobile or laptop given from organization should not used for personal reasons.
• If any of the assets are misused, employer could take action against the violation.
• This policy will help the organization to minimize the risk.

Business Policies

Below covers following policies to increase the stability and decrease risk in an organization.

• Job Rotation, not allowing one to be in same position for ling time.
• Mandatory Vacations, a person needs to leave their job and go for vacation for some time. As during this time the task will be handled by someone, removing the dependency and validate if there was any security violations limiting the ability to commit fraud.
• Separation of Duties, allowing to split knowledge such as to unlock a system keys from two or more persons are needed.
• Clean Desk Policy, no sensitive information should be easily or openly available, always locked both hard copies and soft copies locking the system.
• Least Privilege, only required access provided to perform the action.
• Background Checks, while hiring making sure of your history is clear.
• NDA - Non disclosure agreement, contract preventing employees sharing confidential information.
• Social Media Analysis, using individuals social media policy allowing one to hire or not.
• Onboarding Policy, steps needs to taken care while hiring a new employee such signing the AUP, access to the system, trainings and so on.
• Off-boarding Policy, steps needs to taken care while the employee leaves such as access restriction, preserving encryption keys and so on.
• User Training, below are the ways users needs to be trained to perform the job.
• Gamification
• CTF - Capture the Flag usually security related competition
• Phishing simulation
• CBT - Computer Based Test
• In Person 
• Specialized training, before allowing to take the job to understand the role and security aspects.
• Vendor Policies
• Disciplinary and Adverse Actions, policies specify to consequences for violations.
• Exit Interviews, help to identify workplace factors that lead employee to leave the organization.
• Supply chain assessments, evaluate co-ordination and security process between groups.
• Business partners policies to handle risk effectively. 
• SLA - Service Level Agreement, terms decided on the services uptime, response and so on from the service providers.
• MOU / MOA - Memorandum of Understanding / Memorandum of Agreement, consent signed between both the sides.
• MSA - Measurement System Analysis will assess the measurement process and calculate the uncertainty. Used with quality management systems such as Six Sigma.
• BPA - Business Partnership Agreement, policies such as decision making agreements, owner stake, financial contract.
• ISA - Interconnection Security Agreement, purposes the technical requirement of the interconnections between organizations using shared IT systems.
• EOL - End Of Life, manufacture may end but continue support
• EOSL - End of Service Life, no longer provide support and security patches.

Privacy VS Security

• The value of the information is determined in the impact of it being compromised.
• Data security is an important factor, but privacy is an equal factor as well.
• Privacy: 
• Personnel data is any information about an identifiable individual.
• Data governance required when collecting and processing with personnel data.
• Privacy ensures there are required policies, to identify private data, storage, processing and retention.
• Limited access to the private data only to authorized persons.
• Security:
• Data security focus on CIA attributes.
• Data must kept securely while processing and storage.
• Identity management, allowing only authorized and authenticated person to read or write the data.

Information Life Cycle Management

• Creation
• Distribution
• Use
• Maintenance
• Archive
• Disposal

Data Roles and Responsibility

• Data governance
• Data owner
• Accountable for specific data often a senior officer
• Determine data classification
• Data steward
• Manages the data governance
• Responsible for data accuracy, privacy, compliance and security
• Data classification
• Maps sensitivity labels to the data
• Such as public, private, confidential, personnel, etc
• Data custodian/steward
• Implementing the data classification and security controls
• Data privacy officer (DPO)
• Responsible for the organizations's data security
• Sets policies, implements processes and procedures
• Data controller
• Manages the purposes and means by which personal data is processed.
• Data collector
• Data processor
• Processes the data on behalf of the data controller.

Data Classifications

• Public / Unclassified
• Non sensitive data
• Press releases, marketing materials
• Confidential
• Data reserved for certain employees with an organization
• Critical
• Data should always be available
• Proprietary

• Data disclosed outside the organization on a limited basis. Eg: NDA

• Private / Personal / Restricted / Internal Use Only / Classified
• Data used within specific division, eg: payroll details of employees only for HR and respective individually
• Sensitive
• Severe impact to the organization if it were exposed
• Note
• Data classification should also consider data accuracy, integrity and availability.
• Eg: Publicly classified data should be accurate.

Data Policies

• Privacy Notices
• Data Retention
• Keeps the files that change frequently for version control
• Impact Assessments
• Data Sovereignty 
• Geographical Considerations
• Data Sharing
• Privacy Terms of Agreement
• SLA
• ISA
• NDA
• DSUA
• Credential Policies
• General guidelines are password should not be embedded with the application, passwords in clear text much must not be saved nor be transferred over network.
• Every user needs to have its own user account and personal information should be access to him/her. Users should not have privileged access as this could lead the malware to run as the user easily.
• For administers also should have user account but for performing administration can use elevated access for each.
• Third party accounts can be created for additional vendor based applications access, but these accounts should also be never shared.
• Device accounts for accessing the device, holding the device certificate and require passwords for screen unlock. This can be managed through MDM, Mobile Device Management. For unlock could also include geolocation validation.
• Service accounts, access can be defined for a specific service and the password/key rotation policies should be place avoiding unexpected breaches.
• Administrator or root accounts, these accounts should not be used as normal administration.
• Change management policies ensuring the changes are planned, tested, frequency of the change, installation steps, fallback procedures and scrutiny plan has be done before making changes.
• Asset management, identify and track computing assets, making sure all the security patches are applied, track licenses. 
• FRCP : The United States, Federal Rules of Civil Procedure. have implications for data retention policies.

Data Types

• PII - Personally Identifiable Information
• Customer Data
• PHI - Personal Health Information
• Financial Information
• Government Data

Data Breach Consequences

• Organization Impact
• Reputation Damage
• Identity Theft
• Fines
• IP Theft
• Notification
• Escalation
• Public Notification and Disclosure

Data Protection

• Data at Rest
• Data in Transit
• Data in Use
• Data Exfiltration
• Rights Management Services
• Privacy Enhancing Technologies
• Data minimization
• Deidentification
• Data Masking
• Tokenization
• Aggregation/Banding
• Hashing and Salting
• Anonymization
• Making the data impossible to identify individual data from the dataset.
• The data cannot be reversed to actual data, no way to associate the data to the user.
• Pseudo anonymization
• Data Loss Prevention
• Policy Server
• Endpoint Agents
• Network Agents

Risk Management Process

Risk management process is to identify assets and threats, and risk (high, medium, low) associated to it that could be affected by an attack. Including the total risk to the organization, to make future plans.

Risk can be calculated:

    Risk = Threat * Vulnerability * Impact 

Risk Assessment

• Identify Threat and Vulnerabilities
• External Threats
• Internal Threats
• Legacy Systems
• Multi Party Risk
• Intellectual Property Theft
• Software compliance/licenses
• Quantitative Risk Assessment
• ARO (Annualized Rate of Occurrence)
• Determine the likelihood of occurrence
• Is the estimation possibility of a specific threat taking place in a one-year time period
• SLE (Single Loss Expectancy)
• Determine the magnitude of the impact
• Monetary loss occurs every time
• Formula
• Asset Value * Exposure Factor = SLE
• ALE (Annualized Loss Expectancy) 
• Determine the magnitude of the impact
• Formula
• SLE * ARO = ALE
• Qualitative Risk Assessment
• Risk Register
• Risk Matrix
• Risk Heat Map
• Site Risk Assessment
• Location level risk assessments.
• Recovery plan should include - applications, personal, equipment and work environment
• Insurance planning
• Privacy Impact Assessment - PIA
• Privacy risk needs to be identified in each initiative
• Privacy Threshold Assessment - PTA
• Used to determine system contains PII information for impact analysis and protection of the data.
• Eg: Distributing a questionnaire to system and application owner. 

Risk Analysis

• Risk Register
• Risk Register is strategic component of an organization, providing single point of entry to record information about identified risks.
• Every project has some risk associated with it, identify and document the risk associated with each step.
• Apply all the possible solutions to the identified risks and monitor the risks.
• Helps the organization's risk tolerance and risk appetite are met.
• Risk Matrix / Risk Heat
• Visualize and view the result of the risk assessment, quantify the risk.
• Combines the likelihood (likely, possible, unlikely) of an event with potential impact (negligible, minor, major) and assists with making strategic decisions.

Credits: https://www.balbix.com/app/uploads/risk-heat-map-trimmed.png

• Inherent Risk
• Risk exists in the absence of any controls take place to avoid risk.
• Residual Risk
• Risk exists after the necessary controls taken place. 
• Risk appetite and Risk tolerance
• Risk the organization is willing to take.
• Risk Awareness
• Knowledge about the risk is crucial for everyone in the organization.

Risk Management Strategies

• Risk Mitigation/Deterrence
• Decrease the risk level, by investing in security systems
• Risk Avoidance
• Stop participating in a high risk activity
• Risk Transference
• Buy some cybersecurity insurance
• Risk Acceptance
• Business decision to accept the risk
• Control Risk
• Risk Awareness
• Documentation
• Term of Services / Term of Use / Terms and conditions
• Privacy notice / Privacy policy

Business Impact Analysis 

• Process of determining the potential impacts resulting from the interruption of time sensitive or critical business process
• Risk assessment focuses on the relative likelihood of potential threats to an organization, a BIA focuses on the relative impact of the loss of operational capability on critical business functions.
• Ensure key business is continued:
• BCP - Business Continuity Plan
• COOP - Continuity of Operations of Plan
• ICS - Identification of Critical Systems
• Identify critical systems and components
• Alternative solutions needs to be planned on failure of these
• SPoF - Single Point Failure
• SPoF creates potential risk in bringing the entire system down
• Should be avoided with fault tolerance system and redundance
• MTD - Mobile Threat Defence
• RTO - Recovery Time Objective
• Specifies allowable time to recover
• WRT - Work Recovery Time
• RPO - Recovery Point Objective
• Specifies allowable data loss
• MTTF - Mean Time To Failure
• Length of the time system is expected to work with failures or crashes
• MTTF is collected by running many units and calculate the average based on when components fail
• MTTF is used for non repairable systems
• MTBF - Mean Time Between Failures
• Predict the time between outages or failures
• MTBF is used for repairable systems
• MTTR - Mean Time To Repair
• Time required to fix the issue

Disasters

• Internal Vs External
• Person Made
• Environmental
• Site Risk Assessment
• Disaster Recovery Plans
• Functional Recover Plans
• Mission Essential Functions

References and Credits

https://purplesec.us/wp-content/uploads/2020/02/types-of-security-controls.png

CompTIA Security Plus several notes

Cache Algorithms

 

Introduction

Cache is one of the important strategy while working on performance improvement. Cache means storing the data temporally for quick retrieval avoiding each time reading from the original data source which could be slower. 

In general caching is used by Operating Systems, CPUs, GPUS, web browsers, applications, CDNs - Content Delivery Networks, DNS - Domain Name Systems, Databases, even at ISP - Internet Service Provider level. 

Before we read more in detail about cache let's try to understand about few concepts which we ideally get confused with cache.

Caching Vs Buffering Vs Streaming Vs Register Vs Cookies

• Caching:
• Caching is storing a partial or small data for quick retrieval.
• Buffering: 
• Buffer is used to store the data when there is difference in speed and processing of data between the sender and receiver.
• Streaming: 
• Streaming is real time data broadcasting either audio, video or may be text.
• Register: 
• Registers are very small memory storage in computer processors for fast retrieval by the processor. The data CPUs are processing is generally gets stored here. 
• Cookies: 
• Used in web browsers, usually maintains small data holding user preferences or login details, quite different from caching.

We will see various techniques used to save which particular data temporarily, as we cannot save the entire stuff. And caching techniques could be used in different systems, let's see few of them here. 

Types of Cache:

• Cache memory
• Cache servers
• CPU Cache
• Disk Cache
• Flash Cache
• Persistent Cache

Cache is said to be effective when the client request data and it hits the cache rather reading from the direct memory we call it cache hit else cache miss. Below are few algorithms used to based on the requirement of the application to increase the cache hit ratio.

Spatial

• Spatial is a caching technique used to perform advance read of the nearest data from the recently used data. 
• The idea behind reading closely associated data, there could be high chances of reading this data as well. 
• This will increase the performance as the manual read is avoided and the data is ready to be served. 
• Eg: Data saved in array or similar type of records from table could be read along with the single read instruction from the original source and save in the cache. As this avoids multiple iteration of read requests.

FIFO

• First In First Out - FIFO
• Data is added to the queue as its accessed.
• Once the cache is full, the first added item is removed from the cache.
• The ejection occurs from the order of data being added.
• From the terms of implementation and performance it fast but it is not smart.

LIFO

• Last in First Out - LIFO
• Opposite of FIFO, here once the cache is full, the last added item is removed first.
• The ejection occurs the reverse of the order of data being added.
• Here again it's fast but not smart.

LFU

• Least Frequently Used - LFU
• Here the data keeps tracks how frequent it has been used from the cache, and the count is maintained.
• Once the cache is full, the lowest count data gets removed first.

LRU

• Least Recently Used - LRU
• Initially any read data is added to the cache, but when the cache is full it frees up the least recently used data.
• Here, each time the data is read from the cache its moved to the top of the queue, increasing its significance.
• Fast and most commonly used algorithm.
• Temporal Locality uses similar concept while saving the recently used instructions in cache memory, as there are high chances of it to be used again.

LRU2

• Least Recently Used Twice - LRU2
• Two Caches are maintained here, the items are added to the main cache only when the item is accessed second time.
• Once the cache is full the cache is item least recently accessed item is removed.
• Complex and more space is required as two caches and the count of accesses are maintained.
• But the advantage is the main cache holds the most frequently and recently accessed data.

2Q

• Two Queue - 2Q
• Similar to LRU2, here as well two queues one small and one large are maintained.
• First accessed data is added to smaller LRU queue.
• Second time if the same data is accessed it is moved to the larger LRU and removed from the first queue.
• Fairly performs better than LRU2 and makes it adaptive.

MRU

• Most Recently Used - MRU
• Quite opposite behaviour we have seen in LRU, here most recently accessed data will be removed from the cache.
• Here the approach is more inclined to the older data, which is more likely to be used again.

STBE

• Simple Time based Expiration - STBE
• Once the data is added to the cache, its lifetime tickers gets started.
• Data is removed from the cache after an absolute time period it reaches.
• For example, 5.00pm or particular date or time.

ETBE

• Extended Time Based Expiration - ETBE
• Data from the cache is removed after the relative time period it reaches.
• Here the time to evict is configurable relatively eg 5hrs from now, or every 10mins.

SLTBE

• Sliding Time Based Expiration - SLTBBE
• The time line of the data extends after being accessed from the cache.
• Here the most recently accessed gets more lifetime to stay in the cache.

WS

• Working Set - WS
• Seems to be similar to LRU, but here the flag is created for each access in the cache.
• Periodically the cache is checked, the recently accessed data is considered but the working sets.
• And the non working set data are the candidate for the removals, when the cache is full.

RR

• Random Replacement - RR
• Here the randomly the data is picked from the cache and replaced with the newly accessed data.
• As here it does not keeps track of the history it is less overhead, but cannot guarantee the results

LLF

• Lowest Latency First - LLF
• Here this algorithm keeps track of download latency time.
• The least download latency time data is evicted first, as it could be quickly retrieved again.
• Here the advantage is when complex data needs to be retrieved again it could be easily referred from the cache.

LRD

• Least Reference Density - LRD
• Here based on the reference density, when the cache is full the object with the least reference density is removed. 
• A global reference counter is maintained, containing the sum of all references in the cache. 
• The reference density (RD) is calculated, using the below and from here the least RD is evicted:
• Object's reference counter (RC) meaning number of time it has been accessed
• Total number of all the references (GC)
• Each object has an arrival timestamp (AT), which is current GC value when it's been added to the cache. 
• The reference density - RD is computed as the ratio between the object's reference counter - RC and the number of references added since the object has been included into the cache GC - AT, 
• RC(i) / (GC - AT(i)).

Credits: http://wwwlgis.informatik.uni-kl.de/cms/fileadmin/courses/SS2011/RDBS/lectures/Chapter_04.BufferManagement.pdf

CLOCK

• Second Chance - Clock
• Clock is the efficient version of FIFO, because here the data in cache does not has to constantly pushed to the back of the list rather performs a general function as Second Chance in a circular queue.
• Second Chance is a bit assigned to each object data in the cache, and it is set to be 1 when it has been referenced, giving it second chance.
• When the eviction operation takes place, object follows FIFO queue, but remember the to be evicted position's the object is needs to be 0. So if in the queue the object was recently accessed it would have been, tough after the eviction the flag is reset to 0 again.
• Thus the data gets second chance, of not being replaced during its first consideration.

Credits: http://wwwlgis.informatik.uni-kl.de/cms/fileadmin/courses/SS2011/RDBS/lectures/Chapter_04.BufferManagement.pdf

GCLOCK

• Generalized CLOCK - GCLOCK
• Implements a mixture between LFU (Last Frequently Used) and LRU (Last Recently Used) replacement policies. 
• The reference counts are used to track references to cached objects. 
• Each access request increments the reference count of the object. 
• If the cache is full, the object to be removed is determined by decrementing the reference count for each object.
• After decrement, replace an object with the object which reference count = 0 is found. 
• The implementation tends to replace younger objects first.

ARC

• Adaptive Replacement Cache - ARC
• ARC dynamically balances between recency and frequency using the set of rules and performs self-tuning.
• It keeps track of recently and frequently access data queues, along with entires of recently and frequently recently removed data which is also called as ghost entires.
• These entires helps the algorithm to expand or shrink the LRU or LFU, based on the usage. 
• ARC leads substantial performance gains over commonly used modules.
• There is another variant of ARC - SARC - Sequential Prefetching in Adaptive Replacement Cache which is claimed to be better than ARC.

Credits: https://hal.archives-ouvertes.fr/hal-01700364/document

DeepBM

• A Deep Learning Based Dynamic Page Replacement Policy
• Could find one more interesting paper, 
• https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F18/projects/reports/project16_report.pdf
• Using the Deep Learning Algorithm learns from the past and dynamically adapts to the workload, which predicts the page to be evicted from the cache.

Cache Policies

Cache Policies determines how the cache operates in terms of writes to the storage. 

• Write Around Cache
• Writes to the storage first and skips the cache.
• Here the advantage is when there is large amount of write, the cache would not needs to be overloaded with write I/O.
• But common two problems here, data could be stale in the cache and the read could be slower as the new data would not be present in the cache.
• Write Through Cache
• Write is performed in both the system, cache and actual storage.
• The advantage here is reads would be faster as the most recently written data is available in the cache.
• But write would be slower as it needs to wait until the write is successful in both the systems.
• Write Back Cache
• Write operation takes place in the cache first and considers to be completed if the data is written to the cache.
• From here the data is copied back to the storage.
• Here the read and write both could be faster.
• But the greatest challenge would be inconsistent writes, as there is possibility to be not written in the persistent storage.

Pros and Cons of Caching

Pros

• So far we could have sense Performance will boost given using right cache algorithm
• Can act as middleware when the connectivity is lost, and in offline the operations can take place and once the system is on it could sync.

Cons

• Performance could be impacted if the cache ratio is low, as it takes additional overhead in writing to cache if not used it would definitely back fire.
• Invalid or outdated information could result in misinformation, need to take special care of data in cache is not stale.

Conclusion

New techniques and algorithms continue growing, but for now we have seen few techniques in cache which can help in better performance. 

Please share your ideas or experiences to improve performance in the comment box below. Also if you could find an algorithm matching the above algorithm but different terminology please share.

Let's keep researching and learning!

References and Credits

https://coderanch.com/wiki/660295/Caching-Strategies

https://developers.redhat.com/sites/default/files/blog/2016/02/will-cohen-blog_graphics-02-300x300.png

https://www.youtube.com/watch?v=ccemOqDrc2I&list=LL&index=1

https://www.techtarget.com/searchstorage/definition/cache

https://hal.archives-ouvertes.fr/hal-01700364/document

Digital Forensics - An Introduction

 

Introduction

• Forensic is the process of preserving the evidence and collecting data.
• Digital forensic is a part of collection and protection of information usually during security incident.
• Digital forensics relates to both e-discovery and data recovery
• E-discovery concerns the discovery of the electronically stored information
• Data recovery on the other hand involves in retrieving the lost or corrupted data from the storage device when it is typically inaccessible
• From a forensics standpoint preservation is a most important part, as it is used as evidence for use in legal proceedings.

Digital Forensics Phases

• Digital forensics involves in the process of documentation from initial notification through conclusion.
• Digital forensics process comprises of three standard phases:
• Acquisition of data
• Locate data and any devices of potential evidentiary value
• Identity data of interest
• Analysis of that data
• Create forensic duplicates of the data to review
• Store original data and devices in a manner that preserves integrity
• Perform forensic evaluation and document findings
• Reporting of that data
• Report findings
• This would be ongoing process assuring the organization is complying the laws and regulations.
• Forensics process involves highly in preservation and collection of the data.

Data Breach

• Data breach is an important reason of performing forensics.
• Company of all sizes are concerned about the data breaches.
• There are laws and regulation internationally and nationally, as every state and country follows several standards. Organization operating globally needs abide.

Strategic Counter Intelligence

• Strategic and Counter Intelligence requires after data breach to make sure attackers do not hold the footprints in the organization.
• Active logging and recordings enables us to examine from the time it has been put, and act as an intelligence tool.
• Precautions and measures needs to be taken care to implement detective measures. 

Track Person Hours

• Forensic Investigation can run in thousands of dollars, cost includes person hours and related expenses from the period of acquisition, analysis and reporting.
• An organization needs to assess the cost of investigations against the potential benefits.

Data Hold

• Whenever there is a potential security breach digital forensics comes into play in looking for data.
• Data could be stored into different respective systems and hold for different time spans.
• Each needs to be taken into consideration while scrutinising the breach.

Legal Hold

• The legal hold process ensures that anything that matters to legal proceeding is not destroyed for over a period of time.
• An organization should have a legal hold process to perform e-discovery to preserve and gather information for the later use.
• A legal hold is an important part of forensics process during information breach.
• Often the legal hold data is stored in separate repository.

Chain of Custody

• The chain of custody provides a clear record of the path taken from acquisition to disposal.
• It provides authenticity and non-repudiation establishing the origin of data and proof of custody.
• It is important to create a log of all actions taken.
• Evidence it is useful and must follow below five properties:
• Admissible
• Must follow legal regulations.
• Authentic
• Data must not be tampered. Hash and Checksum are few mechanisms ensuring the data has not been changed.
• Complete
• All the information must be present.
• Reliable
• Data must be gathered based on the order of volatility and avoid destruction of the evidence.
• Multiple copies can be taken also sensitive data needs to be encrypted.
• Believable
• Must be clear to understand
• Document all the transfer of evidence, reason for transfer with the signature from both the parties.
• Proper chain of custody helps to ensure the evidences are handled correctly and strictly secure.
• Blockchain technology could be used track the detailed information.
• Ideally chain of custody means from the time data is gathered no change has been done to it with documentation.

Order of Volatility

• Each data evidence holds different life spans, eg: if the data is present in RAM it is available until system is powered off.
• Evidence collection should follow the order of volatility, collecting the most volatile evidence to least.
• Common evidence collection order follows as below:
• Register
• Caches
• Routing and process table
• System date and time
• Current network connections
• Current open ports and application listening to the ports
• Applications currently running
• Kernel statistics
• Main memory, RAM data, SWAP
• Temporary file system eg: tmp folder
• Secondary memory
• Removable media, Disk
• Operating System
• Write once storage
• Order of volatility demands that evidence be collected first from the most volatile systems (such as registers and caches) and later from the least volatile systems (such as archival media).

Data Acquisition

• Data acquisition is an important concept that involves gathering data or copying data to image or other media, in the forensics process.
• Data acquisition is vital for completeness and accuracy.
• Below are few methods of gathering and capturing data:
• Capture System Images
• Duplicate the copy of  entire system media including volatile and non volatile.
• Capture Screenshots
• Capture screenshots during the investigations and include in the forensic documentation.
• Capture Network Traffic and Logs
• Network traffic can be used to reconstruct network based attacks.
• Capture Event Logs
• Capture event logs which is detailed record of operating system, security and applications
• Capture Video and Photographs
• Recording in crucial areas and entrances can help the forensics confirm based on the evidence gathered in the scene.
• Record Time Offset
• Record the time offset is crucial information, to keep on all data and device collected such as system time off, NTP, hardware configurations and so on. 
• Take Hashes
• Generate checksums or hashes of all the data and applications before and after in-depth analysis performed to validate.
• Collect Witness Interview
• Witness gets to be interviewed by interviewer as part of the investigation.
• Sometimes it can reveal an insider
• But we need to take all this information with pinch of salt, as it could not be 100% accurate.
• Collect additional information
• Some of these data would not be saved in the hard drive, such examples are like browsing history, clipboard information, command history, encryption key, library versions/checksum, number of users logged in and so on.

Forensics in the cloud

• Digital Forensics may not limit to on premises but as well cloud, though it may not be immediate possession as we don't have physical control.
• Hybrid and multi cloud adds more complexity to the forensics process.
• Legally as well there would be control where the data needs to be located in the world.
• Integrity of the data how it has been saved and shared over the network, could be audited.
• In scenarios customers have the right to know where the data resides and any breach needs to be informed.

Reports

• Finally report the findings during the breach in a readable format along with metrics and evidences.
• Reports explain what has exactly occurred during a security incident.
• Usually this holds an overall summary and detailed documentation how data was collected,  processed and analysed. 
• Inferences and conclusions are arrived from the analysis.

Conclusion

• Major concepts behind computer forensics

• Identify the evidence
• Preserve the evidence
• Process the evidence
• Inference from the evidence
• Report the evidence

Credit & References

https://media.itpro.co.uk/image/upload/s--X-WVjvBW--/f_auto,t_content-image-full-desktop@1/v1613578972/Network_forensics_Shutterstock.jpg

Comptia Security Plus course materials