Minimum Requirements for a PCI-DSS Risk Assessment

Note: This blog post orginally appeared on the SecureState blog.

In information security, various regulations require a periodic risk assessment. The Payment Card Industry (PCI) Data Security Standard (DSS) is no exception. For PCI-DSS, the risk assessment process is designed to identify, analyze, and document risks to credit card data. The assessment is the integral component of the risk management strategy, and therefore should be used to manage threats and vulnerabilities, and document control effectiveness.

PCI-DSS Requirement 12.1.2 requires a “formal risk assessment” to be performed. However, what this means remained a topic of debate both among practitioners and those procuring their services. Auditors and their clients often grapple with compliance issues as they interpret the “gray” areas associated with compliance frameworks. In 2012 the PCI Security Council formed a Special Interest Group (SIG) to remove some ambiguity when executing risk assessments to meet PCI-DSS requirements. The output from the SIG was the Information Supplement: PCI DSS Risk Assessment Guidelines V1 document.

Industry adoption of the guidance provides consistency and helps manage merchant expectations, but more importantly should entice entities who store, process, or transmit cardholder data (CHD) to evaluate their processes and reduce their risk posture consistently. This manifests itself in benefits for the brands (e.g., VISA, MC, AmEx), the merchants, and ultimately their customers.

Before we dive into the guidance provided in the Information Supplement, it is important to note that the SIG made it clear the risk assessment is used to determine what additional controls are needed to protect CHD and cannot be used to avoid or bypass any PCI DSS requirements.

The Information Supplement lists a variety of risk frameworks (OCTAVE, ISO 27005, NIST SP 800-30) but does not require a specific framework to be used. Any risk framework can be used that meets the guidelines outlined in the document. Below is a summary of what is minimally required for a risk assessment to be PCI-DSS compliant. There are other items that should be included in a risk assessment, but today we are just looking at the minimum requirements outlined in the SIG document.

• Methodology must be defined and follow a documented process
• Must be performed annually
• Must cover any people, processes or technology which could impact the security of the Cardholder Data Environment (CDE). This is not simply limited to the CDE and must include any people, processes or technologies which are involved in the storage, processing or transmission of CHD. This includes people, processes or technology not directly involved with the processing of CHD that could impact the security of the CDE.
• Asset inventory (people, processes or technology) that covers all payment channels and includes any asset that directly or indirectly impacts the processing, storage, transmission or protection of CHD or the security of the CDE.
• Must identify threats, vulnerabilities and controls that could impact the security of the card holder data
• Can be quantitative, qualitative or a mix of both
• Must include risks posed by outsourcing to third parties
• Identify and score threats (capabilities, impacts, likelihood, etc.)
• Identify organizational and technical vulnerabilities
• Identify controls and the effectiveness of the controls
• Output of the risk assessment provides a prioritized risk mitigation plan

If your organization needs to be PCI-DSS compliant, it is critical that the risk assessment methodology used meets these requirements.

Contact SecureState if you need help performing a risk assessment or are if you are unsure if your current risk assessment process meets PCI-DSS requirement.

You are also invited to attend an exclusive Web Seminar on Tuesday, March 26th at Noon (ET) featuring a discussion of the new PCI-DSS requirements.  CLICK HERE to visit the registration page.

Presenting on the New PCI-DSS Risk Assessment Guidelines at February NEO ISACA Meeting

I'll be presenting "PCI-DSS Risk Assessments – Understanding and Meeting the New PCI-DSS Risk Assessments Guidelines" at the February North East Ohio ISACA meeting. The meeting is February 20^th and takes place in Cleveland, Ohio. This talk focuses on the new PCI Risk Assessment Guidelines published late last year and how to make sure the risk assessment you perform for PCI DSS compliance meet these new guidelines. Below is the abstract for the talk:

Various regulations require a periodic risk assessment, as does PCI-DSS. PCI-DSS requirement 12.1.2 requires a “formal risk assessment” to be performed.  However, what this means remained a topic of debate both among practitioners and those procuring their services.  Auditors and their clients often grapple with compliance issues as they interpret the “gray” areas associated with compliance frameworks. In 2012 the PCI Security Council formed a Special Interest Group (SIG) to remove some ambiguity when executing risk assessments to meeting PCI-DSS requirements.  This presentation will discuss the output of the SIG and the impact on a company’s PCI compliance.  First the guidance provided in the document will be reviewed. Next the presentation will cover how these guidelines could impact you PCI.  The main section of the talk will address recommended way to meet the requirements outlined in the SIG.

Details on the meeting time, location and how to register can be found here.

The Importance of Sample Size in Social Engineering Tests

Note: This blog post orginally appeared on the SecureState blog.

We Have a Problem

Information security has a problem. We make far too many decisions without having reliable data to assist in our decision making process. Because of this, far too many information security professionals use what I call Gut 1.0 to make decisions based on gut feel. To help address this, security professionals need to start gathering reliable and accurate data they can use in the decision making process. To make this data useful, a number of factors need to be looked at. Today we’re going to talk about selecting sample sets that are statically significant so data gathered from a small subset of the population can be applied to the population as a whole. I want to dive into this specifically because I see a lot of companies perform social engineering tests on a handful of their user population and falsely try to apply these results to the entire user population at the company.

Ways to Determine Sample Size

There are a number of different methods and online calculators which can be used to select a sample size that is statistically significant. Today we’re going to be using an online tool provided by Creative Research Systems. In this blog post we won’t go into the math behind this method, but if you are interested you can learn more here. Instead we’ll focus on a practical approach to use this method when performing a social engineering test geared towards measuring the effectiveness of a security awareness training program.

As a general rule, the larger the sample size the more accurately it will represent the lager population. It’s also important to note the relationship between sample size and overall user population is not a simple linear relationship. Before we get into some examples, there are two terms, Confidence Interval (CI) and Confidence Level (CL), which are important to understand when calculating sample sizes.

Key Definitions

The Confidence Interval is what we commonly refer to as the margin of error. The Confidence Interval says the data is accurate plus or minus the CI. For example, if you have a Confidence Interval of 5 and your social engineering tests shows 40% of the sample population fell prey to the attack, you can say the attack would be 35-45% successful against the entire population.

The Confidence Level is basically how sure you are in the reliability of the data. For this article we’ll be running all calculations with a CL of 95%. We chose 95% because that’s a confidence level used by most researchers.

How the Numbers Break Down

In the chart below we show the sample size needed to achieve various CI for different user populations. For example if you have a 500 user company and want to have data that has a CI of 2 you would need to test 414 users, which is pretty close to the entire population. However if you have a 10,000 person company and want test results with the same CI level you need a sample size of 1,936. However if you want to change to a larger CI of 5 the numbers change pretty drastically and you need only 217 users from a 500 person company or 370 users from a 10,000 person company. You can explore the chart below to see the sample size needed for different user populations and CI.

What to Aim For?

What is the correct CI value to aim for? That depends on how you plan to use the data. The important item is to understand what the CI is when interpreting data you gathered or creating a sample size for a test where the results will be applied to a larger population.

Now that we have a general idea on how to the numbers break down lets apply this to a traditional social engineering test where only a handful of users are tested.

Example: Interpreting Test Results

Bob’s Aerospace Widgets has 300 users and wants to run a social engineering test to determine how well users can apply knowledge learned during the security awareness training program. They want to send phishing emails to 10 employees and see if the users recognize it as a malicious email and do not click on the link. After the test of 10 users the company finds that 80% of the users clicked on the link. Now the company wants to use this data to figure out how well this data applies to the overall user base at the company. Using the information above we run the calculations and determine if you have an overall population of 300 and taking a sampling of 10 users you will have a CI of 30.52 when applying this to the general population. So in this case, if the same phishing email was presented to the general population, it would be clicked on by 49.48 to 100% of the population. Overall a range that large isn’t helpful to judge the effectiveness of the program. Is the fact that 8 out of the 10 users clicked on the link a problem that needs to be addressed? Yes. But can this same percentage be applied to the general population? No. In this case the company should determine what a useful CI is and rerun the test with a sample of the proper size to give meaningful data.

SecureState uses this approach when performing social engineering assessments designed to test the effectiveness of a company’s security awareness training program.

Presenting on Smart Grid Security at the Ohio Information Security Conference (O-ISC)

Note: This blog post orginally appeared on the SecureState blog.

I'll be presenting "Smart Grid IQ Test - Assessing the Security of Smart Grid Systems" at the Technology First Ohio Information Security Conference (O-ISC). The conference is March 13^th and is located in Dayton, Ohio. This is a great talk outlining the various assessments that need to occur against smart meters and the infrastructure that supports them. Below is the abstract for the talk:

Today many gas, electric and water utilities are implementing smart grid systems to better monitor and manage their distribution networks. Smart grids present unique security challenges because the critical infrastructure they monitor and manage is deployed into insecure and hostile environments where attackers will have direct access to key components. Additionally, because of the wide geographic area these systems cover, a timely response to intrusions is not assured.

It is essential to make sure the security of these systems is fully tested. The talk will start out by reviewing the common components in a smart grid system. Next it will cover the rules and regulations related to smart grid assessments. The rest of the talk will focus on the different assessments which should be performed on various parts of the infrastructure. The talk will include demos of various smart grid testing tools and techniques.

Registration for the O-ISC is currently open. I’ve presented at three previous O-ISC conferences and it’s a great little conference that pulls in good speakers and attendees. If you are in the Dayton, Cincinnati, Columbus or surrounding areas, I recommend you check it out.

Who is Responsible for Application Security? Development or Security?

Note: This blog post orginally appeared on the SecureState blog.

A recent uptick in the importance of application security driving the debate

During a recent visit to a client site, I took part in a discussion where the Development Department and the Security Department were arguing over which group was responsible for the security of web applications.  Security felt it was the responsibility of the developers, and the developers felt it was the responsibility of security.  I commonly see this debate taking place inside organizations, so I wanted to spend this blog post discussing where that responsibility lies.

Some definitions

When this discussion comes up, the first thing that needs to be defined is what exactly we are talking about when we say “the security of web applications”.  For this blog post, we are going to limit the scope of Web Application Security to “ensuring the web application is securely coded to avoid security vulnerabilities and defects”.  However, it is important to remember that developing and maintaining a secure web application does not stop at making sure the application is properly coded.  A mature Web Application Security program incorporates security into their software development life cycle.  This includes activities such as:

• Application threat modeling being performed when generating requirements for applications
• Security requirements properly gathered
• Reviewing the architecture of the application
• Regularly assessing application until they are removed from production

It is equally important to make sure the infrastructure supporting the web applications is also secure and properly monitored.  However, covering all these items is beyond what can be covered in a short blog post.

It’s both (of course), but what does that mean?

The simple answer to this often debated question is both development and security are responsible for ensuring the security of web applications and they need to work together; neither group can achieve their goals alone.  However, each group has their own separate set of responsibilities when it comes to developing secure web applications.

1. First, developers need to have the proper training to know how to securely code web applications.  This is a key skill left out of most college course on programming.  Second, the developers and security staff need to develop Secure Coding Guidelines for each language used at the company.  These guidelines will provide the application developers with information on how they should securely code common functions.  It is important for security and the developers to work together to generate this document to ensure the guide is useful to developers, allows developers to code in an optimized fashion, and properly addresses security concerns.
2. Once developers have been trained and Secure Coding Practices have been established, the developers need to perform internal checks on the code they are developing to ensure the code is vulnerability free.  These internal checks help ensure the Secure Coding Practices are being followed and are a great way to improve developer’s knowledge of how to securely code web applications.  Throughout this process, the Security or Quality Assurance (QA) group should perform spot checks to make sure the internal developer code checks are being performed.
3. Once the application is coded, it moves into the QA process where the QA staff is responsible for identifying any defects in the application. Classically, QA testing has focused on testing the usability of an application and stress testing an application under load.  However, security vulnerabilities caused by coding are also a defect in the application, because a security vulnerability would allow the application to function in a way it is not intended.  Because of this shift, QA is responsible for ensuring the application is securely coded before it moves into production.  In many organizations large and small, the QA staff does not have the proper training in Web Application Security or the interest in growing that capability in house.  For those organizations, they should team up with an outside company that specializes in Web Application Security to have them perform the QA testing.
4. Once the application passes QA testing, it is time to move the application into staging, which is the final step before it moves into production.  At this point, Security should test the application.  During this test, it is critical the application be set up exactly as it would be in production.  This test must be performed by a group outside of QA and development to ensure the application is reviewed by a fresh set of eyes.  If the Security Department has the capabilities, they can perform this test.  However we find that many Security Departments do not have application security specialist on staff, so this is frequently outsourced to a third party. Security is ultimately responsible for making sure this test occurs and is performed by a qualified party.  If any issues are discovered, development is responsible for making sure they are correctly fixed in a timely manner.
5. Next, the application moves into production.  To ensure the application remains secure, a number of steps must be taken.  All applications that are in production should have quarterly Black Box Scans and annual Grey Box Assessments performed on them.  These Assessments will test the applications for new vulnerabilities that may impact the application.  These steps are mainly the responsibility of the Security Department.  If any functional changes are made to the application, these changes should go through the complete security SDLC process.

As you can see, ensuring web applications are secure is not just the responsibility of one group.  Instead, it takes a team effort to accomplish this important goal.  In closing, I’d like to also state that many security organizations do not have expertise in house to provide training, develop secure coding standards, or properly assess web applications, and instead rely on third parties such as SecureState to provide these services.

So what is your take on this debate?  Who do you think is responsible for security in web applications?  Join the discussion in the comments section of this post!