Evaluation of Field Phishing Study Setup Method (original) (raw)
Related papers
The design of phishing studies: Challenges for researchers
Computers & Security, 2015
In this paper, a role play scenario experiment of people's ability to differentiate between phishing and genuine emails demonstrated limitations in the generalisability of phishing studies. This involves issues around the priming of participants and the diversity of emails used. Only half of our 117 participants were explicitly informed that the study was assessing the ability to identify phishing emails. Results indicate that the informed participants were significantly better at discriminating between phishing and genuine emails than the uninformed participants. This has implications for the interpretation of phishing studies. Specifically, studies where participants are directly asked to identify phishing emails may not represent the performance of real world users, because people are rarely reminded about the risks of phishing emails in real life. Our study also used emails from a larger number and greater diversity of industries than previous phishing studies. Results indicate that participants' performance differs greatly in terms of category (e.g., type of sender) of emails. This demonstrates that caution should be used when interpreting the results of phishing studies that rely on only a small number of emails and/or emails of limited diversity. Hence, when designing and interpreting phishing studies, researchers should carefully consider the instructions provided to participants and the types of emails used.
Designing ethical phishing experiments
IEEE Technology and Society Magazine, 2000
We describe ethical and procedural aspects of setting up and conducting phishing experiments, drawing on experience gained from being involved in the design and execution of a sequence of phishing experiments (second author), and from being involved in the review of such experiments at the Institutional Review Board (IRB) level (first author). We describe the roles of consent, deception, debriefing, risks and privacy, and how related issues place IRBs in a new situation. We also discuss user reactions to phishing experiments, and possible ways to limit the perceived harm to the subjects.
CHI'16 Workshop on Ethical Encounters in HCI, 2016
With employees being still the weakest link in organizational information security, phishing studies are becoming increasingly important and are more frequently employed as a research method. Ensuring the validity of results often calls for the use of deception in phishing research. Yet, deception as a research practice has severe ethical implications: researchers and practitioners have to account for possible emotional harm and distress of participants. Unfortunately, empirical data to estimate this potential harm and distress is still rare. In an ongoing study, we are collecting quantitative and qualitative data on emotional and social effects on employees participating in an organizational phishing study. From this data, we will derive guidelines to estimate possible negative effects and suggest interventions for remediation.
To deceive or not to deceive! Legal implications of phishing covert research
To deceive or not to deceive! Legal implications of phishing covert research, 2013
Whilst studying mobile users’ susceptibility to phishing attacks, we found ourselves subject to regulations concerning the use of deception in research. We argue that such regulations are misapplied in a way that hinders the progress of security research. Our argument analyses the existing framework and the ethical principles of conducting phishing research in light of these regulations. Building on this analysis and reflecting on real world experience; we present our view of good practice and suggest guidance on how to prepare legally compliant proposals to concerned ethics committees.
Informing, simulating experience, or both: A field experiment on phishing risks
PLOS ONE, 2019
Cybersecurity cannot be ensured with mere technical solutions. Hackers often use fraudulent emails to simply ask people for their password to breach into organizations. This technique, called phishing, is a major threat for many organizations. A typical prevention measure is to inform employees but is there a better way to reduce phishing risks? Experience and feedback have often been claimed to be effective in helping people make better decisions. In a large field experiment involving more than 10,000 employees of a Dutch ministry, we tested the effect of information provision, simulated experience, and their combination to reduce the risks of falling into a phishing attack. Both approaches substantially reduced the proportion of employees giving away their password. Combining both interventions did not have a larger impact.
Phishing for the Truth: A Scenario-Based Experiment of Users’ Behavioural Response to Emails
IFIP Advances in Information and Communication Technology, 2013
Using a role play scenario experiment, 117 participants were asked to manage 50 emails. To test whether the knowledge that participants are undertaking a phishing study impacts on their decisions, only half of the participants were informed that the study was assessing the ability to identify phishing emails. Results indicated that the participants who were informed that they were undertaking a phishing study were significantly better at correctly managing phishing emails and took longer to make decisions. This was not caused by a bias towards judging an email as a phishing attack, but instead, an increase in the ability to discriminate between phishing and real emails. Interestingly, participants who had formal training in information systems performed more poorly overall. Our results have implications for the interpretation of previous phishing studies, the design of future studies and for training and education campaigns, as it suggests that when people are primed about phishing risks, they adopt a more diligent screening approach to emails.
Something Smells Phishy: Exploring Definitions, Consequences, and Reactions to Phishing
One hundred fifty-five participants completed a survey on Amazon's Mechanical Turk that assessed characteristics of phishing attacks and requested participants to describe their previous experiences and the related consequences. Results indicated almost all participants had been targets of a phishing with 22% reporting these attempts were successful. Participants reported actively engaging in efforts to protect themselves online by noticing the "padlock icon" and seeking additional information to verify the legitimacy of e-retailers. Moreover, participants indicated that phishers most frequently pose as members of organizations and that phishing typically occurs via email yet they are aware that other media might also make them susceptible to phishing scams. The reported consequences of phishing attacks go beyond financial loss, with many participants describing social ramifications such as embarrassment and of reduced trust. Implications for research in risk communication and design roles by human factors/ergonomics (HF/E) professionals are discussed.
Sixteen Years of Phishing User Studies: What Have We Learned?
ArXiv, 2021
Several previous studies have investigated user susceptibility to phishing attacks. A thorough meta-analysis or systematic review is required to gain a better understanding of these findings and to assess the strength of evidence for phishing susceptibility of a subpopulation, e.g., older users. We aim to determine whether an effect exists; another aim is to determine whether the effect is positive or negative and to obtain a single summary estimate of the effect. OBJECTIVES: We systematically review the results of previous user studies on phishing susceptibility and conduct a meta-analysis. METHOD: We searched four online databases for English studies on phishing. We included all user studies in phishing detection and prevention, whether they proposed new training techniques or analyzed users’ vulnerability. FINDINGS: A careful analysis reveals some discrepancies between the findings. More than half of the studies that analyzed the effect of age reported no statistically significan...
Social engineering cyber-attacks such as phishing emails pose a serious threat to the safety of many organizations. Given that the effectiveness of these attacks heavily relies on poor human decision making, an improved understanding of the individual characteristics that increase cybersecurity vulnerability could inform more targeted training. The current study aimed to identify whether several factors, including phishing email detection ability, confidence in one’s phishing identification decisions, attitudes toward one’s level of responsibility and efficacy, and employee satisfaction and loyalty to the organization, can predict behavior in a naturalistic phishing simulation in an employment setting. We followed up employees of a large organization who had been recently targeted by a phishing simulation and asked them to complete a survey that included a phishing detection task. The employee’s behavior in the phishing simulation was ranked according to its safety: reporting the su...
F for fake: Four studies on how we fall for phish
2011
Abstract This paper reports findings from a multi-method set of four studies that investigate why we continue to fall for phish. Current security advice suggests poor spelling and grammar in emails can be signs of phish. But a content analysis of a phishing archive indicates that many such emails contain no obvious spelling or grammar mistakes and often use convincing logos and letterheads.