Phishing testing is a key part of cybersecurity and specifically security awareness. Protecting your people is more important than ever, as phishing is the leading attack vector for most threat actors. You can create great training material to create awareness, but you need a solution to regularly identify risk within your company. That’s where phishing testing comes in.
So, you’re looking to run a phishing test. Well, not only are you doing a good thing, you’ve come to the right place! Here is a full, comprehensive guide to running an effective phishing test.
What to Know Before you Start
Before you dive right into a phishing test, there are a few things you need to have prepared. You’ll need some data such as your target users, but most importantly, you need to set the right expectations.
Determine your goals and KPIs
Like most experiments, if you don’t have metrics to track or proper goals in place, it will be hard to properly judge effectiveness and make the necessary improvements. Additionally, it will be harder to report to other areas of your company without results that show progress.
Here are a few KPIs we recommended you track:
Per Test:
- Number of email opens
- Number of email clicks
- Users that went even further (entered credentials or downloaded a file)
- Number of users that reported the email
- Number of users that completed training (either a course of point-of-infraction training)
Metrics on an ongoing basis
- Testing results month over month
- 3 and 12-month trend lines
- Open and Click rates over time
- Ongoing course completions for annual training and monthly courses
We recommend using some or all of these metrics to create a solid overall view of your progress.
Gather your users
To send out phishing simulations, you’ll need to know who you’re sending emails to. If you’re using Hook’s phishing simulator, you can add users via manual upload, a CSV, or with integrations like Azure AD and Microsoft Graph.
As far as the information you’ll need for the test, at a minimum you need a name and email. This will provide more than enough for a solid phishing test.
However, there is some additional information you can provide to create some pretty devious custom emails. Information such as;
- Department
- Manager Name
- Title
- Address
- Phone Number
Anything extra across the board that you want to add for context about your own company will just improve the realistic nature of internal-looking emails, AKA Business Email Compromise templates.
Once you’ve gathered your users and uploaded them into the tool of your choice, you’re ready to move on.
Strike A Good Balance Between Effectiveness and Disruption
Phishing testing is a powerful way to identify risk, and coupled with good training materials, can dramatically reduce your cyber risk.
But...these are also your coworkers (or customers). They have work to do and morale to maintain. Your phishing testing should be realistic and effective, but be careful not to toe the line too far into mass panic and angry, frustrated employees.
Here are some phishing tests to probably avoid:
- Teasing raises or bonuses
- Reporting medical data
- Anything that might do more harm than good
[Read: Tribune Publishing apologizes for fake bonus offer in phishing-simulation email]
Phishing Testing, done improperly, can undermine the trust of your employees and create a worse security culture than you had before. Try to follow these guidelines when sending out a phishing test:
- Train everyone. Culture comes from the top down. If your executives are getting tested, this will only help your employees be more receptive
- Train quickly. A phishing test (and subsequent training) shouldn't take a ton of time out of an employee’s day. Keep the training light, communicate that they’re not in major trouble, and send them on their way. This keeps productivity up as you continue to secure your workforce.
- Notify employees in advance. While this may seem counterintuitive, this will help you maintain trust with your workforce. You don’t have to tell them what the email will be, but we encourage you to let them know that something like this is happening. It may even help employees be more secure if they know something could be coming
Truth is, you can still be pretty devious and have some fun with phishing tests, but the more you bring employees into the actual process, the more they’ll take to the training.
Phishing Test Prep
Alright! We have our targets and we have our goals and expectations set. Now let’s get ready to test.
For the purposes of this demonstration, we are going to use Hook Security’s phishing simulator.
Choosing Campaigns
At a bare minimum, you’ll need two templates for your campaign: an email template and a training page template.
If you’re collecting data like credentials, you’ll need an additional landing page to collect that information. Otherwise, if we’re treating the primary action of failure as click, we can send users straight to training.
Email Template
Here’s where you can have a little fun. There are two keys to a good phishing email test: A specific focus and a specific type of email.
The focus of a phishing test will vary and often has some combination of a few, but it’s important to know what it is so you can maximize effectiveness. The focus can be either what the email content contains, or the main red flags that you’ll be testing. This will allow you to provide the best type of feedback and training, as you can highlight specifically what the user should have spotted.
Phishing Test Focus Examples:
- Too Good To Be True - Contains an offer, deal, or promise, that feels “too good to be true”, like a gift card, free items from a brand, etc.
- Bad Links/Bad Sender - An email either from a brand or individual that contains links and/or a sender address that are malicious-looking in their nature.
- Spoofed URL - An email from a sender address that looks eerily similar to either your company or your brand (think dr0pbox or Walrnart).
- Urgency - An email with high levels of urgency, like a password breach, past due notice, or a request from a boss.
Phishing Testing Email Categories
Now that you’ve selected a focus for the email, the email itself may take the form of one of these categories:
- Brand Knock-Offs
- Current Events
- Healthcare
- Information Technology (IT)
- Banking/Finance
- Online Services
- Social Media
- Internal/Business Email Compromise (BEC)
- Shipping/Notifications
- International
Training Page
Once you’ve got your email template selected, you need to page to direct the traffic of those who click. The goal is to provide instant feedback to the user that they clicked on something they shouldn’t have, and how they avoid it in the future.
We recommend keeping this page and its content welcoming, simple, and quick to understand. Many users, once they realize what happened, will freak out and close the tab. So for a small fraction of users, you have a split second to get your point across.
For those who stick around long enough to take in the information, we recommend using a training video that is short, fun at times and delivers a succinct, memorable lesson.
Here’s one of ours, for example:
For those who click but don’t stay around to view the training, you can send them the video as a follow-up email to make sure they get trained.
Finalizing Your Preparation
Ok! You have your targets selected, and you’ve put together a solid campaign. Maybe you’ve gone with a fake Starbucks free drink offer. Or maybe you went with the classic “Boss asking for gift cards” email.
Either way, you are ready to phish.
Running Your Phishing Test
The benefit of doing solid phishing test prep and using a phishing simulation tool is that during the test, well, you frankly don’t have to do much.
Other than pushing the big red button to launch out your test, here’s what we recommend doing during the testing period.
Monitor Deliverability
As the phishing test goes out, make sure the emails are landing in employees’ inboxes and not bouncing. In our Phishing Simulator, you can track deliveries, opens, clicks, and reporting in real-time to keep track of your testing
Notify Help Desk
Depending on your company structure, you may be the help desk. But if not, you should notify them before testing goes out so they can handle support tickets properly.
The last thing you want is for your team to start investigating or pull the alarm on a phishing email that was a simulated one. Just let them know in advance what to expect.
Have a Mid-Test Plan
Similar to the previous point, have an operational plan for how you will handle tickets, inquiries, etc. about the phishing test.
Employees talk. And frankly, it’s a good sign if employees raise the alarm and notify each other of a funky looking email. That being said, you do want to protect the integrity of the phishing test as much as you can.
Have a response plan for inquiries during the test.
“Great job! Now shhh, don’t tell anyone else. Let’s see if they catch it”
After the Phishing Test
Once your campaign is complete, it’s time to tally up the scores, report the data to the necessary stakeholders, and let employees know how they did.
Reporting
This is where we go back to the beginning: our KPIs and metrics.
If this is your first test, then you can report on the “Per Test” metrics: opens, clicks, etc.
If you’re a few months in, that’s where you can begin showing trend lines and progress from test to test.
If you are enrolling your users into training courses in addition to phishing testing, this is where you can report on that as well.
One important note: Some phishing tests are more potent than others (looking at you, Christmas-time UPS missed delivery email). While the goal is to show a downward trend in failures over time, you may see spikes based on the potency of certain emails.
One way to approach this to send a baseline email at the beginning of your phishing test journey, send it again after 12 months of testing and compare the results.
The primary takeaways from reporting should be to understand areas for improvement, show trends over time, and in some cases, demonstrate compliance.
Employee Feedback
After testing, it’s important to give employees feedback on the test because of two possible scenarios:
- The employee clicked but maybe didn’t watch the training
- The employee didn’t click and thus never even knew it happened
Either way, it’s a good idea to let them know what happened. Here’s how:
Reward Successes
Like we mentioned before, employees that pass a phishing test often don’t even know it happened. It’s possible they identified it as phishing and simply deleted it, or maybe they never even opened it.
Let them know! Show them what you sent, congratulate them for not clicking, and even reiterate what it was in the email that they likely picked up on. Again, they have missed the email completely, so showing it to them can go a long way to keep them on the right side of phishing emails.
Coach Failures
At Hook Security, we don’t believe you should fire or even heavily reprimand an employee for failing a phishing test. The most secure employees are ones who have been through training and maybe even failed a phishing test or two. If you fire everyone who fails, hypothetically you’ll never have secure employees. Don’t punish mistakes, coach them, and build a stronger, more secure workforce.
That being said, how do you coach someone who failed a phishing test?
First, let them know that it in fact was a phishing test, and nothing crucial happened. Then let them know what they could have spotted in the email that would give it away. Provide some additional tips and training material to help them in the future.
Finally, let them know that there will be more phishing tests in the future. Tell them what to look out for and how to report an email if it seems fishy.
If an employee is failing repeatedly, you can enroll them in additional training content to better educate them.
Coaching phishing test failures is the most important step in this whole process.
If you can turn failures into passes, you’re on your way to some seriously positive results.
Rinse and Repeat
Congratulations! After reporting and feedback, you will have successfully completed your first phishing test.
What’s next?
Time to do it all over again. We recommend testing monthly, and in some cases even more often for repeat offenders.
Well, there you have it. A full guide to an effective phishing test.
We hope this helps you get started on your phishing testing journey. Security awareness is not a one time project. It’s an ongoing practice, and effecting testing and training is the first step to get there.
Good luck and stay aware out there!
Additional Phishing Test Ideas
Looking for inspiration? Here are a few of our favorite phishing test campaigns.
- A Google “sign-in detected” email
- An internal “Account scheduled for deletion” email
- Starbucks Free Drink
- Netflix - Password Reset
- Apple “support ticket” knockoff email
