Red Teaming with Hackathons: Adding an Extra Layer of Security Testing to Applications
As a cyber security consultant, I have been responsible for overseeing the software development practices and coordinating penetration tests to ensure that after major updates, applications remained secure and aligned with the OWASP Top 10. While these tests effectively assessed compliance with standard security benchmarks, they often missed the types of vulnerabilities that could be uncovered if a user interacted with the application unexpectedly. The issue is twofold, developers have full knowledge of the applications but lack the objectivity to complete thorough testing (it would be equivalent to marking their work), whilst external testers lack the deep product knowledge to test thoroughly beyond the OWASP framework, especially within the constraints of a typical 5-day penetration test. To bridge this gap, I proposed an internal hackathon focused on identifying weaknesses within the codebase. To encourage participation and maximise engagement from the workforce, I suggested monetary rewards for teams that successfully discovered vulnerabilities.
Here’s how it went:
Problem Statement
The hackathon addressed a knowledge issue between the penetration testers and developers, each party lacked the objectivity or knowledge depth to test applications in depth. The hackathon was the right approach since it was collaborative and leveraged the in-depth knowledge possessed by developers. Many organisations struggle with maintaining their codebase, typically contributions have been made over time by multiple authors (developers) with varying approaches to security. This lack of consistency makes keeping the codebase secure problematic and unwieldy. Traditional penetration tests, while valuable, tend to yield a narrow, generic set of findings (usually in alignment with OWASP Top 10). I proposed the hackathon as a low-cost method of improving the codebase by identifying vulnerabilities and cultivating a security mindset, with the findings from the hackathon being logged and resolved internally.
The Challenge
The hackathon was open to the entire workforce, allowing teams of four to participate in a one-day bug-hunting hackathon within a test environment that closely mirrored the production codebase for a single application. Teams were asked to register ahead of time.
Lessons Learned
When setting out to plan a hackathon, I thought the engineering team would find bugs that were outside of what is usually found in a penetration testing engagement, however, the results were better than expected. Here are the key lessons learned from this exercise:
Non-technical employees expressed an interest in the hackathon - this presented an opportunity to champion security outside of the technical teams, this is positive as it makes security a company-wide responsibility rather than solely engineering or security.
Preparation is everything - during the first hackathon, feedback revealed that uncompiled code within the environments ate into the testing time.
Financial rewards are a massive incentive - similar to bug bounty programs that organisations open up to the public, many people are motivated by monetary incentives.
Bugs found will likely highlight a trend in coding practices - single instances of poor code practices provide the opportunity to implement improvements across the entire codebase.
It is likely that some technical employees have a strong interest in security - this enables the nurturing of talent internally and paves the way for the creation of specialised security-focused roles within the engineering and testing teams.
Running an internal bug bounty hackathon is a practical way of building a security-first culture within your engineering team. The benefits gained from the event are determined by careful planning and setting clear objectives. Here are a few guidelines for setting up your bug bounty hackathon:
Clearly define the scope of each hackathon - make the area of focus clear to participants, and ensure that the time assigned is proportionate to the complexity of the application. If the scope is too broad, it may consume more time than planned and disrupt business-as-usual operations. It is better to segment the applications into multiple hackathons across the year than to attempt to shoehorn all of them into one.
Complete a dry run of the event beforehand - ensure the test environment is fully setup to make it efficient for participants, code should be compiled and ready to go. Before the event, attempt to look for bugs and ensure the environment is configured correctly with the necessary tooling in place.
Pick the right timing for the hackathon event - choose the right time for your hackathon to maximise engagement and participation. Assess your organisation and select a time that developers are most available, avoid times with major deadlines and holidays.
A hackathon could be scheduled after a product release to detect security flaws that may have been missed through the development lifecycle. This will serve as an opportunity for post-release security hardening.
Establish a process for logging and resolving bugs - incorporate the hackathon findings into your existing vulnerability management process, capturing details like CVSS score, exploitability, impact and track resolution time. Be sure to explicitly state the hackathon as the source of the finding i.e. March 2024 Hackathon to maintain traceability of the findings.
Create a write-up for the event - document its outcomes and share it across the organisation to further raise awareness and highlight the value it has delivered. Provide details around the number and severity of bugs found, lessons learned and improvements that will be implemented due to the hackathon. Company newsletters and security briefings are often a good platform for sharing.
Compensate participants fairly - ensure that participants are compensated fairly and in a timely manner to encourage them to take part again. Align compensation with the severity of bugs found i.e. critical vulnerabilities should have the highest compensation whilst low criticality vulnerabilities should have the lowest compensation.
The hackathon supplemented the existing penetration testing in place, allowing the teams to uncover obscure and intricate bugs based on their knowledge of the product. This was valuable because any bugs discovered could be fixed across all instances where the code was used, maximising the impact of a single discovery. I encourage everyone to experiment with internal hackathons to improve the quality of their code bases, foster the purple team skill set and increase security awareness across your organisation. By gamifying the security testing process, hackathons serve as an educational and impactful way of bringing security to the forefront.
Key takeaways are:
Many people across the organisation are likely to be curious about cyber security, and use the hackathon to involve them. It will be invaluable for building relationships in non-technical teams and spreading security awareness, promote the hackathon across the entire organisation.
Bugs discovered during the hackathon can help identify skill gaps within the development team, providing a reliable basis for selecting topics for the next round of security awareness training.
A hackathon can enhance cross-departmental communication by encouraging participants to collaborate with colleagues they wouldn’t typically interact with. In a remote environment, many people miss those ‘by chance’ water cooler discussions.
Collecting data on the type of bugs found in each hackathon can provide a number of stakeholders with valuable insights
Board level/Executive: understanding security maturity over time, cost-benefit of investment in secure coding tools
Engineering/Development: code quality improvement over time, gaps in developer knowledge
Cyber Security Risk Management: understanding the high-risk areas of code
If you’d like me to host or assist in planning your hackathon, please reach out via the Contact Me page.
