AI summary by TwinMind

Cracking the Code on Offensive Security With AI ft XBOW CEO and GitHub Copilot Creator Oege de Moor

Sequoia Capital

51m 37s

AI Security Threat Landscape

  • AI code generation enables anyone to create code without security knowledge
  • Foundation models trained on public source code containing vulnerabilities
  • Result: more code generated with more security problems
  • Attackers already using AI to enhance their effectiveness
  • Creates greater threat: more code + more attacks = need for automation like Expo

Expo Company Introduction

  • Founded by Oiad Deore, creator of GitHub Copilot
  • Former Oxford computer science professor
  • Previous startup Seml acquired by GitHub
  • Expo: AI penetration testing company automating offensive security
  • Example of "Service as Software" model
  • 13 people at launch of results

Benchmark Performance Results

  • Initial industry benchmarks from commercial providers (Ports Wigger, Pentland): 75% success rate
  • Suspected training data contamination, created original benchmarks guaranteed not in training sets
  • Original benchmarks scored even better: 85% success rate
  • Comparison with 5 professional pentesters from reputed firms on same 24 challenges
  • Top-tier pentester (multi-billion dollar hedge fund level) scored same as AI
  • Human took 40 hours vs AI took 28 minutes
  • Results exceeded expectations - thought would take at least 1 year to reach reasonable proficiency

Real-World Vulnerability Discovery

  • Working with early design partners daily
  • Found critical vulnerability at major financial institution this morning
  • AI process:
    • Crawled web app, found PHP source code
    • Discovered insecure signing algorithm for host access
    • Continued crawling, found another endpoint
    • Reused previous technique with modifications
    • Found JavaScript code with candidate parameters
    • Gained access to endpoint intended for PDF downloads
    • Discovered could download password files
  • Demonstrates creative, multi-step reasoning similar to human experts
  • "Something like that every day"

Offensive Security Market Definition

  • Current best practice: invite external experts to simulate attacks
  • Highly skilled activity requiring years of training
  • Expensive and slow: typical penetration test costs $18,000
  • Only performed 1-2 times per year due to cost
  • Systems evolve faster than testing frequency
  • Expo automates this process for continuous testing instead of periodic

Market Transformation Through AI

  • Current market small due to limited highly skilled human experts
  • AI expanding market fundamentally:
    • More code generation → more vulnerabilities
    • Attackers using AI → greater threats
    • Automation becomes essential
  • Market expected to grow enormously
  • Analogy to military war games and red teaming exercises
  • National security implications - "first ever AI cyber warrior"

Technology Architecture

  • Most magic built on top of foundation models, not within them
  • Work with multiple foundation model providers
  • Benefit from competition between providers
  • Key components:
    • Benchmarks organized as curriculum to teach models
    • Specialized tools for LLMs (custom web browser for form filling/clicking)
    • Guard rails to prevent harmful actions (prevent "drop table" type disasters)
    • Attack surface discovery to prioritize targets
    • Prompt engineering and chain-of-thought prompting

Technical Challenges and Solutions

  • Hallucination management: run multiple attempts, better models need fewer attempts
  • Stochastic nature handled through repetition
  • Automated validator ensures reports are correct and reproducible
  • Human review still required for final fixes
  • Described AI as "overeager super brilliant teenager" needing supervision

Data and Training Approach

  • Benchmarks critical for evaluation and improvement
  • Multiple data sources:
    • Security expert contractors creating new benchmarks
    • Mining open source (Docker Hub images)
    • Self-generated data from successful attacks
  • Reinforcement learning on specific examples
  • Similar improvement trajectory expected as other games like Go

Cloud-Only Deployment Strategy

  • Available only in cloud, not downloadable software
  • Enables control over scope and usage
  • Customers must prove legitimate ownership of targets
  • Prevents misuse for attacking others' systems
  • Security measure against potential bad actors

Pricing Model Innovation

  • Hybrid approach combining subscription and consumption:
    • Base license based on engineering team size (similar to GitHub)
    • Additional "attack hours" for thorough testing
    • Fixed attack hours included with base license
    • Extra service fees for deeper testing
  • Clear correlation between resources consumed and customer benefit
  • Enables continuous testing after every code change

Future Roadmap

  • Currently in careful early design partner phase with human supervision
  • Once supervision no longer needed, expect rapid deployment
  • Easy deployment: point at existing servers for immediate results
  • Goal: significantly transform web security by next summer
  • Plans for open source demonstrations and platforms like HackerOne

Broader AI Impact Predictions

  • Short term: customer support disruption already happening
  • Social media problems could be mitigated with AI
  • Long term: health and biology biggest impact areas
  • Coding transformation: more people able to create software
  • Professionals focus on conceptual ideas and architecture
  • Future: conversation with model becomes the artifact, not code
  • "English is the coding language"

Personal Journey and Philosophy

  • Left Oxford professorship for commercial impact and speed
  • Small teams with profit motive move faster than academic settings
  • GitHub Copilot launched with only 10 people
  • Focus and speed as key startup advice
  • Imperative to build this technology before "bad guys" do
  • Protecting the Free World's software infrastructure

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome