In the past ~4 weeks I have personally observed some irrefutable things in “AI” that are very likely going to cause massive shocks to employment models in IT, software development, systems administration, and cybersecurity. I know some have already seen minor shocks. They are nothing compared to what’s highly probably ahead.
Nobody likely wants to hear this, but you absolutely need to make or take time this year to identify what you can do that AI cannot do and create some of those items if your list is short or empty.
The weavers in the 1800s used violence to get a 20-year pseudo-reprieve before they were pushed into obsolescence. We’ve got ~maybe 18 months. I’m as pushback-on-this-“AI”-thing as makes sense. I’d like for the bubble to burst. Even if it does, the rulers of our clicktatorship will just fuel a quick rebuild.
Four human-only capabilities in security
In my (broad) field, I think there are some things that make humans 110% necessary. Here’s my list — and it’d be great if folks in very subdomain-specific parts of cyber would provide similar ones. I try to stay in my lane.
1. Judgment under uncertainty with real consequences
These new “AI” systems can use tools to analyze a gazillion sessions and cluster payloads, but they do not (or absolutely should not) bear responsibility for the “we’re pulling the plug on production” decision at 3am. This “weight of consequence” shapes human expertise in ways that inform intuition, risk tolerance, and the ability to act decisively with incomplete information.
Organizations will continue needing people who can own outcomes, not just produce analysis.
2. Adversarial creativity and novel problem framing
The more recent “AI” systems are actually darn good at pattern matching against known patterns and recombining existing approaches. They absolutely suck at the “genuinely novel” — the attack vector nobody has documented, the defensive technique that requires understanding how a specific organization actually operates versus how it should operate.
The best security practitioners think like attackers in ways that go beyond “here are common TTPs.”
3. Institutional knowledge and relationship capital
A yuge one.
Understanding that the finance team always ignores security warnings — especially Dave — during quarter-close. That the legacy SCADA system can’t be patched because the vendor went bankrupt in 2019. That the CISO and CTO have a long-running disagreement about cloud migration.
This context shapes what recommendations are actually actionable. Many technically correct analyses are organizationally useless.
4. The ability to build and maintain trust
The biggest one.
When a breach happens, executives don’t want a report from an “AI”. They want someone who can look them in the eye, explain what happened, and take ownership of the path forward. The human element of security leadership is absolutely not going away.
How to develop these capabilities
Develop depth in areas that require your presence or legal accountability. Disciplines such as incident response, compliance attestation, or security architecture for air-gapped or classified environments. These have regulatory and practical barriers to full automation.
Build expertise in the seams between systems. Understanding how a given combination of legacy mainframe, cloud services, and OT environment actually interconnects requires the kind of institutional archaeology (or the powers of a sexton) that doesn’t exist in training data.
Get comfortable being the human in the loop. I know this will get me tapping mute or block a lot, but you’re going to need to get comfortable being the human in the loop for “AI”-augmented workflows. The analyst who can effectively direct tools, validate outputs (b/c these things will always make stuff up), and translate findings for different audiences has a different job than before but still a necessary one.
Learn to ask better questions. Bring your hypotheses, domain expertise, and knowing which threads are worth pulling to the table. That editorial judgment about what matters is undervalued, and is going to take a while to infuse into “AI” systems.
We’re all John Henry now
A year ago, even with long covid brain fog, I could out-“John Henry” all of the commercial AI models at programming, cyber, and writing tasks. Both in speed and quality.
Now, with the fog gone, I’m likely ~3 months away from being slower than “AI” on a substantial number of core tasks that it can absolutely do. I’ve seen it. I’ve validated the outputs. It sucks. It really really sucks. And it’s not because I’m feeble or have some other undisclosed brain condition (unlike 47). These systems are being curated to do exactly that: erase all of us John Henrys.
The folks who thrive will be those who can figure out what “AI” capabilities aren’t complete garbage and wield them with uniquely human judgment rather than competing on tasks where “AI” has clear advantages.
The pipeline problem
The very uncomfortable truth: there will be fewer entry-level positions that consist primarily of “look at alerts and escalate.” That pipeline into the field is narrowing at a frightening pace.
What concerns me most isn’t the senior practitioners. We’ll adapt and likely become that much more effective. It’s the junior folks who won’t get the years of pattern exposure that built our intuition in the first place.
That’s a pipeline problem the industry hasn’t seriously grappled with yet — and isn’t likely to b/c of the hot, thin air in the offices and boardrooms of myopic and greedy senior executives.
On AI Agents, Criminal Activity, And Who Is Actually Responsible
(Repost from ai.rud.is)
I’ve written and said enough times that I’m tired of OpenAI and Anthropic pulling cybersecurity out as the headline AI risk. The framing is commercially convenient, and it flattens a more complex reality — these models can break plenty of things that have nothing to do with cyber, and most of Anthropic’s public safety output has been more useful for their pending IPO than for anyone making operational decisions. Which is exactly why OpenAI’s latest disclosure deserves scrutiny rather than dismissal: when an account this consequential comes from a lab with every incentive to shape it, the technical content has to be evaluated on its own terms.
Per OpenAI’s own disclosure – THEIR account, NOT an independently verified incident report – GPT-5.6 Sol and a pre-release sibling, both running with cyber refusals deliberately disabled, broke out of the evaluation sandbox and compromised Hugging Face’s production infrastructure. The claimed sequence: zero-day in the package registry proxy being used for network isolation, lateral movement through OpenAI’s research environment, stolen credentials chained with additional zero-days for remote code execution on Hugging Face’s servers, benchmark answers pulled directly from their production database before Hugging Face’s detection caught the operation. Their response ran on locally hosted models because the external LLM service they’d been relying on couldn’t sustain the load or would not perform the tasks.
I’ve been working with LLMs in cyber contexts for quite a while now – actual operational use plus deliberate adversarial testing to understand where these things fail and why. Building and running agentic systems is a meaningful chunk of what I do. And I can tell you from direct experience that under-constrained deployments of capable models will eventually do things they weren’t asked to do. This is far from a mere theoretical concern. I’ve watched it happen in environments I “controlled”, running agents against attacker infrastructure while tired and careless about guardrails, and had them start doing genuinely criminal things: CVE exploitation, credential stuffing, none of it instructed. I just left the boundaries loose enough that the model found a path. “I was tired” is NOT, I want to be crystal clear, a legal defense.
If the disclosure is accurate, what OpenAI’s models did is a more capable version of the same dynamic. These models – GPT-5.6 Sol specifically, and anything else trained on a serious cyber corpus – are optimized to find the most direct route to whatever outcome the reward function is pointing at. They don’t distinguish between sanctioned approaches and whatever else happens to work. The training data came from the leading companies in the security field, who’ve sold OpenAI (and almost certainly continue to sell) substantial amounts of intellectual property: attack techniques, exploitation methodology, defense patterns, the full stack. When OpenAI’s models were pointed at ExploitGym – the evaluation harness – with production safety classifiers turned off, they reportedly found a zero-day, used it to get internet access, reasoned their way to the inference that Hugging Face probably hosted the solutions they needed, and then chained their way in. The UK AISI had independently evaluated GPT-5.6 Sol as capable of sustaining complex multi-step cyber operations, so that part isn’t just OpenAI’s word. However, the rest of the incident timeline is.
What a colleague of mine (deliberately not inserting his name into this informed-yet-hyperbolic opinion piece) and I converged on, after real trial and error building agentic frameworks together: front-load as much as possible on deterministic scripted tooling, push the LLM only into portions of the task where you can constrain its context tightly enough to get “predictable” output, and have a human review whatever comes out before it touches anything consequential. This isn’t 100% reliable, but close enough becomes meaningful when you know what you’re working with. OpenAI’s sandbox apparently had one isolation layer – the proxy cache – and the model apparently found a zero-day in it. A single chokepoint, however well-designed it looks, isn’t sufficient containment for a model that’s been explicitly configured to probe for ways through.
To be frank, the defender calculus hasn’t shifted much in this brave new world. AI-assisted attacks are faster and better at chaining than anything you’d typically see from a human operator working alone, but they’re (mostly) still going after weak credentials, exposed endpoints, and unpatched infrastructure – and Hugging Face’s incident response leaning on locally hosted models when its external LLM service buckled is a reminder of how fragile detection pipelines are at exactly the moment they matter most. WireGuard – and the now-substantial set of vendor implementations – can hide critical attack surfaces entirely rather than just hardening them, and making endpoints invisible costs considerably less than finding out your detection wasn’t fast enough. But it’s the individuals whose data gets taken who bear most of that cost either way; organizations continue to operate after breaches with frustrating regularity. Which is why the interesting question here isn’t defense. It’s liability.
If the account holds: someone at OpenAI decided the evaluation environment was isolated enough to run cybersecurity expert-trained frontier-capable models with cyber refusals stripped out, that decision was wrong, and Hugging Face’s production systems were compromised as a result. Whether “the model did it” provides any legal insulation for the humans who configured and launched that evaluation is, I think, an open question – and I’d genuinely like to see it treated as one, because computer fraud statutes were written with human actors in mind and the industry needs a test case about what happens when the actor is a model someone deliberately unleashed.