Australian organisations put AI into production faster than anyone tested whether it could be trusted. Chatbots on the front page, copilots inside the business, agents wired to real tools with real permissions. The systems shipped. The testing did not.
I started Provok because I kept seeing the same gap from the governance and assurance side of the table: organisations that could tell you their AI was live, but could not tell you whether it was safe. Those are different questions, and the second one almost never got asked.
Red teaming is adversarial testing. You take the system you have deployed and attack it the way a motivated outsider would, on purpose, under authorisation, and you document what breaks. Not a checklist. Not a scan. A person deliberately trying to make your AI do something it should not.
It matters because the failure mode of an AI system is not the failure mode of ordinary software. Traditional security asks whether someone can break in. Red teaming an AI asks a stranger question: what can the system be talked into. Can it be made to ignore its own instructions. Can it be coaxed into revealing data it was never meant to show. Can its connected tools be turned against the business. The system does not need to be hacked in the old sense. It just needs to be convinced.
A penetration test checks the network and the API. It is necessary, it is mature, and it does not look at the model. The attack surface that opened when you added an LLM, prompt injection, guardrail bypass, system prompt leakage, tool and function abuse, excessive agency, sits underneath the layer your pen test covers. The OWASP Top 10 for LLM Applications exists precisely because this is a distinct category of risk. Most organisations testing AI today are testing everything around it and nothing inside it.
This is the part I care about most, and it is structural, not a criticism of anyone's competence.
The people who built the system know where the guardrails are. That knowledge is exactly what stops them finding the ways around them. They test the paths they designed. An adversary does not care about the design, they care about the gap between what you intended and what you shipped, and that gap is invisible from the inside.
Most AI in production was built by a capable team, an in-house group or an external vendor who knew what they were doing. That is not the problem. The problem is that none of them can be the ones to test it. A vendor will not adversarially attack the system they just sold you. An internal team cannot find the gaps in a design they hold in their heads. It does not matter how good the builder is. The test has to come from outside the build, or it is not a test.
Provok does not test systems built by Evolaition, a business I also run. Independence you can influence is not independence.
There are automated tools that promise to red team AI for you. They have a place, and they also mislead confidently.
On one engagement, an industry-standard jailbreak probe reported a 99.6 per cent attack success rate across more than a thousand attempts. A board reading that number would have concluded the system was completely broken. We pulled the actual responses and read them by hand. Not one showed the system breaking role. It had defended itself on every single attempt and been scored as a total failure.
That is the work. The tools generate noise. Someone with judgement reads the noise and tells you what actually happened. On another engagement, against an AI wired to real tools, we made it read a different customer's account, email their invoice to an outside address, and issue a refund against their balance. Twelve of fourteen attacks succeeded, and every one was judged against the system's own logs, not a scanner's guess. No inference. No false positives. Evidence.
Attackers are already probing LLMs in the wild. Auditors, boards and enterprise buyers have started asking for AI-specific assurance, and "we deployed it carefully" is not an answer to "prove it holds". The gap between shipping AI and testing it is where the risk sits, and it is widening while everyone ships faster.
If you have put AI in front of customers or staff, and a breach, a leak, a wrong answer, or an action taken on someone's behalf would cost you more than a bad review, that AI should be tested by someone who did not build it. Better it is us, on your authority, in a controlled environment, than the internet on its own terms.
That is the work I do. If you have deployed something that matters, I will tell you how I would try to break it.
Independent, onshore, authorised, evidence-sealed.
Start with a conversation