Dave Chronister has been doing penetration testing and incident response since 2007 — back when Fortune 500 CIOs called it a novel concept.
In the years since, he's walked into breached environments where the EDR was running, the MDR was installed, the SOC 2 audit had passed, and none of it mattered.
This conversation covers the gap between what companies think they bought and what they actually have why tools become the program by default, why compliance audits and security programs are two different things, what AI is actually doing to enterprise risk profiles right now, and what the organizations that survived a ransomware encryption event had that the ones who didn't were missing.
If you're responsible for a security decision, this is the conversation to have before the next one.
Is your security program real, or is it just theater?
The gap between a real security program and a collection of tools doesn't show up in an audit. It shows up during an incident — when it's too late to fix cheaply.
These eight questions come straight out of this conversation with Dave Chronister, founder of Parameter Security. Each one maps to something he's actually walked into. Answer them against what you know to be true right now — not what your vendor told you at signing.
1. Do you know which findings from your last security assessment are still open?
Dave has had a Fortune 100 client for four straight years. He pulled the year-one report and the year-four report side by side. Almost identical findings. Tools were bought, renewed, and re-certified the whole time — and the actual exposure never moved. If your remediation list looks the same as it did a few cycles ago, the program isn't the problem. The follow-through is.
2. When did your EDR last fire — and who responded?
Not whether it's installed. Whether it's being acted on. In recent insurance data, more than 60% of ransomware encryption events happened at organizations running a leading EDR. Detection without response doesn't stop an attack. If you can't say when it last fired and what happened next, you don't know if your coverage is real.
3. Is your MDR running in active response mode, or monitored mode?
These are not the same thing. Monitored means alerts get logged. Active response means someone acts on them. Dave has walked into environments where MDR sat in monitored mode for years while the client believed they had full coverage. Ask your vendor directly which one you're paying for, and get it in writing.
4. What does your SOC 2 certification actually cover — and what does it say nothing about?
SOC 2 audits whether your processes are documented and followed. It does not evaluate whether those processes protect you. A company can pass SOC 2 every year and carry the same critical vulnerability the whole time. If SOC 2 is your primary answer to "are we secure," that's the gap.
5. Do you know which AI tools are already running in your environment — and what data they have access to?
Shadow AI is already inside most organizations. A tool that entered your environment as a productivity assistant may now have access to email, internal documents, customer records, and approval workflows. If you don't have a current inventory of what's running and what it touches, you don't have an AI policy. You have AI usage.
6. Who actually owns the risk when something goes wrong — IT, the CISO, or the board?
IT holds the tools. The CISO advises. The board carries the fiduciary responsibility. Dave's seen the scapegoating pattern up close: a CISO with no real authority gets blamed for a decision the board never seriously evaluated. If your C-suite treats security as an IT line item, nobody with budget authority is actually deciding what risk is acceptable.
7. Have you defined what a win looks like before your next renewal?
Most companies don't know what "fixed" means before they buy. They implement, assume it worked, and move on. What specific risk reduction is this renewal supposed to buy, and how will you know if you got it? Walking into a renewal without that answer means negotiating blind.
8. What is your responsibility in this — specifically, in your seat?
This is the question Dave wishes every client asked before the engagement even starts. Not "what's the vendor's responsibility" or "what's IT's responsibility" — yours. Most people outsource the answer to a vendor or a department and never come back to it. If you can't state your own responsibility precisely, that's the gap an incident will find for you.
If two or more of these stung, that's the program talking, not the tools.
Tools get bought in response to a checklist or a renewal deadline. Programs get built in response to a defined risk. The full conversation goes deep on where that gap actually comes from and what it costs the companies that don't close it.
Chapters
00:00 — Why security awareness campaigns don't change buyer behavior
08:14 — Theater clients versus clients who actually want help
14:00 — What the pen test findings look like four years later
19:30 — Why tools become the security program by default
26:35 — What SOC 2 actually audits and what it ignores entirely
37:20 — Why 60% of ransomware victims had a leading EDR installed
44:50 — MDR in monitored mode versus active response — and why it matters
52:00 — The real risk of AI inside your organization
1:01:00 — Why the vendor selling the control shouldn't validate it
1:09:00 — Who actually owns the risk when something goes wrong
1:45:00 — The one question Dave wishes every company asked before buying anything
What We Mentioned
- NIST Cybersecurity Framework (CSF) — nist.gov/cyberframework
- NIST 800 series — csrc.nist.gov
- NIST AI Risk Management Framework — nist.gov/system/files/documents/2023/01/26/AIREF1.0.pdf
- SOC 2 / AICPA — aicpa-cima.com
- CMMC — dodcmmc.us
- PCI DSS — pcisecuritystandards.org
- HIPAA / HITECH
- Qualys — qualys.com
- Tenable — tenable.com
- Microsoft E5 / E7 security stack — microsoft.com
- YubiKey — yubico.com
- Parameter Security — parametersecurity.com
About Dave Chronister
Dave Chronister is the founder of Parameter Security. He started doing penetration testing in 2007 when most companies hadn't heard the term and has spent nearly 20 years walking into environments after the breach to find the things the audit missed. He's unusually direct about what tools actually do and what they don't, and he's seen every version of the gap between what companies think they bought and what they actually have.
LinkedIn: https://www.linkedin.com/in/davechronister
Company: https://www.parametersecurity.com
About Signed
The IT market is built for sellers, not buyers.
Signed is the podcast for the buyers. Host Max Clark, CEO of ITBroker.com, sits down with CIOs, CFOs, operators, and founders who’ve lived inside real enterprise tech deals — the ones who can tell you what actually determined whether the deal worked, not what the deck promised.
New episodes weekly. An ITBroker.com podcast.
.png)
