forumNew topic

Using ChatGPT in Red Teaming — would anyone do this during a test we hired them for, and what are the risks?

SSerkan A***Member
Job title
Chief Technology Officer
Sector
Cosmetics
Organization type
medium-sized business
Joined
Jan 2022
Message
13
#1

We're planning a comprehensive pentest and Red Team simulation for our company's e-commerce infrastructure and internal ERP systems. We received quotes from two different cybersecurity consulting firms, with budgets ranging between 90,000 TL and 160.000 TL. During one of the technical calls, I was taken aback when the consulting firm openly mentioned using ChatGPT and similar generative AI tools to craft social engineering scenarios and generate variations of certain custom exploits.

I know AI is used in cybersecurity for both offensive and defensive purposes but how common and ethical is this in a professional Red Team engagement? More importantly, could sensitive configurations, source code snippets or internal employee data from our systems be fed into these public AI models?

Is using external AI in these tests considered standard industry practice or is it just a shortcut that brings data leak and low-effort reporting risks? What kind of restriction clause should we put in the contract regarding this?

NNazlı T***New member
Job title
Accounting clerk
Sector
Seafood
Organization type
40-person manufacturing company
Joined
May 2026
Message
32
Most Helpful#2

Short answer: Red teamers using AI to draft social engineering phishing lures or tweak generic code blocks is becoming increasingly common; however, uploading a target organization's confidential data, IP addresses, or proprietary source code to public cloud models is a massive security and confidentiality violation.

AI tools are mainly used for two things right now: 1) Drafting grammatically flawless, persuasive, and targeted phishing emails during social engineering phases, 2) Speeding up routine scripting or automating the detection of known vulnerabilities. The main risk here is the consultant feeding your company's critical internal architecture logs, network topologies, or proprietary source code into public models. That data could end up in the model's training pool or be stored on third-party servers, causing a breach.

You need to draw clear boundaries at the contract stage. Add a clause to the NDA with the vendor stating: 'No client system outputs, source code, or proprietary information may be uploaded to publicly accessible AI services.' If they're going to do AI-assisted analysis, make it a requirement that they run it exclusively on local, fully open-source, air-gapped models hosted on their own servers.

GGamze K***Member
Job title
QA Tester
Sector
Printing
Organization type
sole proprietorship
Joined
Jan 2025
Message
178
#3

Public models won't generate zero-day exploits directly anyway because of safety filters. What consultants mostly do is refactor known code structures to bypass detection mechanisms. Pasting company code into cloud services—outside of local, air-gapped models—is an unacceptable vulnerability.

YYasemin K***Member
Job title
Board member
Sector
Jewelry
Organization type
cooperative
Joined
Feb 2023
Message
66
#4

During a 120,000 TL pentest we commissioned last month, we caught an engineer pasting internal network error logs into a public AI tool via our network traffic monitoring. We stopped the test immediately and terminated the contract. Monitoring outbound data leaks from the audit team unfortunately falls on the client now.

İİlknur G***VeteranCommunity member
Joined
Nov 2024
Message
80
#5

You have to appreciate their honesty for admitting they use AI, but asking for 160,000 TL just to outsource most of the work to a bot with canned prompts is a total rip-off. The whole point of Red Teaming is creative human intelligence; automated scanners already run standard template attacks anyway.

OOya I***Member
Job title
Board member
Sector
Cosmetics
Organization type
40-person manufacturing company
Joined
Aug 2023
Message
225
#6

Having AI analyze the target company and then invoicing it as Red Team consulting is a lucrative gig. When they deliver the report, watch out for the phrasing—make sure they didn't just copy-paste generic bot responses straight into the findings.

KKemal G***MemberCommunity member
Joined
Nov 2025
Message
5
#7

if theyre using air-gapped models running on local servers theres no issue but if theyre sending data to an external cloud then confidentiality is gone. make sure your contract has heavy penalties for that.

GGürkanMember
Job title
Energy sector
Organization type
boutique agency
Joined
Nov 2023
Message
94
#8

Under the Kişisel Verilerin Korunması Kanunu and information security standards, transferring employee or customer data obtained during testing to AI platforms hosted on foreign servers constitutes a clear regulatory violation. This could expose your company to administrative fines.

SSultan K***Member
Job title
Logistics planning
Sector
Food wholesale
Organization type
early-stage startup
Joined
Dec 2025
Message
105
#9

Your concern is entirely justified. Be sure to request a test methodology document from any bidding firms. Demand that they formally commit in writing to which tools they use at what stages and with what data security measures in place. A professional firm will not hesitate to document this.

IIrmak B***Expert
Job title
Finance Manager
Sector
Logistics
Organization type
a company within a holding
Joined
Jul 2023
Message
150
#10

I think it's hard to be that definitive about red team chatgpt. Forgotten test environments are more often the entry point than live systems.

Processes without records never improve, because you don't know what to fix. Proven by experience.

ZZübeyde K***Member
Job title
Purchasing manager
Sector
Sports and fitness
Organization type
family business
Joined
Feb 2025
Message
165

Doki · Corporate website · 2026

#11

same here.

HHüseyin K***MemberCommunity member
Joined
Jan 2024
Message
368
#12

Quick summary for newcomers: If it's your first time, start small; scaling comes later.

That's all, sorry if I went on too long.

OOrhan E***Member
Job title
Export manager
Sector
Law
Organization type
chain store
Joined
Dec 2025
Message
91

Doki · Vulnerability scanning · 2023

#13

I'd appreciate it if you shared the outcome. Any unwritten clause becomes a point of disagreement later, as both sides remember it differently.

Hope this helps.

NNilMember
Job title
Content manager
Organization type
a company within a holding
Joined
Apr 2024
Message
156
#14

Correct. Payment information changes are never verified through the channel they came from.

When we decide without measuring, we always end up in the same place.

BBatuhanMember
Job title
Freelance developer
Joined
Feb 2024
Message
128
#15

I went through the same thing two years ago. Processes without records never improve, because you don't know what to fix.

Of course, it varies if your situation is different.

SSelin Y***Veteran
Job title
Front office accounting
Sector
Real estate
Organization type
120-person company
Joined
Sep 2024
Message
111
#16

I agree, and I'd like to emphasize that. Taking notes for two weeks yields better results than a six-month estimate.

TTolga Y***Expert
Job title
Export manager
Sector
Printing
Organization type
chain store
Joined
Aug 2023
Message
3
#17

I'll argue the opposite, don't get mad. If you scold false alarms, nobody will report again.

If I were you, I'd go this route.

AAli R***Expert
Job title
Angel Investor
Organization type
two-branch business
Joined
Jun 2023
Message
192
#18

Generally correct, but one part is missing. If you get three different answers on a topic, the question was asked wrong.

If the notification path is long, notifications don't arrive; missing notifications mean delayed incident detection. Of course, it varies if your situation is different.

BBurcuMember
Job title
Frontend developer
Joined
Sep 2024
Message
96
#19

noted, thanks.

TTayfunVeteran
Job title
Software company owner
Joined
May 2023
Message
228

Doki · E-commerce infrastructure · 2024

#20

Absolutely. If I were to add anything: Most time waste accumulates in tasks waiting for approval.

If you have questions, write them; I'll answer as best I can.

Reply