Building an Agentic Pentesting System: Architecture, Safety, and Evidence

A confidential penetration test, many systems to inspect, and a deadline. I wanted a small local AI model to carry some of the work. Running security tools was the easy part. The hard part was giving the model enough structure to choose useful work, remember what changed, resist hostile input, and prove what it found.

What this article is about

Companies sometimes pay people like me to try to break into their systems. Legally. That controlled exercise is a penetration test: find weaknesses before a criminal does, prove the important ones safely, and help the client fix them.

On one client project, there was a lot to test and a fixed deadline. We had scanners and other automated tools, but a pentester still had to choose the next useful test, connect results from different tools, follow new leads, and turn the work into a report.

I wanted to see how much of that work an AI system could carry.

The data could not leave our environment, so a public AI service was not an option. The system had to work locally, including with models that were relatively small and not trained specifically for cybersecurity.

That constraint turned a practical experiment into a more interesting question: what has to exist around an imperfect AI model before it can contribute to a real pentest?

I will start with the job itself, explain what "agentic AI" means without assuming an AI background, and look briefly at the open-source projects I evaluated. From there, the article follows the architecture from the client's rules to task planning, memory, authentication, tool safety, proof, completion, and reporting.

The aim is not an automatic verdict or a replacement for a pentester. It is a useful head start with a reviewable record: what the system tried, what the evidence supports, and what still needs a person.

Penetration testing in two minutes

A pentest begins with permission, not a scanner. The client decides what may be tested, what is off limits, and how much risk is acceptable.

The tester then looks for ways an attacker could enter a system, gain access they should not have, or misuse a feature. Important weaknesses are demonstrated carefully rather than guessed. The client receives a report that explains the risk, the proof, and how to fix it.

The tester may begin with different amounts of information. A black-box test is like approaching a building from the street. A gray-box test might include a key card or part of the floor plan. A white-box test can include the blueprints, such as source code and architecture documents.

Most pentests are time-boxed. The possible surface can include websites, networks, mobile apps, cloud systems, accounts, devices, and APIs, the interfaces software uses to exchange data. There is almost always more to examine than one person can test equally, so the tester has to choose where to spend time and record what was skipped.

The work does not move in a straight line. A new account can reveal pages that were invisible before. Access to one machine can expose another part of the network. A test that succeeds can create an entirely new round of discovery.

NIST SP 800-115 describes this movement between planning, discovery, testing, and reporting. Some of its tool examples are dated, but the central idea still holds. A pentest is an investigation, and the plan changes as the tester learns.

The problem I wanted AI to help solve

The tools we already used saved time within their jobs. Crawlers found pages. Scanners recognized known patterns. Proxies recorded traffic. I still had to connect their results, choose the next useful test, and keep track of what remained.

That connecting work was where I wanted AI to help.

I wanted a system that could carry a bounded part of the investigation. It should choose a focused next step, use the available tools, read the result, update the plan, and continue. At the end, it should give the pentester a clear account of what it tried, what it proved, and what it could not finish.

That required more than attaching a model to a terminal. The system needed to preserve the investigation between assignments and produce something another pentester could review. Confidentiality applied to the whole workflow, including tools, stored evidence, and monitoring.

What "agentic" means here

"Agentic AI" is one of those phrases that now appears everywhere. The idea is less mysterious than the label.

A chatbot usually answers once. An agent is given a goal and can take a series of actions to reach it.

It looks at the situation, chooses an action, uses a tool, reads what happened, and decides what to do next. The loop continues until the goal is complete or the agent reaches something it cannot safely handle.

The ReAct paper described an influential version of this pattern by connecting model reasoning with actions and observations.

The important difference is choice. In a fixed workflow, ordinary software decides the sequence and the model fills in a few blanks. In a more agentic workflow, the model can change direction after it learns something new.

That freedom creates the value. It also creates the risk.

Imagine giving a pentester an assignment brief, a notebook, a library of testing guides, a set of tools, and a badge that opens only the systems the client approved. The pentester still has to decide what to do. The brief sets the rules. The notebook carries knowledge forward. The library provides methods. The badge limits access.

An AI agent needs the same support. The model supplies judgment, but the model alone is not the agent.

For Agentic Pentest, that freedom belongs inside a focused assignment. A worker can investigate and adapt, while the wider system keeps track of what its result changes across the assessment. The architecture below explains how those two levels fit together.

I looked for an existing answer first

Before writing more code, I wanted to know whether an open-source project already solved the problem.

Agentic security is an active open-source field. I reviewed and tried projects that approached the work in different ways. The comparison below captures the architectural idea that mattered for this project, not every feature each tool provides.

In some early trials, an agent would follow an interesting lead deeply, then struggle to recover the wider plan when that lead went nowhere. That experience made continuity across the assessment a priority for me.

I wanted focused assignments to return to a shared investigation record, with testing progress and reusable access kept visible throughout. For our constraints and my intended methodology, building that workflow directly was more practical than adapting one of the projects I tried.

These projects move quickly. The descriptions and links above refer to the repository revisions I reviewed on August 31, 2026.

Six problems appear when the AI can act

Once the model can choose and execute work, mistakes affect more than the quality of an answer. They can waste the testing budget, distort the report, or cause an action the client never permitted. Six problems shaped the design.

Problem 1: Every success changes the map

A pentest begins with an incomplete picture.

Finding one page can reveal a family of related functions. Gaining access to one machine can reveal another network. Creating an account can expose an application that did not exist for an anonymous visitor.

That means the plan cannot be written once and followed blindly. Discovery creates tests. Tests create more discovery. The system needs to return to the map whenever the view changes.

Pentesters often call the mapping work enumeration. The word matters less than the loop. Look, test, learn, and look again.

Problem 2: The same action can mean different things for different users

Imagine an online store with two customers, Alice and Bob.

Alice should see Alice's orders. Bob should see Bob's. A support employee may be allowed to see both.

Testing the order page once as Alice proves very little. The important question is what happens when Bob asks for Alice's order. If Bob can read it, the application has crossed a security boundary. If the support employee can read it, the same behavior may be completely normal.

The page did not change. The person asking did.

So the system has to remember who is signed in, what that user is allowed to do, and who owns the information being requested. Security engineers call this identity, session, and authorization context. The plain-language question is simpler.

Who is doing what to whose data?

Lose that context and the AI can miss a real access-control bug or report a normal result as a vulnerability.

Problem 3: The target can give the AI hostile instructions

An AI worker reads information from the target. That information may include pages, documents, source files, error messages, and tool output.

Some of that text may look like an instruction.

A malicious page could tell the model to ignore its assignment, reveal a password, run another command, or declare the target safe. This is called indirect prompt injection. The AgentDojo paper explores the same general problem in agents that use tools while reading untrusted data.

Telling the model to ignore hostile instructions is useful, but it is not enough. A manipulated worker must still be unable to change what the client allowed, reach unrelated secrets, or silently place a false claim in the report.

Problem 4: A test result can mislead in either direction

A scanner can raise an alert and be wrong. It can also report nothing while leaving an important question unanswered.

A test might never reach the intended part of the system. A firewall could block it, a login could expire, or the tool could stop early. OWASP's architecture guidance describes how protective components can intercept tests and change the responses a tester sees.

An inspector stopped by a guard in the lobby has learned something about the guard. They have not inspected the doors upstairs.

The same distinction matters here. Evidence that a protection layer stopped an attempt can be a useful result. It does not, by itself, settle whether the system behind it has the suspected weakness. And an unexpected response alone does not establish that a firewall was responsible.

So the agent needs to distinguish a test that found no weakness under the conditions checked from one that could not answer the question. Relevant tool records, confirmed access, and defensive logs available to the operator can help explain the difference. Sometimes the next step is an approved follow-up. Sometimes it is a limitation for the pentester to resolve.

Positive results need the same care. A confident explanation is not proof, and asking a model to reconsider is not an independent check. Research on intrinsic LLM self-correction found that correction without external feedback was unreliable in the settings evaluated.

The record should therefore distinguish what happened, what it might mean, and what the evidence actually supports.

Problem 5: An empty task list can still hide missed work

An AI planner creates work from what it currently knows. If its view is incomplete, its to-do list will be incomplete too.

Finishing every item only proves that the list is empty. It does not prove that the system tested the important pages, accounts, workflows, services, or security questions.

Many capture-the-flag challenges reward reaching a particular goal, such as finding a hidden flag. One successful route may be enough. A broader client assessment must also account for the other areas that need attention.

Before stopping, the system needs to check what was answered, what was skipped, and what could not be tested. That is the purpose of a coverage review.

Problem 6: Changing the model changes the quality of the work

One goal was to use a relatively small local model that had not been trained specifically for cybersecurity. That makes model ability a practical design constraint.

Two models can receive the same assignment, notes, and tools and still produce very different work. One may recognize an important clue. Another may misunderstand the output or miss the question worth asking. Size alone does not tell us how well a model will handle this job.

There are two concerns here. Can the model do useful assessment work? And when it makes a mistake, can the system contain the consequences and show what remains uncertain?

A good architecture helps with both, but it cannot make every model equally capable. Supporting different models should mean preserving the same rules, evidence requirements, and review process as models change. It should not be a promise of equal results.

How I structured Agentic Pentest

My central decision was to keep the assessment in a shared record that survives each worker's assignment. The record holds the task plan, the discovered target and its testing history, possible vulnerabilities, and reusable access.

Think of the building inspection again. An inspector can spend an hour examining one door without making the other rooms disappear from the floor plan. Another inspector can continue from the same notes.

The architecture below is a design blueprint, covering the assessment loop and the controls it needs. The demo later in the article is a separate view of the tool in use, not a demonstration of every capability described here.

The assessment loop: Choose, investigate, update

Three roles carry an ordinary task through the loop.

1. The selector chooses a task. It reads the current plan and the relevant records to identify useful work whose prerequisites are satisfied.
2. The worker investigates that task. It reads the information it needs, uses approved tools, and adapts within the assignment. Its activity and results are preserved.
3. One updater brings the record up to date. It reads the activity and existing records, proposes the changes supported by the evidence, preserves unresolved work, and adds justified follow-up tasks.

The surrounding software checks the proposed changes before saving them. The same controlled process handles updates to the task plan, target map, vulnerability record, and access information. A worker's concluding paragraph does not become the assessment's official record on its own.

This gives the system two levels of progress. The worker follows a focused line of investigation. Between assignments, the shared record brings the whole assessment back into view. When nothing is runnable, a review checks for missing work and blocked prerequisites before allowing the report.

AI interprets results and decides what they mean. Ordinary code checks permissions, valid references, and allowed record changes, then preserves the accepted state. Those checks can reject an invalid update. They cannot, by themselves, prove that a vulnerability is real. That still requires evidence.

Model design: Give each model work it can handle

The split keeps each model call responsible for a recognizable job. A worker can concentrate on one assignment without also having to maintain the entire plan and decide whether the pentest should end.

That responsibility needs a clear interface. The worker starts with its objective, the client's rules, and references to the relevant records. It can read more when needed. Its assignment should not require reconstructing the assessment from a long conversation or joining several competing versions of the same fact.

This is useful beyond small models. Lost in the Middle found that the models studied could miss relevant information buried inside long inputs. More available context did not guarantee that every important detail would be used.

For a less capable model, I would favor narrower assignments, a smaller tool selection, clearer guidance, and more human checkpoints. A stronger model may handle broader work. Those choices need evaluation with the actual model and tools, rather than an assumption based on model size.

Prompts should make the objective and expected outcome clear. Where software needs a structured answer, it should ask for only the information it actually consumes. Durable records carry continuity between calls.

A model that cannot finish should leave a specific unanswered question. These choices make its work easier to review and continue. Their value still needs to be measured with the selected model.

The remaining components explain how those decisions carry through an assessment.

Scope: The client sets the boundary

Scope tells the system where it may work, which actions the client allows, and when it must stop or ask for approval.

A page may redirect to another domain. A cloud service may mention a new resource. A password may open a service that was not included in the agreement. The AI can notice those things, but it cannot decide that they are now in scope.

Every run begins with an engagement configuration, the machine-readable version of the client's rules. It records the approved targets, exclusions, accounts, available tools, approval mode, and other instructions.

The brief explains the goal. Machine-readable limits give the execution controls something precise to check. Both are needed: a written exclusion only has force if the tools are prevented from crossing it.

The AI can propose an action. Only the client's rules and the enforcement layer can authorize it.

Those rules also travel with the work. Later, a pentester should be able to tell which target and account an action belonged to without reconstructing the answer from a long transcript.

The engagement should also define which actions need approval and when to hand control back to the operator. The execution layer must enforce those decisions.

Tasks: One coherent piece of work at a time

Tasks divide the assessment into assignments that can be followed, reviewed, and resumed without losing the wider plan.

A useful task sounds more like this.

Map the checkout process for a signed-in customer and save the pages and data objects it uses.

The objective is specific enough to review, but it still leaves room for the worker to investigate.

A task should be large enough for a coherent investigation and small enough to review or resume independently. The system needs to know what the assignment answered and which branches remain open.

Tasks come from several places. The first set comes from the known target and testing guidance. Later tasks come from new discoveries, possible vulnerabilities, new accounts, missing areas, or a worker that needs another bounded pass.

That is how the mapping-and-testing loop becomes actual work. A discovery adds an assignment, and an unfinished investigation keeps a clear continuation.

A task needs a clear objective, the relevant target and account, any prerequisites, and the observations that justify it. A task waits when something necessary is missing. When several are ready, the system chooses useful work in light of the wider assessment.

If a lead goes nowhere, the task keeps a concise handoff and links to the recorded activity. The target map and vulnerability record keep the testing questions and their outcomes. Completing an assignment does not erase unfinished assessment work.

The worker explains what it completed, what remains, and what prevented progress. The updater reconciles that account with the saved activity before changing the shared records. A broken tool or malformed AI answer remains a runtime failure. It is not rewritten as a completed security test.

Memory: Keep the facts together and the context focused

Memory lets each task use what earlier tasks learned, including the context needed to interpret it correctly. Human pentesters take notes for the same reason.

The design gives each kind of durable information a clear home.

• The task plan holds assignments, progress, dependencies, and the handoff needed to resume work.
• The target map holds discovered systems and components, with the tests completed and questions still open for each.
• The vulnerability record holds suspicions and decisions, together with completed validation and the checks still needed.
• The access record holds reusable credentials and sessions, their current usability, and instructions for obtaining or renewing approved access.

This organization keeps related facts together. To understand a component, the worker can read what is known about it and what remains to test in the same place. To understand a possible vulnerability, it can find the claim, supporting evidence, and outstanding validation together.

Tasks link to these records rather than carrying their own competing copies. A finished task and an unresolved security question can therefore coexist without contradiction.

The raw activity is preserved separately. It contains the commands, outputs, errors, and saved artifacts behind the working records. A concise note helps the next decision. Its evidence link lets a reviewer check that note against what actually happened.

A worker starts with references to its task and the related records, then reads what is relevant. That includes information from other tasks when it matters. A newly available account may support a later assessment. A page found earlier may answer part of the current question. A rejected suspicion should not need to be rediscovered.

This gives the system local context and assessment-wide continuity without creating a separate memory store for every scope. The worker's brief is a reading aid, not another source of truth.

The updater's job is to keep the working records consistent with the evidence. A note should preserve whether it is an observation, an unresolved interpretation, or a supported conclusion. It also needs enough source context to identify the task, account, and raw result behind it. This is often called provenance. It lets the pentester ask, "Where did this claim come from?"

Saving an update does not make every statement in it true. Possible vulnerabilities remain possible until validated. A misleading instruction in target content remains untrusted input even if it appears in a saved transcript.

Changes should remain auditable, so a later reader can understand why a conclusion changed. The raw activity also matters after an interruption. Before repeating an action, the system should check whether it already took effect, especially if it created an account or changed the target.

Methodology: Give the agent relevant guidance

Methodology gives the agent relevant questions to investigate and guidance on how to use its tools. Experienced pentesters draw on standards, checklists, documentation, and past work in much the same way.

A fixed checklist brings consistency, but it cannot adapt to every system. A model relying only on its training can adapt, but it may forget a class of tests, use old tool syntax, or invent a method that does not fit.

The design combines model judgment with a curated library of testing methods, tool documentation, and reusable procedures. The useful question is whether the guidance fits this target and this assignment. A network assessment and a mobile application need different reference material.

The worker can receive the relevant guidance without loading the entire library. Searching for useful passages and bringing them into the model's context is called retrieval-augmented generation, or RAG. Think of handing an inspector the relevant pages of a manual. Reusable procedures can also be supplied directly when the task calls for them.

At the beginning of a run, the library helps form the first mapping tasks. During the pentest, it supports individual tests. Before the system stops, it helps check whether an important class of work was missed.

Guidance needs its own quality control. Procedures should identify their source, applicable conditions, and relevant tool version. A successful technique is useful to retain, but only with enough context to know when it applies.

A testing guide can explain why password reset deserves attention. Only evidence from the target can establish whether its password reset is vulnerable.

Authentication: Remember who is signed in

Authentication keeps each test tied to the right user and preserves usable access between tasks. Signing in changes both what the agent can see and what a result means.

Return to Alice and Bob in the online store. If a task creates Alice's account, a later task should be able to reuse Alice's signed-in session. It also has to remember that the session belongs to Alice. Otherwise, the system cannot correctly test whether Bob can reach Alice's data.

A session works like a temporary badge. It belongs to a user, carries a level of access, opens part of the target, and eventually expires.

The access lifecycle should cover accounts supplied by the client and accounts created with permission. It should keep the account, role, target, and session together, notice when access expires, and renew it when allowed. Otherwise, a later task can mistake an expired login for a security result.

Saved credentials and sessions should be available only to work that needs them. The system must distinguish access that is ready to use, access it can obtain through an approved procedure, and access that is blocked by a missing requirement.

That may require a controlled authenticator for time-based codes or a dedicated test mailbox for emailed codes. If a person must approve a step, the system should show that dependency rather than work around it.

Tooling: Put execution behind controls

Tooling lets the agent act on the target while keeping those actions within the client's permission and an environment the operator controls.

A configurable execution layer connects the chosen model to tools such as browsers, scanners, and analysis utilities. The software that runs the model and its tool interactions is often called an agent harness. Different harnesses can fit this design, but each must operate within the same permission and record-integrity controls.

Permission and isolation solve different problems. Permission determines which action is allowed. Isolation limits what a worker can reach if it makes a mistake or follows a hostile instruction.

Each task should run in an isolated environment with only the tools, files, network access, and credentials it needs. Actions should be checked against scope and approval rules, with limits on time and resource use. The system should preserve an activity record and the supporting results for later review.

These controls have to apply to every route the worker can use. Restricting a browser tool achieves little if an unrestricted command can reach the same destination. A new conversation or a fresh process also provides no automatic guarantee of isolation.

A sandbox does not stop the model from misunderstanding a malicious page. It reduces the consequences of that mistake. The resulting notes and claims still need review before they enter shared memory or the report.

Confidentiality also depends on the entire data path. A local reasoning model can still send sensitive material to an external search, embedding, or tracing service. An embedding service, for example, sees the text it converts into a searchable representation. Every service that receives client data needs to fit the engagement's requirements.

The operator should be able to pause the run, inspect pending actions, and stop further execution. Temporary accounts, sessions, and other permitted changes need a cleanup record so the assessment does not leave access behind unintentionally.

Evidence: A promising result is not yet a finding

When an agent thinks it has found a vulnerability, the system checks the claim before presenting it as a finding. Every false positive, a reported problem that is not real, creates more work for the pentester we were trying to help.

During a building inspection, a door that looks unlatched deserves attention. Appearance alone does not establish that someone can enter a restricted room. The inspector still needs to establish who may enter, what actually happens, and whether that behavior breaks the intended rule.

The same distinction applies to software. A suspicious result becomes a candidate, meaning a possible vulnerability with supporting observations and a question still to resolve. Review considers the original evidence, the account used, the affected system, and the expected behavior. If the evidence does not already settle the question, the candidate needs focused validation.

Validation has three useful outcomes.

• Confirmed. The evidence demonstrates the weakness and supports the stated impact. The report includes the affected component, the relevant conditions, the proof, and the recommended fix.
• Rejected. The evidence explains why this particular suspicion was mistaken. That decision remains in the record so another task does not restart the same false alarm.
• Unresolved. A missing account, unavailable tool, permission limit, or inconclusive result prevents a decision. The record states what is missing and what the pentester needs to check.

In Alice and Bob's store, confirmation requires evidence that Bob reached information he should not see. If the required second account is unavailable, the question remains unresolved.

An unsuccessful validation attempt does not automatically reject the candidate. If the session expired or a protection layer prevented the intended check, the evidence may still be inconclusive. The record should say which question remains open.

A deeper check is justified when it can resolve a specific uncertainty within the agreed scope, risk, and time limits. Repeated attempts with no new information should lead to a clear limitation or human review. Several models agreeing does not create new evidence.

Completion: Review the whole assessment before stopping

Completion review checks whether the agreed assessment has been addressed before the system stops, including the work left behind while it followed promising leads.

When no task is runnable, the review looks beyond the task plan. It checks the target map's remaining tests, outstanding vulnerability validation, and access requirements against the applicable methodology and the client's objectives. An assignment may be missing even though the record already contains an unanswered question. Other assignments may exist but be waiting on a prerequisite.

It checks whether important areas remain unexamined, whether a claimed result has evidence, and whether new access revealed something that still needs assessment. Useful remaining work returns to the task pool. Work that requires a person stays visible with its reason.

The run may end because the planned work is addressed, the budget is exhausted, or something prevents further progress. The report should say which happened. Testing that found no weakness, testing that could not run, and an area never examined must remain distinct.

This still cannot guarantee that every vulnerability was found. A professional pentest is limited by its scope, time, access, and methods, as OWASP's reporting guidance makes clear. The defensible claim is that the system can explain what it examined, what the results support, and why it stopped.

Reporting: Build the handoff for a pentester

Reporting turns the assessment record into something a pentester can understand quickly, investigate in detail, and use to continue the work.

When a run ends, the information already exists in its notes, tests, and saved results. It may still be scattered across hundreds of actions. The report should bring that material together, whether the run finished its planned work or stopped with limitations.

I want the pentester to have one review package with several levels of detail.

• The result at a glance. The important confirmed vulnerabilities, their impact and recommended fixes, the agreed scope, the reason the run stopped, and the work that still needs a person.
• A map of the discovered target. The known systems, components, features, and relevant accounts, with the tests and outcomes attached to each. The map should distinguish an area that was discovered from one that was actually assessed.
• The meaningful work. A concise account of discovery, testing, and validation, with the recorded commands or tool actions and the results needed to review each conclusion. Confirmed vulnerabilities need enough supporting detail for a pentester to reproduce them under the approved conditions.
• The remaining questions. Unresolved candidates and tests that were blocked, skipped, or outside the permitted scope. Each should explain why it remains open and what would be needed to resolve it.
• The activity appendix. The full captured activity history and supporting artifacts, linked from the relevant component, test, or finding. The reader can inspect deeper evidence without searching through unrelated logs.

A claim in the summary should lead to its affected component, test, and evidence. The package should carry the context needed to understand the work without collecting separate logs. Large supporting files can remain attachments as long as the links travel with the report.

That package could also support questions through an approved local LLM, with answers linked to the relevant passages or artifacts. A model would still need to read those sources, and missing evidence would still mean a missing answer. Live credentials and unrelated confidential data should not be included in ordinary report exports.

The report saves time when its conclusions are clear and its supporting record is easy to inspect. A shorter summary is useful only if the detail remains available.

Implementation in action (Demo Web)

Here is Agentic Pentest in use. The interesting question is not how many commands an agent can run, but how those actions turn into information a pentester can use.

Conclusion

I started with a practical problem. There was more to examine than the available time allowed, and confidential data had to stay inside our environment. I had limited time to research the approaches, design the architecture, and build the tool. That made the priorities concrete. Give the agent useful work, preserve what it learns, and make its conclusions easy to check.

The encouraging part was how much a small local model could contribute beyond suggesting commands. With focused assignments, relevant guidance, and a shared record, it could carry parts of the investigation forward. That does not make every model equally capable or turn a demonstration into proof of reliability.

The architecture is meant to support different kinds of assessment. The examples here are web-focused. Extending it to another environment means giving it the right methods, access procedures, and tools, then testing whether those pieces work together. A new module earns its place when it helps answer a security question or saves the pentester time.

That is also how I want to judge the project as a whole. A long activity log is not the goal. A useful result tells the pentester what was examined, which weaknesses are supported by evidence, and where to continue. The measures that matter are the quality of that work and the effort it takes to review and finish it.

I would like this project to open a discussion about where agents genuinely help, where supervision matters, and what evidence would earn our trust. The goal is a better starting point for the pentester, with more time left for the work that needs their judgment.