Originally developed in April 2025 | Project retrospective updated October 2026
Artificial intelligence is changing how we approach cybersecurity, information retrieval, and regulatory compliance. But one of the challenges I find most interesting is figuring out how to build AI systems that can work with specialized technical knowledge without sacrificing control, security, or transparency.
That challenge inspired one of my earlier open-source projects: RAG AI Specialist, an experimental Python-based framework exploring how retrieval-augmented generation (RAG) could assist with NIST SP 800-171 cybersecurity assessments.
When I started developing this project in April 2025, my goal was to explore a simple question:
What if a cybersecurity assessor could work alongside an AI specialist capable of retrieving relevant security requirements, interpreting technical evidence, and helping organize compliance findings?
Rather than relying entirely on a general-purpose chatbot, I wanted to investigate an architecture that could eventually support specialized compliance knowledge, modular auditing tools, and controlled information retrieval.
The resulting repository became a starting point for experimenting with those ideas, alongside broader research into offline AI and resilient systems.
Explore the project on GitHub:
https://github.com/CptNope/Rag-AI-Specialist
What Is Retrieval-Augmented Generation?
Retrieval-Augmented Generation, commonly called RAG, is an approach that combines language models with external information retrieval.
Traditional language models generate responses using patterns learned during training. While incredibly capable, they may lack access to current organizational documentation, specialized internal procedures, or the particular version of a technical standard relevant to a task.
RAG introduces an additional step.
Before generating an answer, an application can retrieve relevant information from an approved knowledge source and provide that context to the model.
For a cybersecurity auditing assistant, those sources might include:
- NIST SP 800-171 security requirements
- Official assessment guidance
- System Security Plans (SSPs)
- Policies and operating procedures
- Approved system configuration documentation
- Audit findings and remediation records
The result is the potential for an assistant that can work with a defined body of evidence rather than relying solely on general model knowledge.
However, RAG does not automatically guarantee accuracy, confidentiality, or regulatory compliance. Its effectiveness depends on the underlying retrieval design, access controls, information quality, model behavior, and human oversight.
That distinction is especially important in cybersecurity.
Why I Focused on NIST SP 800-171
NIST Special Publication 800-171 addresses the protection of Controlled Unclassified Information (CUI) in nonfederal systems and organizations.
It is particularly important to organizations working with federal agencies and within the defense industrial base.
Compliance assessments can involve reviewing security policies, collecting evidence, interviewing personnel, inspecting technical configurations, and identifying gaps between documented procedures and actual implementation.
These tasks require considerable expertise, and much of the process involves searching, cross-referencing, and interpreting complex information.
I saw an opportunity to investigate whether AI could support the research and organizational portions of that workflow.
An effective AI-assisted assessment tool could potentially help an assessor locate applicable requirements, identify documentation that needs further review, and prepare structured questions for human investigation.
Importantly, the objective would not be to replace a qualified assessor.
The objective would be to reduce repetitive research and documentation work while keeping the assessor responsible for evidence validation and final conclusions.
A Note on NIST Revisions
Since the project’s original development, it is particularly important to distinguish between versions of the NIST standard.
NIST published SP 800-171 Revision 3 in May 2024, along with its companion assessment publication, SP 800-171A Revision 3.
An AI-assisted compliance system must identify the specific revision and contractual requirements applicable to each assessment. It cannot assume that one collection of controls is appropriate for every organization or assessment program.
For current reference materials, visit the official NIST publication:
The RAG AI Specialist Project
The RAG AI Specialist repository was designed as an experimental foundation for a modular cybersecurity assistant.
The initial architecture explored several connected ideas:
1. Structured Compliance Knowledge
One of the first building blocks was a Python component for retrieving structured security-control information.
The repository includes a NISTControls class designed to load control definitions from YAML and make them accessible by control family and identifier.
The approach separates the compliance reference data from the application logic.
In a more mature system, this could make it easier to update standards, compare revisions, and organize assessment content.
The current public repository contains this loader, but does not include the complete control dataset expected by its default file path.
2. Modular Plugin Architecture
Cybersecurity assessments often require information from numerous operating systems, applications, and infrastructure components.
A Linux server, Windows workstation, cloud environment, and network appliance can all require different collection methods.
Rather than building every capability into one monolithic application, I explored a plugin architecture.
The repository includes a Python plugin manager that discovers classes derived from a shared plugin base.
The concept would allow specialized modules to be introduced for individual audit functions.
Potential future examples include configuration reviews, access-control checks, and evidence collection.
The architecture is present as an early implementation, not a completed catalog of production auditing plugins.
3. Retrieval and Knowledge Management
The original vision extended beyond simply displaying a list of NIST requirements.
I wanted to investigate how an AI agent might retrieve relevant policies, assessment procedures, and approved supporting documents while preserving the context needed for an assessor to evaluate them.
A complete implementation would require document ingestion, indexing, retrieval, source references, and carefully enforced information-access boundaries.
This is where the RAG approach becomes particularly interesting.
Instead of asking a general-purpose AI to explain a security requirement from memory, an assessor could ask a system to locate the applicable requirement, show its source, and identify which organizational documents might be relevant.
The repository contains retrieval-related configuration and architectural material, but the core retrieval adapter remains a placeholder.
4. Communication Between Specialized Agents
Another area I explored was whether distributed agents could eventually cooperate on specialized tasks.
The project includes network and peer-management modules, along with MQTT-related dependencies.
The broader concept was to support communication between independent components, potentially allowing one agent to specialize in configuration data while another handled document retrieval or reporting.
A distributed architecture also introduces important concerns: authentication, authorization, message integrity, transport security, and data leakage.
Those concerns would need to be addressed before such a system could be trusted with sensitive information.
The public repository represents an early exploration of that architecture, rather than a production-ready multi-agent network.
5. Controlled Handling of Sensitive Information
One of my central interests was designing AI workflows that do not unnecessarily expose sensitive organizational information.
In the original proposal, I explored concepts such as sanitized retrieval datasets, tamper-evident audit histories, and controlled sharing of knowledge between agents.
Those remain valuable design goals, but they should not be confused with implemented security guarantees.
For a real deployment involving CUI, I would expect security architecture and authorization decisions to be established before introducing sensitive assessment evidence into an AI workflow.
Encryption, identity management, role-based permissions, data minimization, logging, retention policies, and system boundaries would all be part of that evaluation.
From Compliance AI to Local-First Systems
An interesting part of revisiting this repository is seeing how some of my research interests began to overlap.
Alongside the original NIST auditing assistant, I started exploring offline knowledge access, field-deployable AI, resilient communications, and decentralized computing.
The repository now includes materials associated with the SolarSentinel concept, including plans and examples involving Raspberry Pi systems, ESP32 devices, LoRa communication, offline knowledge bases, progressive web applications, and mission-based training.
Although the compliance assistant and field communications network address different problems, they share an underlying architectural question:
How can we make useful information available to the right people while maintaining greater control over where data lives and how it moves?
That question applies to cybersecurity, emergency communications, industrial environments, and private organizational AI deployments.
It also explains why I continue to find local AI and retrieval-based systems compelling.
What the Open-Source Repository Includes Today
As of this retrospective, the repository is best described as a collection of early-stage implementations, configuration experiments, and architectural documentation.
It contains:
- Python modules for structured compliance-control retrieval and plugin discovery
- Configuration files for agent behavior and memory policies
- Placeholder components for retrieval and network communication
- Docker-related files and Python dependency definitions
- Early dashboard code and templates
- Documentation for local AI, RAG integration, and distributed field-system ideas
- Hardware, communication, and scenario-planning examples associated with SolarSentinel
Some components are functional code foundations. Others are examples, placeholders, or proposals for future development.
The repository should not be treated as a completed NIST assessment platform, certified compliance solution, or secure system for processing CUI.
I believe distinguishing experimental architecture from validated production capabilities is especially important when discussing cybersecurity tools.
Why AI-Assisted Compliance Still Matters
The broader problem that inspired this project remains relevant.
Organizations continue to face significant challenges when managing complex cybersecurity requirements.
Security frameworks evolve. Infrastructure changes. Documentation becomes outdated. And assessment teams must connect technical evidence with the requirements they are evaluating.
AI can offer meaningful assistance when applied to the appropriate tasks.
For example, I see strong potential in systems that help security professionals research controls, organize evidence, search internal documentation, prepare assessment questions, and identify areas that need closer investigation.
But reliability cannot be an afterthought.
A system that invents compliance conclusions, overlooks missing evidence, or retrieves information beyond a user’s authorization could create more risk than value.
That is why I believe future AI-assisted compliance platforms need to be designed around four principles:
Grounding: Answers should be connected to authoritative, versioned source material.
Transparency: Users should be able to inspect the supporting documentation and understand the limits of the answer.
Security: Sensitive evidence should be protected throughout ingestion, retrieval, processing, and storage.
Human accountability: AI-generated findings should be reviewed by qualified professionals before they are used in formal assessments.
These principles apply far beyond NIST SP 800-171.
They are relevant to almost any organization considering AI systems that interact with regulated or confidential data.
What I Would Improve If I Rebuilt It Today
Looking back at the April 2025 prototype, there are several directions I would prioritize in a modern implementation.
First, I would establish a clean separation between the compliance-assessment application and the off-grid field-system experiments.
Second, I would build a versioned standards repository tied directly to official NIST publications and assessment guidance.
Third, I would develop a complete retrieval pipeline that supports approved document ingestion, source citations, permission-aware retrieval, and repeatable testing.
Fourth, I would introduce an evidence-management workflow in which every proposed finding is tied to specific source material and requires human verification.
Finally, I would prioritize threat modeling, automated testing, and deployment architecture before treating any implementation as suitable for organizational use.
For local AI experiments, tools such as Ollama could be useful development options, but running a model locally does not by itself make a system compliant or secure.
In other words, I would approach a modern version as a carefully bounded compliance research assistant first, then expand its functionality as each component is validated.
Explore the RAG AI Specialist on GitHub
I continue to publish technical experiments because I believe sharing early architectures is useful, even when projects are incomplete or evolve in unexpected directions.
RAG AI Specialist represents an early investigation into combining cybersecurity expertise with modular AI-assisted information retrieval.
It also documents ideas that continue to influence how I think about local AI, retrieval architectures, and secure automation.
If you’re a developer, cybersecurity professional, or AI enthusiast interested in those topics, you’re welcome to explore the repository.
GitHub Repository:
https://github.com/CptNope/Rag-AI-Specialist
You can review the source, examine the architecture, and contribute suggestions or improvements.
Final Thoughts
The most exciting applications of artificial intelligence are not necessarily the ones that attempt to automate everything.
Sometimes the most valuable application is a tool that helps a knowledgeable professional find the right information, investigate a problem more efficiently, and make better-informed decisions.
That was the idea behind RAG AI Specialist when I started it in 2025.
And even as the tooling evolves, I believe that principle remains worth pursuing.
AI should strengthen cybersecurity expertise, not substitute for the judgment, evidence, and accountability that effective cybersecurity requires.
About the Author
I’m Jeremy Anderson, a developer and cybersecurity-focused technology consultant specializing in custom web applications, artificial intelligence, automation, and security-conscious systems architecture.
Through Jeremy Anderson Tech, I explore practical ways to combine software engineering, AI, and cybersecurity to solve real-world problems.