Safer Prompting for Executives in Non-profits and Government

Chatbots like ChatGPT, Claude, Gemini, and Co-Pilot (also known as Large Language Models, or LLMs) can speed up routine work, help staff reframe problems, and generate drafts that save valuable time. But as many mission-driven leaders know, the stakes are higher for non-profits and government agencies. We hold sensitive data, serve vulnerable populations, and make decisions that affect communities. That means we need to approach prompting—the way we ask AI for help—with extra care.

This post highlights three areas leaders should understand:

1.        Risks of prompting with sensitive data

2.        Risks of bias in prompts and outputs

3.        Tips for writing safe, effective prompts

1. The Risks of Sensitive Data

Residents, patients, students, participants, staff, and community members trust mission-driven organizations with their sensitive data: donor records, addresses, health information, income data, identity documentation, and records of their participation in sometimes-stigmatized programs.

If staff paste this information directly into an AI tool, it may be stored on someone else’s servers, reused to train the models, used to serve you ads, or, in very rare cases, even leaked in another user’s output. Some chatbots were even putting entire chats up on the public internet, available to find through a web search, when users tried to share chats with their friends or colleagues.

Even if the big U.S.-based models encrypt data in transit and storage, leaders can’t ignore the risks:

• Privacy: Do community members expect their information to be shared with a third-party tech company? (Probably not.)

• Security: Even encrypted data is vulnerable if policies or safeguards change.

• Leakage of strategy: Even if personal data is safe, ideas and strategies can slip out to competitors if prompts are reused or analyzed.

Executive takeaway: Set a clear organizational policy about when to use and not use LLMs. Anonymize data in prompts, break up prompts into parts, or use protective prompting when staff need AI support with sensitive data. And for higher stakes tasks—like performance reviews, legal communications, or high-stakes strategy—consider “scaffolding”: using AI only for support tasks (e.g. spotting gaps, polishing tone) while keeping final outputs entirely human-authored.

2. The Risks of Bias

LLM outputs can include bias.

From training data: LLMs are trained on (approximately) the entire public internet. The internet, as you surely have experienced, is not all sunshine and roses. Although most model-makers try to weed and polish out the most egregious content, subtle bias can slip through.

LLMs don’t have moral reasoning or imagination. If text in the training data echoes stereotypes, gives different advice to people based on their demographic information, or refers to misinformation, it can lead the LLM to perpetuate prejudice.

In the prompt: The way staff phrase a question can reinforce stereotypes or assumptions. For example,“Why do young people resist financial literacy workshops?” assumes that young people do indeed resist workshops more than older people. It also might be assuming that financial literacy is the most important barrier to economic mobility for the population, rather than structural and systemic issues.

Complicating matters, LLMs are designed to be useful to users in the short term. They tend to confirm whatever framing the user provides and even flatter users’ ideas and traits (sometimes rising to the level of sycophancy). That makes them powerful amplifiers of human bias. When the user is not aware of their own biases, confirmation bias makes it very hard for us to see what’s happening and fix it.

Executive takeaway: Encourage staff to treat AI responses as draft input, not authoritative output. Build in review processes that include diverse perspectives and ask the AI explicitly to challenge assumptions or offer critical perspectives.

3. Tips for Writing Safe, Effective Prompts

Good prompts reduce risk and increase the usefulness of AI. A simple template works well for everyday tasks:

Set a role: “You are a program officer at a non-profit serving low-income families in Milwaukee…” This helps the LLM quickly understand some of the basic context it needs to get you what you expect.

Give deep context: Include mission statements, past reports, email threads, vetted sources, and anything else an intern would be referring to if writing it on their first day on the job. The LLM uses this to make tailored, useful content: if your outputs are generic, try more context.

State the task clearly: “Draft three possible introductory paragraphs.” “Compare and contrast these two programs.” “Tell me how I can improve my draft.”

Request a format: Not just “a brief,” but “a 3-5 page brief for an audience of City Council members designed to persuade” or “an internal brief for managers designed to inform.” You can request bullet points, tables, executive summaries, and more.

For more complex work, consider creating structured workflows, context checklists, and custom instructions, like “Please list all assumptions that go into my prompt and your output.” More on complex work in a future post :)

Quick Tip: More context is better. If you aren’t sure what context you might have that could help, you can ask “Before you draft the output, please ask me questions to get all the information you need. Please ask those questions one at a time and incorporate the information I give you as you go.”

Executive takeaway: Safe prompting keeps human judgment in the driver’s seat.

By setting clear policies, understanding risks, and training staff in safe prompting, mission-driven organizations can unlock AI’s benefits without undermining community trust.

I asked an LLM to repurpose a chapter of my book draft to write this :)