Advanced chatbots and generative AI tools act smart, but they are often dead wrong. Trained to be confident and sycophantic, they can persuasively convince you that the equivalent of white is black. When asked about a personal mental illness concern with a short search phrase, Google’s AI Summary will confidently diagnose you (incorrectly) based on a consensus of five “experts” on Reddit. When you ask it for help with a writing task, it will write from a particular point of view based on the writings of millions of mostly English-speaking Americans.
When a large language model (“LLM”) is trained on a huge amount of content (including Reddit) that its developers can get their hands on, you have a very powerful tool which knows next to nothing about you, your problems, your organization, your community, and for most people in the world, your country and your culture. It lacks context. Unfortunately, when it comes to humans, context is everything.
It’s also a technical issue with AI tools. The less AI knows about a specific task, the less well it does. A general purpose image recognizer might be awesome at telling pictures of cats from dogs, but it is not likely to be great at recognizing malaria parasites in an image of a slide smear of blood. It might write a terrific 1,000-word, eloquent essay for your project report, but it doesn’t know your boss prefers 100 words and bullet points.
The current answer from some leading voices in the AI industry is a familiar one: the problem is defective users. Their AI products are super-powerful, on the brink of taking over the world and eliminating your job. Everybody is hearing this story, and bosses are firing people left and right because they believe these claims. But, if you fail to get those kinds of outcomes from an AI product (like 95% of companies in an MIT study), it’s your fault. You didn’t use it correctly or write the right prompt!
Nonprofit leaders are struggling with this dichotomy today. They’ve heard that this technology is going to change everything, but it seems far from what’s being promised. At least for now.
I believe that this will change in the next five years, but I don’t think that the American generative AI (“genAI”) companies are going to solve this problem for most of the nonprofit sector or for most of humanity. For one thing, the economics are going to be a huge barrier: Silicon Valley’s relentless focus on profits means that most companies cannot afford to care about people or the planet, especially the 90% of humanity that is not their target market. The environmental impact (an externality to the AI industry) is another major cause for concern for much of the social sector (and the general public).
My recommendation to the social sector, and most countries in the world, is to grab control of this technology and make it work for you, your organization, your communities and your countries. Don’t rent it from big tech, work with tech nonprofits to own it. The missing ingredient is context: context is going to turn genAI into an effective tool for advancing all of humanity. The social sector may not have infinite money or people, or the latest chips, but we do have one thing the tech industry doesn’t: the context of what 90% of humanity truly needs!
Layers of Context in AI
Context shows up at multiple layers in modern genAI products based on LLMs. The following is a simplified summary of some of these dynamics:
- The base level of context is determined by what content the LLM is trained on. If your community, language, country, or subject area is not included, don’t expect the LLM to be particularly strong on your topics.
- Next is post-training by the model’s developer. Modern LLMs need considerable fine-tuning and reinforcement learning to function well on the specific tasks and roles they are likely to be asked to perform (like being a conversational chatbot, a software developer, or a math tutor). Post-training includes items like system prompts, which provide another layer of context. These prompts are invisible to the user but constrain outputs (like not answering questions about events happening after the cut-off date for the training data). This is also where guardrails are built into the model (for example, refusing to answer questions about how to commit suicide, or how to build a bioweapon).
At this point, the model typically becomes a product available to customers. The focus is on meeting the broad needs of many customers who are likely to pay for the product. The “frontier labs,” like OpenAI and Anthropic, are in a race to be commercially successful, and they are rapidly improving their products.
However, the products they are building are not for the average person in the world. For straightforward economic reasons, they are focusing on areas of maximum market size. It’s not likely to know what medical supplies are available to a rural Zimbabwean, or know the correct local names for medicines, or speak the languages spoken by most people in Zimbabwe. Of course, this issue of market failure has been true of the tech industry for decades, with rare exceptions like the mobile phone.
Even for their target markets, the genAI products don’t arrive understanding their users. More work needs to be done for them to be fit for most tasks that are meaningfully complex. Any user who has asked a genAI product to perform a task and gotten an unsatisfactory result understands this process. Effective individual users have learned how to iterate on their instructions to get a useful result, like how someone supervising a human intern will usually need to provide more detailed instructions for an intern who lacks knowledge and experience about their organizations. This creates a new set of context-providing exercises to take a general-purpose tool and make it useful:
- User knowledge. It is possible to configure a genAI product like Claude with a user profile (it calls it “memory”). This can include your name, role, articles you’ve written, and recent tasks you’ve done with the tool. For certain kinds of projects, it “jump-starts” the tool without needing to inform it of the same facts repeatedly.
- Tasks. Once you’ve successfully gotten a genAI tool to perform a task, you can ask it to re-perform that task at certain times. For example, I have it go through my email early every morning and highlight the highest priority emails (and it has been instructed that emails from my board chair are always top priority). Or, look at my schedule for the week and draft the outline of my weekly meetings report to my team. In each case, I’ve invested the time to get the task to be performed my way, not a generic way.
- Skills. In a similar way, it is possible to create (and even share) a set of instructions for performing a specific task. For example, a nonprofit staffer could have a skill for writing a grant report that sets forth a step-by-step process to follow each time (for example, create a draft grant report based on the original proposal, the funder’s grant agreement, last year’s grant report, and other documents with recent project updates).
- Retrieval-augmented generation (“RAG”) is another way of adding more context. This builds in a step to instruct the LLM to base its answers on a known body of knowledge, rather than on the Internet. Some of the best current uses of genAI technology in nonprofit programs use this approach. It’s particularly effective when the nonprofit has a large body of quality content, such as Digital Green having 10,000 agricultural advice videos in Indian and African languages, or MomConnect (maternal health helpline in South Africa) having many thousands of questions from expectant mothers and answers written by health educators, or Indian Development Review (a major Indian journal) with 3,000 articles. These organizations have genAI tools which are told to base their answers on the organization’s own content (and generally, to avoid making up an answer when it doesn’t exist in that content).
There are even more ways of injecting context in AI technology like those above, all with a common goal. Take a generic genAI tool that is not that good at performing your tasks, and often lacks the relevant information for doing your task well, and work to make it much more useful for you. By adding all of these layers of context, you may have an effective tool for helping people in need, or the people on your team helping people in need, which is the essence of the social sector’s mission. Unfortunately, this amount of technical effort is well beyond the capacity of most nonprofits today.
Freedom from Big AI
Big AI tends to deliver a monolithic message: they are creating super-powerful genAI technology that is dangerous and will completely upend society (and make the developers and investors stupendously rich). But, as delivered, their products don’t work very well, because they lack context of the lives, experiences and knowledge of most of humanity.
Luckily, the rest of humanity (and the social sector) has that context, and we can choose to use it to benefit the communities we serve, rather than providing it to Big AI (or have it stolen from us). We don’t always need to be at the bleeding edge of the technology, which is also the most expensive and probably has an outsized impact on the environment. It is possible for small, skilled software teams to build smaller and cheaper AI models that perform much better for our contexts and our needs. Much better, much more affordable, much more accessible, and with an environmental impact that is likely to be much more acceptable to communities.
What are the ingredients of such a transformation?
- Open LLM models. The big news of the last year has been the advent of open LLMs which are only a few months behind the latest frontier models, and that are available for free to anyone who wants to use them. This way, organizations own their own technology, their own data, the results, and lower costs.
- Post-training the LLM on our relevant data, data we own or have access to. “We” in this case could be a country, or a consortium of nonprofits working in the same field, or even a single technical nonprofit organization. It’s not hard to imagine a powerful open model being tuned on just data from a given country or field of practice (e.g., public health in a certain region).
- Skills to perform common tasks for our use cases. This could include AI coaches to help people who are new to a role get better at their work.
- RAG based on high-quality content from our (larger) organization or a coalition of organizations in a field.
Different organizations will apply context to choose which of the above approaches they will use to improve their genAI outcomes. These ideas are not new to the business community. In recent months, many businesses (both in the U.S. and in Africa) have shifted their genAI activity to these powerful and much more affordable open models. It’s time for the nonprofit sector to get on board, not wait five or ten years before adopting these trends!
What the Social Sector Needs to Do
My initial recommendations for the sector to pursue initiatives that are more mindful of context should not be controversial. Data is the key to context, and the key to all things AI: especially the data the social sector has, which is not well-captured by commercial companies. Above all else, we have to avoid throwing away useful data while respecting privacy and ethics. That means responsible data governance: the social sector specializes in collecting data from vulnerable people about the things that make them vulnerable. I’ve helped create a basic data governance standard called the Better Deal for Data, which commits nonprofits to use data for the benefit of, and in partnership with, the communities they serve. One important element of the Better Deal is committing to not allow for-profit companies access to confidential data for them to hold, expose and exploit.
Beyond data, we should continue to support and encourage AI capacity building in the nonprofit sector. There is a critical need for a modest number of open source initiatives designed to make tech that can be used by hundreds of thousands of nonprofits without professional software development teams. The bulk of this effort needs to focus on those not-super-technical (or not technical at all) nonprofits that make up 98+% of the sector. Not mindlessly exhorting teams to use AI for everything, or forcing them to use it when it makes no sense (or terrifies them). Like every new technology that has arrived in the computer era, we have to directly link tech use to improving the lives of people. One catchphrase is to focus on no-joy and low-joy tasks. For example, few people go into social work or similar people-oriented fields because they love doing data entry, even though our systems often force them to spend most of their time doing paperwork. Applying AI to streamline data entry without sacrificing quality, and you’ll probably find staff eager to adopt new technology.
We will need to build genAI solutions based on open models. To paraphrase the car metaphors going around in industry, why rent a Ferrari (paying the U.S. frontier labs) when you can own your own Toyota (run your own models)? Why pay through the nose for the latest and most expensive technology that needs the latest expensive chips and huge data centers, when you can run better models on your own laptops? Large non-AI companies are already making this switch for these exact reasons: nonprofits should embrace this approach, safeguarding our data, our budgets and our missions!
However, building on open models has to be tempered with professional risk assessment. It is probably best to have skilled nonprofit tech teams pick open models which come from trustworthy sources to minimize the chances of using a model with malware inside.
Beyond investing in the capacity of rank-and-file nonprofit team members, we also need to invest in the growing number of technologists capable of developing quality AI solutions. While this includes traditional software developers, always in short supply in the nonprofit sector, AI has enlarged the pool of nonprofit professionals who can build technology. People who are good with data can become talented AI developers, once experienced software engineers build the foundational LLM technology. They just need to know enough to recognize when the AI has gone off the rails, which will still happen regularly, and get it back on track!
I’m looking forward to a new wave of nonprofits building tech infrastructure, platforms and tools to share responsible AI with the rest of the social sector. A handful of expert technologists who care more about social impact than getting rich can build open tools that substitute for the major commercial platforms. European-based Current AI is already leading the way with their “AI Potluck” project, which lists hundreds of open projects, and has started prototyping their own ethically-created chatbot, Alpha.
By sharing common open source infrastructure, tools, and ethically sourced datasets focused on social impact users, we can create an environment where regular nonprofits and ordinary communities can take a better AI use path. Hundreds or thousands of these infrastructure projects will create the foundation for hundreds of thousands of social good AI applications.
What Could This Mean?
Imagine if…
- We made these investments in common AI infrastructure for the social sector
- Nonprofits had access to the data they need for AI, while ensuring the benefits of data flow back to the communities where it came from, and protecting the privacy of individuals
- We used that data to build genAI tools optimized for social impact
- Every country had the processing power it needed at a locally affordable price
If we made these plans real, the power of the genAI revolution would be available to individuals, nonprofits, local businesses and local governments. I believe this could have a dramatic impact on many of society’s most pressing problems. We won’t know which projects will work until we try and adapt to what we learn. In nonprofit fields I know more about, I think of the following as just a sample of projects which will be worth trying in the next three to five years:
- Mental health coaching chatbots for volunteers and paraprofessionals, which know local languages and cultures, based on best practices (i.e., not sycophantic), and focus on enabling these front-line helpers to support the majority of people with minor mental illness. This could be huge in countries lacking enough mental health professionals.
- Tools that make accessible versions of inaccessible information for students with disabilities. This could go well beyond making PDFs and pictures accessible (something Benetech is currently working on): it’s not hard to imagine a tool which knows exactly how to communicate information to a specific person with a cognitive disability.
- Role-playing coaches for every sort of situation you can imagine, from training crisis hotline counselors (these already exist, but are beyond the means of most hotlines), to getting someone ready to interview for a job, to helping someone on the autism spectrum better deal with a world not adapted to them.
- Tools which know how to repair every piece of medical equipment used in your country, which understands what spare parts are locally available to a medical technician (a dream of Nithya Ramanathan and the Nexleaf Analytics team).
Cheap AI chips from industry that work by themselves (are not dependent on connections to the cloud, working instead at the “edges”) that will know about the physical world are probably arriving near the end of my five-year time horizon, but could revolutionize agriculture, water management and land quality globally. It isn’t hard to imagine ideas like these for any part of the nonprofit sector where limited access to knowledge and expertise has a direct impact on social outcomes.
Conclusion
I’ve spent the last three years talking most nonprofit leaders out of getting carried away with AI applications, especially in program applications. My catchphrase is: “We should not be inflicting bad AI on the people we serve!” In my Nonprofit AI Treasure Map, I advised leaders to test out AI solutions and if they didn’t work, to wait a year. And, to avoid being sucked into building advanced AI technology solutions when they lacked tech people on their teams.
A couple of years have gone by, and the AI future for nonprofits has gradually come into focus. What AI solutions actually work, and which ones are still far from ready, are becoming topics of common knowledge around our sector. We have peer success stories to emulate, and nonprofit leaders trust their peers above any other authority. The advent of powerful open models has created an opening for the nonprofit sector to take control over their AI future. Beyond that, we won’t be financially supporting a segment of the for-profit AI industry that seems to be bent on recklessly moving forward without much regard for its impact on humanity. The social sector emphatically is pro-human, not pro-AI.
More and more of the foundations funding social innovation have been wondering what the genAI revolution means for their grantees and their grantmaking. The initial investments have created early momentum, and it’s time to build on the next phase and reduce the sector’s dependence on a handful of AI companies which are distant from the communities the social sector exists to serve.
At the heart of this AI-for-Good future is the power of context: the power of data applied to benefit the communities we serve. I’m looking forward to sharing much more in-depth plans in the coming months on that detailed roadmap, highlighting the people and groups I think are leading the way. Funders need to fund these clued-in leaders and teams, and nonprofit leaders need to build alliances with them. An AI future increasingly owned and controlled by the social sector should deliver a much better AI future for the world.