The benefits of open-source AI: open-source affords user privacy

In this third post about the benefits of open-source models, we discuss how designing models with open-source in mind benefits the privacy of end users. We investigate privacy concerns with current AI models, and dig into the communities within FOSS which hold privacy to the highest standards.

How open-source AI safeguards privacy

Open-source safeguards privacy, most obviously by allowing users to create local copies of their software. By running AI models locally, users can ensure that models do not pass on information about their conversations to third parties, and it allows them to fully customize their conversation flows. Conversely, proprietary models can offer no easy verifiable guarantee that data stays private between users and the model. What's worse, most private entities seem happy to gobble up as much chat information as they can, as it allows them to design more capable instruction-tuned models. Most egregiously, ChatGPT is mandated to maintain all non-temporary conversations indefinitely. Some end users seem eager to share even the most sensitive data with these models. But from a data privacy perspective, the practices of proprietary model owners can best be described as nightmarish.

For tips on how end users can interact with proprietary models in way which preserves privacy as much as possible, see also Mozilla's excellent overview at Privacy Not Included.

The FOSS community and privacy as an inalienable right

The concept of open-source arises from the hacker community, where privacy is held as a fundamental and inalienable right of users. This has led to many open-source AI initiatives which seek to safeguard user privacy in fundamental ways. Various projects exist which seek to provide users with a privacy-first LLM interaction experience, and when data is shared it is usually done with user consent in mind. In this way, the FOSS community sets a great example for how privacy-preserving AI should be done.

Conclusion

Open-source models offer users privacy in a way which closed-source models often cannot and fundamentally will not do. As such, the privacy-conscious user benefits greatly from open-source initiatives. The FOSS model-developing community represents a great model of how to implement models in a privacy-concious manner, and its projects provide a blueprint for privacy-centric design in the AI space more broadly.