Polish AI

1. I've been pondering a question for a while: do we need a large Polish language model?

2. ChatGPT handles the Polish language very well. It can both answer questions in Polish and translate excellently between Polish and English. It's not perfect, but it does so well that you can use it in Polish only.

3. We have access to good AI products and the APIs on which they are built. So, we can create solutions localized for our market without major obstacles.

• Recently, Grupa Morizon-Gratka implemented a conversational interface for real estate search, based on the ChatGPT API. It works without any problem in Polish.

4. Since we have GPT-4 and others, why should we create a National LLM (Large Language Model)?

5. Knowledge, broadly understood, is essentially international. Despite the fact that humanity is divided into nations, knowledge, especially utilitarian, quickly becomes international and available in many languages.

6. A large model, by definition, has to be large. This means not only a large number of parameters, but also a large amount of training data. There is less data in Polish than in English, and significantly less than in all other (non-Polish) languages combined. This means that a large language model based only on Polish data will have to be poorer and perform worse.

7. A computer generating language is politically problematic. A company or institution controlling a large Polish language model would have to make decisions in ideological matters, how the model responds to sensitive questions. If an institution in Palo Alto does this, we de facto hand these decisions over to people who adhere to values popular among experts living in Palo Alto. This way, we avoid local ideologization of generative AI. Depending on our views, this may be positive or negative for us, but it certainly removes us from responsibility for these decisions.

8. Creating large language models is costly and requires specialized and unique skills. Why duplicate this effort and cost? Centralization seems like a more cost-effective solution.

9. How to profit from this? Will Polish companies choose the Polish LLM when there are good American solutions? What use case scenarios require a Polish LLM?

10. However, I think there are reasons for a national large language model to exist.

11. Poland has a rich literary, scientific, and creative culture. I don't think GPT-4 was trained on all the content available in Polish from our over 1000 years of tradition. If we don't have a Polish model, no one will care to preserve and immortalize this culture in this new way. No one in Palo Alto will care for the quality and depth of preserving and representing our culture, at the level we would wish. Not out of laziness or ill will, just because it's not necessary for their business goals.

• We would like the LLM to write better in Polish, know more idioms, slang, references to Polish popular culture.
• We would like the LLM to know more Poles, niche celebrities, forgotten scientists, and authors.

12. Culture and language intertwine with geography, including hyper-local geography. If we want the language model to accurately answer a question about the features and charm of a certain street, or a specific tenement, in a small town in Poland, we must train it on niche data but also use techniques like Reinforcement Learning from Human Feedback, to guarantee high-quality answers that are not hallucinations.

• The same applies to other geographical aspects and nuances of our country and surroundings. Understanding soil humidity, technical data of bridges, hydrology, etc.

13. State institutions should not only use LLMs hosted on external servers, even if they are servers of geopolitical allies. If the future unfolds as I think it will, LLMs will become an inherent part of every software. If we want an efficiently functioning state and high-quality digital services at the citizen-state interface, we want them to use AI and LLMs. However, I can't imagine that an assistant on the podatki.gov.pl (a platform for tax payers) website would send all my questions and data to a US company.

14. We want uncensored models. Using the ChatGPT API means using a limited version of the model, which has been censored so that we "don't hurt ourselves". That's very nice, but adult people sometimes need to do risky things. For example, we may want to support our military planning with good, detailed AI tips, and we don't necessarily want software for military analysts to be based on systems controlled by foreign companies. It is known that LLMs are already used in this way and will become a standard part of military operation planning in future conflicts. Relying on API calls to ChatGPT is a dead end in this use scenario.

15. Language also implies law. Poles would gladly use tools that understand Polish legal nuances, rulings, procedures, and taxes. This is also hard to achieve without very precise tuning and testing. At the same time, we know that minor errors or changes in law (the famous "or magazines") can have a major impact on the functioning of people and businesses. Therefore, we don't want lawyers and politicians (often politicians are also lawyers) to use models that hallucinate about the nuances of Polish law.

16. I think a national LLM makes sense. It should be created. Technically, this could simply be very precise and methodical tuning of an existing open source model to a "national dataset" and RLHF (Reinforcement Learning from Human Feedback) from Polish experts (or students) in various fields.

17. I think it's an investment that might not make sense in the private market, but could emerge at the intersection of the state treasury, business, and universities. The establishment of an AI Lab, whose mission would be to build a Polish LLM and its development, is a relatively low cost in relation to the potential impact on the lives of Polish citizens.

18. Am I hallucinating?