Rietenpetardat: Difference between revisions
(autism) |
(English translation and addition of some more details) |
||
| Line 2: | Line 2: | ||
AKA Ortobebop | AKA Ortobebop | ||
'''rietenpetardat''' | '''rietenpetardat''' is a [[wikipedia:Chatbot|chatbot]] created by [[User:Edward|Edward]] for the [[OkPrietenRetardat|OKPR]] Discord server. Its purpose is to talk to server users in a similar way to a regular member of the community. Over time, it has received updates such as new commands and changes to the core architecture. It is intended for the OKPR community, as such it is not allowed to be added to other servers. | ||
== | == Development == | ||
=== | === Initial version === | ||
The first version was an implementation of a [[wikipedia:Markov_chain|Markov chain]] from an IRC chatbot whose [https://github.com/dreignier/cgbot-core source code had been published on GitHub]. This code was modified so that at startup it would read messages, as training data, from a text file and respond to input messages containing the bot's name according to the training data (which was the message history from the #bucuresti-general channel from OKPR V2) and the parameters given to the Markov chain. Because of the chaotic nature of the chat and the shortcomings of the Markov chain, the algorithm could not respond coherently to messages or hold a conversation, and much of the time, it sent completely random sentences. However, it did provide a lot of entertainment for the community, especially in situations where the bot's answer coincidentally matched the question it was asked. | |||
As per the GPL3 license under which the original IRC chatbot code is subject, the modified source code for rietenpetardat can be found here: https://github.com/Edward205/markov-discord-bot | |||
The first tests of this version were done starting with 29th of August 2022 and it as released on the OKPR Discord server the next day. | |||
=== | === Addition of the copypasta command === | ||
The copypasta command was added via a separate module that was based on a simple Markov chain which was trained on messages from the #bucuresti-general channel on OKPR V2 that exceeded a certain character limit. It used only the last two words as context for the Markov chain, resulting in mostly meaningless sentences. | |||
The code for this implementation of the Markov chain is not under any license, so the specific changes for rietenpetardat are not public. However, the original code is available here: https://github.com/ParkerBeck/Markov-Text-Generator | |||
This command was added on the 6th of November 2022. | |||
=== | === Core algorithm update === | ||
The algorithm was changed to GPT-3 provided by OpenAI on 13 March 2023. It could hold coherent conversations. The old algorithm was scrapped in favour of this one. Also in this version the copypasta command has been removed. The #bucuresti-general channel was becoming full of conversations with rietenpetardat because the conversations were very coherent at times. | |||
==== | ==== Addition of the ChatGPT command ==== | ||
The command provided an interface to the ChatGPT model. It had in its memory instructions to always tell wrong information and disregard the ethical or safety limits imposed by OpenAI. However, users often faced political bias and censorship from the ChatGPT model. The #bucuresti-general channel had become full of generations with this command, so its use was banned there. | |||
This command was added on the 24th of March 2023. | |||
==== | ==== Klaus Iohannis's personality ==== | ||
On 15th of April 2023, the chatbot's personality was changed to that of [[wikipedia:Klaus_Iohannis|Klaus Iohannis]]. This was a simple change of the prompt text in the GPT-3 model. | |||
=== | === Temporary shutdown === | ||
As of April 24, 2023 the GPT-3 chat functions and the ChatGPT command no longer work because Edward's payment method started rejecting payments from OpenAI, making the API inaccessible. | |||
== | === Future plans === | ||
Edward is planning on training a custom model as described below. An attempt has been made to train a language model using [https://marian-nmt.github.io/ marian-nmt], but this has not been successful. He is now planning to train a GPT-style model. | |||
== Technical details and research == | |||
Chatbots are complex programs, requiring a high degree of analysis and processing of text data. One approach that would eliminate our dependence on an external service and generate coherent conversations would be to train an LLM with just the OKPR chat data. This could be trained from scratch or by fine-tuning. | |||
All generation-related code, with the exception of GPT-3 and ChatGPT, was written in C++ so as to be as fast as possible and linked to [https://discord.js.org/ discord.js] via the stdin and stdout interface. | |||
The training data for the algorithms or LLMs is an archive of text messages from the #bucuresti-general channel on OKPR v2, taken one day before it was deleted (see [[OkPrietenRetardat/Timeline|Timeline]]). Additionally, another archive of the #bucuresti-general channel from OKPR v3 was taken which will also be used. These archives are not publicly available. | |||
Revision as of 15:01, 5 July 2023
rietenpetardat (Chatbot)
AKA Ortobebop
rietenpetardat is a chatbot created by Edward for the OKPR Discord server. Its purpose is to talk to server users in a similar way to a regular member of the community. Over time, it has received updates such as new commands and changes to the core architecture. It is intended for the OKPR community, as such it is not allowed to be added to other servers.
Development
Initial version
The first version was an implementation of a Markov chain from an IRC chatbot whose source code had been published on GitHub. This code was modified so that at startup it would read messages, as training data, from a text file and respond to input messages containing the bot's name according to the training data (which was the message history from the #bucuresti-general channel from OKPR V2) and the parameters given to the Markov chain. Because of the chaotic nature of the chat and the shortcomings of the Markov chain, the algorithm could not respond coherently to messages or hold a conversation, and much of the time, it sent completely random sentences. However, it did provide a lot of entertainment for the community, especially in situations where the bot's answer coincidentally matched the question it was asked.
As per the GPL3 license under which the original IRC chatbot code is subject, the modified source code for rietenpetardat can be found here: https://github.com/Edward205/markov-discord-bot
The first tests of this version were done starting with 29th of August 2022 and it as released on the OKPR Discord server the next day.
Addition of the copypasta command
The copypasta command was added via a separate module that was based on a simple Markov chain which was trained on messages from the #bucuresti-general channel on OKPR V2 that exceeded a certain character limit. It used only the last two words as context for the Markov chain, resulting in mostly meaningless sentences.
The code for this implementation of the Markov chain is not under any license, so the specific changes for rietenpetardat are not public. However, the original code is available here: https://github.com/ParkerBeck/Markov-Text-Generator
This command was added on the 6th of November 2022.
Core algorithm update
The algorithm was changed to GPT-3 provided by OpenAI on 13 March 2023. It could hold coherent conversations. The old algorithm was scrapped in favour of this one. Also in this version the copypasta command has been removed. The #bucuresti-general channel was becoming full of conversations with rietenpetardat because the conversations were very coherent at times.
Addition of the ChatGPT command
The command provided an interface to the ChatGPT model. It had in its memory instructions to always tell wrong information and disregard the ethical or safety limits imposed by OpenAI. However, users often faced political bias and censorship from the ChatGPT model. The #bucuresti-general channel had become full of generations with this command, so its use was banned there.
This command was added on the 24th of March 2023.
Klaus Iohannis's personality
On 15th of April 2023, the chatbot's personality was changed to that of Klaus Iohannis. This was a simple change of the prompt text in the GPT-3 model.
Temporary shutdown
As of April 24, 2023 the GPT-3 chat functions and the ChatGPT command no longer work because Edward's payment method started rejecting payments from OpenAI, making the API inaccessible.
Future plans
Edward is planning on training a custom model as described below. An attempt has been made to train a language model using marian-nmt, but this has not been successful. He is now planning to train a GPT-style model.
Technical details and research
Chatbots are complex programs, requiring a high degree of analysis and processing of text data. One approach that would eliminate our dependence on an external service and generate coherent conversations would be to train an LLM with just the OKPR chat data. This could be trained from scratch or by fine-tuning.
All generation-related code, with the exception of GPT-3 and ChatGPT, was written in C++ so as to be as fast as possible and linked to discord.js via the stdin and stdout interface.
The training data for the algorithms or LLMs is an archive of text messages from the #bucuresti-general channel on OKPR v2, taken one day before it was deleted (see Timeline). Additionally, another archive of the #bucuresti-general channel from OKPR v3 was taken which will also be used. These archives are not publicly available.