Automating your newsletters in your voice

In my last issue, I imitated a chatroom by training data from a massive Facebook Messenger chat, used Codeium to create a script to merge the data, and then loaded that into Oobabooga to create a LoRA model with Facebook’s Llama. The chat brought out everyone’s personalities though had messy formatting and was very random, so I thought of a new experiment for today:

What if I took all my newsletters from RocketFuel and trained a model off of the 40 issues? In these newsletters, I cover what happened in crypto markets and where I think things are going. In theory the model would be analytical but also be forward looking and affirmative. What if I inputted that into a LoRA model and asked it questions, would it sound like me? Well let’s find out!

I’m going to skip the step by step this time because it will be similar to my last substack (be sure to subscribe if you want to absorb my learnings!), but for reference you can see my settings here:

The total text file ended being 2MB, and only took 35 minutes to process. That made me skeptical that it would affect the responses at all. However, I was wrong, it was deeply rooted in content from the newsletters. Let’s start testing this out:

If I ask it just normal questions, it seemed normal…until I started talking about investments:

Now, without the LoRA loaded in, it has no crypto bias, so here is what it said:

Ok let’s ask it a direct investment question:

I asked what I should invest in, and it chillingly said something I would probably have said even today. Which is accurate because BTC and ETH are probably also mentioned the most in the newsletters. For reference this is what the Llama model says without the model:

As you can see, no reference to crypto or actually any specifics. GPT4 gave a very long answer that was about diversification, risk, and random investment strategies.

What if I asked which coins to invest in?

As you can see it actually gave the exact format I have in my newsletters on coins that I analyze. Strangely, the numbers above are made up but very close to numbers today, I did a search for some of the market caps and it didn’t find them in the dataset. Here’s what Llama and ChatGPT said:

What if I asked it for price targets? Would it have any idea? Let’s find out:

Again, this chillingly sounds like my voice and even the format ending “In summary” is something I do in the newsletter. It even gives pretty sound advice with today’s prices which is very coincidental, HOWEVER, it went off the rails to talk about bots which I have never really talked about in the newsletters.

With the other models it didn’t quite work:

And GPT4:

Again a long winded answer avoiding the question, which to be fair is acceptable since the data ends in 2021.

Up until this point, we have been testing on existing content, what if we asked it to write new content in my voice?

This type of prompt isn’t good enough for me to publish, so let us be more specific to today’s news:

How about if we ask it to expand on points?

Even though some of the numbers are off, it actually sounds like it was written by me, though some of it may actually have been picked out from actual content. I needed another approach other than forcing it to keep writing and retrieving outdated data.

So far this experiment proves that you cannot automate a financial newsletter based on previous data without a lot of modifications and up to date additions. But, what if someone else wrote it, and we tried converting it to sound like me? First I’ll get a paragraph from ChatGPT:

I took this prompt, and asked my model to rewrite it, after some initial gibberish, it actually gave content I would have written (even if the prices were wrong):

Llama alone would just rewrite the paragraph, which was extremely similar to ChatGPT:

So in theory, someone could take ChatGPT to write something long, then throw it into their own writing style LoRA model to output something that sounds like them. That doesn’t mean that the article will be accurate or avoid gibberish though. If you were to write a wide range of topics and then create a model off of it, this would work much better. It appears that if you go off topic from your content, it will just stick with Llama’s typical responses. Finally, if you have a lot more newsletters or other written content, the model will sound more and more like you with more and more topics. In theory, automating your newsletters in your voice is actually possible! It just requires a lot of general text in your style.

This was a fun experiment and I’m looking forward to seeing how others find other uses for fine tuning. Next up I will cover some stuff in the Generative Art space.