Why Retrieval-augmented generation (RAG)?

TLDR; I spent far too much time trying to create a fine-tuned "J.D. Meier" skilled LLM model using Python, RAG gave me what I wanted in minutes....

I'm a huge fan of J.D. Meier, the insights and nuggets of knowledge that he shares are just awesome, as is his ability to seemingly churn out this content on such a regular basis!? There are of course years of experience sitting behind this and it set me to thinking, what if I could ask J.D. Meier whatever I wanted, when I wanted, based on everything he has blogged over recent years?? Now that would be awesome, but how?!? Well.. He does have a very comprehensive blog, so that seemed to be the ideal starting point.? In my mind, I wanted to create a fine-tuned LLM, using Phi3 so I could have a local LLM augmented with JD's insights.

So I set to it - the first step was to collect the data, there were some 300+ blog entries, of course too much to do manually, so I created a python script to scrape all the text, including the URL, the Title and the blog text and then formatted into a JSON file.?? I now had my initial set of data, however there was much data cleansing needed to get rid of different ASCII characters, duplicated text etc. etc.? Again a mix of python and manual review was used to do this and after a few hours, I now had my 'clean' data.? I then realised that in many cases the content was larger than the permissible token sizes so more python to try and split the data whilst trying to retain some of the context of the paragraph… this was time consuming and tedious!? Regardless, I persevered and I now had a set of processed data that I was ready to train.

?And again, this wasn't easy, I spent many hours working with GitHub co-pilot to get to a point where my code would run and train. I wound up using GPT2 via Huggingface as I was getting fustrated with many, many CUDA errors for Phi3(!). Eventually I managed to get it all compiled and it successfully ran, yes!

Ecstatic I then move onto code to prompt for a response, this was quick to pull together again with help from GitHub co-pilot, but what I got back was… well... rubbish:

How can I become a high-potential individual?

The answer is simple: You can't become a high-potential individual if you don't have the skills and skillset to become a high-potential individual.

?skillset is the ability to be a high-potential individual.

?

The key to becoming a high-potential individual is to be able to be a high-potential individual.

?skillset is the ability to be a high-potential individual.

?skillset is the ability to be a high-potential individual.

?

The key to becoming a high-potential individual is to be able to be a high-potential individual.

After again research and experimentation with temperatures & repetition penalties I eventually get an output that's somewhat coherent:

How can I become a high-potential individual?        

One of the biggest challenges facing many people is managing your expectations. You need to build an environment where you know what's going on in advance, and when things go wrong with someone or something that doesn't work out well for them at all (or even worse). To help me understand this better more fully here are some tips from my favorite mentors:? People who fail early have much less confidence if they don their best practices first thing through The Big Picture If anyone wants guidance as how not only do others but yourself start making good choices about why we should prioritize our future investments over ourselves it would be great!        

I'm getting there! But it's clear I still have a long, long, long way to go before I get a model that can coherently respond 'as' J.D. Meier. Somewhat despondent I decide to take a RAG approach, I ingest the exact same data into Azure AI Studio, craft an appropriate prompt using a GPT model and minutes later, look at what I get back!

How can I improve my team's productivity?

Improving your team's productivity involves a mix of strategic planning, effective management techniques, and fostering a motivating work environment. Here are several actionable strategies based on insights from the provided documents:

1. Embrace the Integrating Generalist Approach

Integrating Generalists excel in connecting the dots across various disciplines and functions within a business. By encouraging your team to think beyond their specific roles and understand how their work impacts the broader organization, you can foster a more cohesive and efficient work environment. This approach can help in cross-pollinating ideas and creating a more innovative team dynamic.

2. Implement Kanban for Visual Workflow Management

Using a Kanban board with segments such as To Do, Doing, and Done can significantly improve your team's productivity. This method helps in visualizing the workflow, decluttering tasks, and creating a shared view of the work. It allows team members to see the progress and bottlenecks at a glance, facilitating better communication and coordination.

3. Adopt the Rule of Three for Focused Outcomes

Focus on three key outcomes each day, week, and month. This method helps in prioritizing tasks and ensuring that the team is aligned on the most critical objectives. By setting clear, achievable goals, you can help your team stay focused and motivated, leading to more significant and meaningful progress.

4. Foster a Culture of Continuous Learning

Encourage your team to engage in continuous learning and skill development. This can be achieved through 30 Day Sprints, where team members focus on developing a new skill or improving an existing one for 30 days. This approach not only enhances individual capabilities but also contributes to the overall growth and adaptability of the team.

5. Enhance Motivation through Intrinsic and Extrinsic Factors

Understanding what motivates your team members is crucial. Use a combination of intrinsic motivation (internal drive) and extrinsic motivation (rewards and recognition) to keep your team engaged. Create an environment where team members feel their work is meaningful and aligned with their values and goals.

6. Streamline Communication and Collaboration

Encourage open and empathetic communication. Use strategies such as seeking first to understand before being understood and thinking win/win to build a more collaborative and supportive team environment. Effective communication can prevent misunderstandings and ensure that everyone is on the same page.

7. Provide Clear Feedback and Foster Accountability

Establish clear feedback loops and accountability structures. Regularly review progress, celebrate wins, and address challenges. This helps in maintaining momentum and ensuring that everyone is aware of their contributions and areas for improvement.

8. Break Down Large Projects into Manageable Chunks

Avoid letting big projects block smaller, valuable tasks. By breaking down large projects into smaller, manageable chunks and delivering value incrementally, you can maintain a steady flow of productivity and keep the team motivated with visible progress.

9. Utilize Feedback for Continuous Improvement

Create an environment where feedback is regularly sought and used for improvement. This involves tightening feedback loops, ensuring feedback is accurate and relevant, and focusing on actionable insights. This can help your team continuously improve their performance and adapt to changes effectively.

10. Maintain a Balance Between Strategy and Execution

Ensure that your strategy is dynamic and evolves with changing circumstances. Avoid getting bogged down in unnecessary details and focus on executing plans that add value. Regularly review and adjust your strategy to keep it aligned with your goals and the changing business environment.

By integrating these strategies into your management practices, you can enhance your team's productivity, foster a more engaging work environment, and drive better results.

For more detailed insights, refer to the original articles: The Integrating Generalist and the Art of Connecting the Dots, 30 Day Sprints for Personal Change, and 7 Habits of Highly Motivated People.

This is an order of magnitude better than what I had achieved, and I confess somewhat frustratingly to me, in a fraction of the time! Even better, I can cite the URLs as a reference if needed.

So what have I learnt from this?

1. I could have saved many, many hours had I just started with RAG, BUT going through this process has given me a new appreciation for how much is involved in sucesfully fine-tuning LLM models
2. Use WSL if you are on Windows, I again spent far too much time trying to get python & its depedencies needed to work natively in Windows (which I didn't in the end), I eventually enabled WSL and it all just worked.
3. It really suprised me how much of a difference temperatures & repetition penalties made to the inital response when coding locally in Python! Granted it shouldn't have as I've read lots about it, but seeing it in action really brought it home to me
4. Data preparation is key, Spending more time on this and working out how to break out the very long paragraphs, lists & bullet points on a given topic into meaningful, manageable chunks that still made sense in the context of the given topic would have had a significant impact on the results I had. I've also realised that my data was missing a lot of context in the form of missing diagrams, which again meant some of the data didn't make too much sense as it was refering to something that wasn't in my dataset, I truly believe I could have spent days on this and still not got it to where it needed to be to get the outcome I was after - but it's still a scratch that I need to itch, so I may well carry on with this regardless!
5. Use the tools available! I fully confess that I'm a technical chap at heart & I wanted to get deep in the weeds using Python to do this, which obviously is great for personal learning - however, Microsoft has done all the heavy lifting for you in this space, so make sure you take advantage of whats there! Plus I now see that in the US East 2 region there is a preview service for fine tuning a model, which I for sure will be looking at next.
6. GenAI (thankfully!) won't be replacing JD anywhen soon! ??

?