SwiftyNote

Foundation models are a general purpose technology, but getting the right behavior out of them often involves giving up some of that generality and constraining them in some way. I've been thinking about this a lot lately, so I came up with a list of all the ways I can think of to constrain foundation models and arranged them into four general categories: model constraints, input constraints, output constraints, and architectural constraints. I'd be curious to hear what others think about this. Model constraints are changes to the model weights intended to alter the behavior of the model. They're the most fundamental form of constraint. Every form of fine tuning can be thought of as a kind of model constraint. They tame the feral animal that is a pre-trained model, getting it to follow instructions, minimize toxicity, adapt to a specific domain, or induce any number of other characteristics. Inputs constraints shape model behavior by modifying its inputs. The most well-known form of input constraint is prompt engineering, but another common variety is input filtering. Input filtering allows LLM chat apps to block any requests that violate their ToS, but it can also be used for clever tricks like inserting secret instructions to prevent leaking of copyrighted material (https://lnkd.in/eans8_iD). Retrieval augmented generation (RAG) could also be considered a kind of input constraint since it involves injecting relevant documents into the prompt. Output constraints are applied directly to model outputs. Output filtering is just like input filtering but on the opposite end; it's often used for content moderation. Validation is a little like output filtering, but it's usually more grammatically oriented (e.g., checking if JSON matches a schema). Constrained decoding is a powerful technique that guarantees a model's output will conform to a specified format (see https://lnkd.in/gRT6iHFp). Architectural constraints don't control models so much as they limit the importance of any given model output. Techniques like self-consistency or mixture-of-agents that involve comparing foundation model outputs to one another could be included in this category. A less sexy form of architectural constraint is something I'll call BYOR (Bring Your Own Reasoning); instead of asking the model to "think step by step," you think step by step and then explicitly program those steps into your application, only calling the model when necessary. BYOR can make it much easier to catch and correct errors because you're only asking the model to do one thing at a time. The downsides to all of these are that they will increase the complexity of your code and may increase inference costs, but the alternative is to be a thin wrapper. Does this make sense to people? Have I made any glaring omissions? Karl Swanson Kevin Maloy, MD

What if foundation models never reach AGI (artificial general intelligence)? Obviously I don't know for sure, but I'm deeply skeptical that they'll ever get there. But so what? Granted, AGI could be incredibly beneficial (or destructive, depending on whom you ask), but if foundation models don't reach that point, are they a bust? Probably not, but they might look that way at first. That's largely because they were first presented to the public as a general purpose tool. Ask the chatbot anything in your own words and it will give you an answer. Of course, we now know that many of the answers aren't very good, even if they look like they are. Getting reliable performance on most tasks requires prompt engineering (not your own words) or engaging with tools like document stores that aren't part of the core technology. As standalone consumer products, foundation models have some novel use cases, but they're pretty far from revolutionary. The thing is that consumers almost never use general purpose technologies directly. The internet is probably the most impactful tech breakthrough in my lifetime, but I don't "use the internet"; I use products that are enabled by the internet. Electricity is even more essential, but if I had to figure out what to do with it on my own, I would only succeed in burning my house down. They're ubiquitous, but they operate in the background. We only think about them when they stop working or when we have to pay utility bills. In fact, the more pervasive they've become, the less we've talked about them. Remember when every new company put ".com" in their name? Yeah, I feel old, too. Like those earlier technologies, foundation models can be used for many different purposes, but that also means we have to figure out what specific things to do with them. It's like staring at a blank page but on a much larger scale. This is sometimes referred to as capability overhang, and getting through it is hard work, requiring a mix of imagination and experimentation. We skipped those initial growing pains with foundation models. Instead of finding product-market fit, AI companies and their corporate partners gave us direct access to the raw tech or bolted it onto existing products. No wonder people are questioning its value. If this becomes a breakthrough technology or even a viable one, it will be because people build great products on top of it that leverage its unique capabilities. Most of these will be restricted to a single domain, like software engineering or medical documentation. They'll have slick UIs. Chat will be included in a minority, and in many cases, the language model will be hidden from the user entirely. If they become really pervasive, the tech itself will fade into the background. "AI" as part of a company's name will go the way of ".com". The good news is that whether foundation models are a breakthrough or a bust, we'll eventually stop talking about them.

I’m happy to share that I’m starting a new position as Advisor at SwiftyNote!

Glad to see Dr. Cunningham emphasizing the gradual adoption of technologies and the importance of skilled physicians in her recent interview. Integrating technology into processes isn't about replacing humans but making them safer and more efficient. Read more about Dr. Cunningham's insights here: https://lnkd.in/gp3Q8sg8 If I may humbly add: The best technologies seamlessly blend with our processes, often unnoticed. #AIinhealthcare