The Iterative Blueprint for AI Startups: Learn, Experiment, Implement

The AI world is exploding at an unprecedented pace. Every day, a new large language model (LLM), AI agent, product, or innovation is announced, creating a whirlwind of advancements. For those working in the AI space, this rapid evolution can feel overwhelming. The pressure to keep up with the pace of innovation while consistently delivering results is immense. When DeepSeek entered the market, for instance, numerous LLM-based products quickly announced integrations with it, highlighting the urgency to adapt and stay relevant.

Amidst this chaos, there is a way forward. From my experience, I’ve learned that AI teams must continuously brace themselves to learn, experiment, trial, and prototype. The key is to embrace what works and discard what doesn’t. Below, I’ll share some of the principles and lessons I’ve gathered while building CodeSherlock, which have helped us navigate this dynamic landscape.

Continuous Learning

In the AI industry, staying updated is not optional—it’s essential. AI professionals must consistently keep their eyes on the latest developments: new LLMs, agent frameworks, startups, and products. Learning should be a core part of your team’s routine. I recommend dedicating at least X% of your team’s time to learning and exploration. If a particular topic is highly relevant to your work, dive deeper—spend extra time understanding it thoroughly and go above and beyond the basics.

At CodeSherlock, we’ve formalized this approach. Every week, we reserve dedicated hours for learning topics directly related to our work. To ensure consistency and discipline, we’ve integrated learning into our sprint cycles. We also allocate meeting times to discuss and share these learnings. Learning can take many forms: taking online courses, reading articles and blogs, or even hands-on coding experiments. By carving out time for learning within our weekly schedule and discussing it regularly, we’ve managed to keep up with the pace of innovations. We treat learning as equally important as delivering features, which is why we block time for it in our sprints and adhere to it rigorously.

Continuous Experimentation

Learning alone isn’t enough; it must lead to practical trials. If a new technology or platform seems promising in the context of your work, it’s crucial to experiment with it. Allocate time to test it in real-world scenarios and evaluate whether it delivers the desired results.

For example, at CodeSherlock, we spent weeks experimenting with different LLMs to understand their strengths and weaknesses. We created trial programs to compare their outputs and assess which models best suited our needs. This was an intense and time-consuming process, but it was necessary to identify the right solution for the problems we aimed to solve.

Similarly, we invested significant time in experimenting with structured outputs from LLMs. There were instances where we had to write end-to-end code to fully grasp the results. On multiple occasions, we had to discard structured outputs because they didn’t meet our expectations. This required persistence, patience, and a willingness to iterate. For a lean startup, this upfront investment of time and resources can feel daunting, but it’s time well spent. Without such experimentation, we wouldn’t be able to find the best solutions for our clients.

In the AI space, practical experimentation is non-negotiable. There’s often little past knowledge to rely on—everything is so new that you have to learn, try, and implement simultaneously.

Continuous Prototyping

Building on continuous experimentation, there are times when you need to take it a step further and create prototypes. Prototyping allows you to fully understand the potential outcomes of a new technology and make informed decisions.

For instance, during our exploration of structured outputs, we built multiple prototypes to visualize and analyze the results. This hands-on approach gave us a clearer understanding of the outputs and enabled us to make rational, data-driven decisions. Ultimately, this process led us to develop a unique method for extracting structured outputs from LLM responses—a solution we wouldn’t have discovered without prototyping.

Prototyping is particularly valuable in AI because it bridges the gap between theory and practice. It provides tangible insights that can guide your next steps and help you avoid costly mistakes.

Experimentation in AI requires discipline to avoid endless rabbit holes.?Time-box research phases (e.g., 2-3 days) and conduct daily progress checks—if a path shows no results after the allocated time, pivot to alternatives. Compromises are inevitable: prioritize critical metrics (like accuracy) and optimize secondary factors (like speed) later. Structured experimentation turns uncertainty into actionable decisions, ensuring you balance exploration with execution. Without deadlines, curiosity becomes paralysis.

Proactive Learning and Future Proofing

For example, we’ve also been dedicating significant time to experimenting with RAG (Retrieval-Augmented Generation) approaches. Often, while developing experimental or prototype code, we realize that the existing frameworks don’t fully meet our unique requirements. This pushes us to innovate within the framework, creating custom methods that align more closely with our specific needs.

Additionally, we make it a point to go above and beyond in our learning. We often delve deeper into topics than what is immediately necessary because we believe that certain knowledge, while not directly applicable today, could prove invaluable in the future. This proactive approach ensures that when new challenges arise, we’re already prepared to tackle them with confidence and creativity. By staying ahead of the curve, we not only solve current problems but also equip ourselves to handle future opportunities and obstacles effectively.

Conclusion

In the fast-paced world of AI, waiting is a luxury you can’t afford. The only way to stay ahead is to embrace continuous learning, experimentation, and prototyping. Unlike traditional fields, where you can often rely on past knowledge or the experiences of others, AI is still in its infancy. Many of the challenges you face will be uncharted territory, and the solutions will require you to learn, try, and implement in real time.

At CodeSherlock, these principles have been instrumental in our progress. By dedicating time to learning, experimenting relentlessly, and prototyping rigorously, we’ve been able to navigate the complexities of the AI landscape and deliver value to our clients. In a world where innovation never stops, the only way forward is to keep moving—learn, experiment, and iterate. This is the path to sustained progress and success in AI applications.