Emerging Tech & AI - 15th Edition

Welcome to the 15th Edition of?the Emerging Tech & AI Newsletter!

This newsletter aims to help you stay up-to-date on the latest trends in emerging technologies. Subscribe to the newsletter today and never miss a beat!

Subscribe to the newsletter here.

Here's what you can expect in each new issue of the Emerging Tech & AI Newsletter:

• A summary of the top AI / emerging technology news from the past week
• Introductory details/Primer on any emerging technology or a key topic in AI (We explore Small Language Generation this week)
• Examples of how AI is being used or How it will impact the future ( We explore AI Tools for Video Generation )

Last Week in AI/Emerging Tech

The field of AI is experiencing rapid and continuous progress in various areas. Some of the notable advancements and trends from the last week include:

Big Tech in AI:

1. Google to help the Indian Government in developing responsible AI.
2. FOX Sports expands Google Cloud partnership, generative AI to automate archived sports video search.
3. Google's AI will watch YouTube so you don't have to.
4. OpenAI, Microsoft hit with new author copyright lawsuit over AI training.
5. Amazon launches free AI courses on generative AI to skill people globally.
6. xAI’s chatbot ‘Grok’ will launch to X Premium+ subscribers.

Funding & VC Landscape:

1. Adobe buys Indian generative AI startup Rephrase.ai
2. AI firm BRAIIN Holdings forays into India, plans to invest $100 million.
3. Generative AI startup AI21 closed a USD 53 million extension round.
4. Divergent Technologies raised USD 230 million to accelerate the commercialization of its digital manufacturing system that uses generative AI and 3D printing.

Other AI news:

1. Inflection-2 beats Google’s PaLM 2 across common benchmarks.
2. Stability AI Unveils Generative AI Video LLM.
3. SDSU researchers develop an AI-powered model to predict stock market trends.
4. The model Q* demonstrates advanced reasoning capabilities similar to humans.
5. Research: AI system self-organizes to develop features of the brains of complex organisms.

Liked the news summary. Subscribe to the newsletter to keep getting updates every week. Check the comments section on the LinkedIn article for links to the Funding & VC Landscape and Other AI news.

Small Language Models

In the realm of artificial intelligence (AI), the development of language models has revolutionized the way we interact with technology. These models, trained on massive datasets of text and code, can understand and generate human language with remarkable fluency. While large language models (LLMs) have garnered significant attention for their impressive capabilities, small language models (SLMs) are quietly gaining momentum as a viable alternative.

What are Small Language Models?

Small language models are lightweight variants of their larger counterparts. They possess a smaller neural network architecture, fewer parameters, and are trained on a more constrained dataset. This makes them less computationally intensive to train and run, enabling them to operate on devices with limited resources, such as smartphones and edge devices.

How are Small Language Models Created?

The creation of SLMs involves two primary steps: training and fine-tuning. Training involves feeding the model a massive dataset, allowing it to learn the patterns and relationships within the language. Fine-tuning, on the other hand, involves tailoring the model to a specific task or domain by exposing it to additional relevant data.

Where are Small Language Models Used?

SLMs are finding applications in a wide range of domains, including:

• Chatbots and virtual assistants: SLMs power the conversational capabilities of chatbots and virtual assistants, enabling them to understand natural language and provide human-like interactions.
• Text summarization and translation: SLMs can efficiently summarize lengthy texts or translate them between languages, making them valuable tools for content management and communication.
• Content generation: SLMs can generate creative text formats, such as poems, scripts, musical pieces, and email, making them useful for marketing, education, and entertainment.
• Sentiment analysis and classification: SLMs can analyze text to determine its sentiment, whether positive, negative, or neutral, and classify it into relevant categories.

What will be the Role of Small Language Models in the Future?

SLMs are poised to play an increasingly significant role in shaping the future of AI. Their ability to operate on resource-constrained devices and their adaptability to specific domains makes them ideal candidates for embedded AI applications. As AI permeates everyday devices and environments, SLMs will be instrumental in providing seamless and intuitive user experiences.

Here are some specific examples of how SLMs may be used in the future:

• Personalized education: SLMs can tailor learning materials and provide personalized feedback to students, adapting to their individual needs and learning styles.
• Smart homes and wearables: SLMs can control smart home devices, respond to voice commands, and provide personalized recommendations based on user preferences.
• Healthcare diagnosis and treatment: SLMs can assist doctors in analyzing medical data, identifying potential diagnoses, and suggesting treatment options.
• Customer service and support: SLMs can power chatbots and virtual agents to handle customer inquiries, resolve issues, and provide personalized support.

SLM Examples

Below are some examples of available SLMs

1. DistilBERT - a smaller, faster, and lighter version of BERT.
2. Orca 2 - Specialized for zero-shot reasoning.
3. Phi 2 - Designed to be efficient in both cloud and edge deployments and built for mathematical reasoning, common sense, language understanding, and logical reasoning.
4. T5-Small - The Text to Text Transfer Transformer (T5) model from Google.

Conclusion

Small language models represent a significant step forward in the evolution of AI. Their ability to operate on limited resources, their adaptability to specific domains, and their potential for real-time applications make them a powerful tools for shaping the future of technology. As AI continues to integrate into our lives, SLMs will play an increasingly crucial role in bridging the gap between human and machine interaction, enhancing our productivity, creativity, and overall well-being.

Curious to know more? Let us know what follow-up details you would like in the comments and we will plan a detailed article on Tecnologia.

AI Tools for Video Generation

AI-based video generation tools utilize deep learning techniques to transform text descriptions into realistic videos. The process typically involves several stages:

1. Data Preparation:The models are trained on large datasets of videos to learn patterns and features.
2. Model Training:The generative model is trained on the video dataset, adjusting its parameters to minimize a certain loss function.
3. Inference or Generation:During the generation phase, the trained model takes input (random noise for GANs or samples from the latent space for VAEs) to produce new video frames.
4. Post-Processing (Optional):Depending on the application, post-processing steps may be applied to enhance or modify the generated video.
5. Visualization or Output:The generated video can be visualised, saved, or integrated into other applications.

The specific techniques employed by AI models for video generation vary depending on the tool and its underlying algorithms. Some common approaches include:

1. Generative Adversarial Networks (GANs): GANs employ two neural networks competing against each other to generate realistic images or videos.
2. Variational Autoencoders (VAEs): VAEs learn to compress and reconstruct data, allowing them to generate new variations from the learned representations.
3. Autoregressive Models: Autoregressive models predict the next frame based on the previous ones, enabling them to generate sequential data like videos.

Transformer-based Models: Transformer-based models, known for their ability to capture long-range dependencies in language, are increasingly being used for video generation due to their ability to process and understand complex text descriptions. Some of the LLM-based models include

a) NExT-GPT: It is a conversational platform that supports both text input and the uploading of image, video, or audio files. This model possesses the capability to comprehend the content within these inputs and can respond to user queries by generating text, images, video, or audio. Utilizing open-source pre-trained encoders and decoders such as Vicuna and Stable Diffusion, NExT-GPT incorporates trainable neural network layers for enhanced performance.

b) VideoGPT: It allows Video Generation using VQ-VAE and Transformers.

c) VideoDirectorGPT: This framework leverages the capabilities of large language models (LLMs) for video content planning and grounded video generation.

d) GPT4Motion: It is a training-free framework that leverages the planning capability of large language models such as GPT, the physical simulation strength of Blender, and the excellent image generation ability of text-to-image diffusion models to enhance the quality of video synthesis.

There are many paid /freemium tools available for generating end-to-end video content.

For instance, the following tools can be used to generate a 3D video based on custom scenes written using gravitywrite or any other AI-based writing tool

Midjourney: https://www.midjourney.com/, for image generation.

Runway: https://runwayml.com/ or Think Diffusion, convert images to animated video and do more.

Lamau: https://lalamu.studio/demo/ for adding dialogues and lip-syncing videos.

Chip Champ: https://clipchamp.com/en/ for editing videos

Eleven Labs https://elevenlabs.io/ or https://lovo.ai/ for providing voiceovers.

Pixabay: https://pixabay.com/music/ for music, videos and images.

Check out details of some more text-to-video AI tools like DeepBrain AI, Synthesia, Pictory etc here.

Would you like to add any other AI-based Video Generation tool to the list? Do let us know in the comments on the LinkedIn post.

References & Further Reading:

1. Hugging Face Text to Audio.
2. VideoDirectorGPT Research Paper.

Disclosure: Some content in the article was written with the help of Google Bard and ChatGPT.

Thanks for reading. See you next week!

Let's explore the future of technology together!