Having a conversation with ChatGPT
What's this Article about?
The article discusses how Microsoft Cognitive Services can be used to create a conversational AI experience with ChatGPT that allows users to interact with the model through natural language and receive responses in human-like voices. The Text-to-Speech API is used to convert ChatGPT's text-based responses into spoken language, while the Speech-to-Text API is used to transcribe the user's spoken language into text that can be processed by ChatGPT. This combination of technologies allows for a more natural and intuitive interaction with ChatGPT, creating a more engaging and user-friendly experience.
The Business Case
Here are a few examples of business cases where speech-to-text and text-to-speech, in combination with a conversational AI model like ChatGPT, could be useful:
Customer Service: Using speech-to-text, customer service calls can be transcribed in real-time, allowing ChatGPT to analyze the conversation and provide relevant responses to the customer. Text-to-speech can then be used to convert ChatGPT's responses into natural-sounding speech, creating a seamless customer experience.
Healthcare: Speech-to-text can be used to transcribe patient notes during medical appointments, which can then be used to create medical records. ChatGPT can analyze the notes and provide recommendations for treatment plans, while text-to-speech can be used to read back the recommendations to the patient.
Education: Speech-to-text can be used to transcribe lectures or classroom discussions, which can then be used to create study materials. ChatGPT can provide additional explanations and answer questions, while text-to-speech can be used to read the study materials aloud to students.
Finance: Speech-to-text can be used to transcribe financial reports or earnings calls, which can then be used to generate financial analyses. ChatGPT can analyze the data and provide insights or recommendations, while text-to-speech can be used to read the analyses aloud to investors.
Overall, combining speech-to-text, text-to-speech, and ChatGPT can create powerful business applications that improve efficiency, accuracy, and the customer experience.
How could this article be used for Automated Test Frameworks?
In the context of an automated test framework, integrating Microsoft Cognitive Services into the pipeline could provide significant benefits for testers and users alike.
When test cases are triggered from the CI/CD pipeline and saved in repositories, testers could use speech-to-text to record their observations and findings during the testing process. This would allow them to focus on the testing itself, rather than taking notes, and ensure that their feedback is recorded accurately and completely. The resulting text could then be used to generate bug reports or trigger automated workflows.
Text-to-speech could also be used to generate audio feedback for users during the testing process. For example, a test framework could be designed to test the accessibility of a web application. When a test fails, ChatGPT could provide an explanation of the failure and suggest potential solutions, which could then be read back to the user using text-to-speech.
By integrating Microsoft Cognitive Services into the test framework, testers and users would benefit from more efficient and accurate testing processes. Testers could focus on the testing itself, rather than taking notes, while users could receive more comprehensive feedback that is tailored to their needs.
Overall, integrating Microsoft Cognitive Services into an automated test framework has the potential to significantly improve the testing process, leading to better quality products and a more satisfying user experience.
Some examples in a Test Framework or Process
Here are some elaborated examples of integrating Microsoft Cognitive Services into an automated test framework and process. They only serve to get the minds going and create better and more suitable examples:
Overall, integrating Microsoft Cognitive Services into an automated test framework and process can provide significant benefits, including more efficient and accurate testing processes, more comprehensive feedback for users, and improved overall product quality.
The Flow
领英推荐
Speech to text
Microsoft Cognitive Services is a collection of APIs and services that developers can use to add intelligent features to their applications, including the ability to transcribe spoken language into text. The specific API for speech-to-text conversion is the Speech-to-Text API.
To use the Speech-to-Text API, you would first need to obtain an API key from the Microsoft Azure portal. Once you have your API key, you can make requests to the API to transcribe audio recordings or live audio streams in real-time.
When you make a request to the Speech-to-Text API, the audio is sent to Microsoft's cloud-based servers, where it is processed using machine learning algorithms to convert the spoken words into text. The resulting text is then returned to your application as a string.
The Speech-to-Text API supports a variety of audio formats, including WAV, MP3, and AIFF, as well as live audio streams. It also includes features for customizing the transcription process, such as the ability to specify a language model or add custom vocabulary.
Overall, the Speech-to-Text API is a powerful tool that enables developers to quickly and easily add speech recognition capabilities to their applications, allowing users to interact with them using natural language.
Text to Sound
Microsoft Cognitive Services offers a Text-to-Speech API that allows developers to convert written text into spoken language. The API uses deep neural networks to generate natural-sounding voices, and supports a wide range of languages and voices.
To use the Text-to-Speech API, developers first need to obtain an API key from the Microsoft Azure portal. Once they have the key, they can make requests to the API with a text string and other parameters, such as the desired language and voice.
The API then processes the text and generates an audio stream of the spoken words, which can be played back in real-time or saved as an audio file. The resulting voice can be customized with parameters such as voice speed, intonation, and emphasis, allowing developers to create voices that match their application's brand or personality.
In addition to generating spoken language, the Text-to-Speech API also supports the creation of custom voice models. Developers can train the API on their own data to create voices that sound like specific individuals or match the characteristics of a particular demographic group.
Overall, the Text-to-Speech API provides a powerful tool for adding natural-sounding speech capabilities to applications, making them more accessible to users who prefer or require spoken language.
Some examples of Usage
Microsoft Cognitive Services APIs, such as Speech-to-Text and Text-to-Speech, can be integrated into a wide range of business applications, including the Power Platform, SharePoint Online, Dynamics 365, and more.
For example, in the Power Platform, the Speech-to-Text API could be used to transcribe customer service calls, allowing customer service representatives to focus on the conversation rather than taking notes. The resulting text could then be used to generate support tickets or trigger automated workflows.
In SharePoint Online, the Text-to-Speech API could be used to provide audio descriptions of documents and other content, making them more accessible to users with visual impairments. It could also be used to provide audio feedback on form submissions or other user interactions.
In Dynamics 365, the Speech-to-Text API could be used to transcribe sales calls, allowing sales representatives to focus on the conversation rather than taking notes. The resulting text could then be used to generate follow-up tasks or trigger automated workflows.
Overall, integrating Microsoft Cognitive Services APIs into business applications can provide a range of benefits, from improving accessibility to increasing productivity and efficiency.
The Code
Here is some example code of the implementation in a console application. What it does, it takes your voice, turns it into a sound byte, the sound byte is turned into text, the text is given to ChatGPT and the answer of ChatGPT is turned into a sound byte again. Let your mind explore the extended use cases of this simple example.
The code is stupefying simple. That's on purpose.
using System
using System.Diagnostics;
using System.IO;
using System.Threading.Tasks;
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Audio;
using Google.Cloud.Speech.V1;
using Google.Apis.Auth.OAuth2;
using Grpc.Auth;
using Grpc.Core;
using System;
using System.IO;
using System.Threading.Channels;
using OpenAI_API;
using System;
using System.Net;
using System.Speech.Synthesis;
using System.Text;
using OpenAI_API.Completions;
class Program
{
? ? // This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
? ? static string speechKey = "<YOUR KEY>";
? ? static string speechRegion = "westeurope";
? ? static async void OutputSpeechRecognitionResult(Microsoft.CognitiveServices.Speech.SpeechRecognitionResult speechRecognitionResult)
? ? {
? ? ? ? switch (speechRecognitionResult.Reason)
? ? ? ? {
? ? ? ? ? ? case ResultReason.RecognizedSpeech:
? ? ? ? ? ? ? ? Console.WriteLine($"RECOGNIZED: Text={speechRecognitionResult.Text}");
? ? ? ? ? ? ? ? speak(speechRecognitionResult.Text);
? ? ? ? ? ? ? ? break;
? ? ? ? ? ? case ResultReason.NoMatch:
? ? ? ? ? ? ? ? Console.WriteLine($"NOMATCH: Speech could not be recognized.");
? ? ? ? ? ? ? ? break;
? ? ? ? ? ? case ResultReason.Canceled:
? ? ? ? ? ? ? ? var cancellation = CancellationDetails.FromResult(speechRecognitionResult);
? ? ? ? ? ? ? ? Console.WriteLine($"CANCELED: Reason={cancellation.Reason}");
? ? ? ? ? ? ? ? if (cancellation.Reason == CancellationReason.Error)
? ? ? ? ? ? ? ? {
? ? ? ? ? ? ? ? ? ? Console.WriteLine($"CANCELED: ErrorCode={cancellation.ErrorCode}");
? ? ? ? ? ? ? ? ? ? Console.WriteLine($"CANCELED: ErrorDetails={cancellation.ErrorDetails}");
? ? ? ? ? ? ? ? ? ? Console.WriteLine($"CANCELED: Did you set the speech resource key and region values?");
? ? ? ? ? ? ? ? }
? ? ? ? ? ? ? ? break;
? ? ? ? }
? ? }
? ??
? ? static async void speak(string text)
? ? {
? ? ? ? System.Speech.Synthesis.SpeechSynthesizer speechSynthesizer = new System.Speech.Synthesis.SpeechSynthesizer();
? ? ? ? var api = new OpenAI_API.OpenAIAPI("<YOUR KEY>);
? ? ? ? string answer = GetRESTAPICallContent("https://<YOUR ADDRESS>/api/ChatGPT/"
? ? ? ? ? ? ? ? ? ? ? ? + text);
? ? ? ? answer.Replace("Response from AI: ", "");
? ? ? ??
? ? ? ? speechSynthesizer.SelectVoiceByHints(VoiceGender.Female, VoiceAge.Adult, 0, new System.Globalization.CultureInfo("en-US"));
? ? ? ? speechSynthesizer.Speak(answer);
? ? }
? ? public static string GetRESTAPICallContent(string uri)
? ? {
? ? ? ? HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
? ? ? ? string getResponse;
? ? ? ? try
? ? ? ? {
? ? ? ? ? ? WebResponse webResponse = request.GetResponse();
? ? ? ? ? ? Stream webStream = webResponse.GetResponseStream();
? ? ? ? ? ? StreamReader responseReader = new StreamReader(webStream);
? ? ? ? ? ? string response = responseReader.ReadToEnd();
? ? ? ? ? ? getResponse = response;
? ? ? ? ? ? responseReader.Close();
? ? ? ? ? ? return response;
? ? ? ? }
? ? ? ? catch (Exception e)
? ? ? ? {
? ? ? ? ? ? return e.Message;
? ? ? ? }
? ? }
? ? async static Task Main(string[] args)
? ? {
? ? ? ? try
? ? ? ? {
? ? ? ? ? ? var speechConfig = SpeechConfig.FromSubscription(speechKey, speechRegion);
? ? ? ? ? ? speechConfig.SpeechRecognitionLanguage = "en-US";
? ? ? ? ? ? using var audioConfig = AudioConfig.FromDefaultMicrophoneInput();
? ? ? ? ? ? //using var audioConfig = AudioConfig.FromWavFileInput("Recording.wav");
? ? ? ? ? ? using var speechRecognizer = new SpeechRecognizer(speechConfig, audioConfig);
? ? ? ? ? ? Console.WriteLine("Speak into your microphone.");
? ? ? ? ? ? var speechRecognitionResult = await speechRecognizer.RecognizeOnceAsync();
? ? ? ? ? ? OutputSpeechRecognitionResult(speechRecognitionResult);
? ? ? ? }
? ? ? ? catch (Exception ex)
? ? ? ? {
? ? ? ? ? ? Console.WriteLine(ex.Message);
? ? ? ? }
? ? }
};
Conclusion
The previous text explains how Microsoft Cognitive Services can be used to create a conversational AI experience with ChatGPT, where users can interact with the model through natural language and receive responses in human-like voices. This is achieved by using the Text-to-Speech API to convert ChatGPT's text-based responses into spoken language and the Speech-to-Text API to transcribe the user's spoken language into text that can be processed by ChatGPT. This combination of technologies can create a more natural and intuitive interaction with ChatGPT, improving the user experience. Overall, this article highlights the power of Microsoft Cognitive Services in creating conversational AI solutions that can provide value in a variety of business cases.