-
Notifications
You must be signed in to change notification settings - Fork 1
CallingBot
Every time the bot receives a call, the from the CallingController, it will create a new instance of this script in its constructor where it will go through a few phases depending which event was received:
public simplecallbot(ICallingBotService callingBotService)
{
if (callingBotService == null)
throw new ArgumentNullException(nameof(callingBotService));
this.CallingBotService = callingBotService;
CallingBotService.OnIncomingCallReceived += OnIncomingCallReceived;
CallingBotService.OnPlayPromptCompleted += OnPlayPromptCompleted;
CallingBotService.OnRecordCompleted += OnRecordCompleted;
CallingBotService.OnHangupCompleted += OnHangupCompleted;
}This will only be run through Once in every call. So it can be conveniently used to first initialize some variables here. In here, use the incomingCallEvent to gather the details of the participants in the call and use it to clean the user data variables that we will use later.
participant = incomingCallEvent.IncomingCall.Participants;
var id = Guid.NewGuid().ToString();
//remove the persistent variables first
int retries = 0;
BotStateEdit.removeUserData(participant, "activeMode", ref retries);
BotStateEdit.removeUserData(participant, "activeAcc", ref retries);Then lets greet the user by calling GetPromptForText function from Replies class and return the Task with the actions to the next function, which is OnPlayPromptCompleted. GetPromptForText takes in a string and vocalize it using bot framework's default synthetic voice.
incomingCallEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
new Answer { OperationId = id },
Replies.GetPromptForText("Top of the day to you! What would you like to do today?",2)
};
return Task.FromResult(true);In this function, we determine if the user wanted to record his/her voice or talk to a profile. So it sends a text message (Hero Card) to prompt the user, and thereafter prompt for name of profile if response was received from the Hero card.
//get click input here
Replies.GenResponseCard(participant);
activeMode = GetActiveProperty(activeMode, "activeMode", 50);
Debug.WriteLine($"ACTION: {activeMode}");
//Debug.WriteLine($"equal: {activeMode.Equals("None")}");
if (activeMode != null)
{
await SendRecordMessage($"Who would you like to {activeMode} as? 'as ::name::'");
activeAcc = GetActiveProperty(activeAcc, "activeAcc", 50);
Debug.WriteLine($"WHO: {activeAcc}");
}If record was selected, the function will initiate record where GetRecordForText vocalizes string input and creates a prompt object for recording the caller's voice.
playPromptOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
Replies.GetRecordForText("Recording. Please read out the sentence after you received the message and heared the beep.", silenceTimeout: 2)
};If call was selected, it will go into conversation mode:
playPromptOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
Replies.GetPromptForText($"Transferring call to {activeAcc}!",2),
getVoiceAsync("Hello...", 10).Result,
Replies.GetRecordForText(string.Empty, mode: -1)
};- OnRecordCompleted
The function receives recordOutComeEvent object which contains the recorded voice and pass it on to the appropriate processing function.
- RecordOnRecordCompleted - text message user sentences to record after every text
- CallOnRecordCompleted - retrieves audio clip from resourcespace / use built-in synthetic voice to reply the caller
if (activeMode == "record")
{
await RecordOnRecordCompleted(recordOutcomeEvent);
}
else
{
await CallOnRecordCompleted(recordOutcomeEvent);
}- OnHangupCompleted The bot will only touch this function when new Hangup() function is called, it lets the bot knows that the conversation should be terminated.
BingSpeech will allow us to transcribe the caller's voice to words that we can interpret programmatically in ElizaDialog.
Firstly, set up variables with locale and subscription key, we are using "en-US" in this case to transcribe english audio.
public string DefaultLocale { get; } = "en-US";
public string SubscriptionKey { get; } = ConfigurationManager.AppSettings["BingKey"];Before we are able to use the BingSpeech subscription, we have to create the client with the necessary credentials to access it. And make it such that if there are response from the service, pass it to local function OnDataShortPhraseResponseReceivedHandler.
public void CreateDataRecoClient()
{
this.dataClient = SpeechRecognitionServiceFactory.CreateDataClient(
SpeechRecognitionMode.ShortPhrase,
this.DefaultLocale,
this.SubscriptionKey);
this.dataClient.OnResponseReceived += this.OnDataShortPhraseResponseReceivedHandler;
}The recorded audio was received in a stream object so in order to send it to BingSpeech service, iterate through the bytes and send it.
public void SendAudioHelper(Stream recordedStream)
{
int bytesRead = 0;
byte[] buffer = new byte[1024];
try
{
do
{
// Get more Audio data to send into byte buffer.
bytesRead = recordedStream.Read(buffer, 0, buffer.Length);
// Send of audio data to service.
this.dataClient.SendAudio(buffer, bytesRead);
}
while (bytesRead > 0);
}
catch (Exception ex)
{
Debug.WriteLine("Exception ------------ " + ex.Message);
}And when the transmission is done, tell it so by calling .EndAudio().
finally
{
// We are done sending audio. Final recognition results will arrive in OnResponseReceived event call.
this.dataClient.EndAudio();
}
}The response passed back by BingSpeech is a list of sentences it think its appropriate. Each element has some useful variables like length of string result, confidence score and most importantly the text itself.
private async void OnDataShortPhraseResponseReceivedHandler(object sender, SpeechResponseEventArgs e)
{
if (e.PhraseResponse.Results.Length == 0)
{
this.WriteLine("No phrase response is available.");
}
else
{
this.WriteLine("********* Final n-BEST Results *********");
for (int i = 0; i < e.PhraseResponse.Results.Length; i++)
{
this.WriteLine(
"[{0}] Confidence={1}, Text=\"{2}\"",
i,
e.PhraseResponse.Results[i].Confidence,
e.PhraseResponse.Results[i].DisplayText);We would always want the answer with the highest confident level therefore return index 0's transcribed text back to simplecallbot.
_bingresponse(e.PhraseResponse.Results[0].DisplayText);