DATA SCIENCE & MACHINE LEARNING Project using Python & Angular | Telecommunication domain | smartSense Solutions
Anand Tanna
Co-Founder & Head Business Development | AI Forum NZ | Generative AI | SSI | IoT | Product Development | FinTechNZ
Aybell is a telecommunication domain product (POC) we developed for our clients in Silicon Valley, in the USA. In this project we have used data science, machine learning, Angular along with other technologies. Over here, a 3-layered architecture was developed for smooth development and security. Following is the tech stack used in this project:
- Python (Data science, Machine Learning)
- Laravel
- Angular 6
- Android
- iOS (Swift)
- AWS (RDS, EC2)
- GCP
- MySQL
In this project we have also developed a chatbot in Slack. In the chatbot the queries were made to the database of the companies in terms of company name, phone number and work timings.
Our relationship started from the Android and iOS mobile apps, that are used to request call backs from the desired companies for customer service. Here is the review, the clients gave for that work. Then onwards we got the work for backend, admin panel, crawler service and finally the server managed. By this time we were the sole developers of this POC with a team of 10 members on board.
Following are the services developed for this project:
- End user webapp: User can make call requests from here and receive phone calls from the desired department of service from the desired company in the list.
- Admin Console: This is the UI to control all the services of this project. It controls the behaviour of crawler as well as the webapp mentioned above.
Crawler (Datascience & Machine Learning) :
- This is the service totally unknown to users. From the admin console, the admin can add a list of numbers. These numbers are feed in the crawler and the numbers start getting dialled to record the complete IVR of the customer care numbers. From this recording, the crawler prepares the textual representation of the recording using google transcriber. From this speech using data science and machine learning, the crawler will divide the full IVR in menus to such deep level that we even get the department specific extensions.
Brief on Admin Console
The major role of console is to control the crawler to get IVR recordings of numbers and create menus that the webapp users can dial. Other than this the user will be able to add departments for an organization along with location and extensions and much more detailed information for the organization.
To understand this, you will have to be familiar with these terminologies:
Terminology:
- Tree = The full list of menu options of an IVR.
- Root = The first level node of the tree.
- Node = Menu option of an IVR
- Branch = Similar to Node, but branch will always have a child node
- Leaf = The last node of any branch in the tree
- DTMF = Dialling sequence to reach
- STIME = The display of delay of seconds before pressing each value in DTMF
Note: Root, Branch and Leaf are Menu types in the tree From the bottom right corner you can see the raw details of the menu option for debugging as and when needed.
Major features of the console are as follows:
Recording
By default the recording will be No Status. When the user clicks on Start button in Action column it will start the process of recording the menu option selected by dialling the number and the DTMF (if any) with the STIME. Starting it will add the number “In queue”. Next, it will change to “In Process”. And finally once the recording file is generated it will turn to “Done”. You can also see the date and time of this process in Last Updated column.
Transcribing
This process will be same as for Recording in terms of functioning of the 3 columns. In the process, the backend will take the recording file and transcribe to text as seen in the picture above. The triggers (dial X, press X, etc.) will be highlighted in pink.
Parsing of text (with use of data science and machine learning)
This process will be same as for Recording in terms of functioning of the 3 columns. Starting the parsing will tell the crawler to study the text and give the child nodes that can be prepared from the transcription. Example: In the picture, it shows “press star” so a menu will be created with DTMF ending in “ * “ with the appropriate menu name to it.
Note: Two processes can not start together for a number. Also for Parsing, having transcription is must and for Transcribing having the recorded audio file is must.
Note: The refresh icon next to Menu Discovery Status in the picture will refresh the status so you do not have to refresh the full page.
While discovering a number there will be cases where it asks for INPUT like SSN, TAX ID, or any other kind identification or something else. In such cases you can add a menu item from there and enter the required input as the DTMF. Now this will create automated STIME with 1 second pause, but be mindful to change the first value of STIME to match the end time (in transcription) of the fragment where the INPUT is asked for. You can see in the picture above “Please enter NPI” is an example with a long string of digits in DTMF. Once that is added follow the discovery procedure and menu options will be created further.
Following are the images that can help you understand how these processes can be controlled even manually when needed:
Problem & Solution
While recording, transcribing and parsing, the crawler needs to be so intelligent that it must not repeat the menus that are already discovered. Many times the the crawler used to go infinite loop due to the way IVR was designed. But we have later made changes such that it can get out of the loop on its own.
For more details on this project like how end user is benefitted and other features & functions feel free to write us at [email protected]
Best Regards,
Anand Tanna
"Go Big or Go Home"