Search of Brazilian Laws using Dialogflow CX and Matching Engine

Searching for something specific in Brazilian legislation is not a trivial task. Laws are published, parts of these laws are removed, new texts are added, many laws refer to other laws, there are decrees, supplementary laws and so on. To develop a solution given this challenge, rigorous planning and a very solid and planned database are required. The cost of development of this solution in cloud is less than 10 USD. As an option to the Vertex AI Matching Engine, we will deploy the ScaNN (Scalable Nearest Neighbors) index on Cloud Run, which has the benefit of scaling to zero when idle (https://github.com/google-research/ google-research/tree/master/scann).

For this database/user interaction task, nothing better than DialogFlow CX, from Google Cloud. As an improvement from Dialogflow ES, CX version offers an intuitive and simple-to-use work interface, basically composed of flows, which are conversations, and each flow can have a number of specific pages to implement a task or conversation. In each flow, there is a start page with routes that lead to other pages or other flows, and event handlers, which have fulfillments (responses to users).

When entering a flow, each one of the boxes in the figure above shows the details of its pages:

In a flow, the start page offers the routes to be followed, and in the next page, there is, for example, the validation of a username. Upon accessing this page, fulfillment is the response we will give to the user at this stage of the conversation.

Thus, we have the phrases where the user speaks his name, answering the chatbot’s question. The name is an entity, and answering the question is an intent. Mark, for instance, is a person, so it can be indirectly said that many times the entity is a noun. On the other hand, answering the question “What’s your name:” is an intention, an action, a verb. Thus, answering your name is, in this case, an intent.

Entities and intents can be registered one by one in the Dialogflow interface. However, for greater efficiency, we can register these entities and intentions in batch. In intents, you can upload a csv containing the list of different intents. For batch registration of entities, we use the following code:

import requests
import pandas as pd
from google.oauth2 import service_account
from google.cloud.dialogflowcx_v3beta1.types import entity_type
from google.auth.transport.requests import AuthorizedSession

data = pd.read_csv(
    "/names-censos-ibge.csv", header=0, sep=',')

data['Name'] = data['Nome'].astype(str).map(lambda x: x.lower().title())

names = data['Nome'].iloc[0:15000].unique().tolist()
full_list = []
for part in names:
    lista_nomes = {}
    lista_nomes['value'] = part
    lista_nomes['synonyms'] = part
    full_list.append(each_entity_value)

credentials= service_account.Credentials.from_service_account_file(
    '/abcd12345.json')
creds=credentials.with_scopes(
    ['https://www.googleapis.com/auth/cloud-platform'])
session_auth=AuthorizedSession(scoped_creds)
tipo=entity_type.EntityType.Kind.KIND_MAP

project_id='your_project_id'
agent_id='abcdef1234567890'
location='us-central1' 
response=authed_session.post('https://us-central1-dialogflow.googleapis.com/v3/projects/' + project_id + '/locations/' + location + '/agents/' + agent_id + '/entityTypes',
                               json = {
                                   "kind": tipo,
                                   "displayName": "nome",
                                   "entities": full_list
                               }
                               )

response_txt=response.texts        

This will load thousands of instances of the name entity into your Dialogflow CX. Going to the “Manage” tab, you can view these entities:

You can do the same with other entities of interest. The intentions are then registered, mapping the existing entities in the sentences:

The intentions will generate responses (fulfillments) to the user and the entities can be used as parameters in the session:

So you are able to access the username the chatbot is interacting with by using $session.params.name

After this initial interaction, we may further customize the user experience by collecting information about users’ preferences, tastes, etc. Once this is done, the user can then make his/her queries to the chatbot.

Note that this construction structure also applies to voicebots, using an external integration with, for example, VoxImplant. You can also integrate the chatbot with WhatsApp, via Twilio. Another option is to integrate it with a website or Facebook Messenger, or even an Avaya switchboard. This last option is especially useful for CCAI, Contact Center Artificial Intelligence, Google Cloud’s solution for telephone exchanges, which may include service automation and telephone call monitoring.

Given that our chatbot has the function of searching for laws, how is this operationalized? Using an existing solution in the VCC (Virtual Career Center) of Google Cloud, we can make use of the Two Towers model associated with the Vertex AI Matching Engine to carry out this semantic search. It’s relevant to notice that we can either deploy Approximate Nearest Neighbors in Vertex AI Matching Engine or in Cloud Run.

In this specific case, a structured database was manually assembled based on Brazilian laws in a hierarchical manner, Title, Chapter, Section, Article, Paragraph and text of the law.

That database then goes into a Python notebook very similar to the one described in my other article: Using the Vertex AI Matching Engine in Google Cloud’s Virtual Career Center (VCC). This notebook, in its final part, uses a Flask application to receive calls from Dialogflow:

from flask import Flask, request, jsonify
import json

app = Flask(__name__)

@app.route('/predict',methods=['POST'])
def predict():
    if request.get_json():
        x = json.dumps(request.get_json())
        x = json.loads(x)
    else:
        x = {}
    data = x["text"] 
    def predict(text):
        campos=str(text).lower()
        query=np.sum([model.user_model(tf.constant(campos.split()[i])) for i in range(0,len(campos.split()))],axis=0)
        neighbors, distances = searcher.search_batched([query])
        out = df.iloc[neighbors[0],:].nome_vaga
        return "Segundo "+out["Lei/Decreto"]+" "+out["Num_doc"]+", de "+out["Data"]+": "+out["Titulo"]+":"+"\n"+out["Texto_lei"]
    
    return jsonify({ 
    "fulfillment_response": { 
        "messages": [{ 
            "text": {
                 "text": [
                predict(data)
                ] 
            }
        }] 
    } 
})

if __name__ == "__main__":
    app.run(port=8080,host='0.0.0.0',debug=True)        

Note that the return of this application, when calling the endpoint by Dialogflow, must follow the following structure, which is then understood by Dialogflow’s Webhook:

"fulfillment_response": {
        "messages": [{ 
            "text": {
                 "text": [
                predict(data)
                ] 
            }
        }] 
    }         

For this feature to work properly, the Dialogflow CX Webhook must be enabled:

This Flask application plus the notebook containing the trained and saved Two Towers model, with the ANN index (Approximate Nearest Neighbors — code below) will be part of a Docker container that will run on Cloud Run.

searcher = scann.scann_ops_pybind.builder(np.array(indice), 10, "dot_product").tree(
    num_leaves=df.shape[0], num_leaves_to_search=int(df.shape[0]/2), training_sample_size=df.shape[0]).score_brute_force(
    1, quantize=True).build()        

To mount the container image, we need three files:

• requirements.txt

Flask==2.2.2
pandas==1.3.4
numpy==1.22.4
unidecode
scann==1.2.7
tensorflow-recommenders==0.7.0
tensorflow==2.9.1
nltk==3.6.5        

• Dockerfile

FROM python:3.9

EXPOSE 8080
ENV PORT 8080
WORKDIR /home

COPY . /home
USER root
RUN pip install -r /home/requirements.txt
RUN python -c "import nltk;nltk.download('punkt');nltk.download('stopwords')"

CMD python3 /home/prediction.py        

• Notebook with Matching Engine ANN (notebooks here)

So, when everything is ready, we use the gcloud command to mount the container image?…

gcloud builds submit --tag gcr.io/your project_id/container_name . --timeout=85000        

… we push + deploy on Cloud Run?…

gcloud run deploy container_name --image gcr.io/your_project_id/container_name --min-instances 1 --allow-unauthenticated --memory 4Gi --max-instances 8 --region us-central1        

… which will generate an endpoint like:

https://container_name-abcd12345.run.app/predict

At this point, we go back to Dialogflow CX / Manage /Webhooks and add the Cloud Run endpoint address. For simplicity, Cloud Run then accepts unauthenticated calls. However, in order to follow Google Cloud best practices, you must use one of the authentication systems, username and password, key — value, or a CA certificate in DER format.

Then make an interaction in Dialogflow to call the endpoint and see in the Cloud Run metrics if you have code 200 (success):

Once up and running, and certain that we won’t be making any more changes to the structure of Dialogflow, we can then export the conversation history logs to BigQuery. We leave Dialogflow’s work interface, create a new dataset in BigQuery and then go to Console / Logging and create a sink (collection funnel) with the destination for BigQuery:

You can also choose which logs are included and which are left out of this export. Thus, when a new interaction is made in the chatbot, a dialogflow table is created in BigQuery, and the following query can be performed:

SELECT
 timestamp
 AS data,
 jsonPayload.queryResult.diagnosticinfo.execution_sequence[SAFE_OFFSET(0)].step_1.initialstate.flowstate.pagestate.name AS flow,
 jsonPayload.queryResult.text AS input,
 jsonPayload.queryResult.responsemessages[SAFE_OFFSET(0)].text.text AS output,
 labels.session_id
FROM
 `your_project_name.dialogflow.dialogflow_runtime_googleapis_com_requests_20230115`
WHERE
 jsonPayload.queryResult IS NOT NULL ORDER BY session_id 
LIMIT
 1000        

This query will give you the date of the interaction, which flow is being used, what is the user’s input, the chatbot’s response and the session id.

This BigQuery table can then go to Looker Studio where you can generate analytics for interactions, see frequently asked questions, analyze entities, etc., remembering that Dialogflow CX also offers an analytics view:

This is a very interesting end-to-end project to develop and has many practical business applications. You can query databases, validate documents with Computer Vision, integrate with financial institutions to check your balance, issue credit cards, it also serves to collect orders for restaurants, it also allows you to integrate with Google Workspace for educational purposes and many other applications. All of this with the well-known advantage of chatbots, which is their 24/7/365 availability, speed of service, an almost unlimited knowledge base, real-time access to databases, and automation that allows full integration of various services in the same interface.