Redis Series - Building Find Near Me Feature using RediSearch

Hey everyone, when you search these days to find nearby restaurants, hotels, gas stations, what do you think happens behind the scenes? How does the system calculate nearby elements, apply textual search on top of it, add filters and facets depending on your preferences and then give you the most appropriate results back?

If you don’t follow this newsletter, make sure you click the Subscribe button by opening the newsletter page..! ??

Do normal Application Databases handle these use cases?

Let’s say you are using a normal database which gives your application CRUD capabilities and normal database indexes based on B-Trees. Would they satisfy our use cases? If all we want is to search all places within a single city name (not location), it could probably give us the results but the relevance and ranking would not be there.

What we actually need is a database which provides Full Text Search out of the box and can handle read requests at a very high throughput.

What are Full Text Search Indexes?

Full Text Search indexes are special indexes which use an “inverted index” concept in them. Rather than mapping a record to its content, the content is broken, analyzed and then mapped to records.

Let’s say we want to search hotels in our application based on their titles. Do we really want to type in the full name of the hotel to get the result? Obviously not, our system should be smart enough to understand partial titles, phrases and ideally also autocorrect our search results!

Geo-spacial Indexes

Another thing we need when building our application is that we should be able to get results which are “close to us”. Let’s say a user needs to find all gas stations in a 5km radius. This means we also need to calculate the distance of each record from our user’s location and return only those who have a distance less than 5km.

Think about all the gas stations or hotels we can have in our system. Probably millions across the world. Say the user lives in Delhi, should they even be shown the records of Bangalore, much less New York?

To solve this, we need a way to intelligently (or mathematically) prune the records. To do this we need a geo-spacial index, which builds a special data structure on the backend to help our queries prune the data effectively and scale our application read throughput!

RediSearch - Blazing Fast In-Memory FTS Engine

With the given context, I don’t think why not use RediSearch? RediSearch is a module for Redis which empowers us to have a dedicated Full Text Search capable of both the things above - amazing text queries as well as geo indexes.

What’s better is that, as it is powered by Redis on the backend, you can make use of the blazing fast speed of an in-memory database. To use RediSearch, we can either deploy it as a managed module on Redis Cloud as well as through running with Docker.

We will be using Redis Labs to use RediSearch by deploying a completely free Redis cluster with 30MB of data storage.

You can also use the code TIGER200 to get credits for additional storage.

Deploying RediSearch on Redis Cloud for Free

To deploy a free Redis cluster and enable RediSearch, follow the steps below -

1. Sign up and login to Redis Enterprise Cloud

2. Create a new Subscription, choosing Fixed Plan, your preferred cloud provider and region. Make sure the database tier is Standard 30 MB (FREE) so as to stay in the free tier zone.

3. Once you create your subscription, you can now create a Redis Database in it -

• Enter your database name.
• Choose the Type as Redis, not Redis Stack. (Redis Stack also supports RediSearch, but we will not use it for the purposes of this article)
• Choose the RediSearch module to enable it.
• Keeping every other setting as default, copy the password for the default user under the Security tab, as we will need this later.
• Click on Activate Database on the top.

Connecting to your new Managed Redis

On the configuration page, you would now find the Public Endpoint of your newly created database. We will now be using Python and the official Redis library for Python to create our application.

import redis

# # Connect to a database
client = redis.Redis(
???host="<host>",
???port="<port>",
???password="<password>",
)        

Once we have connected successfully, we can now define our options to create a full-text search (FTS) index in Redis. Using the IndexDefinition class, we specify that any keys (specially Hash Keys) with the prefix py_doc: would be used as an input and indexed in this index.

from redis.commands.search.indexDefinition import IndexDefinition?

# # Options for index creation
index_def = IndexDefinition(prefix=["py_doc:"])        

Defining Your Index Structure

Now, let’s define the structure/schema of our index which would tell Redis how to index each field in the FTS index.

from redis.commands.search.field import GeoField, TextField

# # Schema definition
schema = (TextField("title"), TextField("type"), GeoField("location"))        

The above mapping tells us that we have 3 fields in our system right now - one being the title of each place we have, the type of place (gas station, hotel, etc) and the location of each of them.

Building the Index

# # Create an index and pass in the schema
fts_client = client.ft("sampletest")
fts_client.create_index(schema, definition=index_def)        

We now create the index using the index definition and the schema we have defined above. This will go in and activate Redis’s RediSearch module to start indexing any existing keys, as well as new keys defined with the prefix “py_doc:”.

Indexing Data in our Application

Let’s try to ingest 2 sample documents in our application, both of type Hotel. What we do is we add a title, type and location in each of our documents - one hotel is JW Marriott in Delhi and one is Hilton hotel in Bangalore (with corresponding geo coordinates).

Now, we insert them as Hashes (documents) in Redis which internally sees that the prefix matches the one in our created Full Text Search index. Hence, it indexes our newly created documents.

# # A dictionary that represents a document
doc1 = {
???"title": "JW Marroitt, Delhi",
???"type": "HOTEL",
???"location": "77.121562,28.553394",
}

doc2 = {
???"title": "Double Tree Suites by Hilton",
???"type": "HOTEL",
???"location": "77.664822,12.919151",
}

# # Add documents to the database and index them
client.hset("py_doc:1", mapping=doc1)
client.hset("py_doc:2", mapping=doc2)        

Querying our Full Text Search Index

Now we can start querying our index. Let’s say someone wants to search with the name (title) of the hotels, they can just pass in each as a word. Remember, a default TextField kind of mapping breaks down the string and indexes each word (it’s lemma) as a new token for that field. So any tokens generated can be used as query strings.

Over here, we search by the word hilton which gives us only one matching document - the Hilton hotel in Bangalore.

from redis.commands.search.query import Query?

# # Search the index for a string; paging limits the search results to 10
result = fts_client.search(Query("hilton").paging(0,10))

# # The result has the total number of search results and a list of documents
print(result.total)
print(result.docs)        

Let’s say now you wanna find all hotels in Delhi, so you can query with the special query string - “*” and then add a filter, a GeoFilter, to filter only those locations which are within 10 KM of a user in Delhi.

from redis.commands.search.query import Query, GeoFilter?

# # Search the index for a string; paging limits the search results to 10
result = fts_client.search(
???Query("*").add_filter(GeoFilter("location", "77.121562", "28.553394", 10))
)

# # The result has the total number of search results and a list of documents
print(result.total)
print(result.docs)        

Running the above query, would return you the Marriott Hotel in Delhi, but not Hilton in Bangalore.

Under the hood, RediSearch uses a geo-spacial index to build and query geolocation data, which helps in effectively indexing latitude and longitudes in our system.

Conclusion

If you liked this article, make sure you Like and Comment with your feedback..! ??

As we saw above, we can very quickly create, ingest and query data directly from a Redis cluster using the RediSearch module. This gives us added advantage of having In-Memory speed and query performance as it is built directly over Redis.

Using Redis Cloud’s managed Redis offering, we also saw how quick and easy it is to spin up a Redis database and it is just a click to enable the RediSearch module on it.

Learn More

If you are interested in learning more, make sure to visit the following pages:

• Learn more about RediSearch
• Try Redis Cloud for free
• Watch this video on the benefits of Redis Cloud over other Redis providers
• Redis Developer Hub - tools, guides, and tutorials about Redis
• RedisInsight Desktop GUI

This post is in collaboration with Redis.