TT#11: "Tech Talk on Elasticsearch"
Satyam Barsainya
Staff Engineer @ Altimetrik | Ex-Publicis Sapient | Ex-Wipro | Java | Spring Boot | Flutter | HTML | CSS | Angular Specialist | API & Microservices Expert | Technical Blogger
Elasticsearch: The Definitive Guide to Data Navigation ??
Unraveling the Mystery ??: Elasticsearch, a sophisticated and distributed search engine, is designed to navigate and analyze vast volumes of data at breakneck speed. Initially crafted for full-text search capabilities, it has evolved into a versatile tool for diverse data analytics, spanning from log files to intricate business metrics.
Analogous Understanding ???: Picture Elasticsearch as a seasoned navigator in the boundless oceans of data. Much like a navigator charting an efficient course, Elasticsearch adeptly traverses and retrieves valuable information from extensive datasets, providing insights akin to a seasoned guide leading you through unexplored territories.
Technical Insights ??:
In essence, Elasticsearch is the virtuoso navigator of data realms, adept at efficiently steering through massive datasets, empowering businesses and users with the insights needed to make informed decisions in the dynamic landscape of information. ??
How elasticsearch works internally?
Data Arrival ??: Documents, representing products (websites, logs, articles), stream into Elasticsearch like crates arriving at a bustling marketplace.
Indexing ???:
Searching ??:
Key Features ??:
Internal Workings ??:
In essence, Elasticsearch is the market wizard of data realms, adeptly navigating through massive datasets, and empowering businesses and users with the insights needed to make informed decisions in the dynamic landscape of information. ??
Inverted Index Unveiled: Charting the Word Map ???
Explanation ??: An inverted index is a savvy data structure utilized by search engines like Elasticsearch to accelerate the process of locating information within a colossal dataset. It flips the conventional approach on its head, mapping each unique word to a list of documents where that word appears, rather than listing documents and their corresponding words. Picture it as a potent roadmap that directs you to the precise locations of words within an extensive library of documents.
Analogy ???: Envision yourself as an intrepid explorer in a vast library teeming with books on diverse topics. Traditionally, you might have a list of books with their respective contents. However, an inverted index would be akin to having a catalog at the end of each aisle, alphabetically listing every unique word and pinpointing exactly which books (documents) contain that word. It’s as if the library itself morphs into a guide, streamlining your quest for specific information.
Technical Details ??:
In essence, an inverted index is your reliable guide in the library of data, making the search for specific words or information a breeze. It optimizes the process, offering speed, flexibility, and adaptability, much like a well-crafted map that transforms exploration into an efficient and delightful journey. ??
Below is the detailed example of inverted index:
Inverted Index Example with Two Documents: Unveiling Elasticsearch's Power
Documents:
Let's consider two simple documents:
Document ID: 1
Text: "Elasticsearch is a powerful search engine that enables efficient data retrieval and analysis."
Document ID: 2
Text: "Data analytics with Elasticsearch provides valuable insights for informed decision-making."
Inverted Index: Explained
Now, let's delve into how the inverted index works with these two documents.
{
"elasticsearch": [1, 2],
"powerful": [1],
"search": [1, 2],
"engine": [1, 2],
"enables": [1],
"efficient": [1],
"data": [1, 2],
"retrieval": [1],
"analysis": [1],
"analytics": [2],
"provides": [2],
"valuable": [2],
"insights": [2],
"informed": [2],
"decision-making": [2]
}
Search Operation:
Now, if a user performs a search like "data insights," Elasticsearch can efficiently locate the relevant documents:
As a result, Elasticsearch swiftly retrieves both documents, showcasing how the inverted index facilitates rapid and accurate search operations across multiple documents. This fundamental mechanism is at the core of Elasticsearch's capability to handle diverse datasets with speed and precision.
Elasticsearch: Powering Giants in the Digital Landscape ??
Technology Titans ??:
E-commerce Empires ??:
Media & Entertainment Moguls ??:
Travel & Hospitality Hotshots ???:
Finance & Banking Behemoths ??:
These are just a few examples, and the list continues. Elasticsearch’s versatility and scalability make it a sought-after tool for organizations of all sizes across diverse industries, truly embodying the spirit of digital transformation. ??
领英推荐
Here is a list of some commonly used queries in Elasticsearch:
Match Query:
Description: Performs a full-text search on the analyzed text.
Example:
{
"match": {
"field_name": "search text"
}
}
Match Phrase Query:
Description: Matches the entire input phrase, preserving the order of terms.
Example:
{
"match_phrase": {
"field_name": "search text"
}
}
Match Phrase Prefix Query:
Description: Matches a partial phrase with a specified prefix.
Example:
{
"match_phrase_prefix": {
"field_name": "search tex"
}
}
Match Bool Prefix Query:
Description: Combines multiple match queries with boolean operators (AND, OR, NOT).
Example:
{
"bool": {
"must": [
{ "match": { "field1": "value1" } },
{ "match": { "field2": "value2" } }
]
}
}
Match All Query:
Description: Matches all documents in the index.
Example:
{
"match_all": {}
}
Match None Query:
Description: Matches no documents in the index.
Example:
{
"match_none": {}
}
Common Terms Query:
Description: Supports ignoring common terms in the input text.
Example:
{
"common": {
"field_name": {
"query": "search text",
"cutoff_frequency": 0.001
}
}
}
Fuzzy Match Query:
Description: Allows approximate matching with a specified fuzziness level.
Example:
?{
"match": {
"field_name": {
"query": "search text",
"fuzziness": "AUTO"
}
}
}
Term Query:
?Description: Matches documents that contain an exact term in a specified field.
?Example:
?{
"term": {
"field_name": "exact_term"
}
}
Terms Query:
Description: Matches documents that contain any of the specified terms in a field.
Example:
{
"terms": {
"field_name": ["term1", "term2"]
}
}?
These queries cover a range of search scenarios, allowing you to tailor your Elasticsearch queries based on your specific use case.
Below is the code snippet with detailed explanation. ??
@Service
@Slf4j
public class BookService {
@Autowired
private ElasticsearchClient elasticsearchClient;
private static final String BOOK_INDEX = "books";
public void saveBook(List<Book> bookList) throws IOException {
BulkRequest.Builder br = new BulkRequest.Builder();
for (Book book : bookList) {
br.operations(op -> op
.create(idx -> idx
.index(BOOK_INDEX)
.id(book.getId())
.document(book)
)
);
}
BulkResponse result = elasticsearchClient.bulk(br.build());
if (result.errors()) {
log.error("Bulk had errors");
for (BulkResponseItem item : result.items()) {
if (item.error() != null) {
log.error(item.error().reason());
}
}
}
}
public List<Book> searchBooksByMatchQuery(String field, String queryText) throws IOException {
SearchResponse<Book> response = elasticsearchClient.search(searchRequestBuilder -> searchRequestBuilder
.index(BOOK_INDEX)
.query(q -> q
.match(t -> t
.field(field)
.query(queryText)
)
),
Book.class
);
return convertHitsToBooks(response.hits().hits());
}
public List<Book> searchBookByMatchPhraseQuery(String field, String queryText) throws IOException {
SearchResponse<Book> response = elasticsearchClient.search(searchRequestBuilder -> searchRequestBuilder
.index(BOOK_INDEX)
.query(q -> q
.matchPhrase(t -> t
.field(field)
.query(queryText)
)
),
Book.class
);
return convertHitsToBooks(response.hits().hits());
}
public List<Book> searchBookByFuzzyMatchQuery(String field, String queryText) throws IOException {
SearchResponse<Book> response = elasticsearchClient.search(searchRequestBuilder -> searchRequestBuilder
.index(BOOK_INDEX)
.query(q -> q
.fuzzy(t -> t
.field(field)
.fuzziness("AUTO")
.value(queryText)
)
),
Book.class
);
return convertHitsToBooks(response.hits().hits());
}
private List<Book> convertHitsToBooks(List<Hit<Book>> hitBookList) {
List<Book> books = new ArrayList<>();
for (Hit<Book> hit : hitBookList) {
books.add(hit.source());
}
return books;
}
}
Explanation:
Explore the exciting world of Elasticsearch with my latest hello world project! ??
?? Dive into the details: Elasticsearch Example
?? Join the Conversation: Share this post with your friends and colleagues who are passionate about web development and tech innovation. Let's learn and grow together. Your network will thank you! ??