Have you ever experienced long delays while searching for a product on a shopping website? Searching for products or services is the most critical functionality for any e-commerce application. There are various data storage architecture options available to promote quick retrieval of data and accelerate the search process. There was a time when relational database was the most popular option for product engineering & design, where data is scattered among multiple tables. However, a Relational Database slows down when it comes to handling bulk data and fetching search results through SQL queries. As the complexity of applications and databases grew, architects started to look for ways to improve the speed of data handling and associated delays.
There are multiple ways to address these application delays. Architects are adopting new techniques with NoSQL distributed databases rather than continuing with traditional RDBMS.
This where Elasticsearch comes into the picture.
Elasticsearch (ES) is a fast-evolving NoSQL distributed database. From a performance perspective, it is important to learn how Elasticsearch can be used with your existing .NET applications. Below is one approach to address database and application performance challenges.
Elasticsearch benefits
Elasticsearch can be defined as a document-oriented database, designed to store, retrieve and manage document-oriented or semi-structured data. It is a better choice when there is a need to index and search unstructured data from different sources. It offers improved scalability over RDBMS full-text search.
There are number of use cases where Elasticsearch can be implemented with great advantages. Examples—Full-Text Search, Auto Complete, Auto Suggest, Data Aggregation, JSON document storage, Geo Search, etc.
The inclusion of Elasticsearch in the overall architecture can address most data processing issues that are typically faced by users of data intensive applications. Elasticsearch has its own unique capabilities which makes it popular in the application development community.
For example, Elasticsearch has its own domain-specific language in which users can specify the query in JSON format. Typically, applications search on a variety of fields by applying some conditions or values for some predefined fields. Such complicated scenarios can be best processed with the help of a single query.
In terms of scalability, Elasticsearch is a near-real-time search platform. There is a very minimal delay from the time you index data until the time it becomes searchable. Elasticsearch can also play an important role in the overall performance of an application due to its caching mechanism. It can execute complex queries extremely fast. It also caches commonly used queries which ensures faster results. Lastly, as it is API-driven, actions can be performed using a simple RESTful API.
Integrating Elasticsearch into an Application
Elasticsearch is one of the best choices in cases where you need to build advanced search engines to query and analyze massive amounts of data, or when you need to index and search unstructured data from different sources. To utilize .NET in conjunction with Elasticsearch, NEST, a high-level .NET client can be used as it maps closely with Elasticsearch APIs.
Elastic is developed in Java and is released as open source under the terms of the Apache License. Its RESTful web interface accepts schema-free JSON documents as input.
Below are the two official .NET clients for Elasticsearch which can be used in C# code to communicate with Elastic seamlessly:
- Elasticsearch.Net is a low level, abstracted, dependency-free client in which all the Elasticsearch API endpoints are represented as methods, but it’s not capable to build required JSON/request/response objects on its own.
- NEST is a high-level client that has the advantage of auto-mapping all request and response objects. It comes with a query DSL (Domain Specific Language) that maps one to one with the Elasticsearch query DSL. NEST internally uses and exposes, the low-level Elasticsearch.Net client.
NEST, one of the most popular clients, is being used in the architecture below to integrate with Elasticsearch. It takes advantage of .NET features such as covariant results. Let’s see how to perform full-text search using NEST from C#:
- The .NET application layer is sending a request through NEST query to Elastic search API. It offers excellent scalability by defining own nodes in runtime.
- Node: It’s a server that holds the data which involves indexing and grouping. It’s a single elastic search instance.
- The Elastic search indexes are divided into SHARDs which are basically a subset of index documents which allows searching with exceptional performance. All this functionality is abstracted within Elasticsearch and hidden from users.
Step by Step Elasticsearch Integration Process:
1) Download the NEST package from NuGet Package Manager as depicted in the below screenshot:
2) Add the libraries as shown:
3) Implement Elastic Full text search in a function as shown below:
public async Task<searchresult> Search(String querystr, int page, int pageSize)
{
//** eclient : It is one of the interface which we have used in our use case **//
var response = await this._eClient.GetClient().SearchAsync(searchDescriptor => searchDescriptor
.Query(queryContainerDescriptor => queryContainerDescriptor
.Bool(queryDescriptor => queryDescriptor
.Must(queryStringQuery => queryStringQuery
.QueryString(queryString => queryString
.Query(querystr)))
.From((page – 1) * pageSize)
.Size(pageSize));
return MapResponseToSearchResult(response, page, pageSize);
}</searchresult
Conclusion
Elasticsearch can help improve application performance tremendously by:
- Managing huge amounts of data: Elasticsearch can fetch search results in milliseconds, much faster as compared to traditional RDBMS.
- Providing direct, easy and fast access: Documents are stored in proximity to corresponding metadata in the index. This minimizes the number of data reads and in turn accelerates search result response.
- Delivering search engine scalability: As Elasticsearch has a distributed architecture, it can scale up to thousands of servers and accommodate petabytes of data.
Elasticsearch is evolving rapidly with new features and functionality, but the core remains consistent. It is also available as a service on various cloud development and integration platforms like Azure, AWS, etc.
Achieve faster performance with search results in your applications through Elasticsearch.
If you need experts to help you implement Elasticsearch, Emtec’s team can help! Contact us.