⮞ View Other Features Map Topics
Using the dtSearch Engine on Azure and AWS
- General information on using the dtSearch Engine with Azure and AWS, including platform requirements, index storage and and data storage.
- New sample code (released with Version 2023.02) demonstrates use of the dtSearch Engine in an ASP.NET Core application running in a Docker container under Windows (NanoServer) and Linux.
CodeProject Article: "Intelligent Search" with dtSearch Engine and Azure Storage
Article Excerpt:
Azure functions are an extremely versatile platform to operate on. The dtSearch tools allow us to build and manage a search index anywhere that we need intelligent search capabilities ...
What if there was an easier way to store my documents in the cloud for backup purposes and get the same great searching capabilities as when I have those documents stored locally? That’s where dtSearch and Azure functions can together provide a neat experience for managing your documents and their contents in the cloud ...
Azure Storage is a great facility that keeps multiple backups of your documents and data across regions ... Azure Functions allow us to write a little bit of code that can be managed and run when appropriate to interact with those storage locations and perform services for us ...
In the case of dtSearch and our data, this is a perfect match. We can structure our storage location so that it properly houses the archive of documents we want to process with our search engine, and we can also trigger the regeneration of that search index appropriately when a new document is added to the collection. The Azure Function architecture also allows us to make querying that managed search index a trivial operation with an HTTP endpoint, accessible from anywhere in the world.
CodeProject Article: Secure Instant Search in Azure
In this .NET solution, all dtSearch components (including indexes) are cloud-based.
Article Excerpt:
The Microsoft Azure Cloud makes available more data centers than Amazon and Google combined, along with 3X data replication and the most advanced security practices in the industry ...
dtSearch uses its own built-in “document filters” to instantly search terabytes of a broad range of data types: “Office” documents, PDFs, emails with multilevel attachments, web data, and other databases. dtSearch offers over 25 different search options, including faceted searching and multiple advanced data classification options, as well as search results display with highlighted hits ...
Azure Files stores the dtSearch indexes. Azure Files lets you share data across any number of applications through the SMB (Server Message Block) protocol.
Article Excerpt:
Using dtSearch and the techniques in this article will make your data searches lightning fast, making it possible to search terabytes of data with sub-second response time.
But first, two preliminary notes about this blog post. (1) The blog post describes source code data, but the same approach would apply to other data stored in the Microsoft Azure cloud: HTML, XML, MS Office documents -- even email data. (2) While the data in this blog post resides in the Microsoft Azure cloud, the indexes are on a local PC. A subsequent article will address data and indexes in the cloud.
Here is a workplan of our overall project:
CodeProject Article: “Windows Azure SQL Database Development with the dtSearch Engine”
Article Excerpt:
Windows Azure SQL Database does not support the CONTAINS keyword. CONTAINS is important because it supports searches for precise or fuzzy (less precise) matches to single words and phrases, words within a certain distance of one another, or weighted matches. CONTAINS is a predicate used in the
WHERE clause of a Transact-SQL SELECT statement.
But there are ways to get this functionality with the help of third party software. I recently downloaded all the movie information for films that took place in San Francisco. The San Francisco government website provided this interesting information that I wanted to search through very quickly. The data wasn't very normalized so there are many columns that are difficult to search. I downloaded this information and imported it into a database. Specifically I used SQL database hosted in the Windows Azure platform.
My goal was to be able to find records in the database table, searching in all columns and in all rows quickly and efficiently. Essentially, I wanted to build one master index that would allow me to perform the search locally against the generated index and then be able to pull up the specific record in the database table once I found what I was looking for.
dtSearch is a product that allows you to accomplish this goal. I downloaded it and installed it. Of course, I use it for many of my other searching needs as well. dtSearch has a variety of products. This post will focus on how you can index a database table so that you can perform lightning quick full-text searches.