About the course
OpenSearch is a powerful, community-driven open-source search and analytics suite, originated from Elasticsearch and Kibana, that enables users to easily ingest, search, analyse, and visualise vast amounts of data in real-time. It provides scalable capabilities for a wide range of use cases, including enterprise search, log analytics, application monitoring, and security information and event management (SIEM). This comprehensive 2-day training course is designed to provide participants with the in-depth knowledge and practical skills needed to effectively build, secure, and operate solutions using OpenSearch from the ground up.
The course begins by introducing the OpenSearch ecosystem, its relationship to Elasticsearch, and its core architectural concepts like nodes, clusters, shards, and replication, laying the groundwork for understanding how OpenSearch scales and handles data. You will learn how to set up and configure an OpenSearch environment and then dive into the crucial process of data ingestion, understanding different methods for getting data into OpenSearch using the REST and Bulk APIs and common ingestion tools. A significant focus is placed on mastering how OpenSearch stores and indexes data through mappings and effective data modelling for document databases, including structuring indices, handling relationships like nested data, and choosing appropriate settings.
Key skills for interacting with your data are developed through comprehensive modules on searching data using the Query DSL, optimising search queries, and performing powerful data aggregations for analysis and reporting. You will also explore analyzers for controlling text processing. The course covers essential aspects for managing your data over time with index management strategies, including aliases, templates, lifecycle policies with ISM, and using Data Streams for time-series data. Crucially for building production-ready systems, dedicated modules cover OpenSearch Security (authentication, authorisation, TLS, audit logging) and Operational Aspects (cluster monitoring, troubleshooting, sharding/replication best practices, and backup/disaster recovery). Throughout the course, extensive hands-on labs reinforce concepts, enabling participants to gain practical experience in building, securing, and maintaining OpenSearch solutions.
Instructor-led online and in-house face-to-face options are available - as part of a wider customised training programme, or as a standalone workshop, on-site at your offices or at one of many flexible meeting spaces in the UK and around the World.
-
- Understand the OpenSearch ecosystem, its architecture, and core concepts (nodes, clusters, shards, replicas).
- Set up and configure a basic OpenSearch environment.
- Understand different methods for ingesting data into OpenSearch, including using the REST and Bulk APIs and common tools.
- Define effective mappings for indices and choose appropriate OpenSearch data types.
- Apply principles of data modelling for document databases, including denormalization and nested data structures.
- Perform basic and advanced searches using the Query DSL and optimize search queries.
- Perform powerful data aggregations for analysis and reporting.
- Understand and use analyzers to control text processing for search and indexing.
- Manage indices over time using strategies like aliases, templates, ISM, and Data Streams.
- Understand and configure OpenSearch Security for authentication, authorisation, and data protection.
- Use OpenSearch Dashboards for data exploration, visualisation, and interacting with the cluster.
- Understand key operational aspects like cluster monitoring, troubleshooting, best practices, and backup/disaster recovery (Snapshot/Restore).
-
This comprehensive 4-day training course is designed for developers, data engineers, system administrators, DevOps engineers, and architects who need to build, manage, or secure solutions using OpenSearch. It is ideal for:
Developers building search or analytics features into applications using OpenSearch.
Data Engineers responsible for ingesting, processing, and managing data in OpenSearch.
System Administrators and DevOps Engineers setting up, configuring, monitoring, and operating OpenSearch clusters.
Architects designing data platforms and solutions leveraging OpenSearch.
Anyone needing in-depth, practical knowledge covering OpenSearch capabilities, security, and operations.
-
Participants should have:
Basic familiarity with Linux or command-line environments for installation and administration tasks.
Basic understanding of JSON data format.
Basic concepts of databases and data modelling are helpful but not strictly required.
No prior experience with Elasticsearch, OpenSearch, or similar technologies is required.
We can customise the training to match your team's experience and needs - with more time and coverage of fundamental skills for new starters, or a swifter pace for experienced data professionals.
-
This OpenSearch course is available for private / custom delivery for your team - as an in-house face-to-face workshop at your location of choice, or as online instructor-led training via MS Teams (or your own preferred platform).
Get in touch to find out how we can deliver tailored training which focuses on your project requirements and learning goals.
-
Introduction to OpenSearch and Core Concepts
Introduction to the OpenSearch software family: Understanding its origins and relationship to Elasticsearch.
Why Choose OpenSearch?: Key benefits, use cases, and community aspects.
OpenSearch Architecture Overview: High-level understanding of nodes, clusters, indices, shards, and replicas.
Key Terminology: Defining core concepts used throughout the course.
Setting up and Configuration
Installation Options: Setting up a development environment, preferably using Docker, with guidance for other methods.
Essential Configuration Settings: Configuring nodes and clusters for basic operation.
Node Discovery: How nodes find and join a cluster.
Cluster Settings: Configuring cluster-wide behaviours.
Hands-On Lab: Setting up a basic OpenSearch cluster, configuring nodes, exploring cluster settings.
Data Ingestion
How Data Gets into OpenSearch: Overview of common ingestion methods.
Using the REST API: Indexing individual documents via the API.
Using the Bulk API: Efficiently indexing multiple documents.
Introduction to Common Ingestion Tools: Overview of tools like Logstash, Filebeat/OpenSearch Agent, and their role in collecting and processing data.
Considerations for Choosing an Ingestion Strategy.
Hands-On Lab: Indexing data using the REST and Bulk APIs, brief demo or overview of an ingestion tool.
Indexing and Mappings
The Role of Indices: How data is organised and stored in OpenSearch.
Understanding Mappings: Defining the structure and data types of documents within an index.
OpenSearch Data Types: Exploring common data types (text, keyword, numeric, date, boolean, geo, etc.).
Dynamic Mapping: How OpenSearch automatically infers data types.
Explicit Mapping: Defining mappings manually for control.
Resolving Mapping Conflicts: Understanding and handling issues when mappings clash.
Index Settings: Configuring behaviour at the index level (e.g., number of shards/replicas).
Hands-On Lab: Creating indices with explicit mappings, experimenting with dynamic mapping, updating index settings.
Data Modelling and Document Design
Thinking in Documents: Designing your data structure for a document database.
Contrasting Document Modelling with Relational Databases: Understanding the differences and trade-offs.
Denormalization Strategies: Using denormalization to optimise query performance in OpenSearch.
Working with Nested Data and Inner Objects: Modelling complex relationships within documents.
Structuring Your Indices: Designing index names and patterns for scalability and management.
Hands-On Lab: Designing document structures and index strategies for sample use cases (e.g., e-commerce products, logging), implementing nested data structures.
Searching Your Data
The Search API: Introduction to searching documents.
Basic Queries: Using match and term queries.
Full-Text Search: Understanding how text search works.
Query DSL (Domain Specific Language): Building complex queries.
Compound Queries: Combining queries using bool, function_score, etc.
Fuzzy Search and Wildcards.
Searching Nested Documents.
Geo Queries: Searching based on geographical location.
Pagination: Managing search results using from and size, or search_after.
Sorting Search Results.
Optimizing Search Queries: Techniques for improving search performance.
Hands-On Lab: Executing various search queries (basic, compound, nested, geo), practicing pagination and sorting, using explain to analyse query performance.
Aggregating Your Data
Introduction to Aggregations: Performing data analysis and summarisation.
Aggregation Types: Overview of Metric, Bucket, Pipeline, and Matrix aggregations.
Metric Aggregations: Calculating values (e.g., sum, avg, min, max, cardinality).
Bucket Aggregations: Grouping documents into categories (e.g., terms, range, date_histogram).
Combining Metric and Bucket Aggregations: Building complex reports.
Range and Date Range Aggregations.
Hands-On Lab: Executing various aggregations (metrics, buckets), combining aggregations to build reports and dashboards.
Analysing Text with Analyzers
What are Analyzers and why do we use them?: Controlling the text processing pipeline during indexing and searching.
Components of an Analyzer: Character Filters, Tokenizers, and Token Filters.
Exploring Built-in Analyzers.
Creating Custom Analyzers: Defining custom character filters, tokenizers, and token filters.
Practical Examples: Building custom analyzers for specific use cases like telephone numbers, zip codes, or autocomplete.
Hands-On Lab: Using built-in analyzers, defining and testing custom analyzers, applying analyzers to specific fields.
Index Management and Data Lifecycle
Index Aliases: Using aliases to abstract index names for flexibility.
Index Templates: Automatically applying settings and mappings to new indices.
Re-indexing: Migrating data between indices.
Introduction to Index State Management (ISM): Defining policies for index lifecycle (hot, warm, cold, delete).
Data Streams: Managing time-series data with automatically rolling indices and a single writing alias.
Hands-On Lab: Creating and using index aliases and templates, performing a re-index operation, defining and applying a simple ISM policy, working with data streams.
OpenSearch Security
Introduction to OpenSearch Security Plugin: Securing your cluster.
Users, Roles, and Role-Based Access Control (RBAC).
Authentication: Configuring internal database or integrating with external systems (e.g., LDAP, Active Directory - overview).
Authorisation: Defining permissions based on roles (index, document, field level security).
Transport Layer Security (TLS/SSL): Encrypting communication within the cluster and with clients.
Audit Logging: Monitoring security-relevant events.
Hands-On Lab: Setting up basic security (users, roles), configuring authentication (internal), defining basic authorisation rules, enabling TLS (overview/guided setup).
OpenSearch Dashboards
Introduction to OpenSearch Dashboards: The primary user interface.
Discover Panel: Exploring and searching your data.
Visualisations: Creating charts and graphs from your data.
Dashboards: Combining visualisations to monitor data.
Development Tools Console (Dev Tools): Interacting with the cluster using the REST API.
Hands-On Lab: Exploring data in Discover, creating various visualisations, building a dashboard, using the Dev Tools console.
Operational Aspects and Best Practices
Cluster Monitoring: Key metrics for monitoring cluster health and performance (CPU, Memory, Disk, Network).
Logging and Troubleshooting Basics: Understanding OpenSearch logs and identifying common issues.
Sharding and Replication Best Practices: Optimizing for performance and resilience.
Optimizing Indexing and Search: Reviewing key best practices from previous modules.
Backup and Disaster Recovery: Using Snapshot and Restore for creating backups and recovering data.
Hands-On Lab: Monitoring cluster health metrics, simulating a node failure (if possible), performing a snapshot and restore operation.
-
OpenSearch Official Website and Documentation: The central hub for OpenSearch, providing access to documentation, guides, and community resources. https://opensearch.org/docs/
OpenSearch Dashboards User Guide: Official documentation for the OpenSearch user interface, covering data exploration, visualisations, and dashboards. https://opensearch.org/docs/latest/dashboards/
OpenSearch Security Plugin Documentation: Official documentation for configuring security features like authentication, authorisation, and TLS. https://opensearch.org/docs/latest/security-plugin/
OpenSearch API Reference: Comprehensive documentation for the OpenSearch REST API, used for interacting with the cluster for indexing, searching, and managing resources. https://opensearch.org/docs/latest/api-reference/
OpenSearch Data Ingestion Documentation: Explore different methods and tools for getting your data into OpenSearch. https://docs.opensearch.org/latest/getting-started/ingest-data/
Trusted by
