1. How to Access the Pinot Data Explorer User Interface?
Open a browser and navigate to "http://localhost:9000/#/" to access the Pinot User Interface.
2. Cluster Manager Section
Navigate to Cluster Manager from the left navigation bar. This section includes multiple components:
· TENANTS
· CONTROLLER
· BROKER
· SERVER
· MINION
· TASK QUEUES
· CLUSTER CONFIGURATION
2.1 TENANTS section
A default tenant is always available in the TENANTS section.
Click on the "DefaultTenant" link to view the BROKER and SERVER associated with this tenant.
2.2 CONTROLLER section
Controller section gives you the details of the controller configured in your Pinot installation.
Click on the controller link, you can see INSTANCE CONFIG and LIVEINSTANCE CONFIG sections.
In Apache Pinot, the Controller is responsible for managing the cluster, including tenant assignments, table creation, and segment management. The configuration of a Pinot Controller is divided into two types:
Instance Configuration: This defines the static properties of a controller instance. It includes details such as the instance ID, host, port, and tags.
{
"id": "Controller_192.168.0.100_9000",
"simpleFields": {
"HELIX_HOST": "192.168.0.100",
"HELIX_PORT": "9000"
},
"mapFields": {},
"listFields": {
"TAG_LIST": [
"controller"
]
}
}
· "id": The unique identifier for this controller instance.
· "HELIX_HOST": The IP address where the controller is running.
· "HELIX_PORT": The port number used by the controller.
· "TAG_LIST": Labels assigned to the instance (e.g., "controller" to indicate its role).
Live Instance Configuration: This contains dynamic information about a currently active instance. It includes session details, version information, and the size of the task thread pool.
{
"id": "Controller_192.168.0.100_9000",
"simpleFields": {
"CURRENT_TASK_THREAD_POOL_SIZE": "40",
"HELIX_VERSION": "1.3.1",
"LIVE_INSTANCE": "38275@m-jjcw77gyk1",
"SESSION_ID": "1000891bf69000a"
},
"mapFields": {},
"listFields": {}
}
· "CURRENT_TASK_THREAD_POOL_SIZE": Number of threads allocated for processing tasks.
· "HELIX_VERSION": The version of Apache Helix (used for cluster management).
· "LIVE_INSTANCE": The unique identifier for this active instance.
· "SESSION_ID": The session ID assigned to this controller instance during its current run.
In summary,
· Instance config remains static and defines where the controller is running.
· Live instance config changes dynamically based on the current session and operational state.
· The Controller ensures cluster stability, and these configurations help manage its availability and resource usage.
2.3 Broker Section
A Broker in Apache Pinot is responsible for query routing and result aggregation. It provides a REST API endpoint for clients to execute queries and directs the queries to appropriate servers.
Click on the broker instance, you will be taken to the respective broker details.
The Broker UI provides an interface to view and monitor its configuration and status. It consists of three main sections:
Instance Config: This section displays the static configuration of the Broker instance.
{
"id": "Broker_192.168.0.100_8099",
"simpleFields": {
"HELIX_HOST": "192.168.0.100",
"HELIX_PORT": "8099",
"queryMailboxPort": "58730"
},
"mapFields": {},
"listFields": {
"TAG_LIST": [
"DefaultTenant_BROKER"
]
}
}
· "id": Unique identifier for this Broker instance.
· "HELIX_HOST": The IP address where the Broker is running.
· "HELIX_PORT": The port through which the Broker communicates.
· "queryMailboxPort": Port used for inter-broker query execution.
· "TAG_LIST": Specifies that this Broker belongs to the DefaultTenant_BROKER group.
Live Instance Config: This section shows real-time information about the Broker's current status and performance.
{
"id": "Broker_192.168.0.100_8099",
"simpleFields": {
"CURRENT_TASK_THREAD_POOL_SIZE": "40",
"HELIX_VERSION": "1.3.1",
"LIVE_INSTANCE": "39419@m-jjcw77gyk1",
"SESSION_ID": "1000891bf690011"
},
"mapFields": {},
"listFields": {}
}
· "CURRENT_TASK_THREAD_POOL_SIZE": Number of threads allocated for processing queries.
· "HELIX_VERSION": The version of Apache Helix managing this Broker.
· "LIVE_INSTANCE": The unique identifier for the currently active instance.
· "SESSION_ID": The session ID of this Broker instance, which changes when restarted.
Tables Section: This section provides an overview of the tables served by the Broker. It includes:
· Table Name: The name of the table.
· Reported Size: The size of data reported by the table.
· Estimated Size: An approximation of the table’s size.
· Number of Segments: The total number of segments for the table.
· Status: The health and availability of the table in the cluster.
2.4 SERVER section
The Server in Apache Pinot is responsible for storing and processing the data. It handles queries received from the Broker and serves results efficiently. The Server UI in Pinot provides insights into the server’s configuration, real-time status, and the tables it serves.
Click on the server, you can see the details of the server.
The Server UI has three main sections:
2.4.1 Instance Config (Static Configuration): This section displays the static configuration of the Server instance.
{
"id": "Server_192.168.0.100_8098",
"simpleFields": {
"HELIX_HOST": "192.168.0.100",
"HELIX_PORT": "8098",
"adminPort": "8097",
"grpcPort": "8090",
"queryMailboxPort": "58753",
"queryServerPort": "58752",
"shutdownInProgress": "false"
},
"mapFields": {
"SYSTEM_RESOURCE_INFO": {
"numCores": "12",
"totalMemoryMB": "32768",
"maxHeapSizeMB": "8192"
}
},
"listFields": {
"TAG_LIST": [
"DefaultTenant_OFFLINE",
"DefaultTenant_REALTIME"
]
}
}
· "id": Unique identifier for this Server instance.
· "HELIX_HOST": The IP address where the server is running.
· "HELIX_PORT": The port used by the server for communication.
· "adminPort": The port for administrative tasks (e.g., monitoring, debugging).
· "grpcPort": Used for inter-process communication (gRPC).
· "queryMailboxPort" & "queryServerPort": Ports used for distributed query execution.
· "shutdownInProgress": Indicates if the server is in the shutdown process (false means it's active).
Resource Information:
· "numCores": Number of CPU cores allocated (12 in this case).
· "totalMemoryMB": Total system memory available (32GB).
· "maxHeapSizeMB": Maximum heap memory allocated to Pinot (8GB).
Tags:
· "DefaultTenant_OFFLINE" → This server handles offline tables (batch data).
· "DefaultTenant_REALTIME" → This server handles real-time tables (streaming data).
2.4.2 Live Instance Config (Dynamic Information): This section contains real-time information about the active server instance.
{
"id": "Server_192.168.0.100_8098",
"simpleFields": {
"CURRENT_TASK_THREAD_POOL_SIZE": "40",
"HELIX_VERSION": "1.3.1",
"LIVE_INSTANCE": "41176@m-jjcw77gyk1",
"SESSION_ID": "1000891bf690015"
},
"mapFields": {},
"listFields": {}
}
· "CURRENT_TASK_THREAD_POOL_SIZE": Number of threads allocated for processing tasks (40).
· "HELIX_VERSION": Version of Apache Helix managing this instance.
· "LIVE_INSTANCE": Unique identifier for this running instance.
· "SESSION_ID": Session ID assigned when the server starts (changes on restart).
2.4.3. Tables Section
This section provides details about the tables stored and served by this Server.
Includes below information.
· Table Name: Name of the table.
· Reported Size: The actual size of the table.
· Estimated Size: Approximate size of the table.
· Number of Segments: The count of segments stored for the table.
· Status: Whether the table is active, degraded, or unavailable.
3. Query Console
The Query Console in Apache Pinot UI is an interactive interface that allows users to run SQL queries on their Pinot datasets. It provides an easy way to test, debug, and analyze data stored in Pinot tables.
4. Zookeeper Browser
Pinot use Zookeeper to store the metadata. The Zookeeper Browser in Apache Pinot UI provides an interface to explore and manage metadata stored in Apache Zookeeper. Since Pinot relies on Zookeeper for cluster coordination, the browser helps users view and debug cluster state, instance configurations, and table metadata.
5. Swagger REST API
Click on ‘Swagger REST API’ link available at left navigation section, it opens the swagger APIs documentation.
Previous Next Home
No comments:
Post a Comment