In Console, many of the list pages (e.g. Topics, Consumer Groups, Schema Registry, Kafka Connect) are served from an internal table instead of making AdminClient calls to Kafka. Theses tables are updated by a background process we call Indexing. The Indexing process collects changes to your Kafka every 30 seconds and stores that metadata in internal tables. Indexing explained
Detail pages are not using Indexing. As soon as you’re on a page for a specific topic or consumer group, the data is fetched directly from the Kafka cluster.

​
Benefits

Indexing improves user experience and provides functionality that’s not available with out-of-the-box Kafka resources, such as:
  • smart tables with sorting and filtering capabilities, allowing you to get message count, size and much more
  • search and labelling that allows you to organize and find required resources

​
Examples

These are just some examples of how Indexing can help you:
  • identify idle topics: which topics (that have no active consumers and producers) haven’t published data for more than a week?
  • are there any over-partitioned topics - topics that have a large number of partitions while the biggest consumer group only has a few members consuming in parallel?
  • prioritize data at risk topics: which topics are at risk of losing data, because of the replication factor or the min ISR being incorrect?
  • find outliers topics: which topics contain bad or overridden configurations that they shouldn’t have?
  • remove over-replicated applications: which consumer groups have idle members? Typically, this is because the number of consumers exceeded the number of total partitions.

​
Troubleshoot