Many pattern mining algorithms focus on finding patterns that occur frequently in data. However, frequent patterns are not always useful because many simply confirm what is already known. This paper proposes a reasoning-based approach that helps an AI discover patterns that are both frequent and unexpected.
Hypergraphs for Knowledge Representation
The system stores knowledge in a hypergraph called AtomSpace, where concepts and relationships are represented as interconnected nodes and links. This structure allows the AI to represent complex relationships and perform logical reasoning on stored knowledge.
Pattern Mining Through Inference
Rather than searching patterns directly, the proposed method treats pattern mining as a reasoning task. Using OpenCog's Unified Rule Engine (URE), the system generates new patterns through logical inference while pruning patterns that do not satisfy a minimum support threshold. This makes the mining process more flexible and transparent.
Evaluating Interesting Patterns
To reduce the large number of discovered patterns, the paper introduces a measure called surprisingness. Instead of selecting patterns based only on frequency, the system compares the probability predicted by reasoning with the probability observed in the data. A larger difference indicates a more interesting pattern.
Independence-Based Estimation
The paper presents an Independence-Based Surprisingness method that estimates the probability of a pattern by assuming its components are independent. The estimated probability is then compared with the actual probability, allowing the system to identify relationships that are stronger or weaker than expected.
Adaptive Learning
An important feature of the approach is that newly discovered patterns improve the system's future expectations. As more knowledge is accumulated, the AI continuously updates its estimates, reducing surprises that have already been explained and focusing on discovering new ones.
Conclusion
This work combines hypergraph knowledge representation, logical reasoning, and statistical analysis into a unified pattern mining framework. By emphasizing surprising rather than merely frequent patterns, the approach enables AI systems to discover knowledge that is more meaningful and useful for intelligent reasoning.