New Age of Terrain Classification: Proprioceptive Sensors and Uncertainty Quantification

Posted on by Basliel

The challenges of Terrain Classification

Terrain classification (TC) in rover exploration has traditionally relied on computer vision. However, the dynamic conditions of space—such as fluctuating illumination and atmospheric changes—can compromise visual sensors, affecting classification reliability. To address this, researchers are exploring the use of proprioceptive sensors, like inertial measurement units (IMUs) and motor joint states, to classify terrain. This approach aims to enhance the robustness and accuracy of rovers during extra-planetary missions by training Neural Networks (NNs) with proprioceptive data and integrating Uncertainty Quantification (UQ) techniques.

Innovations in Sensor Technology

Traditional TC research has focused on visual sensors like cameras and LADAR, but these are vulnerable to environmental conditions. Proprioceptive sensors, which measure internal states like IMUs and torque sensors, offer a more robust alternative. Recent studies have used these sensors with machine learning methods, showing promising results. However, they lacked quantifiable confidence in predictions. To address this, probabilistic UQ methods such as Monte Carlo Dropout, DropConnect, and Flipout have been integrated into DL models, enhancing their reliability for critical tasks.

Experimental Platform: The AsguardIV Rover

The AsguardIV rover, a hybrid leg-wheel rover designed for unstructured environments, was used to collect data. It features rimless wheels for better obstacle traversal and energy efficiency. Data was collected from various terrains mimicking lunar surfaces, including compact and loose sand, and rocky areas.

This rover has been used to
collect an array of data logs from trials executed across
various locations (Credit: De Lucas Álvarez et al., “Terrain Classification Enhanced With Uncertainty for Space Exploration Robots From Proprioceptive Data.”)

Data Collection and Processing

  • Sensors: IMU (6-feature vector), Joint data (12-feature vector), and fused IMU-joint data (18-feature vector).
  • Data Rate: 100 Hz, approximately six hours of data.
  • Splits: 70% training, 30% testing, with further validation splits.
  • Labelling: Terrains labelled for binary classification: rocky terrains as Class 0 (uneven and undeformable) and sand terrains as Class 1 (even and deformable).

The experimental sites for recording data include terrain that is mostly comprised of sand and rock (Credit: De Lucas Álvarez et al., “Terrain Classification Enhanced With Uncertainty for Space Exploration Robots From Proprioceptive Data.”)

Advanced Sequence Generation

The researchers have used the following two methods:

  1. Sliding Window: Extracting subsequences with predefined widths and steps.
  2. Sequence Subsampling: Selecting every fff-th time step to balance sample number and length.

High-Performance Neural Networks

Three NN architectures—CNN, LSTM, and CNN-LSTM—were optimized using Bayesian Optimization with Hyperband (BOHB).

Uncertainty Quantification Techniques

The researcher have integrated Three UQ techniques to enhance reliability:

  1. Monte Carlo Dropout (MC Dropout): Drops activations at inference time.
  2. DropConnect: Drops weights at inference time.
  3. Flipout: Uses variational inference for weight approximation.

Credit: Tesfu Assefa

Compelling Results

All networks were trained using Nvidia GeForce RTX 3070 and RTX 2070 graphics cards, employing the Adam optimization algorithm. The Bayesian optimizer had a maximum budget of 50 epochs. 216 BOHB studies were conducted, resulting in 6,480 full-budget candidates.

It is demonstrated as the advantage of integrating UQ techniques into TC models for exploration rovers, ensuring high-confidence outputs with low uncertainty, crucial for navigation safety in space missions. UQ-enhanced models, especially those using Monte Carlo Dropout, show superior performance and trustworthiness.

Future work will involve online testing in analogous scenarios and extending classification to more terrain types. We also aim to use multi-objective optimization techniques, incorporating entropy to generate robust and balanced models that optimise both performance and uncertainty.

Reference

Mariela De Lucas Álvarez et al., “Terrain Classification Enhanced With Uncertainty for Space Exploration Robots From Proprioceptive Data,” July 3, 2024, https://export.arxiv.org/abs/2407.03241.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter. 

Web3 Wars: GameFi Dapps Overtaken By Crypto AI Growth, What’s Next?

Posted on by Werner

TL;DR

  • AI Dapps now have 28% of blockchain activity, surpassing GameFi Dapps at 26%
  • GameFi is still growing: 4 million daily active wallets, up 79% month-over-month.
  • Q2 2024 saw $1.1 billion investment in blockchain gaming; July dropped significantly.

Introduction

Blockchain’s decentralized application (Dapp) industry has witnessed a significant shift in recent months, with artificial intelligence (AI) Dapps surpassing crypto gaming (GameFi) Dapps as the leading category for the very first time. 

This development, highlighted in the July 2024 DappRadar Games Report, means that gaming remains a robust and growing sector, but that AI-powered Dapps are the hotter current tech trends, and hold a bit more mindshare in the space right now. 

AI Dapps Take the Lead

Credit: DappRadar

  • In July 2024, the Dapp industry maintained its impressive milestone of over 15 million daily unique active wallets (dUAW) interacting with blockchain applications. 
  • However, the most striking development was the rise of DappRadar’s ‘Other’ category, which primarily consists of AI-based Dapps, to the top position with a 28% share of user activity. 
  • Gaming Dapps still demonstrated significant growth.The sector now represents 26% of DApp activity, engaging 4 million dUAW – a remarkable 79% increase from the previous month. 

The rise of AI Dapps reflects a broader trend in the tech industry, where artificial intelligence and machine learning are being integrated into sectors from finance to entertainment. In the blockchain space, AI applications are leveraging decentralized networks that provide services from decentralized AI computations by projects like Render to AI-driven data analysis and prediction markets.

Gaming Sector Plays On

Despite being overtaken by AI Dapps, the blockchain gaming sector continues to show strength and innovation. The report highlights several key developments and trends:

1. Blockchain Diversity: Ronin remains the leading blockchain for gaming activity, driven by popular titles like Pixels and Lumiterra. Other networks like opBNB, Oasys, NEAR, and Immutable zkEVM are also seeing significant engagement, showcasing the diverse ecosystem of blockchain gaming platforms.

Credit: DappRadar

2. Emerging Titles: New games like SERAPH: In the Darkness, which launched in mid-July, have quickly gained traction, indicating ongoing innovation and user interest in fresh gaming experiences.

3. NFT Trading: Despite a general decline in metaverse-based NFT collections, gaming NFTs continue to see active trading. Gods Unchained and Axie Infinity remain the most traded gaming NFT collections, while newer entries like Guild of Guardians are gaining popularity.

4. Cross-Platform Integration: The success of games published on major platforms like the App Store and Epic Games Store shows the growing acceptance of blockchain and NFT elements in mainstream gaming channels.

Investment Landscape

The investment climate for blockchain gaming and metaverse projects is a mixed picture. July 2024 saw the lowest investment level since Q3 2020 – just $23 million across three deals – but the preceding quarter (Q2 2024) was notably strong. 

Q2 marked the best quarter for blockchain gaming investments since Q3 2022, with $1.1 billion raised – a 314% increase from the previous quarter.

Key investments in Q2 included:

1. a16z Gaming Fund: Raised $600 million for game studios, infrastructure, and the Games x Consumer ecosystem.

2. Bitkraft Venture Fund: Secured $275 million for early-stage investments in gaming and interactive media companies.

3. Metaverse Projects: Significant investments in Baby Shark Universe ($34 million) and The Sandbox ($20 million) demonstrate ongoing interest in metaverse development.

These investments, focused on infrastructure and foundational development, suggest a strategic approach to enriching the Web3 gaming ecosystem. The contrast between the robust Q2 and the subdued July may indicate a temporary summer lull rather than a long-term trend.

Credit: Tesfu Assefa

GameFi Q2 Industry Snapshot and Analysis

1. User Engagement: Blockchain games remain strong in the Web3 industry, accounting for 26% of all Dapp activity and attracting 2.8 million active wallets daily. This persistent engagement suggests that gaming remains a key driver for Web3 adoption.

2. Blockchain Performance: Ronin has reclaimed the top spot among gaming blockchains, with a 100% increase to 1.9 million dUAW. This indicates the Ronin platform has strong user appeal. Newer platforms like Immutable zkEVM and opBNB grow rapidly.

3. Game Performance: Pixels leads the gaming landscape with 48 million unique wallets this quarter, demonstrating the enduring appeal of well-established titles. The success of newer entries like Guild of Guardians, especially following their mobile launch, shows the potential for growth through strategic platform expansions.

4. Metaverse Developments: Metaverse-based NFT collections saw a 29% decline in trading volume and a 21% drop in sales. Projects like Animoca Brands’ Mocaverse continue to dominate, capturing half of the trading volume. This suggests that while the metaverse concept may be experiencing reduced hype, established projects are maintaining their market positions.

5. Technological Advancements: The industry continues to focus on seamless gameplay experiences, investing in infrastructure and cross-chain compatibility. This focus on user experience is crucial to acquire and retain users.

6. Friend or Foe? The rise of AI Dapps presents both a challenge and an opportunity to GameFi. Gaming developers may need to integrate AI elements to stay competitive, potentially leading to more sophisticated and engaging gameplay experiences.

Looking Ahead

Despite the challenges, the blockchain gaming industry shows promising signs for future growth, especially if the 2024/2025 crypto bull run gets back on track.

New game launches are succeeding, and lots of blockchain platforms are getting traction. These trends show continuing innovation in tech, in gameplay, tokenomics, and user engagement strategies. There is potential crossover with traditional gaming platforms (as we’ve seen with Gunzilla and PS5), and interest from established tech companies. Broader industry collaborations will help the GameFi industry scale beyond what we’ve seen.

Significant investments in gaming funds and infrastructure projects continue to lay the groundwork for more sophisticated and scalable blockchain gaming experiences. Also, the rise of AI Dapps may lead to innovative hybrid models, combining elements of AI and gaming to create new, engaging experiences for users.

The rise of AI Dapps presents both a challenge and an opportunity for the GameFi sector to evolve and integrate new technologies. The capital is there. Substantial investments in Web3 gaming, infrastructure, emerging platforms, and cross-industry collaborations are laying the foundations for an exciting next phase of blockchain gaming evolution.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter. 

Combining Knowledge Graphs and Large Language Models

Posted on by YOSEF

Introduction

The growing interest in integrating Knowledge Graphs (KGs) and Large Language Models (LLMs) to enhance Natural Language Processing (NLP) applications is large. LLMs, such as BERT, GPT, and T5, have achieved state-of-the-art performance in various NLP tasks but still exhibit limitations, such as generating hallucinations (false information) and lacking domain-specific knowledge. KGs, on the other hand, provide structured and accurate information about entities and their relationships, which can complement LLMs’ capabilities. Even though LLMs are powerful NLP models trained on vast amounts of text data to understand and generate human language, they sometimes produce incorrect or nonsensical information, especially in domain-specific contexts.

So what are KGs and how can they improve LLMS? Knowledge Graphs (KGs): are databases that represent information in a structured format, with entities (nodes) connected by relationships (edges). They capture the semantics and interconnections between entities, making them valuable for enhancing the factual accuracy and domain knowledge of LLMs.

Methods for Enhancing LLMs with KGs

  1. Knowledge Injection: KAPING (Knowledge-Augmented Pre-trained Language Models) and DRAK (Dynamic Relational Attention Knowledge) are techniques that inject factual knowledge from KGs into LLMs to provide context and improve performance in tasks like zero-shot question answering. This helps LLMs generate more accurate and relevant responses by leveraging structured knowledge. 
  • Example: Injecting information about medical conditions and treatments from a medical KG into an LLM to improve its performance in medical question-answering tasks.
  1. Increasing Explainability: Methods like QA-GNN (Question Answering with Graph Neural Networks) and LMExplainer integrate KGs with LLMs to provide better interpretability and reasoning paths for the model’s outputs. By incorporating KGs, these methods can trace back the sources of information and explain how a particular answer was derived. 
  • Example: Using a KG to provide reasoning paths for answers generated by an LLM, making it easier to understand the logic behind the model’s responses in a legal context.
  1. Semantic Understanding: Approaches like LUKE (Language Understanding with Knowledge-based Embeddings) and R3 (Relational Reasoning for Reading Comprehension) enhance LLMs by adding semantic understanding and entity embeddings from KGs. This allows LLMs to better comprehend the relationships between entities and generate more coherent and contextually appropriate responses.
  • Example: Enhancing an LLM with embeddings from a geographic KG to improve its understanding of geographical entities and relationships in tasks like location-based question answering.

Methods for Enhancing KGs with LLMs

  1. Temporal Forecasting: LLMs are used to predict future facts in Temporal Knowledge Graphs (TKGs) by understanding the semantic meaning of entities and relationships over time. This helps in forecasting events and trends based on historical data stored in KGs.
  • Example: Using an LLM to predict future business trends by analyzing historical data on company performances and market conditions stored in a TKG.
  1. Knowledge Graph Construction: LLMs assist in the construction of KGs by performing tasks like relation extraction, entity recognition, and property identification. This automates the process of building and updating KGs, making it more efficient and accurate.
  • Example: Using an LLM to extract relationships between scientific concepts from research papers and incorporate them into a scientific KG.

Brief Thematic Analysis 

There is a complementary relationship between KGs and LLMs, and their integration can significantly improve the performance and trustworthiness of AI applications. The key themes include:

  1. Accuracy and Reliability: Integrating KGs with LLMs improves the factual accuracy and reliability of the models by providing structured and verified information.
  2. Explainability and Interpretability: Methods that combine KGs and LLMs enhance the explainability of model outputs, making it easier to understand and trust the generated information.
  3. Efficiency and Automation: Using LLMs to construct and maintain KGs automates the process, reducing the time and effort required for manual updates.

Credit: Tesfu Assefa

Conclusion

The integration of KGs and LLMs offers a promising approach to addressing the limitations of LLMs and enhancing the capabilities of KGs. It is a discussion of current and future challenges in this field, providing insights for researchers and practitioners interested in this area. The key takeaways include:

  1. Mutual Benefits: KGs provide structured and domain-specific knowledge to LLMs, reducing hallucinations and improving accuracy, while LLMs enhance the construction and updating of KGs, making the process more efficient.
  2. Research Directions: Future research should focus on developing more sophisticated methods for integrating KGs and LLMs, addressing challenges such as scalability, dynamic updates, and context awareness.
  3. Practical Applications: The integration of KGs and LLMs has significant implications for various NLP applications, including question answering, information retrieval, and knowledge management.

Key Points

  • Knowledge Injection: Techniques like KAPING and DRAK use KGs to provide additional context and facts to LLMs, improving their performance in tasks such as zero-shot question answering.
  • Explainability: Methods like QA-GNN and LMExplainer integrate KGs with LLMs to provide better interpretability and reasoning paths for the models’ outputs.
  • Semantic Understanding: Approaches like LUKE and R3 enhance LLMs by adding semantic understanding and entity embeddings from KGs.
  • Temporal Forecasting: LLMs are used to predict future facts in Temporal Knowledge Graphs (TKGs) by understanding the semantic meaning of entities and relationships over time.
  • KG Construction: LLMs assist in the construction of KGs by performing tasks like relation extraction and property identification, making the process more automated and accurate.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter. 

Ten books to read to understand technology and change

Posted on by David Wood

Looking for a guidebook to help you navigate our changing world?

Has the pace of change in the 21st century got you disorientated? 

Let me draw your attention to ten books I’ve read recently. They each deal with the development of technology in the present and the near future, and its effects on society. Each of them are eye-opening and thought-provoking in their own ways. Indeed, they might change your life path, so beware!

Credit: David Wood

1) Power, Sex, Suicide: Mitochondria and the meaning of life. By Nick Lane.

Fascinating account of the remarkable (and unlikely) evolutionary journey from non-life to modern warm-blooded life. With plenty of insights along the way regarding energy, sex, aging, and death. You’ll wonder why you never knew about this before.

2) Methuselah’s Zoo: What nature can teach us about living longer, healthier lives. By Steven Austad.

A different view regarding what animals can teach us about aging. Many animals live longer, healthier lives than any simple theory would predict – this book explains why and considers the implications for human aging, and for what kind of studies rejuvenation researchers should prioritize.

3) Eve: How the female body drove 200 million years of human evolution. By Cat Bohannon.

Milk. The womb. Menopause. Perception. Tools. Voice. The brain. Love. When you look at the long span of evolution from a female perspective, many things fall into place in an inspiring new way. A welcome reminder that our approach to science often suffers from being male-centric.

4) We Are Electric: The new science of our body’s electrome. By Sally Adee.

A look at biology from a fascinating alternative angle. The electricity throughout our bodies is involved in more processes than we previously thought. Move over genome, epigenome, and biome: make way for the electrome.

5) Sentience: The invention of consciousness. By Nicholas Humphrey.

Why did evolution give rise to phenomenological consciousness? How can we detect and assess consciousness throughout the animal kingdom? And what are the implications for AIs that might be sentient? Lots of captivating biographical asides along the way.

Credit: Tesfu Assefa

6) The Other Pandemic: How QAnon contaminated the world. By James Ball.

Evolution has produced not just intelligence and beauty but also viruses and other pathogens. Mental pathogens (‘memes’) have lots in common with their biological analogues. That’s one reason why the whole world may be on the point of going crazy.

7) The Deadly Rise of Anti-Science: A scientist’s warning. By Peter Hotez.

Part of the growing wave of social irrationality is a determined virulent opposition to the patient methods and hard-won insights of science. Millions have already died as a result. There may be worse ahead. What lies behind these developments? And how can they be parried?

8) End Times: Elites, counter-elites, and the path of political disintegration. By Peter Turchin.

Can we ever have a science of history? Is that idea a fantasy? This book argues that there are important patterns that transcend individual periods of revolutionary turmoil. However, there’s no inevitability in these patterns, provided we are wise and pay attention. You’ll never look at history the same way again.

9) The Coming Wave: Technology, power, and the 21st century’s greatest dilemma. By Mustafa Suleyman.

Current debates about the safety of powerful AI systems should be understood in wider context: economic, political, and historical context. Following a full diagnosis, a ten-stage multi-level plan provides some grounds for optimism.

10) Uncontrollable: The threat of artificial superintelligence and the race to save the world. By Darren McKee.

Will powerful AI systems pose catastrophic risks to humanity? Are you, as an individual, helpless to reduce these risks? Read this book to find out. Written compellingly, with particular clarity.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter. 

A Paean for Privacy and the Accidental Authoritarian Tomorrow

Posted on by SB Fisher

I lost my phone the other day. I retraced my steps around town before giving up and going home – where I could fire up Google on my laptop and quickly locate the device. Not just to a local area, but to the exact hedgerow where it had fallen out of my pocket – right down to the individual shrub. Blessed be! With a quiet and fervent prayer of thanks to Google, I got on with my day – safe in the knowledge Google was tracking every movement I made.

It didn’t take long for my gratitude to turn sour. Of course, I’ve known all along: Google can track me. Law enforcement have been pinging cell towers to track suspects since the 1990s. Yet something about the precision strike on my location felt different. Google doesn’t just know what block I’m on – it knows which couch I’m sitting on.

The End of Individual Freedom

It’s not just Google. An industry handles the data that Google collects, and it’s shared with the government and military under the PRISM project. It’s hard to have any faith that SHA-256 encryption or and Google’s cybersecurity practices mean our data is only handled in just and proper ways. Only recently the company deleted an entire pension fund. Oops.

This is not a polemic against Google, they’re just a useful example. Surveillance capitalism powers most major tech corporations, their market cap riding on the data they process and harvest every centisecond: location data, online interactions, and creepy psychological profiles based on that. 

When I praised the ability to find my device, glad that Google was watching, I exemplified the attitude of consumers who have for decades now ceded autonomy to zaibatsus in exchange for the services they provide. Yet the relentless data breaches, lawsuits, and system outages are starting to make society question, rightly, how senseless this ultimately was. 

Are you comfortable having your preferences tracked to have good and appropriate products advertised to you? Are you any naturally less comfortable with the idea of major corporations being able to construct a better profile of you than your psychotherapist could? Privacy is now a battleground of the individual against the corporation. Our future society depends on the battle. 

Credit: Tesfu Assefa

Wiretapped

You – dear reader – no longer have the right, or indeed the ability, to protect your personal privacy. Google quite literally knows more about you than your family and friends do. After all, does your husband or wife know where you are all the time? And if they had the ability to find out, how comfortable would you be if they kept on tracking you? Probably not; you’d probably file for divorce. Google – and their NSA partners – already know. As would anyone who hacked their systems, or any Google employee with the right access. 

This spectre of panoptic rule by corporations is somewhat diluted by the sheer weight of information we produce. It takes sophisticated algorithms to rapidly index the copious amounts of information being vomited forth every second and make it, as per Google’s credo ‘organised and accessible’. Without the AI, invasions of privacy would have to be tailored, slowing our corporate adversaries. However, the advent of LLMs, AIs, better data indexing tools, and ever more sensory equipment dotted around thanks to the “Internet of Things’ – such complete intrusive oversight of our lives, all our lives, is on the verge of complete reality. 

“The USA Patriot Act of 2001 authorized unprecedented surveillance of American citizens and individuals worldwide without traditional civil liberties safeguards” – it was a scandal at the time and it’s gotten so much worse since. The size, scale and sense that our lives are being recorded has only grown. Do you really believe Alexa isn’t listening? Do you think that our phones are not recording? Do you think Microsoft bought Discord because of the revenue it generates? Many people have had the Baader Meinhof effect where they talk to a friend about an anime series and then see merchandise for that series advertised to them. If that’s happened to you I am sorry to say you’ve been wiretapped. 

Authoritarian Angst

We’ve all been wiretapped. Constantly. For years. We broadly put up with it  with various excuses: “the data I produce isn’t actionable”, “the NSA wouldn’t do anything bad with it”, “we need to stop terrorists”. But the truth – practically and neurologically – is that we addicted to the devices that surveil us. The rise of AI means all that data is actionable. An interviewer will pay a tech corporation for an online profile of you for every job you apply to, and have an LLM review all your recent online activity for red flags. A bank may refuse you a loan because your phone location went to the casino twice this month. Police may visit you if your political leanings are suspect.

This is just the tip of the authoritarian spear. Western propaganda points fingers at China as a fearful vision of an authoritarian future, where facial recognition is common. But do they realise the megacorps of mostly the USA that are architected modern digital surveillance? I am delighted that you found my phone, Google – all it cost me was everything. It is essential to restore privacy to technology, and to create systems where the user controls their data, their applications and their devices – but I fear we may be too late.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.