DRAFT 

Actors: Holder- Issuer- Verifier- Wallet(Agent)- Governing Authority

General Risks

Deepfake and Identity Spoofing

• Risk: Generative AI can create highly realistic fake audio, video, or images, enabling attackers to bypass biometric authentication or impersonate legitimate users. It would be a significant challenge for online(video) authentication. The rapid advancement of AI-driven deepfake technologies, can undermine biometric authentication mechanisms such as facial recognition.
• scenariosuse cases:

  • Using video authentication to log in to the wallet could create a vulnerability.

  • Facial authentication is required to ensure that the verifiable credential (VC) cannot be misused by someone else.

  • Facial Biometric authentication is necessary whenever an action must be performed specifically by the VC owner and not by anyone else who may have access to the wallet. e.x. to grant power of attorney

     
• Solution: Implement deepfake detection tools, multi-factor authentication (MFA), and robust identity verification processes to reduce reliance on single biometric factors. Also update continuously the forgery detection algorithms.
• Complexity of Solution
• Obstacles
• Affected Group: End user/ 
• Active Actor of mitigation

Prompt Injection and Policy Manipulation 

• Risk: Since policies are enforced through cryptographic protocols rather than natural language interpretation, making prompt injection harder. But if the wallet uses AI-driven assistants or automated decision-making (e.g., for verifying credentials or guiding users), attackers can craft malicious prompts to manipulate the AI’s logic.  

  Even if the wallet itself is secure, any connected AI-based helpdesk or verification service could be exploited via prompt injection.

• Solution: Apply prompt hardening techniques, context isolation, and strict input validation. Use allowlists/denylists and sandbox testing for untrusted inputs.

Data Leakage and Membership Inference*

• Risk: If identity-related data is used to train AI models, attackers may infer sensitive attributes or reconstruct original data through model inversion or membership inference attacks. 

  If the wallet uses AI services (e.g., for fraud detection, identity verification, or UX personalization), sensitive identity data might be exposed during model training or inference. If wallet operations involve external AI APIs, data could leak through logs or model updates. Even if raw data isn’t shared, patterns in queries or metadata could allow attackers to infer user attributes. e.g your AI tools on your mobile phone have access to your ID wallet!!

  
  

  If AI is integrated for convenience (e.g., chatbots or automated KYC), and those models access identity data, leakage and inference risks reappear. Metadata (timestamps, transaction patterns) can still be exploited for inference even if credentials are protected.

• Solution: Enforce data minimization,
• Enforce data minimization→ wallet selective dislousure is already there to solve ths problem but by using AI data can reveal werden. (is it really a solution?)
• segregate sensitive datasets
,
• adopt privacy-preserving training methods* (e.g., differential privacy), and secure the entire data lifecycle.

Misinformation and Social Engineering

...

• Risk: Many current digital identity security systems lack the scalability and accuracy required to handle large volumes of users and increasingly sophisticated AI-based attacks. This limitation poses a significant challenge for identity wallets operating at national or cross-border scale.
• Solutions:

  • Deployment of scalable and resilient system architectures

  • Use of AI to automate threat detection and response

  • Continuous improvement of algorithmic accuracy under high-load conditions [2]

Risks as AI-as-a-Service

GenAI: here we mean using GenAI outside the wallet (AI-as-a-Service)

Implicit data leakage (even without “sending data”)

Even if you think you’re only sending:

...

• user attributes

• behavior patterns

• service usage profiles

This is called inference leakage. Over time, the AI provider can reconstruct who you are and what you’re doing — without seeing raw identity data.

Loss of user sovereignty

When AI runs outside the wallet:

...

Result: The wallet becomes a UI, not an agent.

This quietly breaksself-sovereign identity principles.

Policy manipulation & dark negotiation

External AI can:

• bias disclosure decisions

• “optimize” for platform goals

• subtly over-disclose to reduce friction

...

• optimization objectives ≠ user interests

This is algorithmic coercion, not a bug. + Explainable Generative AI*

Prompt and context retention

Most AI services:

• log prompts

• retain context

• reuse data for tuning or monitoring

...

Once logged: You can’t revoke it.

Correlation across wallets and services

A single AI provider serving many wallets can:

...

This recreates centralized identity — without consent.

In addition, if the AIaaS provider experiences an outage, millions of users could be locked out of essential services simultaneously.

Regulatory and jurisdictional drift  

External AI services may:

...

Data may be stored in servers under foreign laws, complicating compliance with national regulations like GDPR and creating legal uncertainty.  

Model hallucination becomes a security risk

Inside a wallet:

• AI mistakes are bounded
  Outside:
  

• hallucinated policy interpretations

• incorrect legal assumptions

• wrong proof selection

...

Hallucination here is not UX noise — it’s identity damage.

"Black Box" Opacity

Complex AI models are not transparent. Users can be denied access (e.g., false non-match) without a clear, explainable reason or recourse.

Algorithmic Bias & Discrimination

AI models can inherit biases, leading to unfair denials of access for specific demographic groups. The system is also vulnerable to adversarial attacks designed to fool it.

*

Membership Inference (or Membership Inference Attack, often shortened to MIA) is a type of privacy attack against machine‑learning models. In this attack, someone (an attacker) tries to figure out whether a specific data sample was part of the model’s training data. In simple words: The attacker wants to know “Was this person’s data used to train the model?” If the attacker can guess this correctly, they can learn private information about individuals.

Privacy-preserving training methods →  Privacy‑preserving training methods are techniques used in machine learning and AI to ensure that sensitive information from the training data cannot be reconstructed, identified, or leaked, while still allowing the model to learn useful patterns.like :

...

1. A. Golda et al., "Privacy and Security Concerns in Generative AI: A Comprehensive Survey," in IEEE Access, vol. 12, pp. 48126-48144, 2024, doi: 10.1109/ACCESS.2024.3381611.  →  https://ieeexplore.ieee.org/document/10478883
2. 'THE EVOLUTION OF IDENTITY SECURITY IN THE AGE OF AI: CHALLENGES AND SOLUTIONS ', International Journal of Computer Engineering and Technology (IJCET)  Volume 16, Issue 1, Jan-Feb 2025, pp. 2305-2319, Article ID: IJCET_16_01_165 Available online at https://iaeme.com/Home/issue/IJCET?Volume=16&Issue=1 ISSN Print: 0976-6367; ISSN Online: 0976-6375; Journal ID: 5751-5249 Impact Factor (2025): 18.59 (Based on Google Scholar Citation) DOI: https://doi.org/10.34218/IJCET_16_01_165 

...