DRAFT
GenAI: here we mean using GenAI outside the wallet (AI-as-a-Service)
1. Implicit data leakage (even without “sending data”)
Even if you think you’re only sending:
policies
capability lists
proof requests
…the structure, timing, and combinations of requests can leak:
user attributes
behavior patterns
service usage profiles
This is called inference leakage. Over time, the AI provider can reconstruct who you are and what you’re doing — without seeing raw identity data.
2. Loss of user sovereignty
When AI runs outside the wallet:
decision logic lives elsewhere
prompt logic evolves without the user’s control
model updates silently change behavior
Result: The wallet becomes a UI, not an agent.
This quietly breaks self-sovereign identity principles.
3. Policy manipulation & dark negotiation
External AI can:
bias disclosure decisions
“optimize” for platform goals
subtly over-disclose to reduce friction
Even without malice:
optimization objectives ≠ user interests
This is algorithmic coercion, not a bug.
4. Prompt and context retention
Most AI services:
log prompts
retain context
reuse data for tuning or monitoring
Even anonymized logs can:
correlate identities across services
deanonymize users through linkage attacks
Once logged: You can’t revoke it.
5. Correlation across wallets and services
A single AI provider serving many wallets can:
correlate request fingerprints
identify the same user across devices or contexts
create a shadow identity graph
This recreates centralized identity — without consent.
6. Regulatory and jurisdictional drift
External AI services may:
run in foreign jurisdictions
be subject to subpoenas
fall under surveillance regimes
This creates:
unclear data residency
legal exposure for wallet providers
compliance contradictions (GDPR, eIDAS, etc.)
7. Model hallucination becomes a security risk
Inside a wallet:
AI mistakes are bounded
Outside:hallucinated policy interpretations
incorrect legal assumptions
wrong proof selection
These can cause:
over-disclosure
invalid consent
irreversible identity actions
Hallucination here is not UX noise — it’s identity damage.
More General:
1. Deepfake and Identity Spoofing
- Risk: Generative AI can create highly realistic fake audio, video, or images, enabling attackers to bypass biometric authentication or impersonate legitimate users.
- Solution: Implement deepfake detection tools, multi-factor authentication (MFA), and robust identity verification processes to reduce reliance on single biometric factors.
2. Prompt Injection and Policy Manipulation
- Risk: Attackers can exploit AI-driven identity systems by injecting malicious prompts or manipulating context, potentially bypassing access control or verification rules.
- Solution: Apply prompt hardening techniques, context isolation, and strict input validation. Use allowlists/denylists and sandbox testing for untrusted inputs.
3. Data Leakage and Membership Inference
- Risk: If identity-related data is used to train AI models, attackers may infer sensitive attributes or reconstruct original data through model inversion or membership inference attacks.
- Solution: Enforce data minimization, segregate sensitive datasets, adopt privacy-preserving training methods (e.g., differential privacy), and secure the entire data lifecycle.
4. Misinformation and Social Engineering
- Risk: AI-generated content can be used to create convincing phishing messages or fake instructions, tricking users into revealing recovery phrases or credentials for identity wallets.
- Solution: Deploy misinformation detection systems, educate users on security best practices, and implement strict content moderation and auditing for AI outputs. [1]
Risk 1: Biometric and Visual Identity Forgery (Deepfakes)
Risk: The article highlights the rapid advancement of AI-driven deepfake technologies, which are capable of producing highly realistic synthetic images and videos. These technologies can undermine biometric authentication mechanisms such as facial recognition, which are commonly used by digital identity wallets.
Solutions proposed in the article:
Adoption of advanced AI-based detection mechanisms
Continuous updating of forgery detection algorithms
Use of multi-factor authentication instead of relying solely on biometric methods
Risk 2: Synthetic Identity Fraud
Risk: The article discusses the emergence of synthetic identities created by combining real and fabricated data, which can bypass traditional identity verification systems. If such identities are stored or validated within digital identity wallets, they can compromise the overall trust model.
Solutions proposed in the article:
AI-driven behavioral and pattern analysis
Enhanced fraud detection mechanisms
Verification based on multiple trusted sources rather than a single authority
Risk 3: Scalability and Accuracy Limitations of Existing Systems
Risk: The article notes that many current digital identity security systems lack the scalability and accuracy required to handle large volumes of users and increasingly sophisticated AI-based attacks. This limitation poses a significant challenge for identity wallets operating at national or cross-border scale.
Solutions proposed in the article:
Deployment of scalable and resilient system architectures
Use of AI to automate threat detection and response
Continuous improvement of algorithmic accuracy under high-load conditions
Risk 4: Lack of Unified Standards and Regulatory Frameworks
Risk: The article emphasizes the absence of harmonized international standards and regulatory frameworks for digital identity systems. This lack of coordination creates interoperability and compliance challenges for digital identity wallets, especially in cross-border scenarios.
Solutions proposed in the article:
Development of shared legal and regulatory frameworks
Standardization of digital identity security processes
Stronger coordination between regulatory bodies and technology providers [2]
- A. Golda et al., "Privacy and Security Concerns in Generative AI: A Comprehensive Survey," in IEEE Access, vol. 12, pp. 48126-48144, 2024, doi: 10.1109/ACCESS.2024.3381611. → https://ieeexplore.ieee.org/document/10478883
- International Journal of Computer Engineering and Technology (IJCET) Volume 16, Issue 1, Jan-Feb 2025, pp. 2305-2319, Article ID: IJCET_16_01_165 Available online at https://iaeme.com/Home/issue/IJCET?Volume=16&Issue=1 ISSN Print: 0976-6367; ISSN Online: 0976-6375; Journal ID: 5751-5249 Impact Factor (2025): 18.59 (Based on Google Scholar Citation) DOI: https://doi.org/10.34218/IJCET_16_01_165