Google Play Steps Up To Regulate Offensive AI Content and Disruptive Notifications

As the proliferation of AI-based apps continues to escalate, Google Play has announced a policy update aimed at curbing the negative impact of such apps by introducing certain changes in its app development and publishing process. The move, set to be enforced starting early next year, is targeted at AI apps that generate potentially offensive content, a problem that has seen a significant surge in recent times. The new policy will necessitate developers to incorporate a feature in their apps that allows users to promptly report or flag any offensive AI-generated content.

The policy change can be largely attributed to some disturbing instances wherein users manipulated apps to create inappropriate or misleading content. A glaring example of this was when the app Lensa was tricked into creating not-safe-for-work imagery. Another app, Remini, which uses AI to enhance headshots, was also criticized for inappropriately altering women's bodies. Even tech giants like Microsoft and Meta had to face the heat when their AI tools were exploited to bypass safeguards and generate offensive imagery.

The consequences of such misuse can be far-reaching and gravely serious. There have been incidents where pedophiles took advantage of open-source AI tools to create child sexual abuse material (CSAM) on a massive scale. With the imminent elections, there are concerns about the rise of deepfakes – fake images created by AI, intending to mislead voters. Google, therefore, emphasizes that all AI content generator apps must adhere to its existing developer policies, which explicitly ban restricted content including CSAM and those that facilitate deceitful behavior.

Google isn't just stopping at the policy level; it's also introducing changes in app permissions. Under the revised policy, apps will be allowed access to photos and videos only if they're directly related to their functionality. In addition, Google Play will conduct an additional review for apps requesting broad photo and video permissions. Moreover, the frequent, intrusive full-screen notifications used by many apps to promote paid subscriptions or offers will be limited to high-priority needs only.

This move by Google to regulate AI apps and chatbots is a leap in the right direction to ensure a safer and more user-friendly experience. Although Apple has typically been the pioneer when it comes to implementing new rules to control app behavior, it has yet to formalize a policy regarding AI or chatbots. Nonetheless, these changes by Google are a testament to the tech giant's commitment to safeguard its users from potential misuse of technology. The policy updates are being rolled out today, giving developers until early 2024 to implement the necessary changes to their apps.