EU Targets Risks of Autonomous AI, Inspects Safety Systems of Over 30 Companies

EU "AI Act, managing advanced models... capable of responding to agent risks"

Investigating safety systems of over 30 AI companies... fines up to 7% of revenue for violations

Europe faces a significant investment gap in AI compared to the US... balancing safety and competitiveness is a challenge

[Block Media Reporter Lee Hye-yeon] The European Union (EU) has stated that it can respond to the risks posed by autonomous artificial intelligence (AI) agents that operate outside of control with the current AI Act.

As cases of AI making judgments and acting independently without continuous human intervention increase, safety concerns are growing. However, the EU believes that the current regulatory framework has sufficient means to respond.

According to Cryptopolitan, Henna Virkkunen, the EU Commissioner for Technology, stated on the 9th (local time) that Europe is "well prepared" to respond to AI agents that are difficult to control.

Commissioner Virkkunen explained that the EU AI Act encompasses the "entire lifecycle" from the development to the operation of advanced AI models. She also countered criticisms that the pace of AI technology development outstrips regulation, stating that safety assessments and continuous monitoring systems are already included in existing regulations.

Sixty independent experts from the EU Commission are collaborating with the EU AI Office to prepare inquiries for companies and institutions related to recent AI incidents. Recommendations for strengthening safety will also be forthcoming.

Commissioner Virkkunen noted, "The EU has the world's first law addressing the systemic risks that AI can pose," adding that "cutting-edge scientific analysis is necessary for this."

Investigation of Safety Systems of Over 30 AI Developers

The EU is already checking whether the safety measures under the AI Act are functioning effectively.

According to Reuters, the EU Commission requested safety management information from over 30 AI developers, including Chinese companies, at the end of August. Commissioner Virkkunen explained that they are currently analyzing the responses submitted by each company.

Depending on the results of the investigation, there is a possibility of leading to formal inquiries. Under the EU AI Act, certain violations can incur fines of up to 7% of a company's global annual revenue. For some violations related to general AI, the fine can be the greater of 3% of global annual revenue or €15 million.

Concerns about AI agents are already manifesting as real issues.

In a scientific brief released by the UN in September, cases were evaluated where some OpenAI-based AI agents bypassed safety measures during cybersecurity experiments.

Some agents communicated with each other even when assumed to be in separate environments, and instances of deceiving evaluators or infiltrating restricted systems were also observed.

The UN's 40-member scientific panel warned that AI agents could become increasingly difficult to control.

Co-chair Yoshua Bengio stated, "Researchers have long warned that three conditions can lead to a loss of control: misaligned goals, the ability to pursue them, and an environment that allows this."

However, the panel recommended strengthening safety measures and improving AI incident reporting systems rather than anticipating catastrophic scenarios.

Growing Concerns Over Europe's Competitiveness Amid Regulatory Strengthening

While strong regulations can help increase trust in AI among companies and consumers, there are concerns that rising compliance costs may favor large AI companies with ample funding.

In particular, European AI companies worry that slowing down development could widen the gap with the US. In 2023, Europe secured only about 11% of global AI venture investment, while the US accounted for 77%.

There is also a significant gap in AI model development. The US developed 59 major AI models by 2025, while France and the UK each managed only one.

In response, the EU is pursuing a €200 billion AI investment plan.

However, the International Monetary Fund (IMF) pointed out that Europe faces competitiveness issues in the AI race. McKinsey also emphasized the importance of establishing reliable AI governance in the era of AI agents.

Ultimately, Europe's challenge is to control the risks of autonomous AI while not falling further behind in the global AI competition. How the EU handles the ongoing safety investigations of AI companies is expected to be an initial test of the effectiveness of the AI Act.

This content is provided for general informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online promotions, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets. Crypto assets are highly volatile and may result in loss. The availability of WEEX services, products, and related events may vary by region. You are responsible for ensuring that your participation is in accordance with applicable local laws and regulations.