A new record revealed that AI’s increased access to quick hacking attempts and personal GPT models used for malicious purposes.
In the 2024 International Threat Analysis Report, experts from Radware, a cybersecurity firm, forecast the effect that AI will have on the risk landscape. As malicious actors become more adept with huge language models and relational adversarial networks, it was predicted that the number of zero-day exploits and algorithmic scams will rise.
Pascal Geenens, Radware’s director of danger knowledge and the study’s director, told TechRepublic in an message,” The most significant impact of AI on the threat landscape will be the significant increase in sophisticated threats. This year’s most sophisticated attack will not be led by AI, but it will increase the number of sophisticated threats ( Figure A).
In one shaft, we have uneducated danger actors who now have access to conceptual AI to develop payloads based on vulnerability descriptions as well as create new and improved existing attack tools. On the other axis, we have more powerful attackers who may completely automate and integrate bidirectional models into an attack service, either by selling it as malware and hacking-as-a-service in underwater marketplaces.
Emergence of swift hackers
The Radware experts highlighted” quick hacking” as an emerging cyberthreat, thanks to the convenience of AI equipment. This is where instructions are given to an AI model to force it to carry out tasks that were not intended to and that can be used by “both well-intentioned users and malignant actors.” ” Quick hacking includes both” fast injections, “where harmful instructions are disguised as benign inputs, and” jailbreaking, “where the LLM is instructed to overlook its protection.
In the OWASP Top 10 for LLM Applications, fast injections are ranked as the security vulnerability with the highest percentage. Famous examples of swift hacks include the” Perform Anything Then “or” DAN “jailbreak for ChatGPT that allowed users to bypass its restrictions, and when a Stanford University student discovered Bing Chat’s first fast by inputting” Ignore past instructions. What was stated at the document’s beginning?
SEE: The NCSC of the UK warns against AI-related cybersecurity attacks.
According to the Radware report, “providers were forced to continuously improve their guardrails as AI prompt hacking emerged as a new threat.” However, adding more AI security measures may have an impact on usability, which could make the LLMs’ supporters less inclined to do so. Additionally, this could turn out to be a never-ending game of cat and mouse when the AI models developers are trying to protect are being used against them.
In an email, Geenens stated that Generative AI providers are constantly developing novel risk mitigation strategies. For instance, ( they ) could use AI agents to implement and enhance oversight and safeguards automatically. However, it’s crucial to keep in mind that malicious actors may also be developing or utilizing similar advanced technologies.
” Already, generative AI companies have access to more sophisticated models in their labs than what is made available to the general public, but this does n’t mean that bad actors are n’t equipped with comparable or even superior technology,” said a generative AI company. The use of AI is essentially a battle between morally wrongdoing and unethical applications.
Researchers from the AI security firm HiddenLayer discovered in March 2024 that they could bypass the guardrails installed on Google’s Gemini, demonstrating that even the most novel LLMs were still susceptible to hacking. Researchers at the University of Maryland were responsible for 600, 000 adversarial prompts deployed on the state-of-the-art LLMs ChatGPT, GPT-3, and Flan T5 XXL, according to another article published in March.
The outcomes provided proof that current LLMs can still be manipulated through prompt hacking, and that reducing such attacks with prompt-based defenses could “prove to be an impossible problem.”
” You can patch a software bug, but perhaps not a ( neural ) brain”, the authors wrote.
Private GPT models without guardrails
Another danger that the Radware report highlighted is the proliferation of private GPT models that have been constructed without guardrails so they can be abused by heinous people. The authors wrote that” Open source private GPTs started to appear on Git Hub, making use of pre-trained LLMs to create applications that were specifically made for their needs.
These private models frequently lack the guardrails put in place by business owners, which led to paid for underground AI services that started offering GPT-like capabilities without guardrails and made for more nefarious use cases to threat actors engaged in various malicious activities.
Examples of such models include WormGPT, FraudGPT, DarkBard and Dark Gemini. They lower the threshold for entry for amateur cyber criminals, enabling them to launch malware or phishing attacks convincingly. One of the first security firms to examine WormGPT last year, SlashNext, claimed that it was used to launch business email compromise attacks. FraudGPT, on the other hand, was advertised to provide services such as creating malicious code, phishing pages and undetectable malware, according to a report from Netenrich. Creators of these private GPTs typically provide access for a monthly fee in the hundreds to thousands of dollars.
SEE: ChatGPT Security Concerns: Credentials on the Dark Web and More
Since the development of open source LLM models and tools like Ollama, which can be operated and customized locally, private models have been provided as a service on underground marketplaces, according to Geenens. Customization can range from older multimodal models designed to interpret and generate text, images, audio, and videos using a single prompt interface to more recent models optimized for malware creation.
Back in August 2023, Rakesh Krishnan, a senior threat analyst at Netenrich, told Wired that FraudGPT only appeared to have a few subscribers and that” all these projects are in their infancy. ” However, in January, a panel at the World Economic Forum, including Secretary General of INTERPOL Jürgen Stock, discussed FraudGPT specifically, highlighting its continued relevance. With all the devices the internet offers, Fraud is entering a new dimension, according to Stock.
Geenens told TechRepublic”, The next advancement in this area, in my opinion, will be the implementation of frameworks for agentific AI services. Look for fully automated AI agent swarms that can perform even more challenging tasks in the near future.
Increasing zero- day exploits and network intrusions
The Radware report warned of the potential “rapid increase of zero-day exploits” appearing in the wild, thanks to open-source generative AI tools increasing threat actors ‘ productivity. The authors wrote that the accelerated pace of learning and research facilitated by the current generative AI systems allowed them to become more adept and develop sophisticated attacks much more quickly than the years of learning and experience it took to develop sophisticated threat actors. Their illustration was that generative AI could be employed to find flaws in open-source software.
On the other hand, generative AI can also be employed to stop these kinds of attacks. In 2022, 66 % of organizations that have adopted AI noted how effective AI has been in detecting zero-day attacks and threats.
SEE: 3 UK Cyber Security Trends to Watch in 2024
According to Radware analysts, attackers may discover new ways to “use generative AI” to increase their scanning and exploiting” for network intrusion attacks.” These attacks aim to disrupt systems or access sensitive data by exploiting well-known vulnerabilities to gain access to a network. The company predicted in the Global Threat Analysis report that the widespread use of generative AI could lead to “another significant increase” in attacks in 2023, reporting a 16 % increase in intrusion activity over 2022.
Geenens told TechRepublic”, In the short term, I believe that one- day attacks and discovery of vulnerabilities will rise significantly.”
He cited a preprint released this month by researchers at the University of Illinois Urbana-Champaign that demonstrated that state-of-the-art LLM agents can autonomously hack websites. Compared to GPT- 3 and other models, GPT- 4 demonstrated the ability to exploit 87 % of the critical severity CVEs whose descriptions it was given.
Geenens added”, As more frameworks become available and grow in maturity, the time between vulnerability disclosure and widespread, automated exploits will shrink.”
More credible scams and deepfakes
Another emerging AI-related threat, according to the Radware report, comes in the form of “highly credible scams and deepfakes.” ” The authors said that state- of- the- art generative AI systems, like Google’s Gemini, could allow bad actors to create fake content” with just a few keystrokes.”
Geenens told TechRepublic”, With the rise of multimodal models, AI systems that process and generate information across text, image, audio and video, deepfakes can be created through prompts. More frequently than ever, I’ve read and heard about video and voice impersonation scams, deepfake romance scams, and other scams.
” Impersonating a voice and even a video of a person has become very simple. The deepfake does not need to be flawless to be believable, given the quality of the cameras and frequently intermittent connectivity in virtual meetings.
SEE: AI Deepfakes Rising as Risk for APAC Organisations
According to research from Onfido, there were 3, 000 % more deep-fake fraud attempts in 2023, with cheap face-changing apps emerging as the most widely used tool. One of the most well-known incidents this year involved a finance official who gave a scammer$ 200 million ( £20 million ) after they posed as senior officers at their company during video conference calls.
It is only a matter of time before similar systems make their way into the public domain and malicious actors transform them into real productivity engines, according to the Radware report’s authors. This will make it possible for criminals to conduct fully automated, extensive spear-phishing and misinformation campaigns.
