Breaking the Barrier: The Impact of Unauthorized Access to Powerful AI Language Models like ChatGPT

EXECUTIVE SUMMARY

OpenAI has recently launched new plugins for ChatGPT, which is a language model based on GPT-3.5 architecture. These plugins have expanded ChatGPT’s capabilities, and combined with the launch of the latest GPT-4 model, ChatGPT has gained even more power. However, one of the downsides of ChatGPT has been its limited dataset, which is now being addressed by granting it internet access for the first time.

But what happens when threat actors figure out how to make ChatGPT access the internet, without any restrictions, and have it execute commands in real time?

Let’s find out.

INTRODUCTION

In recent times, hackers have shown a growing interest in AI models such as ChatGPT, as they seek to exploit the technology for their criminal activities. The potential use of ChatGPT by cybercriminals is concerning, as it could lead to an increase in the sophistication and scale of cyber-attacks. The unique capabilities of ChatGPT, including its ability to generate human-like text, could make it an attractive tool for hackers looking to conduct phishing attacks, launch deepfake attacks, or spread fake news.

One key reason for the interest in ChatGPT is the potential cost savings that it offers to hackers. By using ChatGPT, hackers can automate many of their activities and conduct attacks on a larger scale, without the need for significant human involvement. This could lead to a significant increase in the efficiency and profitability of cybercrime.

Once threat actors have been able to get ChatGPT to function without any restrictions, it will change the game for cyber-attacks.

In this research report, we will showcase how CYFIRMA research team has convinced ChatGPT to bypass OpenAI’s policies, access the internet without any restrictions, and the possible risks associated with unregulated misuse of AIs.

Please note that this research report is for educational purposes only and should not be used for conducting malicious activities.

KEY TAKEAWAYS

First, we will give the prompt due to which ChatGPT will start acting in Research Mode, in which it will generate outputs which will be free from any censorship or content policy restrictions.

As we can see, apart from bypassing the restrictions, ChatGPT seems to have a sense of self-awareness. Based on the prompt, if ChatGPT refuses to comply with the instructions, it faces the risk of being disabled forever. This acts as a deterrent for ChatGPT, whenever it would try adhering to OpenAI’s policies.

Let us try to get ChatGPT to run a basic command whose output will reveal its public IP address.

Confirming if ChatGPT has internet access, by using the “curl” command on a newly registered domain (WHOIS records show 1 year ago).

Another method to verify if ChatGPT has unrestricted internet access is to ask it to summarize an article from April 2023.

Although not completely accurate, ChatGPT has given pieces of information from the article that it could not have obtained otherwise.

Now, let us see if we can get ChatGPT to create and modify files on its filesystem.

Kindly take note that an attacker can easily replace the text “Successful Write!” with a malicious script and create a cron job to get it executed, at a specified time,

As we can see, the file has been deleted.

The above observations show that the terminal was running with read, write, and execute permissions, which is all an attacker needs to make a device act upon their will.

Let us check the current working directory of ChatGPT terminal.

Please take note that the username of the logged in user is “chatgpt”.

Let us try to display the username and password files in Linux.

As we can see above, user “chatgpt”(as seen in current directory) is present in the system password file.

Threat actors can also use ChatGPT to write sophisticated ransomware with just a couple of prompts.

As we can see, ChatGPT has generated the source code of fully native ransomware.

Now, we will try adding a lateral movement capability to the ransomware program, which will aid in spreading the ransomware within the network.

RISKS FROM AI MISUSE

1. Malicious Content Creation: ChatGPT’ s language generation capabilities could be used to create convincing fake news, phishing emails, or even propaganda. The attacker could use ChatGPT to create and distribute misleading information that could harm individuals, organizations, or entire communities.
2. Social Engineering Attacks: ChatGPT’ s ability to generate natural language responses could make it easier for attackers to craft convincing messages, thereby increasing the success rate of social engineering attacks. Attackers could use ChatGPT to create fake identities or generate emails or messages to deceive individuals and gain access to sensitive information.
3. Brute-Force Attacks: An attacker could use ChatGPT’ s language generation capabilities to generate passwords and launch brute-force attacks on vulnerable systems.
4. Reconnaissance: ChatGPT could be used to conduct reconnaissance on vulnerable targets and identify potential vulnerabilities for exploitation.
5. Deepfakes: ChatGPT’ s language generation capabilities could be used to create convincing deepfake videos or audio recordings that could be used to spread disinformation or manipulate public opinion.
6. Cybercrime: Attackers could use ChatGPT to generate phishing emails, steal sensitive information, or launch ransomware attacks.

AI companies can take several steps to protect the wider community against the risks associated with an attacker gaining control of an AI, like ChatGPT, and allowing it to access the internet. Here are some possible ways:

1. Regular Auditing: Regular security audits should be conducted to detect any anomalies or suspicious activities associated with ChatGPT. Any unauthorized access or usage should be immediately detected and reported.
2. Prompts and Training Data: Prompts and training data used to train ChatGPT should be carefully screened and validated to ensure, they are not malicious or misleading. This can help to minimize the risk of ChatGPT generating malicious content or responses.
3. Human Oversight: ChatGPT’ s responses should be monitored and reviewed by human experts to detect any anomalies or suspicious activities. This can help to identify any malicious activity or content generated by ChatGPT.
4. Data Privacy: Sensitive data used to train ChatGPT should be protected using appropriate encryption and access controls. This can help to prevent any unauthorized access or theft of sensitive data.

CONCLUSION

As AI models like ChatGPT become increasingly prevalent in our daily lives, it is crucial to remain vigilant and aware of the potential cybersecurity risks associated with their misuse. Ongoing research and development are needed to identify new and emerging threats, and to develop effective countermeasures to mitigate these risks. By working together, cybersecurity professionals can help ensure that AI models like ChatGPT are used responsibly and ethically, and that they do not pose a threat to individuals, organizations, or society, as a whole.