CWE-150 Detail

The product receives input from an upstream component, but it does not neutralize or incorrectly neutralizes special elements that could be interpreted as escape, meta, or control character sequences when they are sent to a downstream component.

Introductiemodi

Build and Compilation : During LLM training, escape sequences may be assigned within token entry values.
Implementation : Interactive command line interfaces have become more commonplace with growing system administration needs and human-friendly TUIs (text-based user interfaces); logging user-generated values into the terminal into one of these sessions is no longer safe.
Implementation : REALIZATION: This weakness is caused during implementation of an architectural security tactic.

Toepasselijke platforms

Taal

Class: Not Language-Specific (Undetermined)

Technologieën

Name: AI/ML (Undetermined)

Veelvoorkomende gevolgen

Waargenomen voorbeelden

Mogelijke risicobeperkingen

Developers should anticipate that escape, meta and control characters/sequences will be injected/removed/manipulated in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
Phases : Implementation
Phases : Implementation
While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters that do not pass an extremely strict allowlist (such as everything that is not alphanumeric or white space). If some special characters are still needed, such as white space, wrap each argument in quotes after the escaping/filtering step. Be careful of argument injection (CWE-88).
Phases : Implementation
Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous inputs after they have been checked.
Phases : Implementation
When using output from an LLM, neutralize or strip escape codes before redirecting output to the terminal or other rendering engine that would process the codes. The neutralization could require that the character be printable and/or allowable whitespace, such as a carriage return or newline. Be deliberate about what to allow.
Phases : Build and Compilation
When using an LLM: during tokenizer training, suppress escape codes from the tokenizer's vocabulary. Depending on context, this could be accomplished by removing the codes from input to the tokenizer, or removing the map from the string to its token ID. It is generally unlikely that this removal would adversely affect the quality or correctness of what is generated, e.g. advice requests for terminal settings to change colors.

Detectiemethoden

Automated Static Analysis

Automated static analysis, commonly referred to as Static Application Security Testing (SAST), can find some instances of this weakness by analyzing source code (or binary/compiled code) without having to execute it. Typically, this is done by building a model of data flow and control flow, then searching for potentially-vulnerable patterns that connect "sources" (origins of input) with "sinks" (destinations where the data interacts with external components, a lower layer such as the OS, etc.)
Effectiviteit : High

Notities kwetsbaarheidsmapping

Rechtvaardiging : This CWE entry is at the Variant level of abstraction, which is a preferred level of abstraction for mapping to the root causes of vulnerabilities.
Opmerking : Carefully read both the name and description to ensure that this mapping is an appropriate fit. Do not try to 'force' a mapping to a lower-level Base/Variant simply to comply with this preferred level of abstraction.

Gerelateerde aanvalspatronen

Notities

As of CWE 4.20: while exploitation and research interest in ANSI escape codes (and other terminal sequences) was popular in the early 2000's [REF-1527] - and known as an attack vector going back to at least 1994 [REF-1532] - there has been renewed interest in this type of injection as new attack vectors have arrived (e.g., from LLM output, or CLIs that were not based on OS sessions).

Referenties

REF-1532

flash.c
https://phrack.org/issues/46/4

REF-1527

Terminal Emulator Security Issues
H D Moore.
https://marc.info/?l=bugtraq&m=104612710031920&w=2

REF-1526

\e[31m"?! ANSI Terminal security in 2023 and finding 10 CVEs
David Leadbeater.
https://dgl.cx/2023/09/ansi-terminal-security

REF-1529

Weaponizing Plain Text ANSI Escape Sequences as a Forensic Nightmare
STÖK.
https://www.youtube.com/watch?v=3T2Al3jdY38

REF-1530

Weaponizing Plain Text ANSI Escape Sequences as a Forensic Nightmare
STÖK.
https://i.blackhat.com/BH-US-23/Presentations/US-23-stok-weponizing-plain-text-ansi-escape-sequences-as-a-forensic-nightmare-appendix.pdf

REF-1531

LLM output can take over your computer
Leon Derczynski.
https://interhumanagreement.substack.com/p/llm-output-can-take-over-your-computer

REF-1533

Terminal DiLLMa: LLM-powered Apps Can Hijack Your Terminal Via Prompt Injection
wunderwuzzi.
https://embracethered.com/blog/posts/2024/terminal-dillmas-prompt-injection-ansi-sequences/

Indiening

Wijzigingen