sqlmap/extra/esperanto

Esperanto

A DBMS-agnostic blind-extraction engine that speaks one "esperanto" at any SQL back-end, driven by nothing but a boolean oracle.

What this is

Ordinary blind SQL injection retrieval needs to know the back-end first: sqlmap fingerprints the DBMS, loads that dialect's queries.xml row, and extracts with it. When the DBMS cannot be identified - an unknown or heavily-firewalled target, an exotic engine, a fork that answers unlike its parent - that pipeline dead-ends.

Esperanto removes the prerequisite. Given only a boolean oracle - a callable oracle(condition) -> bool that reports whether an arbitrary SQL boolean expression holds at the target - it discovers the dialect from scratch (string concatenation, substring / length / character-code functions, the usable comparison operator, the catalog surface) and then extracts data character-by-character, without ever being told which DBMS is on the other end.

The candidate SQL forms it probes are distilled from sqlmap's own data/xml/queries.xml across all supported DBMSes: rather than fingerprint-then-load-one-dialect, it reuses that accumulated knowledge as a single language it tries to speak at any backend. The engine is self-contained (no sqlmap imports) and runs unmodified on Python 2.7 and Python 3.

Usage

Inside sqlmap, via the --esperanto switch. sqlmap detects the boolean-blind injection as usual, then hands the engine its own oracle (checkBooleanExpression, which rides the full request / comparison / WAF stack) instead of fingerprinting. The normal enumeration switches then work against an unidentified back-end:

python sqlmap.py -u 'http://host/vuln?id=1' --esperanto --banner --tables --dump -T users

Standalone, straight from this directory (no sqlmap, no PYTHONPATH):

python run.py -u 'http://host/vuln?id=1*' --tables --dump -T users
python run.py -u 'http://host/vuln?id=1*' --string <true-marker> --dump -T users -C uname,pass
python run.py --self-test

The * (or an explicit [INFERENCE]) marks the injection point; without one it defaults to the end of the URL. The true/false oracle is taken from --string / --code, or auto-calibrated from the response when none is given.

How it works

Discovery is a set of capability ladders. For each primitive it tries the known SQL forms in turn and keeps the first that a small set of probes proves correct, so a backend that lacks - or a WAF that filters - one form falls through to the next rather than failing:

Primitive	Ladder (first that works wins)
Concatenation	||, CONCAT(), +, &
Substring	SUBSTR / SUBSTRING / MID, else LEFT/RIGHT composition
Length	a length function, else derived from the substring's end
Character read	code point (ASCII/UNICODE/ORD/...), collation-forced byte order, hex, ordinal, = scan
Comparison	>, else BETWEEN, else an operator-free ordered form (SIGN/ABS/LEAST/...), else order-free IN()
No-substring floor	LIKE / GLOB / SIMILAR TO pattern matching

Each candidate is accepted only against a semantic check (for example, the comparison ladder is validated across a signed truth table, not two positive samples), so a rewriting layer that turns > into >= is rejected rather than silently mis-reading every count and length.

Discovery produces a Dialect; identify() then fuses catalog family, required dual-table and version-banner evidence into a best-guess product. Enumeration walks catalogs by keyset (MIN(name) WHERE name > prev) with no dialect row-limiter, and dump() reconstructs rows by a discovered primary/unique key, else a physical row-id, else the row's own value.

Interface

from extra.esperanto import Esperanto

esp = Esperanto(oracle)          # oracle(condition_str) -> bool
esp.discover()                   # ladder out the dialect
esp.identify()                   # best-guess product + evidence
esp.extract("(SELECT ...)")      # one scalar, char-by-char
esp.dump("users", schema="app")  # {columns, rows, complete, exact, keyed_by}
esp.enumerate("table")           # keyset catalog walk

• The oracle contract is strict tri-state: return True / False for an observed result; a transport or observation failure must raise (or return a non-boolean), which the engine treats as undecided rather than a definitive False. A wrong-dialect or unsupported probe is expected to be reported as False by the oracle itself.
• buildHandler() is the sqlmap adapter (--esperanto), a dbmsHandler that maps sqlmap's enumeration calls onto the engine. sqlmap-core imports are deferred inside it so the rest of the package stays dependency-free.
• strategy() freezes the discovered dialect into an immutable InferenceStrategy of pure render_* methods (no oracle, no loop), and hostExtract() is a reference host loop that extracts using only a strategy plus an oracle.

Integrity model

Every recovered value carries an Integrity classification so a caller can tell "we visited every position" apart from "the bytes are provably the source's":

State	Meaning
EXACT	proven byte-identical to the source (or a proven NULL)
WHOLE_BUT_AMBIGUOUS	every position read, but case/accent is uncertain (collation-dependent comparison, no byte-exact primitive)
TRUNCATED	a bounded prefix; the source continues
UNRESOLVED	a character could not be recovered
FAILED	could not observe or verify

The engine is designed to fail closed: it never manufactures a False from an unobservable probe, never silently substitutes or drops a character, distinguishes NULL from empty string, and reports a dump along two independent axes - complete (coverage: every row) and exact (content: the values are byte-faithful). EXACT is claimed only when a byte-faithful witness backs the value.

Scope and limitations

This is a last-resort engine for targets a normal fingerprint cannot reach, not a replacement for sqlmap's native retrieval. Known boundaries:

• Throughput. Extraction is one boolean question per request, char-by-char; it is bounded by the information theory of a boolean oracle (about one bit per request) and by network latency. It is built for reach, not speed.
• Namespace. Database, schema and owner are currently handled as a single scope name. Fully separated, backend-aware qualification (for example SQL Server database.schema.table via a sys.schemas join) is not yet modelled, so schema scoping on those engines is best-effort.
• Encoding. The back-end's declared character set is read from its own catalog and mapped to a codec (with the vendor quirks - MySQL latin1 is Windows-1252, Oracle WE8MSWIN1252 too), resolving the common cases definitively. A column whose character set differs from the database default, or a set outside the mapped list, decodes best-effort and is marked non-exact rather than asserted.
• Strategy hand-off. The exported InferenceStrategy / hostExtract() drive the character comparison modes under an ordered comparator; they do not yet drive the pattern-only or membership-only modes, and the frozen field set does not yet carry every discovered semantic.

Where a value cannot be recovered exactly, the engine surfaces its Integrity rather than presenting it as trustworthy.

Self-test

python run.py --self-test

runs the engine against an in-memory SQLite oracle across every compare mode plus identification, byte extraction, noisy-oracle quorum voting, the fail-closed integrity invariants and the frozen strategy hand-off. The regression corpus lives in tests/test_esperanto.py.

---

Part of sqlmap. See the top-level LICENSE for copying permission.