Cracking an MSSQL Hash with Hashcat — and Why Your GPU Vanishes Inside a VM

TL;DR

You’ve pulled a SQL Server login hash out of a lab box and you want the plaintext. Two things decide whether this goes well: identifying the hash correctly so Hashcat uses the right mode, and having a real GPU so you can actually iterate. Both have a sharp edge.

The hash mode is keyed off the value’s prefix — an MSSQL hash starting 0x0200 is SQL Server 2012-or-later and cracks under -m 1731. Get that wrong and Hashcat runs happily and never cracks anything. Then, when rockyou.txt runs dry, “exhausted” doesn’t mean “uncrackable” — it means “not a common password,” and you climb a ladder: rules, masks, hybrid, bigger lists, a target-specific wordlist, and finally a length-bounded brute force when you know roughly how long the password is.

The trap underneath all of it: I was running Kali inside VMware Workstation, and hashcat -I showed no real GPU. Workstation doesn’t pass your physical card through — you get a virtual display adapter, not CUDA. Everything was crawling on CPU. The fix was to stop fighting the hypervisor and run the crack on the host with the actual GPU.

Authorized use only. This is methodology from authorized lab and CTF practice. Every hash and plaintext here is redacted or a placeholder. Don’t run password attacks against systems you don’t own or aren’t explicitly permitted to test.

Identify before you attack

The single most common way to waste an evening is to point Hashcat at a hash with the wrong mode. It won’t error — it’ll churn through the whole keyspace and crack nothing, because it’s hashing candidates with the wrong algorithm.

MSSQL hashes announce themselves by prefix:

0x0100… — SQL Server 2000/2005 (SHA-1, with the salt). Hashcat mode -m 132 (or -m 131 for the older uppercased variant).
0x0200… — SQL Server 2012 and later (SHA-512). Hashcat mode -m 1731.

The hash in front of me started 0x0200, so this was a modern SHA-512 MSSQL hash → -m 1731. When in doubt, let Hashcat show you the canonical shape:

attacker@kali$ hashcat --example-hashes | grep -i mssql -A 5

If your hash file pairs a username with the hash — the natural sa:0x0200… shape you get from dumping logins — tell Hashcat so it doesn’t try to crack the username as part of the hash:

attacker@kali$ hashcat -m 1731 hash.txt rockyou.txt --username

And before committing hours to any attack, confirm Hashcat actually parsed the line. A quick no-op that proves the format is good:

attacker@kali$ hashcat -m 1731 hash.txt --show

If --show lists your hash (uncracked is fine), the format is valid. If it complains, fix the file before you burn a single GPU-hour.

”Exhausted” is a starting point, not a verdict

The first pass is always the same:

attacker@kali$ hashcat -m 1731 hash.txt /usr/share/wordlists/rockyou.txt
...
Status...........: Exhausted

Exhausted on rockyou tells you exactly one thing: the password isn’t a straight dictionary word from the most famous leaked list. That rules out the easy 90% and points you at the climb. Each rung adds keyspace in the direction real passwords actually live.

Rung 1 — rules

Rules mutate every dictionary word (capitalise, append digits, leetspeak) on the fly. best64 is the cheap, high-yield default; OneRuleToRuleThemAll and dive are heavier and broader:

attacker@kali$ hashcat -m 1731 hash.txt rockyou.txt -r /usr/share/hashcat/rules/best64.rule
attacker@kali$ hashcat -m 1731 hash.txt rockyou.txt -r /usr/share/hashcat/rules/rockyou-30000.rule
attacker@kali$ hashcat -m 1731 hash.txt rockyou.txt -r /usr/share/hashcat/rules/dive.rule

Rung 2 — masks for the corporate shape

So many real passwords are Capital + lowercase + digits + symbol that it’s worth a dedicated mask attack. Spring2025! and Welcome123! are this pattern:

# Capital, five lowercase, three digits, bang — the Spring2025! shape
attacker@kali$ hashcat -m 1731 hash.txt -a 3 '?u?l?l?l?l?l?d?d?d!'

Rung 3 — hybrid

Hybrid attacks bolt a mask onto each dictionary word — the “word plus year plus symbol” habit:

attacker@kali$ hashcat -m 1731 hash.txt -a 6 rockyou.txt '?d?d?d?d!'   # word + 2025!
attacker@kali$ hashcat -m 1731 hash.txt -a 7 '?u' rockyou.txt          # Capital + word

Rung 4 — better and more targeted wordlists

rockyou is one leaked list. SecLists ships much larger and better-ranked ones:

attacker@kali$ sudo apt install seclists
attacker@kali$ hashcat -m 1731 hash.txt \
    /usr/share/seclists/Passwords/Common-Credentials/10-million-password-list-top-1000000.txt \
    -r /usr/share/hashcat/rules/best64.rule

The highest-yield list of all is the one you build from the target. Pull usernames, the company name, hostnames, database names, and words off the app’s own pages into a custom file, then mutate it:

attacker@kali$ hashcat -m 1731 hash.txt custom.txt -r /usr/share/hashcat/rules/best64.rule

Rung 5 — length-bounded brute force

Sometimes you learn the length — a policy hint, a leaked mask, a password manager that shows the field length. Here I had reason to believe the secret was 10–13 lowercase characters. That’s a finite keyspace you can grind directly, walking the lengths with --increment instead of running four separate masks:

attacker@kali$ hashcat -m 1731 hash.txt -a 3 '?l?l?l?l?l?l?l?l?l?l?l?l?l' \
    --increment --increment-min 10 --increment-max 13 -w 3

-w 3 turns the workload up. hashcat --status shows progress; hashcat -m 1731 hash.txt --show prints the plaintext once it falls. And this is exactly the rung where the next problem bit me — because brute force over four lengths of ?l is only tolerable on a GPU.

The real wall: no GPU inside VMware Workstation

A length-10-to-13 lowercase brute force is billions of candidates. On a modern GPU that’s an afternoon; on CPU it’s a different unit of time entirely. Mine was crawling, so I checked what Hashcat could actually see:

attacker@kali$ hashcat -I
...
Backend Device ID #1
  Type...........: CPU
  Name...........: ...
# no NVIDIA device listed

No GPU. The Kali VM was running under VMware Workstation Pro, and Workstation does not do PCIe passthrough — it exposes a virtual SVGA display adapter for desktop 3D, but it never hands the guest your physical NVIDIA card for CUDA/OpenCL compute. You can tick “Accelerate 3D graphics” and install mesa-utils all day; hashcat -I will still only ever show CPU and a virtual adapter. That’s a design limit of the hypervisor, not a missing driver.

Three ways out, easiest first:

Crack on the host. Keep Kali for recon and just move the hash over. Install the NVIDIA driver and Hashcat for Windows, confirm the card with hashcat.exe -I, and run the attack natively on the real GPU. This is the fast, low-friction answer and the one I took.
Dual-boot Linux on bare metal. sudo apt install nvidia-driver nvidia-cuda-toolkit, reboot, verify with nvidia-smi and hashcat -I. The GPU is owned directly, no hypervisor in the way.
Use a hypervisor that does pass GPUs through. This is the home-lab answer — Proxmox or ESXi support VMDirectPath / PCIe passthrough, so a VM can take full ownership of the card. Workstation is the wrong tool for GPU compute; a Type-1 hypervisor is the right one. (It’s the kind of thing the home lab exists to let you try.)

The lesson generalises past Hashcat: any GPU-bound workload — model inference, transcoding, CUDA anything — inside Workstation is silently CPU-only. Always run hashcat -I (or nvidia-smi) and confirm the accelerator is there before you conclude a job is slow. Half the time “slow” is really “running on the wrong processor.”

Detect and prevent

The crack only works because the password was guessable inside a finite keyspace. Defence is about pushing the password out of reach and not leaking the hash in the first place.

Length beats everything. Every rung above — rules, masks, increment — collapses the moment the secret is long and random. A 16-character random sa password makes a ?l-style brute force computationally pointless; a 9-character one is an afternoon. If you set one control, set a length floor.
Prefer Windows/AD authentication over SQL logins. Mixed-mode SQL auth with a weak sa password is the classic foothold. Integrated auth removes the standalone hash entirely.
Treat the sa account as radioactive. Rename or disable it, give it a long random secret, and don’t reuse it across instances. A cracked sa is usually game over for the database — and frequently the host, via xp_cmdshell.
Don’t expose 1433. TCP 1433 reachable from anywhere untrusted turns an offline cracking exercise into an online one against your production server. Firewall it to the app tier.
Detection: alert on dumps of sys.sql_logins/master..syslogins, on xp_cmdshell being enabled, and on bursts of failed sa logins. The hash leaving the database is the event that makes everything in this writeup possible — that’s the thing to catch.

Takeaways

Identify the hash before you attack it. 0x0200 → MSSQL 2012+ → -m 1731; 0x0100 → 2005 → -m 132. Wrong mode means a clean run that cracks nothing.
Prove the format with --show first, and use --username when the file is user:hash. Five seconds here saves hours of a misparse running silently.
“Exhausted” is a fork, not a dead end. Climb deliberately: rules → masks → hybrid → bigger lists → a target-specific list → length-bounded brute force.
Confirm the GPU is real. hashcat -I. VMware Workstation gives you a virtual adapter, not your card — GPU compute belongs on the host, on bare metal, or on a passthrough-capable hypervisor like Proxmox.
On the defensive side, length is the control that makes all of this moot. Long random secrets turn a tractable keyspace into an intractable one.