Crack Me if You Can writeup from Team john-users!

[sleestak]

Team john-users has posted their writeup of this year's Crack Me if You Can contest! Enjoy, and make sure to share widely.

[The Dark Tangent]

To help preserve I’ve pasted here:


Date: Sun, 03 Nov 2019 18:28:25 +0300
From: Aleksey Cherepanov <lyosha@...nwall.com>
To: john-users@...ts.openwall.com
Subject: team john-users write-up for CMIYC 2019

Team john-users competed in the Pro category in Crack Me If You Can
2019 online hash cracking competition held at DEFCON conference. We
are grateful to KoreLogic who organized this wonderful contest. Thank
you! We would like to thank other teams for impressive performance.
Congratulations to teams hashcat and CynoSure Prime! (ch3root asked me
to cut down overly sweet words, but I have to say again that this
contest was AWESOME!!!1eleven Both as a game and as an incentive to
inspect john deeper than usual.)

Contest Website [1]:
[1] https://contest-2019.korelogic.com/


Team Members

Aleksey Cherepanov
Ivan U
Luis Rocha
Matt Weir
Viktor Gazdag aka wucpi
rofl0r
soxrok2212
trebla

We had 8 active members. Some of them were distracted by day jobs.
Also 4 other members provided ideas about types of the additional
files and interpretation of hints.


Hardware Used

CPUs: ~100 cores / ~200 threads.

GPUs: ~18.

That's a rough estimation of hardware resources. Most of the time,
only a part of them was used.

We did not use our FPGAs in this contest because there were not any
hashes of supported formats.


Software Used

- John the Ripper bleeding-jumbo [2]
- hashcat [3]
- PCFG Password Guesser [4]
- EtherCalc, multi-user web spreadsheet [5]
- auxiliary software, including custom scripts to handle cracks and
submissions

[2] https://github.com/magnumripper/JohnTheRipper
[3] https://hashcat.net/hashcat/
[4] https://github.com/lakiw/pcfg_cracker/
[5] https://ethercalc.org/


The Game

A set of scrypt hashes and additional files were provided to
participants. Parameters of most scrypt hashes were so that regular
CPU core could do only ~20 c/s. Not much... But there were the
additional files to help with it: each file was a container of
different type containing a set of weak hashes with old passwords and
a textual hint about way how new password was derived from the old
password. Almost all sets of hashes contained usernames. So an old
password could be tried against one respective scrypt hash usually
without need to check against all scrypt hashes. Some of the textual
hints contained almost straight description how to get passwords for
scrypt hashes from respective additional passwords.

Some of the additional files were protected with simple passwords.
Most of them were cracked with rockyou.txt dictionary. We postponed
one file till the following hint[6] from organizers:
----------------------------------------------------------------------
Hint: log2.rar Take the original rockyou.txt Lines: 52350 351450
920119 920120 2787667 4913443 7280588 7280589 7280590 7280591 7280592
7280593 7280594 7280595 7280596 7280597 - There is a common thread
between them.
----------------------------------------------------------------------
[6] https://twitter.com/CrackMeIfYouCan/status/1159995125600157696

We extracted 14 files with additional hashes (counting two files for
Alaska). The additional files contained weak types of hashes. More
details are available right from the orgs at [7] (but there is nothing
about mysterious 21 lm+nt pairs for Alaska pack). Most passwords were
not very strong and cracking went well. It felt good. OTOH points were
from scrypt hashes only.

[7] https://contest-2019.korelogic.com/stats-hashsets.html

Matt cracked the first scrypt in the first 20 minutes. It had empty
password. Then we found 10 more scrypt hashes with empty password and
it allowed us to stay #1 on the scoreboard for some time. Our next
scrypt crack happened 12 hours later: Luis found "yjovic4" for
user331522553. We checked the username against the additional files
and got the match: JBJ hashes had user331522553 with password
"yjovic8". So we used a one-liner script to pick a password from JBJ
and attack only respective scrypt hash picked by username, replacing
last character with a digit. We got 350 cracks quickly on a single CPU
core. It felt really good: we bypassed another critical point. Phew!

The scrypt hashes and most of the hashes from additional files had
usernames. Not all of usernames were in both lists. Some weak hashes
did not have usernames at all. We used precise matching for some time,
but Ivan questioned that and we found that there were a few usernames
among scrypt hashes with suffixes "-a", "-b", "-c" (e.g.
user3236124088-b). We did not analyze it deeper.

Example of our script to run 1 baseword vs 1 scrypt hash (reformatted):
----------------------------------------------------------------------
while IFS=: read -r u p; do
printf '%s\n' "$p" > twl &&
./JohnTheRipper/run/john \
--users="$u,$u-a,$u-b,$u-c" \
results/uncracked/0.*.target.pw \
--wordlist=twl \
--rules=': sq1 sw2 se3 sr4 st5 sy6 su7 si8 so9 sp0';
done < results/pair_user_crack/14.raw-md5.fast-nosalt.log2.txt
----------------------------------------------------------------------

It was possible to notice that this script might be replaced by john's
single mode: username should be replaced with respective old password
and john would do everything else. Only after the contest, the idea
was explored weighting pros and cons objectively. It is worth a
separate thread[8]. (The first message contains some details specific
to the contest, that are not copied here.)

Subject: approaches to use old password as baseword for new hash
matching by username/login (as in CMIYC 2019)
[8] https://www.openwall.com/lists/john-users/2019/10/06/1

In short, we underused hardware very much because the script was hard
to manage. Underusing hardware, we were slow in exploration of
patterns and missed a lot of cracks. It was the critical mistake.

Single mode should be the right tool for the job, but we did not try
this way at all during the contest. Most probably it was the main
content of the mistake, but it is hard to tell because single mode
would require tricky setup. Maybe we should try all approaches.

~3 hours before the end, Aleksey wrote a quick and dirty script to
distribute attacks. Then the script was running 60 threads across 4
machines picking prepared options+hash+baseword kits from shared pool
of work. It allowed us to finish many postponed ideas for JBJ's old
passwords plus some trivial checks against other packs. The results
were quite good, but it was late to get more.

During the contest, the organizers replaced some target hashes with
hashes of even newer passwords. The lists of new hashes were provided
on the site. But we almost ignored this aspect of the game till the
very end. Then we used cracked passwords for old target hashes and
applied --rules=o1. This job did not finish in time. Most probably
--rules=o1 was a too wide attack, but it was too late for tweaks.

Thanks for reading!

--
Regards,
Aleksey Cherepanov