Python Walkthrough

This is a port being written, not a finished one being shown. Python 3 and its standard library, nothing to install, and every block on this page runs from the repository root as printed. If you only want the finished mappers, the vectors page has all nine in one file. The route there is what's here, including the step where it goes wrong.

The rule throughout: write one stage, check it against the matching section of vectors/gfs-1.0.json, and don't write the next stage until every entry passes. The porting guide gives the reasons behind each detail. This page only does the work.

0. Read the Notes

The vectors file opens with a notes array, and it's the shortest statement of the spec there is.

import json

with open("vectors/gfs-1.0.json") as handle:
    vectors = json.load(handle)

for note in vectors["notes"]:
    print("-", note)
print(vectors["counts"])

- commitment = SHA-256(serverSeed as UTF-8 hex string)
- digest = HMAC-SHA256(key = serverSeed as UTF-8 hex string, message = clientSeed:nonce:cursor)
- float = b0/256 + b1/256^2 + b2/256^3 + b3/256^4 over consecutive 4-byte chunks
- cursor advances once per 32-byte digest (8 floats)
- results must match under canonical JSON comparison
{'commitments': 4, 'digests': 24, 'floats': 12, 'games': 2240}

Four stages, then, and the counts say how many checks each one gets.

1. Commitments

The first note says the commitment is SHA-256 of the server seed "as UTF-8 hex string". So the seed is hashed as the text it's written in.

import hashlib
import json


def commitment(server_seed):
    return hashlib.sha256(server_seed.encode()).hexdigest()


with open("vectors/gfs-1.0.json") as handle:
    cases = json.load(handle)["commitments"]

ok = sum(commitment(c["serverSeed"]) == c["commitment"] for c in cases)
print(f"{ok} of {len(cases)} commitments")
print(commitment("5c1f7d3e8a2b4c6d9e0f1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d"))

4 of 4 commitments
ab37723062965715c5e6eeb54816f909a8e787bd3bfaee67a3cc0a3b90707de7

Four of four, and the last line is the commitment for the public seed that appears on every page of these docs. One function, done.

2. Digests

A digest is HMAC-SHA256 with the server seed as the key and clientSeed:nonce:cursor as the message. Python's hmac.new wants the key as bytes. The seed is hex. Hex turns into bytes with bytes.fromhex, and that's the natural line to write, whatever the second note said.

import hashlib
import hmac
import json


def digest(server_seed, client_seed, nonce, cursor):
    message = f"{client_seed}:{nonce}:{cursor}".encode()
    return hmac.new(bytes.fromhex(server_seed), message, hashlib.sha256).hexdigest()


with open("vectors/gfs-1.0.json") as handle:
    cases = json.load(handle)["digests"]

wrong = [c for c in cases if digest(c["serverSeed"], c["clientSeed"], c["nonce"], c["cursor"]) != c["digest"]]
print(f"{len(wrong)} of {len(cases)} digests do not match")
first = wrong[0]
print("expected", first["digest"])
print("got     ", digest(first["serverSeed"], first["clientSeed"], first["nonce"], first["cursor"]))

24 of 24 digests do not match
expected c1c07718def22d0518555afca618a9cb6edfdbcd0bf1505a3bd709a14e50a0c9
got      9634b92de52fb7482f314d521cc1c447377561a5260f2a03b1ea5f8b64ea0989

Every one of them. A total failure is good news of a kind. Whatever is wrong is wrong for every input, and a digest only has two inputs: the key and the message. The message is one f-string copied from the note.

It's the key. The note said "key = serverSeed as UTF-8 hex string", the same wording as the commitment. The 64 characters are the key, as text. Had we carried on to Dice with this version, the public inputs would have rolled 52.21 and not 56.12, and we'd have been debugging arithmetic that was fine.

import hashlib
import hmac
import json


def digest(server_seed, client_seed, nonce, cursor):
    message = f"{client_seed}:{nonce}:{cursor}".encode()
    return hmac.new(server_seed.encode(), message, hashlib.sha256).digest()


with open("vectors/gfs-1.0.json") as handle:
    cases = json.load(handle)["digests"]

ok = sum(digest(c["serverSeed"], c["clientSeed"], c["nonce"], c["cursor"]).hex() == c["digest"] for c in cases)
print(f"{ok} of {len(cases)} digests")
print(sorted({c["clientSeed"] for c in cases}), sorted({c["nonce"] for c in cases}), sorted({c["cursor"] for c in cases}))

24 of 24 digests
['a', 'galabet'] [0, 1, 65535] [0, 1]

The second line shows what the 24 cases vary: two client seeds, three nonces, two cursors. digest now returns raw bytes, since the next stage wants to slice them.

3. Floats

Each digest gives eight floats, four bytes apiece, big-endian, divided by 2³². When a round needs more than eight, the cursor goes up and another digest is made. Each entry in the floats section holds 16 floats, so it crosses that boundary once.

import hashlib
import hmac
import json


def digest(server_seed, client_seed, nonce, cursor):
    message = f"{client_seed}:{nonce}:{cursor}".encode()
    return hmac.new(server_seed.encode(), message, hashlib.sha256).digest()


def floats(server_seed, client_seed, nonce, count):
    out, cursor = [], 0
    while True:
        block = digest(server_seed, client_seed, nonce, cursor)
        out += [int.from_bytes(block[i:i + 4], "big") / 2**32 for i in range(0, 32, 4)]
        if len(out) >= count:
            return out[:count], cursor
        cursor += 1


with open("vectors/gfs-1.0.json") as handle:
    cases = json.load(handle)["floats"]

ok = sum(floats(c["serverSeed"], c["clientSeed"], c["nonce"], len(c["floats"])) == (c["floats"], c["cursor"]) for c in cases)
print(f"{ok} of {len(cases)} float streams, {sum(len(c['floats']) for c in cases)} floats")

stream, cursor = floats("5c1f7d3e8a2b4c6d9e0f1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d", "galabet", 42, 1)
print(stream[0], cursor)

12 of 12 float streams, 192 floats
0.5611712262034416 0

The comparison is == on lists of floats, with no tolerance, and that is deliberate. A 32-bit integer over 2³² is exact in a double, the JSON number reads back as that same double, and so the right answer is equal to the last bit. An "almost equal" check would wave through a port that used single precision or divided by 2³² − 1, and the porting guide shows that the game vectors don't always catch those.

floats returns the cursor it ended on, because a record stores it. The cursor is the highest one read and not the number of digests, which is why the loop returns before it increments.

The last line is the float behind the public Dice roll, 0.5611712262034416, the same number the Dice page gets from the library.

4. Games

With floats in hand the games are short, and the reference runner has all nine, so this step does two and adds a check that runner leaves out. Dice, because it's one line. Mines, because it brings in the shuffle that Keno and the card games share. The extra check is the cursor field each game vector carries.

import hashlib
import hmac
import json


def digest(server_seed, client_seed, nonce, cursor):
    message = f"{client_seed}:{nonce}:{cursor}".encode()
    return hmac.new(server_seed.encode(), message, hashlib.sha256).digest()


def uint32s(server_seed, client_seed, nonce, count):
    out, cursor = [], 0
    while True:
        block = digest(server_seed, client_seed, nonce, cursor)
        out += [int.from_bytes(block[i:i + 4], "big") for i in range(0, 32, 4)]
        if len(out) >= count:
            return out[:count], cursor
        cursor += 1


def dice(numbers):
    return (numbers[0] * 10001 >> 32) / 100


def shuffle(numbers, size):
    items = list(range(size))
    for n, i in enumerate(range(size - 1, 0, -1)):
        j = numbers[n] * (i + 1) >> 32
        items[i], items[j] = items[j], items[i]
    return items


def mines(numbers, count):
    return sorted(shuffle(numbers, 25)[:count])


with open("vectors/gfs-1.0.json") as handle:
    games = json.load(handle)["games"]

for name, needed, play in (("dice", 1, lambda u, p: dice(u)), ("mines", 24, lambda u, p: mines(u, p["mines"]))):
    rounds = [v for v in games if v["game"] == name]
    results = cursors = 0
    for v in rounds:
        numbers, cursor = uint32s(v["serverSeed"], v["clientSeed"], v["nonce"], needed)
        results += play(numbers, v["params"]) == v["result"]
        cursors += cursor == v["cursor"]
    print(f"{name:6} results {results} of {len(rounds)}, cursors {cursors} of {len(rounds)}")

numbers, cursor = uint32s("5c1f7d3e8a2b4c6d9e0f1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d", "galabet", 42, 24)
print(dice(numbers), mines(numbers, 3), cursor)

dice   results 112 of 112, cursors 112 of 112
mines  results 336 of 336, cursors 336 of 336
56.12 [9, 17, 22] 2

This version never makes a float. Python's integers don't overflow, so u * 10001 >> 32 is the floor of u / 2**32 * 10001 without a division in sight, and the shuffle index works the same way. The guide checks the integer forms against the float forms on every number in the vectors. Either is fine in Python. Pick one and use it everywhere.

Three details in shuffle and mines are the ones a port gets wrong. i walks down from the last position. Each swap takes the next number in order, which is what enumerate is doing. And sorted is part of the spec for Mines and Keno, though not for cards, whose order is the result.

The last line is the public seed pair again: Dice 56.12, and the three-mine board 9, 17, 22 that the Mines page draws as a grid. Mines reads 24 numbers whatever the mine count, so its cursor is always 2.

Limbo is the one mapper to copy character for character from the reference runner. Its operations are in a fixed order in doubles, and the vectors can't catch every rearrangement.

5. Crash Chain

Crash is a separate profile with a separate file. There's no client seed and no nonce. There is a chain of hashes, built backwards from a secret, and the vectors file gives away its secret so that a port can rebuild the whole thing.

import hashlib
import json

with open("vectors/gfs-1.0-crash.json") as handle:
    crash = json.load(handle)

for note in crash["notes"]:
    print("-", note)

chain = [crash["secret"]]
for _ in range(crash["length"]):
    chain.append(hashlib.sha256(chain[-1].encode()).hexdigest())
print("terminating hash matches:", chain[-1] == crash["terminatingHash"])

n = crash["length"]
placed = sum(g["gameHash"] == chain[n - g["index"]] for g in crash["games"])
linked = sum(hashlib.sha256(g["gameHash"].encode()).hexdigest() == g["previousHash"] for g in crash["games"])
print(f"{placed} of {len(crash['games'])} game hashes in place, {linked} of {len(crash['games'])} links verify")

- h_0 = secret; h_{i+1} = SHA-256(h_i hex string); terminatingHash = h_N; game k uses h_{N-k}
- digest = HMAC-SHA256(key = gameHash, message = salt); h = first 52 bits; cents = floor(100 * 2^52 / (h + 1) * (1 - houseEdge)); result = max(100, cents) / 100
terminating hash matches: True
48 of 48 game hashes in place, 48 of 48 links verify

The same trap as step 2 is waiting here, and the same answer. Each link hashes the previous hash as its 64 characters of text. bytes.fromhex would build a perfectly good chain that isn't this one.

chain[0] is the secret and chain[12] is the terminating hash, the value an operator publishes before the first game. Game 1 uses chain[11], game 2 uses chain[10], and so on towards the secret. The link check is what a player runs: hash the game hash they were shown and compare with the one from the round before. They never need the secret. The 48 entries are the 12 games at two salts and two house edges.

6. Crash Results

import hashlib
import hmac
import json
import math


def crash_point(game_hash, salt, house_edge):
    digest = hmac.new(game_hash.encode(), salt.encode(), hashlib.sha256).hexdigest()
    h = int(digest[:13], 16)                                  # 13 hex characters are 52 bits
    micro = 100 * 2**52 * 1_000_000 // (h + 1)                # hundredths, times a million, in integers
    cents = math.floor(float(micro) / 1_000_000 * (1 - house_edge))
    return max(100, cents) / 100


with open("vectors/gfs-1.0-crash.json") as handle:
    games = json.load(handle)["games"]

ok = sum(crash_point(g["gameHash"], g["salt"], g["houseEdge"]) == g["result"] for g in games)
print(f"{ok} of {len(games)} crash results")
for g in games[:4]:
    print(g["index"], g["salt"][:6], g["houseEdge"], crash_point(g["gameHash"], g["salt"], g["houseEdge"]))

48 of 48 crash results
1 0x0000 0.01 1.12
1 0x0000 0 1.13
1 galabe 0.01 1.13
1 galabe 0 1.14

The key is the game hash as text and the message is the salt, whatever string that is. One of the two salts in the file is galabet and the other is a 66-character string starting 0x, which is also used as text, prefix and all.

The division is the step to keep in integers. 100 * 2**52 * 1_000_000 is a 79-bit number, and Python divides it exactly. Only after that does the value become a float, to take the edge off and floor. That mirrors the reference line for line, including the scaling by a million. The guide measures what happens if you tidy the formula into pure integers: a different answer about once in three million rounds.

That's the derivation side of a port: a handful of short functions, passing every vector in both files that they cover. It isn't the whole of GFS. Verifying a record also means checking the client seed rules and hashing canonical JSON, and Python's json.dumps doesn't produce that by default. Canonical JSON and record hash has the Python for it. Ed25519 signatures have their own vector file, vectors/gfs-1.0-sign.json, and nothing on this page touches it.