Site-Map

Notebooks

In this section, you can find a few initial raw technical notebooks related to the site. (They are presented as-is; please excuse stray typos and jargon.) These notebooks attack problems on the ground level (without using any existing specialized modules) to see how things really work before we settle down on any specific path. For the tech-savvy, these samples may provide initial insights for their own experiments and exploration. For those who are technology students, working on your own through these challenges is a fun learning journey in itself – from algorithms to statistical analysis, you will encounter the full spectrum of complexities required to engineer a solution.

( Production level solutions are included in our member-onboarding crash-course. )

Alien Nickname

Let's generate a "nickname-ID" – something that isn't a real name, but certainly feels like one.

To do this, we need a common-names dataset and a Trie data structure.

 import re import json import random import pandas as pd from matplotlib import pyplot as plt from time import time

USA-Names public dataset

Our original dataset contains over 6 million records. By using a group-by-name aggregation and filtering out names without vowels, we arrived at 35K unique, multinational, romanized, and easy-to-pronounce names. The dataset required some cleanup, as some entries lacked a space between the first and last names.

 data = pd.read_csv(f'data/name-stats.csv') data
gender name count vc
0 F Aadhira 7 2
1 F Aadhya 162 2
2 F Aadya 106 2
3 F Aahana 40 1
4 F Aahna 1 1
... ... ... ... ...
35530 M Zyquavious 1 5
35531 M Zyren 1 2
35532 M Zyron 1 2
35533 M Zyrus 1 2
35534 M Zyshonne 1 3

35535 rows × 4 columns

 data = data.sort_values(by='name', key=lambda x: x.str.len()).reset_index() data['name'] 0 Io 1 Pa 2 Ed 3 My 4 Ki ... 35530 Antoniodejesus 35531 Franciscojavier 35532 Johnchristopher 35533 Mariadelosangel 35534 Ryanchristopher Name: name, Length: 35535, dtype: object

Building the Trie

Each path in our Trie corresponds to a common name; leaf nodes (marked with #) contain the actual names. To retrieve a random "true" name, we simply select a random leaf node.

Our objective is to generate fake names that sound natural but remain ethnically neutral, short, and pronounceable. To achieve this, we traverse the Trie branch by branch, occasionally jumping to a different branch while maintaining the same syllable structure. While other strategies exist, this simple approach works perfectly for our use case.

To address the cleanup issue, we insert the shorter names into the Trie first. When we encounter a longer name, we check if it can be broken down into two names that are already present in the Trie. Any names that are too long or cannot be split are dropped.

 class NameTrie: def __init__(self, root: dict = {}): self.root = root self.vows = set('aeoui') self.cons = set('qwrtpsdfghjklzxcvbnmy') def insert(self, word, max_length: int = 10): word = word.lower() node, i = self.root, 0 composite = 0 while i < len(word): ##### check long names ##### if i > 3 and len(word[i:]) > 3 and '#' in node and self.search(word[i:]) is True: #print(word[:i], word[i:]) composite += 1 elif composite > 0 and len(word) > max_length: return # drop composite elif len(word) > max_length + 2: return # drop long ############################ node[word[i]] = node.get(word[i], {}) node, i = node[word[i]], i + 1 node['#'] = word def search(self, word): word = word.lower() node, i = self.root, 0 while i < len(word): if not word[i] in node: return False node, i = node[word[i]], i + 1 return '#' in node and word == node['#'] def get_name(self, prefix: str = None): name = None node = self.root if prefix is None else self.root[prefix] while len(node.keys()) > 0: if '#' in node: name = node['#'].capitalize() x = random.choice(list(node.keys())) if x == '#': return name node = node[x] return name def make_name(self, prefix: str = None, diverge: float = 0.5, max_length: int = 5): if prefix is None: prefix = random.choice(list(self.root.keys())) seq, node = [prefix], self.root[prefix] while len(node.keys()) > 1 and len(seq) < max_length: keys = set(node.keys()) - set(['#']) if seq[-1] in self.vows: keys = keys.difference(self.vows) else: keys = keys.difference(self.cons) if len(keys) == 0: break seq.append(random.choice(list(keys))) node = self.root[seq[-1]] if random.random() < diverge else node[seq[-1]] return ''.join(seq).capitalize() # create new trie = NameTrie() count = 0 for name in data['name']: x = name.encode('ascii', 'ignore').decode('ascii').strip().lower() if len(x) < 3 or x == 'childnotnamed': continue if re.match(r'^[a-z]{2,15}$', x): try: trie.insert(x) except: print(x) else: count += 1 with open('data/names.json','w') as output: json.dump(trie.root, output) count 35450 # load saved with open('data/names.json','r') as source: data = json.load(source) trie = NameTrie(data) trie.search('elon') True # real name starting with R print(trie.get_name('r')) Reghan # made-up name starting with Z print(trie.make_name('z')) Zibuta

The diverge parameter in our make_name method regulates "novelty" – essentially, how far the generated name strays from a "true" name. Zero divergence results in a truncated version of a real name. Lower divergence preserves "familiarity," while higher divergence encourages "weirdness."

 x = [n/1000 for n in range(1001)] y = [] for d in x: count = 0 for _ in range(10000): name = trie.make_name(diverge=d) if trie.search(name): count += 1 y.append(count/100) fig, ax = plt.subplots(figsize=(5, 5)) ax.scatter(x, y, s=1, color='slategray') ax.set_title('Output is a name') ax.set_xlabel('Diverge rate') ax.set_ylabel('Percentage') plt.show()

At very high divergence, the chance of accidentally generating a real name is quite low; if it does happen, we simply retry.

Finally, since we want these to serve as IDs, we often need more than a single word (such as a forename and a surname). Let's explore the size of the search space generated by combining multiple words.

 def generate_name(num_words, diverge, max_length): names = [] while len(names) < num_words: name = trie.make_name(diverge=diverge, max_length=max_length) if not trie.search(name): names.append(name) return ' '.join(names) for _ in range(10): print(generate_name(2, 0.3, 8)) Sumomix Gisuriro Kadorayo Wof Fehi Boraboni Cez Lorala Veletiju Utuwit Miq Tuluta Rugesufi Oj Akizu Copit Kom Qus Wacemera Qi size = 1e7 for max_length in range(5, 11): print(f'\nmax-length: {max_length}') x, y = [1], [] while len(y) == 0 or y[-1] > 0: start = time() result = set() for _ in range(int(size)): name = generate_name(x[-1], 0.5, max_length) result.add(name) y.append((1 - len(result)/size)) print(f'\twords: {x[-1]} collisions: {1 - len(result)/size:.7f}') x.append(x[-1] + 1) max-length: 5 words: 1 collisions: 0.9761968 words: 2 collisions: 0.0147736 words: 3 collisions: 0.0000055 words: 4 collisions: 0.0000000 max-length: 6 words: 1 collisions: 0.8517096 words: 2 collisions: 0.0110747 words: 3 collisions: 0.0000048 words: 4 collisions: 0.0000000 max-length: 7 words: 1 collisions: 0.4981691 words: 2 collisions: 0.0103647 words: 3 collisions: 0.0000050 words: 4 collisions: 0.0000000 max-length: 8 words: 1 collisions: 0.4000669 words: 2 collisions: 0.0103559 words: 3 collisions: 0.0000049 words: 4 collisions: 0.0000000 max-length: 9 words: 1 collisions: 0.3767576 words: 2 collisions: 0.0102748 words: 3 collisions: 0.0000034 words: 4 collisions: 0.0000000 max-length: 10 words: 1 collisions: 0.3751412 words: 2 collisions: 0.0102808 words: 3 collisions: 0.0000046 words: 4 collisions: 0.0000000 [ download ipynb ]