Email Generator 2.1.3
Email addresses in each country's style for C++23
Loading...
Searching...
No Matches
emailgen.hpp
Go to the documentation of this file.
1// SPDX-License-Identifier: MIT
2// Copyright (c) 2026 Diego Dasso Migotto
3
4#ifndef DASMIG_EMAILGEN_HPP
5#define DASMIG_EMAILGEN_HPP
6
7#include "core.hpp"
8#include <algorithm>
9#include <array>
10#include <cstddef>
11#include <cstdint>
12#include <filesystem>
13#include <format>
14#include <optional>
15#include <ostream>
16#include <span>
17#include <stdexcept>
18#include <string>
19#include <string_view>
20#include <system_error>
21#include <unordered_map>
22#include <unordered_set>
23#include <utility>
24#include <vector>
25
26#ifdef DASMIG_EMAIL_EMBED
27#include "emailgen_data.hpp"
28#endif
29
30/// @file emailgen.hpp
31/// @brief Email Generator library — email addresses in each country's style for C++23.
32/// @author Diego Dasso Migotto (diegomigotto at hotmail dot com)
33/// @see See doc/usage.md for the narrative tutorial.
34
35DASMIG_CORE_REQUIRE(1, 1);
36
37namespace dasmig
38{
39
40/// @brief What an email call asks for.
41///
42/// Every generation call takes one; `email_request{}` asks for the defaults.
43/// Choices are never stored on the generator, so one call cannot change the
44/// next. The names are plain values in UTF-8: pass the Latin form of a name
45/// written in another script, or nothing, and a handle style is used.
47{
48 /// @brief ISO-3166 two-letter country code (any case), or a country
49 /// drawn when absent. The country sets the style of the local part
50 /// and the provider domains.
51 std::optional<std::string> country{std::nullopt};
52
53 std::string given{}; ///< Given name; empty when unknown. // NOLINT(readability-redundant-member-init): keeps -Wextra quiet for partial designated initializers
54 std::string family{}; ///< Family name; empty when unknown. // NOLINT(readability-redundant-member-init): keeps -Wextra quiet for partial designated initializers
55 std::string nickname{}; ///< Nickname or handle; empty when unknown. // NOLINT(readability-redundant-member-init): keeps -Wextra quiet for partial designated initializers
56
57 /// @brief Birth year, for the `ana85` and `ana1985` styles; those styles
58 /// are left out when absent.
59 std::optional<int> birth_year{std::nullopt};
60
61 /// @brief Use a reserved domain (`example.com`, `.net`, `.org`, RFC
62 /// 2606), so a message sent to the address reaches nobody.
63 bool safe{false};
64
65 /// @brief Weighted selection (default): countries by population, styles
66 /// and domains by their weights. False selects uniformly.
67 bool weighted{true};
68};
69
70/// @brief Return type for email generation.
71///
72/// Supports implicit conversion to std::string (the address), streaming via
73/// operator<< and JSON output via to_json().
74class email
75{
76 public:
77 std::string country_code; ///< ISO 3166-1 alpha-2 of the country whose style the address follows.
78 std::string local; ///< The part before the `@`: `[a-z0-9._-]`, at most 64 characters.
79 std::string domain; ///< The part after the `@`.
80 std::string kind; ///< `free` (webmail) or `isp` (a provider of the country); empty for a reserved domain.
81 bool reserved{false}; ///< True when the domain is one RFC 2606 reserves (`example.com`, `.net`, `.org`): the `safe` option.
82
83 /// @brief The address.
84 /// @return `local@domain`.
85 [[nodiscard]] std::string address() const { return local + "@" + domain; }
86
87 /// @brief Retrieve the random seed used to generate this address.
88 /// @return The call seed; pass it to get_email(request, seed) with the
89 /// same request and data to replay it.
90 [[nodiscard]] std::uint64_t seed() const { return _signature.seed; }
91
92 /// @brief The seed and the data this address was generated from.
93 /// @return The replay signature; `data` is emg::data_hash() at the time
94 /// of generation.
95 [[nodiscard]] const core::seed_signature& signature() const { return _signature; }
96
97 /// @brief Human-readable summary string.
98 /// @return The address.
99 [[nodiscard]] std::string to_string() const { return address(); }
100
101 /// @brief Write this address as a JSON object.
102 /// @param w Writer positioned where a value is allowed.
103 void to_json(core::json_writer& w) const
104 {
105 w.begin_object()
106 .field("country_code", country_code)
107 .field("address", address())
108 .field("local", local)
109 .field("domain", domain)
110 .field("kind", kind)
111 .field("reserved", reserved)
112 .key("signature");
113 _signature.to_json(w);
114 w.end_object();
115 }
116
117 /// @brief This address as a JSON document.
118 /// @return Compact JSON, for example `{"country_code":"DE","address":"...",...}`.
119 [[nodiscard]] std::string to_json() const
120 {
121 core::json_writer w;
122 to_json(w);
123 return w.str();
124 }
125
126 /// @brief Implicit conversion to std::string.
127 /// @return The address.
128 operator std::string() const // NOLINT(hicpp-explicit-conversions)
129 {
130 return to_string();
131 }
132
133 /// @brief Stream the address to an output stream.
134 /// @param os Output stream.
135 /// @param v The address to stream.
136 /// @return The stream.
137 friend std::ostream& operator<<(std::ostream& os, const email& v)
138 {
139 os << v.to_string();
140 return os;
141 }
142
143 private:
144 core::seed_signature _signature;
145 friend class emg;
146};
147
148/// @brief The tables of the email data, for extending it.
149enum class email_table : std::uint8_t
150{
151 countries, ///< `countries.tsv`: the countries and their populations.
152 fold, ///< `fold.tsv`: letters and their ASCII form.
153 particles, ///< `particles.tsv`: family-name particles email names skip.
154 profiles, ///< `profiles.tsv`: the style profile of each country.
155 email_styles, ///< `email_styles.tsv`: the shapes of an email's local part, by profile.
156 providers, ///< `providers.tsv`: the domains addresses use, by country.
157 words ///< `words.tsv`: the words of the handle styles.
158};
159
160/// @brief Email Generator: email addresses in each country's style.
161///
162/// A call draws a country (by population), a style of the country's profile
163/// (`"style"`), the handle words (`"local"`), the digits (`"number"`) and the
164/// domain (`"domain"`), each from a stream split from the call seed. Output is
165/// identical on every compiler and platform (dasmig-core).
166///
167/// Can be used as a singleton via instance() or constructed
168/// independently.
169///
170/// @par Thread safety
171/// Each instance is independent. Concurrent calls to get_phone(request, seed) on the
172/// **same** instance are safe; calls that use the instance engine, loading and
173/// extending require external synchronisation.
174class emg
175{
176 public:
177 /// @brief Default constructor — creates an empty generator with no
178 /// data.
179 emg() : _tables{make_tables_()} {}
180
181 emg(const emg&) = delete; ///< Not copyable.
182 emg& operator=(const emg&) = delete; ///< Not copyable.
183 emg(emg&&) noexcept = default; ///< Movable.
184 emg& operator=(emg&&) noexcept = default; ///< Movable.
185 ~emg() = default; ///< Destructor.
186
187 /// @brief Access the global singleton instance.
188 ///
189 /// Loads the `lite` tier (or `full` if `lite` is missing) on first
190 /// access, from compiled-in data or an auto-probed `resources/`.
191 /// @return The singleton.
192 static emg& instance()
193 {
194 static emg inst{auto_probe_tag{}};
195 return inst;
196 }
197
198 // -- Email ------------------------------------------------------------
199
200 /// @brief Generate a random email address.
201 /// Use email_request{} for the defaults.
202 /// @param request What to generate.
203 /// @return The address.
204 /// @throws std::runtime_error If no email data has been loaded.
205 /// @throws std::invalid_argument If the country is unknown.
206 [[nodiscard]] email get_email(const email_request& request) { return get_email(request, draw_seed_()); }
207
208 /// @brief Generate a deterministic email address.
209 /// @param request What to generate.
210 /// @param call_seed Seed; the same request, seed and data give the same
211 /// result. Does not touch the instance engine.
212 /// @return The address.
213 /// @throws std::runtime_error If no email data has been loaded.
214 /// @throws std::invalid_argument If the country is unknown.
215 [[nodiscard]] email get_email(const email_request& request, std::uint64_t call_seed) const
216 {
217 require_data_();
218 if (_model.profiles.empty()) throw std::runtime_error("No email data loaded. Call load() first.");
219 const core::rng call{call_seed};
220
221 std::size_t ci = 0;
222 if (request.country)
223 {
224 const auto it = _model.by_region.find(upper_(*request.country));
225 if (it == _model.by_region.end())
226 {
227 throw std::invalid_argument("get_email: unknown country '" + *request.country + "'");
228 }
229 ci = it->second;
230 }
231 else
232 {
233 auto cr = call.split("country");
234 ci = request.weighted ? _model.all_population.pick(cr) : _model.all_uniform.pick(cr);
235 }
236 return make_email_(request, _model.countries[ci], call_seed, call);
237 }
238
239 /// @brief Generate a batch of email results from a seed drawn like
240 /// get_email() draws its own.
241 /// @param request What every item is generated from.
242 /// @param options Count, first index, threads and uniqueness.
243 /// @return The results, in index order.
244 [[nodiscard]] std::vector<email> get_email_batch(const email_request& request,
245 const core::batch_options& options)
246 {
247 return get_email_batch(request, options, draw_seed_());
248 }
249
250 /// @brief Generate a batch of email results, deterministically.
251 ///
252 /// Item `i` is exactly `get_email(request, core::item_seed(batch_seed,
253 /// options.first + i))`; see get_phone_batch().
254 ///
255 /// @param request What every item is generated from.
256 /// @param options Count, first index, threads and uniqueness (the rule
257 /// must be made for `email`).
258 /// @param batch_seed Seed of the whole batch.
259 /// @return The results, in index order.
260 /// @throws std::invalid_argument If the request or options are invalid.
261 /// @throws std::runtime_error If no data has been loaded, or a unique
262 /// batch runs out of attempts.
263 [[nodiscard]] std::vector<email> get_email_batch(const email_request& request,
264 const core::batch_options& options,
265 std::uint64_t batch_seed) const
266 {
267 return core::run_batch(options, batch_seed,
268 [&](std::uint64_t item_seed) { return get_email(request, item_seed); });
269 }
270
271 /// @brief Stream a batch of email results to a sink without holding the
272 /// whole batch.
273 /// @tparam Sink Callable taking `(std::uint64_t, email&)`.
274 /// @param request What every item is generated from.
275 /// @param options Count, first index, threads and uniqueness.
276 /// @param batch_seed Seed of the whole batch.
277 /// @param sink Receives the items.
278 template <typename Sink>
279 void for_each_email(const email_request& request, const core::batch_options& options,
280 std::uint64_t batch_seed, Sink&& sink) const
281 {
282 core::for_each_batch(options, batch_seed,
283 [&](std::uint64_t item_seed) { return get_email(request, item_seed); },
284 std::forward<Sink>(sink));
285 }
286
287 /// @brief Stream a batch of email results with a batch seed drawn like
288 /// get_email() draws its own.
289 /// @tparam Sink Callable taking `(std::uint64_t, email&)`.
290 /// @param request What every item is generated from.
291 /// @param options Count, first index, threads and uniqueness.
292 /// @param sink Receives the items.
293 template <typename Sink>
294 void for_each_email(const email_request& request, const core::batch_options& options, Sink&& sink)
295 {
296 for_each_email(request, options, draw_seed_(), std::forward<Sink>(sink));
297 }
298
299 // -- Data management --------------------------------------------------
300
301 /// @brief Check whether any data has been loaded.
302 /// @return True when at least one country is loaded.
303 [[nodiscard]] bool has_data() const { return !_model.countries.empty(); }
304
305 /// @brief Return the number of countries loaded.
306 /// @return The count; it is `countries().size()`.
307 [[nodiscard]] std::size_t country_count() const { return _model.countries.size(); }
308
309 /// @brief The countries an address can be generated for.
310 /// @return Their ISO-3166 two-letter codes (upper case), in the order of countries.tsv; the
311 /// list a request's `country` is chosen from.
312 [[nodiscard]] const std::vector<std::string>& countries() const { return _model.codes; }
313
314 /// @brief Whether a country is loaded.
315 /// @param country ISO-3166 two-letter code (any case).
316 /// @return True when countries.tsv has the country.
317 [[nodiscard]] bool has_country(std::string_view country) const
318 {
319 return _model.by_region.contains(upper_(country));
320 }
321
322 /// @brief Load data from a resource directory, replacing all data and
323 /// layers loaded before.
324 ///
325 /// The directory must contain `countries.tsv`; the other tables are read
326 /// when present. A path without `countries.tsv` is ignored.
327 ///
328 /// @param dir Directory such as `resources/lite`.
329 /// @throws core::tsv_error If a file is malformed; the data is then
330 /// unchanged.
331 void load(const std::filesystem::path& dir)
332 {
333 std::error_code ec;
334 if (!std::filesystem::is_directory(dir, ec) ||
335 !std::filesystem::is_regular_file(dir / file_name_(email_table::countries), ec))
336 {
337 return;
338 }
339 apply_(read_directory_(dir), core::layer_mode::replace, true);
340 }
341
342 /// @brief Load a dataset tier, replacing all data and layers loaded
343 /// before.
344 ///
345 /// Uses the compiled-in copy when the library is built with
346 /// `DASMIG_EMAIL_EMBED`, otherwise searches `resources/`, `../resources`,
347 /// `email-generator/resources` and `../email-generator/resources`.
348 ///
349 /// @param tier The tier.
350 /// @return False when the tier was not found.
351 /// @throws core::tsv_error If a file is malformed.
352 [[nodiscard]] bool load(dataset tier)
353 {
354 const std::string tier_dir{core::tier_name(tier)};
355 const auto marker = tier_dir + "/" + std::string{file_name_(email_table::countries)};
356 const auto root = core::find_resources(library_name_, marker);
357 documents docs;
358 for (const auto t : all_tables_)
359 {
360 docs.at(index_(t)) = core::load_resource(tier_dir + "/" + std::string{file_name_(t)},
361 embedded_files_(), root);
362 }
363 if (!docs.at(index_(email_table::countries))) return false;
364 apply_(std::move(docs), core::layer_mode::replace, true);
365 return true;
366 }
367
368 /// @brief Add data layers from a directory or a file.
369 ///
370 /// A directory may hold any of the table files (see email_table); all
371 /// are applied together or, if any fails, none. A file must have one of
372 /// those names.
373 ///
374 /// @param path Directory or file.
375 /// @param mode How the layers combine with the data loaded before.
376 /// @throws core::tsv_error If a file cannot be read or a row is
377 /// invalid; the data is then unchanged.
378 /// @throws std::invalid_argument If @p path is a file with another name.
379 void extend(const std::filesystem::path& path, core::layer_mode mode = core::layer_mode::override)
380 {
381 std::error_code ec;
382 if (std::filesystem::is_directory(path, ec))
383 {
384 apply_(read_directory_(path), mode, false);
385 return;
386 }
387 for (const auto t : all_tables_)
388 {
389 if (path.filename() == file_name_(t))
390 {
391 (void)extend(t, core::tsv_document::load(path), mode);
392 return;
393 }
394 }
395 throw std::invalid_argument("extend: expected one of the email tables (countries.tsv, fold.tsv, "
396 "particles.tsv, profiles.tsv, email_styles.tsv, providers.tsv, words.tsv), got " +
397 path.filename().string());
398 }
399
400 /// @brief Add a data layer to one table.
401 ///
402 /// Rows are keyed by the table's key columns (see doc/data.md). In the
403 /// default override mode a row replaces the loaded row with the same key
404 /// and new keys are added; a row with `disabled` set to 1 removes its key.
405 ///
406 /// @param which The table.
407 /// @param doc TSV with that table's columns.
408 /// @param mode How the layer combines with the data loaded before.
409 /// @return What the layer did (rows added, replaced, removed, warnings).
410 /// @throws core::tsv_error If a row is invalid; the data is unchanged.
411 const core::layer_info& extend(email_table which, core::tsv_document doc,
412 core::layer_mode mode = core::layer_mode::override)
413 {
414 documents docs;
415 docs.at(index_(which)).emplace(std::move(doc));
416 apply_(std::move(docs), mode, false);
417 return _tables.at(index_(which)).layers().back();
418 }
419
420 /// @brief Add a data layer to one table from TSV text (for example
421 /// from a WebAssembly host).
422 /// @param which The table.
423 /// @param text TSV with that table's columns.
424 /// @param source Name used in errors.
425 /// @param mode How the layer combines with the data loaded before.
426 /// @return What the layer did.
427 /// @throws core::tsv_error If a row is invalid; the data is unchanged.
428 const core::layer_info& extend_from_string(email_table which, std::string_view text, std::string source,
429 core::layer_mode mode = core::layer_mode::override)
430 {
431 return extend(which, core::tsv_document::parse(text, std::move(source)), mode);
432 }
433
434 /// @brief The layers in effect for one table, in order.
435 /// @param which The table.
436 /// @return The layer records.
437 [[nodiscard]] const std::vector<core::layer_info>& layers(email_table which) const
438 {
439 return _tables.at(index_(which)).layers();
440 }
441
442 /// @brief Hash of the loaded data (all tables), recorded in each
443 /// result's signature().
444 /// @return The hash; 0 when nothing is loaded.
445 [[nodiscard]] std::uint64_t data_hash() const
446 {
447 std::uint64_t h = 0;
448 bool any = false;
449 for (const auto& t : _tables)
450 {
451 any = any || !t.layers().empty();
452 h = core::hash_combine(h, t.data_hash());
453 }
454 return any ? h : 0;
455 }
456
457 private:
458 using table = email_table;
459
460 // -- Tables -------------------------------------------------------------
461
462 static constexpr std::string_view library_name_ = "email-generator";
463 static constexpr std::size_t table_count_ = 7;
464 static constexpr std::array<table, table_count_> all_tables_{
465 table::countries, table::fold, table::particles, table::profiles,
466 table::email_styles, table::providers, table::words};
467
468 using documents = std::array<std::optional<core::tsv_document>, table_count_>;
469 using tables = std::array<core::layered_table, table_count_>;
470
471 static constexpr std::size_t index_(table t) { return static_cast<std::size_t>(t); }
472
473 static constexpr std::string_view file_name_(table t)
474 {
475 switch (t)
476 {
477 case table::countries: return "countries.tsv";
478 case table::fold: return "fold.tsv";
479 case table::particles: return "particles.tsv";
480 case table::profiles: return "profiles.tsv";
481 case table::email_styles: return "email_styles.tsv";
482 case table::providers: return "providers.tsv";
483 case table::words: return "words.tsv";
484 }
485 return "";
486 }
487
488 static core::table_schema schema_(table t)
489 {
490 switch (t)
491 {
492 case table::countries:
493 return {.key = {"region"}, .required = {}, .optional = {"population"}};
494 case table::fold: return {.key = {"from"}, .required = {"to"}, .optional = {}};
495 case table::particles: return {.key = {"particle"}, .required = {}, .optional = {}};
496 case table::profiles: return {.key = {"region"}, .required = {"profile"}, .optional = {}};
497 case table::email_styles: return {.key = {"profile", "pattern"}, .required = {"weight"}, .optional = {}};
498 case table::providers: return {.key = {"domain", "country"}, .required = {"kind", "weight"}, .optional = {}};
499 case table::words: return {.key = {"list", "word"}, .required = {}, .optional = {}};
500 }
501 return {};
502 }
503
504 static tables make_tables_()
505 {
506 return {core::layered_table{schema_(table::countries)},
507 core::layered_table{schema_(table::fold)}, core::layered_table{schema_(table::particles)},
508 core::layered_table{schema_(table::profiles)}, core::layered_table{schema_(table::email_styles)},
509 core::layered_table{schema_(table::providers)}, core::layered_table{schema_(table::words)}};
510 }
511
512 // -- Parsed rows --------------------------------------------------------
513
514 static constexpr std::size_t none_ = static_cast<std::size_t>(-1);
515 // The pieces of an email pattern.
516 enum class tok : std::uint8_t { lit, f, l, fi, li, n, y2, y4, r, r2, adj, animal, jp };
517
518 struct token
519 {
520 tok kind{tok::lit};
521 std::string text; // the characters of a literal
522 };
523
524 struct style_data
525 {
526 std::vector<token> tokens;
527 double weight{0.0};
528 bool needs_given{false};
529 bool needs_family{false};
530 bool needs_nickname{false};
531 bool needs_year{false};
532 std::array<bool, 3> needs_words{};
533 };
534
535 struct provider_data
536 {
537 std::string domain;
538 std::string kind;
539 double weight{0.0};
540 };
541
542 struct country_data
543 {
544 std::string region;
545 double population{1.0};
546 std::size_t profile{none_};
547 std::vector<provider_data> providers;
548 core::weighted_table<std::size_t> provider_weights; // index into providers
549 };
550
551 struct model
552 {
553 std::vector<country_data> countries;
554 std::vector<std::string> codes; // the same countries as their codes, in table order
555 std::unordered_map<std::string, std::size_t> by_region;
556 std::vector<std::vector<style_data>> profiles; // styles by profile
557 std::unordered_map<std::string, std::size_t> profile_index;
558 std::unordered_map<std::uint32_t, std::string> fold; // code point -> ASCII
559 std::unordered_set<std::string> particles;
560 std::array<std::vector<std::string>, 3> words; // adj, animal, jp
561 core::weighted_table<std::size_t> all_population; // every country
562 core::weighted_table<std::size_t> all_uniform;
563 };
564
565 model _model;
566 tables _tables;
567 core::rng _engine{core::rng::from_entropy()};
568
569 struct auto_probe_tag {};
570
571 explicit emg(auto_probe_tag /*tag*/) : _tables{make_tables_()}
572 {
573 if (!load(dataset::lite)) (void)load(dataset::full);
574 }
575
576 // -- Helpers ----------------------------------------------------------
577
578 static std::span<const core::embedded_file> embedded_files_()
579 {
580#ifdef DASMIG_EMAIL_EMBED
581 return embedded::emailgen::files;
582#else
583 return {};
584#endif
585 }
586
587 std::uint64_t draw_seed_() { return _engine(); }
588
589 static std::string upper_(std::string_view s)
590 {
591 std::string out{s};
592 for (auto& c : out)
593 {
594 if (c >= 'a' && c <= 'z') c = static_cast<char>(c - 'a' + 'A');
595 }
596 return out;
597 }
598
599 // An ASCII character as names fold it: letters in lowercase, digits kept; a mark stays in a
600 // nickname (@p keep_marks) and anything else is dropped there ('\0'); in a name every other
601 // character is a space between parts.
602 static char fold_ascii_(char c, bool keep_marks)
603 {
604 if (c >= 'A' && c <= 'Z') return static_cast<char>(c - 'A' + 'a');
605 if ((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9')) return c;
606 if (keep_marks) return is_mark_(c) ? c : '\0';
607 return ' ';
608 }
609
610 // Folds text to lowercase ASCII (see fold_ascii_); a letter with no ASCII form is dropped in a
611 // nickname and is a space in a name.
612 [[nodiscard]] std::string fold_(std::string_view text, bool keep_marks) const
613 {
614 std::string out;
615 for (std::size_t i = 0; i < text.size();)
616 {
617 std::size_t used = 1;
618 const auto cp = decode_(text, i, used);
619 i += used;
620 if (cp != 0 && cp < 0x80)
621 {
622 const char c = fold_ascii_(static_cast<char>(cp), keep_marks);
623 if (c != '\0') out += c;
624 continue;
625 }
626 const auto f = cp == 0 ? _model.fold.end() : _model.fold.find(cp);
627 if (f != _model.fold.end())
628 {
629 out += f->second;
630 }
631 else if (!keep_marks)
632 {
633 out += ' ';
634 }
635 }
636 return out;
637 }
638
639 static std::vector<std::string> split_words_(const std::string& text)
640 {
641 std::vector<std::string> out;
642 std::string cur;
643 for (const char c : text)
644 {
645 if (c == ' ')
646 {
647 if (!cur.empty()) out.push_back(std::move(cur));
648 cur.clear();
649 }
650 else
651 {
652 cur += c;
653 }
654 }
655 if (!cur.empty()) out.push_back(std::move(cur));
656 return out;
657 }
658
659 // First part of a given name; first part of a family name that is not a particle.
660 [[nodiscard]] std::string name_part_(std::string_view text, bool family) const
661 {
662 auto parts = split_words_(fold_(text, false));
663 if (family)
664 {
665 std::erase_if(parts, [&](const std::string& p) { return _model.particles.contains(p); });
666 }
667 return parts.empty() ? std::string{} : parts.front();
668 }
669
670 static bool is_mark_(char c) { return c == '.' || c == '_' || c == '-'; }
671
672 // Keeps [a-z0-9._-], one mark at a time, none at the ends, 64 characters at most.
673 static std::string tidy_local_(std::string_view text)
674 {
675 std::string out;
676 for (const char c : text)
677 {
678 if (is_mark_(c) && (out.empty() || is_mark_(out.back()))) continue;
679 out += c;
680 if (out.size() == 64) break;
681 }
682 while (!out.empty() && is_mark_(out.back())) out.pop_back();
683 return out;
684 }
685
686 static void append_digits_(std::string& out, core::rng& r, int count)
687 {
688 for (int i = 0; i < count; ++i) out += static_cast<char>('0' + core::uniform_int(r, 0, 9));
689 }
690
691 // What the request gives for the local part, folded.
692 struct name_inputs
693 {
694 std::string given;
695 std::string family;
696 std::string nick;
697 bool year_ok{false};
698 };
699
700 [[nodiscard]] name_inputs read_names_(const email_request& request) const
701 {
702 name_inputs n;
703 n.given = name_part_(request.given, false);
704 n.family = name_part_(request.family, true);
705 n.nick = tidy_local_(fold_(request.nickname, true));
706 n.year_ok = request.birth_year && *request.birth_year >= 1000 && *request.birth_year <= 9999;
707 return n;
708 }
709
710 // Whether the inputs can fill the style; a weight of 0 switches a style off.
711 [[nodiscard]] bool style_fits_(const style_data& st, const name_inputs& n) const
712 {
713 const bool names = (!st.needs_given || !n.given.empty()) && (!st.needs_family || !n.family.empty()) &&
714 (!st.needs_nickname || !n.nick.empty()) && (!st.needs_year || n.year_ok);
715 bool words = true;
716 for (std::size_t list = 0; list < st.needs_words.size(); ++list)
717 {
718 words = words && (!st.needs_words.at(list) || !_model.words.at(list).empty());
719 }
720 return names && words && st.weight > 0.0;
721 }
722
723 // A style of the profile the inputs allow, by weight (or evenly when not weighted).
724 [[nodiscard]] const style_data& pick_style_(const std::vector<style_data>& styles, const name_inputs& n,
725 bool weighted, const core::rng& call,
726 const std::string& region) const
727 {
728 const auto share = [&](const style_data& st) { return weighted ? st.weight : 1.0; };
729 double total = 0.0;
730 const style_data* last = nullptr;
731 for (const auto& st : styles)
732 {
733 if (!style_fits_(st, n)) continue;
734 total += share(st);
735 last = &st;
736 }
737 if (last == nullptr) throw std::runtime_error("get_email: no style fits the request in the profile of " + region);
738 auto sr = call.split("style");
739 double x = core::uniform01(sr) * total;
740 for (const auto& st : styles)
741 {
742 if (!style_fits_(st, n)) continue;
743 x -= share(st);
744 if (x < 0.0) return st;
745 }
746 return *last;
747 }
748
749 // The local part: the style's pieces filled in, then tidied.
750 [[nodiscard]] std::string render_local_(const style_data& style, const name_inputs& n, const email_request& request,
751 const core::rng& call) const
752 {
753 auto lr = call.split("local");
754 auto nr = call.split("number");
755 const auto initial = [](const std::string& text) { return text.substr(0, 1); };
756 std::string local;
757 for (const auto& t : style.tokens)
758 {
759 switch (t.kind)
760 {
761 case tok::lit: local += t.text; break;
762 case tok::f: local += n.given; break;
763 case tok::l: local += n.family; break;
764 case tok::fi: local += initial(n.given); break;
765 case tok::li: local += initial(n.family); break;
766 case tok::n: local += n.nick; break;
767 case tok::y2: local += std::format("{:02}", *request.birth_year % 100); break;
768 case tok::y4: local += std::format("{:04}", *request.birth_year); break;
769 case tok::r: append_digits_(local, nr, core::uniform_int(nr, 1, 3)); break;
770 case tok::r2: append_digits_(local, nr, 2); break;
771 case tok::adj: local += pick_word_(0, lr); break;
772 case tok::animal: local += pick_word_(1, lr); break;
773 case tok::jp: local += pick_word_(2, lr); break;
774 }
775 }
776 local = tidy_local_(local);
777 return local.empty() ? "user" : local;
778 }
779
780 // The domain: a reserved one with safe, else one of the country's providers.
781 static void pick_domain_(email& v, const email_request& request, const country_data& country, const core::rng& call)
782 {
783 auto dr = call.split("domain");
784 if (request.safe)
785 {
786 static constexpr std::array<std::string_view, 3> reserved{"example.com", "example.net", "example.org"};
787 v.domain = std::string{reserved.at(core::uniform_int<std::size_t>(dr, 0, reserved.size() - 1))};
788 v.reserved = true;
789 return;
790 }
791 if (country.providers.empty() || (request.weighted && country.provider_weights.empty()))
792 {
793 throw std::runtime_error("get_email: no provider domain for " + country.region);
794 }
795 const auto& p = country.providers[request.weighted ? country.provider_weights.pick(dr)
796 : core::uniform_int<std::size_t>(dr, 0, country.providers.size() - 1)];
797 v.domain = p.domain;
798 v.kind = p.kind;
799 }
800
801 [[nodiscard]] email make_email_(const email_request& request, const country_data& country, std::uint64_t call_seed,
802 const core::rng& call) const
803 {
804 const std::size_t pi = country.profile != none_ ? country.profile : default_profile_();
805 if (pi == none_ || _model.profiles[pi].empty()) throw std::runtime_error("No email styles loaded for " + country.region);
806 const auto names = read_names_(request);
807 const auto& style = pick_style_(_model.profiles[pi], names, request.weighted, call, country.region);
808
809 email v;
810 v._signature = {.seed = call_seed, .data = data_hash()};
811 v.country_code = country.region;
812 v.local = render_local_(style, names, request, call);
813 pick_domain_(v, request, country, call);
814 return v;
815 }
816
817 [[nodiscard]] std::size_t default_profile_() const
818 {
819 const auto it = _model.profile_index.find("default");
820 return it == _model.profile_index.end() ? none_ : it->second;
821 }
822
823 const std::string& pick_word_(std::size_t list, core::rng& r) const
824 {
825 const auto& w = _model.words.at(list);
826 return w[core::uniform_int<std::size_t>(r, 0, w.size() - 1)];
827 }
828
829 static documents read_directory_(const std::filesystem::path& dir)
830 {
831 documents docs;
832 for (const auto t : all_tables_)
833 {
834 const auto file = dir / file_name_(t);
835 std::error_code ec;
836 if (std::filesystem::is_regular_file(file, ec))
837 {
838 docs.at(index_(t)) = core::tsv_document::load(file);
839 }
840 }
841 return docs;
842 }
843
844 // Apply layers to copies of the tables, then commit: all or nothing.
845 // With @p reset (load), tables without a document are emptied.
846 void apply_(documents docs, core::layer_mode mode, bool reset)
847 {
848 tables next = _tables;
849 for (const auto t : all_tables_)
850 {
851 auto& doc = docs.at(index_(t));
852 auto& tbl = next.at(index_(t));
853 if (!doc)
854 {
855 if (reset) tbl.clear();
856 continue;
857 }
858 (void)tbl.extend(std::move(*doc), mode, [t](const core::tsv_row& row) { validate_(t, row); });
859 }
860 auto built = build_model_(next);
861 _tables = std::move(next);
862 _model = std::move(built);
863 }
864
865 // -- Row checks and the model -------------------------------------------
866
867 static void require_region_(const core::tsv_row& row)
868 {
869 if (row["region"].size() != 2) throw row.error("region", "must be an ISO 3166-1 alpha-2 code");
870 }
871
872 static void require_chars_(const core::tsv_row& row, std::string_view column, bool dots, std::string_view message)
873 {
874 for (const char c : row[column])
875 {
876 const bool ok = (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || (dots && (c == '.' || c == '-'));
877 if (!ok) throw row.error(column, std::string{message});
878 }
879 }
880
881 static void validate_provider_(const core::tsv_row& row)
882 {
883 const auto domain = row["domain"];
884 if (!domain.contains('.') || domain.front() == '.' || domain.back() == '.')
885 {
886 throw row.error("domain", "must be a domain name, got '" + std::string{domain} + "'");
887 }
888 require_chars_(row, "domain", true, "must be lowercase letters, digits, dots and hyphens");
889 if (row["country"] != "*" && row["country"].size() != 2)
890 {
891 throw row.error("country", "must be '*' or an ISO 3166-1 alpha-2 code");
892 }
893 if (row["kind"] != "free" && row["kind"] != "isp") throw row.error("kind", "must be 'free' or 'isp'");
894 if (!(row.get<double>("weight") >= 0.0)) throw row.error("weight", "must not be negative");
895 }
896
897 static void validate_(table t, const core::tsv_row& row)
898 {
899 switch (t)
900 {
901 case table::countries:
902 require_region_(row);
903 if (row.get_or<double>("population", 1.0) < 0.0) throw row.error("population", "must not be negative");
904 break;
905 case table::fold:
906 if (decode_one_(row["from"]) == 0) throw row.error("from", "must be one character");
907 for (const char c : row["to"])
908 {
909 if (c < 'a' || c > 'z') throw row.error("to", "must be lowercase ASCII letters");
910 }
911 break;
912 case table::particles:
913 if (row["particle"].empty()) throw row.error("particle", "must not be empty");
914 break;
915 case table::profiles:
916 require_region_(row);
917 if (row["profile"].empty()) throw row.error("profile", "must not be empty");
918 break;
919 case table::email_styles:
920 if (row["profile"].empty()) throw row.error("profile", "must not be empty");
921 (void)parse_style_(row);
922 break;
923 case table::providers:
924 validate_provider_(row);
925 break;
926 case table::words:
927 if (word_list_(row) == none_) throw row.error("list", "must be 'adj', 'animal' or 'jp'");
928 if (row["word"].empty()) throw row.error("word", "must not be empty");
929 require_chars_(row, "word", false, "must be lowercase letters and digits");
930 break;
931 }
932 }
933
934 static std::size_t word_list_(const core::tsv_row& row)
935 {
936 const auto l = row["list"];
937 if (l == "adj") return 0;
938 if (l == "animal") return 1;
939 if (l == "jp") return 2;
940 return none_;
941 }
942
943 // The code point of a one-character UTF-8 string; 0 when it is not exactly one.
944 static std::uint32_t decode_one_(std::string_view text)
945 {
946 if (text.empty()) return 0;
947 std::size_t used = 0;
948 const auto cp = decode_(text, 0, used);
949 return used == text.size() ? cp : 0;
950 }
951
952 // Decodes the character at @p i; @p used is its length (1 for a bad byte, with 0 returned).
953 static std::uint32_t decode_(std::string_view text, std::size_t i, std::size_t& used)
954 {
955 const auto c = static_cast<unsigned char>(text[i]);
956 used = 1;
957 if (c < 0x80) return c;
958 std::size_t n = 0;
959 std::uint32_t cp = 0;
960 if ((c & 0xE0U) == 0xC0U)
961 {
962 n = 2;
963 cp = c & 0x1FU;
964 }
965 else if ((c & 0xF0U) == 0xE0U)
966 {
967 n = 3;
968 cp = c & 0x0FU;
969 }
970 else if ((c & 0xF8U) == 0xF0U)
971 {
972 n = 4;
973 cp = c & 0x07U;
974 }
975 else
976 {
977 return 0;
978 }
979 if (i + n > text.size()) return 0;
980 for (std::size_t k = 1; k < n; ++k)
981 {
982 const auto b = static_cast<unsigned char>(text[i + k]);
983 if ((b & 0xC0U) != 0x80U) return 0;
984 cp = (cp << 6U) | (b & 0x3FU);
985 }
986 used = n;
987 return cp;
988 }
989
990 // The pattern part named @p name, or false when there is none; notes in @p st what it needs.
991 static bool note_part_(std::string_view name, style_data& st, tok& kind)
992 {
993 using part = std::pair<std::string_view, tok>;
994 static constexpr std::array<part, 12> parts{{{"f", tok::f}, {"l", tok::l}, {"fi", tok::fi}, {"li", tok::li},
995 {"n", tok::n}, {"y2", tok::y2}, {"y4", tok::y4}, {"r", tok::r},
996 {"r2", tok::r2}, {"adj", tok::adj}, {"animal", tok::animal},
997 {"jp", tok::jp}}};
998 const auto it = std::ranges::find(parts, name, &part::first);
999 if (it == parts.end()) return false;
1000 kind = it->second;
1001 switch (kind)
1002 {
1003 case tok::f:
1004 case tok::fi: st.needs_given = true; break;
1005 case tok::l:
1006 case tok::li: st.needs_family = true; break;
1007 case tok::n: st.needs_nickname = true; break;
1008 case tok::y2:
1009 case tok::y4: st.needs_year = true; break;
1010 case tok::adj: st.needs_words[0] = true; break;
1011 case tok::animal: st.needs_words[1] = true; break;
1012 case tok::jp: st.needs_words[2] = true; break;
1013 case tok::lit:
1014 case tok::r:
1015 case tok::r2: break;
1016 }
1017 return true;
1018 }
1019
1020 static style_data parse_style_(const core::tsv_row& row)
1021 {
1022 style_data st;
1023 const auto text = row["pattern"];
1024 st.weight = row.get<double>("weight");
1025 if (!(st.weight >= 0.0)) throw row.error("weight", "must not be negative");
1026 std::string lit;
1027 const auto flush = [&] {
1028 if (!lit.empty()) st.tokens.push_back({.kind = tok::lit, .text = std::move(lit)});
1029 lit.clear();
1030 };
1031 for (std::size_t i = 0; i < text.size(); ++i)
1032 {
1033 const char c = text[i];
1034 if (c != '{')
1035 {
1036 const bool literal = (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '.' || c == '_' || c == '-';
1037 if (!literal) throw row.error("pattern", std::string{"unexpected '"} + c + "' in '" + std::string{text} + "'");
1038 lit += c;
1039 continue;
1040 }
1041 const auto close = text.find('}', i);
1042 if (close == std::string_view::npos) throw row.error("pattern", "unclosed '{'");
1043 const auto name = text.substr(i + 1, close - i - 1);
1044 flush();
1045 tok kind{};
1046 if (!note_part_(name, st, kind)) throw row.error("pattern", "unknown part '{" + std::string{name} + "}'");
1047 st.tokens.push_back({.kind = kind, .text = {}});
1048 i = close;
1049 }
1050 flush();
1051 if (st.tokens.empty()) throw row.error("pattern", "must not be empty");
1052 return st;
1053 }
1054
1055 static void read_countries_(model& m, const tables& tbls)
1056 {
1057 for (const auto& row : tbls.at(index_(table::countries)).rows())
1058 {
1059 country_data c;
1060 c.region = upper_(row["region"]);
1061 c.population = std::max(row.get_or<double>("population", 1.0), 1.0);
1062 m.by_region.emplace(c.region, m.countries.size());
1063 m.codes.push_back(c.region);
1064 m.countries.push_back(std::move(c));
1065 }
1066 for (std::size_t i = 0; i < m.countries.size(); ++i)
1067 {
1068 m.all_population.add(i, m.countries[i].population);
1069 m.all_uniform.add(i, 1.0);
1070 }
1071 }
1072
1073 static void read_text_tables_(model& m, const tables& tbls)
1074 {
1075 for (const auto& row : tbls.at(index_(table::fold)).rows())
1076 {
1077 m.fold[decode_one_(row["from"])] = std::string{row["to"]};
1078 }
1079 for (const auto& row : tbls.at(index_(table::particles)).rows()) m.particles.insert(std::string{row["particle"]});
1080 for (const auto& row : tbls.at(index_(table::words)).rows())
1081 {
1082 m.words.at(word_list_(row)).emplace_back(row["word"]);
1083 }
1084 }
1085
1086 static void read_styles_(model& m, const tables& tbls)
1087 {
1088 for (const auto& row : tbls.at(index_(table::email_styles)).rows())
1089 {
1090 const std::string name{row["profile"]};
1091 auto it = m.profile_index.find(name);
1092 if (it == m.profile_index.end())
1093 {
1094 it = m.profile_index.emplace(name, m.profiles.size()).first;
1095 m.profiles.emplace_back();
1096 }
1097 m.profiles[it->second].push_back(parse_style_(row));
1098 }
1099 for (const auto& row : tbls.at(index_(table::profiles)).rows())
1100 {
1101 const auto it = m.by_region.find(upper_(row["region"]));
1102 if (it == m.by_region.end())
1103 {
1104 throw row.error("region", "no country '" + std::string{row["region"]} + "' in countries.tsv");
1105 }
1106 const auto pi = m.profile_index.find(std::string{row["profile"]});
1107 if (pi == m.profile_index.end())
1108 {
1109 throw row.error("profile", "no style for profile '" + std::string{row["profile"]} + "' in email_styles.tsv");
1110 }
1111 m.countries[it->second].profile = pi->second;
1112 }
1113 }
1114
1115 // Each country's providers: its own rows plus the global ones it does not replace, without the
1116 // domains switched off by a weight of 0 (they are not drawn, weighted or not).
1117 static void read_providers_(model& m, const tables& tbls)
1118 {
1119 std::unordered_map<std::string, std::vector<provider_data>> by_country;
1120 for (const auto& row : tbls.at(index_(table::providers)).rows())
1121 {
1122 by_country[upper_(row["country"])].push_back(
1123 {.domain = std::string{row["domain"]}, .kind = std::string{row["kind"]}, .weight = row.get<double>("weight")});
1124 }
1125 const auto global = by_country.find("*");
1126 for (auto& c : m.countries)
1127 {
1128 const auto own = by_country.find(c.region);
1129 if (own != by_country.end()) c.providers = own->second;
1130 if (global != by_country.end())
1131 {
1132 for (const auto& g : global->second)
1133 {
1134 const bool replaced = own != by_country.end() &&
1135 std::ranges::any_of(own->second, [&](const provider_data& d) { return d.domain == g.domain; });
1136 if (!replaced) c.providers.push_back(g);
1137 }
1138 }
1139 std::erase_if(c.providers, [](const provider_data& d) { return !(d.weight > 0.0); });
1140 for (std::size_t k = 0; k < c.providers.size(); ++k) c.provider_weights.add(k, c.providers[k].weight);
1141 }
1142 }
1143
1144 static model build_model_(const tables& tbls)
1145 {
1146 model m;
1147 read_countries_(m, tbls);
1148 read_text_tables_(m, tbls);
1149 read_styles_(m, tbls);
1150 read_providers_(m, tbls);
1151 return m;
1152 }
1153
1154 void require_data_() const
1155 {
1156 if (_model.countries.empty())
1157 {
1158 throw std::runtime_error("No email data loaded. Call load() first.");
1159 }
1160 }
1161};
1162
1163} // namespace dasmig
1164
1165#endif // DASMIG_EMAILGEN_HPP
Return type for email generation.
Definition emailgen.hpp:75
std::string to_string() const
Human-readable summary string.
Definition emailgen.hpp:99
friend std::ostream & operator<<(std::ostream &os, const email &v)
Stream the address to an output stream.
Definition emailgen.hpp:137
std::string address() const
The address.
Definition emailgen.hpp:85
std::string country_code
ISO 3166-1 alpha-2 of the country whose style the address follows.
Definition emailgen.hpp:77
std::string to_json() const
This address as a JSON document.
Definition emailgen.hpp:119
std::string local
The part before the @: [a-z0-9._-], at most 64 characters.
Definition emailgen.hpp:78
const core::seed_signature & signature() const
The seed and the data this address was generated from.
Definition emailgen.hpp:95
bool reserved
True when the domain is one RFC 2606 reserves (example.com, .net, .org): the safe option.
Definition emailgen.hpp:81
void to_json(core::json_writer &w) const
Write this address as a JSON object.
Definition emailgen.hpp:103
std::string domain
The part after the @.
Definition emailgen.hpp:79
std::uint64_t seed() const
Retrieve the random seed used to generate this address.
Definition emailgen.hpp:90
std::string kind
free (webmail) or isp (a provider of the country); empty for a reserved domain.
Definition emailgen.hpp:80
Email Generator: email addresses in each country's style.
Definition emailgen.hpp:175
std::vector< email > get_email_batch(const email_request &request, const core::batch_options &options)
Generate a batch of email results from a seed drawn like get_email() draws its own.
Definition emailgen.hpp:244
void for_each_email(const email_request &request, const core::batch_options &options, std::uint64_t batch_seed, Sink &&sink) const
Stream a batch of email results to a sink without holding the whole batch.
Definition emailgen.hpp:279
email get_email(const email_request &request, std::uint64_t call_seed) const
Generate a deterministic email address.
Definition emailgen.hpp:215
std::size_t country_count() const
Return the number of countries loaded.
Definition emailgen.hpp:307
static emg & instance()
Access the global singleton instance.
Definition emailgen.hpp:192
emg(emg &&) noexcept=default
Movable.
bool has_data() const
Check whether any data has been loaded.
Definition emailgen.hpp:303
std::uint64_t data_hash() const
Hash of the loaded data (all tables), recorded in each result's signature().
Definition emailgen.hpp:445
void extend(const std::filesystem::path &path, core::layer_mode mode=core::layer_mode::override)
Add data layers from a directory or a file.
Definition emailgen.hpp:379
emg & operator=(const emg &)=delete
Not copyable.
bool load(dataset tier)
Load a dataset tier, replacing all data and layers loaded before.
Definition emailgen.hpp:352
emg()
Default constructor — creates an empty generator with no data.
Definition emailgen.hpp:179
const core::layer_info & extend(email_table which, core::tsv_document doc, core::layer_mode mode=core::layer_mode::override)
Add a data layer to one table.
Definition emailgen.hpp:411
void for_each_email(const email_request &request, const core::batch_options &options, Sink &&sink)
Stream a batch of email results with a batch seed drawn like get_email() draws its own.
Definition emailgen.hpp:294
void load(const std::filesystem::path &dir)
Load data from a resource directory, replacing all data and layers loaded before.
Definition emailgen.hpp:331
const std::vector< core::layer_info > & layers(email_table which) const
The layers in effect for one table, in order.
Definition emailgen.hpp:437
emg(const emg &)=delete
Not copyable.
bool has_country(std::string_view country) const
Whether a country is loaded.
Definition emailgen.hpp:317
email get_email(const email_request &request)
Generate a random email address. Use email_request{} for the defaults.
Definition emailgen.hpp:206
const std::vector< std::string > & countries() const
The countries an address can be generated for.
Definition emailgen.hpp:312
const core::layer_info & extend_from_string(email_table which, std::string_view text, std::string source, core::layer_mode mode=core::layer_mode::override)
Add a data layer to one table from TSV text (for example from a WebAssembly host).
Definition emailgen.hpp:428
std::vector< email > get_email_batch(const email_request &request, const core::batch_options &options, std::uint64_t batch_seed) const
Generate a batch of email results, deterministically.
Definition emailgen.hpp:263
email_table
The tables of the email data, for extending it.
Definition emailgen.hpp:150
@ fold
fold.tsv: letters and their ASCII form.
@ providers
providers.tsv: the domains addresses use, by country.
@ countries
countries.tsv: the countries and their populations.
@ particles
particles.tsv: family-name particles email names skip.
@ words
words.tsv: the words of the handle styles.
@ email_styles
email_styles.tsv: the shapes of an email's local part, by profile.
@ profiles
profiles.tsv: the style profile of each country.
What an email call asks for.
Definition emailgen.hpp:47
bool weighted
Weighted selection (default): countries by population, styles and domains by their weights....
Definition emailgen.hpp:67
std::string nickname
Nickname or handle; empty when unknown. // NOLINT(readability-redundant-member-init): keeps -Wextra q...
Definition emailgen.hpp:55
std::optional< std::string > country
ISO-3166 two-letter country code (any case), or a country drawn when absent. The country sets the sty...
Definition emailgen.hpp:51
std::string given
Given name; empty when unknown. // NOLINT(readability-redundant-member-init): keeps -Wextra quiet for...
Definition emailgen.hpp:53
std::string family
Family name; empty when unknown. // NOLINT(readability-redundant-member-init): keeps -Wextra quiet fo...
Definition emailgen.hpp:54
std::optional< int > birth_year
Birth year, for the ana85 and ana1985 styles; those styles are left out when absent.
Definition emailgen.hpp:59
bool safe
Use a reserved domain (example.com, .net, .org, RFC 2606), so a message sent to the address reaches n...
Definition emailgen.hpp:63