> The Unicode input alphabet is up to 4096 symbols. Huh? How did you come up wit... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		jwilk on May 4, 2023 \| parent \| context \| favorite \| on: Why split lexing and parsing into two separate pha... > The Unicode input alphabet is up to 4096 symbols. Huh? How did you come up with this number?

Eliah_Lakhin on May 4, 2023 [–]

ah, I apologize for misdirection. In general it could be much bigger of course. Wikipedia[1] says Utf-8 can encode up to 1,112,064. Anyway, it is quite big "alphabet" than the usual set of Lexis Tokens :)

[1] https://en.wikipedia.org/wiki/UTF-8

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact