[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20260512230930.GH1827@brightrain.aerifal.cx>
Date: Tue, 12 May 2026 19:09:32 -0400
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Re: musl multi-level table format for binary locale images
On Sat, May 09, 2026 at 11:04:13PM -0400, Rich Felker wrote:
> On Fri, May 08, 2026 at 11:22:28PM -0400, Rich Felker wrote:
> > The concepts here have been presented before; what follows is an
> > informal spec of the actual mappable image format that has emerged
> > from earlier design proposals and discussion and from implementation
> > of draft tooling.
> >
> > The multi-level tables used here are in some sense a data-driven
> > generalization of the multi-level tables used elsewhere in musl for
> > character data, with headers at each level defining the range covered
> > and bits examined at the level covered.
> >
> > Some of the details here are not yet matched by the draft tooling, and
> > all may still be subject to further change until integration and
> > release.
> >
> > This is part of the locale support overhaul project, funded by NLnet
> > and the NGI Zero Core Fund.
> >
> >
> >
> >
> >
> > FILE FORMAT
> > [...]
>
> Give me another day or so to clean this up and post it, but I've now
> implemented both the generation and lookup for the binary locale
> images. Generation does not make use of shift!=0 (multi-level)
> functionality, which is presently unneeded outside of collation data;
> this will be added later, and should be easy to do as a
> transformationon the in-memory representation prior to serialization.
>
> Right now I have the code bolted haphazardly to the existing
> parselocale.c to build the image then query back the items in
> parselocale.c's table to check that they round-trip. I'll probably
> clean this up a little bit to be reasonable to commit into the
> development history, then go on to write a simple localedef(1) entry
> point.
>
> This leaves details of collation and musl integration as the main
> remaining parts of the locale project.
An integration of the parser, binary table generation, and
localedef(1) frontend is up at:
https://codeberg.org/dalias/musl-locale-tools-draft
Current version at the time of this email:
https://codeberg.org/dalias/musl-locale-tools-draft/src/commit/19bf9dc524353232b03735f410490895248ee5b1
The code to perform lookups is not yet merged much less hooked up to
any test framework, but I'm attaching a draft to this email. It needs
to be pointed at the start of the actual table (after the 16-byte file
header).
Rich
View attachment "lookup.c" of type "text/plain" (1035 bytes)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
