Requirements for a messages function registry with types

[jldec]

Context

Messages can be defined with variable replacements or match selectors which contain functions.

E.g. message with a plurals function (from MESDK-77 here):

.input {$numProducts}
.match {$numProducts :plural}
zero {{No Products}}
one {{A product}}
other {{{$numProducts} products}}

(this example uses MF2, but inlang v2 Message type semantics are the same)

@loris.sigrist has implementated a registry here as an example to be used by paraglide-js

Requirements for the function registry

1. list the available built-in functions (see MF2 default registry)
   1. plurals (e.g. as implemented by javascript Intl.PluralRules)
   2. string, number and date formatting functions
2. inlude types for function parameters

Proposal

[add api proposal here]

[jldec]

@loris.sigrist, @martin.lysk1
Please add additional requirements if I missed something.

cc: @samuel.stroschein

[LorisSigrist]

The main question for the registry is who owns it. Is it part of the Ecosystem itself via the SDK or is it App-specific, or a mix.

If it's owned by the Ecosystem, how is it exposed to the Apps. Do we have an actual Data-Model + API or is it simply a standard that's implemented in each App with no in-memory representation. Simply having a standard seems better IMO.

We already talked about and decided that the best way to represent types in an app agnostic way is with regexes. The registry should represent types with them.

Eg: Date = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}

Apps that need to give Type-Information can have their own mapping from regex -> App-specific type system.

[samuelstroschein]

The main question for the registry is who owns it. Is it part of the Ecosystem itself via the SDK or is it App-specific, or a mix.

The project owns the registry.

As an app, you have to comply. E.g. the project defines a plural function with input: X, output: Y, it's paraglide's (and other exporters/i18n libs) job now to provide this function during the runtime.

For the registry, I propose a property implementation or something that is a web/js implementation of the function that can be used to execute the function in inlang apps e.g. fink.

Okay, wuuuuuups here might be the answer for paraglide and on how to ensure consitency across different i18n libraries. The project owns the implementation with source code stored in the project itself (see implementation: Record<Platform, File> prop):

{
  "plural": {
    "description": "",
    "implementation": {
      "js": "./functions/plural.js",
      "android": "./functions/plural.java"
    },
    "arguments": {
      "x": "\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}"
    }
    "example": "",
  }
}

// inlang.project/functions/plural.js

export default INTL.pluralRules

[LorisSigrist]

In this case, who would provide the mapping from regex <-> TS Type?

Paraglide would need to know the TS type in order to provide the interface.

The way the example is written implies that the functions/plurals.js file already gets a regular JS Date object. If this should be consistent across Inlang apps that would imply that that mapping happens in the SDK. Am I understanding this correctly?

Would we provide default implementations when creating an Inlang project?

[samuelstroschein]

Would we provide default implementations when creating an Inlang project?

For defaults functions like plural, yes i propose we provide a default implementation. But, I revert my course: I don't think leaking source code into an inlang project file is a good idea.

i18n libs should be responsible for providing the function because this is a dev only problem. there is no need to make life difficult by storing source code in an inlang project.

Summary

• have a mock implementation that can be used to provide translators/designers with "if use this function in this message, this will be the output" capability
• paraglide and other i18n libs need to provide their own implementation that they control (and have to fix if it's wrong)

who would provide the mapping from regex <-> TS Type?

Paraglide/developers in an inlang project.

If the registry contains a type, it's up to developers to provide the correct type for. How you compile this in paraglide will be your job then. You can provide a settings field for paraglide that devs can define.

For default functions like plural, you can ship built-in types in paraglide

[LorisSigrist]

Sounds good! I'll try implementing that to see if it holds up, but it seems like it will work

[LorisSigrist]

Findings after implementing this:

• Both the Interface and Implementation must be provided, it's not enough to just have the implementation & do type inference.
• Standard functions like plural need well-defined semantics & options across all Inlang apps.
• It's not practical to share implementations between Editors like Fink/Sherlock and Paraglide. Devs want to give Paraglide complex, environment-specific objects like Date, which can't all realistically be supported as input options by environment-agnostic editors. They need shared semantics but likely won't be able to share implementations.
• Custom, developer-provided functions are possible in Paraglide but likely won't be practical in Editors. IMO as long as we have standard functions for everything covered by the Intl API we probably won't need to support custom functions.

[samuelstroschein]

• Intl API we probably won't need to support custom functions.

Most likely a wrong assumption but doesn't matter anyways because the moment a function registry exists, we can enable adding custom functions.

• It's not practical to share implementations between apps. Devs want to give Paraglide complex, environment-specific objects like Date, which can't all realistically be supported as input options by environment-agnostic editors.

Does it matter what devs want? Translators need predictable input -> output when creating a translation. Cool that Date can express really complex things but useless if translators can't use it.

[LorisSigrist]

I think we're at risk of confusing concrete Implementation and Interface here.

Yes, a formatDate function would need to take a "date" for both Translators and Developers. This conceptual "date" type is defined in the registry & conceptually shared across all apps.

But, we can't share an implementation of the formatDate function across all apps. An android/iOS dev is not going to be able to use a JS function. What they can do is implement a function for their platform that implements the same options & semantics.

We will need platform-specific implementations of the function anyways. As long as the interface & semantics are the same that doesn't matter.

Translators need predictable input -> output when creating a translation

They have that as long as they use the registry. There is no downside to the platform specific implementation using platform specific types as long as they have the same semantics.

[samuelstroschein]

An android/iOS dev is not going to be able to use a JS function. What they can do is implement a function for their platform that implements the same options & semantics.

Oh yes. Can you confirm with the info below that we talk about the same thing?

1. The function registry contains a spec for a function.
2. The function registry contains a "mock implementation" (in JS) that obeys to the defined spec. Inlang apps can use the mock implementation to preview output.
3. Exporters/i18n libs are responsible for providing functions that obey to the spec

[LorisSigrist]

That works!

The registry defines the functions that are available + the interface. It uses regex strings to define the shared types.

The JS "mock implementations" take strings values that obey the regexes & render correctly. They can be used by Editors / Apps to show previews without needing to use any Platform Types.

Platform i18n libraries (like Paraglide) provide their own platform specific implementations that is equivalent to the mock implementation but may use Platform types instead of just strings.