Does "directly in an enclosing scope" include dialects scopes?

[apblack]

I raised this issue as part of another discussion on 18 June 2020. That was foolish of me, because it got lost.

Here is the issue again:

In the spec, we say:

When interpreting an implicit request of a method named m, the usual rules of lexical scoping apply, so a definition of m in the current scope will take precedence over any definitions in enclosing scopes. However, if m is defined in the current scope by inheritance or trait use, rather than directly, and also defined directly in an enclosing scope, then an implicit request of m is ambiguous, and is an error.

The issue is: when we say "directly in an enclosing scope", did we mean to include dialect scopes?

If we did, then it matters whether a definition appears directly in a dialect, or is reused by the dialect. This means, for example, that a refactoring of the standard dialect to move everything into a trait to enable dialect combination, like the one that I performed in the fall, would cause an ambiguity to disappear (not a big problem!), but that the inverse refactoring (moving from a trait to a direct definition) will create an ambiguity when there didn't use to be one. This seems to me to be Bad.

Practically, it also means that the external representation of a compiled dialect has to distinguish between reused and direct definitions, just because of the disambiguation rule.

Hence, I'm inclined to think that when we wrote "directly in an enclosing scope", we meant "visibly in the text that you can see surrounding the request that you are trying to disambiguate", and did not intend to include the dialect at all. By this interpretation, a reused definition will always override a definition from a dialect, independent of whether the dialect declares the conflicting name directly, or through reuse.

[apblack]

Having just opened this issue, I now propose closing it by changing the wording in the specification as follows:

When interpreting an implicit request of a method named m, the usual rules of lexical scoping apply, so a definition of m in the current scope will take precedence over any definitions in enclosing scopes. However, if m is defined in the current scope by inheritance or trait use,
rather than directly, and also directly defined in a lexically-enclosing scope, then
an implicit request of m is ambiguous, and is an error. The term directly defined in a lexically-enclosing scope excludes an indirect definition of m a dialect scope, and an indirect definition of m by reuse, because such definitions, are indirect, that is, are not visible in the text of the program.

It might be worth factoring-out the terms direct definition and indirect definition and defining them separately.

[kjx]

I'm sure we agreed somewhere not to talk about "compiled representations" of anything...

other than that, it's a pity if we have to make this even more complicated than it already is...

[KimBruce]

It's hard for me to get a deep understanding of the issue. Can you provide an example?

…

On Thu, Feb 23, 2023 at 4:59 PM Andrew Black ***@***.***> wrote: I raised this issue as part of another discussion on 18 June 2020 <#102 (comment)>. That was foolish of me, because it got lost. Here is the issue again: In the spec, we say: When interpreting an implicit request of a method named m, the usual rules of lexical scoping apply, so a definition of m in the current scope will take precedence over any definitions in enclosing scopes. However, if m is defined in the current scope by inheritance or trait use, rather than directly, and also defined directly in an enclosing scope, then an implicit request of m is ambiguous, and is an error. When we say "directly in an enclosing scope", did we mean to include dialects? If we did, then it matters whether a definition appears directly in a dialect, or is reused by the dialect. This means, for example, that a refactoring of the standard dialect to move everything into a trait to enable dialect combination, like the one that I performed in the fall, would cause an ambiguity to disappear (not a big problem!), but that the inverse refactoring (moving from a trait to a direct definition) will create an ambiguity when there didn't use to be one. This seems to me to be *Bad*. Practically, it also means that the external representation of a compiled dialect has to distinguish between reused and direct definitions, just because of the disambiguation rule. Hence, I'm inclined to think that when we wrote "directly in an enclosing scope", we meant "visibly in the text that you can see surrounding the request that you are trying to disambiguate", and did not intend to include the dialect at all. By this interpretation, a reused definition will always override a definition from a dialect, independent of whether the dialect declares the conflicting name directly, or through reuse. — Reply to this email directly, view it on GitHub <#198>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN2D6TGPHDXULOG37RBH4DWZABWNANCNFSM6AAAAAAVGJXVBA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Prof. Kim Bruce Computer Science Pomona College

[apblack]

I've just spent an hour or more constructing an example and explaining everything, only to have Safari throw it all away before I submitted it. So now I'll start again, but with less patience. Sorry!

A working example needs three modules: "trivialBundle" defines the guts of a dialect; the dialect module "trivial" uses "trivialBundle":

module trivialBundle

trait open {
    method hash { myIdentityHash }
}

module trivial

import "trivialBundle" as trivialBundle
use trivialBundle.open

Recall why we define dialects in this way: it's to permit dialects to be combined. Traits can be reused, but modules, being objects, cannot.

Finally, we can define the example module

dialect "trivial"
import "equalityBundle" as equalityBundle

def testObject = object {
    use equalityBundle.open.identityEquality    // defines hash, ==, ≠, etc.
    
    method testMethod {
        print "testObject's hash is {hash}"
    }
}

testObject.testMethod

print "the dialect (i.e., {outer})'s hash is {outer.hash} = {hash}"
print "this module (i.e., {self})'s hash is {self.hash}"

Note thet testObject has a hash method, but it's reused from equalityBundle.open.identityEquality rather than being defined directly.

The interesting request of hash is the receiverless request in the testMethod of testObject.
According to the Grace specification, we have to decide which object is being requested to answer its hash. It might be testObject, but if a lexically-enclosing object also defines hash directly, then the request is ambiguous.

In this example, the request of hash is is not ambiguous, because, although the dialect object is lexically enclosing, and although it also has a hash method, that hash method is not defined directly in the dialect, but rather indirectly through use trivialBundle.open.

The print statement on the penultimate line is there to convince you that the dialect does indeed have a hash method, and that the dialect is indeed the outer object. The final print statement is there to make clear that there is no hash on the example module itself.

Here is the output:

testObject's hash is -559039210
the dialect (i.e., the "trivial" module)'s hash is -559039211 = -559039211
NoSuchMethod: no method hash on the "example" module (defined in module example, line 1).

Now, suppose that the implementor of the trivial dialect decides that our "bundle ... open" convention is overkill for a dialect that contains exactly one definition, and re-writes module trivial as follows:

method hash { myIdentityHash }

With this new version of the "trivial" module, the dialect "trivial" defines hash directly, and testMethod's implicit request of hash has become ambiguous according to our language specification. It's ambiguous because there is a re-used definition of hash ("up") and a direct lexically-enclosing definition of hash ("out").

I believe that this is silly. The implementor of a dialect should be able to restructure the internals of that dialect at will, without having any effect on the consumers of that dialect. This is the reason that confidential in a dialect means "private to the dialect", which is different from confidential in other objects.

If you try this refactoring, you might notice that minigrace does not report an ambiguity. Here's what I wrote about this previously:

This morning I sat down to actually implement ... that names defined by a dialect through reuse don't cause ambiguity, whereas names defined directly in a dialect do cause an ambiguity. ...
After half an hour in the debugger trying to see why it didn't work, I hit myself on the head and realized: it can't possibly work.

Why not? Because, not only don't programmers know which parts of a dialect are inherited and which are local to the dialect module, the compiler also does not know. The dialect module is separately compiled. It's a black box that provides definitions. The compiler of a module that uses a dialect has no idea how that dialect was constructed.

Of course, the fact that minigrace doesn't follow the spec isn't a reason to change the spec; with enough effort I could expose the internal structure of a module as part of the compiled module. I have not been willing to do so because I believe that it's the wrong thing to do. Modules (unlike classes) are intended to be black boxes, and breeching that encapsulation boundary for the sole purpose of making more implicit requests ambiguous seems silly.

[KimBruce]

Sorry to be slow in responding to all of this, but I am trying to get back into it. One question is why we bother only with direct definitions when worrying about ambiguity. Do we really care how (directly or indirectly) hash was defined in the dialect if it was indeed provided by the dialect? It seems like we should get an ambiguity warning either way. After all, it is trivial to fix the issue by adding receiver "self" or "outer". Am I missing something here?

I agree there is something different about modules when used as dialects rather than simply being imported. Confidential items should not be visible when using a dialect. I'm not sure that is analogous to anything else in the language as we are pretending not to see something in an enclosing scope (and typically you see everything in an enclosing scope). Of course we do want to be able to access classes (with confidential methods) defined in a dialect so we can inherit in the client without accidentally overriding existing methods

[apblack]

@KimBruce asks: "why we bother only with direct definitions when worrying about ambiguity". This question applies not just to dialect scopes but to all outer scopes.

I believe that the reason that we care for lexical outer scopes is that a direct definition is visible in the program text, whereas a reused definition is invisible. One might be surprised, then, if an invisible definition acquired through reuse is chosen in preference to a definition visible in an enclosing scope — and that is the reason to prefer "out" to "up".

However, since Grace chooses to tell the programmer about the ambiguity rather than silently pick the wrong method, this argument doesn't really hold — we could, as you say, just require the programmer to disambiguate.

Another reason to treat indirect definitions differently from direct ones is that, in a generalized language, reused classes and traits might not be manifest. Then, we might not be able to tell statically whether or not a request is ambiguous, because we might not know statically what methods an object acquires through reuse. In contrast, we always know about the direct definitions.

@kjx is the authority on what languages do which thing and why. Perhaps he can tell us?