Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
.. contents::

CHANGES
=======
========


5.0.3dev0
---------
Expand Down Expand Up @@ -42,6 +43,8 @@ Bugs

# ``0`` with a given precision (like in ```0`3```) is now parsed as ``0``, an integer number.
#. ``RandomSample`` with one list argument now returns a random ordering of the list items. Previously it would return just one item.
#. Improving parsing ``RowBox`` expressions including ``FormBox`` tags (``` \` ```) inside.



Enhancements
Expand Down
24 changes: 12 additions & 12 deletions mathics/builtin/makeboxes.py
Original file line number Diff line number Diff line change
Expand Up @@ -339,38 +339,38 @@ class BoxForms_(Predefined):


class MakeBoxes(Builtin):
"""
r"""
<dl>
<dt>'MakeBoxes[$expr$]'
<dd>is a low-level formatting primitive that converts $expr$
to box form, without evaluating it.
<dt>'\\( ... \\)'
<dd>is a low-level formatting primitive that converts $expr$ to box form, without evaluating it.

<dt>'\( ... \)'
<dd>directly inputs box objects.
</dl>

String representation of boxes
>> \\(x \\^ 2\\)
>> \(x \^ 2\)
= SuperscriptBox[x, 2]

>> \\(x \\_ 2\\)
>> \(x \_ 2\)
= SubscriptBox[x, 2]

>> \\( a \\+ b \\% c\\)
>> \( a \+ b \% c\)
= UnderoverscriptBox[a, b, c]

>> \\( a \\& b \\% c\\)
>> \( a \& b \% c\)
= UnderoverscriptBox[a, c, b]

#> \\( \\@ 5 \\)
#> \( \@ 5 \)
= SqrtBox[5]

>> \\(x \\& y \\)
>> \(x \& y \)
= OverscriptBox[x, y]

>> \\(x \\+ y \\)
>> \(x \+ y \)
= UnderscriptBox[x, y]

#> \\( x \\^ 2 \\_ 4 \\)
#> \( x \^ 2 \_ 4 \)
= SuperscriptBox[x, SubscriptBox[2, 4]]

## Tests for issue 151 (infix operators in heads)
Expand Down
60 changes: 43 additions & 17 deletions mathics/core/parser/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -308,13 +308,53 @@ def p_RawLeftAssociation(self, token):
def p_LeftRowBox(self, token):
self.consume()
children = []
self.box_depth += 1
self.bracket_depth += 1
# If this does not happen, it would be because
# it was called when a `FormBox` (or any other)
# was found. More generally, we could use
# ``token.tag == "LeftRowBox"
# if there were other Tokens with a
# similar behaviour (see bellow).

if token.tag != "FormBox":
self.box_depth += 1
self.bracket_depth += 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every place there is some depth that increases by one, there should be another place that decreases the depth by one.

Where is that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the tat is not LeftRowBox then we never reach the decrement.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are saying something more or less like what the code says. Why is the decrement not reached. What is the depth used for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are saying something more or less like what the code says. Why is the decrement not reached. What is the depth used for?

The deep was used to handle nested expressions like \( a b \( InputForm \` \(OutputForm \` c d\) e \) \)

When a \` is found, then the previous tokens are collected, and evaluated as a Symbol, a Box expression or StandardForm if there is no token before \`. The result of this step is stored in a variable, to be used as the second argument of FormBox.
Then, a new call to the method is done, with FormBox as a tag. In the new call, the next elements are processed until the bracket and box deeps reach 0. Then, these elements are collected into a String or a RowBox. Then, the instance where \` was found takes the format and the other collected elements, builds a FormBox[], and returns it (before reaching the line where the decrements happen).

I tried to find a way to put the increment and the decrement together, but it just was more involved than this approach: I should have allowed the increment of the deep variables, and then doing the decrement before returning the FormBox. I didn't think that it was clearer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the new comment helps to understand the logic.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the description. Something about this seems a little to complicated and that there should be something that follows the specification in a little more direct way:

I would like to think about and reflect on this some more. I think we can make this cleaner, clearer and simpler.

What would be ideal is if we could do a cleanup step first - no changes to behavior yet.
Then add the list of tests that are wrong and that we want to fix.

And after this, then come up with a way to implement this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • ∖(∖`input∖) yields FormBox[input,RawForm].

Actually, in WMA,

In[1]:= \(\`input\)                                                             

Out[1]= FormBox[input, StandardForm]

I would like to think about and reflect on this some more. I think we can make this cleaner, clearer and simpler.

I am sure it is possible, just that after trying for a while, I didn't find a better way.

What would be ideal is if we could do a cleanup step first - no changes to behavior yet. Then add the list of tests that are wrong and that we want to fix.

What would be a cleanup step? The list of (simple) tests to check are the tests that I added in test/core/parser/test_parser.py.

And after this, then come up with a way to implement this.

OK, in that case, I am going to put this as a draft for a while.


token = self.next()
while token.tag not in ("RightRowBox", "OtherscriptBox"):
while token.tag not in ("RightRowBox", "OtherscriptBox", "FormBox"):
newnode = self.parse_box(0)
children.append(newnode)
token = self.next()

# FormBox token has a particular behaviour: if it is found inside
# a RowBox, it splits the Rowbox in two pieces: the part at the
# left is taken as a "Format" and the part at the right is parsed
# as the Box to be formatted, in a way that
# \(a_1, a_2 ... \` b_1 b_2 ... \) is parsed as
# FormBox[RowBox[{b_1, b_2,...}], RowBox[a_1, a_2, ...]]
# This kind of parsing is not supported by the standard mechanism,
# so we need to processing it here instead of using a p_FormBox
# method.
# The strategy to deal with is that, when a "FormBox" tag is found,
# the collected elements are used to build the second argument of the
# output, and then the method is called again to parse the rest of the
# box as it had started at the "FormBox" tag. In this new call,
# neither `self.box_depth` or `self.bracket_depth` are going to be
# incremented, but are decremented at the end of the child expression.
if token.tag == "FormBox":
if len(children) == 0:
fmt = Symbol("StandardForm")
elif len(children) == 1:
fmt = children[0]
if type(fmt) is String:
fmt_name = fmt.value
if is_symbol_name(fmt_name):
fmt = Symbol(fmt_name)
else:
fmt = Node("Removed", String("$$Failure"))
else:
fmt = Node("RowBox", Node("List", *children))
rest = self.p_LeftRowBox(token)
return Node("FormBox", rest, fmt)
if len(children) == 0:
result = String("")
elif len(children) == 1:
Expand Down Expand Up @@ -840,20 +880,6 @@ def b_FractionBox(self, box1, token, p):
box2 = self.parse_box(q + 1)
return Node("FractionBox", box1, box2)

def b_FormBox(self, box1, token, p):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because with the current logic, to implement the behavior of FormBox is not possible using this mechanism, so in my proposal, this method is never used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is what I mean about the myopic kind of thing. FormBox needs to worry about precedence and the b_ routines are where this kind of thing is done. So if you are defeating that, then maybe the thinking around this is flawed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was aware about the possible issue with the precedence. So I checked it against WMA, and it also seems to neglect the operator precedence of \`, so it gives the maximum precedence to it. On the other hand, the tokens that follows \` are processed using all the precedence rules.

What makes me think that this is the right approach (or at least a better approach than the current one) is that it does not break the existing tests, and it just makes a difference if a \` token is found inside a LeftRowBracket/RightRowBracket expression.

q = misc_ops["FormBox"]
if q < p:
return None
if box1 is None:
box1 = Symbol("StandardForm") # RawForm
elif is_symbol_name(box1.value):
box1 = Symbol(box1.value, context=None)
else:
box1 = Node("Removed", String("$$Failure"))
self.consume()
box2 = self.parse_box(q)
return Node("FormBox", box2, box1)

def b_OverscriptBox(self, box1, token, p):
q = misc_ops["OverscriptBox"]
if q < p:
Expand Down
12 changes: 8 additions & 4 deletions test/core/parser/test_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -694,10 +694,14 @@ def testFraction(self):
self.check("\\( \\/ \\)", 'FractionBox["", ""]')

def testFormBox(self):
self.check("\\( 1 \\` b \\)", 'FormBox["b", Removed["$$Failure"]]')
self.check("\\( \\` b \\)", 'FormBox["b", StandardForm]')
self.check("\\( a \\` b \\)", 'FormBox["b", a]')
self.check("\\( a \\` \\)", 'FormBox["", a]')
self.check(r"\( \` b \)", 'FormBox["b", StandardForm]')
self.check(r"\( a \` b \)", 'FormBox["b", a]')
self.check(r"\( a \` \)", 'FormBox["", a]')
self.check(r"\( a \` b + c \)", 'FormBox[RowBox[{"b", "+", "c"}], a]')
self.check(r"\( a \` b \` c \)", 'FormBox[FormBox["c", b], a]')
self.check(r'\( "a" \` b \)', 'FormBox["b", Removed["$$Failure"]]')
self.check(r"\( 3.2 \` b \)", 'FormBox["b", Removed["$$Failure"]]')
self.check(r"\( 3.2 + a \` b \)", 'FormBox["b", RowBox[{"3.2", "+", "a"}]]')

def testRow(self):
self.check("\\( \\)", String(""))
Expand Down