extend concept matching description #477
Conversation
|
As stated in the meeting, I am against adding such text to the main spec, but it may be useful to continue to clarify the Core dictionary workflows (in mathml-docs). As we agreed in the meeting on 2023-10-19, the "arity" field is a practical shorthand that describes the argument count in (an optional) speech hint. It has to match the number of arguments in the hint to be useful, and as such it is the hint rule that is of primary relevance. That makes arity non-normative, as are also the hints themselves. A concept entered into the Open list without a speech hint should not be specified to be an "unknown concept" - an implementer may add support for it without updating the list. I would also endorse Open lists that do not have an arity column at all, if that can be auto-inferred from the hint rule. Recall that I'm also not in favour of the requirement of a "fixity property" for a concept entry, especially in the Open realm. We have discussed this at length in the past, so I won't rehash that here. As per the current text:
I think that is already sufficient. Speech hints should not be a requirement for Open list inclusion, and should be an implementation detail of the lists, rather than part of the main MathML spec. My perspective is that the main contribution of annotating with intent is anchoring what was meant - e.g. that a superscript was power - rather than to provide exhaustive narration directives for AT implementers. A tool conforming to Core should be able to handle all concepts in the Core list, but not necessarily use the exact speech hints (and their arity) in that list. |
You were actually the one who said that the arity matching description should be in the spec and not just in the core concept list if it is to be kept, and the meeting gave me an explicit action item to make this PR, so while your concern is noted I don't think that should block the PR.
Arities can not always be inferred from the speech template, for example you can have a speech hint for log(x,10) or power(x,2) that matches a 2-argument use but does not mention the all the arguments. In most cases they can (or if you include the side conditions perhaps they always can) but it makes no sense to make every implementer have to parse the ad hoc template speech fragments to work that out when we can have a column of the table.
I actually can't guess what you mean by that, the arity is not a rule that can be normative or non normative, it is simply a statement of fact about a particular explicit instance. If you have This PR does not change the normative behaviour of the spec in any way, we already reference the core concept list and the text at the top of that file just describes the format of the table, and this PR just adds a version of that description to the main spec. |
Yes of course we make explicit in multiple places that an implementation is not obliged to use the speech forms suggested by the speech templates. |
|
Personally, I think the proposed text could be slightly less prescriptive (and limiting). Perhaps the sentence starting "The use of a concept matches an entry in the dictionary if..." could be "A speech hint from the dictionary is applicable when...". Similarly, the clause for when it does not match ("then it is treated as...") might be "then it may be treated as...". |
@brucemiller hmm OK I'll adjust a bit. Actually there is nothing prescriptive because the formulation is similar to the operator dictionary that an implementation should have a concept dictionary that should have at least the things in core. So if an implementation has some rule to read some intent somehow then it has an entry in its concept dictionary so by definition if it doesn't match it's an unknown concept and the spec has always said they should be read as is. But I agree, given that, it could be made to read less prescriptive... The text is not intended to imply that if a concept/arity/property combination is not in the specific core list we provide that it has to be read literally. Implementations can always support more entries, from the open list or elsewhere. (Perhaps it wouldn't hurt to say that explicitly again here?) All this is saying is what we have always said is that if a particular concept/arity/property combination is not known to the AT system it should read the concept literally in the style indicated by the fixity property. so |
What we have always stated/agreed on is the clause "if a particular concept is not known to the AT system". Codifying how a concept is recognized by a system seems to lift the implementation choices of one implementer to the level of MathML Intent conformance. As someone who built an intent prototype back in 2021, I can see at least a couple of different ways to organize the necessary abstractions internally to a system. For example, a simple template-based approach would need neither fixity nor arity exposed as surface constructs. An intent expression I have a well-documented record of disagreeing that "fixity properties" should be a conformance part of Intent - i.e. it is important to me that Intent should remain usable without reliance on these properties - and I think we've been successful in making the spec reflect that. My rationale for fixity also extends to arity - any specific "Intent List" may choose to include arity information (for the associated speech hint), but a generic "Intent expression" should be possible to create and consume without "arity" playing any mandatory role. |
If you have a rule that says A to the missing power then you have recognised that this is a 1 argument form, which is an explicit recognition of the arity. There is nothing here that stops an implementation having such a rule, it is just not in the core list (which would imply every implementation should have such a rule).
Noted, but we can not re-do every decision the group has made over the past four years You can of course generate intent without using these properties, but if they are there and you are consuming them then the core properties should be honoured, that's why they are core. |
|
This confusion (if that's what it is) is exactly what I was trying to avoid by a bit of constructive hair-splitting: As I understand it, in the AT system we're designing, whether a concept is "known", or not, is relevant only to the extent there is a speech template (or algorithm) for the specific use case (arity, fixity) encountered. The core dictionary provides a set of suggested speech templates for a collection of concept names, arity and fixity. If an intent instance doesn't match name, arity, fixity then it is not "known". But if this very narrow, AT-specific use of the word "known" is problematic, possibly in the context of unknowable future applications of the open dictionary, it may be better to talk about whether a speech hint is applicable, rather than whether a concept is known. Or, perhaps I've completely missed the whole point of this debate. |
|
@brucemiller yes was hoping to reword based on your previous comments but I had hardware issues this week and only got access again yesterday, I'll adjust the text a bit |
I fear that perception is a result of focusing too closely on a single implementation. Here is a different scenario, which I have partially raised in early discussions: We have briefly mentioned that NMT (neural machine translation) approaches, akin to Google Translate, could be usable over MathML+Intent trees to produce speech strings (or even Braille). I had sent around some ChatGPT proof-of-concept examples which worked surprisingly well for a generalist model. In such a setting, the Core/Open lists would be used as coverage targets, for which to design training datasets. The lists are still important for an NMT approach, as using a natural+consistent vocabulary can ensure that generator tools emit intent expressions that are in-distribution for the ML model. In a mainstream modeling setup, we would no longer have any explicit mention of fixity, arity or speech hints - instead there would be just tuples of input pmml trees (annotated with intent expressions) and output language strings, in each desired natural language. The MathML Intent spec, as currently written, can be used by completely different classes of algorithms. If speech hints become normative, then only symbolic AT systems can be conformant, as the "hints" (and their arity) would become "rules". To me that seems like an unfortunate limitation to start adding at this late stage of spec work.
It is an implicit recognition of the arity - consider a That is the core of the debate - is arity an implementation detail, or a declarative spec requirement. |
Not at all; a single purpose
Well, neither, really. And it certainly is not a restriction on any other use you may want to make about a concept. |
|
Sorry for coming to the party late -- I thought I'd get to this on Sunday but people submitted several MathCAT bugs and I got bogged down dealing with one of them (it reminded me of lisp's "quote" and that was troublesome for me 50 years ago and apparently still is). Anyway... I think it is productive to think outside the box like Deyan did with the NMT mention. It brings up a different way of thinking about the information we provide. But it doesn't change what Assuming everyone agrees that systems should respect the author's intent, then if the author says they want a prefix reading by using the Similarly for an NMT, if something is in core, the NMT should be trained on what's in core. We all agree that the concept of To answer Deyan's question:
It is as fundamental to a specific instance in the core concept list as the name is. So in one sense, yes arity is part of a declaritive spec requirement. But in another sense, it is as much an implementation detail as the choice of name for a concept. To try and be a bit more complete: the name, number of arguments (could be nary), and any properties are the signature of a concept. The core list specifies those concepts that "everyone" should be able to count on/implement. But "everyone" is free to use/implement other concepts, whether they vary from core in name, number of arguments, or properties. |
I agree with that completely.
To me this is the matching signature of a speech rule (or hint), and should be specific to systems that employ such rules. |
It is neither. You are reading things in to this that are simply not there. arity as used here is not an abstract concept that we could choose to promote or not promote, it is a statement of fact about each concrete instance of an intent function call expression, it is the number of commas plus 1. If a document has If the core concept list has an entry for a root concept of arity 2 it means an implementation SHOULD have rules for root($a,3) to say cube root or whatever other templates are suggested by the dictionary. It doesn't mean root with three arguments is an error or that an implementation should not have rules to handle it, it's just that any such rules are not in the core list, and not expected in all implementations. I fail to see why any of this is even remotely controversial. |
That is indeed not what I have been discussing. For that framing, let's talk about the "number of arguments in an intent application". Then I can give you an example that it's not that simple as long as we have literals, which are often conveniently omitted (and I have to re-litigate each time): |
|
In which case the speech template for |
|
I don't want to over-interpret a thumbs up, but I think the only issue here is to find a clear way of saying that the arity/fixity is only about matching concepts to a speech string/algorithm. None of it was ever about restricting concepts themselves (or shouldn't have been). |
yes it really is that simple.
I assume you meant
that is arity 5 as it has 4 commas. That is perfectly fine but the core concept dictionary says nothing about it. The core concept dictionary will necessarily say nothing about lots of intent expressions. |
As Murray mentioned during the last group discussion that touched on this PR - the example can also be seen as a Core use of The key problem is that the question "arity of what?" has multiple independent answers that are "statement of fact": arity of a speech hint, arity of an intent application, arity of a mathematical function. As Bruce suggested in his last comment, at the least the PR needs some rewording. My main position was that arity should be a non-normative implementation detail for "matching speech hints". As I've illustrated, there are algorithms that don't need to use speech hints. Hence, this shouldn't be a question addressed by the main spec text, unless speech hints became normative for a separate reason (such as the one of #478 ). But that could also wait for a future version of Intent, it also adds complexity. |
|
On Mon, 27 Nov 2023 at 15:55, Deyan Ginev ***@***.***> wrote:
then I can give you an example that it's not that simple
yes it really is that simple.
intent="power:silent($1,_lifted,_to,_exponent_,$2)"
that is arity 5 as it has 4 commas. That is perfectly fine but the core
concept dictionary says nothing about it. The core concept dictionary will
necessarily say nothing about lots of intent expressions.
As Murray mentioned during the last group discussion that touched on this
PR - the example can also be seen as a Core use of power with arity 2. I
was happy someone with sufficient distance from the PR could recognize the
technical point I tried to raise.
I don't see how there can be any confusion here but we can add more words
anyway, the point of the dictionary is to look up terms in an intent
expression so it "arity of what" is clear, it's the arity of an intent
expression, and the value is 5.
Obviously one may infer that this expression is denoting the mathematical
arity 2 power operation by adding some extra arguments with literals, which
is all fine but explicitly a non-core use where you are forcing some speech.
The key problem is that the question "arity of *what?*" has multiple
independent answers that are "statement of fact": arity of a speech hint,
arity of an intent application, arity of a mathematical function. As Bruce
suggested in his last comment, at the least the PR needs some rewording.
As I say I can't see how in this context it isn't clear that it's the arity
of the intent function term that is meant, but we can say that explicitly.
------------------------------
My main position was that arity should be a non-normative implementation
detail for "matching speech hints". As I've illustrated, there are
algorithms that don't need to use speech hints. Hence, this shouldn't be a
question addressed by the main spec text, *unless* speech hints became
normative for a separate reason (such as the one of #478
<#478> ). But that could also wait
for a future version of Intent, it also adds complexity.
You need something to say what the entries in the core concept dictionary
means and when an item should be used.
The point of having a core list is that implementations _should_ support
those entries, That means if they read a `power(a,b)` in an intent
expression they _should_ use the core entry for power. It's "only" a
_should_ so they can still not do that and there is a lot of freedom to use
different words anyway so it's not really a testable constraint. But if
they read `power:silent($1,_lifted,_to,_exponent_,$2)` the core concept
dictionary doesn't apply at all (as it has 5 arguments) but the `:silent`
property means that the system _should_ read this as $1 lifted to exponent
$2.
I don't understand why you see this as controversial and not a natural
consequence of the design of `intent` as already agreed by the group.
Message ID: ***@***.***>
… |
As discussed in the call of 2023-10-19, extending the description of matching to a concept dictionary entry to make explicit the fact that arity and fixity need to match which previously was only explicitly spelled out in the core concept dictionary itself.
The core concept list has the text
This PR just adds the same in the existing paragraph describing case insensitive matching and normalising of
-and.