Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC4211: WebXDC on Matrix #4211

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 156 additions & 0 deletions proposals/4211-webxdc.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation requirements:

  • Client

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I'd like to see feedback from a web client regarding how difficult this would be to implement, including the required sandboxing.

Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# MSC4211: WebXDC on Matrix
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isnt WebXDC just a widget with less functionality or am I missing something? Not sure if we need a second widget api 🤔

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've already mentionned this in the "Alternatives" section:

Widgets(MSC1236), while powerful, also suffer from a similar agnosticity problem with
other platforms, and has been noted as not having a "[...] canonical document to describe
[it]" last year. WebXDC also has stricter security concerns(e.g: clients cannot have
sensitive information leaked, unlike widgets), thus making it safer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the lack of a canonical document is a Matrix spec issue rather than a shortcoming of widgets themselves? IIUC, the main selling point of WebXDC appears to be that it doesn't require another server.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the lack of a canonical document is a Matrix spec issue rather than a shortcoming of widgets themselves?

Seeing it like that, yeah, will correct that back

IIUC, the main selling point of WebXDC appears to be that it doesn't require another server.

It also is. WebXDC is mostly self-contained (and here, the only real requirement is that you can actually get a MXC URI of the actual WebXDC container)

Copy link

@toger5 toger5 Oct 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its super cool to see someone looking into matrix+WebXDC. I have spend some time looking into it as well and liked the potential.

My appraoch during this investigation was to build a WebXDC container widget. It would load the WebXDC content and forward messages back and forth. It could also take care of putting everything into threads (this is the only way we can have proper back pagination with the widget api as of today) and abstract all this "bridging work" away from the client.

As mentioned before this would be sth each client would need to support.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. How does this interact with clients that do not, or can't include a web runtime?
  2. I'd been thinking about a similar MSC to be able to define how a given event type would be rendered. This would allow non-chat clients to interoperate with chat clients more clearly, by being able to specify that eg. a bug tracker can have it's events be rendered as "User opened new issue xyz".



In Matrix, users may want to send each other interactive content (for example a game
in a groupchat, or maybe a collaborative board). Previously, integrations were used
to provide such content, but they are not in-band (thus needs another server, which
tends to be unmaintained as of now). As such, this proposal aims to resolve this by
adding [WebXDC](https://webxdc.org/) into the Matrix client ecosystem, with no
server-side changes or additions needed.

## Proposal

A common method, with a well-defined API and that isn't locked to the Matrix ecosystem
(which may be useful for, e.g bridges, which would greatly benefit from such a common
system, and even client developers, who can simply rely on pre-existing tooling) and
that wouldn't require server administrators to either host another software solution
or users to resort to a centralised hub(as integrations) would be highly appreciable.

(Please note that the following simply describes how WebXDC calls transmate to
Matrix events. More information is available at WebXDC's own specification)

To start a WebXDC event, a client may start by sending a `m.webxdc.start` event,
formatted as such:
```jsonc
{
"content": {
"name": "Storm on Mt. Ooe", // optional, must be a string
tulir marked this conversation as resolved.
Show resolved Hide resolved
"url": "mxc://hell.club/WebXDCFileForClients", // MUST be a .xdc file
"icon": "mxc://hell.club/AnIconForTheWebXDC", // optional,
// content.icon_mime must exist
// iff present
"icon_mime": "image/png" // present IFF content.icon is present
},
"room_id": "!former:hell.club",
"event_id": "$WebXDCStart1",
"sender": "@yuugi:hell.club",
"type": "m.webxdc.start"
...
}
```
where the `content.url` field must be a MXC URI pointing to the WebXDC media.
Once a user starts, a client can show the event(alongside `content.name`, and
`content.icon`/`content.icon_mime`) as an invitation to enter the WebXDC context.

If so, it MUST set `window.webxdc.selfAddr` and `window.webxdc.selfName` to the
user's MXID and name in the room(or the global name stored in the profile API if
unavailable, otherwise fallback to the MXID), respectively.

Whenever a client calls the JavaScript `window.webxdc.sendUpdate(update, desc)`
function, it MUST send a `m.room.message` with a relation to the original event
with `rel_type=m.webxdc`, the `content.body` field being the description if existent,
and the *required* `m.webxdc.data` containing a JSON-encoded object as a representation
of `update`, which must be stored as a string if it contains floating-point numbers,
like so:
```js
// Client tries to send an update
window.webxdc.sendUpdate(
{
payload: {graze: 430, score: 5300},
info: window.webxdc.selfName + ' got over the Spellcard!',
summary: "Score: 5300"
},
"New Score on Mt. Ooe"
);
```

```jsonc
{
"content": {
"m.relates_to": { "event_id": "$WebXDCStart1", "rel_type": "m.webxdc" },
"body": "Marisa got over the Spellcard!",
"m.webxdc.data": {
"payload": {
"graze": 430,
"score": 5300
},
"info": "Marisa got over the Spellcard!",
"summary": "Score: 5300"
}
},
"sender": "@marisa:magic.forest",
"room_id": "!former:hell.club",
"event_id": "$WebXDCEvent",
"type": "m.room.message",
...
}
```

Clients receiving such an event(including themselves) must process it with
the listener set with `window.webxdc.setUpdateListener`, with the update's
`payload` being the `content->m.webxdc.data` object, it's `info`, `document`
and `summary` being the `content->m.webxdc.data->info`, `content->m.webxdc.data->document`,
and `content->m.webxdc.data->summary` fields respectively, the
`serial`/`max_serial` fieldd being the current and last event's `origin_server_ts`
fields respecticely. They must also check for such events (ideally via the
relations API).

The `window.webxdc.sendToChat(message)` function shall send a regular event to the
current room with the `content->body` field being set to the `message.text` value,
and being sent as the correct type of media if the `media.file` value exists.


## Potential issues

Non WebXDC-capable clients currently do not get a fallback for WebXDC start events,
which means that such messages will either not display, or will be hidden. Note that
this isn't a big issue, since clients that simply do not handle WebXDC generally have
no real reason to have the file in the first place.

Handling of floating-point values within events within the spec(which effectively do
not allow such things) may be an issue for WebXDC applications which require it.

Servers may be out of sync with the `origin_server_ts`, thus causing conditions where
two WebXDC updates `A` and `B`, such that `A` comes before `B`, may be processed in the
order `B` *then* `A`.

As it is experimental, this proposal does not provide a mapping between
`window.webxdc.joinRealtimeChannel` and a real, Matrix-side construct. The author also
doesn't see a good way to make it function alongside the existing event system properly,
and recommends waiting until it is stable within the WebXDC API before making an MSC
which supports it. As such, clients MUST set the function to the JavaScript `undefined`.


## Alternatives

Another way to have interactive content in rooms would be to use an integration manager.
However, these would require an additional server to be setup(which are not maintained),
and would effectively be limited to the Matrix ecosystem only, instead of being platform-
agnostic.
Widgets(MSC1236), while powerful, also suffer from a similar agnosticity problem with
other platforms.

## Security considerations

WebXDC still suffers from some issues, due to its nature, which is effectively a packaged web
application. As such, client developers will need to be careful to properly conceal secrets
(like a user's access token), and must follow [the WebXDC constraints](https://webxdc.org/docs/spec/messenger.html#webview-constraints-for-running-apps).
A malicious user could also use WebXDC as a vector for existing vulnerabilities in, e.g, a
browser/JavaScript engine, thus being able to break out of sandboxes. Clients must first
consult the user on whenever they want to start a WebXDC session, and must warn of potential
risks, especially with untrusted entities.


## Unstable prefix

Until this MSC is merged, client shall replace the following namespaces:
- `m.webxdc`->`at.kappach.at.webxdc`
- `m.webxdc.start`->`at.kappach.at.webxdc.start`
- `m.webxdc.data`->`at.kappach.at.webxdc.data`

As this proposal acts at the event layer, there is no need to ask the server for if it has
support for this proposal.

## Dependencies

This MSC is independent of any other unmerged MSCs.