Audio Support for Frames

[alecsavvy]

Audio Support for Frames

Overview

At the @AudiusProject we recently implemented frame support when sharing links to music on the farcaster network. To expand on the frames protocol we'd like to propose a way to allow users to listen to music directly from a frame, ipod style.

Here you can find the full proposal that documents multiple flows as well as more details around how buttons could be used to support playback.
https://audiusproject.notion.site/Farcaster-Frame-Audio-Support-Proposal-c132268c37714cc096edc6302655e2c1

Example

Here's a quick example of what tags and a button could look like for supporting listening to a track. This would output a button at the index the developer defined that simply plays and pauses the track sourced from the fc:frame:audio tag.

<meta property="fc:frame" content="vNext" />
<meta property="fc:frame:audio" content="https://url/to/track.mp3" />
<meta property="fc:frame:button:1" content="▶⏸" />
<meta property="fc:frame:button:1:action" content="audio" />

Thank you for taking the time to consider our proposal! Would appreciate any feedback and look forward to enhancing the user experience of Farcaster and Frames.

[gigamesh]

Very cool. It seems like it'd be really valuable to be able to include metadata in as well? ex: artist & title, so clients can display it in a player.

Maybe a standardized json schema in fc:frame:state ?

[alecsavvy]

That could be an interesting way to pass the metadata to clients. Would the standardized json have to be part of the frame spec?

Since the frame specification is an extension of the open graph protocol could this data be provided to clients with additional og tags?

<!--same as before-->
<meta property="fc:frame" content="vNext" />
<meta property="fc:frame:audio" content="https://url/to/track.mp3" />
<meta property="fc:frame:button:1" content="▶⏸" />
<meta property="fc:frame:button:1:action" content="audio" />

<!--additional og tags clients could use should they be present-->
<meta property="og:title" content="Love Bites - Def Leppard" />
<!--same as fc:frame:audio-->
<meta property="og:image" content="https://url/to/track/art.jpg" />
<meta property="og:description" content="Description of track" />
<meta property="music:musician" content="https://audius.co/musician" />

[gigamesh]

I was thinking it would just be a JSON schema as part of your proposal, passed with the fc:frame:state tag described at the bottom of this table. That way, the schema can evolve independently of farcaster.

[alecsavvy]

I see, so something like this would be included as an optional structure for frame state in the frame spec?

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Music Metadata",
  "description": "Schema for music metadata",
  "type": "object",
  "properties": {
    "imageUrl": {
      "type": "string",
      "format": "uri",
      "description": "URL to artwork for the track, may be different than what's rendered in frame."
    },
    "artist": {
      "type": "string",
      "description": "Name of the artist who owns the track."
    },
    "description": {
      "type": "string",
      "description": "A brief description of the track."
    }
  },
  "required": ["imageUrl", "artist", "description"]
}

[jemgold]

pleeeeeease please please please please

[Mrmagnitude]

This is a great idea

[horsefacts]

Wanted to ack this, thank you for taking the time to write this up! I'd love to extend frames to other media types.

I like the simplicity of a single play/pause button, but can imagine clients wanting to do more. Perhaps this should be the minimum required behavior but clients could optionally enhance the controls.

We proxy image requests and probably need to consider the privacy implications of loading other content types.

[alecsavvy]

I like the simplicity of a single play/pause button, but can imagine clients wanting to do more. Perhaps this should be the minimum required behavior but clients could optionally enhance the controls.

Really like this approach. Wanted to keep it simple for the initial proposal.

[JWMatheo]

Would love to see it!

[varunsrin]

This should now be possible with the new mini-app experience in frames