docs: add-content

src/pages/_meta.json

@@ -1,6 +1,6 @@
 {
   "docs": {
-    "title": "🏠",
+    "title": "Docs",
     "type": "page"
   }
 }

src/pages/docs/_meta.json

@@ -1,6 +1,8 @@
 {
-  "index": "Start",
+  "index": "Welcome",
+  "quickstart": "Quickstart",
   "w3cli": "Command line",
   "w3up-client": "JS Client",
+  "how-to": "How to",
   "concepts": "Concepts"
 }

src/pages/docs/concepts/_meta.json

@@ -1,4 +1,15 @@
 {
-  "ucan": "UCAN",
-  "did": "DID"
+  "ucans-and-web3storage": "UCANs and web3.storage",
+  "content-addressing": "Content-addressing",
+  "upload-vs-store": "Upload vs. Store",
+  "architecture-options": "Architecture options",
+  "car": "Content Archive (CAR) files",
+  "filecoin-storage": "Filecoin",
+  "ipfs-gateways": "IPFS Gateways",
+  "did": {
+    "display": "hidden"
+  },
+  "ucan": {
+    "display": "hidden"
+  }
 }

src/pages/docs/concepts/architecture-options.md

@@ -0,0 +1,136 @@
+# Possible architectures using web3.storage to upload
+
+UCAN opens up a number of options in how to integrate with w3up: Should you, the developer, own the Space? Should you delegate permissions to your users? Or should your user own their own Space? Broadly, there are three ways to integrate:
+
+- Client-server: You (the developer) own the Space, and your user uploads to your backend infra before you upload it to the service
+- Delegated: You own the Space, but you give a delegated UCAN token to your user's Agent to upload directly to the service (rather than needing to touch the upload in your backend)
+- User-owned: Your user owns the Space and registers it and they use it to upload directly with the service; if you want to instrument visibility into what they're uploading, you'll have to write separate code in your app for it
+
+In the How-tos section of the docs, we focused on the first two options, as they are the most familiar today. However, you can implement each of these in a number of ways, but we talk through some considerations when implementing a given option.
+
+## Client-server
+
+```mermaid
+sequenceDiagram
+participant User
+w3up-client in backend-\>\>w3up-client in backend: Client set with Agent with delegation from Space
+User-\>\>w3up-client in backend: Upload data
+w3up-client in backend-\>\>web3.storage w3up service: Upload data
+```
+
+- For your backend to be scalable, you might consider using serverless workers or a queue in front of a server
+- In either case, you'll need a registered Space, and your client instance in your backend to have an Agent with a delegation from this Space
+  - (Recommended) It's likely easiest to create and register your Space using [w3cli](https://github.com/web3-storage/w3cli) rather than using w3up-client to do so (especially if your backend isn't persistent); you can then generate your own Agent and delegate the ability to upload to your Space using something like [this example](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#bringing-your-own-agent-and-delegation)
+  - If your backend is persistent, you can do this or do everything in the client directly ([create Space](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#creating-and-registering-spaces) and [get delegation](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#delegating-from-space-to-agent))
+- After this, once your user uploads data to your backend, you can run any of the upload methods
+
+## Delegated
+
+```mermaid
+sequenceDiagram
+participant w3up-client in user
+participant w3up-client in backend
+participant web3.storage w3up service
+w3up-client in backend-\>\>w3up-client in backend: Client created with Agent and delegation from Space
+w3up-client in user-\>\>w3up-client in user: Client instantiated with default Agent
+w3up-client in user-\>\>w3up-client in backend: Request delegation with user's Agent DID
+w3up-client in backend-\>\>w3up-client in user: Send delegation from Space to user's Agent DID
+w3up-client in user-\>\>web3.storage w3up service: Upload data
+```
+
+- You will likely have w3up-client running in your end-user's client code, as well as backend code that's able to generate UCANs that delegate the ability to upload and pass them to your users (e.g., w3up-client running in a serverless worker)
+- For your backend to be scalable, you might consider using serverless workers or a queue in front of a server
+- As the developer, you'll need a registered Space, and your client instance in your backend to have an Agent with a delegation from this Space
+  - (Recommended) It's likely easiest to create and register your Space using [w3cli](https://github.com/web3-storage/w3cli) rather than using w3up-client to do so (especially if your backend isn't persistent); you can then generate your own Agent and delegate the ability to upload to your Space using something like [this example](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#bringing-your-own-agent-and-delegation)
+  - If your backend is persistent, you can do this or do everything in the client directly ([create Space](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#creating-and-registering-spaces) and [get delegation](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#delegating-from-space-to-agent))
+- Your user does not need a registered Space - just an Agent with a delegation from your Space
+  - w3up-client in the end user environment should have a unique Agent for each user, which should happen by default (since when w3up-client is instantiated it creates a new Agent anyway, or uses the one in local Store)
+  - From there, when your end user is ready to upload, they should request from your backend a delegation from your developer-owned Space to their Agent (which can be derived via [client.agent()](https://github.com/web3-storage/w3up/blob/main/packages/w3up-client/docs-Client#agent))
+    - In your backend, you can call [client.createDelegation()](https://github.com/web3-storage/w3up/blob/main/packages/w3up-client/docs-Client#createDelegation) passing in the Agent object from client.agent() in your end user's instance, and passing through options? params to limit the scope of the delegation (e.g., store/add, upload/add, expiration time)
+    - You can serialize this using delegation.archive() and send it to your user
+    - The end user instance of the client should not need to call client.authorize(email), as it is not claiming any delegations via email address (but rather getting the delegation directly from your backend)
+- Once your user receives the delegation, they can deserialize it using [ucanto.Delegation.extract()](https://github.com/web3-storage/ucanto/blob/c8999a59852b61549d163532a83bac62290b629d/packages/core/src/delegation.js#L399) and pass it in using client.addSpace(), and from there they can run any of the upload methods
+  - Note that this alone does not give visibility into which of your end users are uploading what; to track this, you'll probably need them to send you that information separately (e.g., once they've run upload and get back a content CID, you can have them send that CID to you for tracking)
+- A code example that does this can be found below
+
+```js
+import { CarReader } from '@ipld/car'
+import * as DID from '@ipld/dag-ucan/did'
+import * as Delegation from '@ucanto/core/delegation'
+import { importDAG } from '@ucanto/core/delegation'
+import * as Signer from '@ucanto/principal/ed25519'
+import * as Client from '@web3-storage/w3up-client'
+
+async function backend(did: string) {
+  // Load client with specific private key
+  const principal = Signer.parse(process.env.KEY)
+  const client = await Client.create({ principal })
+
+  // Add proof that this agent has been delegated capabilities on the space
+  const proof = await parseProof(process.env.PROOF)
+  const space = await client.addSpace(proof)
+  await client.setCurrentSpace(space.did())
+
+  // Create a delegation for a specific DID
+  const audience = DID.parse(did)
+  const abilities = ['store/add', 'upload/add']
+  const expiration = Math.floor(Date.now() / 1000) + 60 \* 60 \* 24 // 24 hours from now
+  const delegation = await client.createDelegation(audience, abilities, { expiration })
+
+  // Serialize the delegation and send it to the client
+  const archive = await delegation.archive()
+
+  return archive.ok;
+}
+
+/** @param {string} data Base64 encoded CAR file */
+async function parseProof(data) {
+  const blocks = []
+  const reader = await CarReader.fromBytes(Buffer.from(data, 'base64'))
+  for await (const block of reader.blocks()) {
+    blocks.push(block)
+  }
+  return importDAG(blocks)
+}
+
+async function frontend() {
+  // Create a new client
+  const client = await Client.create()
+
+  // Fetch the delegation from the backend
+  const apiUrl = `/api/w3up-delegation/${client.agent().did()}`
+  const response = await fetch(apiUrl)
+  const data = await response.arrayBuffer();
+
+  // Deserialize the delegation
+  const delegation = await Delegation.extract(new Uint8Array(data));
+  if (!delegation.ok) {
+    throw new Error('Failed to extract delegation');
+  }
+
+  // Add proof that this agent has been delegated capabilities on the space
+  const space = await client.addSpace(delegation.ok);
+  client.setCurrentSpace(space.did())
+
+  // READY to go!
+}
+```
+
+## User-owned
+
+```mermaid
+sequenceDiagram
+participant User
+participant Application backend
+participant web3.storage w3up service
+Application backend-\>\>User: Front end code that includes w3up-client
+User-\>\>web3.storage w3up service: (If needed) Create Space and register it
+User-\>\>web3.storage w3up service: (If needed) Use Agent email verification to "log in" to Space
+User-\>\>web3.storage w3up service: Upload data using w3up-client
+```
+
+- If you want your user to own their own Space, you'll likely be relying on the w3up-client methods to create a Space, authorize the Space, and authorize the Agent on the end user-side; from there they can run any of the upload methods
+  - Doing this does take some of the UX out of your control; for instance, when web3.storage fully launches with w3up, your users will have to set up their payment methods with web3.storage
+  - Note that this alone does not give visibility into which of your end users are uploading what; to track this, you'll probably need them to send you that information separately (e.g., once they've run upload and get back a content CID, you can have them send that CID to you for tracking)
+- There is a world of possibilities with your users "bringing their own identity" for their Space; you could explore how crypto wallet private keys, Apple Passkey, and more might map to Space DIDs and have the client use those
+- If you have code snippet(s) that works for you, please share them in a PR or [Github issue](https://github.com/web3-storage/w3up/issues) and we'll link them here!

src/pages/docs/concepts/car.md

@@ -0,0 +1,134 @@
+# Working with Content Archive (CAR) files
+
+When you upload files to web3.storage using the [client library](https://web3.storage/docs/reference/js-client-library/), your data is converted into a graph of data structures, which are then packed into a format called a Content Archive (CAR) before being sent to the web3.storage service.
+
+For most use cases, you never need to know about this process, as the conversion happens behind the scenes when using the client library. However, you might want to know more about how to manipulate CAR files directly, especially if you're using `upload` and `store` separately in web3.storage.
+
+## What is a Content Archive?
+
+The [Content Archive format](https://ipld.io/specs/transport/car/) is a way of packaging up [content addressed data](https://web3.storage/docs/concepts/content-addressing/) into archive files that can be easily stored and transferred. You can think of them like [TAR files](https://en.wikipedia.org/wiki/Tar_(computing)) that are designed for storing collections of content addressed data.
+
+The type of data stored in CARs is defined by [IPLD](https://ipld.io/), or InterPlanetary Linked Data. IPLD is a specification and set of implementations for structured data types that can link to each other using a hash-based Content Identifier (CID). Data linked in this way forms a Directed Acyclic Graph, or DAG, and you'll likely see a few references to DAGs in the documentation for IPLD and IPFS.
+
+IPFS files are one example of IPLD data, but IPLD can also be used to access data from Ethereum, Git, and other hash-addressed systems. You can also use IPLD as a general purpose format for your structured data, sort of like a Web3-flavored JSON. See Advanced IPLD formats below for more information.
+
+## CARs and web3.storage
+
+When the web3.storage client and CLI pack up regular files into a CAR to store on IPFS, the CAR contains data encoded in the same format used by IPFS when importing files using the command line or other IPFS APIs.
+
+This format uses an IPLD "codec" called [dag-pb](https://ipld.io/docs/codecs/known/dag-pb/), which uses [Protocol Buffers](https://developers.google.com/protocol-buffers) to encode an object graph. Inside the graph are [UnixFS objects](https://docs.ipfs.io/concepts/file-systems/#unix-file-system-unixfs) that describe the files and their contents. We do this client-side for a few reasons.
+
+First, formatting everything on the client allows us to calculate the root Content Identifier for the data you're uploading before we send any data to the remote service. This means that you can compare the CID returned by the web3.storage service to the one you calculated locally, and you don't have to trust the service to do the right thing.
+
+Another reason to use CARs is to support large files, which would otherwise hit size limits on the web3.storage backend platform. The data in a CAR is already chunked into small blocks, which makes CARs easy to shard into small pieces that can be uploaded in batches.
+
+CAR files are a format that pretty much any IPFS tool or implementation can interact with. You can export data from your personal IPFS node into a CAR file and upload it to web3.storage using `w3 up --car` or `client.uploadCar`. As a result, we dive into the various ways you might interact with CAR files.
+
+## Command line tools
+
+There are a few ways to create and interact with CAR files from the command line.
+
+### ipfs-car
+
+The [ipfs-car](https://github.com/web3-storage/ipfs-car) JavaScript package includes a command-line tool for easily creating, unpacking, and verifying CAR files.
+
+You can install the command globally:
+
+```sh
+npm install -g ipfs-car
+```
+
+The --pack flag will create a new CAR file from a collection of input files:
+
+```sh
+ipfs-car --pack path/to/files --output path/to/write/a.car
+```
+
+Extract files from a CAR with --unpack:
+
+```sh
+ipfs-car --unpack path/to/my.car --output path/to/unpack/files/to
+```
+
+List the contents of a CAR with --list:
+
+```sh
+ipfs-car --list path/to/my.car
+```
+
+For more usage information, run ipfs-car --help.
+
+### Kubo
+
+Kubo is the reference implementation of the IPFS protocol. Among many other features, go-ipfs supports exporting any IPFS object graph into a CAR file and importing data from CAR files into your local IPFS repository.
+
+The [ipfs dag export](https://docs.ipfs.io/reference/cli/#ipfs-dag-export) command will fetch an IPFS object graph by its Content ID (CID), writing a stream of CAR data to standard output.
+
+To create a CAR file using go-ipfs, you can redirect the output of ipfs dag export to a file:
+
+```sh
+cid="bafybeigdmvh2wgmryq5ovlfu4bd3yiljokhzdep7abpe4c4lrf6rukkx4m"
+ipfs dag export $cid > path/to/output.car
+```
+
+Note that you should replace the value of cid inside the quotes with the CID you want to export.
+
+If you don't have the CID in your local IPFS repository, the dag export command will try to fetch it over the IPFS network.
+
+To add the contents of a CAR file to your local IPFS repository, you can use ipfs dag import:
+
+```sh
+ipfs dag import path/to/input.car
+```
+
+## Javascript libraries
+
+#### ipfs-car
+
+The ipfs-car package includes library functions for packing and unpacking files into CARs, using the IPFS UnixFs data model. The library includes the same functionality as the ipfs-car command line utility [described above](https://web3.storage/docs/how-tos/work-with-car-files/#ipfs-car).
+
+See the [ipfs-car README](https://github.com/web3-storage/ipfs-car#api) for API documentation and usage examples.
+
+#### @ipld/car
+
+The [@ipld/car](https://github.com/ipld/js-car)[package](https://github.com/ipld/js-car) contains the main JavaScript implementation of the CAR specification and is used by ipfs-car under the hood. If you want to store non-file data using [advanced IPLD formats](https://web3.storage/docs/how-tos/work-with-car-files/#advanced-ipld-formats), you should use @ipld/car directly.
+
+@ipld/car also provides the CarReader interface used by the web3.storage client's [putCar](https://web3.storage/docs/reference/js-client-library/#store-car-files)[method](https://web3.storage/docs/reference/js-client-library/#store-car-files).
+
+Here's a simple example of loading a CAR file from a Node.js stream and storing it with web3.storage:
+
+```js
+import { createReadStream } from 'fs'
+import { CarReader } from '@ipld/car'
+
+async function storeCarFile(filename) {
+  const inStream = createReadStream(filename)
+  const car = await CarReader.fromIterable(inStream)
+  const client = makeStorageClient()
+  const cid = await client.uploadCar(car)
+  console.log('Stored CAR file! CID:',cid)
+}
+```
+
+CarReader.fromIterable accepts any iterable of Uint8Array data, including Node.js streams. If you have all your CAR data in a single Uint8Array already, you can use [CarReader.fromBytes](https://github.com/ipld/js-car#CarReader__fromBytes) instead.
+
+The CarReader type shown above will read the entire contents of the CAR into memory, which may cause issues with large files. On Node.js, you can use [CarIndexedReader](https://github.com/ipld/js-car#carindexedreader), which reads CAR data from disk directly and uses less memory than CarReader.
+
+## [Advanced IPLD formats](https://web3.storage/docs/how-tos/work-with-car-files/#advanced-ipld-formats)
+
+IPLD can also be used as a general purpose data format like JSON. In fact, you can use JSON directly as IPLD just by using a special convention for linking to other IPLD objects. This convention is defined in the [dag-json](https://ipld.io/docs/codecs/known/dag-json/)["codec"](https://ipld.io/docs/codecs/known/dag-json/).
+
+Here's an example of a `dag-json` object:
+
+```json
+{
+  "name": "Have you seen this dog?",
+  "description": "I have now...",
+  "image":{"/":"bafybeihkqv2ukwgpgzkwsuz7whmvneztvxglkljbs3zosewgku2cfluvba"}
+}
+```
+
+The image field uses the special "link type" to reference another IPLD object. The link is just a regular JSON object with a single key named /, whose value is a Content Identifier.
+
+Although dag-json is familiar and easy to use, we recommend using the similar [dag-cbor](https://ipld.io/docs/codecs/known/dag-cbor/)[codec](https://ipld.io/docs/codecs/known/dag-cbor/) instead. dag-cbor uses the [Concise Binary Object Representation](https://cbor.io/) to more efficiently encode data, especially binary data which must be Base64-encoded when using dag-json.
+