diff --git a/src/pages/_meta.json b/src/pages/_meta.json index 7c46b9b..72ece55 100644 --- a/src/pages/_meta.json +++ b/src/pages/_meta.json @@ -1,6 +1,6 @@ { "docs": { - "title": "🏠", + "title": "Docs", "type": "page" } } \ No newline at end of file diff --git a/src/pages/docs/_meta.json b/src/pages/docs/_meta.json index 0d0d10c..5231dfa 100644 --- a/src/pages/docs/_meta.json +++ b/src/pages/docs/_meta.json @@ -1,6 +1,8 @@ { - "index": "Start", + "index": "Welcome", + "quickstart": "Quickstart", + "how-tos": "How to", + "concepts": "Concepts", "w3cli": "Command line", - "w3up-client": "JS Client", - "concepts": "Concepts" + "w3up-client": "JS Client" } \ No newline at end of file diff --git a/src/pages/docs/concepts/_meta.json b/src/pages/docs/concepts/_meta.json index a989ebb..655b93d 100644 --- a/src/pages/docs/concepts/_meta.json +++ b/src/pages/docs/concepts/_meta.json @@ -1,5 +1,15 @@ { - "ucan": "UCAN", - "did": "DID", - "car": "CAR" + "ucans-and-web3storage": "UCANs and web3.storage", + "content-addressing": "Content-addressing", + "upload-vs-store": "Upload vs. Store", + "architecture-options": "Architecture options", + "car": "Content Archive (CAR) files", + "filecoin-storage": "Filecoin", + "ipfs-gateways": "IPFS Gateways", + "did": { + "display": "hidden" + }, + "ucan": { + "display": "hidden" + } } \ No newline at end of file diff --git a/src/pages/docs/concepts/architecture-options.md b/src/pages/docs/concepts/architecture-options.md index 90710b2..79b3f56 100644 --- a/src/pages/docs/concepts/architecture-options.md +++ b/src/pages/docs/concepts/architecture-options.md @@ -1,12 +1,10 @@ # Architecture options -Possible architectures of using web3.storage to upload - UCAN opens up a number of options in how to integrate with w3up: Should you, the developer, own the Space? Should you delegate permissions to your users? Or should your user own their own Space? Broadly, there are three ways to integrate: - Client-server: You (the developer) own the Space, and your user uploads to your backend infra before you upload it to the service - Delegated: You own the Space, but you give a delegated UCAN token to your user's Agent to upload directly to the service (rather than needing to touch the upload in your backend) -- (Most complex) User-owned: Your user owns the Space and registers it and they use it to upload directly with the service; if you want to instrument visibility into what they're uploading, you'll have to write separate code in your app for it +- User-owned: Your user owns the Space and registers it and they use it to upload directly with the service; if you want to instrument visibility into what they're uploading, you'll have to write separate code in your app for it In the How-tos section of the docs, we focused on the first two options, as they are the most familiar today. However, you can implement each of these in a number of ways, but we talk through some considerations when implementing a given option. diff --git a/src/pages/docs/concepts/content-addressing.md b/src/pages/docs/concepts/content-addressing.md index e64dc83..050e86e 100644 --- a/src/pages/docs/concepts/content-addressing.md +++ b/src/pages/docs/concepts/content-addressing.md @@ -1,16 +1,14 @@ -## Content addressing - -Content addressing in brief +# Content addressing web3.storage's decentralized file storage relies on _content addressing_ to find, reference, and retrieve your files on the network. Content addressing is a technique for organizing and locating data in a system in which the key used to locate content is derived from the content itself, rather than its location. While you don't need to understand content addressing to be able to incorporate web3.storage in your apps and services, if you're curious about what's going on under the hood, read on. -### [The basic problem](https://web3.storage/docs/concepts/content-addressing/#the-basic-problem) +## The basic problem Consider what happens when you resolve a link like web3.storage/docs/concepts/content-addressing. First, your operating system queries a global shared key-value store, split into many domains — you may know this as the Domain Name System (DNS). The DNS returns an IP address that your network card can use to send HTTP requests over the network, where this site's naming conventions turn the key /concepts/content-addressing into a response payload. The problem is, components of an address like web3.storage/docs/concepts/content-addressing are _mutable_, meaning they can change over time. In the context of the web, where _everything_ is mutable and dynamic, this is just the way it's always been. As a result, [link rot](https://en.wikipedia-on-ipfs.org/wiki/Link_rot) is just something we've all learned to live with. -### [CIDs: Location-independent, globally unique keys](https://web3.storage/docs/concepts/content-addressing/#cids-location-independent-globally-unique-keys) +## CIDs: Location-independent, globally unique keys However, thanks to content addressing, link rot may become a thing of the past. A content-addressed system such as web3.storage is like our key-value-based DNS, with one significant difference: You no longer get to choose the keys. Instead, the keys are derived directly from the file contents using an algorithm that will always generate the same key for the same content. @@ -18,7 +16,7 @@ As a result, we no longer need to coordinate among multiple writers to our store This type of key is called a _content identifier (CID)_. Once you know the CID of a file on the web3.storage network, you have all you need for the network to locate and return the file back to you. -[web3.storage CIDs under the hood](https://web3.storage/docs/concepts/content-addressing/#web3storage-cids-under-the-hood) +## web3.storage CIDs under the hood web3.storage uses CIDs to make its decentralized file storage work, with help from [IPFS](https://ipfs.io/) for locating files and making sure they're always available. @@ -33,14 +31,14 @@ There are two types of CIDs that web3.storage interacts with: In the vast majority of cases, users should focus on content CIDs, as this is what they'll be using to fetch their content. If you stick with using the recommended client and CLI methods, then you won't really have to ever worry about the shard CIDs. -### [Summary](https://web3.storage/docs/concepts/content-addressing/#summary) +## Summary Using content addressing for locating files rather than the legacy web's method of location-dependent addressing responds to several critical weaknesses of the legacy web: - Content addressing solves for the problem behind link rot — the mutability of location-dependent storage systems — by using a hashing algorithm to generate a unique CID for each file that can be used as the lookup key for a file rather than a URL. - In addition to making sure files don't get lost if they're moved, content addressing also ensures that users intending to retrieve a specific version of a file will be guaranteed to retrieve that version for as long as it exists anywhere on the network. -### [Learn more](https://web3.storage/docs/concepts/content-addressing/#learn-more) +## Learn more Want a deep dive into content addressing, how it works, and why it's important? Check out [ProtoSchool](https://proto.school/content-addressing/) for an in-depth look at content addressing on the decentralized web, plus a wealth of other interactive tutorials on DWeb concepts, protocols, and tools. diff --git a/src/pages/docs/concepts/filecoin-storage.md b/src/pages/docs/concepts/filecoin-storage.md index e95a336..5ad1bf4 100644 --- a/src/pages/docs/concepts/filecoin-storage.md +++ b/src/pages/docs/concepts/filecoin-storage.md @@ -1,8 +1,8 @@ -## Filecoin storage +# Filecoin storage -### Filecoin storage +The Filecoin network is a great building block for any decentralized storage system. Independent storage providers periodically must cryptographically prove that they are physically storing your specific data for a specific duration of time. When they submit these proofs to the network, other nodes verify these proofs, and this is what ends up on the Filecoin blockchain. So, anyone at any given moment can trustlessly verify that specific content is persisted, the number of copies on the network, and with who they are stored with. -The Filecoin network is a great building block for any decentralized storage system. Independent storage providers periodically must cryptographically prove that they are physically storing your specific data for a specific duration of time. When they submit these proofs to the network, other nodes verify these proofs, and this is what ends up on the Filecoin blockchain. So, anyone at any given moment can trustlessly verify that specific content is persisted, the number of copies on the network, and with who they are stored with. Read more about Filecoin proofs [here](https://filecoin.io/blog/posts/what-sets-us-apart-filecoin-s-proof-system/). +Read more about Filecoin proofs [here](https://filecoin.io/blog/posts/what-sets-us-apart-filecoin-s-proof-system/). Because of the open nature of the Filecoin network allowing anyone to participate, it's very inexpensive to store data on the Filecoin network. As a result, web3.storage uses the network today to back up all its data on the decentralized web. In the future, we will launch new products that allow users to take advantage of Filecoin for "colder," disruptively expensive data storage. diff --git a/src/pages/docs/concepts/ipfs-gateways.md b/src/pages/docs/concepts/ipfs-gateways.md index 32151e5..b96c2e5 100644 --- a/src/pages/docs/concepts/ipfs-gateways.md +++ b/src/pages/docs/concepts/ipfs-gateways.md @@ -1,6 +1,4 @@ -## IPFS Gateways - -### IPFS HTTP Gateways +# IPFS Gateways web3.storage uses the [InterPlanetary File System (IPFS)](https://ipfs.io/) as a key part of its storage and retrieval infrastructure. @@ -12,7 +10,7 @@ As more browsers like [Brave](https://brave.com/ipfs-support/) and [Opera](https For more information about fetching content that you uploaded through an IPFS HTTP gateway, see the Retrieve section. -## [Types of gateway](https://web3.storage/docs/concepts/w3link/#types-of-gateway) +## Types of gateway The official [IPFS documentation on gateways](https://docs.ipfs.io/concepts/ipfs-gateway/) is helpful for understanding the types of gateways in the IPFS ecosystem and how they're used. @@ -20,7 +18,7 @@ One of the key things to understand for our purposes is the different [resolutio If you check the [list of public gateways](https://ipfs.github.io/public-gateway-checker/), you'll see that some support "subdomain" style URLs, while others only support path-style URLs. Below is a short refresher on the distinction between the two styles. -### [Path style URLs](https://web3.storage/docs/concepts/w3link/#path-style-urls) +### Path style URLs A "path style" URL puts the IPFS CID into the path portion of the gateway URL, like this: @@ -30,7 +28,7 @@ If the CID points to a directory listing, you can append the name of a file with [https://w3s.link/ipfs/bafybeid4gwmvbza257a7rx52bheeplwlaogshu4rgse3eaudfkfm7tx2my/hi-gateway.txt](https://w3s.link/ipfs/bafybeid4gwmvbza257a7rx52bheeplwlaogshu4rgse3eaudfkfm7tx2my/hi-gateway.txt) -### [Subdomain style URLs](https://web3.storage/docs/concepts/w3link/#subdomain-style-urls) +### Subdomain style URLs A "subdomain style" gateway URL puts the CID into the host portion of the URL, as a subdomain of the gateway host, like this: diff --git a/src/pages/docs/concepts/index.md b/src/pages/docs/concepts/ucans-and-web3storage.md similarity index 69% rename from src/pages/docs/concepts/index.md rename to src/pages/docs/concepts/ucans-and-web3storage.md index 4e21fe0..da0252f 100644 --- a/src/pages/docs/concepts/index.md +++ b/src/pages/docs/concepts/ucans-and-web3storage.md @@ -1,20 +1,16 @@ -## Concepts - -## UCANs and web3.storage - -How web3.storage uses UCAN +# UCANs and web3.storage For authorization, w3up services use [ucanto](https://github.com/web3-storage/ucanto), a Remote Procedure Call (RPC) framework built around [UCAN](https://ucan.xzy/), or User Controlled Authorization Networks. UCANs are a powerful capability-based authorization system that allows fine-grained sharing of permissions through a process called _delegation_ on top of [public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography). You can think about UCAN replacing bearer tokens in traditional APIs for authorization with w3up. Since any actor can be represented by a cryptographic keypair and permissions can be delegated to them, users can interact with w3up directly in cases where a developer might have needed to previously run additional back-end infrastructure to keep API keys secure. This can be extended even to have end users using applications integrated with w3up using their own keypair-based identity. -### How w3up and w3up-client use UCANs +## How w3up and w3up-client use UCANs Our client and CLI use ucanto to take care of the details of UCANs for you, but a few of the underlying terms and concepts may "bubble up" to the surface of the API, so we'll cover the basics. We'll also go over some terms that are specific to web3.storage that you might not have encountered elsewhere. UCAN-based APIs are centered around _capabilities_, which are comprised of an _ability_ and a _resource_. Together, the ability and resource determine what action a client can perform and what objects in the system can be acted upon. When invoking a service method, a client will present a UCAN token that includes an ability and resource, along with _proofs_ that verify that they should be allowed to exercise the capability. The proof might be signed directly by the capability owner, or have a chain of signatures (_delegations_) where the actor invoking the capability has been verifiably delegated permission to do so. -#### [Space](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#space) +### Space When you upload data to w3up, your uploads are linked to a unique _Space_ that acts as a "namespace" for the data you upload. Each Space corresponds to a _DID_, or [Decentralized Identity Document](https://www.w3.org/TR/did-core/). In web3.storage's implementation of w3up, these Space DIDs generally use the key DID method, of the form did:key:publicKey with a corresponding private signing key. @@ -22,117 +18,77 @@ When creating a Space, it generates this private key and did:key for you locally Under the hood in the email registration process, your Space delegates the capabilities needed to use w3up to your email address, and this delegation is stored by web3.storage. If you need access to your Space in the future from any device, web3.storage allows you to reclaim those capabilities the same way you would reset a password in other services - using an email verification process. This means you don't need to store or manage Space private keys to use w3up - just create a new space, register it with w3up and use it from as many devices as you like. More on this "sign in" process is detailed in the next section on Agents. -#### Agent +### Agent To invoke a capability like store/add on a Space using the client or CLI, the client must have an _Agent_. Like a Space, an Agent corresponds to a did:key whose private key is generated locally. An Agent is useful once the client or CLI has a UCAN delegation where a registered Space(s) delegates the Agent its capabilities. (An imperfect analogy is Agent to login session.) The delegation from a Space to your Agent that w3up-client needs can be passed either by verifying the email address the Space is registered to and claiming the UCAN delegation (authorize(email) then capability.access.claim) or directly if you have the UCAN delegation available locally (addSpace(delegation)). -#### Delegation to other actors +### Delegation to other actors Just like Spaces can delegate permissions to Agents you own, you can also delegate permissions to other actors' Agents. One common application of this could be you delegating permission to upload to your Space to your users. Here's a code snippet demonstrating this from the Upload section: -``` +```javascript import { CarReader } from '@ipld/car'; - -import \* as DID from '@ipld/dag-ucan/did'; - -import \* as Delegation from '@ucanto/core/delegation'; - +import * as DID from '@ipld/dag-ucan/did'; +import * as Delegation from '@ucanto/core/delegation'; import { importDAG } from '@ucanto/core/delegation'; - -import \* as Signer from '@ucanto/principal/ed25519'; - -import \* as Client from '@web3-storage/w3up-client'; +import * as Signer from '@ucanto/principal/ed25519'; +import * as Client from '@web3-storage/w3up-client'; async function backend(did: string) { - -// Load client with specific private key - -const principal = Signer.parse(process.env.KEY); - -const client = await Client.create({ principal }); - -// Add proof that this agent has been delegated capabilities on the space - -const proof = await parseProof(process.env.PROOF); - -const space = await client.addSpace(proof); - -await client.setCurrentSpace(space.did()); - -// Create a delegation for a specific DID - -const audience = DID.parse(did); - -const abilities = ['store/add', 'upload/add']; - -const expiration = Math.floor(Date.now() / 1000) + 60 \* 60 \* 24; // 24 hours from now - -const delegation = await client.createDelegation(audience, abilities, { - -expiration, - -}); - -// Serialize the delegation and send it to the client - -const archive = await delegation.archive(); - -return archive.ok; - + // Load client with specific private key + const principal = Signer.parse(process.env.KEY); + const client = await Client.create({ principal }); + + // Add proof that this agent has been delegated capabilities on the space + const proof = await parseProof(process.env.PROOF); + const space = await client.addSpace(proof); + await client.setCurrentSpace(space.did()); + + // Create a delegation for a specific DID + const audience = DID.parse(did); + const abilities = ['store/add', 'upload/add']; + const expiration = Math.floor(Date.now() / 1000) + 60 * 60 * 24; // 24 hours from now + const delegation = await client.createDelegation(audience, abilities, { + expiration, + }); + + // Serialize the delegation and send it to the client + const archive = await delegation.archive(); + return archive.ok; } -/\*\* @param {string} data Base64 encoded CAR file \*/ - +/** @param {string} data Base64 encoded CAR file */ async function parseProof(data) { - -const blocks = []; - -const reader = await CarReader.fromBytes(Buffer.from(data, 'base64')); - -for await (const block of reader.blocks()) { - -blocks.push(block); - -} - -return importDAG(blocks); - + const blocks = []; + const reader = await CarReader.fromBytes(Buffer.from(data, 'base64')); + for await (const block of reader.blocks()) { + blocks.push(block); + } + return importDAG(blocks); } async function frontend() { + // Create a new client + const client = await Client.create(); -// Create a new client - -const client = await Client.create(); - -// Fetch the delegation from the backend - -const apiUrl = `/api/w3up-delegation/${client.agent().did()}`; // backend method is exposed at this API URL - -const response = await fetch(apiUrl); - -const data = await response.arrayBuffer(); - -// Deserialize the delegation - -const delegation = await Delegation.extract(new Uint8Array(data)); - -if (!delegation.ok) { - -throw new Error('Failed to extract delegation'); - -} - -// Add proof that this agent has been delegated capabilities on the space - -const space = await client.addSpace(delegation.ok); + // Fetch the delegation from the backend + const apiUrl = `/api/w3up-delegation/${client.agent().did()}`; // backend method is exposed at this API URL + const response = await fetch(apiUrl); + const data = await response.arrayBuffer(); -client.setCurrentSpace(space.did()); + // Deserialize the delegation + const delegation = await Delegation.extract(new Uint8Array(data)); + if (!delegation.ok) { + throw new Error('Failed to extract delegation'); + } -// READY to go! + // Add proof that this agent has been delegated capabilities on the space + const space = await client.addSpace(delegation.ok); + client.setCurrentSpace(space.did()); + // READY to go! } ``` @@ -145,7 +101,7 @@ You can see the following flow: - When `frontend` function is called in the user's environment: - An Agent DID is created - The `backend` function hosted at an API endpoint is called, passing in the Agent DID - - The client is set up with a UCAN delegating upload capabilties to the Agent + - The client is set up with a UCAN delegating upload capabilities to the Agent - It's now ready to upload! -However, there's other interesting possibilities - for instance, you could create an app where your users make Spaces and delegate permission to your app to read their uploads. Read the Architecture options section to explore more. +However, there's other interesting possibilities - for instance, you could create an app where your users make Spaces and delegate permission to your app to read their uploads. Read the [Architecture options](/docs/concepts/architecture-options/) section to explore more. diff --git a/src/pages/docs/concepts/upload-vs-store.md b/src/pages/docs/concepts/upload-vs-store.md index 5608421..0609727 100644 --- a/src/pages/docs/concepts/upload-vs-store.md +++ b/src/pages/docs/concepts/upload-vs-store.md @@ -1,41 +1,27 @@ -## Upload vs. Store in web3.storage - -## Upload vs. Store capabilities in web3.storage +# Upload vs. Store in web3.storage There are two types of content identifiers (CIDs) that web3.storage interacts with: -- Content CIDs: The CIDs used to reference and access uploads in the format generally useful to users (e.g., files, directories). These CIDs are generally prefixed by `bafy…`. -- Shard CIDs: The CID of the serialized shards of data itself that are produced client-side, sent to web3.storage, and stored. These CIDs are generally prefixed by `bag…`. +1. Content CIDs: The CIDs used to reference and access uploads in the format generally useful to users (e.g., files, directories). These CIDs are generally prefixed by `bafy…`. +2. Shard CIDs: The CID of the serialized shards of data itself that are produced client-side, sent to web3.storage, and stored. These CIDs are generally prefixed by `bag…`. One example of where you might see both is uploading a large file with the CLI: -``` +```sh w3 up gb.file - -1 file (1GB) - + 1 file (1GB) bagbaierao... - bagbaieraq... - bagbaieraj... - bagbaierai... - bagbaierax... - bagbaieraf... - bagbaierac... - bagbaierax... - bagbaierax... - bagbaiera4... ⁂ Stored 1 file - ⁂ https://w3s.link/ipfs/bafybeiaxwvg4... ``` @@ -45,7 +31,7 @@ In the vast majority of cases, users should focus on content CIDs, as this is wh However, if you're interested in learning more about how web3.storage uses both, read on! -### Upload vs. Store +## Upload vs. Store There are two similar-sounding, complementary, but separate concepts in web3.storage: Upload and Store. The place you most readily see this is in the lower-level client methods like `Client.capabilities.upload.*` and `Client.capabilities.store.*`, and CLI methods like `w3 can upload *`, and `w3 can store *`. @@ -61,26 +47,24 @@ However, from web3.storage's perspective, it doesn't necessarily know whether th In cases where the user is uploading a whole file or directory like in the example above, it's safe to know that the series of CAR shards uploaded by the user correspond to the file's content CID that it cares about. That's why the higher-level `w3 up` method is appropriate to use, and should represent the vast majority of upload use cases by web3.storage users. -### When should I care about Store and shard CIDs? +## When should I care about Store and shard CIDs? There's a few cases to note when caring about shard CIDs themselves and the `store` series of capabilities. -**Sharing data across content CIDs** - -Because IPFS interacts with DAGs, you can actually share blocks between different content CIDs. One simple example would be in a directory: +### Sharing data across content CIDs -\ +Because IPFS interacts with DAGs, you can actually share blocks between different content CIDs. One simple example would be in a directory -If a user uploads this directory using `w3 up`, the content CID for this directory will be registered with their account. This is actually the root CID of the directory itself, but isn't the only content CID in reality that was made available on the network: each file within this directory actually has their own content CID (highlighted). +If a user uploads a directory using `w3 up`, the content CID for this directory will be registered with their account. This is actually the root CID of the directory itself, but isn't the only content CID in reality that was made available on the network: each file within this directory actually has their own content CID (highlighted). Let's say you want to be able to have two different directories registered as uploads in your account: one that has 3 of the files above, and another that has all of them. One way to do this would be to upload each file separately, then locally compute the content CIDs of each directory. You can then call `w3 can upload add` to register each of these directories. This results in only a single copy of each file being stored, but you being able to interact with two different directories! -**Removing data from account** +### Removing data from an account web3.storage tracks usage for payment (i.e., how much storage is utilized by a user) using the volume of data associated with shard CIDs. This should make sense after learning about the difference between `store` and `upload` - web3.storage is storing the CAR shards themselves, and `upload`s are more users tracking. -Fortunately, this shouldn't make things any more complicated - we go into more detail below, but in general, when you remove a content CID from your account, you'll want to remove the shard CIDs as well (e.g., in the client calling `Client.remove(contentCID, shards=True)`). +Fortunately, this shouldn't make things any more complicated - we go into more detail below, but in general, when you remove a content CID from your account, you'll want to remove the shard CIDs as well (e.g., in the client calling `Client.remove(contentCID, shards=true)`). However, if you are a power user interacting with shard CIDs as well (like in the previous section), then you need to be more cautious about removing shard CIDs from your account. (This is why the default for the client and CLI methods is for shards to be maintained after removing a content CID). Learn more about how to do this in the Remove section. diff --git a/src/pages/docs/how-tos/_meta.json b/src/pages/docs/how-tos/_meta.json new file mode 100644 index 0000000..d339ebe --- /dev/null +++ b/src/pages/docs/how-tos/_meta.json @@ -0,0 +1,9 @@ +{ + "create-account": "Create account and space", + "upload": "Upload", + "retrieve": "Retrieve", + "list": "List", + "remove": "Remove", + "plan": "Change price plan", + "receipts": "Query UCAN receipts" +} diff --git a/src/pages/docs/how-tos/index.md b/src/pages/docs/how-tos/create-account.md similarity index 91% rename from src/pages/docs/how-tos/index.md rename to src/pages/docs/how-tos/create-account.md index fe7357d..374bb19 100644 --- a/src/pages/docs/how-tos/index.md +++ b/src/pages/docs/how-tos/create-account.md @@ -1,14 +1,13 @@ -# How-tos -## Create Account and Space +# Create Account and Space -**How to create an account** +## How to create an account In the How-tos section of the docs, we show the most familiar, straightforward way to use web3.storage: by setting up an account for you, the developer, that you use in your application. For an overview of the various ways web3.storage can be integrated with your application, check out the "Architecture options" concepts section. You first need to create and register a Space you'll upload to. A Space acts as a namespace for your uploads. It is created locally and associated with a private key, and is then registered with web3.storage and associated with your email address. But don't worry about keeping track of the Space's private key! web3.storage's email authorization system allows this private key to be treated as a throwaway key. -**The easiest way to create and register a Space is by using the CLI.** +## The easiest way to create and register a Space is by using the CLI 1. Install the CLI from npm using your command line: `npm install -g @web3-storage/w3cli` 2. Run `w3 authorize [alice@example.com](mailto:alice@example.com)` in the command line using your email address. This will send an email to your inbox with a link for validation. @@ -16,7 +15,7 @@ You first need to create and register a Space you'll upload to. A Space acts as 4. Create a new Space for storing your data and register it: w3 space create -**w3 space register** +## `w3 space register` Separately, you can visit console.web3.storage, sign up with your email and select a plan, and create a space using the UI, but we recommend that developers get familiar with the CLI since it's a powerful tool for many things you might want to do. diff --git a/src/pages/docs/how-tos/upload.md b/src/pages/docs/how-tos/upload.md index 534aae9..69296fe 100644 --- a/src/pages/docs/how-tos/upload.md +++ b/src/pages/docs/how-tos/upload.md @@ -1,6 +1,6 @@ -## Upload +# Upload -**How to upload data using web3.storage** +## How to upload data using web3.storage In this how-to guide, you'll learn how to store data programmatically for your development projects using the web3.storage client library in JavaScript using your (developer-owned) Space. This includes various architecture options for the data pipeline for your users to upload to web3.storage, which then makes your data available on the decentralized IPFS network with persistent long-term storage provided by Filecoin. @@ -12,63 +12,67 @@ Later in this section, we also cover uploading data using the CLI or web console ⚠️❗ Permanent Data ♾️: Removing files from w3up will remove them from the file listing for your account, but that doesn't prevent nodes on the decentralized storage network from retaining copies of the data indefinitely. web3.storage itself generally retains and charges users for any uploaded data for a minimum of 30 days. Do not use w3up for data that may need to be permanently deleted in the future. -**web3.storage API authorization** +## web3.storage API authorization In the previous section, you created a Space that has a unique DID. To use the client to upload data to this Space, you need to give it permission to do so. This is done by passing an Agent into the client and a UCAN token delegating permissions to this Agent. An Agent has a DID with an underlying private key (like a Space), but can be thought of as parallel to a login session (whereas the Space can be thought of more like a bucket or account). To learn more about how UCAN works and how web3.storage uses it, read our docs on UCAN authorization. -**Using the Javascript client to upload** +## Using the Command line `w3cli` -This section discusses using the web3.storage JavaScript client, w3up-client, with your (developer-owned) Space in your application. web3.storage's Javascript client provides a simple interface for storing data using syntax inspired by familiar web APIs such as [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)and [File](https://developer.mozilla.org/en-US/docs/Web/API/File). +If you followed the Create account and Space section, you will already have the CLI set up with a Space. However, you might be using the CLI on a new machine, in which case you can follow these instructions: +1. (If not yet installed) Install the CLI from npm using your command line: `npm install -g @web3-storage/w3cli`. +2. Run `w3 authorize [alice@example.com](mailto:alice@example.com)` in the command line using your email address. Click on the validation link sent to your email. +3. After successfully running `authorize`, your CLI Agent has been delegated access to all Spaces associated with your email address. You can see a list of these Spaces using `w3 space ls` and select the one you'd like to upload to using `w3 space use \`. + +When the right Space is selected, you are ready to upload! You can do so by running `w3 up \`. + +There are a few useful flags (check out the reference docs to see a full list): + +``` +--no-wrap //Don't wrap input files with a directory. +-H, --hidden //Include paths that start with ".". +-c, --car //File is a CAR file. ``` -mermaid -flowchart TD +## Using the Javascript client -B[w3up-client instance] --\>|Automatic if specific Agent is not passed when client object created|B(Pass local Agent DID and key) +This section discusses using the web3.storage JavaScript client, w3up-client, with your (developer-owned) Space in your application. web3.storage's Javascript client provides a simple interface for storing data using syntax inspired by familiar web APIs such as [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)and [File](https://developer.mozilla.org/en-US/docs/Web/API/File). +```mermaid +flowchart TD +B[w3up-client instance] --\>|Automatic if specific Agent is not passed when client object created|B(Pass local Agent DID and key) S --\> C(Get UCAN delegation from Space to Agent) - C --\> D(Upload to Space using Agent) - ``` All uses of w3up-client to upload with web3.storage follow the flow above. -**Installing the client** +### Installing the client In your JavaScript project, add the web3.storage package to your dependencies: -``` -npminstall@web3-storage/w3up-client +```sh +npm install @web3-storage/w3up-client ``` -**Creating a server-side client instance** +### Creating a server-side client instance The package provides a [static](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#create)[create](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#create)[function](https://github.com/web3-storage/w3up/tree/main/packages/w3up-client#create) that returns a [Client](https://web3-storage.github.io/w3up-client/classes/client.Client.html)[object](https://web3-storage.github.io/w3up-client/classes/client.Client.html). How you initialize it depends on the backend environment. -**Claim delegation via email validation: For persistent backend only** - -``` -mermaid +### Claim delegation via email validation: For persistent backend only +```mermaid sequenceDiagram - Client-\>\>web3.storage service: Here is my email address and Agent DID - web3.storage service--\>\>Client: Please click the link to validate - Client--\>\>web3.storage service: Email address validated - web3.storage service-\>\>Client: Here is a UCAN delegating permission from Space DID to Agent DID - ``` You can use web3.storage's email authorization flow to give permissions to your server-side client. This can be good if your backend environment will be persistent (otherwise it would be prohibitive to click an email validation link every time the client is re-instantiated. -``` +```javascript import { create } from '@web3-storage/w3up-client' - const client = await create() ``` @@ -76,108 +80,80 @@ By default, clients will create a new [Agent](https://web3-storage.github.io/w3p Then you can authorize your Agent with your email address. Calling authorize will cause an email to be sent to the given address. -``` +```javascript await client.authorize('zaphod@beeblebrox.galaxy') ``` Once a user clicks the confirmation link in the email, the authorize method will resolve. Make sure to check for errors, as authorize will fail if the email is not confirmed within the expiration timeout. Authorization needs to happen only once per agent. This also claims all delegations available with your email address, so from there, you can select the Space you'd like to use. -``` +```javascript await client.setCurrentSpace(space.did()) # select the relevant Space DID that is associated with your account ``` -**Bring your own Agent: For any backend (including non-persistent and/or serverless)** +### Bring your own Agent -``` -mermaid +For any backend (including non-persistent and/or serverless) +```mermaid sequenceDiagram - Developer-\>\>Developer: Create Agent private key and DID - Developer-\>\>Developer: Delegate UCAN from Space to Agent - Developer-\>\>Client: Here is my Agent private key and UCAN delegating permissions - ``` An option that works for any backend environment is to define an Agent and delegate a UCAN from your Space to this Agent before initializing the client. This is especially useful if you're using the client in a serverless Node environment (e.g., Lambda). In your command line wherever the CLI is configured with the Space you want to use (e.g., where you created the Space): -``` +```shell # the following command returns what will be your Agent private key and DID -npx ucan-key ed --json - # store the private key in environment variable KEY +npx ucan-key ed --json # the following command returns the UCAN that will delegate - # make sure `w3 space use` is set to the Space you intend on using - -w3 delegation create \ | base64 +# if you want to limit permissions being passed to the Agent, you can specify which permissions to give, e.g., `--can 'store/add' --can 'upload/add'` limits to just being able to upload +w3 delegation create | base64 # store the output in environment variable PROOF - -# if you want to limit permissions being passed to the Agent, you can specify which permissions to give, e.g., `--can 'store/add' --can 'upload/add'` limits to just being able to upload ``` - Then, when you initialize and configure the client, you can pass in this Agent and UCAN. -``` +```javascript import { create } from '@web3-storage/w3up-client' - -import \* as Signer from '@ucanto/principal/ed25519' // Agents on Node should use Ed25519 keys +import * as Signer from '@ucanto/principal/ed25519' // Agents on Node should use Ed25519 keys async function main () { - -// Load client with specific private key - -const principal = Signer.parse(process.env.KEY) - -const client = await Client.create({ principal }) - -// Add proof that this agent has been delegated capabilities on the space - -const proof = await parseProof(process.env.PROOF) - -const space = await client.addSpace(proof) - -await client.setCurrentSpace(space.did()) - -// READY to go! - + // Load client with specific private key + const principal = Signer.parse(process.env.KEY) + const client = await Client.create({ principal }) + // Add proof that this agent has been delegated capabilities on the space + const proof = await parseProof(process.env.PROOF) + const space = await client.addSpace(proof) + await client.setCurrentSpace(space.did()) + // READY to go! } -/\*\* @param {string} data Base64 encoded CAR file \*/ - +/** @param {string} data Base64 encoded CAR file \*/ async function parseProof (data) { - -const blocks = [] - -const reader = await CarReader.fromBytes(Buffer.from(data, 'base64')) - -for await (const block of reader.blocks()) { - -blocks.push(block) - -} - -return importDAG(blocks) - + const blocks = [] + const reader = await CarReader.fromBytes(Buffer.from(data, 'base64')) + for await (const block of reader.blocks()) { + blocks.push(block) + } + return importDAG(blocks) } ``` If you're doing this in a non-persistent or serverless backend, you might consider using an in-memory [Store](https://github.com/web3-storage/w3up/tree/main/packages/access-client) for your Agent information rather than the default on-disk: -``` +```javascript import { StoreMemory } from '@web3-storage/access/stores/store-memory' - const client = await Client.create({ principal, store: new MemoryStore() }) ``` -**Uploading to web3.storage** +### Uploading to web3.storage Now that your backend client instance is set up with being able to interact with your Space, you're ready to upload! Call uploadFile to upload a single file, or uploadDirectory to upload multiple files. @@ -186,199 +162,111 @@ There are two main options to getting content into your Space: - Upload data to web3.storage from the backend client itself (e.g., you're storing data that your users are uploading to your backend) - Upload data to web3.storage directly from your user's environment (like your application's user's browser) by delegating a UCAN that has permission to upload to your Space -**Upload from backend client directly** - -``` -mermaid +### Upload from backend client directly +```mermaid sequenceDiagram - User-\>\>w3up-client in backend: Upload data - w3up-client in backend-\>\>web3.storage service: Upload data - ``` -You are already set up to upload using your client instance as data becomes available to your backend - you can call uploadFile or uploadDirectory with it. Here's a complete example of what this might look like: +You are already set up to upload using your client instance as data becomes available to your backend - you can call uploadFile or uploadDirectory with it. -``` +```javascript import { create } from '@web3-storage/w3up-client' +import { filesFromPaths } from 'files-from-path' -import \* as Signer from '@ucanto/principal/ed25519' // Agents on Node should use Ed25519 keys - -const principal = Signer.parse(process.env.KEY) // Agent private key - -const client = await create({ principal }) - -async function main () { - -// Load client with specific private key - -const principal = Signer.parse(process.env.KEY) - -const client = await Client.create({ principal }) - -// Add proof that this agent has been delegated capabilities on the space - -const proof = await parseProof(process.env.PROOF) - -const space = await client.addSpace(proof) - -await client.setCurrentSpace(space.did()) - -// READY to go! - -} - -/\*\* @param {string} data Base64 encoded CAR file \*/ - -async function parseProof (data) { - -const blocks = [] - -const reader = await CarReader.fromBytes(Buffer.from(data, 'base64')) - -for await (const block of reader.blocks()) { - -blocks.push(block) +// e.g "./best-gifs" +const path = process.env.PATH_TO_FILES +const files = await filesFromPaths([path]) -} - -return importDAG(blocks) - -} +const client = await create() +const directoryCid = await client.storeDirectory(files) ``` -**Delegate UCAN for your user to upload directly** +In the example above, `directoryCid` resolves to an IPFS directory. -``` -mermaid -sequenceDiagram +### Delegate UCAN for your user to upload directly +```mermaid +sequenceDiagram participant w3up-client in user - participant w3up-client in backend - participant web3.storage service - w3up-client in user-\>\>w3up-client in user: Client instantiated with default Agent - w3up-client in user-\>\>w3up-client in backend: Request delegation with user's Agent DID - w3up-client in backend-\>\>w3up-client in user: Send delegation from Space to user's Agent DID - w3up-client in user-\>\>web3.storage service: Upload data - ``` Your backend instance can also be used to delegate upload permissions directly to your user to upload. The code snippet below shows an example of how you might set up a client instance in your application frontend and how it might interact with your backend client. You can see how the frontend client Agent DID is used for the backend client to delegate permissions to; from there, it will be the frontend client that will call the `upload` method. -``` +```javascript import { CarReader } from '@ipld/car'; - -import \* as DID from '@ipld/dag-ucan/did'; - -import \* as Delegation from '@ucanto/core/delegation'; - +import * as DID from '@ipld/dag-ucan/did'; +import * as Delegation from '@ucanto/core/delegation'; import { importDAG } from '@ucanto/core/delegation'; - -import \* as Signer from '@ucanto/principal/ed25519'; - -import \* as Client from '@web3-storage/w3up-client'; +import * as Signer from '@ucanto/principal/ed25519'; +import * as Client from '@web3-storage/w3up-client'; async function backend(did: string) { - -// Load client with specific private key - -const principal = Signer.parse(process.env.KEY); - -const client = await Client.create({ principal }); - -// Add proof that this agent has been delegated capabilities on the space - -const proof = await parseProof(process.env.PROOF); - -const space = await client.addSpace(proof); - -await client.setCurrentSpace(space.did()); - -// Create a delegation for a specific DID - -const audience = DID.parse(did); - -const abilities = ['store/add', 'upload/add']; - -const expiration = Math.floor(Date.now() / 1000) + 60 \* 60 \* 24; // 24 hours from now - -const delegation = await client.createDelegation(audience, abilities, { - -expiration, - -}); - -// Serialize the delegation and send it to the client - -const archive = await delegation.archive(); - -return archive.ok; - + // Load client with specific private key + const principal = Signer.parse(process.env.KEY); + const client = await Client.create({ principal }); + + // Add proof that this agent has been delegated capabilities on the space + const proof = await parseProof(process.env.PROOF); + const space = await client.addSpace(proof); + await client.setCurrentSpace(space.did()); + + // Create a delegation for a specific DID + const audience = DID.parse(did); + const abilities = ['store/add', 'upload/add']; + const expiration = Math.floor(Date.now() / 1000) + 60 * 60 * 24; // 24 hours from now + const delegation = await client.createDelegation(audience, abilities, { + expiration, + }); + + // Serialize the delegation and send it to the client + const archive = await delegation.archive(); + return archive.ok; } -/\*\* @param {string} data Base64 encoded CAR file \*/ - +/** @param {string} data Base64 encoded CAR file */ async function parseProof(data) { - -const blocks = []; - -const reader = await CarReader.fromBytes(Buffer.from(data, 'base64')); - -for await (const block of reader.blocks()) { - -blocks.push(block); - -} - -return importDAG(blocks); - + const blocks = []; + const reader = await CarReader.fromBytes(Buffer.from(data, 'base64')); + for await (const block of reader.blocks()) { + blocks.push(block); + } + return importDAG(blocks); } async function frontend() { + // Create a new client + const client = await Client.create(); -// Create a new client - -const client = await Client.create(); - -// Fetch the delegation from the backend - -const apiUrl = `/api/w3up-delegation/${client.agent().did()}`; // backend method is exposed at this API URL - -const response = await fetch(apiUrl); - -const data = await response.arrayBuffer(); - -// Deserialize the delegation - -const delegation = await Delegation.extract(new Uint8Array(data)); + // Fetch the delegation from the backend + const apiUrl = `/api/w3up-delegation/${client.agent().did()}`; + const response = await fetch(apiUrl); + const data = await response.arrayBuffer(); -if (!delegation.ok) { + // Deserialize the delegation + const delegation = await Delegation.extract(new Uint8Array(data)); + if (!delegation.ok) { + throw new Error('Failed to extract delegation'); + } -throw new Error('Failed to extract delegation'); - -} - -// Add proof that this agent has been delegated capabilities on the space - -const space = await client.addSpace(delegation.ok); - -client.setCurrentSpace(space.did()); - -// READY to go! + // Add proof that this agent has been delegated capabilities on the space + const space = await client.addSpace(delegation.ok); + client.setCurrentSpace(space.did()); + // READY to go! } ``` -**[Preparing files and uploading](https://web3.storage/docs/how-tos/store/#preparing-files-for-upload)** +### Preparing files and uploading You are now ready to upload using the client! In general, the easiest way to upload data is using the uploadFile or uploadDirectory method. @@ -386,20 +274,15 @@ uploadFile expects a "Blob like" input, which can be a [Blob](https://developer. uploadDirectory requires File-like objects instead of Blobs, as the file's name property is used to build the directory hierarchy. -**Tip** - -When uploading multiple files, try to give each file a unique name. All the files in a storeDirectory request will be bundled into one content archive, and linking to the files inside is much easier if each file has a unique, human-readable name. +**Tip** When uploading multiple files, give each file a unique name. All the files in a `storeDirectory` request will be bundled into one content archive, and linking to the files inside is much easier if each file has a unique, human-readable name. You can control the directory layout and create nested directory structures by using / delimited paths in your filenames: -``` -const files = [ +```javascript +const files = [ new File(['some-file-content'], 'readme.md'), - new File(['import foo'], 'src/main.py'), - new File([someBinaryData], 'images/example.png'), - ] const directoryCid = await client.storeDirectory(files) @@ -409,195 +292,78 @@ In the example above, directoryCid resolves to an IPFS directory with the follow ``` . - ├──images - -│└──example.png - +| └──example.png ├──readme.md - └──src - └──main.py ``` -There are a few different ways of creating File objects available, depending on your platform. +There are a few different ways of creating `File` objects available, depending on your platform. In the browser, you can use a [file input element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/file) to allow the user to select files for upload: -``` -functiongetFiles(){ - -constfileInput=document.querySelector('input[type="file"]') - -returnfileInput.files - +```javascript +function getFiles () { + const fileInput = document.querySelector('input[type="file"]') + return fileInput.files } ``` You can also manually create File objects using the native File constructor provided by the browser runtime. This is useful when you want to store data created by your application, instead of files from the user's computer. -``` -functionmakeFileObjects(){ - -//YoucancreateFileobjectsfromaBlobofbinarydata - -//see:https://developer.mozilla.org/en-US/docs/Web/API/Blob - -//Herewe'rejuststoringaJSONobject,butyoucanstoreimages, - -//audio,orwhateveryouwant! - -constobj={hello:'world'} - -constblob=newBlob([JSON.stringify(obj)],{type:'application/json'}) - -constfiles=[ - -newFile(['contents-of-file-1'],'plain-utf8.txt'), - -newFile([blob],'hello.json') - -] - -returnfiles - +```javascript +function makeFileObjects () { + // You can create File objects from a Blob of binary data + // see: https://developer.mozilla.org/en-US/docs/Web/API/Blob + // Here we're just storing a JSON object, but you can store images, + // audio, or whatever you want! + const obj = { hello: 'world' } + const blob = new Blob([JSON.stringify(obj)], { type: 'application/json' }) + + const files = [ + new File(['contents-of-file-1'], 'plain-utf8.txt'), + new File([blob], 'hello.json') + ] + return files } ``` -In Node.js, the web3.storage package exports some helpful utility functions from the [files-from-path](https://www.npmjs.com/package/files-from-path)[module](https://www.npmjs.com/package/files-from-path) that allow you to easily read File objects from the local file system. The getFilesFromPath helper asynchronously returns an array of Files that you can use directly with the put client method: +In Node.js, the [files-from-path](https://www.npmjs.com/package/files-from-path)[module](https://www.npmjs.com/package/files-from-path) module reads File objects from the local file system. The `getFilesFromPaths` helper asynchronously returns an array of Files that you can use directly with the put client method: -``` -import{getFilesFromPath}from'web3.storage' +```javascript +import { getFilesFromPaths } from 'files-from-path' ``` If you expect to be loading a lot of large files, you may be better served by the [filesFromPath](https://github.com/web3-storage/files-from-path#filesfrompath)[helper](https://github.com/web3-storage/files-from-path#filesfrompath). It reduces memory pressure by yielding File objects one by one as they're loaded from disk, instead of loading everything into memory. You can then issue multiple put requests to send each file to web3.storage. You can also manually create File objects by importing a Node.js implementation of File from the web3.storage package. This is useful when you want to store data created by your application, instead of files from the user's computer. -``` -import{File}from'web3.storage' - -asyncfunctiongetFiles(path){ - -constfiles=awaitgetFilesFromPath(path) - -console.log(`read${files.length}file(s)from${path}`) - -returnfiles - +```javascript +async function getFiles (path) { + const files = await getFilesFromPath(path) + console.log(`read ${files.length} file(s) from ${path}`) + return files } -functionmakeFileObjects(){ - -//YoucancreateFileobjectsfromaBufferofbinarydata - -//see:https://nodejs.org/api/buffer.html - -//Herewe'rejuststoringaJSONobject,butyoucanstoreimages, - -//audio,orwhateveryouwant! - -constobj={hello:'world'} - -constbuffer=Buffer.from(JSON.stringify(obj)) - -constfiles=[ - -newFile(['contents-of-file-1'],'plain-utf8.txt'), - -newFile([buffer],'hello.json') - -] - -returnfiles - +function makeFileObjects () { + // You can create File objects from a Buffer of binary data + // see: https://nodejs.org/api/buffer.html + // Here we're just storing a JSON object, but you can store images, + // audio, or whatever you want! + const obj = { hello: 'world' } + const buffer = Buffer.from(JSON.stringify(obj)) + const files = [ + new File(['contents-of-file-1'], 'plain-utf8.txt'), + new File([buffer], 'hello.json') + ] + return files } ``` -**Using the CLI or web console to upload** - -You can also use the CLI or the web3.storage web console to upload. - -**CLI** - -If you followed the Create account and Space section, you will already have the CLI set up with a Space. However, you might be using the CLI on a new machine, in which case you can follow these instructions: - -1. (If not yet installed) Install the CLI from npm using your command line: `npm install -g @web3-storage/w3cli`. -2. Run `w3 authorize [alice@example.com](mailto:alice@example.com)` in the command line using your email address. Click on the validation link sent to your email. -3. After successfully running `authorize`, your CLI Agent has been delegated access to all Spaces associated with your email address. You can see a list of these Spaces using `w3 space ls` and select the one you'd like to upload to using `w3 space use \`. - -When the right Space is selected, you are ready to upload! You can do so by running `w3 up \`. - -There are a few useful flags (check out the reference docs to see a full list): - -``` ---no-wrap //Don't wrap input files with a directory. --H, --hidden //Include paths that start with ".". --c, --car //File is a CAR file. -``` - -**Web console** - -You can also upload data using the web console. First, go to console.web3.storage. Enter your email if you're not yet logged in (and click on the link sent to your inbox). - - - -Once logged in, select the Space you'd like to upload to. - - - -From there, just click the Upload box and select your file or directory. - - - -**Content vs. shard CIDs** - -There are two types of content identifiers (CIDs) that web3.storage interacts with: - -- Content CIDs: The CIDs used to reference and access uploads in the format generally useful to users (e.g., files, directories). These CIDs are generally prefixed by `bafy…`. -- Shard CIDs: The CID of the serialized shards of data itself (CAR files) that are produced client-side, sent to web3.storage, and stored. These CIDs are generally prefixed by `bag…`. - -One example of where you might see both is uploading a large file with the CLI: - - -``` -w3 up gb.file - -1 file (1GB) - -bagbaierao... - -bagbaieraq... - -bagbaieraj... - -bagbaierai... - -bagbaierax... - -bagbaieraf... - -bagbaierac... - -bagbaierax... - -bagbaierax... - -bagbaiera4... - -⁂ Stored 1 file - -⁂ https://w3s.link/ipfs/bafybeiaxwvg4... -``` - -The CLI sharded the 1GB upload into 10 shards, each with a `bag…`-prefixed CID. The content CID of the file itself is included in the `w3s.link/ipfs/bafy…` link at the bottom. - -In the vast majority of cases, users should focus on content CIDs, as this is what they'll be using to fetch their content. If you stick with using the recommended client and CLI methods, then you won't really have to ever worry about the shard CIDs. -However, there are some cases where interacting with shard CIDs can be useful. This requires using the lower-level methods that the client and CLI expose (`capability.*` methods in the client, and `w3 can *` methods in the CLI). You can read more about the differences in the Upload vs. Store section. -**Next steps** +## Next steps Learn more about how to fetch your data using the CID in the next section, Retrieve. diff --git a/src/pages/docs/quickstart.md b/src/pages/docs/quickstart.md new file mode 100644 index 0000000..74f1f2e --- /dev/null +++ b/src/pages/docs/quickstart.md @@ -0,0 +1,95 @@ +# Quickstart + +Ready to get started using web3.storage? Get up and running in minutes by following this quickstart guide. In this guide, we'll walk through the following steps: + +1. Install the CLI. +2. Create a Space to upload your files and register it with web3.storage. +3. Upload a file. +4. Get your uploaded file using your browser or curl. + +This guide uses our CLI, w3cli, since it's the fastest way to get started using web3.storage programmatically. In the "How-tos" section of the docs, we also include instructions on using the Javascript client or web interface to create an account, store data, and more. + +## You will need + +Node.js version `18` or higher and npm version `7` or higher to complete this guide. Check your local versions like this: + +```shell +node --version && npm --version + +> v18.17.1 +> 7.18.1 +``` + +Install the CLI from npm using your command line: + +```sh +npm install -g @web3-storage/w3cli +``` + +## Create an account + +You need to create a web3.storage account associated with an email address and set it up so you can start uploading to a Space. The Space is created locally and associated with a private key, and is then registered with web3.storage and associated with your email address. But don't worry about keeping track of the Space's private key! web3.storage's email authorization system allows this private key to be treated as a throwaway key. + +1. Run `w3 authorize [alice@example.com](mailto:alice@example.com)` in the command line using your email address. This will sent an email to your inbox with a link for validation. +2. Once you click on the validation link, you'll be taken to a webpage where you can enter your payment information and select a plan (like our Free tier). +3. Create a new Space for storing your data and register it: + +```sh +w3 space create Documents # pick a good name! + +w3 space register +``` + +## Upload + +You can now upload a file or directory using the command line: + +```sh +$ w3 up lets-go.txt + 1 file 0.6KB +⁂ Stored 1 file +⁂ https://w3s.link/ipfs/bafybeib4ht2a53pttgipw6mgckqqhmgkifnmh2glzsju2c6ait5ibnkow4 +``` + +The CLI content-addresses your files, packs them into 1 or more CAR files, and uploads them to web3.storage for indexing and inclusion in Filecoin storage deals. It will show an http gateway URL that includes the content CID (content identifier) of your upload e.g: + +https://w3s.link/ipfs/bafybeib4ht2a53pttgipw6mgckqqhmgkifnmh2glzsju2c6ait5ibnkow4 + +## Get your file + +Your upload is now available over the public IPFS network using the content CID of your upload. The easiest way to fetch it is using the w3s.link gateway using the link that `w3 up` provided. w3s.link is optimized for content uploaded to web3.storage. + +```sh +curl -L 'https://w3s.link/ipfs/bafybeib4ht2a53pttgipw6mgckqqhmgkifnmh2glzsju2c6ait5ibnkow4/lets-go.txt' + + ___. ________ __ +__ _ __ ____ \_ |__ \_____ \ _______/ |_ ____ _______ _____ ____ ____ +\ \/ \/ /_/ __ \ | __ \ _(__ < / ___/\ __\ / _ \ \_ __ \\__ \ / ___\ _/ __ \ + \ / \ ___/ | \_\ \ / \ \___ \ | | ( <_> ) | | \/ / __ \_ / /_/ >\ ___/ + \/\_/ \___ > |___ //______ / /\/____ > |__| \____/ |__| (____ / \___ / \___ > + \/ \/ \/ \/ \/ \/ /_____/ \/ +``` + + +You can also fetch your content p2p style over bitswap with an IPFS implementation like `helia` or `kubo`. + +```sh +ipfs cat bafybeib4ht2a53pttgipw6mgckqqhmgkifnmh2glzsju2c6ait5ibnkow4/lets-go.txt + + ___. ________ __ +__ _ __ ____ \_ |__ \_____ \ _______/ |_ ____ _______ _____ ____ ____ +\ \/ \/ /_/ __ \ | __ \ _(__ < / ___/\ __\ / _ \ \_ __ \\__ \ / ___\ _/ __ \ + \ / \ ___/ | \_\ \ / \ \___ \ | | ( <_> ) | | \/ / __ \_ / /_/ >\ ___/ + \/\_/ \___ > |___ //______ / /\/____ > |__| \____/ |__| (____ / \___ / \___ > + \/ \/ \/ \/ \/ \/ /_____/ \/ +``` + +## Next steps + +Congratulations! You've just covered the basics of web3.storage. To learn more, take a look at these useful resources: + +- For a deep dive into storing files, including using the Javascript client to do so, visit the Store how-to guide. +- Read more about the power of UCANs and IPFS, and learn about the various options to integrate web3.storage with your application. +- Try out our image gallery example to see how easy it is to take advantage of these decentralized protocols using web3.storage. +- Visit the reference API section for more details on what else you can do with the web3.storage client and how to integrate it into your own projects. + diff --git a/src/pages/docs/w3up-client.md b/src/pages/docs/w3up-client.md index 4551d81..5a1f9bd 100644 --- a/src/pages/docs/w3up-client.md +++ b/src/pages/docs/w3up-client.md @@ -36,9 +36,9 @@ See the [client reference docs][reference-w3up-client#create] for more creation ## Create and register a space -When you upload things to web3.storage, each upload is associated with a "space," which is a unique identifier that acts as a namespace for your content. +When you upload things to web3.storage, each upload is associated with a "space," which is a unique identifier that acts as a namespace for your content. -Spaces are identified by DID using keys created locally on your devices. To use a space for uploads, it needs to be registered with the storage service by providing an email address. +Spaces are identified by DID using keys created locally on your devices. To use a space for uploads, it needs to be registered with the storage service by providing an email address. To create a space using `w3up-client`, use the [`createSpace` client method][reference-w3up-client#createSpace]: @@ -106,7 +106,7 @@ In the example above, `directoryCid` resolves to an IPFS directory with the foll The `uploadFile` and `uploadDirectory` methods described in the previous step both return a CID, or Content Identifier, encoded as a string. -To create a link to view your file on an IPFS gateway, create a URL of the form `https://${cid}.ipfs.${gatewayHost}`, where `${cid}` is the CID of the content you want to view, and `${gatewayHost}` is the domain of the gateway. To use our own gateway at `w3s.link`, your url would be `https://${cid}.ipfs.w3s.link`. +To create a link to view your file on an IPFS gateway, create a URL of the form `https://${cid}.ipfs.${gatewayHost}`, where `${cid}` is the CID of the content you want to view, and `${gatewayHost}` is the domain of the gateway. To use our own gateway at `w3s.link`, your url would be `https://${cid}.ipfs.w3s.link`. Opening the gateway URL in a browser will take you to your uploaded file, or a directory listing of files, depending on what you uploaded.