Skip to content

Commit

Permalink
fix: upload section
Browse files Browse the repository at this point in the history
  • Loading branch information
Alan Shaw committed Dec 13, 2023
1 parent bbe9393 commit 2f9b604
Show file tree
Hide file tree
Showing 6 changed files with 95 additions and 75 deletions.
4 changes: 2 additions & 2 deletions src/pages/docs/concepts/car.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,13 @@ The --pack flag will create a new CAR file from a collection of input files:
ipfs-car pack path/to/files --output path/to/write/a.car
```

Extract files from a CAR with --unpack:
Extract files from a CAR with `--unpack`:

```sh
ipfs-car unpack path/to/my.car --output path/to/unpack/files/to
```

List the contents of a CAR with --list:
List the contents of a CAR with `--list`:

```sh
ipfs-car list path/to/my.car
Expand Down
46 changes: 28 additions & 18 deletions src/pages/docs/concepts/upload-vs-store.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,41 @@
# Upload vs. Store capabilities in web3.storage

There are two types of content identifiers (CIDs) that web3.storage interacts with:
There are three types of content identifiers (CIDs) that web3.storage interacts with:

1. **Content CIDs**: The CIDs used to reference and access uploads in the format generally useful to users (e.g., files, directories). These CIDs are generally prefixed by `bafy…`.
2. **Shard CIDs**: The CID of the serialized shards of data itself that are produced client-side, sent to web3.storage, and stored. These CIDs are generally prefixed by `bag…`.
3. **Piece CIDs**: The primary means of referencing data stored in a sector of a Filecoin Storage Provider. You can think of them as another way to reference a shard. These CIDs are generally prefixed by `bafk…`.

One example of where you might see both is uploading a large file with the CLI:
One example of where you might see all three is uploading a large file with the CLI:

```sh
w3 up gb.file
1 file (1GB)
bagbaierao...
bagbaieraq...
bagbaieraj...
bagbaierai...
bagbaierax...
bagbaieraf...
bagbaierac...
bagbaierax...
bagbaierax...
bagbaiera4...

w3 up gb.file --verbose
1 file 1.0GB
bagbaierakgivx2igjaydgkdholxohq77gsmjceb44wghn6l7rcxjnxlb2ysq 132.1MB
└── Piece CID: bafkzcibextmt6fvx6vkm56k3je76dirfpwzbqhs75agxen7yidv5lo5usnfdbetzcu
bagbaiera5hekddouwxfgue3wrbe5ddwcefr6l7z3majbqsjk3rxlruziw4nq 132.1MB
└── Piece CID: bafkzcibextmt6fujkiyd45vkvyd2gppji2wzcpsvu72y3sshzppmi2wnxp4q5pblay
bagbaierazfqaaabmnaksetr4ivqu7ytchnikygij7g26cycbjbbnfja6oxfa 132.1MB
└── Piece CID: bafkzcibextmt6ftpb5xepzfvg4ot5bv7uhuplrdbdrwnznpwnpaempif4i76gxbmdm
bagbaiera3elp7gxxbwvgyrgzrjjqsrhefofnr4t2zayktvkamqbjxfuundoa 132.1MB
└── Piece CID: bafkzcibextmt6frkjuqg6pxgqaq2zi5ktwsprflbikk2cnqshxifvsumntmqedgrba
bagbaieraamcctnhnqvwzuf7zhg4jyedg5c76fqn625wi7pico4xtnid5bgba 132.1MB
└── Piece CID: bafkzcibextmt6frloqsvau6qhwzfndrh55ahdjech52ddhspakf4gnxwksuks6uiem
bagbaiera6kel2dvxaiowgtj563kic2ftosnzqwue7etiph242w7w5s262fha 132.1MB
└── Piece CID: bafkzcibextmt6frcrmqxujkfxiy34yutheho55cwhlaqdypafjkhgusphcn5kyp3da
bagbaieranxcdfpqqqvmyugisyxmautsduivwrsm4jg2x2akys4ia3u4oyh7q 59.0MB
└── Piece CID: bafkzcibfzxlncayvf3ndiybf3hpigkfd4o27oqrfgmpr6y7ut5nhcyjdtjhqsvstwyqq
bagbaierakynqcaqedv2brd6qs772s3uqnrxwa62guoeqqjq2bi6kr76vfvla 132.1MB
└── Piece CID: bafkzcibextmt6fvobnshzubgtr4szqsztg35yjkoczy3bry74zfy2bu5fm2x5caefq
⁂ Stored 1 file
⁂ https://w3s.link/ipfs/bafybeiaxwvg4...
⁂ https://w3s.link/ipfs/bafybeidt227tuki2axtb3xcolwd5fw7relv6hfwxrlhgxjzyzwh4cowymy
```

The CLI sharded the 1GB upload into 10 shards, each with a `bag…`-prefixed CID. The content CID of the file itself is included in the `w3s.link/ipfs/bafy…` link at the bottom.
The CLI sharded the ~1GB upload into 8 shards, each with a `bag…`-prefixed CID. The content CID of the file itself is included in the `w3s.link/ipfs/bafy…` link at the bottom.

In the vast majority of cases, users should focus on content CIDs, as this is what they'll be using to fetch their content. If you stick with using the recommended client and CLI methods, then you won't really have to ever worry about the shard CIDs.

However, if you're interested in learning more about how web3.storage uses both, read on!
However, if you're interested in learning more about how web3.storage uses all three, read on!

## Upload vs. Store

Expand Down Expand Up @@ -68,3 +74,7 @@ web3.storage tracks usage for payment (i.e., how much storage is utilized by a u
Fortunately, this shouldn't make things any more complicated - we go into more detail below, but in general, when you remove a content CID from your account, you'll want to remove the shard CIDs as well (e.g., in the client calling `client.remove(contentCID, { shards: true })`).

However, if you are a power user interacting with shard CIDs as well (like in the previous section), then you need to be more cautious about removing shard CIDs from your account. (This is why the default for the client and CLI methods is for shards to be maintained after removing a content CID). Learn more about how to do this in the Remove section.

## When should I care about Piece CIDs?

Piece CIDs are how you can reference your data when it is stored in Filecoin Storage Providers. You can think of Piece CIDs as another way to reference a shard - they are in fact calculated from shard data. Piece CIDs are used in [PoDSI (Proof of Data Segment Inclusion)](/docs/concepts/podsi/) - a proof that a piece is included in a larger piece, which allows users and third parties to prove their data is stored with a Filecoin Storage Provider.
2 changes: 1 addition & 1 deletion src/pages/docs/how-to/create-account.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The easiest way to create an account is by using `w3cli`.
2. If you don't have an account, the website will ask you for your email address. Enter your email address and submit the form.
3. Check your email for a message from web3.storage including a link to confirm your intention to authenticate using the email you provided.

## Using the JS library
## Using the JS client

1. Install the client library from npm using your command line: `npm install @web3-storage/w3up-client`.
2. Call `client.login('[email protected]')`, and wait for the promise to resolve. This will send an email to your inbox with a link for validation.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { Callout } from 'nextra/components'

# How to create a space

In this how-to guide, you'll learn how to create a web3.storage Space to organize stored data. For an overview of the various ways web3.storage can be integrated with your application, check out [Architecture Options](/docs/concepts-architecture-options/).
Expand Down Expand Up @@ -26,9 +28,11 @@ Separately, you can visit [console.web3.storage](https://console.web3.storage/),
The Space you create can be used to [upload](/docs/how-to/upload/) data using the CLI, the w3up client, or when you log into the web console.
## Using the JS library
## Using the JS client
1. Install the client library from npm using your command line: `npm install @web3-storage/w3up-client`.
2. Call `client.createSpace('Documents')` and wait for the promise to resolve.
Note: the space must be provisioned by an account before it can be used for uploads. See [our guide](/docs/w3up-client/#create-and-register-a-space) for details.
<Callout>
The space must be provisioned by an account before it can be used for uploads. See [our guide](/docs/w3up-client/#create-and-provision-a-space) for details.
</Callout>
2 changes: 1 addition & 1 deletion src/pages/docs/how-to/retrieve.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -92,4 +92,4 @@ Replace `<YOUR_CID>`, `<FILE_NAME>`, and `<OUTPUT_FILE>` with their respective v

## Next steps

Next, you'll learn about how to [list](/docs/how-to/list/)
Next, you'll learn about how to [list](/docs/how-to/list/) uploaded content.
Loading

0 comments on commit 2f9b604

Please sign in to comment.