Enable pubsub by default

[alanshaw]

In js-ipfs we're integrating gossipsub and moving pubsub out of "experimental" status. There are some changes we want to make to support this:

1. Enable pubsub by default
   • This will add a small overhead to normal operation of the node but is worth it to allow people to pubsub without having to jump through hoops
2. Add a config option Pubsub.Enabled that will allow power users to disable pubsub if necessary (done: feat: enabling pubsub and ipns-pubsub via config flags #8510)
3. Deprecate --enable-pubsub-experiment flag for ipfs daemon
   • In js-ipfs we will just remove this, but I understand if you want to continue to support it for a period

[Stebalien]

@aschmahmann think you could tackle this?

[aschmahmann]

@Stebalien No problem. Any thoughts on deprecating vs removing the experimental flag? Should we deprecate in 0.5.0 and remove and 0.6.0?

[Stebalien]

I'd leave it and hide it for now (maybe issue a warning). We can clean it up eventually.

[Stebalien]

Also:

1. Use pubsub's new built-in discovery system instead of literally putting a block advertising our interest in the topic into the blockstore.
2. Manage pubsub topics. This may involve changing the API. At the moment, calling pubsub publish will "join" the topic permanently.
3. Pass pubsub.WithReadyness(pubsub.MinTopicSize(1)) on publish so we don't publish into the void. We can pass a context with a short (1-10s) timeout to avoid blocking forever.
4. Use the non-deprecated topic.Publish function so we can actually pass in a context.

[Stebalien]

Other blockers:

• Memory usage: Reduce memory usage for receive buffers libp2p/go-libp2p-pubsub#350
• Silent when not in use: On-demand pubsub libp2p/go-libp2p-pubsub#332

[lidel]

Note for later:

Many people run --enable-pubsub-experiment (or --enable-namesys-pubsub)
and we cannot break their infra when experiment becomes the default.

Right now (go-ipfs 0.8.0), when a flag is removed, go-ipfs returns hard error:

$ ipfs daemon --removed-fla
Error: unknown option "removed-flag"

[Stebalien]

We usually avoid removing flags and just deprecate & ignore them.

[FledgeXu]

Any progress about this?

[lidel]

Quick update:

• added config flags for enabling pubsub and ipns over pubsub in feat: enabling pubsub and ipns-pubsub via config flags #8510 (ETA: go-ipfs 0.11)
  • the flags are disabled by default
  • CLI ones still work
• JS-IPFS will be getting support for Fetch protocol soon (feat: add fetch protocol libp2p/js-libp2p#1036), which means we could be leveraging decentralized IPNS reproviding across the stack in near future.

I created #8591 for tracking IPNS-over-pubsub chunk separately.
Let's keep this one around pubsub alone.

Are there any other blockers?

[Jorropo]

This require too much out of goal work so we are gonna kill it in 0.20: #9717