Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Blueprints: Directory Resources (#1793)
## Description
Adds a Directory resource type to enable loading file trees from git
repositories, local hard drive, deeply nested zip archives etc:
```json
{
"steps": [
{
"step": "installPlugin",
"pluginData": {
"resource": "git:directory",
"repositoryUrl": "https://github.com/WordPress/wordpress-playground.git",
"ref": "HEAD",
"path": "packages/docs"
}
},
{
"step": "installPlugin",
"pluginData": {
"resource": "literal:directory",
"name": "hello-world",
"files": {
"README.md": "Hello, World!",
"index.php": "<?php\n/**\n* Plugin Name: Hello World\n* Description: A simple plugin that says hello world.\n*/",
}
}
}
]
}
```
## Motivation
This PR opens the door to:
* Seamless Git integration. Import path mapping from git to Playground
is now just a few steps referencing specific git directories. No more
custom logic required!
* Blueprint-based site imports and exports without any Playground
webapp-specific logic.
* Runtime-specific "resource overrides", e.g.
`--resource-override=GUTENBERG:./gutenberg.zip` in CLI to test a
Blueprint with a my local version of Gutenberg. The same logic would be
used by the Blueprints builder to use files selected via `<input
type="file">` controls.
### Schema
Every step can declare which kinds of resources it accepts (file-based
resources vs directory-based resources). Using a single `pluginData`
property in the `installPlugin` step means less choices for the
developer. It also makes local resource overrides easy, e.g. we could
tell Playground CLI to load a local Gutenberg directory instead of a
remote Gutenberg zip. This wouldn't be as easy had we used separate
options for passing ZIP-based and directory-based resources.
On one hand, `pluginData` is less informative than `pluginZipFile`. On
the other, the name accommodates for non-zip resources such as
directories.
## Developer notes about specific API changes introduced in this PR
This PR changes introduces a new `literal:directory` resource that can
be used in Blueprints as follows:
```json
{
"steps": [
{
"step": "installPlugin",
"pluginData": {
"resource": "literal:directory",
"name": "hello-world",
"files": {
"README.md": "Hello, World!",
"index.php": "<?php\n/**\n* Plugin Name: Hello World\n* Description: A simple plugin that says hello world.\n*/",
}
}
}
]
}
```
Or via the JS API:
```ts
await installTheme(php, {
themeData: {
name: 'test-theme',
files: {
'index.php': `/**\n * Theme Name: Test Theme`,
},
},
ifAlreadyInstalled: 'overwrite',
options: {
activate: false,
},
});
```
It also introduces a new `writeFiles` step:
```ts
{
"steps": [
{
"step": "writeFiles",
"writeToPath": "/wordpress/wp-content/plugins/my-plugin",
"filesTree": {
"name": "my-plugin",
"files": {
"index.php": "<?php echo '<a>Hello World!</a>'; ?>",
"public": {
"style.css": "a { color: red; }"
}
}
}
}
]
}
```
Specific changes:
* Adds a `Resource<Directory>` resource type that provides a `name:
string` and `files: FileTree`.
* Renames `pluginZipFile` to `pluginData` in the `installPlugin` step
* Renames `themeZipFile` to `themeData` in the `installPlugin` step
* Adds a new `writeFiles` step for writing entire directory trees
* Adds a new `literal:directory` resource type where an entire file tree
can be specified inline
* Adds a new `git:directory` resource type that throws an error for now,
but will load arbitrary directories from git repositories once
#1764 lands
## Remaining work
- [x] Discuss the scope and the ideas
- [x] Add unit tests
- [x] Update the documentation
- [x] Adjust the `installPlugin` and `installTheme` step for
compatibility with it's former signature. Ensure the existing packages
consuming those functions from the `@wp-playground/blueprints` package
will continue to work.
- [x] Confirm we can safely omit streaming from the system design at
this point without setting ourselves up for a grand refactor a few
months down the road.
* I think we can! Streaming support could be an addition to the system,
not a change in how the system works. For example, there could be a new
`DirectoryStream` resource type producing an `AsyncDirectoryIterator`
with streamable `File` or `Blob` objects as its leafs. It would work
nicely with remote APIs or the ZIP streaming plumbing in
`@php-wasm/stream-compression`. Any existing code expecting a
`DirectoryResource` should be relatively easily adaptable to use these
async iterators instead.
## Follow-up work
- [ ] Include actual git support once the [Git sparse checkout
PR](#1764) lands
- [ ] Ship a Playground CORS proxy to enable using git checkout in the
webapp
- [ ] Once we have a use-able `git:directory` resource, expand the
developer notes from this PR and other related PRs and write a post on
https://make.wp.org/playground
## Tangent – Streaming and a shorthand URL notation
Without streaming, the entire directory must be loaded into memory. Our
git sparse checkout implementation buffers everything anyway, but we
will want to stream-read directory resources in the future. For example:
```js
{
"steps": [
// Stream plugin files directly from a doubly zipped Git artifact
{
"step": "installPlugin",
"pluginData": {
"resource": "zip:github-artifact",
"zipFile": {
"resource": "url",
"url": "https://github.com/WordPress/guteneberg/pr/54713/artifacts/build.zip"
}
}
}
}
}
```
That's extremely verbose, I'd love to explore a shorthand notation. One
idea would be to make it a valid
[URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) shaped
after the [data URL
syntax](https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data):
```js
const dataUri = `data:text/html;base64,%3Cscript%3Ealert%28%27hi%27%29%3B%3C%2Fscript%3E`;
const githubArtifactUri = `zip-github-artifact+url:https://github.com/WordPress/gutenberg/pr/54713/artifacts/build.zip`;
const gitResourceUri = `git:branch=HEAD;path=src,https://github.com/WordPress/hello-dolly.git`;
```
It wouldn't allow easy composition of the resources, e.g. a directory
inside a zip sourced from a GitHub repo. Maybe that's for the best,
though, since such a string would be extremely dense and difficult for
humans to read. The object-based syntax might still be the most
convenient way of to declare those.- Loading branch information
