@@ -36,7 +36,7 @@
Now, you can make a pull request targeting the master branch of the MarkBind repository.
Fill in the pull request template.
Wait for the CI checks to pass.
For first-time contributors, you may need to wait for the MarkBind team to approve your PR before the CI checks can run.
Verify that the PR preview is working.
The PR preview is generated by netlify/markbind-master/deploy-preview.
You can find the link to the PR preview in the checks section of the PR.
Open the link to the PR preview and ensure that your changes are reflected in the rendered site
Follow up with PR reviews and make changes as requested.
Since this is a hypothetical change, you do not need to raise a PR. You can refer to PR #1924 for an example.
Checkpoint
For actual documentation changes, check that the following are true:
The PR template has been filled in
The PR preview is working
Summary
In this task, you have learned how to make documentation changes to MarkBind!
While documentation changes are not as exciting as code changes, they are still important. In particular, our documentation is generated by MarkBind itself, utilizing the same features that we are documenting. This means that our documentation is a great place to discover and analyze how MarkBind is used!
Here are some additional tasks for you to try out:
Review a real documentation change (e.g issue #1772 was resolved in PR #1924 to improve the user guide)
Check if there are existing issues tagged with the a-Documentation label
-
If there are, try to resolve them!
If there are not, raise an issue when you discover a documentation bug or improvement (e.g make this bootcamp documentation better)!
When you are ready, move on to learn how to squash some bugs!
Update your GitHub repository setting for deploying to GitHub Pages.
-
Navigate to the Settings > Pages section on GitHub for that repository and set the source to Deploy from a branch and the branch to gh-pages and /(root).
Checkpoint
Check that the following are true:
The triggered GitHub action runs are successful
The site is deployed and available at https://<username>.github.io/mb-dev-xxx/ (after a few seconds to a few minutes)
Congratulations! ๐๐๐ You have deployed your MarkBind site!
Summary
In the first part of this bootcamp, we experimented with MarkBind as a user. We have set up the master branch of MarkBind and created a new MarkBind site. We have also created content with MarkBind components/syntax and deployed our site.
Here are some additional tasks for you to try out:
Navigate to the Settings > Pages section on GitHub for that repository and set the source to Deploy from a branch and the branch to gh-pages and /(root).
Checkpoint
Check that the following are true:
The triggered GitHub action runs are successful
The site is deployed and available at https://<username>.github.io/mb-dev-xxx/ (after a few seconds to a few minutes)
Congratulations! ๐๐๐ You have deployed your MarkBind site!
Summary
In the first part of this bootcamp, we experimented with MarkBind as a user. We have set up the master branch of MarkBind and created a new MarkBind site. We have also created content with MarkBind components/syntax and deployed our site.
Here are some additional tasks for you to try out:
Run npm run test in the root directory to run the entire test suite.
See that the test passes
Update Documentation
If the bug is related to a feature that is documented, we should update the documentation to reflect the changes. This is especially important if the bug fix changes the behavior of the feature.
In this case, we will not need to update the documentation.
Create PR & Follow up with the Reviews
Now that we have completed our changes, we can create a PR to merge the changes into the master branch.
For first-time contributors, you may need to wait for the MarkBind team to approve your PR before the CI checks can run.
Verify that the PR preview is working.
-
The PR preview is generated by netlify/markbind-master/deploy-preview.
You can find the link to the PR preview in the checks section of the PR.
Open the link to the PR preview and ensure that your changes are reflected in the rendered site
Follow up with PR reviews and make changes as requested.
For changes that involve code logic, some discussion may be required to determine the best way to fix the bug. Hence, it may be helpful to get feedback early either by creating a draft PR or by discussing the approach in the issue thread.
Summary
In this task, you have learned how to deal with a bug report. While the example bug report is simple, the steps are generally the same for more complex bug reports.
Here are some additional tasks for you to try out:
The PR preview is generated by netlify/markbind-master/deploy-preview.
You can find the link to the PR preview in the checks section of the PR.
Open the link to the PR preview and ensure that your changes are reflected in the rendered site
Follow up with PR reviews and make changes as requested.
For changes that involve code logic, some discussion may be required to determine the best way to fix the bug. Hence, it may be helpful to get feedback early either by creating a draft PR or by discussing the approach in the issue thread.
Summary
In this task, you have learned how to deal with a bug report. While the example bug report is simple, the steps are generally the same for more complex bug reports.
Here are some additional tasks for you to try out:
By this time, you should have a good idea about how tests and documentation are written. Where possible, unit tests should also be added to ensure that the feature works as expected. Utilizing the new feature in our documentation is also encouraged so that we can get a better idea of how the feature can be used.
When adding intra-site links, use tags to differentiate between the deployment environment when appropriate, as mentioned in Adding Intra-Site Links to Documentation!
Create PR & Follow up with the Reviews
(Same as what we mentioned in the previous task)
Fill in the pull request template.
Wait for the CI checks to pass.
For first-time contributors, you may need to wait for the MarkBind team to approve your PR before the CI checks can run.
Verify that the PR preview is working.
-
The PR preview is generated by netlify/markbind-master/deploy-preview.
You can find the link to the PR preview in the checks section of the PR.
Open the link to the PR preview and ensure that your changes are reflected in the rendered site
Follow up with PR reviews and make changes as requested.
For features, it is also crucial to get early feedback on implementation details. This is because some considerations may need to be exercised when introducing new user-facing syntax. Another problem is that the implementation may not be the most appropriate or straightforward one, sometimes due to how existing code is structured.
Issue #1475 is an example where the work in PR #1988 involved changing the implementation midway.
Summary
In this part, we have reviewed the steps to implement a new feature. Working on a new feature can be a daunting task, but it is also very rewarding to own a feature from start to finish.
Here are some additional tasks for you to try out:
Look through the mentioned PRs and try to understand how the features were implemented.
Look through the issue tracker and see if there are any features that you would like to work on.
And...that's it!๐๐๐
You have completed the bootcamp! Take a well-deserved rest and we look forward to your contributions!
The PR preview is generated by netlify/markbind-master/deploy-preview.
You can find the link to the PR preview in the checks section of the PR.
Open the link to the PR preview and ensure that your changes are reflected in the rendered site
Follow up with PR reviews and make changes as requested.
For features, it is also crucial to get early feedback on implementation details. This is because some considerations may need to be exercised when introducing new user-facing syntax. Another problem is that the implementation may not be the most appropriate or straightforward one, sometimes due to how existing code is structured.
Issue #1475 is an example where the work in PR #1988 involved changing the implementation midway.
Summary
In this part, we have reviewed the steps to implement a new feature. Working on a new feature can be a daunting task, but it is also very rewarding to own a feature from start to finish.
Here are some additional tasks for you to try out:
Look through the mentioned PRs and try to understand how the features were implemented.
Look through the issue tracker and see if there are any features that you would like to work on.
And...that's it!๐๐๐
You have completed the bootcamp! Take a well-deserved rest and we look forward to your contributions!
New developers are often confused about how to get started in brown-field projects. This onboarding bootcamp is created to ramp up developer familiarity with the MarkBind codebase and get you started with contributing (in a variety of ways) to MarkBind.
In the following pages, we will go through a series of tasks and you are encouraged to follow along to walk through some of the typical development workflows. When in doubt, you can always refer to the rest of the developer guide for more information.
Structure
Each Task will include a "TLDR" section that summarizes the key objectives. This is intended for developers who wish to attempt the task on their own. The rest of the page will include a step-by-step guide and checkpoints
-to help you complete the task.
This onboarding bootcamp is intended for developers who are keen to make more than just one contribution to the project. It is particularly well-suited for onboarding NUS Independent Work Module (CP3108A and CP3108B) and Thematic Systems Project (CS3281 aka Software Engineering in Live Projects) students, but may also be useful to any external developers who wish to dive deeper into the codebase. While we encourage all new developers to go through this process, it is optional and developers are welcome to jump straight into making contributions if they prefer.
The maintenance of this onboarding bootcamp document does require effort. If you encounter any issues as the materials become outdated or have suggestions, we highly encourage you to submit an issue or make a pull request to update the materials. This will help us keep the onboarding process up-to-date and effective for future developers.
This onboarding bootcamp is intended for developers who are keen to make more than just one contribution to the project. It is particularly well-suited for onboarding NUS Independent Work Module (CP3108A and CP3108B) and Thematic Systems Project (CS3281 aka Software Engineering in Live Projects) students, but may also be useful to any external developers who wish to dive deeper into the codebase. While we encourage all new developers to go through this process, it is optional and developers are welcome to jump straight into making contributions if they prefer.
The maintenance of this onboarding bootcamp document does require effort. If you encounter any issues as the materials become outdated or have suggestions, we highly encourage you to submit an issue or make a pull request to update the materials. This will help us keep the onboarding process up-to-date and effective for future developers.
This page provides an overview of the MarkBind's architecture.
The above diagram shows the key classes and in MarkBind. You may note the following from these:
Key classes
The 3 key classes representing different types of content are as follows:
Page โ a page, as specified by the user in their various site configuration glob or src properties. These are directly managed by the Site instance.
Layout โ a single layout file, as contained in the _markbind/layouts folder, collectively managed by LayoutManager.
Note that Layout instances do not generate any output files directly, but rather, they store intermediate processing results to be used in a Page.
External โ source files referenced in a Page, Layout or even another External that result in a output file. These output files are loaded dynamically and on-demand in the browser. These instances are managed by a single ExternalManager instance.
For example, the file referenced by a <panel src="xx.md"> generates a separate xx._include_.html, which is requested and loaded only when the panel is expanded.
Content processing flow
Every Page, Layout and External follows the same overall content processing flow, that is:
Nunjucks Markdown Html
Note that generation of Externals (e.g. <panel src="...">) do not fall within the content processing flow where they are referenced.
These are only flagged for generation, and then processed by ExternalManagerafter, in another similar content processing flow within an External instance.
Rationale
This simple three stage flow provides a simple, predictable content processing flow for the user, and removes several other development concerns:
As the templating language of choice, Nunjucks is always processed first, allowing its templating capabilities to be used to the full extent.
-Its syntax is also the most compatible and independent of the other stages.
Secondly, Markdown is rendered before HTML, which produces more HTML. This also allows core Markdown features (e.g. code blocks) and Markdown plugins with eccentric syntaxes to be used without having to patch the HTML parser.
Having processed possibly conflicting Nunjucks and Markdown syntax, HTML is then processed last.
+Its syntax is also the most compatible and independent of the other stages.
Secondly, Markdown is rendered before HTML, which produces more HTML. This also allows core Markdown features (e.g. code blocks) and Markdown plugins with eccentric syntaxes to be used without having to patch the HTML parser.
Having processed possibly conflicting Nunjucks and Markdown syntax, HTML is then processed last.
This page gives you an overview of the MarkBind's internal project structure.
Packages
The MarkBind project is developed in a (MarkBind/markbind) of 4 packages:
The core library, which parses and processes MarkBind's various syntaxes, resides in the packages/core/ directory.
The command-line interface (CLI) application, which accepts commands from users and then uses the core library to parse and generate web pages, resides in the packages/cli/ directory.
The core web library, which contains a generated web bundle from various setup scripts and the UI components library, resides in packages/core-web/.
The UI components library, which MarkBind authors can use to create content with complex and interactive structure, resides in the packages/vue-components/ directory.
Stack used: Node.js, Vue.js
MarkBind core library
The core library mainly houses:
Functions and libraries used to parse and process MarkBind into usable output are stored in src. The architecture described in Architecture is contained here. A brief rundown of what it includes:
Various key functionalities in processing MarkBind syntax into valid HTML output, stored in html. The other part of the content processing flow is found in variables, which manages site variables and facilitates the Nunjucks calls.
Page files generate a single page of the site, and are managed by the Site instance. Site uses the Page model's interface to generate pages, and performs various other utility-like functions related to site generation such as copying of external assets into the output folder.
Layout holds the files relating to the layout of the site and are managed by LayoutManager. Similarly, External files, which are separate output files to be loaded dynamically and on-demand, are managed by a ExternalManager instance.
Various libraries (contained in lib) and plugins (in plugins) are also stored here. Some external libraries have also been amended to suit MarkBind's purpose โ see patches.
MarkBind's templates, used in the markbind init command.
Unit Tests (though there are more unit tests and functional tests in the cli library)
The key external libraries used are:
markdown-it, which does the Markdown parsing and rendering. There are also several customized markdown-it plugins used in MarkBind, which are located inside the src/lib/markdown-it/ directory.
Several markdown-it plugins are installed to enhance the existing Markdown syntax. They can be found in src/package.json. Some of them are patched in the src/lib/markdown-it/patches/ directory to fit MarkBind's needs.
Additionally, there are some markdown-it plugins in the src/lib/markdown-it/plugins/ directory (either forked, modified or written to enhance existing functionalities).
htmlparser2, a speedy and forgiving HTML parser which exposes a DOM-like object structure to work on. To comply with the markdown spec, and our custom requirements, src/patches/htmlparser2.js patches various behaviours of this library.
cheerio, which is a node.js equivalent of jQuery. Cheerio uses htmlparser2 to parse the HTML as well, hence our patches propagate here.
Nunjucks, which is a JavaScript templating engine. Nunjucks is used to support our variable system to help with reusing small bits of code in multiple places. The package is patched and stored in src/patches/nunjucks to make it compatible with other MarkBind syntax processing steps.
MarkBind CLI
The CLI application uses and further builds on the interface exposed by the core library's Site model to provide functionalities for the author, such as markbind serve which initiates a live reload workflow.
live-server, which is a simple web server for local development and preview of a MarkBind site. The package is patched and stored in src/lib/live-server with our custom fine tuning.
MarkBind core-web library
This package houses the various frontend assets used in the core package.
Some external assets included are Vue.js, Bootstrap bundles, and FontAwesome bundles.
Internal bundles are also present, generated from setup scripts, custom stylesheets and the UI components library.
UI components library
This package consists of a mix of Bootstrap and proprietary components rewritten in Vue.js based on our needs for educational websites.
This page gives you an overview of the MarkBind's internal project structure.
Packages
The MarkBind project is developed in a (MarkBind/markbind) of 4 packages:
The core library, which parses and processes MarkBind's various syntaxes, resides in the packages/core/ directory.
The command-line interface (CLI) application, which accepts commands from users and then uses the core library to parse and generate web pages, resides in the packages/cli/ directory.
The core web library, which contains a generated web bundle from various setup scripts and the UI components library, resides in packages/core-web/.
The UI components library, which MarkBind authors can use to create content with complex and interactive structure, resides in the packages/vue-components/ directory.
Stack used: Node.js, Vue.js
MarkBind core library
The core library mainly houses:
Functions and libraries used to parse and process MarkBind into usable output are stored in src. The architecture described in Architecture is contained here. A brief rundown of what it includes:
Various key functionalities in processing MarkBind syntax into valid HTML output, stored in html. The other part of the content processing flow is found in variables, which manages site variables and facilitates the Nunjucks calls.
Page files generate a single page of the site, and are managed by the Site instance. Site uses the Page model's interface to generate pages, and performs various other utility-like functions related to site generation such as copying of external assets into the output folder.
Layout holds the files relating to the layout of the site and are managed by LayoutManager. Similarly, External files, which are separate output files to be loaded dynamically and on-demand, are managed by a ExternalManager instance.
Various libraries (contained in lib) and plugins (in plugins) are also stored here. Some external libraries have also been amended to suit MarkBind's purpose โ see patches.
MarkBind's templates, used in the markbind init command.
Unit Tests (though there are more unit tests and functional tests in the cli library)
The key external libraries used are:
markdown-it, which does the Markdown parsing and rendering. There are also several customized markdown-it plugins used in MarkBind, which are located inside the src/lib/markdown-it/ directory.
Several markdown-it plugins are installed to enhance the existing Markdown syntax. They can be found in src/package.json. Some of them are patched in the src/lib/markdown-it/patches/ directory to fit MarkBind's needs.
Additionally, there are some markdown-it plugins in the src/lib/markdown-it/plugins/ directory (either forked, modified or written to enhance existing functionalities).
htmlparser2, a speedy and forgiving HTML parser which exposes a DOM-like object structure to work on. To comply with the markdown spec, and our custom requirements, src/patches/htmlparser2.js patches various behaviours of this library.
cheerio, which is a node.js equivalent of jQuery. Cheerio uses htmlparser2 to parse the HTML as well, hence our patches propagate here.
Nunjucks, which is a JavaScript templating engine. Nunjucks is used to support our variable system to help with reusing small bits of code in multiple places. The package is patched and stored in src/patches/nunjucks to make it compatible with other MarkBind syntax processing steps.
MarkBind CLI
The CLI application uses and further builds on the interface exposed by the core library's Site model to provide functionalities for the author, such as markbind serve which initiates a live reload workflow.
live-server, which is a simple web server for local development and preview of a MarkBind site. The package is patched and stored in src/lib/live-server with our custom fine tuning.
MarkBind core-web library
This package houses the various frontend assets used in the core package.
Some external assets included are Vue.js, Bootstrap bundles, and FontAwesome bundles.
Internal bundles are also present, generated from setup scripts, custom stylesheets and the UI components library.
UI components library
This package consists of a mix of Bootstrap and proprietary components rewritten in Vue.js based on our needs for educational websites.
You may refer to MarkBind's project structure to get a better understanding of how the packages work together.
What is Server-side Rendering and Why?
In MarkBind's context, SSR refers to the generation of proper HTML strings from Vue components on the server (core library) instead of shifting that responsibility to the client-side (browser).
The main motivation that we had for introducing SSR into MarkBind is to enhance user experience by resolving the unsightly issue of Flash-of-Unstyled-Content (FOUC), which occurs due to our reliance on Client-side Rendering (CSR).
Client-side Hydration
SSR and Client-side Hydration are 2 concepts that go hand-in-hand. Essentially, once we produce the static HTML via SSR and send it over to the client-side, Vue on the client-side will execute what is known as Client-side Hydration on the static HTML.
During the hydration process, Vue essentially compares your SSR HTML markup against the virtual DOM generated by the render function on the client-side. If any difference is found, meaning that the application that we have on the client-side (the virtual DOM tree) differs from the SSR HTML mark-up that we send to the client, Vue will reject the SSR HTML output, bail Client-side Hydration, and execute full CSR.
This is known as "Hydration Issue" and it is one of the main challenges you will face with SSR in MarkBind.
Using production build
In production build, Vue does not perform hydration check. Any errors resulting from hydration will trigger full CSR without warning. MarkBind uses the development build by default when you serve it in development mode (-d option).
Penalties of Hydration Issue
When hydration fails, on top of the wasted time and effort in executing SSR, we will also incur the additional time penalty of executing Client-side Hydration (where CSR will follow afterwards).
Fortunately, even if we face hydration issues and execute full CSR, the FOUC problem will still be resolved nonetheless. The reason for this is because the SSR HTML markup should resemble the CSR HTML markup to a large extent.
Supposedly, hydration issues typically occurs due to minor differences between client-side rendered virtual DOM tree and the server-rendered content. Of course, this is assuming that we are adhering to the concept of "universal application" as much as possible, which will be explained in the following section.
Avoiding Hydration Issue
Conceptually, to prevent hydration issue, what we should always strive to achieve is a "universal application".
It is not difficult to achieve a "universal application" per-se because we merely have to ensure two things:
the initial state is the same between client-side and server-side.
after compiling and rendering the Vue page application, the SSR HTML mark-up is not modified.
Beyond achieving a "universal application", note that the HTML specifications should also be adhered to.
Some common mistakes are as such:
Having block-level elements within <p> tag
Having unknown HTML elements within our Vue application during compilation/rendering (though this can be easily resolved by adding v-pre to the unknown element, so that Vue will ignore that element during compilation).
If you are unsure what elements are allowed within other elements, or what constitutes invalid HTML in general, a good resource to reference would be the MDN Web Docs.
Note that the list only included the common causes of hydration issue that MarkBind developers have ran into. There may be other causes of hydration issue that are not listed here (although unlikely).
You may refer to MarkBind's project structure to get a better understanding of how the packages work together.
What is Server-side Rendering and Why?
In MarkBind's context, SSR refers to the generation of proper HTML strings from Vue components on the server (core library) instead of shifting that responsibility to the client-side (browser).
The main motivation that we had for introducing SSR into MarkBind is to enhance user experience by resolving the unsightly issue of Flash-of-Unstyled-Content (FOUC), which occurs due to our reliance on Client-side Rendering (CSR).
Client-side Hydration
SSR and Client-side Hydration are 2 concepts that go hand-in-hand. Essentially, once we produce the static HTML via SSR and send it over to the client-side, Vue on the client-side will execute what is known as Client-side Hydration on the static HTML.
During the hydration process, Vue essentially compares your SSR HTML markup against the virtual DOM generated by the render function on the client-side. If any difference is found, meaning that the application that we have on the client-side (the virtual DOM tree) differs from the SSR HTML mark-up that we send to the client, Vue will reject the SSR HTML output, bail Client-side Hydration, and execute full CSR.
This is known as "Hydration Issue" and it is one of the main challenges you will face with SSR in MarkBind.
Using production build
In production build, Vue does not perform hydration check. Any errors resulting from hydration will trigger full CSR without warning. MarkBind uses the development build by default when you serve it in development mode (-d option).
Penalties of Hydration Issue
When hydration fails, on top of the wasted time and effort in executing SSR, we will also incur the additional time penalty of executing Client-side Hydration (where CSR will follow afterwards).
Fortunately, even if we face hydration issues and execute full CSR, the FOUC problem will still be resolved nonetheless. The reason for this is because the SSR HTML markup should resemble the CSR HTML markup to a large extent.
Supposedly, hydration issues typically occurs due to minor differences between client-side rendered virtual DOM tree and the server-rendered content. Of course, this is assuming that we are adhering to the concept of "universal application" as much as possible, which will be explained in the following section.
Avoiding Hydration Issue
Conceptually, to prevent hydration issue, what we should always strive to achieve is a "universal application".
It is not difficult to achieve a "universal application" per-se because we merely have to ensure two things:
the initial state is the same between client-side and server-side.
after compiling and rendering the Vue page application, the SSR HTML mark-up is not modified.
Beyond achieving a "universal application", note that the HTML specifications should also be adhered to.
Some common mistakes are as such:
Having block-level elements within <p> tag
Having unknown HTML elements within our Vue application during compilation/rendering (though this can be easily resolved by adding v-pre to the unknown element, so that Vue will ignore that element during compilation).
If you are unsure what elements are allowed within other elements, or what constitutes invalid HTML in general, a good resource to reference would be the MDN Web Docs.
Note that the list only included the common causes of hydration issue that MarkBind developers have ran into. There may be other causes of hydration issue that are not listed here (although unlikely).
Second, ensure you have a basic knowledge of the following:
JavaScript (ES6)
Node.js
HTML & CSS
Markdown
Command-line environment
Third, ensure you have spent time with MarkBind and familiar with the content of the user guide. It's important to know the product well before you start contributing code.
Now, follow these steps to get started on contributing:
Second, ensure you have a basic knowledge of the following:
JavaScript (ES6)
Node.js
HTML & CSS
Markdown
Command-line environment
Third, ensure you have spent time with MarkBind and familiar with the content of the user guide. It's important to know the product well before you start contributing code.
Now, follow these steps to get started on contributing:
Follow steps in Using CI Platforms but update the config files for the various CI Platforms to use the correct Node.js version. Try deploying and ensure there are no problems with deployment.
If a different npm version is needed
Install the correct npm version before running npm commands.
Update documentation
Update Node.js and npm version in documentation. See Update to use Node 16 as an example.
-Don't forget to update the version numbers in the example config files in Deploying the Site!
@@ -41,7 +41,7 @@
For example: git commit --no-verify -m "Rename core/src/html to TypeScript".
Step 3: "Adapt" Commit
Start automatic compilation on change/save. More on this at Editing backend features section in the Workflow page.
Change the CommonJS module import/export syntax to TypeScript's equivalent syntax (use ES6 syntax only if possible).
A common error in newly renamed files comes from the way modules are imported/exported. Those statements can be converted according to the Import/Export Syntax Reference section below. The steps are as follows:
Start by changing the import/export statements into TypeScript equivalent syntax. Be careful with imports as you have to match them with the export syntax that the modules use, be it TypeScript equivalent or ES6.
Check whether it is possible to change the export to use ES6 syntax. As mentioned in the syntax reference, ES6 should not be used if the module only exports a single thing. Change to ES6 syntax only if possible, otherwise keep with TypeScript equivalent syntax.
You might have to adjust how the files are imported by other TypeScript (.ts) files. If you are changing the export to the TypeScript equivalent syntax, then it must be imported the same way as well. The same goes for the ES6 syntax.
Adapt the files fully to TypeScript.
The errors from TypeScript and typescript-eslint can guide you on what to fix. Only fix what is necessary and be careful with accidentally modifying any code functionality. Avoid using any as best as you can.
If you happen to encounter type errors related to using names (of constant, properties, members, etc.) from another MarkBind import that is still in JavaScript (.js), you can try to infer simple types in your file first as a "stand-in" of the more robust types when that internal dependency is eventually migrated.
On the flip side of the above situation, once you have developed a robust type for your own file, adjust how other TypeScript (.ts) files type names referenced from your files as applicable. Some type declarations might need to be made stricter, or can be removed if it's not necessary anymore, and so on. This will ensure that the files will be incrementally stricter.
Make sure everything is in order by running npm run test in the root directory.
The core package tests can directly run on the .ts files (powered by ts-jest), and the cli package tests uses the compiled files from core. Therefore, passing the cli tests means that it is very likely the compiled files work as expected.
Stage the adapted files and the typings from Step 1 for commit. Verify the changes before committing.
Commit the changes with the message Adapt <file-or-folder> to TypeScript.
For example, Adapt core/src/html to TypeScript.
You are now ready to create a pull request for the changes to the repository.
(Optional) Step 4: Changing the commit
While refactoring you may encounter scenarios where you might have to amend your code after a commit. There are 2 common scenarios which we will cover.
Scenario 1: You have 2 commits and want to add on to the second commit.
Scenario 2: You have more than 2 commits and want to make it into 2 commits.
Example of Migrated Works
You can see these pull requests to observe the finished migration works:
Here is the reference on changing the import/export statements from CommonJS syntax into those that TypeScript support: TypeScript equivalent syntax, and ES6 syntax.
The TypeScript equivalent syntax is designed to compile to the exact same code as CommonJS. You can read more about this syntax in the official documentation.
The ES6 syntax is the newer JavaScript syntax that TypeScript recommends, but in certain cases it does not compile to the exact same code as CommonJS. We simplify the concepts here for illustration, refer to the StackOverflow post here for more details.
In the tables below, a "thing" is defined as a JavaScript construct that can be exported, such as constants, functions, classes, etc. In ES6 syntax, a "thing" can also be TypeScript constructs like types, interfaces, enums, etc.
Exports
What to Export
CommonJS
TypeScript equivalent
ES6
One thing only
module.exports = x
export = x
export default xย ^
Object of one/more things
module.exports = { x, y, z }
export = { x, y, z }
export { x, y, z }
^: For compatibility, do not useexport default during migration. It is compiled differently and does not play well with dependants that are still in .js, unless some clunky modifications are made in the .js files (refer to the StackOverflow post previously linked).
Imports from a module that exports one thing only
What to Import
CommonJS
TypeScript equivalent
ES6
Exported thing
const x = require(a)
import x = require(a)ย ^
import x from aย ^
^: You can only use TypeScript equivalent syntax when the exported module is also in the TypeScript equivalent syntax. The same goes for the ES6 syntax.
Imports from a module that exports an object of one/more things
What to Import
CommonJS
TypeScript equivalent
ES6
Entire object
const w = require(a)
import w = require(a)
import * as w from a
Selected things
const { x } = require(a)
N/A ย ^
import { x } from a
^: While not equivalent on the compile-level, this can be achieved by importing the entire object then destructuring the object.
Explanation Notes
Why do we need to separate "Rename" and "Adapt" commits
We want to keep the file history intact throughout this migration, meaning that we should still be able to see the change history of the .ts files even from the time when they were still in .js.
When we rename files, we have to keep the Git similarity index between the old file and the new file above a certain threshold (50%) to keep the history, or else it will be regarded as a different file with its own history.
Combining the two steps into one - that is, adapting files immediately after renaming - will lower the similarity index, therefore making it possible for the files, especially small ones or those that need a lot of changes when adapting, to hit below the similarity threshold and lose their history.
The solution to this is to make the rename and adapt commits separate, in order for Git to recognize first that the file has changed into .ts, and so the changes are compared not against the old .js file, but to the renamed .ts file instead.
Why do we need a rebase and merge commit
Even if the migration developer has kept the history intact through the separate "Rename" and "Adapt" commits, this is only intact at their working branch. At the end of the day, the totality of the changes in the working branch is compared against the target branch, in which the same consideration would apply.
If we do the usual squash commit, the changes from the two commits are combined into a new commit and only that commit will be pushed into the target branch. The original two commits are omitted, therefore the history of the working branch that we have tried to keep intact is stripped away. This results in a linear history, as if the changes in the working branch were made
-after the latest changes in the target branch.
@@ -34,7 +34,7 @@
in the root folder of your cloned repo.
Congratulations! Now you are ready to start modifying MarkBind code.
Setting up the git hooks (optional but recommended)
There are a few Git hooks implemented using the pre-commit tool that runs common tasks like linting, automated tests and compiling typescript files.
Install the hooks by running python3 ./pre-commit/pre-commit-2.20.0.pyz install in the root folder of your cloned repo.
If you ever need to uninstall the hooks, simply run uninstall instead of install.
Common setup questions
Does MarkBind work with all operating systems?
Yes! We support all operating systems.
How does MarkBind manage dependencies?
MarkBind uses lerna, a popular
multi-package development tool, to manage its dependencies. It is
essentially a high level wrapper over node and npm's functionalities.
How do I move back to the released version of MarkBind?
To go back to the released version of MarkBind, run
-npm un -g markbind-cli, followed by npm i -g markbind-cli.
Some of my front-end components are not working as expected when running markbind serve.
Try running either markbind serve -d or npm run build:web to view frontend changes (especially after pulling a frontend update that someone else may have pushed). You can see more details here.
+npm un -g markbind-cli, followed by npm i -g markbind-cli.
Some of my front-end components are not working as expected when running markbind serve.
Try running either markbind serve -d or npm run build:web to view frontend changes (especially after pulling a frontend update that someone else may have pushed). You can see more details here.
We follow our style guides. Using a linter will help check and fix some of the code style errors in your code. It will save time for both you and your code reviewer. The linting tool we use is ESLint and StyleLint. Here is a gist with an explanation of the ESLint rules chosen in markbind-cli.
Before making a commit or pull request, you should lint your code by running the following commands from the root of your project:
To lint a specific file: eslint path/to/specificfile.js
To lint all files: npm run lint
It is also possible to auto-fix some (not all) style errors, using npm run lintfix.
ESLint has integrations with popular editors. They offer features such as "fix errors on save", which will make development smoother.
Git hooks
We have three git hooks in our project. We use pre-commit to manage our git hooks.
-The pre-commit scripts are located in ./pre-commit/pre-commit-scripts and the config is found in ./pre-commit-config.yaml.
To skip running the pre-commit hook or pre-push hook, you can use the --no-verify flag (e.g. git push --no-verify origin fork_branch).
post-checkout: When you checkout to another branch, this hook will run clean and build the backend.
pre-commit: This hook will run clean, build the backend, and run lintfix on all the files.
pre-push: This hook will run clean, build the backend, and run all the test cases.
Add your dependency into the appropriate package.json file inside packages/*. If this is a development dependency to be used across all packages (e.g. ESLint), add it the the root package.json.
Then, simply rerun the npm run setup command to update the package-lock.json files.
To delete a dependency
The safest way is to first remove the particular dependency entry from the package.json file of the respective directory. Then, run npm run setup in the root directory to clean up the local dependencies and update the package-lock.json file.
To update a dependency
First, follow the instruction to delete the dependency. Then, follow the instruction to add the latest dependency back. Also, when updating dependencies, ensure that it is updated in all packages using that dependency.
Dependency updates are not trivial, and can be the source of subtle bugs. You should always check the respective dependency changelogs before doing so!
Points for consideration
There are a few ways to incorporate external packages into MarkBind, each with its pros and cons. The following table shows some of the common trade-offs:
Approach
Pros
Cons
Installing
Ease of upgrade
Traceable in package.json
May not satisfy custom behavior
Become vulnerable if the source repo is no longer actively maintained
Forking
Relatively easy to upgrade
Leverage upstream testing procedures
Benefit others who share the same use cases
May become out-of-sync with the latest version
Difficult to maintain it externally in the long run
Patching
Quick - no need to maintain / publish more npm packages or setup release procedures etc
Ensure the changes propagate to other dependencies
Enjoy the benefits of monorepo
Difficult to upgrade
As the choice is highly dependent on context and details of the implementation, below are some additional questions to ask before proceeding:
Is the package actively maintained?
How big is the package?
How invasive are the proposed changes?
Are there existing APIs/plugin system from the package (to modify default behaviors)?
Find out more about the key external libraries used in MarkBind from the project structure section. Also, the rationales behind most existing patches are documented in their respective files, read them (and their respective PRs/issues) for more context!
Updating PlantUML
PlantUML is a third-party library used by MarkBind to create UML diagrams. MarkBind runs the PlantUML JAR file found in packages/core/src/plugins/default when building the site to generate the diagrams.
Rename the file to plantuml.jar (if required), and replace the existing JAR file located in packages/core/src/plugins/default.
Check the HTML pages that contain PlantUML diagrams, i.e. /userGuide/components/imagesAndDiagrams.html.
Updating Bootstrap and Bootswatch
As Bootswatch is built on Bootstrap, ensure that the versions of both are in sync to avoid unexpected differences in styling behavior between default and other themes. Both are currently using version 5.1.3.
Add your dependency into the appropriate package.json file inside packages/*. If this is a development dependency to be used across all packages (e.g. ESLint), add it the the root package.json.
Then, simply rerun the npm run setup command to update the package-lock.json files.
To delete a dependency
The safest way is to first remove the particular dependency entry from the package.json file of the respective directory. Then, run npm run setup in the root directory to clean up the local dependencies and update the package-lock.json file.
To update a dependency
First, follow the instruction to delete the dependency. Then, follow the instruction to add the latest dependency back. Also, when updating dependencies, ensure that it is updated in all packages using that dependency.
Dependency updates are not trivial, and can be the source of subtle bugs. You should always check the respective dependency changelogs before doing so!
Points for consideration
There are a few ways to incorporate external packages into MarkBind, each with its pros and cons. The following table shows some of the common trade-offs:
Approach
Pros
Cons
Installing
Ease of upgrade
Traceable in package.json
May not satisfy custom behavior
Become vulnerable if the source repo is no longer actively maintained
Forking
Relatively easy to upgrade
Leverage upstream testing procedures
Benefit others who share the same use cases
May become out-of-sync with the latest version
Difficult to maintain it externally in the long run
Patching
Quick - no need to maintain / publish more npm packages or setup release procedures etc
Ensure the changes propagate to other dependencies
Enjoy the benefits of monorepo
Difficult to upgrade
As the choice is highly dependent on context and details of the implementation, below are some additional questions to ask before proceeding:
Is the package actively maintained?
How big is the package?
How invasive are the proposed changes?
Are there existing APIs/plugin system from the package (to modify default behaviors)?
Find out more about the key external libraries used in MarkBind from the project structure section. Also, the rationales behind most existing patches are documented in their respective files, read them (and their respective PRs/issues) for more context!
Updating PlantUML
PlantUML is a third-party library used by MarkBind to create UML diagrams. MarkBind runs the PlantUML JAR file found in packages/core/src/plugins/default when building the site to generate the diagrams.
Rename the file to plantuml.jar (if required), and replace the existing JAR file located in packages/core/src/plugins/default.
Check the HTML pages that contain PlantUML diagrams, i.e. /userGuide/components/imagesAndDiagrams.html.
Updating Bootstrap and Bootswatch
As Bootswatch is built on Bootstrap, ensure that the versions of both are in sync to avoid unexpected differences in styling behavior between default and other themes. Both are currently using version 5.1.3.
@@ -43,7 +43,7 @@
Bundlephobia may be useful to quickly look up the size of a package!
Dependencies
When choosing to use a third-party library or package, it should ideally be well-maintained and not have too many dependencies.
While dependencies may be inevitable, a package with dependencies on large libraries may lag behind the most recent releases of these libraries, which may become a blocker for MarkBind to migrate to these recent releases as well.
For instance, if bootstrap-vue depends on Bootstrap and Vue, we will need to wait for bootstrap-vue to migrate to the newest versions of both Bootstrap and Vue before MarkBind can migrate to these versions of Bootstrap and Vue as well.
Feel free to raise any concerns during the initial discussion phase for other devs to weigh in on the tradeoffs!
Attributes and Slots
MarkBind components may support , or both.
Some components allow users to supply the same content as either a slot or an attribute.
-If an author provides the same content as both a slot and an attribute, in most cases, the slot should override the attribute.
MarkBind should also log a warning to inform the author of this conflict!
Tag properties are top-level properties of the tag configuration object. The following table lists what these properties are used for:
Property
Values
Default
Remarks
isSpecial
true or false
false
Allows configuring whether any tag is to be parsed "specially" like a <script> or <style> tag. This allows configuring custom tags that may contain conflicting syntax, such as the <puml> tag used for UML diagram generation.
attributes
Array of attribute configurations
[]
Contains the attribute configurations of the tags.
Attribute Properties
The following table lists what the possible properties for configuring the attributes of a tag, and what they are used for:
Property
Values
Default
Remarks
name
attribute name
none
The string name of the attribute.
isRelative
true or false
false
Should be true if this attribute may contain a relative link. This tells MarkBind to properly resolve such relative links when used in <include>s, by converting the link into a baseUrl preceded absolute link.
isSourceFile
true or false
false
Should be true if the attribute points to a source file. This allows flagging other source files to trigger page regeneration during live reload.
Lifecycle hooks
You may also need to maintain some plugin state during site generation, then reset this when the site or pages are regenerated.
To do this, you may implement the beforeSiteGenerate method.
beforeSiteGenerate(): Called during initial site generation and subsequent regenerations during live preview.
-
Create a new branch from master to work on your changes.
Make necessary modifications to the workflow files.
Update the repository's README.md to reflect the changes.
Testing the action
Include a workflow file in your to test your modified version of the action.
Instead of using uses: MarkBind/markbind-action@v2 in the workflow file, use uses: yourGithubName/markbind-action@yourBranch to reference the unpublished version of the action that you are currently developing.
-
e.g. uses: tlylt/markbind-action@main
Trigger the action as needed, check and validate the results in the action logs.
You can now submit PRs to improve MarkBind's GitHub actions! ๐
Create and validate a release on a release branch (such as release/v1) before creating the release tag (for example, v1.0.2).
Create a release using semantic versioning. For more information, see "Creating releases."
Move the major version tag (such as v1, v2) to point to the Git ref of the current release. For more information, see "Git basics - tagging."
Introduce a new major version tag (v2) for changes that will break existing workflows. For example, changing an action's inputs would be a breaking change.
Create and validate a release on a release branch (such as release/v1) before creating the release tag (for example, v1.0.2).
Create a release using semantic versioning. For more information, see "Creating releases."
Move the major version tag (such as v1, v2) to point to the Git ref of the current release. For more information, see "Git basics - tagging."
Introduce a new major version tag (v2) for changes that will break existing workflows. For example, changing an action's inputs would be a breaking change.
Find out the details of the existing reusable workflows in its README.md.
As there is no need to release the reusable workflows in order for others to use, the workflows will be updated and maintained in the master branch (without doing a release).
Find out the details of the existing reusable workflows in its README.md.
As there is no need to release the reusable workflows in order for others to use, the workflows will be updated and maintained in the master branch (without doing a release).
diff --git a/devGuide/index.page-vue-render.js b/devGuide/index.page-vue-render.js
index 1565147f..eebcd3fe 100644
--- a/devGuide/index.page-vue-render.js
+++ b/devGuide/index.page-vue-render.js
@@ -5,6 +5,6 @@ with(this){return _c('div',{attrs:{"id":"app"}},[_c('header',{attrs:{"sticky":""
};
var pageVueStaticRenderFns = [function anonymous(
) {
-with(this){return _c('div',[_c('footer',[_c('div',{staticClass:"text-center"},[_c('small',[_v("[Generated by "),_c('a',{attrs:{"href":"https://markbind.org/"}},[_v("MarkBind 5.2.0")]),_v(" on Sat, 24 Feb 2024, 6:52:28 UTC]")]),_c('br'),_v(" "),_c('small',[_v("This site is powered by "),_c('a',{attrs:{"href":"https://www.netlify.com/"}},[_v("Netlify")]),_v(".")])])])])}
+with(this){return _c('div',[_c('footer',[_c('div',{staticClass:"text-center"},[_c('small',[_v("[Generated by "),_c('a',{attrs:{"href":"https://markbind.org/"}},[_v("MarkBind 5.3.0")]),_v(" on Sat, 24 Feb 2024, 7:47:49 UTC]")]),_c('br'),_v(" "),_c('small',[_v("This site is powered by "),_c('a',{attrs:{"href":"https://www.netlify.com/"}},[_v("Netlify")]),_v(".")])])])])}
}];
\ No newline at end of file
diff --git a/devGuide/projectManagement.html b/devGuide/projectManagement.html
index 31c1c721..34fbda98 100644
--- a/devGuide/projectManagement.html
+++ b/devGuide/projectManagement.html
@@ -3,7 +3,7 @@
-
+
MarkBind - Project management
@@ -260,7 +260,7 @@
to include 'tlylt' in .all-contributorsrc and README.md.
The senior dev approves and merge the automatic PR.
The senior dev deletes the automatic PR.
Note that:
All contribution types specified here can be used.
E.g. code, doc, mentoring, question, test, bug
Multiple contribution types can be included at once or updated later on.
E.g. "@all-contributors please add tlylt for code, doc"
Contribution types can be updated later on.
-
E.g "@all-contributors please add tlylt for code"
Some time later: "@all-contributors please add tlylt for doc"
Result: 'tlylt' will be updated to have both icons(code and doc)
