Skip to content

Latest commit

 

History

History
217 lines (166 loc) · 13.8 KB

README_BuildProcess.md

File metadata and controls

217 lines (166 loc) · 13.8 KB
Raw

Espruino Internals: Build Process

Directories and Files

  • ChangeLog: What's new
  • boards/: Information on boards, used to auto-generate a lot of the code
  • doxygen/: Directory for auto-generated code documentation - see doxygen/README.md for more info
  • gen/: Auto-Generated Source Files
  • libs/: Optional libraries to include in Espruino (Math, Filesystem, Graphics, etc)
  • make/: Makefile fragments for each architecture
  • misc/: Other useful things
  • scripts/: Scripts for generating files in gen, and for analysing code/compilation/etc
  • src/: Main source code
  • targetlibs/: Libraries for targeted architectures
  • targets/: Specific code for targeted architectures
  • tests/: JavaScript Testcases
  • benchmark/: JavaScript Benchmarks
  • dist_*: files to be copied into distribution zip file
  • Makefile: the Makefile which controls the build process

Build Commands

Building for an Espruino board is generally done by typing:

BOARD=BOARDNAME make

And for release, by typing:

RELEASE=1 BOARD=BOARDNAME make

Valid board names are in comments at the head of the Makefile..

Release builds have all the assertions removed, reducing flash usage. Normal builds still have the assertions, and debug builds (with DEBUG=1 have more debug info in the assertions, as well as the debug info needed for proper GDB debugging).

The Make system has the following targets:

  • clean - Clean the build
  • all (or unspecified) - Build, including binaries needed for flashing the firmware
  • flash - Generate the firmware files and flash to the board
  • serialflash - Generate the firmware files and flash to the board via serial bootloader (used for STM32, especially the Original Espruino and Espruino Pico)

Build Process

  • Overall build logic is in the Makefile.
  • The Board Definition File in boards/$BOARD.py is parsed by scripts/get_makefile_decls.py to work out extra definitions for the Makefile (chip family, part number, binary name, whether it has a bootloader), which control the build.
  • Architecture-specic build logic is in the make directory - this is called based off family in the Board Definition File in boards/.
  • The Makefile adds source files to $SOURCES, and 'wrapper' source files to $WRAPPERSOURCES. Wrapper files are files that contain functions that are exposed to JS.
  • The script scripts/build_platform_config.py is run which generates gen/platform_config.h from boards/$BOARD.py - this contains information like the amount of RAM, as well as buffer sizes and the amount of variables that will be stored.
  • The script scripts/build_pininfo.py creates the pin definitions and puts them in gen/jspininfo.c
  • The script scripts/build_jswrapper.py is then run on $WRAPPERSOURCES - it generates gen/jswrapper.c - a hard-coded symbol table of built-in functions - from the Wrapper files.
  • Source files are built
  • On embedded targets, a linker file gen/linker.ld is auto-generated by scripts/build_linker.py based on the board definition (available flash, ram, bootloader etc). In some targets (especially non-STM32) a pre-made linker file will be used instead.
  • Everything is linked using link time optimisation (where possible) - this helps to inline code that wouldn't otherwise have been inlined, and generally makes for a much more efficient binary.
  • The elf file is converted with objdump to:
    • bin (for uploading to Espruino board)
    • hex (for uploading to mbed/etc)
    • lst (assembler listing - used for debugging)
  • scripts/check_size.sh does a sanity check of the bin file's size against what's described in the board definition, and fails if it won't fit into available flash memory.

Oddities

  • malloc and free are not included, because they're not needed and by default they use a large pool of valuable RAM to speed up memory allocation.
  • strcpy, memcpy, memset are all reimplemented in jsutils.h - again because of initial issues with high flash memory usage
  • Espruino on STM32 uses its own Maths library for double arithmetic, because when building initially the libraries included by GCC dynamically allocated memory - which pulled in malloc. If we could move back to GCC libraries at some point it'd be a huge bonus

Board Definition File

The board definition files are .py files that reside in the boards folder (for example the Pico one). They're used all over the place:

These contain:

info

  • Board name and link (for the HTML file)
  • Default console device, pins, and baus rate (the device Espruino goes to when USB is unplugged or not built in)
  • variables - The number of variables to use (this depends on the amount of RAM available). Less than 1023 vars use 12 bytes per var, more uses 16 bytes. You have to adjust this such that there is spare room for the stack and static variables (on Espruino boards this means leaving around 16kB free, but on smaller boards it can be reduced a lot)
  • bootloader - whether the binary image needs compiling with a special USB-VCP bootloader (Espruino boards only)
  • binary_name - the name of the binary that'll be produced
  • binaries - available binaries - this is used by the Web IDE to allow the user to choose which binary to upload
  • build - controls what gets build via the Makefile:
    • optimizeflags - flags like -O3 to give to the compiler
    • libraries - list of libraries to include
    • makefile - list of commands/definitions to execute in the Makefile

chip

  • part - Chip part number (this is defined in the compiler - eg if the part is STM32F401CDU6, -DSTM32F401CDU6 is put on the GCC command-line)
  • family - Chip family - also defined, but used in the Makefile to define what gets built
  • package - Used for ST chips when working out Pin Definitions to ensure that the right pins are included
  • ram on chip in KB
  • flash on chip in KB
  • speed of chip in MHz
  • Number of usarts
  • Number of spis
  • Number of i2cs
  • Number of adcs
  • Number of dacs
  • saved_code - how and where to save JS code in the chip's flash memory
    • The address of the start of the code in flash memory. Note that on STM32 chips this is at 0x08000000 even though code executes from 0x00000000 (the two areas are mirrored but 0x00000000 is faster to execute from while 0x08000000 is what's needed for writing to flash)
    • The page_size in bytes of the first flash page containing code (not used now?)
    • The number of flash pages we're using for code (not used now?)
    • flash_available in KB, used as a sanity check by scripts/check_size.sh after the build completes
  • place_text_section what address to start the program code at - This is 0 if not specified, but is useful in devices like the Pico where varying page sizes mean that the program code may be located after the saved program code (and/or bootloader)

board

Names of rows of pins - used solely by scripts/build_board_docs.py to generate the nice HTML description.

There can also be a _css element which contains raw Cascading Style Sheets, which contain carefully tweaked positions of the rows of pins defined in board, to align with the background image in boards/img

devices

This is a list of built-in stuff on the board that is made accessible to Espruino. It's:

  • Parsed and turned into a series of #defines in gen/platform_config.h by this code.
  • Used for the magenta tags shown near pins in the board's HTML file - that warn users that the pin might be used for some other function.

Stuff you can use is LED1-LED8, BTN1-BTN4, USB, LCD (for boards with FSMC LCDs built in), SD (SD card), JTAG (when JTAG pins are defined, but we need to make sure we leave them alone when the board resets). You can also define your own.

Pin Definitions

The pin definitions are created by scripts/build_pininfo.py and are stored in gen/jspininfo.c. The script calls get_pins in boards/$BOARD.py and a structure of the following form is returned:

[
   { "name":"PD12", "sortingname":"D12", "port":"D", "num":"20", "functions":{}, "csv":{} },
   { "name":"PD13", "sortingname":"D13", "port":"D", "num":"23", "functions":{ "SPI1_SCK":0 }, "csv":{} },
   ...
]

get_pins can define the raw array (like boards/MICROBIT.py), or on boards with STM32 chips like the Pico some utility functions auto-generate the code from CSV files that were copied from ST's datasheets.

  • name is the pin name - due to random historical reasons (from ST datasheets) it needs prefixing with P
  • sortingname is the name, but padded so that when it's sorted everything appears in the right order
  • port is the actual port - on ESP8266 this might not be needed and could just default to D
  • num is the pin number - this doesn't have to match D - it's what is needed internally to access the hardware. For instance Olimexino has 'logical' pins that actually map all over the place.
  • function is a map of pin functions to their 'alternate functions' (an STM32 chip thing - STM32F4 chips can have different peripherals on each pin, so the alternate function is a number that you shove in that pin's register in order to connect it to that peripheral). The format, for instance I2C1_SDA is important as it's parsed later and is used to build gen/jspininfo.c. The code to parse them is here
  • csv isn't needed, but when using data grabbed from csv files from ST's datasheets like this it contains the raw data for debugging)

Wrapper Files

Wrapper files are normal C source files that contain functions that are exposed to JavaScript code in the interpreter. They're prefixed jswrap_.

See libs/README.md for a short tutorial on how to add your wrapper files.

Each function to be exposed has a specially formatted comment above it containing JSON. The format of this JSON is specified in this file

An example of a wrapper file might be:

#include "jswrap_hello.h"  // We need the declaration of the jswrap_hello_world function
#include "jsinteractive.h" // Pull in the jsiConsolePrint function

// Let's define the JavaScript class that will contain our `world()` method. We'll call it `Hello`
/*JSON{
  "type" : "class",
  "class" : "Hello"
}
Some info about this class
*/

// Now, we define the `jswrap_hello_world` to be a `staticmethod` on the `Hello` class
/*JSON{
  "type" : "staticmethod",
  "class" : "Hello",
  "name" : "world",
  "generate" : "jswrap_hello_world"
}
We can write simple markdown-formatted comments in here. These
then get scanned and turned into the Espruino [Reference](http://www.espruino.com/Reference)

 * Bulletted
 * List
*/
void jswrap_hello_world() {
    jsiConsolePrint("Hello World!\r\n");
}

In the build process, scripts/build_jswrapper.py parses all these files, pulls out the headers, and generates gen/jswrapper.c - a hard-coded symbol table that resides on flash memory. It's consulted whenever

scripts/build_jswrapper.py also needs to be aware of what definitions were passed to the compiler - for instance if SAVE_ON_FLASH is defined, several non-vital functions will not be compiled in (so should not appear in the symbol table).

The wrapper files are also parsed by:

  • scripts/build_docs.py, which builds the HTML file used for the Espruino Reference.
  • scripts/build_tern_json.js, which generates a JSON description of all functions (along with their documentation) which is used for code completion in the Web IDE and to parse all code examples in order to produce the 'Examples' links in the Reference.