# Getting Started Guide

This guide provides step-by-step instructions for setting up the environment and reproducing the experiments presented in the paper.
You can find the official website at https://csg-tokyo.github.io/bluescript/.

## Requirements

### Hardware

* An [M5Stack FIRE](https://docs.m5stack.com/en/core/fire) (v2.6 or v2.7) as the **microcontroller**.
    * If you don't have an M5Stack Fire, Go to "[Getting Started Guid and Step-by-Step Instructions without microcontroller](#getting-started-guid-and-step-by-step-instructions-without-microcontroller)".
* A computer running Windows, Linux, or macOS as the **host machine**.
* A USB cable to connect the **microcontroller** and the **host machine**.

### Software

* [Docker](https://docs.docker.com/get-docker/)
* [Docker Compose](https://docs.docker.com/compose/install/)
* [esptool](https://docs.espressif.com/projects/esptool/en/latest/esp32/)
* [Google Chrome](https://www.google.com/chrome/)
    * The BlueScript interactive environment (REPL) communicates with the microcontroller via Web Bluetooth, which is currently best supported by Google Chrome.

---

## Installation and Setup

### Host Machine Setup

1.  Navigate to the project's root directory and start the BlueScript server using Docker Compose.
    ```bash
    cd bluescript
    docker compose up
    ```
2.  Verify that the server has started.
    * Open **Google Chrome** and navigate to `http://localhost:3000/`.
    * You should see the BlueScript REPL (Read-Eval-Print Loop) interface.

### Microcontroller Setup

1.  Build the BlueScript local runtime.
    In the project's root directory, run the following command:
    ```bash
    docker run --rm -v $PWD:/project -w /project/microcontroller/ports/esp32 -u $UID -e HOME=/tmp espressif/idf:release-v5.0 idf.py build
    ```
2.  Connect the microcontroller to your host machine via USB and identify its serial port.
    Common serial port naming patterns are:
    * Windows: `COM` followed by a number (e.g., `COM3`).
    * macOS: `/dev/tty.` followed by a name (e.g., `/dev/tty.usbserial-0001`).
    * Linux: `/dev/ttyUSB` or `/dev/ttyACM` followed by a number.

    For more details, refer to the official documentation: [Establish Serial Connection with ESP32](https://docs.espressif.com/projects/esp-idf/en/stable/esp32/get-started/establish-serial-connection.html).

    Once you have identified the port, set the `PORT` environment variable.
    ```bash
    # Example for macOS
    PORT=/dev/tty.usbserial-0001
    ```

3.  Flash the BlueScript runtime onto the microcontroller.
    ```bash
    esptool.py --chip esp32 -b 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 40m --flash_size 4MB \
    0x1000 ./microcontroller/ports/esp32/build/bootloader/bootloader.bin \
    0x10000 ./microcontroller/ports/esp32/build/bluescript.bin \
    0x8000 ./microcontroller/ports/esp32/build/partition_table/partition-table.bin
    ```
---

## Verifying the Setup

1.  Open **Google Chrome** and navigate to `http://localhost:3000/`.

2.  Click the **Start** button. You’ll be prompted to select a Bluetooth device. Choose `BLUESCRIPT` and pair with it.
    * If you cannot find the device, ensure Bluetooth is enabled on your host machine and that Chrome has permission to use it. You may need to grant permissions in your browser or operating system settings.

3.  Type the following code into the first cell:
    ```typescript
    print("Hello world!");
    ```
4.  Click the execution (▶) button to the left of the cell to execute the code.
    * You should see `Hello world!` appear in the **OUTPUT** area in the bottom-right corner of the page.

---

# Overview of Claims

In the paper, we present the design and implementation of BlueScript and evaluate it from the following four perspectives, which you can reproduce using this artifact.

* **Execution Time** (Section 4.1, Figures 3 & 4)
* **Interactivity** (Section 4.2, Figure 5)
* **Performance changes when applying the dynamic compiler** (Section 4.3, Figures 6 & 7)
* **Memory Footprint** (Section 4.4, Figure 8)

---

# Step-by-Step Instructions

**Important Note**: If you don't have microcontroller devices, you cannot run this section. Go to [Getting Started Guid and Step-by-Step Instructions without microcontroller](#getting-started-guid-and-step-by-step-instructions-without-microcontroller)

## Benchmark Suites

We use the following benchmark suites, which are all included in this repository.

1.  **Typed "Are We Fast Yet?" benchmarks**: Located at `benchmarks/are-we-fast-yet/bluescript`
2.  **Untyped "Are We Fast Yet?" benchmarks**: Located at `benchmarks/are-we-fast-yet/bluescript-without-type`
3.  **Typed "ProgLangComp" benchmarks**: Located at `benchmarks/prog-lang-comp/bluescript`
4.  **Untyped "ProgLangComp" benchmarks**: Located at `benchmarks/prog-lang-comp/bluescript-without-type`
5.  **Interactivity benchmarks (from micro:bit examples)**: Located at `benchmarks/microbit-examples`

## Execution time (Section 4.1)

This experiment measures the execution time of various benchmark programs, corresponding to **Figures 3 and 4** in the paper. We use benchmark suites 1-4.

**Steps to Reproduce:**

1.  For each benchmark program in suites 1, 2, 3, and 4, repeat the following steps:
    1.  Open Google Chrome and navigate to `http://localhost:3000/`.
    2.  Connect to the microcontroller by clicking the "Start" button and selecting `BLUESCRIPT`.
    3.  **Disable the dynamic compiler** by unchecking the "Use JIT" box at the top of the page.
    4.  Copy the entire content of a benchmark file (e.g., `benchmarks/are-we-fast-yet/bluescript/bounce.bs`) and paste it into the first cell.
    5.  Execute the first cell by clicking the execution button to the left of the cell.
    6.  In the second cell, paste the following code snippet and execute it. This code runs the benchmark function and prints the average execution time.
        ```typescript
        import { getTimeMs } from 'timer';
        
        // Run the benchmark 3 times and calculate the average
        const start = getTimeMs();
        benchmark(3); // The argument '3' is an example iteration count
        const end = getTimeMs();
        print((end - start) / 3);
        ```
    7.  The measured execution time (in milliseconds) will be displayed in the OUTPUT area. Record this value.

## Interactivity (Section 4.2)

This experiment measures the compilation, execution, and communication times for each code fragment in an interactive session, corresponding to **Figure 5** in the paper. We use benchmark suite 5.

In these benchmarks, the program for each scenario is split into commented sections like `// Cell1`, `// Cell2`, etc. These correspond to code fragments #1, #2, etc., in Figure 5.

**Steps to Reproduce:**

1.  First, enable logging by replacing a file:
    * Copy the contents of `benchmarks/microbit-examples/bluescript/repl-context-with-log.tsx` and paste them over the contents of `notebook/src/hooks/repl-context.tsx`.
2.  Open Google Chrome and navigate to `http://localhost:3000/`.
3.  Connect to the microcontroller.
4.  For each scenario in benchmark suite 5, perform the following:
    1.  Open the browser's Developer Tools (usually by pressing F12 or Ctrl+Shift+I) and go to the Console tab.
    2.  Copy the code under the `// Cell1` comment and paste it into the first cell of the BlueScript REPL.
    3.  Execute the cell. You will see log output in the console similar to this:
        ```
        push execution button, current: 34024.19999998808
        receive output, current: 38863.89999997616
        compilation: 920.2223750000121
        execution: 3.9179999828338623
        ```
        From this output, calculate the times for fragment #1:
        * **Compilation time** = `920.2223750000121`
        * **Execution time** = `3.9179999828338623`
        * **Communication time** = `(38863.89999997616 - 34024.19999998808) - 920.2223750000121 - 3.9179999828338623`
    4.  Repeat the process for `// Cell2`, `// Cell3`, etc., pasting the code into new cells and recording the timing data from the console for each fragment.

## Dynamic compilation (Section 4.3)

This experiment investigates the performance improvement from the dynamic compiler, corresponding to **Figures 6 and 7**. We measure the execution time of the untyped benchmarks (suites 2 and 4) over 20 iterations.

**Steps to Reproduce (Cold Run, Warm Run, and Iterations):**

1.  For each benchmark in suites 2 and 4, repeat the following:
    1.  Go to `http://localhost:3000/` in Google Chrome and connect to the microcontroller.
    2.  **Ensure the "Use JIT" checkbox is checked (enabled).**
    3.  Paste the benchmark program into the first cell and execute it.
    4.  In the second cell, paste and execute the following code. It will run the benchmark every 2 seconds for 20 iterations.
        ```typescript
        import { setInterval, clearInterval, getTimeMs } from 'timer';

        let count = 0;
        const timerId = setInterval(() => {
            if (count >= 20) {
                clearInterval(timerId);
                return;
            }
            const start = getTimeMs();
            benchmark(1);
            const end = getTimeMs();
            print(end - start);
            count++;
        }, 2000);
        ```
    5.  Wait for about 40 seconds. The OUTPUT area will display the execution times for all 20 iterations.
        * The **1st** value corresponds to the **Cold run** in Figure 6.
        * The **15th** value corresponds to the **Warm run** in Figure 6.
        * All **20** values correspond to the data points in Figure 7.

**Steps to Reproduce (Baseline):**

1.  For each benchmark in suites 2 and 4, repeat the following:
    1.  Go to `http://localhost:3000/` and connect to the microcontroller.
    2.  **Disable the dynamic compiler** by unchecking the "Use JIT" box.
    3.  Paste the benchmark program into the first cell and execute it.
    4.  In the second cell, paste and execute the following code:
        ```typescript
        import { getTimeMs } from 'timer';

        const start = getTimeMs();
        benchmark(1);
        const end = getTimeMs();
        print(end - start);
        ```
    5.  The result in the OUTPUT area is the **Baseline** execution time for Figure 6.

## Memory Footprint (Section 4.4)

This experiment measures the flash memory footprint of the BlueScript runtime on the microcontroller, corresponding to **Figure 8**.

**To measure the full memory footprint (with Bluetooth):**

1.  In the project's root directory, execute:
    ```bash
    docker run --rm -v $PWD:/project -w /project/microcontroller/ports/esp32 -u $UID -e HOME=/tmp espressif/idf:release-v5.0 idf.py size
    ```
2.  The output will show a size summary. The `Total image size` at the end is the value used in Figure 8.
    ```
    ...
    Total image size:  752009 bytes (.bin may be padded larger)
    ```

**To measure the memory footprint without Bluetooth:**

1.  Open `microcontroller/core/include/protocol.h` and comment out line 10:
    ```c
    // #define BS_PROTOCOL_USE_BLUETOOTH
    ```
2.  Re-run the `idf.py size` command from the previous step.
    ```bash
    docker run --rm -v $PWD:/project -w /project/microcontroller/ports/esp32 -u $UID -e HOME=/tmp espressif/idf:release-v5.0 idf.py size
    ```
3.  The new `Total image size` corresponds to the memory footprint without the Bluetooth component.

---

# Getting Started Guide and Step-by-Step Instructions without microcontroller
Even if you don't have an M5Stack device, you can still reproduce the **Execution time** experiments on your host machine.
Experiments requiring the microcontroller—**Interactivity**, **Dynamic compilation**, and **Memory Footprint**—cannot be reproduced this way.

**Important Note**: Performance results obtained on the host machine will differ from those reported in the paper, as the code will be running on your host CPU instead of the ESP32 microcontroller.

**Requirements**
* [Docker](https://docs.docker.com/get-docker/)
* [Docker Compose](https://docs.docker.com/compose/install/)

**Steps to Reproduce The Execution Time Experiment**

1.  For each benchmark program in suites 1, 2, 3, and 4 in [Benchmark Suites](#benchmark-suites), repeat the following steps:
    1.  In the project's root directory, execute the following command to launch the BlueScript interactive shell on the host machine.
        ```bash
        docker compose run -it --rm server npm run shell
        ```
    2.  Inside the shell, load prerequisite files.
        ```bash
        > .load ../modules/std/std-for-host.bs
        > .load ../modules/timer/timer-for-host.bs
        ```
    3.  Load the benchmark file (e.g. `benchmarks/are-we-fast-yet/bluescript/bounce.bs`).
        ```bash
        > .load ../benchmarks/are-we-fast-yet/bluescript/bounce.bs
        ```
    4.  Execute the following code:
        ```bash
        > const start=getTimeMs(); benchmark(3); const end = getTimeMs(); print(end-start);
        ```
    5.  The measured execution time (in milliseconds) will be displayed in the shell.
    6.  Exit the shell by typing `.quit` or `Ctrl-D`
