Added gRPC functions to manage libraries in profiles by cmaglie · Pull Request #3019 · arduino/arduino-cli (original) (raw)

Hi @dankeboy36!

Thank you for your valuable feedback, as always.

If an IDE opens two sketches with different profiles (e.g. arduino:avr:uno and esp32:esp32:esp32da), should each sketch have its own gRPC client or session?

The basic idea is that when a profile is in use, the platforms and libraries installed globally should have no effect, and only the platforms and libraries listed in the profile should be visible. This idea was implemented by adding the profile parameter in the Init method:

message InitRequest { // An Arduino Core instance. Instance instance = 1; // Profile to use. string profile = 2; // The path where the sketch is stored. string sketch_path = 3; }

An instance initialized with a profile should be able to see only the libraries listed in the profile, and I've actually checked that this is the case, using the following profile:

profiles:
  uno:
    fqbn: arduino:avr:uno
    platforms:
      - platform: arduino:avr (1.8.3)
    libraries:
      - Servo (1.1.6)

If I try to initialize an instance of the CLI with the profile and try to get the list of installed libraries:

    err := srv.Init(&rpc.InitRequest{
        Instance:   inst,
        SketchPath: sketchPath.String(),
        Profile:    "uno",
    }, stream)
    res, err := srv.LibraryList(ctx, &rpc.LibraryListRequest{
        Instance: inst,
        All:      true,
    })
    require.NoError(t, err)
    for _, lib := range res.InstalledLibraries {
        fmt.Println("LIB>",
            lib.GetLibrary().GetName(),
            lib.GetLibrary().GetVersion(),
            lib.GetLibrary().GetLocation(),
            lib.GetLibrary().GetInstallDir())
    }

I got the expected:

LIB> Servo 1.1.6 LIBRARY_LOCATION_PROFILE /home/cmaglie/.arduino15/internal/Servo_1.1.6_663e17729cbd9f93/Servo
LIB> Wire 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/Wire
LIB> EEPROM 2.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/EEPROM
LIB> HID 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/HID
LIB> SPI 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/SPI
LIB> SoftwareSerial 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/SoftwareSerial

The Servo library is listed as LOCATION_PROFILE, the other libraries are PLATFORM_BUILTIN.

So, based on the previous test, the answer to the following question is:

When switching or activating a profile, does the CLI reinitialize the build environment (toolchains, platforms, libraries), or are these shared and cached? For example, if libA is installed in profileA but not in profileB, will a library search over gRPC return different results for each profile?

Yes, the result will differ depending on the profile. By the way, there is a distinction between a library search (i.e., searching for a library in the Library Index) and listing installed libraries. In the first case, searching for a library will always produce the same result regardless of the selected profile; however, listing the installed libraries will show different results based on the selected profile.

If multiple profiles use the same library, will it be installed multiple times, or can they share a global copy? Since each sketch can have its own sketch.yaml, does that mean the same dependency could be downloaded many times?

Libraries and platforms installed through profiles are placed in a private folder inside $DATA_DIR/internal/. The same package will not be downloaded twice; there are no duplicates here. Those directories should be read-only. By the way, there are no protections in place to prevent users from making changes.

$ ls ~/.arduino15/internal/ -l
totale 200
drwxr-xr-x  3 cmaglie cmaglie 4096 ott  3  2024  Adafruit_BusIO_1.16.1_ba866f2a3757c639
drwxr-xr-x  4 cmaglie cmaglie 4096 ott  9  2024 'Adafruit SleepyDog Library@1.6.5_a3c453d34b90db4b'
drwxrwxr-x  3 cmaglie cmaglie 4096 mag 30 16:02  arduino_arduinoOTA_1.2.1_c939277d7d79c18d
drwxr-xr-x  3 cmaglie cmaglie 4096 gen 23  2024  arduino_arduinoOTA_1.3.0_8902221f9b4869ce
drwxr-xr-x  6 cmaglie cmaglie 4096 mag 30 16:02  arduino_arm-none-eabi-gcc_7-2017q4_7b7be9f526b2cb64
drwxr-xr-x  8 cmaglie cmaglie 4096 nov 17 11:55  arduino_avr_1.8.3_40b0bc3206d75f60
drwxr-xr-x  8 cmaglie cmaglie 4096 gen 23  2024  arduino_avr_1.8.4_7ad8ae8e9f6a9dc9
drwxr-xr-x  9 cmaglie cmaglie 4096 gen 23  2024  arduino_avr_1.8.5_70334d3546efee53
drwxr-xr-x  9 cmaglie cmaglie 4096 lug  1  2024  arduino_avr_1.8.6_78e0815c6047bc54
drwxr-xr-x  4 cmaglie cmaglie 4096 gen 23  2024  arduino_avrdude_6.3.0-arduino17_b00caa3114ef923d
drwxr-xr-x  7 cmaglie cmaglie 4096 mag 30 16:03  arduino_avrdude_6.3.0-arduino9_734fb0aeab0d748f
drwxr-xr-x  8 cmaglie cmaglie 4096 gen 23  2024  arduino_avr-gcc_7.3.0-atmel3.6.1-arduino7_fc423089e09cfcab
drwxr-xr-x  3 cmaglie cmaglie 4096 lug  7 17:10  ArduinoBearSSL_1.7.6_3fd6f33f302ce290

When using gRPC, should compile, upload, etc. always load fqbn, port, and other settings from the active profile, or should clients omit them?

The Compile and Upload commands should retrieve the FQBN and Port from the profile, and the gRPC API allows overriding them if the FQBN and Port parameters are filled. At least, that was the intention.
I put the above statement to the test, and I discovered that:

The above must surely be fixed in the next releases. I'll open an issue for that.