Partition Schemes in the Arduino IDE

pizza.jpg

A guide on how to add or edit the flash memory partitions in the Arduino IDE

Sometimes when you are creating a complicated Sketch especially when using WiFi or Bluetooth libraries you can run out of space in the flash memory on your device. You see an error like this

Sketch too big. Sketch uses 2233498 bytes (113%) of program storage space. Maximum is 1966080 bytes.

Most of the time this can be fixed by simply choosing a different in-built partition scheme from the Tools menu such as Huge App:

Arduino IDE Choose Partition Scheme

However if you need the OTA (Over the Air) partition or need to store data that survives power off and also need more space than is available in the IDE schemes then you can tweak the partitions to change the space available in different parts of the flash storage. Partitions can be thought of as slices of a cake or pizza. You can make them larger or smaller but the overall space is the same.

Create Partitions

The partition schemes are stored as CSV files. They can be found in one of two places, depending on how you installed the Arduino IDE.

Arduino IDE installed from the Windows Store:

C > Users > *your-user-name* > Documents > ArduinoData > packages > esp32 > hardware > esp32 > 1.0.1 > tools > partitions

Arduino IDE installed from the Arduino website:

C > Users > *your-user-name* > AppData > Local > Arduino15  > packages > esp32 > hardware > esp32 > 1.0.1 > tools > partitions

They look like this when opened in Notepad

Min Spiffs Partition

Huge App Partition

You can see the min_spiffs partition.csv scheme has two app partitions (app0 and app1) of 0x1E0000. Using a Hex to Decimal online converter the 0x1E0000 hex number converts to 1966080 bytes hence the error message above  Sketch uses 2233498 bytes (113%) of program storage space. Maximum is 1966080 bytes

The huge_app partition scheme has only one app partition of 0x300000 which is 3145728 bytes. This is plenty for the application but there is nowhere to store the permanent data as app0 is overwritten every time the board is flashed.

Persistent Storage

I needed to add a new partition for saving computed faces from a face recognition application. The computed faces are stored in a part of the on-board flash that has to be set up in certain way. A new partition type is needed. I created a new partition scheme file like this:

Fr Partition

In this file I changed the size of the app0 partition to be 0x280000 which is 2621440 bytes. I also created a second partition called fr with 0xEF000 (978944 bytes) of space . This second partition is used by my application to store permanent data. I used the name, type and subtype from this code for fr_flash.h here

#define FR_FLASH_TYPE 32
#define FR_FLASH_SUBTYPE 32
#define FR_FLASH_PARTITION_NAME “fr”

You can download my partition file from here: https://robotzero.one/wp-content/uploads/2019/04/rzo_partitions.csv

The offset of each partition should be the value of the previous offset plus the size of the previous partition. With SSDs and flash memory there are recommended sizes for partitioning disks. I had an error about offsets when I tried to create this using my own values and in the end I used HEX values from other examples to create my partitions. If someone knows the recommendations please let me know.

Adding the New Partition Scheme to the IDE

Once the new partition scheme file has been created you need to add a reference to it in the boards manager so it can be selected in the Arduino IDE. The boards.txt file is found in one of two places

Arduino IDE installed from the Windows Store:

C > Users > *your-user-name* > Documents > ArduinoData > packages > esp32 > hardware > esp32 > 1.0.1

Arduino IDE installed from the Arduino website:

C > Users > *your-user-name* > AppData > Local > Arduino15  > packages > esp32 > hardware > esp32 > 1.0.1

The boards.txt file can be edited in Notepad. Each definition is separated by a line like this: #########

I copied ESP Dev Board definition and changed the name and added three new lines for the new partition file. You could just add these lines to an existing definition if you prefer:

rzo.menu.PartitionScheme.rzo_partition=Face Recognition (2621440 bytes with OTA)
rzo.menu.PartitionScheme.rzo_partition.build.partitions=rzo_partitions
rzo.menu.PartitionScheme.rzo_partition.upload.maximum_size=2621440

You can copy and paste the definition below to create a new board if you have used my rzo partition definition file above.

##############################################################

rzo.name=Robot Zero One

rzo.upload.tool=esptool_py
rzo.upload.maximum_size=1310720
rzo.upload.maximum_data_size=2621440
rzo.upload.wait_for_upload_port=true

rzo.serial.disableDTR=true
rzo.serial.disableRTS=true

rzo.build.mcu=esp32
rzo.build.core=esp32
rzo.build.variant=esp32
rzo.build.board=ESP32_DEV

rzo.build.f_cpu=240000000L
rzo.build.flash_size=4MB
rzo.build.flash_freq=40m
rzo.build.flash_mode=dio
rzo.build.boot=dio
rzo.build.partitions=default
rzo.build.defines=

rzo.menu.PSRAM.disabled=Disabled
rzo.menu.PSRAM.disabled.build.defines=
rzo.menu.PSRAM.enabled=Enabled
rzo.menu.PSRAM.enabled.build.defines=-DBOARD_HAS_PSRAM -mfix-esp32-psram-cache-issue

rzo.menu.PartitionScheme.default=Default
rzo.menu.PartitionScheme.default.build.partitions=default
rzo.menu.PartitionScheme.minimal=Minimal (2MB FLASH)
rzo.menu.PartitionScheme.minimal.build.partitions=minimal
rzo.menu.PartitionScheme.no_ota=No OTA (Large APP)
rzo.menu.PartitionScheme.no_ota.build.partitions=no_ota
rzo.menu.PartitionScheme.no_ota.upload.maximum_size=2097152
rzo.menu.PartitionScheme.huge_app=Huge APP (3MB No OTA)
rzo.menu.PartitionScheme.huge_app.build.partitions=huge_app
rzo.menu.PartitionScheme.huge_app.upload.maximum_size=3145728
rzo.menu.PartitionScheme.min_spiffs=Minimal SPIFFS (Large APPS with OTA)
rzo.menu.PartitionScheme.min_spiffs.build.partitions=min_spiffs
rzo.menu.PartitionScheme.min_spiffs.upload.maximum_size=1966080
rzo.menu.PartitionScheme.fatflash=16M Fat
rzo.menu.PartitionScheme.fatflash.build.partitions=ffat
rzo.menu.PartitionScheme.rzo_partition=Face Recognition (2621440 bytes with OTA)
rzo.menu.PartitionScheme.rzo_partition.build.partitions=rzo_partition
rzo.menu.PartitionScheme.rzo_partition.upload.maximum_size=2621440

rzo.menu.CPUFreq.240=240MHz (WiFi/BT)
rzo.menu.CPUFreq.240.build.f_cpu=240000000L
rzo.menu.CPUFreq.160=160MHz (WiFi/BT)
rzo.menu.CPUFreq.160.build.f_cpu=160000000L
rzo.menu.CPUFreq.80=80MHz (WiFi/BT)
rzo.menu.CPUFreq.80.build.f_cpu=80000000L
rzo.menu.CPUFreq.40=40MHz (40MHz XTAL)
rzo.menu.CPUFreq.40.build.f_cpu=40000000L
rzo.menu.CPUFreq.26=26MHz (26MHz XTAL)
rzo.menu.CPUFreq.26.build.f_cpu=26000000L
rzo.menu.CPUFreq.20=20MHz (40MHz XTAL)
rzo.menu.CPUFreq.20.build.f_cpu=20000000L
rzo.menu.CPUFreq.13=13MHz (26MHz XTAL)
rzo.menu.CPUFreq.13.build.f_cpu=13000000L
rzo.menu.CPUFreq.10=10MHz (40MHz XTAL)
rzo.menu.CPUFreq.10.build.f_cpu=10000000L

rzo.menu.FlashMode.qio=QIO
rzo.menu.FlashMode.qio.build.flash_mode=dio
rzo.menu.FlashMode.qio.build.boot=qio
rzo.menu.FlashMode.dio=DIO
rzo.menu.FlashMode.dio.build.flash_mode=dio
rzo.menu.FlashMode.dio.build.boot=dio
rzo.menu.FlashMode.qout=QOUT
rzo.menu.FlashMode.qout.build.flash_mode=dout
rzo.menu.FlashMode.qout.build.boot=qout
rzo.menu.FlashMode.dout=DOUT
rzo.menu.FlashMode.dout.build.flash_mode=dout
rzo.menu.FlashMode.dout.build.boot=dout

rzo.menu.FlashFreq.80=80MHz
rzo.menu.FlashFreq.80.build.flash_freq=80m
rzo.menu.FlashFreq.40=40MHz
rzo.menu.FlashFreq.40.build.flash_freq=40m

rzo.menu.FlashSize.4M=4MB (32Mb)
rzo.menu.FlashSize.4M.build.flash_size=4MB
rzo.menu.FlashSize.2M=2MB (16Mb)
rzo.menu.FlashSize.2M.build.flash_size=2MB
rzo.menu.FlashSize.2M.build.partitions=minimal
rzo.menu.FlashSize.16M=16MB (128Mb)
rzo.menu.FlashSize.16M.build.flash_size=16MB
rzo.menu.FlashSize.16M.build.partitions=ffat

rzo.menu.UploadSpeed.921600=921600
rzo.menu.UploadSpeed.921600.upload.speed=921600
rzo.menu.UploadSpeed.115200=115200
rzo.menu.UploadSpeed.115200.upload.speed=115200
rzo.menu.UploadSpeed.256000.windows=256000
rzo.menu.UploadSpeed.256000.upload.speed=256000
rzo.menu.UploadSpeed.230400.windows.upload.speed=256000
rzo.menu.UploadSpeed.230400=230400
rzo.menu.UploadSpeed.230400.upload.speed=230400
rzo.menu.UploadSpeed.460800.linux=460800
rzo.menu.UploadSpeed.460800.macosx=460800
rzo.menu.UploadSpeed.460800.upload.speed=460800
rzo.menu.UploadSpeed.512000.windows=512000
rzo.menu.UploadSpeed.512000.upload.speed=512000

rzo.menu.DebugLevel.none=None
rzo.menu.DebugLevel.none.build.code_debug=0
rzo.menu.DebugLevel.error=Error
rzo.menu.DebugLevel.error.build.code_debug=1
rzo.menu.DebugLevel.warn=Warn
rzo.menu.DebugLevel.warn.build.code_debug=2
rzo.menu.DebugLevel.info=Info
rzo.menu.DebugLevel.info.build.code_debug=3
rzo.menu.DebugLevel.debug=Debug
rzo.menu.DebugLevel.debug.build.code_debug=4
rzo.menu.DebugLevel.verbose=Verbose
rzo.menu.DebugLevel.verbose.build.code_debug=5

In the IDE the new scheme will be available for the new board or any board you add the new definition to:

Arduino IDE Custom Partition Scheme

References

Why programmers use Hex: https://www.i-programmer.info/babbages-bag/478-hexadecimal.html?start=1
Hex to Binary Converter: https://www.rapidtables.com/convert/number/hex-to-binary.html
Hex Calculator: https://ncalculators.com/digital-computation/hex-addition-calculator.htm
Partitioned Pizza Photo: Vita Marija Murenaite on Unsplash

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

scroll to top