kaa example too big to fit esp8266 - kaa

Question

I'm experimenting with the kaa IoT platform on an ESP8266 and have followed this guide. I've set up a new build environment in an Lubuntu virtual box and the build appears to run ok however it complains of a lack of code space (see below).
It seems that the ESP8266 can't handle encryption so I made sure that was turned off in the cmake file using "-DWITH_ENCRYPTION=OFF" but that does not help. Weirdly, the build still seems to mention encryption so perhaps I've not disabled it properly.
I don't understand why it mentions "Built target mbedtls" when I've specified the target as esp8266.
Anyway, here's the build output in case anyone can help me :-)
martin#martin-VirtualBox:/media/sf_martin/kaaSDKs/esp8266-sample/build$ make clean
martin#martin-VirtualBox:/media/sf_martin/kaaSDKs/esp8266-sample/build$ make
-- Toolchain path: /opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf
-- Default SDK location will be used: /opt/Espressif/esp-rtos-sdk
==================================
BUILD_TYPE = MinSizeRel
KAA_PLATFORM = esp8266
KAA_MAX_LOG_LEVEL = 3
==================================
BOOTSTRAP ENABLED
PROFILE ENABLED
USER EXTENSION ENABLED
KAA WILL BE INSTALLED TO /usr/local
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Could NOT find cppcheck
-- Configuring done
-- Generating done
-- Build files have been written to: /media/sf_martin/kaaSDKs/esp8266-sample/build
[ 1%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/aes.c.obj
[ 2%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/asn1parse.c.obj
[ 3%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/asn1write.c.obj
[ 5%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/bignum.c.obj
[ 6%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/ctr_drbg.c.obj
[ 7%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/entropy.c.obj
[ 8%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/entropy_poll.c.obj
[ 10%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/md.c.obj
[ 11%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/md_wrap.c.obj
[ 12%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/oid.c.obj
[ 13%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/pk.c.obj
[ 15%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/pkwrite.c.obj
[ 16%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/pem.c.obj
[ 17%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/pk_wrap.c.obj
[ 18%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/rsa.c.obj
[ 20%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/sha256.c.obj
[ 21%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/pkparse.c.obj
[ 22%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/sha1.c.obj
[ 23%] Building C object libs/kaa/tools/kaa_encryption/rsa_key_gen/mbedtls/CMakeFiles/mbedtls.dir/base64.c.obj
[ 25%] Linking C static library libmbedtls.a
[ 25%] Built target mbedtls
[ 26%] Creating directories for 'kaa_rsa_keygen'
[ 27%] No download step for 'kaa_rsa_keygen'
[ 28%] No patch step for 'kaa_rsa_keygen'
[ 30%] No update step for 'kaa_rsa_keygen'
[ 31%] Performing configure step for 'kaa_rsa_keygen'
-- Configuring done
-- Generating done
-- Build files have been written to: /media/sf_martin/kaaSDKs/esp8266-sample/build/libs/kaa/kaa_keys
[ 32%] Performing build step for 'kaa_rsa_keygen'
[ 83%] Built target mbedtls
[ 91%] Built target rsa_keygen
[100%] Built target generate_rsa_keys
[ 33%] Generating RSA headers
[ 35%] Completed 'kaa_rsa_keygen'
[ 35%] Built target kaa_rsa_keygen
[ 36%] Building C object libs/kaa/CMakeFiles/extension_profile.dir/src/extensions/profile/kaa_profile.c.obj
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/extensions/profile/kaa_profile.c: In function 'kaa_profile_manager_is_profile_set':
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/extensions/profile/kaa_profile.c:195:64: warning: unused parameter 'self' [-Wunused-parameter]
bool kaa_profile_manager_is_profile_set(kaa_profile_manager_t *self)
^
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/extensions/profile/kaa_profile.c: In function 'kaa_profile_manager_update_profile':
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/extensions/profile/kaa_profile.c:394:71: warning: unused parameter 'self' [-Wunused-parameter]
kaa_error_t kaa_profile_manager_update_profile(kaa_profile_manager_t *self, kaa_profile_t *profile_body)
^
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/extensions/profile/kaa_profile.c:394:92: warning: unused parameter 'profile_body' [-Wunused-parameter]
kaa_error_t kaa_profile_manager_update_profile(kaa_profile_manager_t *self, kaa_profile_t *profile_body)
^
[ 37%] Linking C static library libextension_profile.a
[ 37%] Built target extension_profile
Scanning dependencies of target kaac
[ 38%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/logger.c.obj
[ 40%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/kaa_client.c.obj
[ 41%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/configuration_persistence.c.obj
[ 42%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/status.c.obj
[ 43%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/tcp_utils.c.obj
[ 45%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/time.c.obj
[ 46%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/exit.c.obj
[ 47%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/snprintf.c.obj
[ 48%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/common/sha.c.obj
[ 50%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/common/kaa_failover_strategy.c.obj
[ 51%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/common/ext_log_storage_memory.c.obj
[ 52%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/common/ext_log_upload_strategies.c.obj
[ 53%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/common/key_utils.c.obj
[ 55%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/common/kaa_tcp_channel.c.obj
[ 56%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_protocols/kaa_tcp/kaatcp_parser.c.obj
[ 57%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_protocols/kaa_tcp/kaatcp_request.c.obj
[ 58%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/avro_src/io.c.obj
[ 60%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/avro_src/encoding_binary.c.obj
[ 61%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/collections/kaa_list.c.obj
[ 62%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/utilities/kaa_aes_rsa.c.obj
[ 63%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/utilities/kaa_log.c.obj
[ 65%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/utilities/kaa_mem.c.obj
[ 66%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/utilities/kaa_buffer.c.obj
[ 67%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_platform_utils.c.obj
[ 68%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_platform_protocol.c.obj
[ 70%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_channel_manager.c.obj
[ 71%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_common_schema.c.obj
[ 72%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_status.c.obj
[ 73%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa.c.obj
[ 75%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/kaa_extension.c.obj
[ 76%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/common/kaa_htonll.c.obj
[ 77%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/gen/kaa_profile_gen.c.obj
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_profile_gen.c: In function 'kaa_profile_empty_data_deserialize':
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_profile_gen.c:47:76: warning: unused parameter 'reader' [-Wunused-parameter]
kaa_profile_empty_data_t *kaa_profile_empty_data_deserialize(avro_reader_t reader)
^
[ 78%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/gen/kaa_logging_gen.c.obj
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_logging_gen.c: In function 'kaa_logging_empty_data_deserialize':
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_logging_gen.c:47:76: warning: unused parameter 'reader' [-Wunused-parameter]
kaa_logging_empty_data_t *kaa_logging_empty_data_deserialize(avro_reader_t reader)
^
[ 80%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/gen/kaa_configuration_gen.c.obj
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_configuration_gen.c: In function 'kaa_configuration_empty_data_deserialize':
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_configuration_gen.c:47:88: warning: unused parameter 'reader' [-Wunused-parameter]
kaa_configuration_empty_data_t *kaa_configuration_empty_data_deserialize(avro_reader_t reader)
^
[ 81%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/gen/kaa_notification_gen.c.obj
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_notification_gen.c: In function 'kaa_notification_empty_data_deserialize':
/media/sf_martin/kaaSDKs/esp8266-sample/libs/kaa/src/kaa/gen/kaa_notification_gen.c:47:86: warning: unused parameter 'reader' [-Wunused-parameter]
kaa_notification_empty_data_t *kaa_notification_empty_data_deserialize(avro_reader_t reader)
^
[ 82%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/gen/kaa_chat.c.obj
[ 83%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/gen/kaa_chat_definitions.c.obj
[ 85%] Linking C static library libkaac.a
[ 85%] Built target kaac
[ 86%] Building C object libs/kaa/CMakeFiles/extension_user.dir/src/extensions/user/kaa_user.c.obj
[ 87%] Linking C static library libextension_user.a
[ 87%] Built target extension_user
[ 88%] Building C object libs/kaa/CMakeFiles/extension_bootstrap.dir/src/extensions/bootstrap/kaa_bootstrap_manager.c.obj
[ 90%] Linking C static library libextension_bootstrap.a
[ 90%] Built target extension_bootstrap
[ 91%] Building C object CMakeFiles/kaa_demo_s.dir/user/user_main.c.obj
[ 92%] Building C object CMakeFiles/kaa_demo_s.dir/driver/uart.c.obj
[ 93%] Building C object CMakeFiles/kaa_demo_s.dir/src/kaa_demo.c.obj
[ 95%] Linking C static library libkaa_demo_s.a
[ 95%] Built target kaa_demo_s
Scanning dependencies of target kaa_demo
[ 96%] Building C object CMakeFiles/kaa_demo.dir/blank.c.obj
[ 97%] Linking C executable kaa_demo
/opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: kaa_demo section `.text' will not fit in region `iram1_0_seg'
/opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: region `iram1_0_seg' overflowed by 24817 bytes
collect2: error: ld returned 1 exit status
CMakeFiles/kaa_demo.dir/build.make:122: recipe for target 'kaa_demo' failed
make[2]: *** [kaa_demo] Error 1
CMakeFiles/Makefile2:107: recipe for target 'CMakeFiles/kaa_demo.dir/all' failed
make[1]: *** [CMakeFiles/kaa_demo.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
martin#martin-VirtualBox:/media/sf_martin/kaaSDKs/esp8266-sample/build$

Answer 1

I had the exact same problem when first building the sample app with a custom SDK according to the tutorial you referenced.
First of, the Built target mbedtls refers to the build targets specified by the make file, not your target platform. As you can see in the beginning, the KAA_PLATFORM = esp8266 shows the correct value. Later on, make compiles the correct files as well (for example: [ 38%] Building C object libs/kaa/CMakeFiles/kaac.dir/src/kaa/platform-impl/esp8266/logger.c.obj). This should be fine.
Encryption is disabled in your build. Otherwise it would print ENCRYPTION ENABLED below USER EXTENSION ENABLED.
Here are a couple of possible solutions:
1. Adjust the linker file
One problem for me was that the linker file eagle.app.v6.ld (which should be in the folder ld in your project's root) had to be adjusted so that some parts of the code are moved from RAM to ROM.
I use the linker file from the Kaa sample app and refer to that in the following explanation.
In the section .text : ALIGN(4), find the line containing:
*( .literal .text .literal.* .text.* .stub .gnu.warning .gnu.linkonce.literal.* .gnu.linkonce.t.*.literal .gnu.linkonce.t.*)
Remove the sections .literal.* .text.* from there and move them to .irom0.text : ALIGN(4) by adding the following line after _irom0_text_start = ABSOLUTE(.);:
*(.literal.* .text.*)
Now all sections starting with .literal.and .text. are located in the ROM. This is sort of brute force because regardless of the *.a module (as defined by the lines below that in the section .irom0.text : ALIGN(4)), all sections of this type are moved to ROM. It might have some complications which I did not yet experience. There is definitely room to optimize the linker mapping.
For me this saved a huge amount of memory in RAM and the program compiled, linked and ran correctly.
2. Minimize extensions
When configuring the build with cmake, you can pass different arguments to cmake to turn extensions on or off (e.g. -DWITH_EXTENSION_PROFILE=OFF to turn off profiles).
In your case, the following extensions seem to be enabled:
BOOTSTRAP ENABLED
PROFILE ENABLED
USER EXTENSION ENABLED
In my experience, some extensions are just to big to be compiled into the final image. I successfully included the Configuration and Logging extensions. Adding any other extensions has lead to an overflow of the RAM.
Make sure to enable only those extensions which are really needed by your endpoint SDK. If it still fails, try to turn all extensions off and enable them step-by-step to see which ones fail.
Informational: See which parts of your code get mapped to which section (RAM or ROM)
The following command prints out a list to the file listing.txt detailing which code is mapped to which section. It might help finding out which parts of the code use the most memory in RAM and ROM. (replace name_of_your_image with the actual name of the image generated by the make command)
xtensa-lx106-elf-objdump -x name_of_your_image > listing.txt