blackbeard420/esp-idf
1
0
Fork 0
mirror of https://github.com/espressif/esp-idf synced 2025-03-12 18:49:08 -04:00
Code Issues Projects Releases Wiki Activity

Merge branch 'feature/wl_ext_sector_size' into 'master'

Browse Source 
Feature/wl ext sector size

See merge request !1038
This commit is contained in:
Ivan Grokhotkov 2017-08-07 17:20:34 +08:00
commit 72ee0120cd
8 changed files with 534 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
components/wear_levelling/Kconfig Executable file
View File

@ -0,0 +1,50 @@
menu "FAT FS Wear Levelling Settings"
choice WL_SECTOR_SIZE
bool "FAT FS sector size"
default WL_SECTOR_SIZE_FAT
help
Specify the FAT sector size.
You can set default sector size or size that will
fit to the flash device sector size.
config WL_SECTOR_SIZE_FAT
bool "512"
config WL_SECTOR_SIZE_FLASH
bool "4096"
endchoice
config WL_SECTOR_SIZE
int
default 512 if WL_SECTOR_SIZE_FAT
default 4096 if WL_SECTOR_SIZE_FLASH
choice WL_SECTOR_MODE
bool "Sector store mode"
default WL_SECTOR_MODE_PERF
help
Specify the mode to store data into the flash.
config WL_SECTOR_MODE_PERF
bool "Perfomance"
help
In Performance mode a data will be stored to the RAM and then
stored back to the flash. Compare to the Safety mode, this operation
faster, but if by the erase sector operation power will be off, the
data from complete flash device sector will be lost.
config WL_SECTOR_MODE_SAFE
bool "Safety"
help
In Safety mode a data from complete flash device sector will be stored to the flash and then
stored back to the flash. Compare to the Performance mode, this operation
slower, but if by the erase sector operation power will be off, the
data of the full flash device sector will not be lost.
endchoice
config WL_SECTOR_MODE
int
default 0 if WL_SECTOR_MODE_PERF
default 1 if WL_SECTOR_MODE_SAFE
endmenu

27
components/wear_levelling/README.rst
View File

@ -4,17 +4,30 @@ Wear Levelling APIs
Overview
--------
Most of the flash devices and specially SPI flash devices that are used in ESP32
have sector based organization and have limited amount of erase/modification cycles
per memory sector. To avoid situation when one sector reach the limit of erases when
have sector based organization and have limited amount of erase/modification cycles
per memory sector. To avoid situation when one sector reach the limit of erases when
other sectors was used not often, we have made a component that avoid this situation.
The wear levelling component share the amount of erases between all sectors in the
The wear levelling component share the amount of erases between all sectors in the
memory without user interaction.
The wear_levelling component contains APIs related to reading, writing, erasing,
memory mapping data in the external SPI flash through the partition component. It
also has higher-level APIs which work with FAT filesystem defined in
memory mapping data in the external SPI flash through the partition component. It
also has higher-level APIs which work with FAT filesystem defined in
the :doc:`FAT filesystem </api-reference/storage/fatfs>`.
The wear levelling component does not cache data in RAM. Write and erase functions
The wear levelling component, together with FAT FS component, works with FAT FS sector size 4096
bytes which is standard size of the flash devices. In this mode the component has best performance,
but needs additional memoty in the RAM. To save internal memory the component has two additional modes
to work with sector size 512 bytes: Performance and Safety modes. In Performance mode by erase sector
operation data will be stored to the RAM, sector will be erased and then data will be stored
back to the flash. If by this operation power off situation will occur, the complete 4096 bytes
will be lost. To prevent this the Safety mode was implemented. In safety mode the data will be first
stored to the flash and after sector will be erased, will be stored back. If power off situation will
occur, after power on, the data will be recovered.
By default defined the sector size 512 bytes and Performance mode. To change these values please use
the configuration menu.
The wear levelling component does not cache data in RAM. Write and erase functions
modify flash directly, and flash contents is consistent when the function returns.
@ -37,6 +50,6 @@ filesystem-specific functions.
Memory Size
-----------
The memory size calculated in the wear Levelling module based on parameters of
The memory size calculated in the wear Levelling module based on parameters of
partition. The module use few sectors of flash for internal data.

163
components/wear_levelling/WL_Ext_Perf.cpp Executable file
View File

@ -0,0 +1,163 @@
// Copyright 2015-2017 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "WL_Ext_Perf.h"
#include <stdlib.h>
#include "esp_log.h"
static const char *TAG = "wl_ext_perf";
#define WL_EXT_RESULT_CHECK(result) \
if (result != ESP_OK) { \
ESP_LOGE(TAG,"%s(%d): result = 0x%08x", __FUNCTION__, __LINE__, result); \
return (result); \
}
WL_Ext_Perf::WL_Ext_Perf(): WL_Flash()
{
this->sector_buffer = NULL;
}
WL_Ext_Perf::~WL_Ext_Perf()
{
free(this->sector_buffer);
}
esp_err_t WL_Ext_Perf::config(WL_Config_s *cfg, Flash_Access *flash_drv)
{
wl_ext_cfg_t *config = (wl_ext_cfg_t *)cfg;
this->fat_sector_size = config->fat_sector_size;
this->flash_sector_size = cfg->sector_size;
this->sector_buffer = (uint32_t *)malloc(cfg->sector_size);
if (this->sector_buffer == NULL) {
return ESP_ERR_NO_MEM;
}
this->size_factor = this->flash_sector_size / this->fat_sector_size;
if (this->size_factor < 1) {
return ESP_ERR_INVALID_ARG;
}
return WL_Flash::config(cfg, flash_drv);
}
esp_err_t WL_Ext_Perf::init()
{
return WL_Flash::init();
}
size_t WL_Ext_Perf::chip_size()
{
return WL_Flash::chip_size();
}
size_t WL_Ext_Perf::sector_size()
{
return this->fat_sector_size;
}
esp_err_t WL_Ext_Perf::erase_sector(size_t sector)
{
return this->erase_sector_fit(sector, 1);
}
esp_err_t WL_Ext_Perf::erase_sector_fit(uint32_t start_sector, uint32_t count)
{
// This method works with one flash device sector and able to erase "count" of fatfs sectors from this sector
esp_err_t result = ESP_OK;
uint32_t pre_check_start = start_sector % this->size_factor;
for (int i = 0; i < this->size_factor; i++) {
if ((i < pre_check_start) || (i >= count + pre_check_start)) {
result = this->read(start_sector / this->size_factor * this->flash_sector_size + i * this->fat_sector_size, &this->sector_buffer[i * this->fat_sector_size / sizeof(uint32_t)], this->fat_sector_size);
WL_EXT_RESULT_CHECK(result);
}
}
result = WL_Flash::erase_sector(start_sector / this->size_factor); // erase comlete flash sector
WL_EXT_RESULT_CHECK(result);
// And write back only data that should not be erased...
for (int i = 0; i < this->size_factor; i++) {
if ((i < pre_check_start) || (i >= count + pre_check_start)) {
result = this->write(start_sector / this->size_factor * this->flash_sector_size + i * this->fat_sector_size, &this->sector_buffer[i * this->fat_sector_size / sizeof(uint32_t)], this->fat_sector_size);
WL_EXT_RESULT_CHECK(result);
}
}
return ESP_OK;
}
esp_err_t WL_Ext_Perf::erase_range(size_t start_address, size_t size)
{
esp_err_t result = ESP_OK;
if ((start_address % this->fat_sector_size) != 0) {
result = ESP_ERR_INVALID_ARG;
}
if (((size % this->fat_sector_size) != 0) || (size == 0)) {
result = ESP_ERR_INVALID_ARG;
}
WL_EXT_RESULT_CHECK(result);
// The range to erase could be allocated in any possible way
// ---------------------------------------------------------
// | | | | |
// |0|0|x|x|x|x|x|x|x|x|x|x|x|x|0|0|
// | pre | rest | rest | post | <- check ranges
//
// Pre check - the data that is not fit to the full sector at the begining of the erased block
// Post check - the data that are not fit to the full sector at the end of the erased block
// rest - data that are fit to the flash device sector at the middle of the erased block
//
// In case of pre and post check situations the data of the non erased area have to be readed first and then
// stored back.
// For the rest area this operation not needed because complete flash device sector will be erased.
ESP_LOGV(TAG, "%s begin, addr = 0x%08x, size = %i", __func__, start_address, size);
// Calculate pre check values
uint32_t pre_check_start = (start_address / this->fat_sector_size) % this->size_factor;
uint32_t sectors_count = size / this->fat_sector_size;
uint32_t pre_check_count = (this->size_factor - pre_check_start);
if (pre_check_count > sectors_count) {
pre_check_count = sectors_count;
}
// Calculate post ckeck
uint32_t post_check_count = (sectors_count - pre_check_count) % this->size_factor;
uint32_t post_check_start = ((start_address + size - post_check_count * this->fat_sector_size) / this->fat_sector_size);
// Calculate rest
uint32_t rest_check_count = sectors_count - pre_check_count - post_check_count;
if ((pre_check_count == this->size_factor) && (0 == pre_check_start)) {
rest_check_count++;
pre_check_count = 0;
}
uint32_t rest_check_start = start_address + pre_check_count * this->fat_sector_size;
// Here we will clear pre_check_count amount of sectors
if (pre_check_count != 0) {
result = this->erase_sector_fit(start_address / this->fat_sector_size, pre_check_count);
WL_EXT_RESULT_CHECK(result);
}
if (rest_check_count > 0) {
rest_check_count = rest_check_count / this->size_factor;
result = WL_Flash::erase_range(rest_check_start, rest_check_count * this->flash_sector_size);
WL_EXT_RESULT_CHECK(result);
}
if (post_check_count != 0) {
result = this->erase_sector_fit(post_check_start, post_check_count);
WL_EXT_RESULT_CHECK(result);
}
return ESP_OK;
}

161
components/wear_levelling/WL_Ext_Safe.cpp Executable file
View File

@ -0,0 +1,161 @@
// Copyright 2015-2017 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "WL_Ext_Safe.h"
#include <stdlib.h>
#include "esp_log.h"
static const char *TAG = "wl_ext_safe";
#define WL_EXT_RESULT_CHECK(result) \
if (result != ESP_OK) { \
ESP_LOGE(TAG,"%s(%d): result = 0x%08x", __FUNCTION__, __LINE__, result); \
return (result); \
}
#ifndef FLASH_ERASE_VALUE
#define FLASH_ERASE_VALUE 0xffffffff
#endif // FLASH_ERASE_VALUE
#ifndef WL_EXT_SAFE_OK
#define WL_EXT_SAFE_OK 0x12345678
#endif // WL_EXT_SAFE_OK
#ifndef WL_EXT_SAFE_OFFSET
#define WL_EXT_SAFE_OFFSET 16
#endif // WL_EXT_SAFE_OFFSET
struct WL_Ext_Safe_State {
public:
uint32_t erase_begin;
uint32_t local_addr_base;
uint32_t local_addr_shift;
uint32_t count;
};
WL_Ext_Safe::WL_Ext_Safe(): WL_Ext_Perf()
{
}
WL_Ext_Safe::~WL_Ext_Safe()
{
}
esp_err_t WL_Ext_Safe::config(WL_Config_s *cfg, Flash_Access *flash_drv)
{
esp_err_t result = ESP_OK;
result = WL_Ext_Perf::config(cfg, flash_drv);
WL_EXT_RESULT_CHECK(result);
this->state_addr = WL_Flash::chip_size() - 2 * WL_Flash::sector_size();
this->dump_addr = WL_Flash::chip_size() - 1 * WL_Flash::sector_size();
return ESP_OK;
}
esp_err_t WL_Ext_Safe::init()
{
esp_err_t result = ESP_OK;
ESP_LOGV(TAG, "%s", __func__);
result = WL_Ext_Perf::init();
WL_EXT_RESULT_CHECK(result);
result = this->recover();
return result;
}
size_t WL_Ext_Safe::chip_size()
{
ESP_LOGV(TAG, "%s size = %i", __func__, WL_Flash::chip_size() - 2 * this->flash_sector_size);
return WL_Flash::chip_size() - 2 * this->flash_sector_size;
}
esp_err_t WL_Ext_Safe::recover()
{
esp_err_t result = ESP_OK;
WL_Ext_Safe_State state;
result = WL_Flash::read(this->state_addr, &state, sizeof(WL_Ext_Safe_State));
WL_EXT_RESULT_CHECK(result);
ESP_LOGI(TAG, "%s recover, start_addr = 0x%08x, local_addr_base = 0x%08x, local_addr_shift = %i, count=%i", __func__, state.erase_begin, state.local_addr_base, state.local_addr_shift, state.count);
// check if we have transaction
if (state.erase_begin == WL_EXT_SAFE_OK) {
result = this->read(this->dump_addr, this->sector_buffer, this->flash_sector_size);
WL_EXT_RESULT_CHECK(result);
result = WL_Flash::erase_sector(state.local_addr_base); // erase comlete flash sector
WL_EXT_RESULT_CHECK(result);
// And write back...
for (int i = 0; i < this->size_factor; i++) {
if ((i < state.local_addr_shift) || (i >= state.count + state.local_addr_shift)) {
result = this->write(state.local_addr_base * this->flash_sector_size + i * this->fat_sector_size, &this->sector_buffer[i * this->fat_sector_size / sizeof(uint32_t)], this->fat_sector_size);
WL_EXT_RESULT_CHECK(result);
}
}
// clear transaction
result = WL_Flash::erase_range(this->state_addr, this->flash_sector_size);
}
return result;
}
esp_err_t WL_Ext_Safe::erase_sector_fit(uint32_t start_sector, uint32_t count)
{
esp_err_t result = ESP_OK;
uint32_t local_addr_base = start_sector / this->size_factor;
uint32_t pre_check_start = start_sector % this->size_factor;
ESP_LOGV(TAG, "%s start_sector=0x%08x, count = %i", __func__, start_sector, count);
for (int i = 0; i < this->size_factor; i++) {
if ((i < pre_check_start) || (i >= count + pre_check_start)) {
result = this->read(start_sector / this->size_factor * this->flash_sector_size + i * this->fat_sector_size, &this->sector_buffer[i * this->fat_sector_size / sizeof(uint32_t)], this->fat_sector_size);
WL_EXT_RESULT_CHECK(result);
}
}
result = WL_Flash::erase_sector(this->dump_addr / this->flash_sector_size);
WL_EXT_RESULT_CHECK(result);
result = WL_Flash::write(this->dump_addr, this->sector_buffer, this->flash_sector_size);
WL_EXT_RESULT_CHECK(result);
WL_Ext_Safe_State state;
state.erase_begin = WL_EXT_SAFE_OK;
state.local_addr_base = local_addr_base;
state.local_addr_shift = pre_check_start;
state.count = count;
result = WL_Flash::erase_sector(this->state_addr / this->flash_sector_size);
WL_EXT_RESULT_CHECK(result);
result = WL_Flash::write(this->state_addr + 0, &state, sizeof(WL_Ext_Safe_State));
WL_EXT_RESULT_CHECK(result);
// Erase
result = WL_Flash::erase_sector(local_addr_base); // erase comlete flash sector
WL_EXT_RESULT_CHECK(result);
// And write back...
for (int i = 0; i < this->size_factor; i++) {
if ((i < pre_check_start) || (i >= count + pre_check_start)) {
result = this->write(local_addr_base * this->flash_sector_size + i * this->fat_sector_size, &this->sector_buffer[i * this->fat_sector_size / sizeof(uint32_t)], this->fat_sector_size);
WL_EXT_RESULT_CHECK(result);
}
}
result = WL_Flash::erase_sector(this->state_addr / this->flash_sector_size);
WL_EXT_RESULT_CHECK(result);
return ESP_OK;
}

22
components/wear_levelling/private_include/WL_Ext_Cfg.h Executable file
View File

@ -0,0 +1,22 @@
// Copyright 2015-2017 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef _WL_Ext_Cfg_H_
#define _WL_Ext_Cfg_H_
#include "WL_Config.h"
typedef struct WL_Ext_Cfg_s : public WL_Config_s {
uint32_t fat_sector_size; /*!< virtual sector size*/
} wl_ext_cfg_t;
#endif // _WL_Ext_Cfg_H_

46
components/wear_levelling/private_include/WL_Ext_Perf.h Executable file
View File

@ -0,0 +1,46 @@
// Copyright 2015-2017 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef _WL_Ext_Perf_H_
#define _WL_Ext_Perf_H_
#include "WL_Flash.h"
#include "WL_Ext_Cfg.h"
class WL_Ext_Perf : public WL_Flash
{
public:
WL_Ext_Perf();
~WL_Ext_Perf() override;
esp_err_t config(WL_Config_s *cfg, Flash_Access *flash_drv) override;
esp_err_t init() override;
size_t chip_size() override;
size_t sector_size() override;
esp_err_t erase_sector(size_t sector) override;
esp_err_t erase_range(size_t start_address, size_t size) override;
protected:
uint32_t flash_sector_size;
uint32_t fat_sector_size;
uint32_t size_factor;
uint32_t *sector_buffer;
virtual esp_err_t erase_sector_fit(uint32_t start_sector, uint32_t count);
};
#endif // _WL_Ext_Perf_H_

42
components/wear_levelling/private_include/WL_Ext_Safe.h Executable file
View File

@ -0,0 +1,42 @@
// Copyright 2015-2017 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef _WL_Ext_Safe_H_
#define _WL_Ext_Safe_H_
#include "WL_Flash.h"
#include "WL_Ext_Cfg.h"
#include "WL_Ext_Perf.h"
class WL_Ext_Safe : public WL_Ext_Perf
{
public:
WL_Ext_Safe();
~WL_Ext_Safe() override;
esp_err_t config(WL_Config_s *cfg, Flash_Access *flash_drv) override;
esp_err_t init() override;
size_t chip_size() override;
protected:
esp_err_t erase_sector_fit(uint32_t start_sector, uint32_t count) override;
// Dump Sector
uint32_t dump_addr; // dump buffer address
uint32_t state_addr;// sectore where state of transaction will be stored
esp_err_t recover();
};
#endif // _WL_Ext_Safe_H_

31
components/wear_levelling/wear_levelling.cpp
View File

@ -17,7 +17,10 @@
#include <sys/lock.h>
#include "wear_levelling.h"
#include "WL_Config.h"
#include "WL_Ext_Cfg.h"
#include "WL_Flash.h"
#include "WL_Ext_Perf.h"
#include "WL_Ext_Safe.h"
#include "SPI_Flash.h"
#include "Partition.h"
@ -79,7 +82,7 @@ esp_err_t wl_mount(const esp_partition_t *partition, wl_handle_t *out_handle)
goto out;
}
wl_config_t cfg;
wl_ext_cfg_t cfg;
cfg.full_mem_size = partition->size;
cfg.start_addr = WL_DEFAULT_START_ADDR;
cfg.version = WL_CURRENT_VERSION;
@ -88,6 +91,8 @@ esp_err_t wl_mount(const esp_partition_t *partition, wl_handle_t *out_handle)
cfg.updaterate = WL_DEFAULT_UPDATERATE;
cfg.temp_buff_size = WL_DEFAULT_TEMP_BUFF_SIZE;
cfg.wr_size = WL_DEFAULT_WRITE_SIZE;
// FAT sector size by default will be 512
cfg.fat_sector_size = CONFIG_WL_SECTOR_SIZE;
// Allocate memory for a Partition object, and then initialize the object
// using placement new operator. This way we can recover from out of
@ -101,13 +106,37 @@ esp_err_t wl_mount(const esp_partition_t *partition, wl_handle_t *out_handle)
part = new (part_ptr) Partition(partition);
// Same for WL_Flash: allocate memory, use placement new
#if CONFIG_WL_SECTOR_SIZE == 512
#if CONFIG_WL_SECTOR_MODE == 1
wl_flash_ptr = malloc(sizeof(WL_Ext_Safe));
if (wl_flash_ptr == NULL) {
result = ESP_ERR_NO_MEM;
ESP_LOGE(TAG, "%s: can't allocate WL_Ext_Safe", __func__);
goto out;
}
wl_flash = new (wl_flash_ptr) WL_Ext_Safe();
#else
wl_flash_ptr = malloc(sizeof(WL_Ext_Perf));
if (wl_flash_ptr == NULL) {
result = ESP_ERR_NO_MEM;
ESP_LOGE(TAG, "%s: can't allocate WL_Ext_Perf", __func__);
goto out;
}
wl_flash = new (wl_flash_ptr) WL_Ext_Perf();
#endif // CONFIG_WL_SECTOR_MODE
#endif // CONFIG_WL_SECTOR_SIZE
#if CONFIG_WL_SECTOR_SIZE == 4096
wl_flash_ptr = malloc(sizeof(WL_Flash));
if (wl_flash_ptr == NULL) {
result = ESP_ERR_NO_MEM;
ESP_LOGE(TAG, "%s: can't allocate WL_Flash", __func__);
goto out;
}
wl_flash = new (wl_flash_ptr) WL_Flash();
#endif // CONFIG_WL_SECTOR_SIZE
result = wl_flash->config(&cfg, part);
if (ESP_OK != result) {