Memory Technology Device (MTD) Subsystem for Linux.
Description: A generic subsystem for handling memory technology devices under Linux
Keywords: Disk-On-Chip, CompactFlash, Compact Flash, Disk On Chip, M-Systems, Linux, MTD, Flash, Memory Technology Device

# Memory Technology Device (MTD) Subsystem for Linux

## Design aims

We're working on a generic Linux subsystem for memory devices, especially Flash devices.

The aim of the system is to make it simple to provide a driver for new hardware, by providing a generic interface between the hardware drivers and the upper layers of the system.

Hardware drivers need to know nothing about the storage formats used, such as FTL, FFS2, etc., but will only need to provide simple routines for `read`, `write` and `erase`. Presentation of the device's contents to the user in an appropriate form will be handled by the upper layers of the system.

## Mailing list

There is a majordomo-managed mailing list: lists-mtd@lists.infradead.org - Full archives are available at http://lists.infradead.org/pipermail/linux-mtd/. Now that I've finally managed to get Mailman to display them properly, this includes all the previous messages from before the changeover to Mailman.

To subscribe, go here or send "`subscribe`" in the body of a mail to linux-mtd-request@lists.infradead.org

NOTE: DO NOT SEND YOUR SUBSCRIPTION REQUEST TO THE LIST ITSELF.
SEND IT TO linux-mtd-request@lists.infradead.org AS THE ABOVE SAYS.

## Download and CVS

Very occasionally, I make snapshot releases. Now that the MTD code is in the 2.4 kernel, it's become even rarer. Your best option is to obtain the latest code from CVS, by following the instructions below. We do break the CVS build occasionally, but we're also fairly good at fixing it quickly when we do so.

Anonymous CVS access is available, and full CVS access may also be granted to developers on request.

    cvs -d :pserver:anoncvs@cvs.infradead.org:/home/cvs login (password: anoncvs)
    cvs -d :pserver:anoncvs@cvs.infradead.org:/home/cvs co mtd

If you can't access anoncvs for some reason, daily snapshots are also available at ftp://ftp.uk.linux.org/pub/people/dwmw2/mtd/cvs/

## Documentation

There is now some MTD API documentation available. It's a little out of date - the API has been evolving over the last few months as we encounter strange pieces of hardware that we hadn't anticipated, and occasionally when I'd just overlooked something blatantly obvious. Volunteers to update the docs are welcome.
13th Feb 2001: A mtd-jffs-HOWTO is now also available under CVS. Not all topics are covered yet, but it's a start.
5th May 2002: A NAND Flash document is now available. It covers some technical topics abaout NAND and filesystems and contains a HOWTO.

## Booting

You can now replace the firmware on the DiskOnChip 2000, and possibly also the DiskOnChip Millennium, with a version of GNU Grub which runs directly from the flash.

The patches to make Grub aware of the DiskOnChip and the NFTL format used on it, along with a first-stage loader to load Grub itself into memory from the DiskOnChip, are in the CVS repository.

## TODO

I'm impressed. The TODO list is getting much shorter.

- Add an MTD driver for PCMCIA memory cards
- Make the FFS2 code work on 2.3 kernels.
- Implement XIP support in the 'map' layer and a read-only filesystem to use it.
- Decrease the amount of flash space reserved by JFFS2 garbage collection

## Changelog

- Latest CVS
  - JFFS v2 -
    - Compression
    - Improved wear levelling - we deal with individual erase blocks now
    - Improved concurrency - no longer all under one big semaphore
    - Smaller RAM footprint - at least theoretically. And we forget stuff under memory pressure to be nice
    - Hard link support
  - New 'mtdblock' driver which allows you to use any normal filesystem on flash chips, albeit rather non-robust if you're writing to it. See below for more information.
  - New map drivers, including the Itsy (and iPAQ) driver.
  - Massive NFTL overhaul. ECC, bad block handling, journalling fixes, etc.
  - Verified DiskOnChip 2000 and Millennium support - some wonderful person seems to have gone through the code and the docs and ensured that we meet all the timing constraints that I ignored when I threw the original code together.
  - Support all sensible geometries for AMD/Fujitsu command set CFI flash
  - Start (most?) of JFFS support for NAND flash
  - Hardware support for a handful of NAND flash chips
  - Beat JFFS up till it actually passes filesystem stress tests
  - Allow CFI reads to suspend in-progress erases.
  - Merged DiskOnChip 2000 and Millennium driver into one
  - Partitioning of flash chips into multiple devices
  - Map area drivers for various SBC boards
  - New DiskOnChip Millennium driver, contributed by Ollie Lho
  - Assorted JFFS fixes
  - CFI code for AMD/Fujitsu Standard command set, mapped in 2x16 mode, contributed by Masami Komiya
- mtd-20000704
  Patch: mtd-20000704.diff
  Tarball: mtd-20000704.tar.gz
  - Kernel patches for both 2.2.16 and 2.4.0 kernels to add MTD support, and a script file to move the source files into the right place.
  - New `map` architecture for making mmap'd areas available for use by generic chipset drivers
  - Probe code for Common Flash Interface devices
  - Run-time address alias detection for CFI chips
  - Driver code for Intel Extended Command Set devices - connected any way you like as long as it's in 16-bit mode and not interleaved
  - JFFS ported to 2.2 kernel
  - Remove the /dev/mtdx character device driver from `mtd.o` and make it separately configurable as `mtdchar.o`
  - Rename `mtd.o` to `mtdcore.o` just in case I didn't confuse enough people with the above
  - Split the DiskOnChip probe routine into a separate file, allowing DiskOnChip 2000 and DiskOnChip Millennium drivers to be developed separately.
  - Separate `CONFIG_NFTL_RW` option allowing you to compile safe read-only drivers (by turning it off)
  - Imported Axis' Journalling Flash File System (JFFS)
  - Updated FTL driver to the latest version from David Hinds' PCMCIA code. Still some elusive bugs which cause it to corrupt the FTL - must get some PCMCIA flash hardware and check it worked before I started playing with it.
  - Litter FTL driver with cpu_to_le16() et al calls in an attempt to make it work on big-endian hardware
  - Some cleanups to make the code conform to the new rules.
  - Add `get_mtd_device()` and `put_mtd_device` back again, having gone over the locking rules with a fine tooth comb and decided that they were OK if done properly.
  - Added patch to GNU Grub which teaches it about the DiskOnChip. When it can also decode NFTL, it'll be more useful.
- mtd-20000131
  Patch: mtd-20000131.diff
  Tarball: mtd-20000131.tar.gz
  Complete patch against 2.3.41 kernel: mtd-20000131-against-2.3.41.diff.gz
  - Updated to work with 2.3.recent kernel. Apart from the obvious changes, this also includes turning off the block device request merging code, which seems to be duplicating requests rather than just merging them.
  - Remove the byteorder workaround - there's a patch included which should fix the kernel.
  - Added Grub loader for DiskOnChip - now someone just has to add DiskOnChip support to Grub itself.
- mtd-20000119
  Patch: mtd-20000119.diff
  Tarball: mtd-20000119.tar.gz
  - NFTL now registers the block device before asking the gendisk code to scan it for partitions - it works better like that :)
  - Various code cleanups, fixes for PPC and older kernels - mostly thanks to Pavel Roskin.
  - No patch against 2.2 or 2.3.current kernels - I'll resume that practice as soon as I've got more time to look after it.
- mtd-19991025
  Patch: mtd-19991025.diff
  Tarball: mtd-19991025.tar.gz
  Complete patch against 2.3.23 kernel: mtd-19991025-against-2.3.23.diff.gz
  - Ported to 2.3 kernels (still compiles with 2.2)
  - NFTL write support seems fairly reliable and interoperates with M-Systems' own drivers - more testing required
  - Various bugfixes
- mtd-19990820
  Patch: mtd-19990820.diff
  Tarball: mtd-19990820.tar.gz
  - Add `nftl_format` program (M-Systems' own drivers don't like the resulting NFTL yet)
  - First version of write support for NFTL (Likewise)
- mtd-19990817
  Patch: mtd-19990817.diff
  Tarball: mtd-19990817.tar.gz
  - Support ECC for NFTL on DiskOnChip
  - Behave correctly with multiple DiskOnChip
  - Add primitive MAKEDEV script
  - Move include directory to make it easier to compile into kernel
  - Add `/proc/mtd` support
- mtd-19990810
  Patch: mtd-19990810.diff
  Tarball: mtd-19990810.tar.gz
  - Various endianness and wordsize fixes
  - Compile fixes for Linux/Alpha - DiskOnChip & NFTL now works
  - Sanity checks in NFTL driver to prevent oopsen on certain MTDs
- mtd-19990809
  Tarball: mtd-19990809.tar.gz
  Large merge, including:
  - Preliminary drivers for DiskOnChip 2000 and NFTL filesystem
  - Microsoft FFS2 filesystem
  - Octagon-5066 and vmax301 drivers for SBC boards
  - Various other improvements from jgg.
  - Boot loader code

## MTD User modules

These are the modules which provide interfaces that can be used directly from userspace. The user modules currently planned include:

- Raw character access:
  A character device which allows direct access to the underlying memory. Useful for creating filesystems on the devices, before using some of the translation drivers below, or for raw storage on infrequently-changed flash, or RAM devices.
- Raw block access
  A block device driver which allows you to pretend that the flash is a normal device with sensible sector size. It actually works by caching a whole flash erase block in RAM, modifying it as requested, then erasing the whole block and writing back the modified data.
  This allows you to use normal filesystems on flash parts. Obviously it's not particularly robust when you are writing to it - you lose a whole erase block's worth of data if your read/modify/erase/rewrite cycle actually goes read/modify/erase/poweroff. But for development, and for setting up filesystems which are actually going to be mounted read-only in production units, it should be fine.
  There is also a read-only version of this driver which doesn't have the capacity to do the caching and erase/writeback, mainly for use with uCLinux where the extra RAM requirement was considered too large.
- Flash Translation Layer (FTL)
- NFTL
  Block device drivers which implement an FTL/NFTL filesystem on the underlying memory device. FTL is fully functional. NFTL is currently working for both reading and writing, but could probably do with some more field testing before being used on production systems.
- Microsoft Flash Filing System
  This provides a filesystem directly from the flash, rather than the less efficient filesystem-on-FTL-on-flash setup. The current driver is in alpha-testing, but should allow read-only access to such filesystems.
- Journalling Flash File System
  This was developed by Axis Communications, and provides a filesystem directly on the flash, rather than emulating a block device. For more information, see Axis' developer pages.

## MTD hardware device drivers

These provide physical access to memory devices, and are not used directly - they are accessed through the user modules above.

- On-board memory
  Many PC chipsets are incapable of correctly caching system memory above 64M or 512M. A driver exists which allows you to use this memory with the linux-mtd system.
- PCMCIA devices
  There are already a number of PCMCIA memory devices supported by Linux. The MTD device subsystem is being designed with compatibility in mind, so that the existing PCMCIA drivers can cleanly interface to the new MTD subsystem. This work has not yet been undertaken, though.
- Common Flash Interface (CFI) onboard memory
  This is one of the most interesting options for small embedded systems. With a filesystem directly on the flash, rather than through an extra layer like FTL, we can get execute-in-place (XIP) of programs and libraries on the flash, without needing to use up valuable system memory as a cache for the executable code.
  A patch to the Linux kernel which supports this is available, and it should shortly be ported to the generic MTD system as a hardware driver.
- M-Systems' ISA Flash cards and DiskOnChip 1000
  These devices are very simple - they seem to consist of a large array of Intel flash memory, with a simple paging mechanism reminiscent of EMS, which allows different pages of the flash to be mapped into the device's 8K memory window. While working for Nortel a couple of years ago, I developed drivers for these devices.
  Unfortunately, I don't think the Disk-On-Chip 1000 is available any more.
- M-Systems' DiskOnChip 2000 and Millennium
  M-Systems have now provided me with full documentation for the DiskOnChip 2000 and Millennium. The current driver is capable of full operation on a DiskOnChip 2000, and I have received a sample DiskOnChip Millennium from M-Systems, so I should get round to adding Millennium support soon. The write support for NFTL appears to pass my simple tests for interoperability - after I mount it with my drivers, thrash it for a while, and unmount it, the NFTL is still recognised by M-Systems' own drivers. More testing of this would be very much appreciated.
  The Linux and Disk-On-Chip 2000 page is available here. Please note that, now that M-Systems have provided me with complete specifications with which to write a free driver for the device, there is no longer any reason to avoid using it with Linux systems.

- CompactFlash - http://www.compactflash.org/
  CompactFlash emulates an IDE disk, either through the PCMCIA-ATA standard, or by connecting directly to an IDE interface.
  As such, it has no business being on this page, as to the best of my knowledge it doesn't have any alternative method of accessing the flash - you have to use the IDE emulation - I mention it here for completeness.

$Id: index.html,v 1.1.1.1 2003/11/25 22:15:12 jharrell Exp $